Git Product home page Git Product logo

google / earlgrey Goto Github PK

View Code? Open in Web Editor NEW
5.6K 233.0 734.0 47.22 MB

:tea: iOS UI Automation Test Framework

Home Page: http://google.github.io/EarlGrey/

License: Apache License 2.0

Objective-C 95.36% HTML 0.38% Swift 1.11% Ruby 1.94% Python 0.19% Shell 0.76% CSS 0.26%
earlgrey ios-automation ui-automated-tests ios test-framework testing-tools automation ios-device ios-development ios-test

earlgrey's Introduction

Deprecation: EarlGrey 1.0 is deprecated in favor of EarlGrey 2.0 which integrates it with XCUITest. Please look at the earlgrey2 branch. EarlGrey 1.0 is not being maintained internally with iOS 13.

EarlGrey

Apache License CC-BY 4.0 License Build Status Carthage compatible CocoaPods Gem Version

For EarlGrey 2, please go to earlgrey2 branch.

EarlGrey is a native iOS UI automation test framework that enables you to write clear, concise tests.

With the EarlGrey framework, you have access to enhanced synchronization features. EarlGrey automatically synchronizes with the UI, network requests, and various queues; but still allows you to manually implement customized timings, if needed.

EarlGrey’s synchronization features help to ensure that the UI is in a steady state before actions are performed. This greatly increases test stability and makes tests highly repeatable.

EarlGrey works in conjunction with the XCTest framework and integrates with Xcode’s Test Navigator so you can run tests directly from Xcode or the command line (using xcodebuild).

Getting Started

The EarlGrey documentation for users is located in the EarlGrey/docs folder. To get started, review the EarlGrey features, check for backward compatibility, and then install/run EarlGrey with your test target. After everything is configured, take a look at the EarlGrey API and start writing your own tests.

Getting Help

If you need help, several resources are available. First check the FAQ. If you have more questions after reading the FAQ, see Known Issues. You can bring more specific issues to our attention by asking them on stackoverflow.com using the #earlgrey tag. You can also start new discussions with us on our Google group or request to join our slack channel.

Analytics

To prioritize and improve EarlGrey, the framework collects usage data and uploads it to Google Analytics. More specifically, the framework collects the MD5 hash of Bundle ID, Test Class Names and Test Method Names. This information allows us to measure the volume of usage. For more detailed information about our analytics collection, please peruse the GREYAnalytics.m file which contains the implementation details. If they wish, users can choose to opt out by disabling the Analytics config setting in their test’s - (void)setUp method:

In Objective-C:

// Disable analytics.
[[GREYConfiguration sharedInstance] setValue:@(NO) forConfigKey:kGREYConfigKeyAnalyticsEnabled];

In Swift:

// Disable analytics.
GREYConfiguration.sharedInstance().setValue(false, forConfigKey: kGREYConfigKeyAnalyticsEnabled)

For Contributors

Please make sure you’ve followed the guidelines in CONTRIBUTING.md before making any contributions.

Setup an EarlGrey Project

  1. Clone the EarlGrey repository from GitHub:
git clone https://github.com/google/EarlGrey.git
  1. After you have cloned the EarlGrey repository, download all the dependencies using setup-earlgrey.sh.
  2. After the script completes successfully, open EarlGrey.xcodeproj and ensure that all the targets build.
  3. You can now use EarlGrey.xcodeproj to make changes to the framework.

Add and Run Tests

Unit Tests

To add unit tests for EarlGrey, use UnitTests.xcodeproj located at Tests/UnitTests. To run all unit tests, select the UnitTests Scheme and press Cmd+U.

Functional Tests

To add functional tests for EarlGrey, use the FunctionalTests.xcodeproj located at Tests/FunctionalTests. To run all functional tests, select the FunctionalTests Scheme and press Cmd+U.

earlgrey's People

Contributors

arcank avatar axi0mx avatar bootstraponline avatar fernyb avatar gimite avatar kazucocoa avatar kebernet avatar khandpur avatar kylefin avatar leonatan avatar lewislebentz avatar mbaxley avatar mpetrov avatar petaren avatar phsong-google avatar ravimandala avatar rohit-rao avatar rolyatwilson avatar rotemmiz avatar saurabhj80 avatar sferrini avatar sid-github avatar sigito avatar sskhandp avatar st3fan avatar stkhapugin avatar suriyaakudoisc avatar tirodkar avatar wuhao5 avatar ynzhang0509 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

earlgrey's Issues

grey_replaceText() as alternative to grey_typeText()

I'd like to set text directly on the UITextField without using the keyboard. This doesn't seem possible to do with the existing API. In Espresso, there's a replaceText() action.

UITextField has a setText method.

Xcode build fails

Related to #30, xcode-build fails because the signing is set to Don't Code Sign

$ xcodebuild -project EarlGrey.xcodeproj -configuration Release -target EarlGrey CONFIGURATION_BUILD_DIR=../..

=== BUILD TARGET EarlGrey OF PROJECT EarlGrey WITH CONFIGURATION Release ===

Check dependencies
CodeSign error: code signing is required for product type 'Framework' in SDK 'iOS 9.3'

** BUILD FAILED **

How should people be making builds of EarlGrey? I think the setting should be set to automatic.

  • Project (EarlGrey) → Code Signing → Code Signing Identity → Automatic ...

grey_kindOfClass doesn't accept a String

| |--<_UIAlertControllerView:0x7a7b4600; AX=N; AX.label='Login Failure';

EarlGrey().selectElementWithMatcher(grey_allOfMatchers(
    grey_accessibilityLabel("Login Failure"),
    grey_kindOfClass("_UIAlertControllerView")))

Why can't I use a string for grey_kindOfClass? I can't reference _UIAlertControllerView directly as it appears to be a private class.

AX Exchange error when MFMailComposeViewController has been initialized.

Description

when EarlGrey tries to open a viewController where the MFMailComposeViewController has been initialized, many warning messages start being printed on the console. and earlgrey keeps waiting forever until the test is marked as failed.

The warning message is:

AX Exchange error: Error Domain=Accessibility Code=0 "Remote service does not respond to _accessibilityMachPort" UserInfo={NSLocalizedDescription=Remote service does not respond to _accessibilityMachPort}

observations:

If you turn off synchronization the test continues until it finish, which marks the test as success. but the warning messages keeps appearing.

My custom matcher push an error " caught "NSInvalidArgumentException", "-[UIRemoteKeyboardWindow placeholder]: unrecognized selector sent to instance ..."

image

code:
-(id)matcherForTextFieldWithPlaceHolder:(NSString*) placeholder{

MatchesBlock matches = ^BOOL(UITextField *textfield) {
    return [textfield.placeholder isEqualToString:placeholder];

};
DescribeToBlock describe = ^void(id<GREYDescription> description) {

    [description appendText:@"matcherForTextFieldWithPlaceHolder"];

};
return [[GREYElementMatcherBlock alloc] initWithMatchesBlock:matches

                                            descriptionBlock:describe];

}

Thanks for your attention,
How to fix this issue ?

Cannot execute the test target

Facing the problem below :

Test target IntegrationTests encountered an error (Early unexpected exit, operation never finished bootstrapping - no restart will be attempted)

Issue in running setup-earlgrey.sh

I tried running setup-earlgrey.sh several times.
But, it says - "There was an issue moving Fishhook as per the EarlGrey specification.".

What's wrong ?

First test always fails

Whenever I try running a couple tests, the first one always fails when it should pass. When I rename methods to change execution order, the test that was failing now passes, and the passing test that is now running.

Running on the 5S(8.4) sim. Initially on Xcode 7.3, but reverted to 7.0.1 and still had this issue (side note: execution was a lot faster on 7.0.1).

firstrun
secondrun

Unit test failures on Travis

Two unit tests are failing on Travis:

    • [GREYOperationQueueIdlingResourceTest testOccupiedQueueIdleAfterTaskCompletion]
      GREYOperationQueueIdlingResourceTest.m:73: error: -[GREYOperationQueueIdlingResourceTest testOccupiedQueueIdleAfterTaskCompletion] : (([idlingRes isIdleNow]) is false) failed - Non-empty queue should not be in idle state.
    • [GREYOperationQueueIdlingResourceTest testOccupiedQueueNotIdle]
      GREYOperationQueueIdlingResourceTest.m:61: error: -[GREYOperationQueueIdlingResourceTest testOccupiedQueueNotIdle] : (([idlingRes isIdleNow]) is false) failed - Non-empty queue should not be in idle state.

See full failure logs here:
https://s3.amazonaws.com/archive.travis-ci.org/jobs/110559080/log.txt

Carthage docs & example

Now that Carthage is able to fetch and build EarlGrey, it'd be nice to have an example similar to the existing cocoapods demo. Also the install-and-run doc should be updated to include the steps necessary to get started with carthage.

The Demo project does not work out of the box.

After doing a pod install, the demo project does not work. grey_accessibilityID seems to be the culprit which I also can't resolve when I add EarlGrey to my own project. This is making it extremely difficult to want to pursue when the setup is so rickety.

Swift Example:
screen shot 2016-06-08 at 10 02 19 am

Objective-C Example:
screen shot 2016-06-08 at 10 03 43 am

Jenkins Command to Run EarlGrey tests

I was wondering what the command is to run the EarlGrey UI tests in Jenkins?

Is it the same as this?
"-sdk iphonesimulator -destination 'platform=iOS Simulator,name=iPhone 6,OS=9.0' test"

I use that command for XCUITest.

I am evaluating frameworks for my work and we need it to run in Jenkins.

Thanks!

setup fails to end

Env: iOS 9.2, iPhone 5s simulator + Xcode 7.2.1
In my test setup, the line
[[[XCUIApplication alloc] init] launch];
never returns.
If I remove it the tests run correctly. Is this line needed ?

Remove OCHamcrest imports from Unit and Functional Tests Project.

Since we have a wrapper around OCHamcrest, we do not need an explicit import nor any linking against it in the Test Projects themselves.

Remove all instances of -f OCHamcrest from the UnitTests.xcodeproj and FunctionalTest.xcodeproj. Also, remove any uneeded imports containing import <OCHamcrest/...>.

Some keys can not be tapped on non English keyboard.

Hi,

I tried the EarlGrey, if you want to test non-English app, some keys can not be tapped, like "space". I tried with "fr", the space key should be "espace".

Besides of that, EarlGrey is great, thanks for making this.

Switch keyplane failed when the responder is a UITextField with secureTextEntry enabled and the language setting is non-English.

The switch keyplane key's accessibilityLabel on Software keyboard triggered by normal UITextField is "more, numbers" or "more, letters" in any language setting.
But with secureTextEntry enabled, the accessibilityLabel of the key will be localized, which failed every test related to this.
Currently I'm using KIF's mechanism to bypass this, which directly calls the first responder view's "setText:" method.

Great solution for UITesting,thx:)

Swift support & example?

Is it possible to use EarlGrey with Swift? If so, are there any plans for an example repository to demonstrate simple usage?

Carthage 0.15.2 Failed to read file or folder at..

I am having trouble using EarlGrey with Carthage. I managed to build it with carthage by specifying master branch as source but then when I try to run it I get " Failed to read file or folder at"
which I found similar to issue Carthage/Carthage#1096

I was looking at EarlGrey-Info.plist and did not find CFBundlePackageType property with FMWK value instead there is APPL.

Should we create another .plist for the xcscheme for Carthage or can we just change APPL to FMWK?

t

G

Unable to Pod Install in the demo project

When I'm trying to run 'pod install' the following error pops up

[!] Invalid Podfile file: undefined method `project' for #Pod::Podfile:0x007fe73c1918f8. Updating CocoaPods might fix the issue.

I've tried this on multiple combinations of Cocoapods (using both stable and latest --pre) and Ruby, nothing seems to work.

Pan from edge gesture fails on devices

On devices, the following code which tries to perform a Pan-From-Edge gesture fails:

[[EarlGrey selectElementWithMatcher:grey_keyWindow()] performAction:grey_swipeSlowInDirectionWithStartPoint(kGREYDirectionRight, 0.001f, 0.5f)]

I've verified it fails on at least the following devices:

  • iPhone 5s iOS 9.3
  • iPhone 6s iOS 9.3
  • iPad iOS 7.0

Assertion failure in -[GREYElementProvider dataEnumerator]

I keep getting this issue where I navigate to another view controller then then nothing works,
I do have a custom matcher and I can see that it does match the element I'm looking but I cannot do anything with it. As this is the error I'm getting.

For some reason it's only on a specific view controller where it doesn't work but everything else works fine.

*** Assertion failure in -[GREYElementProvider dataEnumerator], /Users/tirodkar/egg/google3/third_party/objective_c/EarlGrey/EarlGrey/Provider/GREYElementProvider.m:152

0   CoreFoundation                      0x000000010cba8e65 __exceptionPreprocess + 165
1   libobjc.A.dylib                     0x000000010c61edeb objc_exception_throw + 48
2   CoreFoundation                      0x000000010cba8cca +[NSException raise:format:arguments:] + 106
3   Foundation                          0x000000010c0344de -[NSAssertionHandler handleFailureInMethod:object:file:lineNumber:description:] + 198
4   EarlGrey                            0x0000000121017437 __37-[GREYElementProvider dataEnumerator]_block_invoke + 2711
5   EarlGrey                            0x0000000121015708 -[GREYDataEnumerator nextObject] + 72
6   CoreFoundation                      0x000000010cabd2f5 -[NSEnumerator countByEnumeratingWithState:objects:count:] + 53
7   EarlGrey                            0x0000000120ff1e79 -[GREYElementFinder elementsMatchedInProvider:] + 1817
8   EarlGrey                            0x0000000120ff280d -[GREYElementInteraction matchedElementsWithTimeout:error:] + 1117
9   EarlGrey                            0x0000000120ff42e6 __46-[GREYElementInteraction performAction:error:]_block_invoke + 502
10  EarlGrey                            0x00000001210248bd __59-[GREYUIThreadExecutor executeSyncWithTimeout:block:error:]_block_invoke + 429
11  CoreFoundation                      0x000000010cad4a1c __CFRUNLOOP_IS_CALLING_OUT_TO_A_BLOCK__ + 12
12  CoreFoundation                      0x000000010caca6a5 __CFRunLoopDoBlocks + 341
13  CoreFoundation                      0x000000010cac9e02 __CFRunLoopRun + 850
14  CoreFoundation                      0x000000010cac9828 CFRunLoopRunSpecific + 488
15  EarlGrey                            0x000000012102338c -[GREYUIThreadExecutor executeSyncWithTimeout:block:error:] + 1932
16  EarlGrey                            0x0000000120ff3981 -[GREYElementInteraction performAction:error:] + 2049
17  EarlGrey                            0x0000000120ff310c -[GREYElementInteraction performAction:] + 76
18  mip_write_review_test               0x0000000120f87380 -[MIPWriteReviewTest test14TapReviewFromProfileScreen] + 240
19  CoreFoundation                      0x000000010ca971cc __invoking___ + 140

#import "EarlGrey/EarlGrey.h" not found

I installed the project by manual and with object c project. After I followed the install document and try to do something, the above error occured and I can click the EarlGreay keyword goto the file.
Also @import module will failed.

I created a new project and the .xcodeproj file in under the root dic. But the one which failed the orignal .xcodeproj is under e.g. source/projectname/ After I set the file path in settings like "$(SRCROOT)/EarlGrey.xcodeproj" the full file path is like ..../source/projectname/...earlGrey.xcodeproj. So maybe this is the issue? How do I drag the xcodeproj to the project and how to set the ring file path here? Thanks.

EarlGrey crash when trying to test a game using Spritekit

I tried to do an example test in my test project with below code.
[EarlGrey selectElementWithMatcher:grey_keyWindow()];
[[EarlGrey selectElementWithMatcher:grey_keyWindow()] assertWithMatcher:grey_sufficientlyVisible()];

The first line worked fine but the second one got crash. Any idea why this happen?

Support XC UI Testing

Integrate EarlGrey with XC UI Testing. Please follow this issue for progress updates.

EarlGrey() has wrong line number for page objects

In EarlGrey.swift, EarlGrey is defined as:

public func EarlGrey() -> EarlGreyImpl! {
  return EarlGreyImpl.invokedFromFile(#file, lineNumber: #line)
}

I define a login page object:

public class LoginPage {

    //# MARK: - Login Page Elements

    private var domainField: GREYElementInteraction {
        return EarlGrey().selectElementWithMatcher(grey_accessibilityID("domain_field"))
    }

which is then used:

func assertPageObjects() {
    domainField.assertWithMatcher(grey_sufficientlyVisible())
}

The line number is wrong because EarlGrey() sets it to be on the EarlGrey().selectElementWithMatcher... line.

I've had to update EarlGrey.swift like this:

//# MARK: - Custom EarlGrey file/line

extension EarlGreyImpl {
    // Use @nonobjc to fix A declaration cannot be both 'final' and 'dynamic' error
    @nonobjc public static var file: String = ""
    @nonobjc public static var line: UInt = 0

    public static func setFromFile(file: String, _ line: UInt) {
        self.file = file
        self.line = line
    }
}

public func grey_invokedFromFile(file: String, _ line: UInt) {
    EarlGreyImpl.setFromFile(file, line)
}

public func EarlGrey() -> EarlGreyImpl! {
  return EarlGreyImpl.invokedFromFile(EarlGreyImpl.file, lineNumber: EarlGreyImpl.line)
}

and update the page object:

func assertPageObjects(file: String = #file, _ line: UInt = #line) {
    grey_invokedFromFile(file, line)

    domainField.assertWithMatcher(grey_sufficientlyVisible())
}

now the line number is correctly identified in the test. Thoughts on how to best update EarlGrey.swift to handle this use case?

about the source url

When I install the EarlGrey with the command pod install --verbose --no-repo-update, I got the failure.

Downloading dependencies

-> Installing EarlGrey (1.0.0)

Http download
$ /usr/bin/curl -f -L -o
/var/folders/f8/40y7v4md00xdlvxnfy00stm40000gn/T/d20160219-30346-1tpz669/file.zip
http://www.github.com/google/EarlGrey/releases/download/1.0.0/EarlGrey-1.0.0.zip --create-dirs
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- 0:00:02 --:--:-- 0curl: (56) Recv failure: Connection reset by peer

While downloading EarlGrey, the request is http not https.

In the file EarlGrey.podspec.json, the source is the http request
"source": {
"http": "http://www.github.com/google/EarlGrey/releases/download/1.0.0/EarlGrey-1.0.0.zip"
},

How can I change the path to https?

Some functional tests run very slow, causing travis to abort the test

From https://travis-ci.org/google/EarlGrey/jobs/118931370

The following tests take an enormous amount of time to complete:

  1. testBasicInteractionWithViews (31.436 seconds)
  2. testActionErrorAPI (23.584 seconds)
  3. testEmailTyping (81.767 seconds)
  4. testLetterTyping (26.599 seconds)
  5. testNumbersTyping (38.642 seconds)
  6. testTypingWordsThatTriggerAutoCorrect (154.971 seconds)
  7. testUIKeyboardTypeEmailAddress (46.818 seconds)
  8. testUIKeyboardTypeTwitter (61.335 seconds)
  9. testUIKeyboardTypeURL (75.466 seconds)
  10. testUpperCaseLettersTyping (124.599 seconds)
  11. testInteractingWithElementsAfterRotation (53.374 seconds)
  12. testScrollToTopWorksWithNegativeInsets (38.443 seconds)
  13. testScrollToTopWorksWithPositiveInsets (39.259 seconds)
  14. testJavaScriptExecution (49.297 seconds)

Swift conditional example

Is this the correct way to handle pointers with Swift?

I think this would be useful as a FAQ entry. It works in my test suite.

GREYCondition(name: "Wait for collection view to populate", block: { _ in
    let errorOrNil = UnsafeMutablePointer<NSError?>.alloc(1)
    errorOrNil.initialize(nil)

    EarlGrey().selectElementWithMatcher(collectionViewMatcher)
              .assertWithMatcher(grey_notNil(), error: errorOrNil)

    let success = errorOrNil.memory == nil
    errorOrNil.destroy()
    errorOrNil.dealloc(1)

    return success
}).waitWithTimeout(20.0)

Build Failed - Semantic Issue

The EarlGrey project is not building. [Xcode ver 6.4(6E35b)]
Steps:

  1. Clones EarlGrey repo.
  2. Ran 'setup-earlgrey.sh'.
  3. Opened EarlGrey.xcodeproj and tried building the project.

Error:
GREYTapAction.m - Semantic Issue Method override for the designated initializer of the superclass '-initWithName:constraints:' not found

image

EarlGrey Failure - “Keyboard did not disappear after resigning first responder status”

This error occurs when attempting to use the action to type text into a textfield.

If you chain responder status between textfields, EarlGrey will fail, reporting that "Keyboard did not disappear after resigning first responder status". This is because when your client implementation immediately moves firstResponder to a new textfield, the keyboard will not disappear (nor should it).

This error occurs because EarlGrey checks for Auto Correction on the textfield, disables autocorrection on the textfield, and then calls "resignFirstResponder" followed by "becomeFirstResponder", in order to effect the change. If your app chains firstResponder status, then the first call to "resignFirstResponder" will cause your app to pass responder status to the next text field.

Inspecting the EarlGrey source, it looks like the issue could be on line 182 in GREYActions.m. Here, we aren't checking to see that auto correction status is actually on. So even if your text field doesn't use auto correction, this will always invoke "resignFirstResponder" followed by "becomeFirstResponder". We should actually check to make sure that the prior value was "on" before performing this dance.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.