xcpretty / xcpretty Goto Github PK

View Code? Open in Web Editor NEW

3.9K 69.0 284.0 3.73 MB

Flexible and fast xcodebuild formatter

License: Other

Ruby 79.20% HTML 6.41% Gherkin 14.38%

ios ci continuous-integration

xcpretty's Introduction

xcpretty is a fast and flexible formatter for xcodebuild.
It does one thing, and it should do it well.

Installation

$ gem install xcpretty

Usage

$ xcodebuild [flags] | xcpretty

xcpretty is designed to be piped with xcodebuild and thus keeping 100% compatibility with it. It's even a bit faster than xcodebuild itself, since it saves your terminal some prints.

Important: If you're running xcpretty on a CI like Travis or Jenkins, you may want to exit with same status code as xcodebuild. CI systems usually use status codes to determine if the build has failed.

$ set -o pipefail && xcodebuild [flags] | xcpretty
#
# OR
#
$ xcodebuild [flags] | xcpretty && exit ${PIPESTATUS[0]}

Raw xcodebuild output

You might want to use xcpretty together with tee to store the raw log in a file, and get the pretty output in the terminal. This might be useful if you want to inspect a failure in detail and aren't able to tell from the pretty output.

Here's a way of doing it:

$ xcodebuild [flags] | tee xcodebuild.log | xcpretty

Formats

--simple, -s (default)
--test, -t (RSpec style)
--tap (Test Anything Protocol-compatible output)
--knock, -k (a simplified version of the Test Anything Protocol)

ANSI / UTF-8

--[no-]color: Show build icons in color. (you can add it to --simple or --test format). Defaults to auto-detecting color availability.
--[no-]utf: Use unicode characters in build output or only ASCII. Defaults to auto-detecting the current locale.

Reporters

--report junit, -r junit: Creates a JUnit-style XML report at build/reports/junit.xml, compatible with Jenkins and TeamCity CI.
--report html, -r html: Creates a simple HTML report at build/reports/tests.html.
--report json-compilation-database, -r json-compilation-database: Creates a JSON compilation database at build/reports/compilation_db.json. This is a format to replay single compilations independently of the build system.

Writing a report to a custom path can be specified using --output PATH.

Extensions

xcpretty supports custom formatters through the use of the --formatter flag, which takes a path to a file as an argument. The file must contain a Ruby subclass of XCPretty::Formatter, and return that class at the end of the file. The class can override the format_* methods to hook into output parsing events.

Known extensions

xcpretty-travis-formatter: support for cleaner output in TravisCI using code folding

The recommended format is a gem containing the formatter and named with an xcpretty- prefix, for easier discovery.

Team

xcpretty's People

Contributors

Stargazers

Watchers

Forkers

fabiopelosin lingling2012 fritexvz pingwinator-archive alloy sypproject jakecataford libin vickeryj shahzadmajeed donjordano bengottlieb schukin neonichu alcherk evobilis simonekalb jonaswitt foozmeat ogantopkaya viteinfinite sethkontny nvdnkpr mohammedeltayeb ssilk attomos hari-tw antonyzhao cenllv krutarth ohadschn robertwijas mangofever mtauraso norio-nomura ufosky-server yas375 ashton-w greenhouseci developervox sbyer sependa ashfurrow poplax 0xced davieds creantan anwol004 pspdfkit-labs pcantrell mfrawley duk42111 revolter antonini sadisasha pikuseru sebreh refreshwork rnhsp ambhojgoyal sndgeek monits lobster-king pofeng2000 jaredgrubb benasher44 huynguyen masbog gmorales-phunware backelite birmacher codeguru85 mtitolo korzonek timrbula chrisfsampaio raeidsaqur brunodlz andybest daerwang 23andme tanujkumar1640 cm-kato-jun finnchen felixlinbh guyo714 jwfriese is00hcw ashgti cn-chengjiangang dhanushram irshadpc ruudputs will-pb mgrebenets billbai potemkinaleksey saltzmanjoelh jonnyyu urmyfaith

xcpretty's Issues

JUnit support for performance testing

Errors are not printed

When I pipe my xcodebuild output through xcpretty the error describing eg. syntax errors in the code are not shown on screen.

If for instance I omit a semicolon at the end of a statement in a source file I get output like this:

The following build commands failed:
CompileC build/Intermediates/MyApp.build/Inhouse-iphoneos/MyAppStaging.build/Objects-normal/armv7/AbstractRefreshingTableViewController.o MyApp/Controllers/AbstractRefreshingTableViewController.m normal armv7 objective-c com.apple.compilers.llvm.clang.1_0.compiler
(1 failure)

But without xcpretty I get the much more useful error

/repos/Apps/MyApp/iOS/MyApp/Controllers/AbstractRefreshingTableViewController.m:45:30: error: expected ';' after expression
[self.tableView reloadData]
^
;
1 error generated.

(although hidden in thousands of lines of spammy output).

How can I get xcpretty to output the detailed error?

US-ASCII encoding problems

during test I get the following exception:
/Users/admin/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/gems/2.0.0/gems/xcpretty-0.1.0/lib/xcpretty.rb:25:in =~': invalid byte sequence in US-ASCII (ArgumentError)
[15:07:47][Step 4/9] from /Users/admin/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/gems/2.0.0/gems/xcpretty-0.1.0/lib/xcpretty.rb:25:in block in handle' [15:07:47][Step 4/9] from /Users/admin/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/gems/2.0.0/gems/xcpretty-0.1.0/lib/xcpretty.rb:24:in each'
[15:07:47][Step 4/9] from /Users/admin/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/gems/2.0.0/gems/xcpretty-0.1.0/lib/xcpretty.rb:24:in detect' [15:07:47][Step 4/9] from /Users/admin/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/gems/2.0.0/gems/xcpretty-0.1.0/lib/xcpretty.rb:24:in handle'
[15:07:47][Step 4/9] from /Users/admin/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/gems/2.0.0/gems/xcpretty-0.1.0/bin/xcpretty:66:in block in <top (required)>' [15:07:47][Step 4/9] from /Users/admin/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/gems/2.0.0/gems/xcpretty-0.1.0/bin/xcpretty:65:in each_line'
[15:07:47][Step 4/9] from /Users/admin/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/gems/2.0.0/gems/xcpretty-0.1.0/bin/xcpretty:65:in each_line' [15:07:47][Step 4/9] from /Users/admin/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/gems/2.0.0/gems/xcpretty-0.1.0/bin/xcpretty:65:in <top (required)>'
[15:07:47][Step 4/9] from /Users/admin/.rvm/gems/ruby-2.0.0-p247/bin/xcpretty:23:in load' [15:07:47][Step 4/9] from /Users/admin/.rvm/gems/ruby-2.0.0-p247/bin/xcpretty:23:in

'
[15:07:47][Step 4/9] from /Users/admin/.rvm/gems/ruby-2.0.0-p247/bin/ruby_noexec_wrapper:14:in eval' [15:07:47][Step 4/9] from /Users/admin/.rvm/gems/ruby-2.0.0-p247/bin/ruby_noexec_wrapper:14:in '`

Probably it's related to #36.
What kind of information can I provide in addition?

Compile failure message gets consumed

The test output is great for both success / fail scenarios.

However when compiling if things don't go well the behavior is a little ambiguous. We seem to simply get an error indicating that it didn't work.

Let me know if I can help to fix this. And thanks again for xcpretty - fantastic tool.

Parsing fails (invalid byte sequence in US-ASCII)

In this Travis build, the parser fails with the following error message (ruby 2.0.0p247, xcpretty 0.1.4):

/Users/travis/.rvm/gems/ruby-2.0.0-p247/gems/xcpretty-0.1.4/lib/xcpretty/parser.rb:265:in `===': invalid byte sequence in US-ASCII (ArgumentError)
    from /Users/travis/.rvm/gems/ruby-2.0.0-p247/gems/xcpretty-0.1.4/lib/xcpretty/parser.rb:265:in `update_test_state'
    from /Users/travis/.rvm/gems/ruby-2.0.0-p247/gems/xcpretty-0.1.4/lib/xcpretty/parser.rb:182:in `parse'
    from /Users/travis/.rvm/gems/ruby-2.0.0-p247/gems/xcpretty-0.1.4/lib/xcpretty/formatters/formatter.rb:63:in `pretty_format'
    from /Users/travis/.rvm/gems/ruby-2.0.0-p247/gems/xcpretty-0.1.4/lib/xcpretty/printer.rb:15:in `pretty_print'
    from /Users/travis/.rvm/gems/ruby-2.0.0-p247/gems/xcpretty-0.1.4/bin/xcpretty:73:in `block in <top (required)>'
    from /Users/travis/.rvm/gems/ruby-2.0.0-p247/gems/xcpretty-0.1.4/bin/xcpretty:72:in `each_line'
    from /Users/travis/.rvm/gems/ruby-2.0.0-p247/gems/xcpretty-0.1.4/bin/xcpretty:72:in `each_line'
    from /Users/travis/.rvm/gems/ruby-2.0.0-p247/gems/xcpretty-0.1.4/bin/xcpretty:72:in `<top (required)>'
    from /Users/travis/.rvm/gems/ruby-2.0.0-p247/bin/xcpretty:23:in `load'
    from /Users/travis/.rvm/gems/ruby-2.0.0-p247/bin/xcpretty:23:in `<main>'
    from /Users/travis/.rvm/gems/ruby-2.0.0-p247/bin/ruby_executable_hooks:15:in `eval'
    from /Users/travis/.rvm/gems/ruby-2.0.0-p247/bin/ruby_executable_hooks:15:in `<main>'

This seems to be related to the Travis configuration (Mac OS X 10.8, Xcode 5.0.2). I can't reproduce this error on my machine (Mac OS X 10.9, Xcode 5.1).

That's not much to go on, but do you have an idea what could be the problem?

Prettify test titles

Not sure what you think about this @supermarin, but I think it would be really nice to parse the test names and remove certain characters. This makes the output a little more readable.

Take the following example

FFGLocationControllerSpec
    ✓ Location_Controller_current_location_monitoring_should_raise_an_exception_when_releasing_a_location_without_requsting_a_location (0.001 seconds)
    ✓ Location_Controller_current_location_monitoring_should_request_location_when_location_monitoring_is_available (0.007 seconds)
    ✓ Location_Controller_current_location_monitoring_should_allow_us_to_release_location_after_successfully_requesting_location (0.002 seconds)
    ✓ Location_Controller_current_location_monitoring_should_fail_to_request_location_if_location_monitoring_is_not_available (0.006 seconds)

Could be converted to:

FFGLocationControllerSpec
    ✓ Location Controller, current location monitoring should raise an exception when releasing a location without requesting a location (0.001 seconds)
    ✓ Location Controller, current location monitoring should request location when location monitoring is available (0.007 seconds)
    ✓ Location Controller, current location monitoring should allow us to release location after successfully requesting location (0.002 seconds)
    ✓ Location Controller, current location monitoring should fail to request location if location monitoring is not available (0.006 seconds)

By substituting _ for a space. Let me know if you like this, I could probably make a pull request 👍.

`===': invalid byte sequence in US-ASCII (ArgumentError)

I had xcpretty working really well on a Jenkins CI server but I am now getting the following error. The error is only occurring on one of my projects.

xcodebuild \
-scheme SchemeName \
-sdk iphoneos \
-destination "name=Dev iPhone 5" \
-config Debug \
test \
| xcpretty \
-s \
--no-utf \
-r junit \
--output Reports/junit.xml \
&& exit ${PIPESTATUS[0]}

xcodebuild[19512:4a2f]  DeveloperPortal: Using pre-existing current store at URL (file:///Users/Administrator/Library/Developer/Xcode/DeveloperPortal%205.1.1.db).
/Library/Ruby/Gems/2.0.0/gems/xcpretty-0.1.5/lib/xcpretty/parser.rb:288:in `===': invalid byte sequence in US-ASCII (ArgumentError)
    from /Library/Ruby/Gems/2.0.0/gems/xcpretty-0.1.5/lib/xcpretty/parser.rb:288:in `update_test_state'
    from /Library/Ruby/Gems/2.0.0/gems/xcpretty-0.1.5/lib/xcpretty/parser.rb:202:in `parse'
    from /Library/Ruby/Gems/2.0.0/gems/xcpretty-0.1.5/lib/xcpretty/formatters/formatter.rb:65:in `pretty_format'
    from /Library/Ruby/Gems/2.0.0/gems/xcpretty-0.1.5/lib/xcpretty/printer.rb:15:in `pretty_print'
    from /Library/Ruby/Gems/2.0.0/gems/xcpretty-0.1.5/bin/xcpretty:74:in `block in <top (required)>'
    from /Library/Ruby/Gems/2.0.0/gems/xcpretty-0.1.5/bin/xcpretty:73:in `each_line'
    from /Library/Ruby/Gems/2.0.0/gems/xcpretty-0.1.5/bin/xcpretty:73:in `each_line'
    from /Library/Ruby/Gems/2.0.0/gems/xcpretty-0.1.5/bin/xcpretty:73:in `<top (required)>'
    from /usr/bin/xcpretty:23:in `load'
    from /usr/bin/xcpretty:23:in `<main>'

Formatting issue in Jenkins

I'm not exactly sure why this happens. . however, Jenkins is not printing the pretty icons, and is instead mojibaking them.

It looks like the attached picture:

Are you using UTF-8 to print those to the terminal? Or are they special terminal commands (never done something like that before).

It might be related to file encoding. . . the other day, I noticed Jenkins tell me: "CocoaPods requires UTF-8 encoding". . . meaning perhaps the server was started in another encoding.

Some errors get swallowed

It's all nice when the build passes, but xcodebuild is often printing error messages to stdout instead of stderr.

Some of them are not implemented, I'll try to list some here:

haven't caught the whole output, but starts with fatal error:
Check dependencies Code Sign error: No code signing identites found: No valid signing identities (i.e. certificate and private key pair) matching the team ID ‚ÄúCAT6HF57NJ‚Äù were found. CodeSign error: code signing is required for product type 'Application' in SDK 'iOS 7.0'

Show only failures

When there are more than 30 or so tests, the total number of lines of RSpec output exceeds what can be displayed in a terminal window.

This means that some failures will not easily be seen by a developer when the tests are re-run.

xcpretty is supressing certain failures (application terminating due to uncaught exception)

XCPretty says it failed but doesn't indicate why (I have to manually run xcodebuild to find out):

$ make tests
=> Running tests
** TEST FAILED **

 ▸ Building Pods/Pods-Redacted-AFNetworking [Debug]
 ▸ Building Pods/Pods-Redacted-GoogleAnalytics-iOS-SDK [Debug]
 ▸ Building Pods/Pods-Redacted-BPXLUUIDHandler [Debug]
 ▸ Building Pods/Pods-Redacted-TestFlightSDK [Debug]
 ▸ Building Pods/Pods-Redacted-CCLDefaults [Debug]
 ▸ Building Pods/Pods-Redacted-FLKAutoLayout [Debug]
 ▸ Building Pods/Pods-Redacted-FormatterKit [Debug]
 ▸ Building Pods/Pods-Redacted-ARAnalytics [Debug]
 ▸ Building Pods/Pods-Redacted-ISO8601DateFormatter [Debug]
 ▸ Building Pods/Pods-Redacted-KFData [Debug]
 ▸ Building Pods/Pods-Redacted-MFLImageView [Debug]
 ▸ Building Pods/Pods-Redacted-MFStoryboardPushSegue [Debug]
 ▸ Building Pods/Pods-Redacted-Mantle [Debug]
 ▸ Building Pods/Pods-Redacted-NMRangeSlider [Debug]
 ▸ Building Pods/Pods-Redacted-NSAttributedString+CCLFormat [Debug]
 ▸ Building Pods/Pods-Redacted-ORStackView [Debug]
 ▸ Building Pods/Pods-Redacted-ReactiveCocoa [Debug]
 ▸ Building Pods/Pods-Redacted-Reveal-iOS-SDK [Debug]
 ▸ Building Pods/Pods-Redacted-SSKeychain [Debug]
 ▸ Building Pods/Pods-Redacted-SVProgressHUD [Debug]
 ▸ Building Pods/Pods-Redacted-libextobjc [Debug]
 ▸ Building Pods/Pods-RedactedTests-CCLRequestReplay [Debug]
 ▸ Building Pods/Pods-RedactedTests-Expecta [Debug]
 ▸ Building Pods/Pods-RedactedTests-Specta [Debug]
 ▸ Building Pods/Pods-RedactedTests-Stubbilino [Debug]
 ▸ Building Pods/Pods-Redacted [Debug]
 ▸ Building Redacted/Redacted [Debug]
 ▸ Running script 'Check Pods Manifest.lock'
 ▸ Running script 'Set Application Version'
 ▸ Running script 'Copy Pods Resources'
 ▸ Building Pods/Pods-RedactedTests [Debug]
 ▸ Building Redacted/RedactedTests [Debug]
 ▸ Running script 'Check Pods Manifest.lock'
 ▸ Running script 'Copy Pods Resources'
 All tests
 Test Suite RedactedTests.xctest started
 FFGAPIClientSpec

xcodebuild output:

014-05-16 16:42:43.570 Redacted[66921:60b] FFGDataStore: Error loading data store (Error Domain=NSCocoaErrorDomain Code=134130 "The operation couldn’t be completed. (Cocoa error 134130.)" UserInfo=0xba7fd50 {URL=file:///Users/kylef/Library/Application%20Support/iPhone%20Simulator/7.1/Applications/5E12E40F-BB6A-4AB2-984D-4ED41CD070CB/Documents/DataStores/Production, metadata={
    NSPersistenceFrameworkVersion = 479;
    NSStoreModelVersionHashes =     {
        <redacted>
    };
    NSStoreModelVersionHashesVersion = 3;
    NSStoreModelVersionIdentifiers =     (
        ""
    );
    NSStoreType = SQLite;
    NSStoreUUID = "B10D1D38-7DD0-4285-9464-CB8FFCEC8D6E";
    "_NSAutoVacuumLevel" = 2;
}, reason=Can't find model for source store})
2014-05-16 16:42:43.572 Redacted[66921:60b] Defaults: Unknown country code (US) defaulting to GB
2014-05-16 16:42:43.573 Redacted[66921:60b] Forcing language to en-GB (instead of en)
2014-05-16 16:42:43.589 Redacted[66921:60b] Defaults: Unknown country code (US) defaulting to GB
2014-05-16 16:42:43.682 Redacted[66921:60b]  INFO: Reveal server started.
= All tests
  Test Suite 'All tests' started at 2014-05-16 15:42:44 +0000

  = Redacted.xctest
    Test Suite 'Redacted.xctest' started at 2014-05-16 15:42:44 +0000

    = FFGAPIClientSpec
      Test Suite 'FFGAPIClientSpec' started at 2014-05-16 15:42:44 +0000

      -[FFGAPIClientSpec an_API_client_should_set_the_language_header_for_the_user_s_language]
        Test Case '-[FFGAPIClientSpec an_API_client_should_set_the_language_header_for_the_user_s_language]' started.
2014-05-16 16:42:44.508 Redacted[66921:3503] *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'This NSPersistentStoreCoordinator has no persistent stores.  It cannot perform a save operation.'
*** First throw call stack:
(
    0   CoreFoundation                      0x02c521e4 __exceptionPreprocess + 180
    1   libobjc.A.dylib                     0x028248e5 objc_exception_throw + 44
    2   CoreData                            0x00971b0c -[NSPersistentStoreCoordinator executeRequest:withContext:error:] + 5260
    3   CoreData                            0x0099e56c -[NSManagedObjectContext save:] + 764
    4   Redacted                       0x001dffa6 __63-[NSManagedObjectContext(KFData) performWriteBlock:completion:]_block_invoke + 134
    5   CoreData                            0x00a3bfe3 developerSubmittedBlockToNSManagedObjectContextPerform_privateasync + 83
    6   libdispatch.dylib                   0x02f664d0 _dispatch_client_callout + 14
    7   libdispatch.dylib                   0x02f54047 _dispatch_queue_drain + 452
    8   libdispatch.dylib                   0x02f53e42 _dispatch_queue_invoke + 128
    9   libdispatch.dylib                   0x02f54de2 _dispatch_root_queue_drain + 78
    10  libdispatch.dylib                   0x02f55127 _dispatch_worker_thread2 + 39
    11  libsystem_pthread.dylib             0x03295dab _pthread_wqthread + 336
    12  libsystem_pthread.dylib             0x03299cce start_wqthread + 30
)
libc++abi.dylib: terminating with uncaught exception of type NSException

Add building... indicator on RSpec formatter

It'd probably be good to have an animated progress indicator while it's building with the RSpec format.

With larger projects and when using xclisten, sometimes build times can take longer and you're not sure if everything's ok.

oclint needs "include" command to link to the pch in the project folder

This issue occurs when building for OCLint. The "include" command should not link to the pch located in the DerivedData folder but it should rather contain the absolute path of the pch in the working folder, e.g:

/Users/USER/Library/Developer/Xcode/DerivedData/MyProject-Library-fsljtldcjttttseqpqryyyihhqjz/Build/Intermediates/PrecompiledHeaders/MyProject-Library-Prefix-uyziozhtyxkrezcmujrjnshapoct/MyProject-Library-Prefix.pch

should be transformed into

/Users/USER/Works/MyProject-Library/MyProject-Library/MyProject-Library-Prefix.pch

Put a UTF-8 BOM at the start of xcretty output

I've run into a number or problems regarding file encoding with xcpretty.

The first was getting the output to format correctly in Jenkins by setting the file.encoding property to UTF-8. . (this may depend on your locale and/or workstation).

The 2nd problem occurred when I published the generated report to GitHub-pages, so that I could link to it from my project's ReadMe.

Again, the browser would display the file all mojibaked, with none of the pretty checkmarks, etc occurring.

To fix this problem, I added a UTF-8 BOM at the start of the text file, and it now looks good:

http://jasperblues.github.io/XcodeEditor/build-status/test-results-pretty.txt

Perhaps xcpretty could do this at the start of the output by default. . I can't see it causing any problems, and I think it would fix a lot of issues that otherwise require painful config.

For reference here's the build-file that does this - calling out to a script-called utf8-bom.sh

<echo file="${temp.dir}/runTests.sh">
            #!/bin/sh
            xcodebuild -workspace ${module.name}.xcworkspace test -scheme Tests -configuration Debug > ${reports.dir}/tests/test-results.txt
            xcpretty ${reports.dir}/tests/test-results.txt > ${reports.dir}/tests/test-results-pretty.txt
            ${scripts.dir}/utf8-bom.sh ${reports.dir}/tests/test-results-pretty.txt
        </echo>
        <chmod perm="+x" file="${temp.dir}/runTests.sh"/>
        <exec executable="${temp.dir}/runTests.sh" failonerror="true" failifexecutionfails="true"/>

And the contents of the script are in this SO answer:

http://stackoverflow.com/questions/20697616/set-file-encoding-attributes-of-txt-file-on-webserver/20699001#20699001

Ability to suppress build output

I'm not that interested in the build process unless it fails, I'd love to be able to skip having xcpretty output that.

How to integrate with Jenkins?

Our iOS build is done with Jenkins using the Xcode Plugin:
https://wiki.jenkins-ci.org/display/JENKINS/Xcode+Plugin

I was trying to integrate XCPretty into this since it looks very useful.

I tried adding to the "Custom xcodebuild arguments", e.g.:

BUILD_NUMBER=${BUILD_NUMBER} ONLY_ACTIVE_ARCH=NO | xcpretty

However I get an error in Jenkins:

[workspace] $ /usr/bin/xcodebuild -scheme makeshift -workspace iOS/makeshift.xcworkspace -configuration Dev clean build CONFIGURATION_BUILD_DIR=/Users/Shared/Jenkins/Home/jobs/makeShift-iOS-Dev-TestFlight/workspace/iOS/build BUILD_NUMBER=240 ONLY_ACTIVE_ARCH=NO | xcpretty
Build settings from command line:
    BUILD_NUMBER = 240
    CONFIGURATION_BUILD_DIR = /Users/Shared/Jenkins/Home/jobs/makeShift-iOS-Dev-TestFlight/workspace/iOS/build
    ONLY_ACTIVE_ARCH = NO

xcodebuild: error: Unknown build action '|'.
Build step 'Xcode' marked build as failure

It seems to be choking on the pipe. 😕

I know this isn't an XCPretty bug, per se, but have you guys investigated how to make this work with Jenkins? Would it be possible to update the documentation if so?

Thanks!
Jeremy

Teamcity custom test reporting

Hi! Thanks for the awesome gem, it' the good point for starting the idea I have.
Teamcity has an "API" for build output to generate reports http://confluence.jetbrains.com/display/TCD7/Build+Script+Interaction+with+TeamCity and I want to implement this output format for xcpretty.

My question - do you already have plans about deeper CI integration or should I start working on this feature?

Print full static analyzer results

Thanks for a great project!

Currently xcpretty reports static analyzer issues thusly:

The following commands produced analyzer issues:
    Analyze AWSiOSSDK/src/ThirdParty/JSON/AWS_SBJsonParser.m
    Analyze AWSiOSSDK/src/Amazon.S3/AmazonS3Client.m
    Analyze AWSiOSSDK/src/Amazon.S3/Model/S3CompleteMultipartUploadRequest.m
    Analyze AWSiOSSDK/src/Amazon.S3/Model/S3GetObjectResponse.m
    Analyze HockeySDK/Classes/BITHockeyHelper.m
    Analyze UrbanAirship-iOS-SDK/Airship/External/UA_asi-http-request/UA_ASIHTTPRequest.m
    Analyze UrbanAirship-iOS-SDK/Airship/External/UA_fmdb/UA_FMDatabase.m
    Analyze UrbanAirship-iOS-SDK/Airship/External/UA_ZipFile-OC/UA_minizip/UA_zip.c
    Analyze PonyDebugger/ObjC/PonyDebugger/NSManagedObject+PDRuntimePropertyDescriptor.m
    Analyze PonyDebugger/ObjC/PonyDebugger/PDDefinitions.m
    Analyze /Users/Shared/Jenkins/Home/workspace/IFTTTKit/IFTTTKit/Model/IFTTTChannel.m
(11 commands with analyzer issues)

That text appears at the start of xcpretty's output, but the actual static analyzer issues themselves are never printed. I'd love an option to print full static analyzer output.

xcodebuild failed and xcpretty exited with 0

I'm running into a case where xcodebuild fails, but xcpretty exits with a "0" resulting in Travis flagging the build as a success.

$ xcodebuild -workspace Register.xcworkspace -scheme RegisterAppTests -sdk iphonesimulator7.0 test | xcpretty --color --test
2013-12-13 13:25:11.794 xcodebuild[2314:1207] [MT] DVTAssertions: Warning in /SourceCache/DTDeviceKit/DTDeviceKit-3697/DTDeviceKitBase/DTDeviceKitBase_Utilities.m:503
Details:  Failed to register "/System/Library/CoreServices/CoreTypes.bundle/Contents/Library/MobileDevices.bundle" with Launch Services, error code: -10811.
Function: __RegisterUTIsIfNeeded_block_invoke
Thread:   <NSThread: 0x7fdd4bc0eeb0>{name = (null), num = 1}
Please file a bug at http://bugreport.apple.com with this warning message and any useful information you can provide.
** TEST FAILED **


The following build commands failed:
    CompileC /Users/travis/Library/Developer/Xcode/DerivedData/Register-dzcwixjqhnydsmaymaktjhicoudj/Build/Intermediates/Register.build/Test-iphonesimulator/register.build/Objects-normal/i386/SKCustomerSelectionViewController.o Classes/View\ Controllers/Customer/SKCustomerSelectionViewController.m normal i386 objective-c com.apple.compilers.llvm.clang.1_0.compiler
(1 failure)

The command "xcpretty --color --test" exited with 0.

Done. Your build exited with 0.

Some more errors from stdout don't get printed

ld: library not found for -lPods-AwesomeApp
clang: error: linker command failed with exit code 1 (use -v to see invocation)

If the linker fails with duplicate symbols, it should output the error

First off, thanks for the awesome utility!

In the following output:

duplicate symbol _OBJC_IVAR_$ClassName._ivarName in:
    /Users/username/Library/Developer/Xcode/DerivedData/App-arcyyktezaigixbocjwfhsjllojz/Build/Intermediates/App.build/Debug-iphonesimulator/App.build/Objects-normal/i386/ClassName.o
    /Users/username/Library/Developer/Xcode/DerivedData/App-arcyyktezaigixbocjwfhsjllojz/Build/Products/Debug-iphonesimulator/libPods.a(DuplicateClassName.o)
ld: 1 duplicate symbol for architecture i386
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Xcpretty should output the duplicate symbol. Right now, xcpretty just outputs:

Linking App
⌦ clang: error: linker command failed with exit code 1 (use -v to see invocation)

Cobertura File Format Export

Is it possible to create test coverage report in cobertura format.

oclint needs absolute paths in json-compilation-database

Currently, we only have relative ones in there.

Include failures in the HTML output

HTML output isn't as useful if the build fails.
For having a first-class html view, we need to show why a certain build failed (if it wasn't a failing test).

cc/ @kattrali

Console reporter

Make reporting to the console optional. If you just want a html or junit report you are forced to have the console output.

--report console

junit report file package names

In Jenkins test results for several targets are viewed all together in a root package. If TEST_TARGET is added in front of the class name, tests for each test target will be separated for each test target.

Support for exportArchive command

Are there any plans to support the new -exportArchive command?

Jenkins result code issue

So, i'm trying to integrate XCPretty with my Jenkins setup, and i ran into following issue.

I'm using custom shell command to execute unit tests and if tests succeed - build ipa and upload application to Jenkins. When test fail without xcpretty, build stops there and fails - just like it should. However, with XCPretty Jenkins assumes that build step was successful, even if output prints out that some tests failed. And it goes on and builds ipa and uploads it to TestFlught successfully, which, of course, it should not do.

Here's my xcodebuild command:

xcodebuild clean test -scheme MyScheme -workspace MyProject.xcworkspace -destination "name=iPhone Retina (4-inch),OS=7.0"

and here it is with XCPretty:

xcodebuild clean test -scheme MyScheme -workspace MyProject.xcworkspace -destination "name=iPhone Retina (4-inch),OS=7.0" | xcpretty -t

Maybe i'm missing something?

Summary is printed only once

When there are many destinations (2 iPhones and iPad), Executed 367 tests, with 0 failures (0 unexpected) in 1.138 (1.267) seconds is printed only on the first run.

XCPretty fails although xcodebuild tests succeeded

Two of our test sets are failing although xcodebuild succeeded.

We are using the junit test formatter which reports no failing tests.

xcodebuild -workspace XING.xcworkspace -scheme 'Test Full' -sdk iphonesimulator -destination platform='iOS Simulator',OS=$OS,name='iPhone Retina (4-inch)' clean test | xcpretty -s -c --report junit; exit ${PIPESTATUS[0]}

We too use the ${PIPESTATUS[0]} makro. If we take out xcpretty from our builds the builds run through.

End of the output:

�[32;1mExecuted 93 tests, with 0 failures (0 unexpected) in 1922.860 (1933.212) seconds
�[0m
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
Build step 'Groovy Postbuild' marked build as failure

Test ASCII failure only on Travis?

I'm trying to figure out why xcpretty prints these tests fine locally:

SSCoreDataSourceTests
    ✓ testCountsItems (0.046 seconds)
    ✓ testDeletingItemDeletesRow (0.011 seconds)
    ✓ testForwardsMovesToMoveBlock (0.007 seconds)
    ✓ testInsertingItemInsertsRow (0.007 seconds)
    ✓ testRetrievesItems (0.007 seconds)
    ✓ testUpdatingItemReloadsRow (0.012 seconds)

But xcpretty errors out and fails the build when the same project, with the same build script, is built on Travis:

SSCoreDataSourceTests
/Users/travis/.rvm/gems/ruby-2.0.0-p353/gems/xcpretty-0.1.5/lib/xcpretty/parser.rb:288:in `===': invalid byte sequence in US-ASCII (ArgumentError)
from /Users/travis/.rvm/gems/ruby-2.0.0-p353/gems/xcpretty-0.1.5/lib/xcpretty/parser.rb:288:in `update_test_state'
from /Users/travis/.rvm/gems/ruby-2.0.0-p353/gems/xcpretty-0.1.5/lib/xcpretty/parser.rb:202:in `parse'
from /Users/travis/.rvm/gems/ruby-2.0.0-p353/gems/xcpretty-0.1.5/lib/xcpretty/formatters/formatter.rb:65:in `pretty_format'
from /Users/travis/.rvm/gems/ruby-2.0.0-p353/gems/xcpretty-0.1.5/lib/xcpretty/printer.rb:15:in `pretty_print'
from /Users/travis/.rvm/gems/ruby-2.0.0-p353/gems/xcpretty-0.1.5/bin/xcpretty:74:in `block in <top (required)>'
from /Users/travis/.rvm/gems/ruby-2.0.0-p353/gems/xcpretty-0.1.5/bin/xcpretty:73:in `each_line'
from /Users/travis/.rvm/gems/ruby-2.0.0-p353/gems/xcpretty-0.1.5/bin/xcpretty:73:in `each_line'
from /Users/travis/.rvm/gems/ruby-2.0.0-p353/gems/xcpretty-0.1.5/bin/xcpretty:73:in `<top (required)>'
from /Users/travis/.rvm/gems/ruby-2.0.0-p353/bin/xcpretty:23:in `load'
from /Users/travis/.rvm/gems/ruby-2.0.0-p353/bin/xcpretty:23:in `<main>'
from /Users/travis/.rvm/gems/ruby-2.0.0-p353/bin/ruby_executable_hooks:15:in `eval'
from /Users/travis/.rvm/gems/ruby-2.0.0-p353/bin/ruby_executable_hooks:15:in `<main>'

The command "script/ci" exited with 1.

Piping xcodebuild to xcpretty doesn't show passing tests

When I pipe xcodebuild test to xcpretty -c, I do not get any output regarding passing tests, only if there was a failure:

$ xcodebuild -workspace App.xcworkspace -scheme App -sdk iphonesimulator test | xcpretty -c

yields the following output:

▸ Building App/AppTests [Debug]
All tests
Test Suite AppTests.xctest started
AppSpec

I confirmed that if I run xcodebuild without piping it to xcpretty, I get output related to each passing spec and how long it took.

Versions:

$ xcodebuild -version
Xcode 5.1
Build version 5B130a

$ xcpretty -v
0.1.3

Tets failures aren't printed out on a multiple run

We have a script that runs tests on 2 iPhones and 2 iPads.
The first round passes, and then when an iPad fails, we don't see the failure output + snippet.

option to rename junit reporter output file

I have 3 separate tests targets in the project and it will be nice to add something like
xcpretty -c -r junit -o "report.xml" to have possibility to grab output from each target.

Logo?

Do you have a logo?

That way I can stick it o the CI-build section of my Github projects. .

Build-Passing

*** Reports **

Build Log (powered by XcPretty)
Test Results
Coverage Reports
API Docs

Support for different formats of output

Hi,

I've just discovered your project and it seems promising as a replacement for xctool. I was wondering if there were any plans to support different formats of output like xctool does with its reporters.

Thanks

Format junit report into suites, the way ocunit2junit does

Format junit report into suites/class-names, the way ocunit2junit does .. .

Any plans do do this?

CpResource regular expression doesn't work with relative paths

I have a project where relative paths are used for CpResource operations:

CpResource CocoaChip/Argentina.png build/Release/CocoaChip.app/Contents/Resources/Argentina.png

The regular expression for CpResource matching is expecting these to be absolute paths so xcpretty doesn't output a "Copying" line for this resource.

Travis reports success when used with XCPretty even with failed tests

I got a failed test here.
https://travis-ci.org/seivan/SHAlertViewBlocks/builds/15574058#L213
Travis reports this as a success.

Not sure what the intended behavior is here or if this should be with the Travis team.

Only compilations of .m are considered for output

The COMPILE_MATCHER regex only matches for files with the .m extension. I think other common filetypes in Objective-C projects like .c or .mm should also be considered. Thoughts?

Show pending tests differently from passing and failing ones

It probably makes sense to use a different symbol/color for pending tests.

Unhandled cases

So far, i've caught these when compiling an OSX app:

Touch /Users/musalj/Library/Developer/Xcode/DerivedData/Alcatraz-aobuxcinaqyzjugrnxjjhfzgwaou/Build/Products/Debug/AlcatrazTests.octest
    cd /Users/musalj/code/OSS/Alcatraz
    /usr/bin/touch -c /Users/musalj/Library/Developer/Xcode/DerivedData/Alcatraz-aobuxcinaqyzjugrnxjjhfzgwaou/Build/Products/Debug/AlcatrazTests.octest

TiffUtil eye_icon.tiff
    cd /Users/musalj/code/OSS/Alcatraz
    /usr/bin/tiffutil -cathidpicheck /Users/musalj/code/OSS/Alcatraz/Alcatraz/[email protected] /Users/musalj/code/OSS/Alcatraz/Alcatraz/eye_icon.png -out /Users/musalj/Library/Application\ Support/Developer/Shared/Xcode/Plug-ins/Alcatraz.xcplugin/Contents/Resources/eye_icon.tiff

Feel free to pick it up / submit a PR

Analyze steps are not printed

If a project is configured to use the static analyzer, there will be separate analyze steps during the build like so:

...
Analyze foo.m
cd /Users/dustin/src/foo
setenv LANG en_US.US-ASCII
/Applications/Xcode4.6/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -x c -arch i386 -fmessage-length=0
…

xcpretty doesn't match on /^Analyze/ so these operations are ignored. It would be nice if xcpretty could output "Analyzing foo.m".

Exit status of xcodebuild

I did what is suggested in the README, i.e. to exit with xcodebuild status on Travis.

Then the tests fail yet the build is marked as passed.

I’m not sure if it’s a xcpretty usage issue or a Travis issue.

Locale / Jenkins / Encoding

�/Library/Ruby/Gems/2.0.0/gems/xcpretty-0.0.3/lib/xcpretty/printer.rb:45:in `===': invalid byte sequence in US-ASCII (ArgumentError)
    from /Library/Ruby/Gems/2.0.0/gems/xcpretty-0.0.3/lib/xcpretty/printer.rb:45:in `update_test_state'
    from /Library/Ruby/Gems/2.0.0/gems/xcpretty-0.0.3/lib/xcpretty/printer.rb:36:in `pretty_print'
    from /Library/Ruby/Gems/2.0.0/gems/xcpretty-0.0.3/bin/xcpretty:32:in `block in <top (required)>'
    from /Library/Ruby/Gems/2.0.0/gems/xcpretty-0.0.3/bin/xcpretty:31:in `each_line'
    from /Library/Ruby/Gems/2.0.0/gems/xcpretty-0.0.3/bin/xcpretty:31:in `each_line'
    from /Library/Ruby/Gems/2.0.0/gems/xcpretty-0.0.3/bin/xcpretty:31:in `<top (required)>'
    from /usr/bin/xcpretty:23:in `load'
    from /usr/bin/xcpretty:23:in `<main>'
** TEST FAILED **

"-v" parameter only works when piped with xcodebuild

Possible to have Xcode's cmd + u run through xcpretty?

Couldn't find any documentation on it and figured it might be for the greater good. I'm pretty tired of having unreadable tests in the console.

Suppress log output from xcodebuild

There is a bunch of log information printed at the top of my xcpretty output and I have no idea what it's about, maybe some kind of Xcode bug.

$ make tests
=> Running tests
2014-04-16 14:37:45.110 xcodebuild[2459:1007] [MT] PluginLoading: Required plug-in compatibility UUID A2E4D43F-41F4-4FB9-BB94-7177011C9AED for plug-in at path '~/Library/Application Support/Developer/Shared/Xcode/Plug-ins/XVim.xcplugin' not present in DVTPlugInCompatibilityUUIDs
2014-04-16 14:37:46.314 xcodebuild[2459:1007] [MT] IDELogStore: Log record's backing file ("/Users/kylef/Library/Developer/Xcode/DerivedData/FitnessFirst-ehlmmtklfpmwrjgydjgjpuufphxg/Logs/Test/0DBA141B-E017-40F8-A5A3-0FD27991A518.xcactivitylog") is missing. Skipping.
2014-04-16 14:37:46.315 xcodebuild[2459:1007] [MT] IDELogStore: Log record's backing file ("/Users/kylef/Library/Developer/Xcode/DerivedData/FitnessFirst-ehlmmtklfpmwrjgydjgjpuufphxg/Logs/Test/34012713-E879-432F-AF4C-4AB6BF2A83D6.xcactivitylog") is missing. Skipping.
2014-04-16 14:37:46.315 xcodebuild[2459:1007] [MT] IDELogStore: Log record's backing file ("/Users/kylef/Library/Developer/Xcode/DerivedData/FitnessFirst-ehlmmtklfpmwrjgydjgjpuufphxg/Logs/Test/74F0866D-7002-476F-A2E3-31404439BAC3.xcactivitylog") is missing. Skipping.
2014-04-16 14:37:46.315 xcodebuild[2459:1007] [MT] IDELogStore: Log record's backing file ("/Users/kylef/Library/Developer/Xcode/DerivedData/FitnessFirst-ehlmmtklfpmwrjgydjgjpuufphxg/Logs/Test/7153DA24-3A1C-46CA-AB9E-5801FAC019C2.xcactivitylog") is missing. Skipping.
2014-04-16 14:37:46.315 xcodebuild[2459:1007] [MT] IDELogStore: Log record's backing file ("/Users/kylef/Library/Developer/Xcode/DerivedData/FitnessFirst-ehlmmtklfpmwrjgydjgjpuufphxg/Logs/Test/3A6D009E-0FE9-44AF-BD89-2C2745A0458C.xcactivitylog") is missing. Skipping.
2014-04-16 14:37:46.316 xcodebuild[2459:1007] [MT] IDELogStore: Log record's backing file ("/Users/kylef/Library/Developer/Xcode/DerivedData/FitnessFirst-ehlmmtklfpmwrjgydjgjpuufphxg/Logs/Test/19F6AA74-AF70-48D3-A1A2-45FED326008F.xcactivitylog") is missing. Skipping.
2014-04-16 14:37:46.316 xcodebuild[2459:1007] [MT] IDELogStore: Log record's backing file ("/Users/kylef/Library/Developer/Xcode/DerivedData/FitnessFirst-ehlmmtklfpmwrjgydjgjpuufphxg/Logs/Test/7DE0DA6A-DF94-41B7-B779-82D68B36DFC0.xcactivitylog") is missing. Skipping.
 ▸ Building Pods/Pods-FitnessFirst-AFNetworking [Debug]
 ▸ Building Pods/Pods-FitnessFirst-BPXLUUIDHandler [Debug]
 ▸ Building Pods/Pods-FitnessFirst-TestFlightSDK [Debug]
 ▸ Building Pods/Pods-FitnessFirst-GoogleAnalytics-iOS-SDK [Debug]
 ▸ Building Pods/Pods-FitnessFirst-CCLDefaults [Debug]
 ▸ Building Pods/Pods-FitnessFirst-FormatterKit [Debug]
 ▸ Building Pods/Pods-FitnessFirst-ISO8601DateFormatter [Debug]
 ▸ Building Pods/Pods-FitnessFirst-ARAnalytics [Debug]
 ▸ Building Pods/Pods-FitnessFirst-KFData [Debug]
 ▸ Building Pods/Pods-FitnessFirst-MFLImageView [Debug]
 ▸ Building Pods/Pods-FitnessFirst-MFStoryboardPushSegue [Debug]
 ▸ Building Pods/Pods-FitnessFirst-Mantle [Debug]
 ▸ Building Pods/Pods-FitnessFirst-NMRangeSlider [Debug]
 ▸ Building Pods/Pods-FitnessFirst-NSAttributedString+CCLFormat [Debug]
 ▸ Building Pods/Pods-FitnessFirst-ORStackView [Debug]
 ▸ Building Pods/Pods-FitnessFirst-Reveal-iOS-SDK [Debug]
 ▸ Building Pods/Pods-FitnessFirst-SSKeychain [Debug]
 ▸ Building Pods/Pods-FitnessFirst-SVProgressHUD [Debug]
 ▸ Building Pods/Pods-FitnessFirstTests-Expecta [Debug]
 ▸ Building Pods/Pods-FitnessFirstTests-OHHTTPStubs [Debug]
 ▸ Building Pods/Pods-FitnessFirstTests-Specta [Debug]
 ▸ Building Pods/Pods-FitnessFirstTests-Stubbilino [Debug]
 ▸ Building Pods/Pods-FitnessFirst [Debug]
 ▸ Building FitnessFirst/FitnessFirst [Debug]
 ▸ Running script 'Check Pods Manifest.lock'
 ▸ Running script 'Set Application Version'
 ▸ Running script 'Copy Pods Resources'
 ▸ Building Pods/Pods-FitnessFirstTests [Debug]
 ▸ Building FitnessFirst/FitnessFirstTests [Debug]
 ▸ Running script 'Check Pods Manifest.lock'
 ▸ Compiling FFGUserModelTests.m
 ▸ Linking FitnessFirstTests
 ▸ Generating 'FitnessFirstTests.xctest.dSYM'
 ▸ Running script 'Copy Pods Resources'
 All tests
 Test Suite FitnessFirstTests.xctest started

"User interaction is not allowed message" gets eaten up by XCPretty

This basically happens, when keychain, containing distribution certificate, is locked, or does not allow access from applications. All output that we get is following:

** BUILD FAILED **

The following build commands failed:
CodeSign Build/Products/Nightly-iphoneos/GT-Nightly.app
(1 failure)
** ARCHIVE FAILED **

The following build commands failed:
CodeSign Build/Intermediates/ArchiveIntermediates/Foo/InstallationBuildProductsLocation/Applications/GT-Nightly.app
(1 failure)

Feature request: Detect terminal feature support

xcpretty could use something like tput colors to decide whether to its output should use colors or not. In a similar vein, locale charmap could be used to choose a default output encoding.

Just a suggestion. Great project!