Git Product home page Git Product logo

fastlane-plugin-firebase_test_lab's Introduction

Firebase Test Lab plugin for fastlane

This project is a fastlane plugin. You can add it to your fastlane project by running

fastlane add_plugin firebase_test_lab

fastlane Firebase Test Lab plugin

About Firebase Test Lab plugin

Firebase Test Lab let you easily test your iOS app (Android support forthcoming) on a variety of real or virtual devices and configurations. This plugin allows you to submit your app to Firebase Test Lab by adding an action into your Fastfile.

Getting started

If you are not current user of Firebase

You need to set up Firebase first. These only needs to be done once for an organization.

Configure Google credentials through service accounts

To authenticate, Google Cloud credentials will need to be set for any machine where fastlane and this plugin runs on.

If you are running this plugin on Google Cloud Compute Engine, Kubernetes Engine or App Engine flexible environment, a default service account is automatically provisioned. You will not need to create a service account. See this for more details.

In all other cases, you need to configure the service account manually. You can follow this guide on how to create a new service account and create a key for it. You will need to set the GOOGLE_APPLICATION_CREDENTIALS environment variable pointing to the service account key file according to the document.

No matter if you are a using an automatically provisioned service account or a manually created one, the service account must be configured to have the project editor role.

Enable relevant Google APIs

  • You need to enable the following APIs on your Google Cloud API library (see this for instructions how):
    1. Cloud Testing API
    2. Cloud Tool Results API

Find the devices you want to test on

If you have gcloud tool, you can run

gcloud beta firebase test ios models list

to get a list of supported devices and their identifiers.

Alternatively all available devices can also be seen here.

Actions

firebase_test_lab_ios_xctest

Submit your iOS app to Firebase Test Lab and run XCTest. Refer to this document for more details about Firebase Test Lab specific arguments.

scan(
  scheme: 'YourApp',                  # XCTest scheme
  clean: true,                        # Recommended: This would ensure the build would not include unnecessary files
  skip_detect_devices: true,          # Required
  build_for_testing: true,            # Required
  sdk: 'iphoneos',                    # Required
  should_zip_build_products: true     # Must be true to set the correct format for Firebase Test Lab
)
firebase_test_lab_ios_xctest(
  gcp_project: 'your-google-project', # Your Google Cloud project name
  devices: [                          # Device(s) to run tests on
    {
      ios_model_id: 'iphonex',        # Device model ID, see gcloud command above
      ios_version_id: '11.2',         # iOS version ID, see gcloud command above
      locale: 'en_US',                # Optional: default to en_US if not set
      orientation: 'portrait'         # Optional: default to portrait if not set
    }
  ]
)

Available parameters:

  • app_path: You may provide a different path in the local filesystem (e.g: /path/to/app-bundle.zip) or on Google Cloud Storage (gs://your-bucket/path/to/app-bundle.zip) that points to an app bundle as specified here. If a Google Cloud Storage path is used, the service account must have read access to such file.
  • gcp_project: The Google Cloud project name for Firebase Test Lab to run on.
  • gcp_requests_timeout: The timeout, in seconds, to use for all requests to the Google Cloud platform (e.g. uploading your app bundle ZIP). If this parameter is omitted, Google Cloud SDK's default requests timeout value will be used. If you are finding that your ZIP uploads are timing out due to the ZIP file being large and not completing within the set timeout time, increase this value.
  • gcp_additional_client_info: Additional information to include in the client information of your test job. For example, if you'd like to include metadata to use in a Google cloud function in response to your test job's outcome. See https://firebase.google.com/docs/reference/functions/functions.testLab.ClientInfo for more information.
  • oauth_key_file_path: The path to the Google Cloud service account key. If not set, the Google application default credential will be used.
  • devices: An array of devices for your app to be tested on. Each device is represented as a ruby hash, with ios_model_id, ios_version_id, locale and orientation properties, the first two of which are required. If not set, it will default to iPhone X on iOS 11.2. This array cannot be empty.
  • async: If set to true, the action will not wait for the test results but exit immediately.
  • timeout_sec: After how long will the test be abandoned by Firebase Test Lab. Duration should be given in seconds.
  • result_storage: Designate which location on Google Cloud Storage to store the test results. This should be a directory (e.g: gs://your-bucket/tests/)

Issues and Feedback

If you have any other issues and feedback about this plugin, we appreciate if you could submit an issue to this repository.

You can also join the Firebase slack channel here.

Troubleshooting

For some more detailed help with plugins problems, check out the Plugins Troubleshooting doc in the main fastlane repo.

Using fastlane Plugins

For more information about how the fastlane plugin system works, check out the Plugins documentation in the main fastlane repo.

About fastlane

fastlane automates building, testing, and releasing your app for beta and app store distributions. To learn more about fastlane, check out fastlane.tools.

fastlane-plugin-firebase_test_lab's People

Contributors

bpollman avatar janpio avatar johnrogers avatar krausefx avatar martycatawiki avatar nancy-gomez avatar powerivq avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

fastlane-plugin-firebase_test_lab's Issues

Use double space after printing an emoji

image

Looking at, there is an extra space missing after each emoji. It's the way emojis are rendered on the terminal, by adding an extra space after each, it will look better for the user ๐Ÿ‘

Use of `UI` wrapper to print output

image

Looking at the screenshot above (posted on #20) it looks like we're sometimes printing content via puts instead of the fastlane provided methods (UI.message, UI.error, etc.).

By using the fastlane provided methods, you get lots of free stuff, like the user can customize their output, we get the timestamp prefix, colored output, etc

More information about those are available on https://github.com/fastlane/fastlane/tree/master/fastlane_core#interacting-with-the-user.

How to fix issue with google cloud storage conflict

Could you help me to fix problem with cloud storage conflict in your plugin
Error loading plugin 'fastlane-plugin-firebase_test_lab': Unable to activate fastlane-plugin-firebase_test_lab-1.0.7, because google-cloud-storage-1.26.1 conflicts with google-cloud-storage (~> 1.25.1)

forbidden Google::Cloud::PermissionDeniedError

forbidden: xxxx does not have storage.buckets.get access to the Google Cloud Storage bucket.
Please note:

  • I was able to upload manually files to the bucket.
  • format of my bucket "gs://[bucket]/tests"
  • .rvm/rubies/ruby-2.6.3/lib/ruby/gems/2.6.0/gems/google-cloud-storage-1.25.1

Please help here if you guys have any insights or fix - Thanks

Waiting for results for 3 hours

Hey all,

We're attempting to run nightly UI tests on CircleCI; I looked last night and the build runs, uploads and starts the tests successfully.

However, the plugin waits for results for 3 hours and then eventually posts the following error:

[09:34:14]: Network error when attempting to get test results, type: Faraday::ConnectionFailed, message: Net::OpenTimeout

I followed the link to the test results on Firebase Console and the tests complete successfully, it appears that the plugin is not being informed of the tests finishing properly.

Any ideas about what could be the cause of this?

Fastlane/Ruby Gem Dependency Issue: google-cloud-storage

Hey,

I am moving our testing over to Firebase and set up our Fastlane with with plugin the backend of last week, but looks like there is is dependency issues.

We are running fastlane_version "2.113.0"

The error when running the lane for Firebase is below:

Error loading plugin 'fastlane-plugin-firebase_test_lab': Unable to activate fastlane-plugin-firebase_test_lab-1.0.2, because google-cloud-storage-1.15.0 conflicts with google-cloud-storage (~> 1.13.0)

Myself and one of our devs have looked at many different ways to solve this, but to no avail.

Any chance this can be looked into?

Cheers,

Grae

Plugin not activated

There is a version conflict in the plugin that restrict plugin to activate.

Error loading plugin 'fastlane-plugin-firebase_test_lab': Unable to activate fastlane-plugin-firebase_test_lab-1.0.5, because google-cloud-storage-1.16.0 conflicts with google-cloud-storage (~> 1.15.0)
[06:54:43]: It seems like you wanted to load some plugins, however they couldn't be loaded

Scheme count error on Test Plans

Hello,

I'm running latest version of the plugin with Xcode 11.5 and now switched to Test Plans. The .xctestrun file now has:
CodeCoverageBuildableInfos
TestConfigurations
TestPlan
xctestrun_metadata

image
So, this is failing at
unless size == 1 UI.user_error!("The app bundle may contain only one scheme, #{size} found.") end
and I see the error

The app bundle may contain only one scheme, 3 found.

Could you please update the plugin to be able to use with TestPlans ?

Thanks

Fastlane >= 2.183.0 support

Would we be able to get a new version released containing eee528b ?

I'm struggling to install this plugin with fastlane 2.183.0

แ… bundle
Fetching gem metadata from https://rubygems.org/.........
Resolving dependencies...
Bundler could not find compatible versions for gem "google-cloud-storage":
  In Gemfile:
    fastlane (~> 2.183.0) was resolved to 2.183.2, which depends on
      google-cloud-storage (~> 1.31)

    fastlane-plugin-firebase_test_lab (~> 1.0.7) was resolved to 1.0.7, which depends on
      google-cloud-storage (~> 1.25.1)

Thank you!

Scheme Count Issue

The app bundle may contain only one scheme, 3 found.. Getting this error even after checking that zip file contains only 1 xctestrun and one debug iphoneos folder. Manually uploaded this to test lab and was able to run its successfully but through fastlane getting issue.

Scheme Count Issue

The app bundle may contain only one scheme, 3 found.. Getting this error even after checking that zip file contains only 1 xctestrun and one debug iphoneos folder. Manually upload this to test lab and was able to run its successfully but through fastlane getting issue.

Requests Timeout

Hey all.

My upload requests are timing out.

Please see my PR for this issue here: #62

1.0.1 Scheme count, app verification errors

Hi all,

After reading about the release of 1.0.1 in this thread, I thought the issue may be solved but it doesn't appear to fix it for me.

I am running Xcode 10.1, and version 1.0.1 of the plugin.

I get the error The app bundle may contain only one scheme, 2 found; when I remove the __xctestrun_metadata__ dict and associated keys from the build_products.zip file and re-package, I get the following error:

app verification failed: There must be only one .xctestrun files in the ZIP file.

There's definitely only one .xctestrun file in my ZIP - that and contents of the Debug-iphoneos folder are the only two items in there.

I tried uploading the ZIP manually via Firebase Console however it gives me the same error (there's a Stack Overflow issue here about it).

Any suggestions?

Can you please publish final version to resolve Conflict on Google Cloud Storage 1.43.0 ?

I am trying to run the plugin on my local and Bitrise. This is the error I am getting. It seems too many people experienced a similar issue and it is resolved but not published at all.

Error loading plugin 'fastlane-plugin-firebase_test_lab': Unable to activate fastlane-plugin-firebase_test_lab-1.0.7, because google-cloud-storage-1.43.0 conflicts with google-cloud-storage (~> 1.25.1)

Trying with forked repositories also does not affect at all , fastlane somehow tries to use the one from rubygems.org.

Can you please publish this version to rubygems.org

Need the output from the Firebase command (bucket url)

Your plugin does not output the actual link to where the test results are being stored in the google cloud bucket, this is vital for downloading results. Can you adjust the output so it still displays that instead of consuming the output?

Add Xcode version selection option (Xcode 10.2 now default on Firebase Test Lab)

Hello! So I am in the Firebase Community Slack group and I saw in the #ftl-announcements room there was an announcement:

"""
Hey everyone! For iOS, it's looking like we'll roll out Xcode 10.2 next week [Stated on Friday April 19th]. As a reminder, if you're still building with Xcode 10.1, please specify it as an argument to either gcloud or Flank using --xcode-version=10.1, as the new default will immediately be 10.2 (and it's looking like not all tests are forwards-compatible).

Xcode 10.2 is live!
Note that for those using the default Xcode version, you'll switch over automatically to Xcode 10.2.
"""

And was wondering if we can add an additional option flag for specifying which xcode version to use :)

Provide an error when the ios zip is invalid

There are many reasons why an ios zip assembly is valid, for example:

  1. There are multiple .xctestrun files in it;
  2. The xctestrun XML file does not include all required fields. (which can be fixed by flipping build flags in xcode).

It would be nice to provide a direct in-tool feedback when it detects one of the following circumstance, and let the user know exactly what needs to be done to rectify this.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.