achimr / mteval Goto Github PK

Root cause:
The method to authenticate Github Actions code to Google Cloud is described here (this is code provided by Google). After a checkout of the repository, the authentication methods create temporary JSON files in the repository folders. However, nbdev 2.0 CI code checks out the repository again using the Github actions/checkout@v3 with the default clean=true option, overwriting the Google Cloud JSON authentication files. Subsequent calls to Google Cloud therefore cannot be authenticated anymore.

The proper way to fix this would be:

1. Fork fastai/workflows
2. Add an option clean_repo to fastai/workflows/nbdev-ci/actions.yml to allow setting the clean option for actions/checkout@v3 (default: true)
3. Add the code to add Google Cloud authentication to this repository in .github/workflows/test.yaml with the option clean_repo=false set
4. Test this fix with the forked fastai workflows
5. Submit PR to fastai
6. After PR is accepted change .github/workflows/test.yaml to use fastai/workflows again instead of the fork

COMET unit testing has to be run with GPU. Github Actions doesn't provide GPU instances for testing. It is possible to run actions on self-hosted runners, but setting this up is very involved.

The easier workaround is to run unit tests with nbdev_test on a local Jupyter notebook instance (with GPU support) or on Google Colab (with GPU runtime).