[BUG] Issues with `codecov` on `cudf` CI about cudf HOT 3 OPEN

...
We need to check on the GitHub application configuration and make sure it's set up correctly. We should also update to the newer Codecov CLI to upload, and use an organization token to authenticate our uploads. Currently we are using the older Codecov uploader (see the install step and shared-workflows call).

PRs that address this:

• rapidsai/ci-imgs#141
• rapidsai/shared-workflows#212

from cudf.

I had a call with a couple folks from Sentry (who create Codecov), Sabiha and Vlad. We talked through a few topics related to coverage in RAPIDS.

• Codecov's carryforward option may help us with the problems we face with only running CI on a nightly basis. This may resolve (3) above. We may want to use -F <component> (like -F cudf) so that we can more easily determine coverage for each part of a repository.

• I requested a feature in Codecov. The UI for Codecov should have an option to stay in sync with the GitHub default branch (currently https://app.codecov.io/gh/rapidsai/cudf/ uses branch-23.06, which is a year old). Currently the branch must be manually adjusted, and we don't want to add work to the release process. Vlad will see what can be done on their side (a UI option to stay in sync with GitHub would be ideal).

• I reported a bug in Codecov: the "latest commit" reported for branch-24.06 on Codecov is 2 weeks old. However, there are newer coverage reports on the commit list for branch-24.06. Vlad said this seems to be a bug and they'll take a look.

• On the RAPIDS side, we're getting rate limited during Codecov uploads, which prevents us from getting full coverage reports on every nightly test run. Copying from that log below for posterity:

[2024-05-17T07:24:36.632Z] ['info'] Pinging Codecov: https://codecov.io/upload/v4?package=uploader-0.7.2&token=*******&branch=branch-24.06&build=9124031981&build_url=https%3A%2F%2Fgithub.com%2Frapidsai%2Fcudf%2Factions%2Fruns%2F9124031981&commit=6d5f9653debe57c7eb52f42fb980d38451a9a460&job=test&pr=&service=github-actions&slug=rapidsai%2Fcudf&name=&tag=&flags=&parent=
[2024-05-17T07:24:36.632Z] ['verbose'] Passed token was 0 characters long
[2024-05-17T07:24:36.632Z] ['verbose'] https://codecov.io/upload/v4?package=uploader-0.7.2&branch=branch-24.06&build=9124031981&build_url=https%3A%2F%2Fgithub.com%2Frapidsai%2Fcudf%2Factions%2Fruns%2F9124031981&commit=6d5f9653debe57c7eb52f42fb980d38451a9a460&job=test&pr=&service=github-actions&slug=rapidsai%2Fcudf&name=&tag=&flags=&parent=
        Content-Type: 'text/plain'
        Content-Encoding: 'gzip'
        X-Reduced-Redundancy: 'false'
[2024-05-17T07:24:36.844Z] ['error'] There was an error running the uploader: Error uploading to [https://codecov.io:](https://codecov.io/) Error: There was an error fetching the storage URL during POST: 429 - {'detail': ErrorDetail(string='Rate limit reached. Please upload with the Codecov repository upload token to resolve issue. Expected time to availability: 681s.', code='throttled')}
[2024-05-17T07:24:36.846Z] ['verbose'] The error stack is: Error: Error uploading to [https://codecov.io:](https://codecov.io/) Error: There was an error fetching the storage URL during POST: 429 - {'detail': ErrorDetail(string='Rate limit reached. Please upload with the Codecov repository upload token to resolve issue. Expected time to availability: 681s.', code='throttled')}
    at main (/snapshot/repo/dist/src/index.js)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
[2024-05-17T07:24:36.846Z] ['verbose'] End of uploader: 532 milliseconds
[2024-05-17T07:24:36.846Z] ['info'] Codecov will exit with status code 0. If you are expecting a non-zero exit code, please pass in the `-Z` flag

We need to check on the GitHub application configuration and make sure it's set up correctly. We should also update to the newer Codecov CLI to upload, and use an organization token to authenticate our uploads. Currently we are using the older Codecov uploader (see the install step and shared-workflows call).

from cudf.

Amazing! Thanks @AyodeAwe. I will circle back and try this again soon. I think we may get nightly coverage reports for branch-24.06 since that's currently active in CI, but most new PRs are targeting branch-24.08 so we may be out-of-sync for a while until the 24.06 release is complete and we move our nightly CI runs to branch-24.08.

from cudf.