The filter-test-dev CI script doesn't seem to be working about 2024 HOT 7 CLOSED

thanks BTW for the work on fixing the actions!

from 2024.

I suspect that's because of some security restrictions of github actions. It might be necessary to put some of those tests in a different workflow_run file as I've done in deploy-for-review.yml. See for example https://stackoverflow.com/a/71683208

from 2024.

Alright so I did a LOT of new commits to fix the scripts for when an external PR is made.

I found some issues:

1. When an external person makes a PR, we have to manually "Approve & Run" the actions for it. This is doable manually but very annoying. -> TODO
2. When the filters fail, the comment wouldn't get written at all. This would also cause the S3 upload to bail prematurely. -> [FIXED in both cases]

from 2024.

The new file for writing comments on filter failures is here: https://github.com/iclr-blogposts/2024/blob/main/.github/workflows/comment-on-error.yaml

Here is an example PR from an external account that. You can tell by the automated comments on that PR that at first, I tested a success case, and then I swapped to a failure case. #52

from 2024.

Regarding your first point, I believe we have to manually approve it only if it's a new github account (I don't know how github determines exactly that an account is new)

from 2024.

I believe we have to manually approve it only if it's a new github account

ah ok I see! Maybe we can add a small message in the PR template to mention that if their run doesn't start automatically, they have to @ us to approve it

from 2024.

yes that seems like a good idea

from 2024.