PyTorch-2.1.2-foss-2023a-CUDA-12.1.1.eb still fails with too many errors after #20156 about easybuild-easyconfigs HOT 5 OPEN

I would not say that 3 is too many, we allow for 50 failures in the easyconfig because we know that there are many test that are unreliable. Are those 3 failed test very important?

from easybuild-easyconfigs.

No clue, I'm more wondering why everybody elses test build passed with only 2 errors...

And we shouldn't have an EC in a release that fails to build. So we either have to increase allowed failures or fix some of the above.

from easybuild-easyconfigs.

I see, this is related to recent changes from #20156
It seems too aggressive to have lowered the number of failed test down to 2.

from easybuild-easyconfigs.

Also from the confcall:

• test_proxy_tensor likely requires rebuild Z3 as the non-suffixed version now includes the Python bindings.
• test_nn is known to fail randomly
• test_jit might be an issue with the memory (leak) detector on some systems
• the other 2 don't happen for me, no idea here without more details

2 might indeed be to low and was mainly intended to shake out issues in that PR where it seemingly worked well enough.
The idea was to have this as low as possible to work for most such that when more fails a conscious decision can be made to use --ignore-test-failures or --skip-test-step

from easybuild-easyconfigs.

The extra V100/A40 problems was due to not having rebuilt one of the dependencies.

from easybuild-easyconfigs.