[Feature] Precompiled r0vm binary with metal or cuda support about risc0 HOT 5 OPEN

Will need to test if I can embed the Cuda into the Python module without side effects.

^ I would circle back to this in case it really becomes a problem.

from risc0.

This is dependent on making the CUDA libraries late-binding at runtime rather than binding at compile time. It's been on my TODO list for awhile.

from risc0.

just to ask a clarifying question. Do you mean that, for Linux, if the code is compiled with CUDA but later runs on a platform that does not have the cuda library, the executable will fail because: by default, Rust told Linux dynamic linker to load all the dynamic libraries when the program starts (related to rust-lang/rust#89751), so it would try to load lib-cuda even if the code is never going to actually use it.

There are two solutions:

• see if one can fix the issue by playing with the linker argument to ask Rust to do it differently. The issue here rust-lang/rust#89751 does not seem to be fully resolved with extra linker arg (rust-lang/cargo#7984 a confirmation. a full-fledged linker script is too hacky.

• otherwise, manually implement lazy loading (i.e., late binding) here. When the CUDA core is on and is going to call some CUDA functions, it finds the CUDA library and tries to load it (or more specifically, load necessary functions). The problem is that this may require certain changes. Currently, RISC0 is usingcust. Obviously, cust does not do lazy loading, but asks Rust to tell LLVM to indicate dynamic linkage. One could change cust-raw so that all calls to the CUDA library would be redirected to an intermediary that does the function translation, but the side effect is that this is not an official cust feature. Note that one does not need to feel guilty to fork cust---the last release is two years ago.

Is that correct? And if so, what is your plan of attack?

Edited

from risc0.

This is on my TODO and we'll get around to it eventually. Is this really a blocker for you now?

from risc0.

Depends on #1025

from risc0.