Moving model between GPU and CPU about openrlhf HOT 3 CLOSED

This is a bit complicated, I'm not sure if DeepSpeed supports this, but I recommend using deepspeed's offload feature

from openrlhf.

Thanks. Deepspeed's offload feature is only offloading the optimiser on the CPU if I'm not mistaken. What I'm trying to achieve is to move both the model (with the parameters) and the optimiser from the GPU to the CPU and release the previously allocated GPU memory. Is there perhaps a proposed way to approach this? Thanks

from openrlhf.

Thanks. Deepspeed's offload feature is only offloading the optimiser on the CPU if I'm not mistaken. What I'm trying to achieve is to move both the model (with the parameters) and the optimiser from the GPU to the CPU and release the previously allocated GPU memory. Is there perhaps a proposed way to approach this? Thanks

deepspeed supports model weights offloading

from openrlhf.