[BUG?] It is not possible to use more than one GPU for training on Kaggle about rvc_cli HOT 7 CLOSED

I also have a similar issue but with 2x 8GB AMD Radeon 5500 XTs in a local Docker container.

from rvc_cli.

I tested it, and this problem extends to Applio, and doesn't happen on the RVC mainline. it seems to be a problem with containers

from rvc_cli.

@shirounanashi

I don't think it's a container issue. As you mentioned, I ran mainline RVC inside Docker, and multi-GPU training worked there. I believe the problem of parallel training not functioning has been around in Applio for a while. Not everyone has two or more GPUs for machine learning purposes. In fact, @blaisewf is already aware of this but doesn't have additional cards to test other than going to the cloud.

from rvc_cli.

I get it @TheTrustedComputer , I think I was wrong then, and Indeed, not everyone has multiple GPUs, I don't have it myself haha And I think it's impossible to discover the problem just through kaggle

from rvc_cli.

I don't think it's an applio bug, I tried to train with rvc-mainline with a very long dataset, which only allowed me to have a batch size of 7, when I wanted to test the two gpu's of the kaggle, then I got the error of insufficient vram.
-- Bad Traductor

from rvc_cli.

Using one GPU with 8 GB of VRAM, I can train with a batch size of 8 in mainline regardless of the dataset size. However, two GPUs require reducing it to 4, or I'll get out-of-memory errors. Whereas in Applio/RVC_CLI, training doesn't even start with 2+ GPUs.

from rvc_cli.

Fixed

from rvc_cli.