Multi GPU with transfer.py about sentiment-discovery HOT 1 OPEN

So the way DataParallel works is the .forward method takes cpu data (note that the tensors have to be on cpu) and broadcasts it to all the available GPUs where it passes the GPU data to the model's forward method.
What you tried didn't work because you only passed the data to the model's forward method not DataParallel's forward method. So you need to

1. Use the full dataparallel module for forward
2. Any time you need to access a model attribute, access it from the original modelpar.module module
3. Use cpu data type so that the model's forward can automatically send the data to the right gpu.

We had an implementation for this working in our original release of the codebase if you'd like a reference https://github.com/NVIDIA/sentiment-discovery/releases/tag/v0.1. It was too difficult to maintain while trying to add new features so we deprecated it.

from sentiment-discovery.