Loss stuck issue about dalle2-pytorch HOT 2 CLOSED

@goldiusleonard hey Goldius, the repository is still about 3-4 weeks away from being trainable by the general public. there's a number of engineering work to be done, considering it involves 3+ model (multiple unets in the cascade in the decoder)

however, if you want to continue tinkering for education, i highly recommend trying out the training wrapper i designed at https://github.com/lucidrains/dalle2-pytorch#training-wrapper-wip

from dalle2-pytorch.

@lucidrains I'm using the wrapper provided in train_diffusion_prior.py for training. However, I get stuck on scaler.scale(loss).backward(). I tried getting rid of the scaler, but it still doesn't proceed.
The output looks like this:

100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 1992.54it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<?, ?it/s]
  0%|                                                                                                                                                  | 0/42 [00:00<?, ?it/s]
  0%|                                                                                                                                                  | 0/42 [00:00<?, ?it/s]

The run doesn't return any errors, but gets stuck there.
Any idea why this could be happening?

from dalle2-pytorch.