Can you share the loss curve of the Swin-UNETR pre-training process about research-contributions HOT 4 OPEN

Hi @upupming

I see. We trained the model using 4 nodes with 8 GPUs (total 32). We noticed scaling up the pre-training to multi-node is important for convergence.

from research-contributions.

Hi @ahatamiz, i meet the same problem ,as i pre-training the model on a single GPU with batch size 2.My input and ground-turth are shown in the image, but the output does not seem to have any resemblance to the original image, do you mean to train on multiple GPUs to have good results

x1_aug

x1_gt

x1_recon

from research-contributions.

hI @GaoHuaZhang ,

Thanks, batch size should be a key point. We don't have the record for this training any more, but we have this
https://github.com/Project-MONAI/tutorials/tree/main/self_supervised_pretraining
which is a similar strategy pre-trianing, you could see loss curves with this tutorial.

Thanks

from research-contributions.

Hi @ahatamiz, i meet the same problem ,as i pre-training the model on a single GPU with batch size 2.My input and ground-turth are shown in the image, but the output does not seem to have any resemblance to the original image, do you mean to train on multiple GPUs to have good results x1_aug x1_gt x1_recon

Have you solved the problem? I have the same problem as you

from research-contributions.