Inconsistence with the results between the tutorial in MONAI and the UNETR paper about research-contributions HOT 5 CLOSED

Hi,

I am very appreciate for your reply, and I have figured out this.

Thanks again for your great work and your kindful reply!!!

Merry Christmas in advance:)

from research-contributions.

Thanks so much for the reply. May I know where the additional 50 dataset you get to make the training to 80 data?

from research-contributions.

Thanks so much for the reply. May I know where the additional 50 dataset you get to make the training to 80 data?

@ahatamiz

from research-contributions.

Hi @Puzzled-Hui

Thanks for the comments and questions. There are some differences that cause the discrepancies in the numbers you mentioned. Firstly, our goal in the tutorial was to relax the memory constrains and making it more accessible for users. For this purpose, the UNETR model is trained on a different resolution which is 1.5* 1.5 * 2 as opposed to 1* 1* 1. Secondly, we only trained on 1 fold without any extensive data augmentation. For the paper, we utilized a five-fold cross validation scheme and used ensemble of 10 different models, from two different splits, for test server submission. Also noting that, the 0.79 performance comes from training on a single fold as outlined in the tutorial and research contribution repository ( link to json file)

In addition, as you are aware, BTCV server has 2 different submission tracks, namely standard and free competition. For the free competition, it is customary to use extra training data as done by previous works that are listed on the challenge leaderboard. Considering this, we also increased the dataset size to 80 to be comparable with these approaches and again utilized five fold cross validation for test server submission.

I hope these explanations are useful.

Thanks

from research-contributions.

Hi @Puzzled-Hui

Thanks for the comments and compliments.

Happy Holidays :)

from research-contributions.