fets-ai / models Goto Github PK

Hi,

I am trying to reproduce the results from this paper, by extracting (i.e. copy-pasting) the pytorch code as-is from the various repos in the FeTS-AI org, as needed. It is relevant to do this, instead of directly installing openfl, to ensure reproducibility across different frameworks (I plan to extend to other FL frameworks that support pytorch when this issue is solved).

The paper states that, using the 'Public Initial Model', an average DSC score of roughly 0.8 was achieved, corresponding to a DSC loss value of 0.2.
However, using the pretrained model provided in this repo, I get very high loss values (around 0.9), corresponding to a DSC score of 0.1!

You can find what I did here, and if you scroll to the bottom you'll see the bad loss values.

I understand that there may be some slight differences in performance, but not by such a large factor. Maybe the provided model is not the one referenced in the paper?

Thank you!
Francesco