ShiftNet loss about highres-net HOT 1 OPEN

Hello @veseln - thanks for raising this issue. I can confirm that this is indeed a bug.

You are entirely correct on the discrepancy between equation 5

and its implementation
https://github.com/ElementAI/HighRes-net/blob/aa022561c55ee3875b7d2e7837d37e620bf19315/src/train.py#L187

It seems to be a typo, and more than likely this is how the results were obtained.

A few notes:

1. I can confirm that the intention of Line 187 was to regularize the norm of the shifts (output by ShiftNet). As such, Line 187 is not strictly speaking the ShiftNet loss (or anything's loss for that matter). The ShiftNet parameters are learned by virtue of the quality of the reconstruction (the cPSNR-based loss). The regularizer was meant to keep the predicted shifts from deviating too much.

2. The bad news is that, as you rightly suggested, the mean of all shifts will tend to zero, and therefore its square will converge quadratically faster to zero. All this does is regularize the mean magnitude, so it's fair to say that Line 187 is totally moot.

3. The good bit of news is that, in retrospect, although we have not been regularizing the ShiftNet output all along, the end-to-end pipeline can work without this regulatization. The question remains however, on how the regularization affects overall performance. I would be grateful if you performed an ablation with the corrected line, and reported your results here.

In summary
ShiftNet is still learned to accommodate for the shifts between the SR and HR ground-truth. But the predicted shifts are effectively not regularized at all.

Hope this helps.

from highres-net.