Input and output mean scales updated during training (X-UMX) about asteroid HOT 5 CLOSED

Oh, okay. That's a bit weird to me, but if it works and the original UMX also did it that way I guess it's better to keep it like that.

Thanks,
David

from asteroid.

Hello !

I don't master this architecture, people from Sony do : @r-sawata WDYT ?

from asteroid.

Sorry to have kept you waiting. I was busy due to CVPR deadline, and it has finally gone yesterday. I'll check this in a few days.

from asteroid.

The easiest solution would be using requires_grad=False as done with the STFT window, but this can generate problems if people try to do transfer learning or fine-tuning and start freezing/unfreezing parts of the model without being aware of this (I've recently gone through this problem and was the reason I found this bug).

I would suggest registering both the scalers and the STFT window as buffers instead of as parameters since that's the general recommendation in Pytorch for model's tensors that are not being optimized. Doing this with the STFT window is quite straightforward (I can do a PR with a fix I've been testing that works well and is backward compatible with pre-trained models) but I'm unsure about how to do this with the scales since Pytorch doesn't have a nn.BufferDict function to replace nn.ParameterDict.

from asteroid.

Sorry for being so late. I confirmed it carefully, and found that mean and scale with requires_grad=True is fine. Namely, optimizing them during training is our intention.

As you may know, this X-UMX is the extended version of the original one, Open-Unmix (UMX). As shown in the initialization of their implementation, input_mean and input_scale should be learned during training. Please see here:
https://github.com/sigsep/open-unmix-pytorch/blob/4318fb278e1863f4cf8556b513987faf14a15832/openunmix/model.py#L84-L95

from asteroid.