Are conv2d_transpose layers shift variant, too? about antialiased-cnns HOT 8 CLOSED

Thanks for the question!

Yes, upsample with ConvTranspose2d with groups=in_channels, and then use [1, 1] or [1, 2, 1] as your weights.

For TensorFlow, perhaps you can refer to the StyleGan2 code/paper, which does smoothed upsampling as well.

from antialiased-cnns.

Thanks for the question. Upsampling convolution is shift-invariant, but if you do it naively, it is aliased and leads to stippling patterns.

If you do bilinear upsampling (convolve with a [1 2 1] filter after doing the naive upsampling), then you are better antialiased. The StyleGAN paper does this, for example.

from antialiased-cnns.

Yes, it makes a difference. The second (BlurPool-->Conv-->ReLU) works worse, because it destroys information as the first step.

from antialiased-cnns.

@richzhang
Thanks for the answer (and also the reference to StyleGAN example)!

Can I ask one more question?

In case of the strided convolutions, does the performance vary depending on whether the BlurPool is put before or after the convolution?
(i.e., Conv (stride 1) -> ReLU -> BlurPool (stride 2) vs. BlurPool (stride 2) -> Conv (stride 1) -> ReLU)
It seems that putting BlurPool before Conv could save computation, but I wonder how the performance would be.

Could you share your thoughts or experiences on this?

Thank you!

from antialiased-cnns.

but subpixel upscale layer is not antialiased?

from antialiased-cnns.

In paper there is not much information on antialiased upsampling(or I just don't get it):

We mirror the downsampling by applying the same filter after upsampling. Note that applying the Rect-2 and Tri-3 filters while upsampling correspond to “nearest” and “bilinear” upsampling, respectively

Is it should be done via torch.nn.ConvTranspose2d with groups= in_channels and with same filter as in downsampling code?

Also seems there is not groups parameter in TensorFlow for conv_transpose, is there any workaround? i.e. use some layer that can do x2 upscale(bilinear resize, conv_traspose, pixel_shuffle, etc) and then add 'blur' layer(avr pool with stride 1 and kernel size 2x2) on top?

from antialiased-cnns.

Looking at StyleGan2 code, seems they are doing it like:

blur(upsample(x)):
https://github.com/NVlabs/stylegan2/blob/7d3145d23013607b987db30736f89fb1d3e10fad/training/networks_stylegan.py#L519

downsample(blur(x)):
https://github.com/NVlabs/stylegan2/blob/7d3145d23013607b987db30736f89fb1d3e10fad/training/networks_stylegan.py#L607

I wonder is this approach(i.e. via 2 steps) is different from resampling-by-convolution(i.e. via 1 step) ? https://clouard.users.greyc.fr/Pantheon/experiments/rescaling/index-en.html

from antialiased-cnns.

It's the same. The two steps can be folded into one. We don't actually blur the whole map at every spatial location and then subsample by factor of 2, since many of the calculations are thrown away. We simply evaluate the blur at every other location.

from antialiased-cnns.