how to use this model for image generation? about vit-pytorch HOT 3 OPEN

@basamelatex no one has shown that this can work with a straight encoder yet afaik, but people have discretized the pixel space and then used a decoder to generate the image as with iGPT and Image Transformer

from vit-pytorch.

Thanks a lot for your answer, I checked out the papers you mentioned above. I noticed that they were able to generate only quite small images such as 64x64 and used relatively small datasets like CIFAR10. On the other hand, in the Vit paper they were suggesting that the model doesn't work well on small datasets. Do you think this would be the case in image generation as well? Do we really need a huge dataset for Vit to work on image generation? I would like to give it a try, but I feel a bit skeptical after seeing 300M dataset they use..

from vit-pytorch.

I would like to know how to use this model for spatial-temporal state forecast, such as nowcasting using radar echo, like ConvLSTM.

from vit-pytorch.