Why the time-variable t need to be input into FourierFeatures class？ about fisor HOT 3 CLOSED

Actually, such timestep embedding method comes from Transformer (Attention is All You Need) and has been widely utilized in diffusion models and Transformer-like network structures. If the scaler timestep is used as input of the network, compared with other high-dimensional inputs, the final output will be difficult to heavily depend on it, which is not what we want. Therefore, encoding the scaler timestep to a vector that has a similar dimension as other inputs can solve this problem.

from fisor.

Thanks for your reply! I may check out later about the relation between fourier features and generalization, see if there are any literature furthur explain it. I would share my findings as soon as I finished the research.

Best regards!

from fisor.

Hi!
Thanks for your interests on our work. This is a good question and we did not investigate this in-depth. Actually, this is a common choice in many popular diffusion policies like IDQL, Decision Diffuser, et al and also many diffusion models for image generation. So, we directly followed this impementation.

From a quick search, I found the fourier feature can improve generalization ability for unseen times, and help the neural network to extract high-frequency features from the scalar time variable, and enjoys other good properties. We would also very happy to hear from your further findings on the advantages of fourier features.

from fisor.