some questions about motion flow warp in code about videoinr-continuous-space-time-super-resolution HOT 4 CLOSED

Thanks for your interests! These different variables are generated by warping different maps (feature map, input image map, etc) based on the same motion flow field estimated by our network. They are concatenated together as the input of the decoding network for predicting the RGB value at the input coordinate (x, y, t).

from videoinr-continuous-space-time-super-resolution.

Thanks for your answer. And here are some other Qs, thank you very much!

1. why did you estimate two flow in the work?
2. where is the process of SpatialINR after warping in code, is SpatialINR “feat_imnet” in code? But I didnt find the usage after warp? So how did you do this process?

from videoinr-continuous-space-time-super-resolution.

1. At the beginning we want to apply a bi-directional architecture for the motion flow (flow t->0 and flow t->1), and we applied some explicit constraints on the network to learn such bi-directional flows. After extensive experiments we found that learning two flows without constraints yields better results, comparing with one single flow or two flows with bi-directional constraints.
2. feat_imnet is used to generate the continuous feature corresponding to the input 3D coordinate (x,y,t).

from videoinr-continuous-space-time-super-resolution.

Thanks for the good Research.
But I have similar problem understanding the function of TemporalINR and calculating motionflow.

1. How TemporalINR which is SIREN could get motion flow as output?
2. Also, in paper you mentioned that 'we generate two independent flows for the motion flow field, and concatenate corresponding warped features'. What is the two independent flow ? Is it flow from frame I_0 to frame I_1 frame flow and frame I_1 to frame I_0 frame?

from videoinr-continuous-space-time-super-resolution.

@zychen-ustc Hello! Also thank you for great research and code!
any comment on above question?
Is 'encode_imnet' is SpatialINR for warpped spatial coord?

from videoinr-continuous-space-time-super-resolution.