why the inference results are not aligned with the validation results? about motiondirector HOT 3 OPEN

Hi. Typically, this is because the randomly sampled noises are different from the training stage.
Could you please provide more details? Like are you training MotionDirector on a single video or multiple videos?

from motiondirector.

Hi. Typically, this is because the randomly sampled noises are different from the training stage. Could you please provide more details? Like are you training MotionDirector on a single video or multiple videos?

Thanks for your reply, your answer helps, it is maybe different settings between training and inference. I have another question, how is the generalization ability of MotionDirector? For example, if i use a custom dreambooth weight, which is different from training. Can the temporal weights trained from MotionDirector work under this situation?

from motiondirector.

If you use DreamBooth to only finetune the spatial layers, I think it is OK, just like the results shown here. If the temporal layers are also changed, I'm not sure what will happen. You can try it out. Looking forward to your insights.

from motiondirector.