question about the input data of locomotion about stylegestures HOT 2 CLOSED

Dear @nlpofwhat,

I have read your anther paper of Moglow (...) It seems to be a autogressive model totally with locomotion?

The StyleGestures paper builds on our earlier MoGlow preprint, which has been accepted for presentation at SIGGRAPH Asia this year. Since the architectures are relatively similar in both works, I would personally argue that both of them are examples of "MoGlow"("motion Glow") architectures, with the main difference being the application: The original MoGlow paper preprint demonstrates that the architecture works for different locomotion-generation tasks, while the StyleGestures follow-up work shows that the architecture can be adapted to gesture synthesis and also adds style control.

So, in summary: My view is that MoGlow is an architecture or a method; locomotion synthesis and speech-driven gesture generation are applications for which we have demonstrated that the method gives good results. MoGlow is thus not an "autogressive model totally with locomotion".

I try to reproduce the results but I can't figure out what the original input data is. For example, you generate gestures by audio then the main input data is audio.

The original MoGlow paper investigates path-based locomotion control from motion-capture data of humans and dogs. For each frame, the main input (the control signal) contains 3 numbers that specify:

1. The current forward velocity of the character in a body-centric coordinate system.
2. The current sideways velocity of the character in a body-centric coordinate system.
3. The current turning velocity of the character.

Together, these 3 numbers act to steer the moving character a bit like driving a car. Over time, the control signal defines a path through space that the root node will follow, along with the character's facing direction at different times. The task of the probabilistic model is then to generate a sequence of poses for the character that represent convincing locomotion along that path.

Like you suggest, this is a very different problem than generating gesture motion using speech acoustics as input features. We think it is a strength of the method that the same basic approach gives such compelling results on such disparate problems.

I can't figure out what the original input data is

If your question is about data availability and coding (e.g., which numbers represent what), the pre-processed locomotion data we used in the paper can be downloaded from: https://kth.box.com/s/quh3rwwl2hedwo32cdg1kq7pff04fjdf. (Edited in 2022: Our university is no longer using Box, invalidating the link originally provided here. Please see the repository readme for the latest files.) Code for training the MoGlow locomotion systems from the article is available in this repository, but may be split off to its own GitHub repo in the near future. Additional information regarding how to use the locomotion training data can be found in issue #1, for instance in this comment as well as this comment and its response. @simonalexanderson knows more about the details of the code and might be able to follow up with any information that I may have missed.

I hope this answers your questions.

from stylegestures.

I've got my answers. Thank you for answering my questions so patiently.

from stylegestures.