ubisoft / ubisoft-laforge-animation-dataset Goto Github PK

When im trying to clone this repo, a github bandwidth limitation kicks in:

 git clone https://github.com/ubisoft/ubisoft-laforge-animation-dataset
Cloning into 'ubisoft-laforge-animation-dataset'...
remote: Enumerating objects: 97, done.
remote: Counting objects: 100% (44/44), done.
remote: Compressing objects: 100% (12/12), done.
remote: Total 97 (delta 38), reused 32 (delta 32), pack-reused 53
Receiving objects: 100% (97/97), 244.81 KiB | 6.80 MiB/s, done.
Resolving deltas: 100% (39/39), done.
Downloading c3d/lafan1_c3d.zip (357 MB)
Error downloading object: c3d/lafan1_c3d.zip (ecc3276): Smudge error: Error downloading c3d/lafan1_c3d.zip (ecc3276f8a1d3c6105e88e191fef7abc83a9bc2ff3909b73488e13237e786d47): batch response: This repository is over its data quota. Account responsible for LFS bandwidth should purchase more data packs to restore access.

Errors logged to 'C:\Users\ericd\Downloads\ubisoft-laforge-animation-dataset\.git\lfs\logs\20230122T124700.0833764.log'.
Use `git lfs logs last` to view the log.
error: external filter 'git-lfs filter-process' failed
fatal: c3d/lafan1_c3d.zip: smudge filter lfs failed
warning: Clone succeeded, but checkout failed.
You can inspect what was checked out with 'git status'
and retry with 'git restore --source=HEAD :/'

So is it me who has to pay for LSF or is it the owner of the repo?
Context from github:
https://docs.github.com/de/billing/managing-billing-for-git-large-file-storage/upgrading-git-large-file-storage

Hi there,
Thanks a lot for sharing this data! I'm trying to import the animations into Blender for visualization but I'm having issues with the orientations of the provided FBX rig and the BVH animation data. It seems the rest poses of the rigs are not aligned so the animation is garbled. Do you have a recommended workflow for rendering? I saw in issue #3 you mentioned using Motion Builder but I'm using macOS which is unfortunately not supported.

Thanks again!

Thank you for sharing these valuable data with the community.

We are thinking to compute SMPL body parameters by applying mosh++ and add them to AMASS which is large unified mocap dataset.

With this background I wanted to quickly ask whether the gender of the subjects is provided somewhere or if you could also expose them. Knowing the gender considerably improves the quality of the reconstructed surface.

Best
Nima

Great work!
I'd like to make some compare with your model and further study, so is there any way to get the model, training code or some demos of this paper?
Thanks a lot!

I am having an issue importing the animations into unreal, the mesh is twisted when importing the animation.
(Imported animation attached left, no animation on right)

I imagine there is some step that I could take to fix this. I would really appreciate some help.

My steps were :

Autodesk FBX converter for FBX model to FBX Binary. (This model imported into unreal with no problems)
Open the bvh animation files with blender.
Add a root node to the animation skeletons.
Export as a FBX
Import into Unreal attaching the FBX model skeleton as skeleton.

Hello @XefPatterson:
I am studying your brilliant work. Followed your paper, I write a pytorch-based code. However, it's hard for me to reproduce your results.
The situation I met
Without the discriminator, training after iterating 75,000 steps shows some reasonable results. (In my experiments, pytorch' RNN has the slower convergence rate than the tensorflow' RNN). But the feet sliding slightly on the ground. So I added the discriminator futher to alleviate this sliding problem. However, my training loss has suddenly increased after adding the discriminator. (At the same time, the discriminator's loss has decreased.)
In your paper, you scale all of your losses to be approximately equal on the LaFAN1 dataset for an untrained network. But I am not sure how to scale the loss of discriminator because the discriminators' losses are different every time(about from 1 to 50).
Discriminator structure
So I suspect that I have written discriminators wrong, which followed by:
Conv1d(input_channel=3+(num_joints-1)*6, output_channel=512, window=10 or 2)->Relu()->Conv1d(512, 256, 1)->Relu()->Conv1d(256, 1, 1),
where 3+(num_joints-1)*6 is the dimension of the vector concatenated by root velocity, all bones' offset related to the root bone and all bones' velocity, the same as your paper said. And the output layer is feed forward layer which means it has not activation layer.
Loss calculation process
The output has the shape of (batch_size, numbers of the sliding window), and the loss is calculated as followed:
loss=mean(square(subtract(output,label)))
I first calculate the loss of generated result and the label 1, and the optimizer step backward to update the generator's weight. And then I calculate the loss of discriminator reuslt, followed by:
loss(generated result,0)+loss(ground truth, 1)

Optimizer
I use the same optimizer and the same loss as the generator for the discriminator.

I stuck in reproducing the similar results as your work for few days. I am very very apperaciate it if you find any mistakes I made or point out any potential problems. Thank you very much!

Hi,

what is 1 unit in the LAFAN1 dataset in "real world" metrics? Is it 1cm / 1 inch / 1 meter or something completely different?
We need to know that in order to transform the scaling to the default scaling that Unity uses (1 unit = 1 meter)

Best,
Simon

Hi Felix and colleagues,

Thanks for sharing this incredibly useful dataset.

As far as I know, the BVH files include the number of joints and links corresponding to the pose sequences but do not actually include rigged mesh or rigid bodies corresponding to the character model. This is evident when I load the BVH files onto Blender and can only see armatures without any geometry. I envision that one needs to combine the BVH poses with an FBX character model with mesh + rigging.

So I guess we need to bring our own rigged character models in order to render the results. I wonder if you have an example character model that we can use (e.g. the mocap suit character visible in the teaser image).

If not, would you be willing to give us some hints about bringing our own characters e.g. Mixamo characters? Namely, what are the steps and tools needed to apply these poses to them?

Is there an tpose in the data file? So, I can retarget to other model.

Hi, I am trying to reimplement Learned Motion Matching. In Learned Motion Matching, the framerate is 60hz. However, this bvh dataset is under 30hz.

Coud you open source 60hz bvh files? Or is there any convenient method to extend 30hz bvh to 60hz bvh?

Thanks a lot!

Downloading c3d/lafan1_c3d.zip (357 MB)
Error downloading object: c3d/lafan1_c3d.zip (ecc3276): Smudge error: Error downloading c3d/lafan1_c3d.zip (ecc3276f8a1d3c6105e88e191fef7abc83a9bc2ff3909b73488e13237e786d47): batch response: This repository is over its data quota. Account responsible for LFS bandwidth should purchase more data packs to restore access.

Is there another way I can download these files?

Hi, I found an issue in benchmark.py at line 92 and 116. There is a np.concatenate operation that expands the dimension of (B, trans , J , 3) at axis 2. The original number of joints is 22. After that operation, the joints number is 23.

The current implementation of the BVH file reader interprets the Euler angles for rotation as extrinsic rotations. However, according to the BVH file format specification, the rotations should be interpreted as intrinsic rotations. This discrepancy may lead to incorrect animation data being processed.

# lafan1/extract.py
def read_bvh(filename, start=None, end=None, order=None):
    ...
    rotations = utils.euler_to_quat(np.radians(rotations), order=order)
    ...

# lafan1/utils.py
# this is an extrinsic rotation
def euler_to_quat(e, order='zyx'):
    """

    Converts from an euler representation to a quaternion representation

    :param e: euler tensor
    :param order: order of euler rotations
    :return: quaternion tensor
    """
    axis = {
        'x': np.asarray([1, 0, 0], dtype=np.float32),
        'y': np.asarray([0, 1, 0], dtype=np.float32),
        'z': np.asarray([0, 0, 1], dtype=np.float32)}

    q0 = angle_axis_to_quat(e[..., 0], axis[order[0]])
    q1 = angle_axis_to_quat(e[..., 1], axis[order[1]])
    q2 = angle_axis_to_quat(e[..., 2], axis[order[2]])

    return quat_mul(q0, quat_mul(q1, q2))

Proposed Solution:
Update the code to apply Euler angles as intrinsic rotations.

Hello, it is a nice work! I just wonder the order of the 22 joints and their corresponding names. Is the order the same to that of SMPL joints? Thank you in advance!