axis in SMPL result about romp HOT 18 OPEN

Sorry for the delayed reply! At this moment, I have something urgent at hand. But I will look into the SPIN joints you mentioned later. I think it wouldn't need any training.

from romp.

I have submitted a commit to support the extra 24 joints in SPIN (compared with 25 openpose joints we already support).
With this commit, the result files will contain a key named "j3d_spin24", which is what you are asking for.

Let me know if there are any problems.
Besides, have you converted the output file into bvh? Would you want to share the detailed process with other developers? It seems important for the others. While I am not familiar with these parts.
Best.

from romp.

Thanks for your detailed bug report. I have tested the code at a different codebase. There were something wrong while I pasted the changes to this repository.

At my service. Aha~

from romp.

Good question. The j3d_smpl24 and kp3d_smpl24_results are both in shape B x 24 x 3. The last dimension is in order of (x,y,z). While the results[img_path][subject_idx]['j3d_smpl24'] is in shape 24 x 3.
If you want to swap axis (Y and Z), simply add a line below the line you quoted.
results[img_path][subject_idx]['j3d_smpl24'] = results[img_path][subject_idx]['j3d_smpl24'][:,[0,2,1]]

from romp.

Thanks for you reply.
I have another question.
As I mentioned before that I'm trying to animate Avatar in Unity using joints position from CenterHMR. From CenterHMR we can get SMPL 24 joints or OpenPose 25 joints, that are already discussed in #21, but in both cases there is no top_head joint which is causing problem to head movement. Check the attached screenshot that I tested with CenterHMR (no top_head) and with other model (with top_head joint from SPIN).

I read about SPIN joints from which we can get Head joint (from h36m) and top_head joint (LSP). Many models are using SPIN regressor and joints for better motion but from CenterHMR I'm unable to get better motion in case of head even CenterHMR can output head joint in case of 24 SMPL joints and nose joints in case of OpenPose joints but still it can't solve head movement as compared to SPIN joints.

How we can get SPIN joints in CenterHMR by using SPIN regressor? It will need training with CenterHMR?

I think this functionality should be added in CenterHMR as it already can output OpenPose keypoints.

from romp.

It would be appreciated

from romp.

I am also waiting for this next commit.

from romp.

Thanks for this great commit.
I am also looking for BVH step. @ahsan3803 would you like to update about it?

from romp.

@Arthur151 Thanks for updating the commit for SPIN joints.

Sorry for late reply.
I didn't complete the BVH task, still I'm working on it. I will update ASAP whenever I will complete it.

Regarding SPIN joints, I tested them and seems that joints are wrong. SPIN joints order as you mentioned here in constants file is right if we ignore OpenPose joints. I'm getting the wrong joints, to verify it, I draw the skeleton for one frame after getting SPIN joints from here.

Below is the skeletal result for SPIN joints from CenterHMR.

From above attached result, we can see that some joints Neck_LSP[12], Head_top[13], Jaw_H36M[17], Head_H36M[18] are missing and some joints Thorax_MPII[15], Spine_H36M[16] are not at right place.
Furthermore the other problem I noticed that, right_knee, left_knee, right_ankle and left_ankle are at two positions.
right_knee is at [1] and [15].
left_knee is at [4] and [14].
right_ankle is at [0] and [18].
left_ankle is at [5] and [17].

Below is the complete order of currently joints for SPIN in CenterHMR and it seems that this order is not right according to SPIN 24 joints
[0] right_ankle
[1] right_knee
[2] pelvis
[3] left_hip
[4] left_knee
[5] left_ankle
[6] right_wrist
[7] right_elbow
[8] right_shoulder
[9] left_shoulder
[10] left_elbow
[11] left_wrist
[12] right_hip
[13] spine
[14] left_knee
[15] right_knee
[16] thorax
[17] left_ankle
[18] right_ankle
[19] nose
[20] left_eye
[21] right_eye
[22] left_ear
[23] right_ear

Following attached screenshot is the skeleton limbs and joints connection for above attached skeletal result.

I have tried my best to explain all the problems. Let me know if anything is creating confusion, I will explain in more detail with visual results.

from romp.

Sorry for the bug.
I have uploaded a new commit to fix it.

Now the joint order is just as the same as what I define in constrants.py.

from romp.

Thanks, it's working now. It was minor bug.

from romp.

It happens to a good developer 😜

from romp.

Thanks for your detailed bug report. I have tested the code at a different codebase. There were something wrong while I pasted the changes to this repository.

At my service. Aha~

thx for ur job，my smpl model is just a mesh(obj file, only mesh and texture without skeleton and weights), how I can get a rigged smpl model(fbx) from my own smpl model(obj file) to replace basicModel_m_lbs_10_207_0_v1.0.2.fbx?

from romp.

�SMPL is a statistical linear skinned model. Every smpl mesh is just a state of the model in a specific parameters.
Please refer to https://smpl.is.tue.mpg.de/ for the details.
I am afraid you may have to regress the pose and shape parameters from your smpl mesh via iterative fitting the model. Then you can drive the fbx model to the state of your mesh.

from romp.

Good question. The j3d_smpl24 and kp3d_smpl24_results are both in shape B x 24 x 3. The last dimension is in order of (x,y,z). While the results[img_path][subject_idx]['j3d_smpl24'] is in shape 24 x 3. If you want to swap axis (Y and Z), simply add a line below the line you quoted. results[img_path][subject_idx]['j3d_smpl24'] = results[img_path][subject_idx]['j3d_smpl24'][:,[0,2,1]]

this line and this file are not exist in new version. what is the new place?

from romp.

Hi, there.
If you want to switch the dimention of estimated 3D keypoints.
You can add a line before this line:

outputs['joints'] = outputs['joints'][:, [0,2,1]]

The tracking_ids of all subjects in the image are outputs['track_ids']. You can use it to get the motion sequence of each subjects.

from romp.

@Arthur151
When you try to execute the following command, the function you sent will not be executed

python3 -u -m romp.predict.image --configs_yml='configs/image.yml'

from romp.

@Arthur151 Thanks for updating the commit for SPIN joints.

Sorry for late reply. I didn't complete the BVH task, still I'm working on it. I will update ASAP whenever I will complete it.

Regarding SPIN joints, I tested them and seems that joints are wrong. SPIN joints order as you mentioned here in constants file is right if we ignore OpenPose joints. I'm getting the wrong joints, to verify it, I draw the skeleton for one frame after getting SPIN joints from here.

Below is the skeletal result for SPIN joints from CenterHMR.

From above attached result, we can see that some joints Neck_LSP[12], Head_top[13], Jaw_H36M[17], Head_H36M[18] are missing and some joints Thorax_MPII[15], Spine_H36M[16] are not at right place. Furthermore the other problem I noticed that, right_knee, left_knee, right_ankle and left_ankle are at two positions. right_knee is at [1] and [15]. left_knee is at [4] and [14]. right_ankle is at [0] and [18]. left_ankle is at [5] and [17].

Below is the complete order of currently joints for SPIN in CenterHMR and it seems that this order is not right according to SPIN 24 joints [0] right_ankle [1] right_knee [2] pelvis [3] left_hip [4] left_knee [5] left_ankle [6] right_wrist [7] right_elbow [8] right_shoulder [9] left_shoulder [10] left_elbow [11] left_wrist [12] right_hip [13] spine [14] left_knee [15] right_knee [16] thorax [17] left_ankle [18] right_ankle [19] nose [20] left_eye [21] right_eye [22] left_ear [23] right_ear

Following attached screenshot is the skeleton limbs and joints connection for above attached skeletal result.

I have tried my best to explain all the problems. Let me know if anything is creating confusion, I will explain in more detail with visual results.

@ahsan3803 @Arthur151
What a great job! I want to plot the 3D skeleton of my video, can I get the bvh file of the object?
Could you tell me how can I get the 3d points file to plot the skeleton of my video?

I'm a bit ashamed to ask this but I'm so glad if you teach me something!

from romp.