jeff-sjtu / niki Goto Github PK
View Code? Open in Web Editor NEWCode of "NIKI: Neural Inverse Kinematics with Invertible Neural Networks for 3D Human Pose and Shape Estimation", CVPR 2023
Code of "NIKI: Neural Inverse Kinematics with Invertible Neural Networks for 3D Human Pose and Shape Estimation", CVPR 2023
if not vis, what is the fps with GPU ?
When would you plan to release the code?
Env: Windows 10, anaconda 24.1 , python 3.8.19
sends the error:
Loading model from exp/checkpoint_49_cocoeft.pth...
Loding LGD model from exp/niki_model_28.pth
Try to use other video encoders...
Traceback (most recent call last):
File "scripts/demo.py", line 208, in
assert write_stream.isOpened(), 'Cannot open video for writing'
AssertionError: Cannot open video for writing
Hi,can you provide training data?
Hello~ 请问有没有类似 HybrIK 的 Blender 插件代码能渲染生成的 pt 文件
Could you please also provide the pre-trained model that was used with AGORA? The current pre-trained model performs significantly worse than reported in the paper. These are the results I'm getting right now:
INFO:root:{'precision': 0.89, 'recall': 0.52, 'f1': 0.65, 'kid_precision': 'nan', 'kid_recall': 'nan', 'kid_f1': 'nan', 'body-MPJPE': 115.0, 'kid-body-MPJPE': nan, 'body-NMJE': 176.9, 'kid-body-NMJE': 'nan', 'body-MVE': 263.3, 'kid-body-MVE': nan, 'body-NMVE': 405.1, 'kid-body-NMVE': 'nan'}
Hello, very impressive work and I am wondering if you are going to release the demo code for external videos. Thanks!
Hello!
Is it possible to apply SMPL-X with this model?
Thanks for the help
Tested on some images from Yoga-82 dataset with the default demo.py script. The model seems too constrained by IK on this dataset, and sometimes it incorrectly detects a person.
What is the easiest way to improve results for this kind of pictures? Does it require fine-tuning the IK network, or can the IK constraints be adjusted in some config?
Bad results:
Ok results:
Hey,
Thanks for this amazing project, I was wondering if there is any way to add texture or mesh_cloth on the 3d avatar, for example the texture file in figure#1 and the mesh generated should be with this texture applied on it as in figure#2
----------------------------Figure #1 : Texture To Apply----------------------------------
----------------------------Figure #2 : Final Mesh----------------------------------
When I run the code, it says I am missing this file. How do I download it please?
Hi, Jeff,
I'm a little confused how you normalize joints. It's not consistent in the codes. Some place use joints[0], some use mean joints[1,2].
In the following line of demo.py
, the 3d joints are normalized against to center of joints[1,2].
Line 348 in 69f94e7
But, in the forward pass of FlowRegressor
the joints is renormalized with respect to joints[0] which makes the previous normalization unnecessary.
NIKI/niki/models/NIKI_1stage.py
Line 85 in 69f94e7
It seems most of the time you are using joints[0] for normalization. But sometimes use joints[1,2]'s center, e.g. in FlowRegressor.projection2uv()
:
NIKI/niki/models/NIKI_1stage.py
Line 247 in 69f94e7
When should the joints being normalized against to joints[0] vs joints[1,2]'s center?
When will the paper and code be released?
FileNotFoundError: [Errno 2] No such file or directory: './model_files/basicModel_neutral_lbs_10_207_0_v1.0.0.pkl'
Can you share the script you used to generate AGORA .pkl files so I can reproduce the results for the AGORA dataset? Thank you.
Hi, I have a question about the 3DPW-OCC test.
The results that you've reported in the table and the results from PARE's paper are different each other.
For example, in 3DPW-OCC set,
Methods | PA-MPJPE | MPJPE | PVE |
---|---|---|---|
SPIN | 60.8 | 95.6 | 121.6 |
HMR-EFT | 60.9 | 94.4 | 111.3 |
PARE (Res-50) | 56.6 | 90.5 | 107.9 |
Methods | PA-MPJPE | MPJPE | PVE |
---|---|---|---|
SPIN | 62.5 | 98.4 | 135.1 |
HMR-EFT | 62.0 | 95.8 | 120.5 |
PARE | 57.4 | 91.4 | 115.3 |
I think there are some differences in the evaluation code (e.g., setting bounding boxes), could you share how you evaluated each model? Thank you so much for your help in advance!
When I tried to run the demo.py, this error occured. The input model is from "checkpoint_49_cocoeft.pth".
Any method to solve this?
'''
RuntimeError: Error(s) in loading state_dict for HRNetSMPLCam:
size mismatch for smpl.shapedirs: copying a param with shape torch.Size([6890, 3, 10]) from checkpoint, the shape in current model is torch.Size([6890, 3, 300]).
'''
Thanks for sharing such fantastic works.
When I was trying to run the train/validate/demo according to README.md, I got the following error:
Traceback (most recent call last):
File "scripts/train.py", line 8, in <module>
from niki.datasets.naive_dataset_temporal import (
File "/home/ubuntu/NIKI/niki/datasets/naive_dataset_temporal.py", line 22, in <module>
from .hp3d_dataset_temporal import hp3d_dataset_temporal
File "/home/ubuntu/NIKI/niki/datasets/hp3d_dataset_temporal.py", line 16, in <module>
from niki.utils.pose_utils import (calc_cam_scale_trans_const_scale,
ImportError: cannot import name 'calc_cam_scale_trans_const_scale' from 'niki.utils.pose_utils' (/home/ubuntu/NIKI/niki/utils/pose_utils.py)
After digging into niki/utils/pose_utils
, I was unable to find the function calc_cam_scale_trans_const_scale
.
Is this a bug?
Thanks for your great work! Could you provide the trainning data so we can reproduce the results?
- './data/annot/h36m_train_25fps_occ_smpl_db_xyz17new_add_cam_bbox_hrnetw48_pred_amb_sigma.pt'
- './data/annot/3dpw_train_occ_db_xyz17_uv24new_addcam_bbox_hrnetw48_pred_amb_sigma_adduv.pt'
- './data/annot/mpii3d_train_scale12_occ_db_add_cam_bbox_hrnetw48_pred_amb_sigma_adduv.pt'
Can you make a google collab Notebook of NIKI that would be very helpful !
Hi, how is J_regressor_h36m.npy made?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.