bharat-b7 / ipnet Goto Github PK

View Code? Open in Web Editor NEW

219.0 12.0 27.0 2.7 MB

Repo for "Combining Implicit Function Learning and Parametric Models for 3D Human Reconstruction, ECCV'20(Oral)""

Python 98.46% Cython 1.54%

ipnet's Introduction

Implicit Part Network (IP-Net)

Repo for "Combining Implicit Function Learning and Parametric Models for 3D Human Reconstruction, ECCV'20 (Oral)"

Link to paper: http://arxiv.org/abs/2007.11432

Prerequisites

Cuda 10.0
Cudnn 7.6.5
Kaolin (https://github.com/NVIDIAGameWorks/kaolin) - for SMPL registration
MPI mesh library (https://github.com/MPI-IS/mesh)
Trimesh
Python 3.7.6
Tensorboard 1.15
Pytorch 1.4
SMPL pytorch from https://github.com/gulvarol/smplpytorch. I have included these files in this repo.

Download pre-trained models

Download IPNet weights: https://datasets.d2.mpi-inf.mpg.de/IPNet2020/IPNet_p5000_01_exp_id01.zip
IPNet single surface: https://nextcloud.mpi-klsb.mpg.de/index.php/s/4nomcDH8EGwbzNi
mkdir <IPNet directory>/experiments
Put the downloaded weights in <IPNet directory>/experiments/

Preprocess data for training

Normalize scans: python utils/preprocess scan.py <scan.obj> <body_shape.obj> <save_name> <save_path>
Register SMPL+D to the scan: smpl_registration/fit_SMPLD.py <scan_path.obj> <save_path>
Note that SMPL fitting is much more stable with correct gender.
Generate query points: python boundary_sampling_double.py <scan_scaled.obj> <body_shape_scaled.py> <smpld_registration.obj> <save_path> --sigma <sigma> --sample_num 100000 --ext_in 01 --ext_out 01
We used sigma=0.15 and 0.015, ext_in and ext_out are just suffix for naming files.
Generate voxelized input : python voxelized_pointcloud_sampling.py <scan_scaled.obj> <save_path> --ext 01 --res 128 --num_points 5000

Run demo IP-Net

Test on single scan/PC: python test_IPNet.py assets/scan.obj experiments/IPNet_p5000_01_exp_id01/checkpoints/checkpoint_epoch_249.tar out_dir -m IPNet
(It is better to use dataloader for testing on a dataset: python generate.py -dist 0.5 0.5 -std_dev 0.15 0.015 -res 128 -m IPNet -ext 01 -suffix 01 -pc_samples 5000 -exp_id 01)
Fit SMPLD to IPNet predictions: python smpl_registration/fit_SMPL_IPNet.py out_dir/body.ply out_dir/full.ply out_dir/parts.npy out_dir/cent.npy out_dir/

For training/ testing on dataset, you'd need the following directory structure if you'd like to use our dataloaders:

[DATASETS]
-[dataset]
--[subject_01]
---[scan.obj]
---[smpld_registration.obj]
---[boundary_sampling]
---- <query points for implicit function, see boundary_samplin_double.py, we use sigma=[0.15, 0.015]>
---[voxels]
---- <voxelized scan, see voxelized_pointcloud_sampling.py>
--[subject_02]

Train IP-Net

python train.py -dist 0.5 0.5 -std_dev 0.15 0.015 -batch_size 4 -res 128 -m IPNet -ext 01 -suffix 01 -pc_samples 5000 -num_sample_points 20000 -exp_id 01

Fit SMPL to IP-net predictions

python smpl_registration/fit_SMPL_IPNet.py <scan_path.obj> <scan_labels.npy> <scale_file.npy> <save_path>

Cite us:

If you use this code please cite:

@inproceedings{bhatnagar2020ipnet,
    title = {Combining Implicit Function Learning and Parametric Models for 3D Human Reconstruction},
    author = {Bhatnagar, Bharat Lal and Sminchisescu, Cristian and Theobalt, Christian and Pons-Moll, Gerard},
    booktitle = {European Conference on Computer Vision ({ECCV})},
    month = {aug},
    organization = {{Springer}},
    year = {2020},
}

License

Please read carefully the following terms and conditions and any accompanying documentation before you download and/or use this software and associated documentation files (the "Software").

The authors hereby grant you a non-exclusive, non-transferable, free of charge right to copy, modify, merge, publish, distribute, and sublicense the Software for the sole purpose of performing non-commercial scientific research, non-commercial education, or non-commercial artistic projects.

Any other use, in particular any use for commercial purposes, is prohibited. This includes, without limitation, incorporation in a commercial product, use in a commercial service, or production of other artefacts for commercial purposes.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

You understand and agree that the authors are under no obligation to provide either maintenance services, update services, notices of latent defects, or corrections of defects with regard to the Software. The authors nevertheless reserve the right to update, modify, or discontinue the Software at any time.

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. You agree to cite the Combining Implicit Function Learning and Parametric Models for 3D Human Reconstruction paper in documents and papers that report on research using this Software.

ipnet's People

Contributors

Stargazers

Watchers

ipnet's Issues

Could I get skeleton and skinning weights after alignments?

Hi,

First, thanks for this great work! I read your paper and knew that you could get skeleton and skinning weights after alignments. However, the results I got from your demo do not contain this information. How could I save skeleton and skinning weights accompany with *.obj results?

Thanks for your time and look forward to your reply!

The difference between `args.SV` and `args.model == 'IPNetSingleSurface'`

Hi, I want to train the model on my own dataset using partial pointclouds, but from line 70 to line 111 in the file train.py I find two arguemnets args.SV and args.model == 'IPNetSingleSurface'. What is the difference of them? And which one should I use for partial pointcloud?
Thanks a lot!

Results of "Run demo IP-Net"

Thank you for providing the wonderful paper and code.

I followed your ReadMe and ran "Run demo IP-Net".
As a result, I got a mesh that looks like the mesh on the right in the image below.

The head mesh is strange, is this the expected and correct output result? In Fig. 6 in the paper, even the hairstyle seems to be well reproduced.
This happened with other subjects as well.

Was it wrong that I followed issue 11?

Boundary points of single view data?

As I understood, the input "points" was sampled from the boundary of the mesh, which is reconstructed from complete scan of a real world object; my question is how to get the sampled points (for input) when we only have the points cloud from a single view (e.g. one depth camera)? as we don't have the complete mesh reconstructed, we can't sample the area which have no depth point clouds right?

Thank you for your time.

Regarding Fine-tuning Model on Personalized Dataset

Dear Authors,

Thank you for your amazing work! I have been following your work on IPNET with great interest and admiration. Your approach has shown remarkable results in your paper.

I am reaching out to inquire about the feasibility and steps involved in fine-tuning your model on a personalized dataset that my team and I have curated. Specifically, we have gathered a dataset consisting of paired 3D scans and SMPL representations for individuals in a lying-down posture. Our objective is to leverage your methodology to fine-tune the model on our dataset and further enhance its performance for our specific application.

Given your expertise and experience in this domain, I would greatly appreciate any suggestions or guidance you could offer on the steps involved in this process. Additionally, if there are any specific considerations or best practices that we should be aware of when fine-tuning the model on a personalized dataset, your insights would be invaluable to us.

Thank you very much for considering my inquiry.

Warm regards,
David

CUDA out of memory

Hello @bharat-b7,
When I run the function test_IPNet.py, I meet the error as follows:

RuntimeError: CUDA out of memory. Tried to allocate 4.01 GiB (GPU 0; 10.91 GiB total capacity; 7.29 GiB already allocated; 2.68 GiB free; 7.33 GiB reserved in total by PyTorch)

While I look my GPU:

Thu Nov  5 17:38:12 2020       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 450.80.02    Driver Version: 450.80.02    CUDA Version: 11.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  GeForce GTX 108...  Off  | 00000000:65:00.0  On |                  N/A |
| 27%   42C    P8    18W / 250W |    392MiB / 11170MiB |      3%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+

It seems that I have enough GPU memory to run the code, but why is there a error like this?
Thanks a lot

data preparation question.

Hi, Thank you very much for sharing your work.
I am trying to prepare the training data. However, I come across two questions. (1) In step 1. we are supposed to get scaled_scan.obj and scaled_body_shape.obj . I have input scan.obj, but what is the body_shape.obj, it is obtained by SMPL registered to scan.obj ? how could I get it ?
(2) in step 2, This step is to get smpld_registration.obj right ?

Scaled_scan.obj and scaled_body_shape.obj will be used in boundary_sampling_double.py/boundary_sampling function to get boundary query points. and smpld_registration.obj will be used in boundary_sampling_double.py/part_labelling for get body parts labels.
Thanks a million for your time.

the pre-trained model is damage

I download the model by https://datasets.d2.mpi-inf.mpg.de/IPNet2020/IPNet_p5000_01_exp_id01.zip, but it damaged

Suggestions to improve the execution time

Hi,

First, thank for share you great job.
I'm running IPNet with my local computer, and the execution takes ~15 minutes on the inner-outer surface inference and ~7 minutes on SMPL optimization.
Is there a way to improve the execution time of the method, that you recommend?
Maybe changing some parameter o doing some modification in code?

Any suggestion will be helpful for me.

Thanks in advance!

kaolin install error

error:
/usr/local/cuda/include/cuda_fp16.hpp(1680): error: more than one instance of overloaded function "isinf" matches the argument list:
function "isinf(float)"
function "std::isinf(float)"
argument types are: (float)

/usr/local/cuda/include/cuda_fp16.hpp(1691): error: more than one instance of overloaded function "isinf" matches the argument list:
function "isinf(float)"
function "std::isinf(float)"
argument types are: (float)

/home/notebook/code/personal/80299257/conda/envs/ipnet/lib/python3.7/site-packages/torch/include/c10/core/TensorTypeSet.h(44): warning: integer conversion resulted in a change of sign

2 errors detected in the compilation of "/tmp/tmpxft_00005287_00000000-6_mesh_intersection_cuda_kernel.cpp1.ii".
error: command '/usr/local/cuda/bin/nvcc' failed with exit status 1

I was unable to install kaolin correctly, could anyone give me some advice,thank you !

configuration:
linux server
pytorch=1.4
cuda=10.0.130
Python=3.7.6

Point to Surface Distance

Question about the part probabilities

@bharat-b7
Hi, Bharat~
I really interesting about your paper and code, but I have a question about the part probabilities:
For each query point p, you calculate a probability parts_softmax (shape: B×14×N) to judge which part of the SMPL the query point p belongs to. (The related code is in the line 124: https://github.com/bharat-b7/IPNet/blob/master/models/local_model_body_full.py)
You divide the SMPL body into 14 parts. Why is the shape of probability parts_softmax not B×(14+2)×N? What if the query point p does not belong to any part of the SMPL body (e.g. point p is inside or outside the surface)?

I cannot figured it out, and I really hope you can answer me. Thanks in advance~

fit_SMPLD results are bad

I use fit_SMPLD.py to fit scan.obj, but the result is not good. Could you please point out where I was wrong?

displacments in IPNetMano()

May I ask what's this for, thank you.

Question about Single Surface mode

Thank you for the excellent work. Dose the pretrained model of Single Surface mean that the input is from a single view and pointcloud is partial? if not, would you provide the model for partial pointcloud as input?
Thanks a lot!

Can I input point clouds without faces

Hi @bharat-b7 ,
Thanks a lot for the release of code, and very interesting work!
In the code fit_SMPL.py, the input scan.obj should contain faces. Could it possible that I only have point clouds (x,y,z) WITHOUT faces?

BEST

Using Kinect point cloud as input

I gave the non parametric part of the IPNet a point cloud captured by Azure kinect. (running test_IPNet.py ) But I feel like the results are not very promising. The results are for the body file. I wanted to check with you to see if I'm doing something wrong. (I know I have to fit the SMPL model to the non-parametric reconstruction to get the final result, But I feel like if the SMPL fitting code used these results as input it wouldn't work very well. ) In all the following I use -batch_points 100000 since otherwise I would get a GPU out of memory error.

This is the point cloud file I have used as a csv file (just three columns of x,y,z first row is column names.)
frame_20.zip
I added these lines in the test_IPNet.py under the main function to read the point cloud:

def main(args):
# Load kinect point cloud from csv file
    kinect_pc = np.genfromtxt('kiya_data/frame_20.csv', delimiter=',')
    kinect_pc = kinect_pc[1:]

so instead of sending pc.vertices to pc2vox I sent kinect_pc which is a numpy array.

I have tried this point cloud as input : (it is around 36k points)

my first attempt: (I reversed the direction of the Y axis since I thought that might help the network since your sample was also like that)
python kiya_test_IPNet.py assets/scan.obj experiments/IPNet_p5000_01_exp_id01/checkpoints/checkpoint_epoch_249.tar kiya_out_dir -m IPNet -batch_points 100000

Then I tried reversing the direct of the Z axis as well and I got :

Finally I also tried increasing the resolution of the input voxelized figure to the network but it actually made the result worse:
python kiya_test_IPNet.py assets/scan.obj experiments/IPNet_p5000_01_exp_id01/checkpoints/checkpoint_epoch_249.tar kiya_out_dir -m IPNet -batch_points 100000 -res 200

IPNetMano IPNetSingleSurface pre-trained model and examples

Thank you for the excellent work. By the way, would you also share your pre-trained models for the IPNetMano (Parts, especially hands modeling) and IPNetSingleSurface (for the case, where there is only one single depth frame as input), and also the corresponding test examples? Thank you.

Preprocess data for training

Hi,
I'm using smpl_registration/fit_SMPLD.py by testing with a scan file from Renderpeople, but I met a error AttributeError: 'add' object has no attribute 'pose_subjects'. How can I get this .pkl (such as lbs_tj10smooth6_0fixed_normalized/male/model.pkl) or a pre-computed prior (such as assets/pose_prior.pkl)
Any idea on how to fix it would be helpful.

Can not download IPNet weight？

a great work！
but i can not download the IP-net model from this link https://datasets.d2.mpi-inf.mpg.de/IPNet2020/IPNet_p5000_01_exp_id01.zip

import errors with libmesh on windows

This error continues to exist on Windows, I run the command - python test_IPNet.py assets/scan.obj experiments/IPNet_p5000_01_exp_id01/checkpoints/checkpoint_epoch_249.tar out_dir -m IPNet , I get the error -

(IP-Net) D:\CG_Source\NeRFs\3D_Avatar_Pipeline\IPNet>python test_IPNet.py assets/scan.obj experiments/IPNet_p5000_01_exp_id01/checkpoints/checkpoint_epoch_249.tar out_dir -m IPNet
Traceback (most recent call last):
  File "test_IPNet.py", line 12, in <module>
    from utils.voxelized_pointcloud_sampling import voxelize
  File "D:\CG_Source\NeRFs\3D_Avatar_Pipeline\IPNet\utils\voxelized_pointcloud_sampling.py", line 7, in <module>
    import utils.implicit_waterproofing as iw
  File "D:\CG_Source\NeRFs\3D_Avatar_Pipeline\IPNet\utils\implicit_waterproofing.py", line 9, in <module>
    from .libs.libmesh import check_mesh_contains
  File "D:\CG_Source\NeRFs\3D_Avatar_Pipeline\IPNet\utils\libs\libmesh\__init__.py", line 1, in <module>
    from .inside_mesh import check_mesh_contains, MeshIntersector, TriangleIntersector2d
  File "D:\CG_Source\NeRFs\3D_Avatar_Pipeline\IPNet\utils\libs\libmesh\inside_mesh.py", line 3, in <module>
    from .triangle_hash import TriangleHash as _TriangleHash
ModuleNotFoundError: No module named 'utils.libs.libmesh.triangle_hash'

I compiled and installed libmesh from utils/libs/libmesh successfully -

(IP-Net) D:\CG_Source\NeRFs\3D_Avatar_Pipeline\IPNet\utils\libs\libmesh>python setup.py install
running install
running bdist_egg
running egg_info
writing Hello_world_app.egg-info\PKG-INFO
writing dependency_links to Hello_world_app.egg-info\dependency_links.txt
writing top-level names to Hello_world_app.egg-info\top_level.txt
reading manifest file 'Hello_world_app.egg-info\SOURCES.txt'
writing manifest file 'Hello_world_app.egg-info\SOURCES.txt'
installing library code to build\bdist.win-amd64\egg
running install_lib
running build_ext
creating build\bdist.win-amd64\egg
creating build\bdist.win-amd64\egg\utils
creating build\bdist.win-amd64\egg\utils\libs
creating build\bdist.win-amd64\egg\utils\libs\libmesh
copying build\lib.win-amd64-3.7\utils\libs\libmesh\triangle_hash.cp37-win_amd64.pyd -> build\bdist.win-amd64\egg\utils\libs\libmesh
creating stub loader for utils\libs\libmesh\triangle_hash.cp37-win_amd64.pyd
byte-compiling build\bdist.win-amd64\egg\utils\libs\libmesh\triangle_hash.py to triangle_hash.cpython-37.pyc
creating build\bdist.win-amd64\egg\EGG-INFO
copying Hello_world_app.egg-info\PKG-INFO -> build\bdist.win-amd64\egg\EGG-INFO
copying Hello_world_app.egg-info\SOURCES.txt -> build\bdist.win-amd64\egg\EGG-INFO
copying Hello_world_app.egg-info\dependency_links.txt -> build\bdist.win-amd64\egg\EGG-INFO
copying Hello_world_app.egg-info\top_level.txt -> build\bdist.win-amd64\egg\EGG-INFO
writing build\bdist.win-amd64\egg\EGG-INFO\native_libs.txt
zip_safe flag not set; analyzing archive contents...
utils.libs.libmesh.__pycache__.triangle_hash.cpython-37: module references __file__
creating 'dist\Hello_world_app-0.0.0-py3.7-win-amd64.egg' and adding 'build\bdist.win-amd64\egg' to it
removing 'build\bdist.win-amd64\egg' (and everything under it)
Processing Hello_world_app-0.0.0-py3.7-win-amd64.egg
removing 'c:\miniconda3\envs\ip-net\lib\site-packages\Hello_world_app-0.0.0-py3.7-win-amd64.egg' (and everything under it)
creating c:\miniconda3\envs\ip-net\lib\site-packages\Hello_world_app-0.0.0-py3.7-win-amd64.egg
Extracting Hello_world_app-0.0.0-py3.7-win-amd64.egg to c:\miniconda3\envs\ip-net\lib\site-packages
Hello-world-app 0.0.0 is already the active version in easy-install.pth

Installed c:\miniconda3\envs\ip-net\lib\site-packages\hello_world_app-0.0.0-py3.7-win-amd64.egg
Processing dependencies for Hello-world-app==0.0.0
Finished processing dependencies for Hello-world-app==0.0.0

Originally posted by @codesavory in #10 (comment)

Install Kaolin failed

hi, I tried to install Kaolin, but it failed.
System:
OS: Ubuntu 18.04
Python version: 3.7.6
PyTorch version: 1.7.1
Torchvision version: 0.8.2
CUDA/cuDNN version: 11.1/8.0.4
GPU: RTX3080
Do you have any suggestions?
Can you provide Kaolin for me?

Fitting SMPL pose failed

Hi,
I'm using IPNet code by testing with this scan file:
scan_man.zip
but code fails in pose optimization step. I used Ubuntu 18.04, and installed dependencies as readme file says.
There are some screenshots of the results.
the scan that i'm using:

and the inner surface(runinng 'test_IPNet.py'):

but for the SMPL optimization step(running 'fit_SMPL_IPNet.py'), pose optimization isn't working. I have shown a few iterations and the pose changes in the wrong direction with each iteration.
Iter 1:

Iter 5:

Iter 15:

Iter 30:

I tried to figure out whats the problem here, changing learning rate of Adam optimizer, but the result are the same.
I think the problem is in pose initialization, and the pose is taken from 'pose_prior.pkl' obtaining mean smpl pose to initialize, but i don't understand how to change initial pose.
Other way is use a different optimizer than ADAM, any suggestions?

Any idea on how to fix it would be helpful.

Texture registration SMPL?

If I have a texture island map and want it to register to the SMPL model along with my scan obj file for the same. How can we do that?

Fitting SMPL

I have tried running the :

python smpl_registration/fit_SMPL_IPNet.py <scan_path.obj> <scan_labels.npy> <scale_file.npy> <save_path>

but I get the following error:

Traceback (most recent call last):
  File "smpl_registration/fit_SMPL_IPNet.py", line 381, in <module>
    _, _, _ = fit_SMPL([args.inner_path], scan_labels=[args.inner_labels], display=args.display, save_path=args.save_path,
  File "smpl_registration/fit_SMPL_IPNet.py", line 212, in fit_SMPL
    smpl_faces = sp.get_faces()
  File "/home/logi/Desktop/ipnet/IPNet/lib/smpl_paths.py", line 111, in get_faces
    return np.load(fname)
  File "/home/logi/anaconda3/envs/ipnet/lib/python3.8/site-packages/numpy/lib/npyio.py", line 416, in load
    fid = stack.enter_context(open(os_fspath(file), "rb"))
FileNotFoundError: [Errno 2] No such file or directory: '/BS/RVH/work/data/template/faces.npy'

So my question is:

Do I need to download the smpl models from the paper's website ? if yes where should I put them ?

Also what should the faces.npy file look like ? and how can I get it ?

Test on Single-View Point Cloud

@bharat-b7
Hi, Bharat~

I really appreciate your great work!
I run the demo on my own single-view pointcloud, but met some problems, the IP-Net predictions and SMPL fitting outputs are very strange.
Here are my:
input pc,
body.ply,
full.ply,
body_smpl.ply,
full_smpld.ply.

I used the pre-trained model you offered in README(IPNet weights: https://datasets.d2.mpi-inf.mpg.de/IPNet2020/IPNet_p5000_01_exp_id01.zip).
I wonder if the pre-trained model is only used for complete scan input?
Can it work on single-view point cloud input?
Would you release the pre-trained model for single-view point cloud input?
I would be very grateful if you could provide it!

Best wishes~

No such file or directory: body25_regressor.pkl face_regressor.pkl hand_regressor.pkl

There seems to be some files missing (import errors)

I was trying your code but it seems there are functions/ files missing for the imports.

this is the error message I get when I try to run the demo:

(kaolin2) PS C:\Users\User\Desktop\Kiarash\IPNet> python test_IPNet.py assets/scan.obj experiments/IPNet_p5000_01_exp_id01/checkpoints/checkpoint_epoch_249.tar out_dir -m IPNet
Traceback (most recent call last):
  File "C:\Users\User\Desktop\Kiarash\IPNet\utils\implicit_waterproofing.py", line 9, in <module>
    from .libs.libmesh import check_mesh_contains
  File "C:\Users\User\Desktop\Kiarash\IPNet\utils\libs\libmesh\__init__.py", line 1, in <module>
    from .inside_mesh import check_mesh_contains, MeshIntersector, TriangleIntersector2d
  File "C:\Users\User\Desktop\Kiarash\IPNet\utils\libs\libmesh\inside_mesh.py", line 3, in <module>
    from .triangle_hash import TriangleHash as _TriangleHash
ModuleNotFoundError: No module named 'utils.libs.libmesh.triangle_hash'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "test_IPNet.py", line 12, in <module>
    from utils.voxelized_pointcloud_sampling import voxelize
  File "C:\Users\User\Desktop\Kiarash\IPNet\utils\voxelized_pointcloud_sampling.py", line 7, in <module>
    import utils.implicit_waterproofing as iw
  File "C:\Users\User\Desktop\Kiarash\IPNet\utils\implicit_waterproofing.py", line 12, in <module>
    from .libs_py27.libmesh import check_mesh_contains
ModuleNotFoundError: No module named 'utils.libs_py27'

I am using python 3.6 on windows 10.

I suspect I should install cython for the first error (although it was not in the requirements), But for the second error such a file doesn't even exist.

your implicit_waterproofing.py code (in the utils folder) has:
from .libs.libmesh import check_mesh_contains
but there is no module check_mesh_contains !?

thank you

How to get a female SMPL fitting result?

I tried to use female data as input and changed the gender in fit SMPL part as 'female'. However，I still got the male-like SMPL result (SMPL-D was good). Could I get a female SMPL fitting result via fitting the SMPL part？

How the pkl file to view

I wanna to know how the pkl, npy file can be view. I will be appreciate if anyone can answer my question.

ModuleNotFoundError: No module named 'kaolin.metrics.mesh'

Hi guys, does anyone encounter this problem?

python smpl_registration/fit_SMPL_IPNet.py out_dir/body.ply out_dir/full.ply out_dir/parts.npy out_dir/cent.npy out_dir/

Traceback (most recent call last):
  File "smpl_registration/fit_SMPL_IPNet.py", line 16, in <module>
    from kaolin.metrics.mesh import point_to_surface, laplacian_loss  # , chamfer_distance
ModuleNotFoundError: No module named 'kaolin.metrics.mesh'

I used kaolin==0.13, I really don't know how to solve it.~

How to obtain the “/BS/RVH/work/data/template/faces.npy”

Hi,

Thanks for your great work!

when I run the demo code, an error is caused by missing the "/BS/RVH/work/data/template/faces.npy" file.

Although you have pointed to the "smpl-pytorch" repo in one of the issues, however, I still confused about it.

Could you tell me how to obtain this file more specifically? Thanks a lot in advance!

Inner surface error with frontal point cloud

Hi,
I'm trying to test IPNet using a frontal view obtained from Kinect V2 capture.
The input file: depth_segmented.zip
and looks like this:

But inner surface is very incompleted:

and optimized SMPL looks bad:

Is possible change some parameter to use IPNet with incompleted frontal input?
If the answer is not, Can you provide IPNet pretrained model for single view reconstruction?

Thanks for you great work!

CUDA out of memory

Hello Bharat! Thank you so much for sharing this work!

I have just managed to install all the required environments. While I try the demo with provided command

python test_IPNet.py assets/scan.obj experiments/IPNet_p5000_01_exp_id01/checkpoints/checkpoint_epoch_249.tar out_dir -m IPNet

The scripts is taking a lot of time "Loading weights from, experiments/IPNet_p5000_01_exp_id01/checkpoints/checkpoint_epoch_249.tar". And after a while, the system gives me the following error:

RuntimeError: CUDA out of memory. Tried to allocate 4.01 GiB (GPU 15.78 GiB total capacity; 11.29 GiB already allocated; 235.19 MiBe; 11.33 GiB reserved in total by PyTorch)

I was running this demo on a docker environment, using an empty "tesla v100-pcie-16gb" GPU. I was wondering whether this is enough space for a demo? If my problem is abnormal, do you have any idea why this is happening? I am very much appreciated for your help!

Best

Can you provide a test example？the code does not work for my own data

The testing code does not work on my own data， hope you to provide one example.

SMPL object has no property pose_subjects

Hey guys I'm trying to use your smpl_registration library to register SMPLD model to my scans,

I am using google colab with all the dependencies specified in your repo setup except for my python version being 3.6

when I run the following : python smpl_registration/fit_SMPLD.py <scan_obj> <result_dir>

this is the error I'm getting

Traceback (most recent call last):
File "smpl_registration/fit_SMPLD.py", line 174, in
gender=args.gender)
File "smpl_registration/fit_SMPLD.py", line 96, in fit_SMPLD
pose, betas, trans = fit_SMPL(scans, None, gender, save_path, display)
File "/content/drive/My Drive/IPNet/smpl_registration/fit_SMPL.py", line 247, in fit_SMPL
prior = get_prior(gender=gender)
File "/content/drive/My Drive/IPNet/lib/th_smpl_prior.py", line 25, in get_prior
prior = Prior(dp_prior.get_smpl())
File "/content/drive/My Drive/IPNet/lib/th_smpl_prior.py", line 52, in init
self.pose_subjects = sm.pose_subjects
AttributeError: 'add' object has no attribute 'pose_subjects'

I also checked what were the members of the sm(smpl object) and it was the following
['J', 'J_regressor', 'J_regressor_prior', 'J_transformed', 'T',

I also did not see any vertex or face member included (I've not worked with smpl-pytorch yet so I'm unsure what are the expected member variables)

Do you guys have any idea what might be causing the issue?

I faced some issues with missing files(face.npy and model.pkl), which I resolved by creating face.npy by converted the faces from (https://smpl.is.tue.mpg.de/) directly to numpy and for model.pkl I just used the male smpl model from (https://smpl.is.tue.mpg.de/)

The std of boundary sampling

As in the training cmd, the project use 0.15 and 0.015 to multiply on a normal distribution for points shift augmentation; my question is that what size is your voxel (in meter) since those shift will directly add onto points sampled from mesh, and what parameter & voxel size will you use while training IPNetMANO? I assume that uniformly sample from voxel grid lead to a worse results and you guys tried some set of stds to get the best results.

How do I get the SMPL model 'lbs_tj10smooth6_0fixed_normalized' ?

Hello, thank you for the great work.

I would like to test your code on the RenderPeople(free-model) mesh model,
but I got some problems.

The first is the the hard-coded file path.
lib/smpl_paths.py code
ROOT = '/BS/RVH/work/data/'

And,
where could I find the SMPL model 'lbs_tj10smooth6_0fixed_normalized'
lib/spml_paths.py code
smpl_file = join(ROOT, 'smpl_models', 'lrotmin', 'lbs_tj10smooth6_0fixed_normalized', self.gender, 'model.pkl')

Thank you.

Python 3.6 or 3.7

While trying to run demo IP Net
python test_IPNet.py assets/scan.obj experiments/IPNet_p5000_01_exp_id01/checkpoints/checkpoint_epoch_249.tar out_dir -m IPNet

This error was returned:
RuntimeWarning: compiletime version 3.6 of module 'utils.libs.libmesh.triangle_hash' does not match runtime version 3.7

The documentation states python 3.7 as a pre-requisite but the above error suggests we use 3.6. Can this be confirmed? Thanks.

RuntimeError: CUDA error: device-side assert triggered

Hi, thank you for your great work.

When I try to run the IP-Net demo and fit SMPLD to IPNet predictions, I got RuntimeError: CUDA error: device-side assert triggered

Optimizing SMPL global orientation: 0%| | 0/30 [00:00<?, ?it/s]/pytorch/aten/src/THC/THCTensorIndex.cu:361: void indexSelectLargeIndex(TensorInfo<T, IndexType>, TensorInfo<T, IndexType>, TensorInfo<long, IndexType>, int, int, IndexType, IndexType, long) [with T = float, IndexType = unsigned int, DstDim = -1, SrcDim = -1, IdxDim = -1, IndexIsMajor = true]: block: [72,0,0], thread: [32,0,0] Assertion srcIndex < srcSelectDimSize failed.
/pytorch/aten/src/THC/THCTensorIndex.cu:361: void indexSelectLargeIndex(TensorInfo<T, IndexType>, TensorInfo<T, IndexType>, TensorInfo<long, IndexType>, int, int, IndexType, IndexType, long) [with T = float, IndexType = unsigned int, DstDim = -1, SrcDim = -1, IdxDim = -1, IndexIsMajor = true]: block: [72,0,0], thread: [33,0,0] Assertion srcIndex < srcSelectDimSize failed.
/pytorch/aten/src/THC/THCTensorIndex.cu:361: void indexSelectLargeIndex(TensorInfo<T, IndexType>, TensorInfo<T, IndexType>, TensorInfo<long, IndexType>, int, int, IndexType, IndexType, long) [with T = float, IndexType = unsigned int, DstDim = -1, SrcDim = -1, IdxDim = -1, IndexIsMajor = true]: block: [72,0,0], thread: [34,0,0] Assertion srcIndex < srcSelectDimSize failed.
/pytorch/aten/src/THC/THCTensorIndex.cu:361: void indexSelectLargeIndex(TensorInfo<T, IndexType>, TensorInfo<T, IndexType>, TensorInfo<long, IndexType>, int, int, IndexType, IndexType, long) [with T = float, IndexType = unsigned int, DstDim = -1, SrcDim = -1, IdxDim = -1, IndexIsMajor = true]: block: [72,0,0], thread: [35,0,0] Assertion srcIndex < srcSelectDimSize failed.
/pytorch/aten/src/THC/THCTensorIndex.cu:361: void indexSelectLargeIndex(TensorInfo<T, IndexType>, TensorInfo<T, IndexType>, TensorInfo<long, IndexType>, int, int, IndexType, IndexType, long) [with T = float, IndexType = unsigned int, DstDim = -1, SrcDim = -1, IdxDim = -1, IndexIsMajor = true]: block: [72,0,0], thread: [36,0,0] Assertion srcIndex < srcSelectDimSize failed.
/pytorch/aten/src/THC/THCTensorIndex.cu:361: void indexSelectLargeIndex(TensorInfo<T, IndexType>, TensorInfo<T, IndexType>, TensorInfo<long, IndexType>, int, int, IndexType, IndexType, long) [with T = float, IndexType = unsigned int, DstDim = -1, SrcDim = -1, IdxDim = -1, IndexIsMajor = true]: block: [72,0,0], thread: [37,0,0] Assertion srcIndex < srcSelectDimSize failed.
/pytorch/aten/src/THC/THCTensorIndex.cu:361: void indexSelectLargeIndex(TensorInfo<T, IndexType>, TensorInfo<T, IndexType>, TensorInfo<long, IndexType>, int, int, IndexType, IndexType, long) [with T = float, IndexType = unsigned int, DstDim = -1, SrcDim = -1, IdxDim = -1, IndexIsMajor = true]: block: [72,0,0], thread: [38,0,0] Assertion srcIndex < srcSelectDimSize failed.
/pytorch/aten/src/THC/THCTensorIndex.cu:361: void indexSelectLargeIndex(TensorInfo<T, IndexType>, TensorInfo<T, IndexType>, TensorInfo<long, IndexType>, int, int, IndexType, IndexType, long) [with T = float, IndexType = unsigned int, DstDim = -1, SrcDim = -1, IdxDim = -1, IndexIsMajor = true]: block: [72,0,0], thread: [39,0,0] Assertion srcIndex < srcSelectDimSize failed.
/pytorch/aten/src/THC/THCTensorIndex.cu:361: void indexSelectLargeIndex(TensorInfo<T, IndexType>, TensorInfo<T, IndexType>, TensorInfo<long, IndexType>, int, int, IndexType, IndexType, long) [with T = float, IndexType = unsigned int, DstDim = -1, SrcDim = -1, IdxDim = -1, IndexIsMajor = true]: block: [72,0,0], thread: [40,0,0] Assertion srcIndex < srcSelectDimSize failed.
/pytorch/aten/src/THC/THCTensorIndex.cu:361: void indexSelectLargeIndex(TensorInfo<T, IndexType>, TensorInfo<T, IndexType>, TensorInfo<long, IndexType>, int, int, IndexType, IndexType, long) [with T = float, IndexType = unsigned int, DstDim = -1, SrcDim = -1, IdxDim = -1, IndexIsMajor = true]: block: [72,0,0], thread: [41,0,0] Assertion srcIndex < srcSelectDimSize failed.
/pytorch/aten/src/THC/THCTensorIndex.cu:361: void indexSelectLargeIndex(TensorInfo<T, IndexType>, TensorInfo<T, IndexType>, TensorInfo<long, IndexType>, int, int, IndexType, IndexType, long) [with T = float, IndexType = unsigned int, DstDim = -1, SrcDim = -1, IdxDim = -1, IndexIsMajor = true]: block: [72,0,0], thread: [42,0,0] Assertion srcIndex < srcSelectDimSize failed.
/pytorch/aten/src/THC/THCTensorIndex.cu:361: void indexSelectLargeIndex(TensorInfo<T, IndexType>, TensorInfo<T, IndexType>, TensorInfo<long, IndexType>, int, int, IndexType, IndexType, long) [with T = float, IndexType = unsigned int, DstDim = -1, SrcDim = -1, IdxDim = -1, IndexIsMajor = true]: block: [72,0,0], thread: [43,0,0] Assertion srcIndex < srcSelectDimSize failed.
/pytorch/aten/src/THC/THCTensorIndex.cu:361: void indexSelectLargeIndex(TensorInfo<T, IndexType>, TensorInfo<T, IndexType>, TensorInfo<long, IndexType>, int, int, IndexType, IndexType, long) [with T = float, IndexType = unsigned int, DstDim = -1, SrcDim = -1, IdxDim = -1, IndexIsMajor = true]: block: [72,0,0], thread: [44,0,0] Assertion srcIndex < srcSelectDimSize failed.
/pytorch/aten/src/THC/THCTensorIndex.cu:361: void indexSelectLargeIndex(TensorInfo<T, IndexType>, TensorInfo<T, IndexType>, TensorInfo<long, IndexType>, int, int, IndexType, IndexType, long) [with T = float, IndexType = unsigned int, DstDim = -1, SrcDim = -1, IdxDim = -1, IndexIsMajor = true]: block: [72,0,0], thread: [45,0,0] Assertion srcIndex < srcSelectDimSize failed.
/pytorch/aten/src/THC/THCTensorIndex.cu:361: void indexSelectLargeIndex(TensorInfo<T, IndexType>, TensorInfo<T, IndexType>, TensorInfo<long, IndexType>, int, int, IndexType, IndexType, long) [with T = float, IndexType = unsigned int, DstDim = -1, SrcDim = -1, IdxDim = -1, IndexIsMajor = true]: block: [72,0,0], thread: [46,0,0] Assertion srcIndex < srcSelectDimSize failed.
/pytorch/aten/src/THC/THCTensorIndex.cu:361: void indexSelectLargeIndex(TensorInfo<T, IndexType>, TensorInfo<T, IndexType>, TensorInfo<long, IndexType>, int, int, IndexType, IndexType, long) [with T = float, IndexType = unsigned int, DstDim = -1, SrcDim = -1, IdxDim = -1, IndexIsMajor = true]: block: [72,0,0], thread: [47,0,0] Assertion srcIndex < srcSelectDimSize failed.
/pytorch/aten/src/THC/THCTensorIndex.cu:361: void indexSelectLargeIndex(TensorInfo<T, IndexType>, TensorInfo<T, IndexType>, TensorInfo<long, IndexType>, int, int, IndexType, IndexType, long) [with T = float, IndexType = unsigned int, DstDim = -1, SrcDim = -1, IdxDim = -1, IndexIsMajor = true]: block: [72,0,0], thread: [48,0,0] Assertion srcIndex < srcSelectDimSize failed.
/pytorch/aten/src/THC/THCTensorIndex.cu:361: void indexSelectLargeIndex(TensorInfo<T, IndexType>, TensorInfo<T, IndexType>, TensorInfo<long, IndexType>, int, int, IndexType, IndexType, long) [with T = float, IndexType = unsigned int, DstDim = -1, SrcDim = -1, IdxDim = -1, IndexIsMajor = true]: block: [72,0,0], thread: [49,0,0] Assertion srcIndex < srcSelectDimSize failed.
/pytorch/aten/src/THC/THCTensorIndex.cu:361: void indexSelectLargeIndex(TensorInfo<T, IndexType>, TensorInfo<T, IndexType>, TensorInfo<long, IndexType>, int, int, IndexType, IndexType, long) [with T = float, IndexType = unsigned int, DstDim = -1, SrcDim = -1, IdxDim = -1, IndexIsMajor = true]: block: [72,0,0], thread: [50,0,0] Assertion srcIndex < srcSelectDimSize failed.
/pytorch/aten/src/THC/THCTensorIndex.cu:361: void indexSelectLargeIndex(TensorInfo<T, IndexType>, TensorInfo<T, IndexType>, TensorInfo<long, IndexType>, int, int, IndexType, IndexType, long) [with T = float, IndexType = unsigned int, DstDim = -1, SrcDim = -1, IdxDim = -1, IndexIsMajor = true]: block: [72,0,0], thread: [51,0,0] Assertion srcIndex < srcSelectDimSize failed.
/pytorch/aten/src/THC/THCTensorIndex.cu:361: void indexSelectLargeIndex(TensorInfo<T, IndexType>, TensorInfo<T, IndexType>, TensorInfo<long, IndexType>, int, int, IndexType, IndexType, long) [with T = float, IndexType = unsigned int, DstDim = -1, SrcDim = -1, IdxDim = -1, IndexIsMajor = true]: block: [72,0,0], thread: [52,0,0] Assertion srcIndex < srcSelectDimSize failed.
/pytorch/aten/src/THC/THCTensorIndex.cu:361: void indexSelectLargeIndex(TensorInfo<T, IndexType>, TensorInfo<T, IndexType>, TensorInfo<long, IndexType>, int, int, IndexType, IndexType, long) [with T = float, IndexType = unsigned int, DstDim = -1, SrcDim = -1, IdxDim = -1, IndexIsMajor = true]: block: [72,0,0], thread: [53,0,0] Assertion srcIndex < srcSelectDimSize failed.
/pytorch/aten/src/THC/THCTensorIndex.cu:361: void indexSelectLargeIndex(TensorInfo<T, IndexType>, TensorInfo<T, IndexType>, TensorInfo<long, IndexType>, int, int, IndexType, IndexType, long) [with T = float, IndexType = unsigned int, DstDim = -1, SrcDim = -1, IdxDim = -1, IndexIsMajor = true]: block: [72,0,0], thread: [54,0,0] Assertion srcIndex < srcSelectDimSize failed.
/pytorch/aten/src/THC/THCTensorIndex.cu:361: void indexSelectLargeIndex(TensorInfo<T, IndexType>, TensorInfo<T, IndexType>, TensorInfo<long, IndexType>, int, int, IndexType, IndexType, long) [with T = float, IndexType = unsigned int, DstDim = -1, SrcDim = -1, IdxDim = -1, IndexIsMajor = true]: block: [72,0,0], thread: [55,0,0] Assertion srcIndex < srcSelectDimSize failed.
/pytorch/aten/src/THC/THCTensorIndex.cu:361: void indexSelectLargeIndex(TensorInfo<T, IndexType>, TensorInfo<T, IndexType>, TensorInfo<long, IndexType>, int, int, IndexType, IndexType, long) [with T = float, IndexType = unsigned int, DstDim = -1, SrcDim = -1, IdxDim = -1, IndexIsMajor = true]: block: [72,0,0], thread: [56,0,0] Assertion srcIndex < srcSelectDimSize failed.
/pytorch/aten/src/THC/THCTensorIndex.cu:361: void indexSelectLargeIndex(TensorInfo<T, IndexType>, TensorInfo<T, IndexType>, TensorInfo<long, IndexType>, int, int, IndexType, IndexType, long) [with T = float, IndexType = unsigned int, DstDim = -1, SrcDim = -1, IdxDim = -1, IndexIsMajor = true]: block: [72,0,0], thread: [57,0,0] Assertion srcIndex < srcSelectDimSize failed.
/pytorch/aten/src/THC/THCTensorIndex.cu:361: void indexSelectLargeIndex(TensorInfo<T, IndexType>, TensorInfo<T, IndexType>, TensorInfo<long, IndexType>, int, int, IndexType, IndexType, long) [with T = float, IndexType = unsigned int, DstDim = -1, SrcDim = -1, IdxDim = -1, IndexIsMajor = true]: block: [72,0,0], thread: [58,0,0] Assertion srcIndex < srcSelectDimSize failed.
/pytorch/aten/src/THC/THCTensorIndex.cu:361: void indexSelectLargeIndex(TensorInfo<T, IndexType>, TensorInfo<T, IndexType>, TensorInfo<long, IndexType>, int, int, IndexType, IndexType, long) [with T = float, IndexType = unsigned int, DstDim = -1, SrcDim = -1, IdxDim = -1, IndexIsMajor = true]: block: [72,0,0], thread: [59,0,0] Assertion srcIndex < srcSelectDimSize failed.
/pytorch/aten/src/THC/THCTensorIndex.cu:361: void indexSelectLargeIndex(TensorInfo<T, IndexType>, TensorInfo<T, IndexType>, TensorInfo<long, IndexType>, int, int, IndexType, IndexType, long) [with T = float, IndexType = unsigned int, DstDim = -1, SrcDim = -1, IdxDim = -1, IndexIsMajor = true]: block: [72,0,0], thread: [60,0,0] Assertion srcIndex < srcSelectDimSize failed.
/pytorch/aten/src/THC/THCTensorIndex.cu:361: void indexSelectLargeIndex(TensorInfo<T, IndexType>, TensorInfo<T, IndexType>, TensorInfo<long, IndexType>, int, int, IndexType, IndexType, long) [with T = float, IndexType = unsigned int, DstDim = -1, SrcDim = -1, IdxDim = -1, IndexIsMajor = true]: block: [72,0,0], thread: [61,0,0] Assertion srcIndex < srcSelectDimSize failed.
/pytorch/aten/src/THC/THCTensorIndex.cu:361: void indexSelectLargeIndex(TensorInfo<T, IndexType>, TensorInfo<T, IndexType>, TensorInfo<long, IndexType>, int, int, IndexType, IndexType, long) [with T = float, IndexType = unsigned int, DstDim = -1, SrcDim = -1, IdxDim = -1, IndexIsMajor = true]: block: [72,0,0], thread: [62,0,0] Assertion srcIndex < srcSelectDimSize failed.
/pytorch/aten/src/THC/THCTensorIndex.cu:361: void indexSelectLargeIndex(TensorInfo<T, IndexType>, TensorInfo<T, IndexType>, TensorInfo<long, IndexType>, int, int, IndexType, IndexType, long) [with T = float, IndexType = unsigned int, DstDim = -1, SrcDim = -1, IdxDim = -1, IndexIsMajor = true]: block: [72,0,0], thread: [63,0,0] Assertion srcIndex < srcSelectDimSize failed.

bharat-b7 / ipnet Goto Github PK

ipnet's Introduction

Implicit Part Network (IP-Net)

Prerequisites

Download pre-trained models

Preprocess data for training

Run demo IP-Net

Train IP-Net

Fit SMPL to IP-net predictions

Cite us:

License

ipnet's People

Contributors

Stargazers

Watchers

Forkers

ipnet's Issues

Recommend Projects

Recommend Topics

Recommend Org