rasmusrpaulsen / deep-mvlm Goto Github PK

Hi, thank you for your wonderful work, I have learned a lot from your code. But could you have me with this question?
I want to prepare rendered data using my raw data (.wrl, .bmp, and the landmark .txt files). But when run the preparedata.py file, I got the following problems. Unfortunately, I can not solve it. Do you know anything about it? I guess there is some format difference between your data and mine. If it is possible, could you share a set of example data for training? Maybe the data from DTU dataset because the BU_3DFE dataset is not available to download freely.

Preparing BU-3DFE data
Read 10 file ids
Processing 7 file ids for training
Processing F0001\F0001_AN02WH
F0001\F0001_AN02WH is locked - skipping
Processing F0001\F0001_AN03WH
F0001\F0001_AN03WH is locked - skipping
Processing F0001\F0001_AN04WH
F0001\F0001_AN04WH is locked - skipping
Processing F0001\F0001_DI01WH
Rendering F0001\F0001_DI01WH
Error near line 2: parse error
Traceback (most recent call last):
File "D:/07_AutomaticLandmarking/Paulsen_DeepMVLM/preparedata.py", line 349, in
main(cfg_global)
File "D:/07_AutomaticLandmarking/Paulsen_DeepMVLM/preparedata.py", line 334, in main
prepare_bu_3dfe_data(config)
File "D:/07_AutomaticLandmarking/Paulsen_DeepMVLM/preparedata.py", line 327, in prepare_bu_3dfe_data
process_file_bu_3dfe(config, base_name, output_dir)
File "D:/07_AutomaticLandmarking/Paulsen_DeepMVLM/preparedata.py", line 97, in process_file_bu_3dfe
pd = vrmlin.GetRenderer().GetActors().GetLastActor().GetMapper().GetInput()
AttributeError: 'NoneType' object has no attribute 'GetMapper'

Hi everybody,

I have two questions:

1. Can I place the facial landmarks on the following MRI head model (STL or PLY or OBJ) with Deep-MVLM? It has 500.000 vertices and 800.000 faces. However, there is no information other than the STL file.

2. By navigating around a head prototype with the depth camera, I obtained 10 RGB-D data. Can I create the head model (3d head.stl) using this code?

Thank you for this sharing

First of all, thanks to share your great 3D face landmark project.
I am trying to recognize a face from a .xyz cloud point data file, however you support OBJ/WRL/VTK/PLY only, because I am using a TOF sensor and able to capture xyz data only.
So would you please help me to make sure the xyz format able to support in your project.

Thanks & Best Regards
Sui

Hi, Prof .Rasmus R. Paulsen,
Is the DTU-3D dataset public available?
I try to train the dataset with SOTA network to improve the precision.

I am converting point cloud into mesh using Poison Reconstruction then applying the Deep-MVLM model. Any recommendations for how I can improve the accuracy of the landmarking?

The 3D model from the above image:
point_cloud_recon2.zip
Note that I have set "align_center_of_mass" and "write_pre_aligned" to true.

Did anyone successfully implemented ,the repo on google colab?

What are the modifications , we need , in order to run the repo on the ubuntu/aws? Could you please be kind to reply to the above queries?

Thanks,
satyam.

@Aodamiaomiao @Easy-Shu @iPsych @jonike @xrcodes @RasmusRPaulsen

Hi!
Thank you for sharing your great work with the community.
I would like to ask wich config file should I use for the model below, I tried geometry and depth, but without sucess. Is it something wrong with my mesh? Thank you.

Hi,
Can you share the implementation of 3D landmark annotation of DTU3D ?
The orignal annotation of BU-3DFE dataset seems unsatisfactory and does not provide the annotation of the nose tip that your annotation does.

Any help would be appreciate !

Hi. I have a query regarding the implementation using custom dataset.
I have .obj mesh file (example screenshot is attached)

I wanted to test the model on it but I get the message "Not enough valid view lines for landmark" for all the landmarks.

I wanted to ask if you can guide on how to train the model on custom dataset and how to annotate.

Hello, may I ask, if I test my own data, I find that first of all, my .obj file has vn, while the test data does not, will this affect? I also find that when I import the data into the model, the generated views are all white images. What is the reason? Thank you very much for your answer

When I try to run this code on an AWS SageMaker notebook instance:
python predict.py --c configs/DTU3D-RGB.json --n assets/testmeshA.obj

I run in to the following error:
Processing assets/testmeshA.obj
Initialising model
Loading checkpoint
Getting device
Warning: There's no GPU available on this machine,prediction will be performed on CPU.
Loading checkpoint: https://shapeml.compute.dtu.dk/Deep-MVLM/models/MVLMModel_DTU3D_RGB_07092019-c1cc3d59.pth
Traceback (most recent call last):
File "predict.py", line 70, in
main(global_config)
File "predict.py", line 51, in main
process_one_file(config, name)
File "predict.py", line 12, in process_one_file
dm = deepmvlm.DeepMVLM(config)
File "/home/ec2-user/SageMaker/Deep-MVLM/deepmvlm/api.py", line 37, in init
self.device, self.model = self._get_device_and_load_model_from_url()
File "/home/ec2-user/SageMaker/Deep-MVLM/deepmvlm/api.py", line 75, in _get_device_and_load_model_from_url
checkpoint = load_url(check_point_name, model_dir, map_location=device)
File "/home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/hub.py", line 506, in load_state_dict_from_url
return torch.load(cached_file, map_location=map_location)
File "/home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/serialization.py", line 529, in load
return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
File "/home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/serialization.py", line 702, in _legacy_load
result = unpickler.load()
File "/home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/pathlib.py", line 1002, in new
% (cls.name,))
NotImplementedError: cannot instantiate 'WindowsPath' on your system

On the left is the my mesh put through the Deep-MVLM algorithm and visualised with its built in functions. On the right is when I open the same mesh in mesh lab. How do I get the lighting/shading similar to how you all did it? It brings out the contours much better and it would help to know exactly how you did it.

Thanks.

I found my test obj model is vary small in offscreen rendering image, and you hardcode the SetParallelScale(150),so can I modify this magic number to adjust my model size.

Hello,
I have an issue when I am running the explained command : python predict.py --c configs\DTU3D-depth.json --n man_bust.obj

The landmarks does not calculate themself well and don't appear on the render window.

I did not change any part of the code.
What can I do ?

Thanks