Hello, I was wondering if there is a way to export an .obj/,fbx file along with corresponding materials from this?
If not, do you have any suggestions as to how to go about that if I were to try extend the code to incorporate that functionality?

Hello,

First of all, thank you for releasing the code for your seminal work. I really think neural volumes is one of the works that popularized differentiable rendering and inspired future works such as neural radiance fields.

My question is whether this codebase includes the code for the hybrid rendering method outlined in section 6.2 of the paper. I'm trying to fit Neural Volumes to multi-view video of a full-body human being, similar to the 5th subfigure in Fig. 1 of the main paper, but after reading it more carefully it seems as though I would need to use hybrid rendering to be able to render the fine details of the human being.

Could you

1. confirm the existence of hybrid rendering in this codebase AND
2. whether or not hybrid rendering was used to render the full-bodied human being in Fig. 1 of the main paper.

Thank you in advance.

Hi,

I am working on MIT dataset to test the network. When I specify a camera to render, it looks fine throughout timeline. However, while rendering the rotating video, the cameras are misaligned as shown in attached screenshot. All cameras look like clustered at the center and views are spread around within the range cameras cover. Is it possible to be any error in KRT or configuration?

Any suggestion is welcome.

Hi my name is Luan I am trying to render a Person Image but I am not being able to run can you create and for me a folder with the Setting setup to use a person image? Thank you.

Hi @stephenlombardi ,

Thanks for sharing this great work. I was wondering do you have any plan to release all the data that you used in the paper (apart from the dryice)?

Best,
Zirui

Hello,
Thanks for releasing your code. I am impressed by your work. Now I hope to run your code with my our dataset. I have two questions.

Firstly, I see the pose.txt is used in the code to put the objects in the center. If I use my own data, will the file still work?

Secondly, I see the code set the raypos is among -1 and 1. Is it the matrix in this pose file that narrows the range to -1 to 1? My own dataset' range is different.

Thirdly, does the code limit the scope of the template? Does it have to be between 0-255?

Thanks a lot in advance!

Hello,
Thanks for uploading the code, I would like to ask if your method is able to generalize to unseen training data ?

Hi,
is there any paper describing the used block-wise weight initialization scheme?

Hi there,

I wonder whether the origin of the volume is (0,0,0)?

I'm testing the method on a public dataset (http://people.csail.mit.edu/drdaniel/mesh_animation), and I know exactly where (0,0,0) is in the images. But the volume seems to float around the scene. This is the first preview for training process:

Each camera is pointing to the opposite side of the scene, so I expect the same for the volume location in images. But for some reason, they are on the same side in the images. Can you help?

Thank you.

Hi,
I have a few questions on how the data should be formatted and the data format of the provided dryice1.

• The model expects world space coordinate in meters? i.e if my extrinsics are already in meters do I still need the world_scale=1/256. in config.py file?
• The extrinsics are in world2cam and the rotation convention is like opencv? i.e, y-down,z-forward and x-right, assuming identity for pose.txt file?
• how long do I need to train for about 200 frames? And in the config.py file it seems you are skipping some frames? This is ok to do for my own sequence as well?
• in the KRT file, I see that there's 5 parameters above the RT matrix. This is the distortion correction in opencv format? But it is not used yes?
• I did not visualize your cameras, so I am not sure how they are distributed. Is it gonna be a problem if I use 50 cameras equally distributed in a half-hemisphere and the subject is already at world origin and 3.5 meters from every cameras? My question is do I need to filter the training cameras so that the back side of subject that is not seen by input 3 cameras is excluded?
• How do I choose the input cameras? I have a visualization of the cameras . Which camera config should I use? Is this more a question of which testing camera poses I intend to have, i.e narrower the testing cameras' range of view, the closer input training cameras can be? Config_0 is more orthogonal and Config_1 sees less of the backside.