fbxiang / neutex Goto Github PK

Hi @fbxiang , thanks for sharing the codes.
I did some simple experiments, and it seems that removing the cycle loss leads to better novel view synthesis (but maybe more distorted uv map).
In Fig. 3 of the paper, removing cycle loss will lead to very bad mapping results which didn't happen in my experiments. May I know how to get the results in the last column of Fig. 3? Thanks.

hello，this is a great work and thanks for your sharing.
when I run the demo(dtu.sh) you provided, I'm a bit confused and I need your help. In your paper "NeuTex", you mentioned that you used Chamfer loss during F_uv^-1 initialization. However, loss_chamfer_weight=-1 means that it has not been used. If I want to try it, When do I need to set loss_chamfer_weight=1 so that I can get the result similar to Figure 3 in paper ? And how do I understand this word in 4.2Training detail，initialize F_uv^-1 with a point cloud from COLMAP.

zeros = torch.zeros(camera_position.shape[0], dtype=torch.long, device=camera_position.device)
geometry_embedding = self.net_geometry_embedding(zeros) 
...
point_array_2d, points_3d = self.net_atlasnet(geometry_embedding)
...

In the code, does the geometric latent code represent the point cloud here ？

I can't find the code where the texture modification is injected to the network, could you please give some guidance?

Hi, thanks for the amazing work!
I train the network on scan 114 data and got accurate rendering results like this:

However, when I run visualize_nerf_atlas_radiance.py to visualize the geometry and texture map, I found they are strange.
point cloud:

mesh:

texture map:

Hello,
Could you give me more detailed guidelines for using custom datasets?
I tried to follow your guidelines, but I was confused about how to set these fields:

I also wonder should I make all of these files in the 'dtu_dataset.py' following above fields?
self.campos = np.load(self.data_dir + "/in_camOrgs.npy")
self.camat = np.load(self.data_dir + "/in_camAts.npy")
self.focal = np.load(self.data_dir + "/in_camFocal.npy")
self.princpt = np.load(self.data_dir + "/in_camPrincpt.npy")
self.extrinsics = np.load(self.data_dir + "/in_camExtrinsics.npy")
self.point_cloud = trimesh.load(self.data_dir + "/pcd_down_unit.ply")
I guessed "pcd_down_unit.ply" is a polygon file of point clouds, but I don't know how to make other files.

I'd really appreciate it if you reply to my question.

非常感谢您能开源代码！

在复现过程中遇到了点问题：

1. run文件夹下的dtu.sh文件
2. 跑run文件夹下的dtu-freeze.sh时，需要'./checkpoints//114/latest_states.pth'

Amazing work. I have some q
I find there is only UV as inputs in TexMLP, but as mentioned in the paper, there is view direction and uv as inputs.
So how to decide which direction to use when we unpack the texture to cubemap?

Hi @fbxiang
Great work!

Could you share the weights/checkpoints of the models you have trained?