How to get more detail about the camera pose? about scnerf HOT 10 CLOSED

Thank you very much for your reply. It helps me a lot.

from scnerf.

For the files named "poses_bounds.npy", I strongly recommend referring to the codes below, which is the process of extracting bounded poses. It might help you.

https://github.com/weiyithu/NerfingMVS/blob/8c8f96244146b929a1495caf2719c090b48ac082/utils/pose_utils.py#L72
https://github.com/Fyusion/LLFF/blob/master/llff/poses/pose_utils.py

I'm not really sure what you are targeting so, I'll assume that you are trying to render your custom scenes with our code. Based on my understanding, our code does not use simplices. Thus, could you check whether the file "simplices.npy" is necessary for your code?

from scnerf.

The article here(https://github-wiki-see.page/m/pelednoam/scatter_3d_to_surface/wiki/Preprocessing) could help you understand what "slices.npy" exactly is. I hope this link helps you.

from scnerf.

Thank you for your reply. It works now. I'm looking forward to the final result.

from scnerf.

Thanks for taking an interest in our work.

The estimated camera poses are stored in ".tar" file, the ckpt file with the standard torch format.

If you take a look at the key "camera_model", you could find all the estimated camera parameters. To be precise, you are able to find the camera parameters by the script below.

torch.load([path to tar file])["camera_model"]

I might guess you would have difficulty loading the estimated camera poses since it is divided into two parameters: "extrinsics_noise" and "extrinsics_init". As described in our main paper, we use a different camera pose estimation, 6D representation. Thus, I recommend referring to the code here.

• The code to convert 6D representations to rotation matrices.
  

  SCNeRF/model/camera_utils.py

  Line 78 in dc57e9f

  def ortho2rotation(poses):

• The code loading trained model
  

  SCNeRF/NeRF/create_nerf.py

  Line 172 in dc57e9f

  camera_model.load_state_dict(ckpt["camera_model"])

To simply extract the file, here is the easiest way to checkout camera poses.

import torch
pretrained = torch.load([path to pretrained])
camera_model = pretrained["camera_model"]
rot_6D_trans_3D = camera_model["extrinsics_initial"] + camera_model["extrinsics_noise"] * <your noise scale> 
ortho2rotation(rot_6D_trans_3D[:, :6]) -> rotation matrices
rot_6D_trans_3D[:, -3:] -> translation vectors [edited]

[path to pretrained] = path to the .tar file you are planning to open.
[your noise scale] = the extrinsic noise scale you have used during training.

If this reply was not sufficient for you, then let me know what problem you are suffering from.

Again, thanks for taking an interest in our work.

from scnerf.

If you have any difficulty extracting camera poses, please reopen the issue. I would provide more detailed descriptions for it.

from scnerf.

Hello author,

In this way, I can get the 4x4 matrice about the camera pose. I notice in your code that the translation vectors consist of the first three numbers of rot_6D_trans_3D. And I find the translation vector is a part of rotations matrice, like this:

So is that a mistake? Maybe the translation vector should be the last three numbers of rot_6D_trans_3D.(The shape of rot_6D_trans_3D is batch * 9)

Another question is that I notice in your paper you initialize all the rotation matrices, the translation vectors, and focal lengths to an identity matrix, zero vector, and height and width of the captured images. And your code also have a file named load_blender.py. If I want to estimate camera pose of my own images by rgbd camera. Can I choose 'blender' dataset_type? (I think the Blender dataset is closer to the RGBD camera collection)

from scnerf.

Oh, sorry for being late. As you pointed out, the last three elements in the ckpt file are translation vectors. Sorry for the mistake above. I've just fixed it.

The reason why we include the 'blender' dataset is that the very first version of our paper included noise injection ablation studies. For clearer paper writing, we have removed such experiments. In these experiments, we have added synthetic noises on camera parameters to verify that our model recovers the noises. I could find a few traces of experiments we have done in this code. If you ever want to use the scenes in the blender dataset, you could use it but I cannot ensure whether this code still perfectly works.

from scnerf.

I got it. And another question is that I find the date include a pose_bounds.npy and a simplices.npy. I read the file. It show me some matrices (The shape of matrices about pose_bounds and simplices are n * 17 and n * 3). I use colmap to process my data and get the result in folder 'sparse' and database.db, but I don't know the meaning about the two .npy files and how to get it?
In addition, I have sent an email to you about this question. If you answered here, please ignore the email.

Finally, thank you for your reply.

from scnerf.

Glad to hear the good news! 👍

from scnerf.