Comments (7)
In case someone else will find this helpful.. I believe COLMAP follows the format of the projection matrix given as transforming 3D camera coordinates to world coordinates. Hence, to go from the above image as formed from W -> C transformation to this image,
try the following:
# Given 3x3 W -> C SO(3) matrix and r, the translation vector, form the correct 4x4 transformation matrix
# X_w = R^T X_c - R^T t (cam to world, what was needed)
# X_c = R X_w + t (world to cam, what I had before)
Rt = np.matmul(so3.transpose(), r)
trans = np.vstack((np.hstack((so3.transpose(), -Rt.reshape(-1, 1))), [0, 0, 0, 1]))
You can then view using python view_data.py <data_root>
. All one needs is images, poses, and intrinsics that follow the above format (no bbox.txt or other files strictly needed).
from svox2.
I am little bit confused about the intrinsic matrix, shouldn't it be like this?
fx 0 cx 0
0 fy cy 0
0 0 1 0
0 0 0 1
from svox2.
Hi, thanks for the question.
If you want to use your own camera poses, you will have to process them our NSVF-based format, which is fairly simple anyway (see below). Other than proc_colmap there is also proc_record3d.py
which processes captures from the iPhone app Record3D to our format; this might be a helpful example.
Currently svox2 itself only supports the pinhole model fx/fy/cx/cy.
The run_colmap.py
script (called by proc_colmap.sh
) actually estimates radial distortion parameters by default with COLMAP but will undistort the images. For simplicity, you can also use OpenCV to undistort your own images.
Format reference:
intrinsics.txt
: 4x4 matrix,
fx 0 0 cx
0 fy 0 cy
0 0 1 0
0 0 0 1
images/
or rgb/
: images (*.png or *.jpg)
pose/
: 4x4 c2w pose matrix for each image (*.txt), OpenCV convention
from svox2.
Thank you kindly! I may try undistorting all my images, though the distortion coefficients are very small here, so I'm going to ignore them for the moment.
I was able to get the nsvf dataset loader working after formatting my images and poses to the following convention (had to add in a conversion from grayscale to rgb)
<dataset_name>
|-- bbox.txt # bounding-box file
|-- intrinsics.txt # 4x4 camera intrinsics
|-- images
|-- 0_000001.png # target image for each view
...
|-- 1_000001.png
...
|-- pose
|-- 0_000001.txt # camera pose for each view (4x4 matrices)
...
|-- 1_000001.txt
...
I'll continue training and testing, granted there are quite a number of hyperparamers to adjust here, but hoping I can start to see the rough formation of my imaged object.
Do you know what convention rotation matrices are to follow for NSVF? Having a difficult time determining if my axes are aligned with its standard. For example, here is my distribution of camera poses
from svox2.
images/
orrgb/
: images (*.png or .jpg)pose/
: 4x4 c2w pose matrix for each image (.txt), OpenCV convention
Apologies, I totally missed this remark! Would have saved myself a headache 😂
from svox2.
how can i use views with different intrinsics (images captured by multi-cameras)?
from svox2.
what does <CHECKPOINT.npz> <data_dir> mean?
from svox2.
Related Issues (20)
- Set size of voxels?
- Why the learning rate of sigma `lr_sigma` is set so large
- How to add a new loss? HOT 1
- RuntimeError: CUDA out of memory HOT 3
- segmentation fault HOT 1
- How to get RGB color at sampled 3D points HOT 17
- How to extract mesh from a pretrained svox2 model(.npz)? HOT 1
- About the render_ray_kernel
- svox2 install error - need help! HOT 2
- how to implement camera parameters free (such as pose, need not solmap) based on this Plenoxels?
- Depth Map Issues
- About colmap2nsvf.py
- ingp2nsvf.py doesn't work
- to_svox1 and open with volrend
- ERROR: Failed building wheel for svox2 Running setup.py clean for svox2 Failed to build svox2 ERROR: Could not build wheels for svox2, which is required to install pyproject.toml-based projects
- Error when running proc_colmap.sh HOT 1
- mipnerf-360 dataset - parameters HOT 8
- _C is always the nonetype HOT 1
- Weight decay only applied when printing averaged stats
- installation failed, subprocess-exited-with-error HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from svox2.