Comments (7)
never mind, replaced by
from nr3d_lib.graphics.pack_ops import merge_two_packs_sorted_aligned
from nr3d_lib.graphics.raysample import packed_sample_pdf
from lightning-nerf.
Hi, sorry for the late reply. Since nr3d_lib
has recently had a major code update, you may need to use a previous version of nr3d_lib
. As mentioned in the README, we have tested nr3d_lib
with version 0.3.1 (more specifically, commit id: e4eba51
). Thus, if you do not want to modify the code, you may checkout this commit in nr3d_lib
.
I am also grateful for your mentioning how to modify the import script to align with the updated nr3d_lib
. Thank you for your time.
from lightning-nerf.
Hi, why do we use nr3d_lib
in this project? Can it be replaced by some modules in NeRFStudio?
Thanks in advance.
from lightning-nerf.
@szhang963
Hi, thank you for your interest.
Briefly, our sampling process (to generate sample points along rays) consists of two steps:
- Sampling guided by an occupancy grid.
- (Optionally) Perform importance sampling based on previous sampled results, similar to the fine stage of the hierarchical sampling used in Vanilla NeRF.
The second round of importance sampling may assist in sampling near surface areas. If the second round of importance sampling is disabled, then nr3d_lib
is not required.
- You may set
--pipeline.model.pdf-num-samples-per-ray 0
in the training script to disable the importance sampling. - Then, you may comment out the import of
nr3d_lib
used in lightning_nerf/sampler.py.
The reason is as follows.
This project uses an occupancy grid
sampler to sample points along rays, resulting in different numbers of samples per ray. Therefore, we cannot use conventional representations like (num_rays_per_batch, num_samples_per_ray, ...)
used in NeRFStudio to perform a second round of importance sampling. Instead, the output of occupancy grid
sampler is represented as (num_samples_per_batch, ...)
along with an index to indicate which ray the sample belongs to. nr3d_lib
supports a range of operations for this representation. Thus, we made a dependency on nr3d_lib
.
from lightning-nerf.
@XJay18 Thank you for your detailed reply.
What do the render results change if the importance sampling is disabled?
from lightning-nerf.
@szhang963
Basically, turning off the importance sampling would have a slight effect on interpolation metrics. However, the extrapolation results would be inferior since extrapolation requires a more accurate sampling around true surfaces.
Please see the following illustration (especially lane markings) of an extrapolation experiment for an intuitive understanding.
from lightning-nerf.
@XJay18 Thanks for your interpretation in detail. I will try it.
from lightning-nerf.
Related Issues (11)
- Missing 'meters' from 'nerfstudio.utils' HOT 1
- data preprocessing for lidar HOT 2
- AttributeError: module 'nerfacc.cuda.csrc' has no attribute 'ray_aabb_intersect' HOT 1
- Can you provide some guidance on preparing the custom dataset? HOT 2
- Renderer Output HOT 2
- refine depth HOT 3
- How to train using NeRF Studio HOT 1
- proposal sampler vs occupancy grid sampler HOT 1
- RuntimeError: min(): Expected reduction dim to be specified for input.numel() == 0. Specify the reduction dim with the 'dim' argument. HOT 1
- background model capacity HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from lightning-nerf.