Comments (5)
dendenxu
commented on August 20, 2024
1
Hi, thanks for your interest in our code.
From the error above, it seems that you're missing a proper X11 installation:
CMake Error at /tmp/pip-build-env-wz5irisp/overlay/lib/python3.9/site-packages/cmake/data/share/cmake-3.29/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
Could NOT find X11 (missing: X11_X11_INCLUDE_PATH X11_X11_LIB)
Does this machine have a GUI? Real-time visualization requires a graphical interface to function correctly. So if this a display-less server, you could try running the offline rendering commands beginning with evc -t test as listed here: https://github.com/zju3dv/4K4D?tab=readme-ov-file#rendering-of-trained-model
# Testing with input views
evc -t test -c configs/projects/realtime4dv/rendering/4k4d_0013_01.yaml,configs/specs/eval.yaml,configs/specs/vf0.yaml # Only render some of the view of the first frame
evc -t test -c configs/projects/realtime4dv/rendering/4k4d_0013_01.yaml,configs/specs/eval.yaml # Only rendering some selected testing views and frames
# Rendering rotating novel views
evc -t test -c configs/projects/realtime4dv/rendering/4k4d_0013_01.yaml,configs/specs/eval.yaml,configs/specs/spiral.yaml,configs/specs/ibr.yaml,configs/specs/vf0.yaml # Render a static rotating novel view
evc -t test -c configs/projects/realtime4dv/rendering/4k4d_0013_01.yaml,configs/specs/eval.yaml,configs/specs/spiral.yaml,configs/specs/ibr.yaml # Render a dynamic rotating novel viewfrom 4k4d.
dendenxu
commented on August 20, 2024
1
Looks like the code could not find your dataset, please check whether join(self.data_root, self.images_dir) exists.
The dataset structure for EasyVolcap is documented here: https://github.com/zju3dv/4K4D?tab=readme-ov-file#datasets
from 4k4d.
dendenxu
commented on August 20, 2024
1
Strangely, I'm unable to reproduce the errors on my end. Can you help me make sure the pre-trained model is loaded?
There should be a line like: easyvolcap.utils.net_utils -> load_network: Loaded network data/trained_model/4k4d_0013_01/1599.npz at epoch -1 in your log.
For training, 4K4D shouldn't require more than 5GB of VRAM for the DNA-Rendering dataset.
from 4k4d.
hhhddddddd
commented on August 20, 2024
Thank you very much for your advice!
My machine haven't a GUI, so I try executing the follow command:evc -t test -c configs/projects/realtime4dv/rendering/4k4d_0013_01.yaml,configs/specs/eval.yaml However, I have a new error:
(4k4d) xuankai@Baicheng:~/code/4K4D$ evc -t test -c configs/projects/realtime4dv/rendering/4k4d_0013_01.yaml,configs/specs/eval.yaml
2024-03-29 11:39:50.187596 easyvolcap.runners -> <module>: Failed to import volumetric_video_viewer.py, missing __init__.py:10
package: imgui_bundle
2024-03-29 11:39:50.194870 __main__ -> preflight: Starting experiment: 4k4d_0013_01, command: test main.py:80
2024-03-29 11:39:54.231495 easyvolcap.utils.console_utils -> inner: Runtime exception: index 0 is out of console_utils.py:391
bounds for axis 1 with size 0
╭────────────────────────────────────────────── Traceback (most recent call last) ──────────────────────────────────────────────╮
│ /home/xuankai/code/4K4D/easyvolcap/utils/console_utils.py:388 in inner │
│ │
│ ❱ 388 │ │ │ │ return func(*args, **kwargs) │
│ │
│ /home/xuankai/code/4K4D/easyvolcap/scripts/main.py:272 in main │
│ │
│ ❱ 272 │ else: globals()[args.type](cfg) # invoke this (call callable_from_cfg -> call_from_cfg) I: utilize @callable_from_ │
│ │
│ /home/xuankai/code/4K4D/easyvolcap/engine/registry.py:61 in inner │
│ │
│ ❱ 61 │ │ return call_from_cfg(func, cfg) │
│ │
│ /home/xuankai/code/4K4D/easyvolcap/engine/registry.py:52 in call_from_cfg │
│ │
│ ❱ 52 │ return func(**call_args) # the args of function is prepared. │
│ │
│ /home/xuankai/code/4K4D/easyvolcap/scripts/main.py:165 in test │
│ │
│ ❱ 165 │ val_dataloader: "VolumetricVideoDataloader" = DATALOADERS.build(val_dataloader_cfg) # reuse the validataion I: val │
│ │
│ /home/xuankai/code/4K4D/easyvolcap/engine/registry.py:305 in build │
│ │
│ ❱ 305 │ │ return self.build_func(*args, **kwargs, registry=self) # runner & viewer: self.build_func is build_from_cfg │
│ │
│ /home/xuankai/code/4K4D/easyvolcap/engine/registry.py:136 in build_from_cfg │
│ │
│ ❱ 136 │ return call_from_cfg(obj_cls, args) │
│ │
│ /home/xuankai/code/4K4D/easyvolcap/engine/registry.py:52 in call_from_cfg │
│ │
│ ❱ 52 │ return func(**call_args) # the args of function is prepared. │
│ │
│ /home/xuankai/code/4K4D/easyvolcap/dataloaders/volumetric_video_dataloader.py:94 in __init__ │
│ │
│ ❱ 94 │ │ dataset: VolumetricVideoDataset = DATASETS.build(dataset_cfg) │
│ │
│ /home/xuankai/code/4K4D/easyvolcap/engine/registry.py:305 in build │
│ │
│ ❱ 305 │ │ return self.build_func(*args, **kwargs, registry=self) # runner & viewer: self.build_func is build_from_cfg │
│ │
│ /home/xuankai/code/4K4D/easyvolcap/engine/registry.py:136 in build_from_cfg │
│ │
│ ❱ 136 │ return call_from_cfg(obj_cls, args) │
│ │
│ /home/xuankai/code/4K4D/easyvolcap/engine/registry.py:52 in call_from_cfg │
│ │
│ ❱ 52 │ return func(**call_args) # the args of function is prepared. │
│ │
│ /home/xuankai/code/4K4D/easyvolcap/dataloaders/datasets/image_based_dataset.py:56 in __init__ │
│ │
│ ❱ 56 │ │ call_from_cfg(super().__init__, kwargs, skip_loading_images=skip_loading_images) │
│ │
│ /home/xuankai/code/4K4D/easyvolcap/engine/registry.py:52 in call_from_cfg │
│ │
│ ❱ 52 │ return func(**call_args) # the args of function is prepared. │
│ │
│ /home/xuankai/code/4K4D/easyvolcap/dataloaders/datasets/volumetric_video_dataset.py:276 in __init__ │
│ │
│ ❱ 276 │ │ self.load_paths() # load image files into self.ims │
│ │
│ /home/xuankai/code/4K4D/easyvolcap/dataloaders/datasets/volumetric_video_dataset.py:382 in load_paths │
│ │
│ ❱ 382 │ │ self.ims_dir = join(*split(dirname(self.ims[0, 0]))[:-1]) # logging only │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
IndexError: index 0 is out of bounds for axis 1 with size 0
*** index 0 is out of bounds for axis 1 with size 0
> /home/xuankai/code/4K4D/easyvolcap/dataloaders/datasets/volumetric_video_dataset.py(382)load_paths()
380 ims = [np.asarray(ims[i])[:min([len(i) for i in ims])] for i in range(len(ims))] # deal with the fact that some
381 self.ims = np.asarray(ims) # V, N
--> 382 self.ims_dir = join(*split(dirname(self.ims[0, 0]))[:-1]) # logging only
383
384 # TypeError: can't convert np.ndarray of type numpy.str_. The only supported types are: float64, float32, float16
(Pdbr) exit
I wonder if you have any advice on the problems I am facing? Thank you very much!
from 4k4d.
hhhddddddd
commented on August 20, 2024
Thank you very much for helping me find the cause of the above problem！
I encountered a new problem when executing evc -t test -c configs/projects/realtime4dv/rendering/4k4d_0013_01.yaml,configs/specs/eval.yaml,configs/specs/vf0.yaml
2024-03-29 15:21:35.520479 easyvolcap.utils.console_utils -> inner: Runtime exception: The size of tensor a console_utils.py:391
(203840) must match the size of tensor b (440608) at non-singleton dimension 1
╭────────────────────────────────────────────── Traceback (most recent call last) ──────────────────────────────────────────────╮
│ /home/xuankai/code/4K4D/easyvolcap/utils/console_utils.py:388 in inner │
│ │
│ ❱ 388 │ │ │ │ return func(*args, **kwargs) │
│ │
│ /home/xuankai/code/4K4D/easyvolcap/scripts/main.py:272 in main │
│ │
│ ❱ 272 │ else: globals()[args.type](cfg) # invoke this (call callable_from_cfg -> call_from_cfg) I: utilize @callable_from_ │
│ │
│ /home/xuankai/code/4K4D/easyvolcap/engine/registry.py:61 in inner │
│ │
│ ❱ 61 │ │ return call_from_cfg(func, cfg) │
│ │
│ /home/xuankai/code/4K4D/easyvolcap/engine/registry.py:52 in call_from_cfg │
│ │
│ ❱ 52 │ return func(**call_args) # the args of function is prepared. │
│ │
│ /home/xuankai/code/4K4D/easyvolcap/scripts/main.py:181 in test │
│ │
│ ❱ 181 │ launcher(**kwargs, runner_function=runner.test, runner_object=runner) │
│ │
│ /home/xuankai/code/4K4D/easyvolcap/scripts/main.py:52 in launcher │
│ │
│ ❱ 52 │ runner_function() │
│ │
│ /home/xuankai/code/4K4D/easyvolcap/runners/volumetric_video_runner.py:275 in test │
│ │
│ ❱ 275 │ │ self.test_epoch(epoch) │
│ │
│ /home/xuankai/code/4K4D/easyvolcap/runners/volumetric_video_runner.py:423 in test_epoch │
│ │
│ ❱ 423 │ │ for _ in test_generator: pass # the actual calling │
│ │
│ /home/xuankai/code/4K4D/easyvolcap/runners/volumetric_video_runner.py:435 in test_generator │
│ │
│ ❱ 435 │ │ │ │ scalar_stats = self.evaluator.evaluate(output, batch) │
│ │
│ /home/xuankai/code/4K4D/easyvolcap/runners/evaluators/volumetric_video_evaluator.py:31 in evaluate │
│ │
│ ❱ 31 │ │ img, img_gt, _ = super().generate_type(output, batch, Visualization.RENDER) │
│ │
│ /home/xuankai/code/4K4D/easyvolcap/runners/visualizers/volumetric_video_visualizer.py:176 in generate_type │
│ │
│ ❱ 176 │ │ │ img_gt = img_gt + output.bg_color * (1 - batch.msk) │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
RuntimeError: The size of tensor a (203840) must match the size of tensor b (440608) at non-singleton dimension 1
2024-03-29 15:21:35.806174 easyvolcap.runners.volumetric… 0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/3 0:00:18 < -:--:-- ? it/s v…
-> test_generator:
*** The size of tensor a (203840) must match the size of tensor b (440608) at non-singleton dimension 1
> /home/xuankai/code/4K4D/easyvolcap/runners/visualizers/volumetric_video_visualizer.py(176)generate_type()
174 if img_gt is not None and 'bg_color' in output and 'msk' in batch:
175 # Fill gt with input BG colors
--> 176 img_gt = img_gt + output.bg_color * (1 - batch.msk)
177
178 if self.store_image_error and img_gt is not None:
(Pdbr)
I wonder if you have any advice on the problems I am facing? In addition, I would also like to ask about the video memory requirements of the NVIDIA 3090 for the experiment in 4k4d. Thank you very much!
from 4k4d.
hhhddddddd
commented on August 20, 2024
Thank you very much for helping me find the cause of the above problem！
I encountered a new problem when executing evc -t test -c configs/projects/realtime4dv/rendering/4k4d_0013_01.yaml,configs/specs/eval.yaml,configs/specs/vf0.yaml
(4k4d) xuankai@user-SYS:~/code/4K4D$ evc -t test -c configs/projects/realtime4dv/rendering/4k4d_0013_01.yaml,configs/specs/eval.yaml,configs/specs/vf0.yaml
2024-03-30 16:27:03.104763 easyvolcap.runners -> <module>: Failed to import volumetric_video_viewer.py, missing __init__.py:10
package: imgui_bundle
2024-03-30 16:27:05.603352 __main__ -> preflight: Starting experiment: 4k4d_0013_01, command: test main.py:80
2024-03-30 easyvolcap.… Preparing vhulls data/renbody/0013_01/surfs VAL 100% ━━━━━━━━━━━━━ 1/1 0:00:20 < 0:00:00 ? it/s v…
16:27:29.795… ->
load_vhulls:
2024-03-30 easyvolca… Loading mask bytes for data/renbody/0013_01/masks VAL 100% ━━━━━━━━━━ 60/60 0:00:03 < 0:00:00 33.85 p…
16:27:33.… -> it/s
load_resi…
2024-03… easyvol… Loading imgs bytes for data/renbody/0013_01/images_calib VAL 100% ━━━━━━━━ 60/60 0:00:09 < 0:00:00 9.654 p…
16:27:4… -> it/s
load_re…
2024-03-30 easyvolcap.… Cropping msks imgs for data/renbody/0013_01 VAL 100% ━━━━━━━━━━━ 60/60 0:00:05 < 0:00:00 21.56 it/s p…
16:27:48.70… ->
decode_crop…
2024-03-30 16:27:57.680329 easyvolcap.runners.visualizers.volumetric_video_visualizer -> volumetric_video_visualizer.py:80
__init__: Visualization output: data/result/4k4d_0013_01/{RENDER}
2024-03-30 16:27:57.778188 easyvolcap.runners.recorders -> __init__: Saved config file to recorders.py:105
data/record/4k4d_0013_01/4k4d_0013_01_1711787277.yaml
2024-03-30 16:27:57.782684 __main__ -> launcher: Launching runner for experiment: 4k4d_0013_01 main.py:50
2024-03-30 easyvolcap.models.s… Caching rgbw and center 100% ━━━━━━━━━━━━━━━━━━━━━ 1/1 0:00:27 < 0:00:00 ? it/s s…
16:28:38.710354 ->
_load_state_dict_po…
2024-03-30 easyvolcap.models… Computing features for caching 100% ━━━━━━━━━━━━━━━━━━━ 1/1 0:00:00 < 0:00:00 ? it/s s…
16:28:38.768409 ->
_load_state_dict_…
2024-03-30 easyvolcap.models.s… Caching spherical harmonics 100% ━━━━━━━━━━━━━━━━━━━ 1/1 0:00:01 < 0:00:00 ? it/s s…
16:28:40.093285 ->
_load_state_dict_po…
2024-03-30 easyvolcap.models.s… Caching radius and alpha 100% ━━━━━━━━━━━━━━━━━━━━━ 1/1 0:00:00 < 0:00:00 ? it/s s…
16:28:40.262554 ->
precompute_geometry:
2024-03-30 16:28:40.270231 easyvolcap.utils.net_utils -> load_network: Loaded network net_utils.py:436
data/trained_model/4k4d_0013_01/1599.npz at epoch -1
2024-03-30 16:28:56.893129 easyvolcap.models.samplers.point_planes_sampler -> prepare_opengl: Init point_planes_sampler.py:491
eglctx with h, w: 1124, 392
2024-03-30 16:28:59.408833 easyvolcap.models.samplers.point_planes_sampler -> prepare_opengl: Init point_planes_sampler.py:499
cudagl with size: 262144
2024-03-30 16:29:07.705786 easyvolcap.utils.gl_utils -> init_textures: Created texture of h, w: 1124, 392 gl_utils.py:1289
2024-03-30 16:29:19.083550 easyvolcap.utils.metric_utils -> lpips: Initializing LPIPS network metric_utils.py:39
2024-03-30 16:29:45.801998 easyvolcap.utils.console_utils -> inner: Runtime exception: CUDA out of memory. console_utils.py:391
Tried to allocate 1.20 GiB (GPU 0; 23.69 GiB total capacity; 8.86 GiB already
allocated; 148.62 MiB free; 23.69 GiB allowed; 9.25 GiB reserved in total by
PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb
to avoid fragmentation. See documentation for Memory Management and
PYTORCH_CUDA_ALLOC_CONF
╭────────────────────────────────────────────── Traceback (most recent call last) ──────────────────────────────────────────────╮
│ /home/xuankai/code/4K4D/easyvolcap/utils/console_utils.py:388 in inner │
│ │
│ ❱ 388 │ │ │ │ return func(*args, **kwargs) │
│ │
│ /home/xuankai/code/4K4D/easyvolcap/scripts/main.py:272 in main │
│ │
│ ❱ 272 │ else: globals()[args.type](cfg) # invoke this (call callable_from_cfg -> call_from_cfg) I: utilize @callable_from_ │
│ │
│ /home/xuankai/code/4K4D/easyvolcap/engine/registry.py:61 in inner │
│ │
│ ❱ 61 │ │ return call_from_cfg(func, cfg) │
│ │
│ /home/xuankai/code/4K4D/easyvolcap/engine/registry.py:52 in call_from_cfg │
│ │
│ ❱ 52 │ return func(**call_args) # the args of function is prepared. │
│ │
│ /home/xuankai/code/4K4D/easyvolcap/scripts/main.py:181 in test │
│ │
│ ❱ 181 │ launcher(**kwargs, runner_function=runner.test, runner_object=runner) │
│ │
│ /home/xuankai/code/4K4D/easyvolcap/scripts/main.py:52 in launcher │
│ │
│ ❱ 52 │ runner_function() │
│ │
│ /home/xuankai/code/4K4D/easyvolcap/runners/volumetric_video_runner.py:275 in test │
│ │
│ ❱ 275 │ │ self.test_epoch(epoch) │
│ │
│ /home/xuankai/code/4K4D/easyvolcap/runners/volumetric_video_runner.py:423 in test_epoch │
│ │
│ ❱ 423 │ │ for _ in test_generator: pass # the actual calling │
│ │
│ /home/xuankai/code/4K4D/easyvolcap/runners/volumetric_video_runner.py:435 in test_generator │
│ │
│ ❱ 435 │ │ │ │ scalar_stats = self.evaluator.evaluate(output, batch) │
│ │
│ /home/xuankai/code/4K4D/easyvolcap/runners/evaluators/volumetric_video_evaluator.py:42 in evaluate │
│ │
│ ❱ 42 │ │ │ │ metrics[compute.__name__] = compute(img, img_gt) # actual computation of the metrics │
│ │
│ /home/xuankai/code/4K4D/easyvolcap/utils/metric_utils.py:45 in lpips │
│ │
│ ❱ 45 │ return lpips.compute_lpips(x.cuda().permute(0, 3, 1, 2) * 2 - 1, y.cuda().permute(0, 3, 1, 2) * 2 - 1).mean().item() │
│ │
│ /home/xuankai/anaconda3/envs/4k4d/lib/python3.9/site-packages/torch/nn/modules/module.py:1501 in _call_impl │
│ │
│ ❱ 1501 │ │ │ return forward_call(*args, **kwargs) │
│ │
│ /home/xuankai/anaconda3/envs/4k4d/lib/python3.9/site-packages/lpips/lpips.py:124 in forward │
│ │
│ ❱ 124 │ │ │ diffs[kk] = (feats0[kk]-feats1[kk])**2 │
│ │
│ /home/xuankai/anaconda3/envs/4k4d/lib/python3.9/site-packages/torch/_tensor.py:40 in wrapped │
│ │
│ ❱ 40 │ │ │ return f(*args, **kwargs) │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
OutOfMemoryError: CUDA out of memory. Tried to allocate 1.20 GiB (GPU 0; 23.69 GiB total capacity; 8.86 GiB already allocated; 148.62 MiB free; 23.69 GiB allowed; 9.25 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
2024-03-30 16:29:47.748662 easyvolcap.runners.volumetric… 0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/3 0:01:07 < -:--:-- ? it/s v…
-> test_generator:
*** CUDA out of memory. Tried to allocate 1.20 GiB (GPU 0; 23.69 GiB total capacity; 8.86 GiB already allocated; 148.62 MiB free;
23.69 GiB allowed; 9.25 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb
to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
> /home/xuankai/anaconda3/envs/4k4d/lib/python3.9/site-packages/torch/_tensor.py(40)wrapped()
38 if has_torch_function(args):
39 return handle_torch_function(wrapped, args, *args, **kwargs)
---> 40 return f(*args, **kwargs)
41 except TypeError:
42 return NotImplemented
(Pdbr)
Environment
GPU: NVIDIA GeForce RTX 3090 (Memory 24GB)
The OOM error happened during pre-computation. How do I lower the batch size ？In addition, I also want to know how to use the 4k4d_0013_01_mb to render？
from 4k4d.
Related Issues (20)
- Are there any plans to release missing files? HOT 2
- CUDA out of memory HOT 2
- About real-time rendering HOT 1
- Is there any plan to add the training code? HOT 2
- Error in opengl HOT 3
- How to convert CamPose.inf to extri.yml? HOT 1
- An error occurred(Segmentation fault) while loading the model HOT 17
- Test view HOT 3
- Question about KPlanes result of DyNeRF dataset? HOT 4
- Questions about the mask loss in the paper HOT 3
- Minimal Dataset extract_images 解帧发绿 HOT 1
- Error during rendering HOT 12
- How to get a 3d object from a new video? HOT 5
- No mask directory in Mobile-stage dataset HOT 7
- EasyVolcap can't import xxx HOT 2
- Get in trouble with pytorch3d 's installing and tiny-cuda-nn 's compiling. HOT 2
- where is the preprocess scripts for DNA-Rendering dataset? HOT 1
- Are there any detailed steps for training our own data? HOT 10
- Bad results than expected. HOT 10
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from 4k4d.