Git Product home page Git Product logo

Comments (9)

darglein avatar darglein commented on July 2, 2024

Are you on the latest commit, with all submodules updated?

Do the other scenes work?

Also please try setting in the GUI:

  1. use_custom_camera
  2. use_gl_tonemapping
  3. set color_flags to 1

image

from adop.

mickey1356 avatar mickey1356 commented on July 2, 2024

Yes, I am on the latest commit with the most updated submodules. I have tried changing those settings in the GUI as well, but the neural view is still a black screen. This is happening for all 5 of the scenes.

from adop.

darglein avatar darglein commented on July 2, 2024

If you set color_flags to 1 do you get weird colors in the debug view or is it also black?

I have committed a patch a few minutes ago. Can you please try this one and send me the console output?

from adop.

mickey1356 avatar mickey1356 commented on July 2, 2024

When I set color_flags to 1, I get weird colors in the debug view.

I have pulled the latest changes, and recompiled the binaries, but I'm still getting the same issues with the neural view. The console output is as follows:

(base) jeremy@jeremy-ubuntu:~/Documents/ADOP$ ./build/bin/adop_viewer --scene_dir scenes/tt_train/
register neural render info
Ref. 
=============== Saiga ===============
| Saiga Version     1.3.2           |
| Eigen Version     3.3.91          |
| Compiler          GNU             |
|   -> Version      9.3.0           |
| Build Type        RelWithDebInfo  |
| Debug             0               |
| Eigen Debug       0               |
| ASAN              0               |
| Asserts           1               |
| Optimizations     0               |
=====================================
Initializing GLFW.
Initializing GLFW sucessfull!
Creating GLFW Window. 1920x1080 Mode=3 Fullscreen=1 Borderless=0
=========================== OpenGL ===========================
| OpenGL Version    3.2.0 NVIDIA 495.29.05                   | 
| GLSL Version      1.50 NVIDIA via Cg compiler              | 
| Renderer          NVIDIA GeForce GTX 1660 SUPER/PCIe/SSE2  | 
| Vendor            NVIDIA Corporation                       | 
==============================================================
[Renderer] Target resized to 1848x1016
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/imgui_gl.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/tone_map.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/tone_map_linear.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/compute/bloom_extract_bright.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/compute/bloom_downsample.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/compute/bloom_upsample.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/compute/bloom_combine_simple.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/compute/copy_image.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/compute/compute_blur.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/compute/compute_blur.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/lighting/light_directional.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/lighting/light_directional.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/lighting/light_point.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/lighting/light_point.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/lighting/light_spot.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/lighting/light_spot.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/lighting/light_point.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/lighting/light_spot.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/lighting/stenciltest.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/post_processing/post_processing.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/lighting/blitDepth.glsl
Deferred Renderer initialized. Render resolution: 1848x1016
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/post_processing/imagedisplay.glsl
Program Initialized!
Loading Scene scenes/tt_train/
====================================
Scene Loaded
  Name       tt_train
  Path       /home/jeremy/Documents/ADOP/scenes/tt_train
  Image Size 1920x1080
  Aspect     1.77778
  K          1159.63 1166.22 960     540     0      
  ocam       1920x1080 affine(1, 0, 0, 0, 0) cam2world() world2cam()
  ocam cut   1
  normalized center 0 0
  dist       -0.0634429   0.0416044    0            0            0            0            -0.000792661 -0.000517484
  Points     11818812
  Colors     1
  Normals    1
  Num Images 301
  Num Cameras 1
====================================
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/asset/ColoredAsset.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/asset/ColoredAsset.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/asset/ColoredAsset.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/asset/LineVertexColoredAsset.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/asset/LineVertexColoredAsset.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/asset/LineVertexColoredAsset.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/tone_map.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/tone_map_linear.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/compute/bloom_extract_bright.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/compute/bloom_downsample.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/compute/bloom_upsample.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/compute/bloom_combine_simple.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/compute/copy_image.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/compute/compute_blur.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/compute/compute_blur.glsl
Found experiment experiments//2021-10-30_09:17:38_multi_scene_pose/ with 1 epochs
Found experiment experiments//2021-10-30_09:14:22_boat/ with 1 epochs
loading checkpoint 2021-10-30_09:17:38_multi_scene_pose -> ep0400
Load Checkpoint render
Total Model Params: 574651
> Reset cuda cache
GPU memory - Point Cloud: 283.651MB
Pinhole Intrinsics:
Tensor [1, 13] float cuda:0 Min/Max -0.0634429 1166.22 Mean 294.294 Sum 3825.82 req-grad 1
GPU memory - Texture: 189.101MB
Load Checkpoint points 11818812 max uv: 11818811
Load Checkpoint texture. Texels: 11818812 Channels: 4
Load Checkpoint pose
First pose before SE3(Quatwxyz(0.984287,-0.0118274,-0.174542,0.0239534),Vec3(0.811265,0.401157,4.70207))
First pose after SE3(Quatwxyz(0.984284,-0.0118084,-0.174564,0.0239311),Vec3(0.811051,0.400861,4.70222))
Load Checkpoint intrinsics
Load Checkpoint environment_map
Load Checkpoint vignette
Load Checkpoint response
Load Checkpoint exposures_values
Using Adam texture optimzier
optimizing texture with lr 0.08/0.004
optimizing environment_map with lr 0.02
optimizing response with lr 0.001
optimizing exposure with lr 0.0005
optimizing vignette with lr 5e-06
optimizing poses with lr 0.005
Setting Neural Render Size to 1920x1080
GPU memory - Texture: 189.101MB
Current Best (img,cam) = (0,0) EV: 0.84082
[Renderer] Target resized to 733x354
[DeferredRenderer] Resize 733x354
loading shader shader//point_render.glsl
loading checkpoint 2021-10-30_09:17:38_multi_scene_pose -> ep0400
Load Checkpoint render
Total Model Params: 574651
> Reset cuda cache
GPU memory - Point Cloud: 283.651MB
Pinhole Intrinsics:
Tensor [1, 13] float cuda:0 Min/Max -0.0634429 1166.22 Mean 294.294 Sum 3825.82 req-grad 1
GPU memory - Texture: 189.101MB
Load Checkpoint points 11818812 max uv: 11818811
Load Checkpoint texture. Texels: 11818812 Channels: 4
Load Checkpoint pose
First pose before SE3(Quatwxyz(0.984284,-0.0118084,-0.174564,0.0239311),Vec3(0.811051,0.400861,4.70222))
First pose after SE3(Quatwxyz(0.984284,-0.0118084,-0.174564,0.0239311),Vec3(0.811051,0.400861,4.70222))
Load Checkpoint intrinsics
Load Checkpoint environment_map
Load Checkpoint vignette
Load Checkpoint response
Load Checkpoint exposures_values
Using Adam texture optimzier
optimizing texture with lr 0.08/0.004
optimizing environment_map with lr 0.02
optimizing response with lr 0.001
optimizing exposure with lr 0.0005
optimizing vignette with lr 5e-06
optimizing poses with lr 0.005
GPU memory - Texture: 189.101MB
Current Best (img,cam) = (0,0) EV: 0.84082
loading checkpoint 2021-10-30_09:17:38_multi_scene_pose -> ep0400
Load Checkpoint render
Total Model Params: 574651
> Reset cuda cache
GPU memory - Point Cloud: 283.651MB
Pinhole Intrinsics:
Tensor [1, 13] float cuda:0 Min/Max -0.0634429 1166.22 Mean 294.294 Sum 3825.82 req-grad 1
GPU memory - Texture: 189.101MB
Load Checkpoint points 11818812 max uv: 11818811
Load Checkpoint texture. Texels: 11818812 Channels: 4
Load Checkpoint pose
First pose before SE3(Quatwxyz(0.984284,-0.0118084,-0.174564,0.0239311),Vec3(0.811051,0.400861,4.70222))
First pose after SE3(Quatwxyz(0.984284,-0.0118084,-0.174564,0.0239311),Vec3(0.811051,0.400861,4.70222))
Load Checkpoint intrinsics
Load Checkpoint environment_map
Load Checkpoint vignette
Load Checkpoint response
Load Checkpoint exposures_values
Using Adam texture optimzier
optimizing texture with lr 0.08/0.004
optimizing environment_map with lr 0.02
optimizing response with lr 0.001
optimizing exposure with lr 0.0005
optimizing vignette with lr 5e-06
optimizing poses with lr 0.005
GPU memory - Texture: 189.101MB
Current Best (img,cam) = (0,0) EV: 0.84082

from adop.

darglein avatar darglein commented on July 2, 2024

Thank you! Let's do it again. Pull the latest commit, build ADOP, run the viewer and send me the console output :D

from adop.

mickey1356 avatar mickey1356 commented on July 2, 2024

Thanks for being so responsive!

Here's the latest console output:

(base) jeremy@jeremy-ubuntu:~/Documents/ADOP$ ./build/bin/adop_viewer --scene_dir scenes/tt_lighthouse/
register neural render info
Git ref: f34052239a9d80ccd94a35a4dd8658a91455555d
Ref. 
=============== Saiga ===============
| Saiga Version     1.3.2           |
| Eigen Version     3.3.91          |
| Compiler          GNU             |
|   -> Version      9.3.0           |
| Build Type        RelWithDebInfo  |
| Debug             0               |
| Eigen Debug       0               |
| ASAN              0               |
| Asserts           1               |
| Optimizations     0               |
=====================================
Initializing GLFW.
Initializing GLFW sucessfull!
Creating GLFW Window. 1920x1080 Mode=3 Fullscreen=1 Borderless=0
=========================== OpenGL ===========================
| OpenGL Version    3.2.0 NVIDIA 495.29.05                   | 
| GLSL Version      1.50 NVIDIA via Cg compiler              | 
| Renderer          NVIDIA GeForce GTX 1660 SUPER/PCIe/SSE2  | 
| Vendor            NVIDIA Corporation                       | 
==============================================================
[Renderer] Target resized to 1848x1016
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/imgui_gl.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/tone_map.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/tone_map_linear.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/compute/bloom_extract_bright.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/compute/bloom_downsample.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/compute/bloom_upsample.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/compute/bloom_combine_simple.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/compute/copy_image.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/compute/compute_blur.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/compute/compute_blur.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/lighting/light_directional.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/lighting/light_directional.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/lighting/light_point.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/lighting/light_point.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/lighting/light_spot.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/lighting/light_spot.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/lighting/light_point.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/lighting/light_spot.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/lighting/stenciltest.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/post_processing/post_processing.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/lighting/blitDepth.glsl
Deferred Renderer initialized. Render resolution: 1848x1016
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/post_processing/imagedisplay.glsl
Program Initialized!
Loading Scene scenes/tt_lighthouse/
====================================
Scene Loaded
  Name       tt_lighthouse
  Path       /home/jeremy/Documents/ADOP/scenes/tt_lighthouse
  Image Size 2048x1080
  Aspect     1.8963
  K          1666.68 1661.18 1024    540     0      
  ocam       2048x1080 affine(1, 0, 0, 0, 0) cam2world() world2cam()
  ocam cut   1
  normalized center 0 0
  dist       -0.126731   0.0217503   0           0           0           0           0.000406465 0.000123456
  Points     12313620
  Colors     1
  Normals    1
  Num Images 309
  Num Cameras 1
====================================
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/asset/ColoredAsset.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/asset/ColoredAsset.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/asset/ColoredAsset.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/asset/LineVertexColoredAsset.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/asset/LineVertexColoredAsset.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/asset/LineVertexColoredAsset.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/tone_map.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/tone_map_linear.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/compute/bloom_extract_bright.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/compute/bloom_downsample.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/compute/bloom_upsample.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/compute/bloom_combine_simple.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/compute/copy_image.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/compute/compute_blur.glsl
loading shader /home/jeremy/Documents/ADOP/External/saiga/shader/compute/compute_blur.glsl
Found experiment experiments//2021-10-30_09:17:38_multi_scene_pose/ with 1 epochs
Found experiment experiments//2021-10-30_09:14:22_boat/ with 1 epochs
loading checkpoint 2021-10-30_09:17:38_multi_scene_pose -> ep0400
Load Checkpoint render
Total Model Params: 574651
> Reset cuda cache
GPU memory - Point Cloud: 295.527MB
Pinhole Intrinsics:
Tensor [1, 13] float cuda:0 Min/Max -0.126731 1666.68 Mean 376.289 Sum 4891.75 req-grad 1
GPU memory - Texture: 197.018MB
Load Checkpoint points 12313620 max uv: 12313619
Load Checkpoint texture. Texels: 12313620 Channels: 4
Load Checkpoint pose
First pose before SE3(Quatwxyz(0.990142,-0.0887136,0.107975,0.00948205),Vec3(0.115149,0.198961,4.53537))
First pose after SE3(Quatwxyz(0.990146,-0.0887041,0.107948,0.00948975),Vec3(0.114891,0.198873,4.53539))
Load Checkpoint intrinsics
Load Checkpoint environment_map
Load Checkpoint vignette
Load Checkpoint response
Load Checkpoint exposures_values
Using Adam texture optimzier
optimizing texture with lr 0.08/0.004
optimizing environment_map with lr 0.02
optimizing response with lr 0.001
optimizing exposure with lr 0.0005
optimizing vignette with lr 5e-06
optimizing poses with lr 0.005
Setting Neural Render Size to 1024x540
GPU memory - Texture: 197.018MB
Setting Debug Output Size to 1024x540
Current Best (img,cam) = (0,0) EV: -0.239136
[Renderer] Target resized to 733x354
[DeferredRenderer] Resize 733x354
loading shader shader//point_render.glsl

from adop.

darglein avatar darglein commented on July 2, 2024

Did you change the render_scale in the dataset parmams of the lighthouse scene? This is fine, but its better to send --render_scale 0.5 to adop_viewer because the checkpoint was trained with a different render scale.

Can you please try changing half_float and channels_last in the viewer GUI?

image

from adop.

mickey1356 avatar mickey1356 commented on July 2, 2024

Oh, disabling half_float worked.

Thank you so much for your help and time! I will close this issue then!

from adop.

darglein avatar darglein commented on July 2, 2024

Thank you too! I'm glad that it works now. I guess this is an issue of older GPUs not supporting half-float operations. A bit strange though that now errors are generated.

from adop.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.