RealSense Support about splatam HOT 15 OPEN

What configs should I use for RealSense D435i?
Could you please provide a config file?

from splatam.

I recommend not using the Replica Config file as your base config. The iPhone config file is more reasonable in terms of learning rate and other parameters.

Unfortunately, we don't currently have a way to look at the reconstruction while SplaTAM is running. That's in the work. We generally use the visualize_tracking_loss config to debug out the tracking. Once tracking works, the rest of the pipeline typically works seamlessly.

from splatam.

Hi @hhcxx2006, it would be great if you could share the data so that we can take a look (if things aren't still working).

Thanks for your interest in a RealSense demo. We will potentially consider this for V2.

from splatam.

And even I changed the picture resolution from 1280 * 720 to 640 * 360. The error is also occured. CUDA is out of memory.

from splatam.

Hi, the error could be due to the following reasons:

1. Could you instead use torch.eye(4).float() for the dummy pose? The GradSLAM dataloader converts all the poses relative to the first frame, where the first frame is identity. We use that identity pose to initialize our first frame pose. This might be causing potential bugs in that step.

2. Is the depth scaling & intrinsics correct?

3. If it's a Tracking Failure:

How many iterations of tracking and mapping are you using? Also, what's the frame rate of the capture, and how much is the sensor moving between each frame?

In the current version, SplaTAM's number of tracking iterations depends on the camera motion between the frames.

An easy way to debug this would be to set the following config to True and see if your tracking is converging to the correct pose:

You can also set the num_frames to a shorter number (10-50) to see if things are working.

from splatam.

Hi, the depth scaling & intrinsics are correct, I am sure. The intrinsics in above picture is not the newest.

I use 40 iterations of tracking and 60 iterations of mapping, just the same as the config file of Replica.

The frame rate is 10 FPS, maybe the frame rate is too low. I will use 30 FPS to have another test. Thank you!

By the way, I wonder if there is a way to visualize the whole process of reconstruction? The words in the terminal are not quite intuitive. Can I visualize the whole process for RealSense data?

from splatam.

Okay!
Thank you for your help! I get it. I will have another try. Thank you!

from splatam.

Hi，I have collected data many times. During the collection process, I move the camera slowly and walk slowly. I use 30 fps to collect the data, but finally the code still shows cuda is out of memory. The 32G memory is used up. What should I do? What's wrong with this?The resolution is 1280 * 720. If num_frames set to a shorter number (10-50), it can run sucessfully, but I need to reconstruct the whole scene. I need the whole data. Thank you!

from splatam.

If I provide external camera pose and do not want to use your tracking method, what should I do?

from splatam.

from splatam.

Thank you for providing the code and for your question. I am using a Realsense D435i camera. Could you please explain how to create a dataset? Also, what modifications are needed for the author's code? I am encountering the following error.Looking forward to your reply.

from splatam.

Also interested in this! Starting to try with Realsense D455

from splatam.

I'm using the config file for replica to test datasets collected by realsense d435.It turns out that the mapping quality is not good enough,especially the edges and corners cannot be reconstructed.(The intrinsics and depth scale are set accurately).Can I get some suggestions on selecting configuration parameters?

from splatam.

If I provide external camera pose and do not want to use your tracking method, what should I do?

Just set use_gt_pose=True, as `configs/replica/splatam.py':

    tracking=dict(
        use_gt_poses=False, # Use GT Poses for Tracking
        forward_prop=True, # Forward Propagate Poses
        num_iters=tracking_iters,
        use_sil_for_loss=True,
        sil_thres=0.99,
        use_l1=True,
        ignore_outlier_depth_loss=False,
        loss_weights=dict(
            im=0.5,
            depth=1.0,
        ),
        lrs=dict(
            means3D=0.0,
            rgb_colors=0.0,
            unnorm_rotations=0.0,
            logit_opacities=0.0,
            log_scales=0.0,
            cam_unnorm_rots=0.0004,
            cam_trans=0.002,
        ),
    ),

But in my experience, even if I use gt_poses, the mapping result isn't good enough. I come up with some reasons for this :

• Tracking trajectory isn't complicated enough. I mean maybe good mapping result need up and down trajectory like replica dataset.
• Gaussians prune method. Maybe the authors finetuned those methods to accommodate those datasets they evaluate at.
• As @Nik-V9 mentioned, learning rate issue.

from splatam.

Hello, have you solved this problem? Even if I set the parameters very low, my computer will still run out of GPU memory.

from splatam.