Guide on exporting scene from COLMAP about adop HOT 22 CLOSED

So, for those who have similar questions, footage processing pipeline:

1. Create new project in COLMAP

2. Start autoreconstruction and go watch film or something

3. run colmap2adop tool:
   ./build/bin/colmap2adop --sparse_dir <sparse/0 in your project folder> --image_dir <your original footage dir> --point_cloud_file <find fused.ply in your project folder> --scale_intrinsics 1 --render_scale 1 --output_path <ADOP dir>/scenes/<your scene name>

4. Note, that you want to set render_scale to <1 to fit images to gpu memory, but value depends on initial images scale and your gpu

5. @darglein could you explain what scale_intrinsics does, please

6. you can train ADOP on your data now!

from adop.

So the .ply point cloud from COLMAP does not have normals?

from adop.

It doesn't

from adop.

This is weird. The dense point cloud from COLMAP does usually have normals. Do you run an old version maybe? Or did you export the sparse point cloud by accident?

from adop.

I installed it literally yesterday from the last release in their github (So don't be surprized about stupid questions)
I did Undistortion, Stereo, Fusion steps in Dense reconstruction window, and chose latest model, so it should be dense model, right?

from adop.

I opened fused.ply and it contains nx, ny, nz values it looks like normals, but previous time I tried to set point_cloud_file in comap2adop to fusion result it went to endless loop

from adop.

Yes fused.ply is the correct file you should use. What do you mean by endless loop? At which point does it get stuck? For large point clouds colmap2adop converter can take a few minutes to complete.

from adop.

At removing duplicate points

from adop.

UPD: fixed it by passing path to original images directory to --image_dir.

I found this in output of colmap2adop:

Num Point3D 68762
Num         = [116]
Min,Max     = [-nan,-nan]
Mean,Median,Rms = [-nan,-nan,-nan]
sdev,var    = [-nan,-nan]
dynamic range: -nan

Is it ok?

from adop.

So, now the problem is that for some reason RemoveDoubles function takes a lot more time than on example scenes.
It also worth noting that my scene is 5 times smaller than yours.

from adop.

I would like to take a look at your fused.ply point cloud and see why preprocessing it takes so long. Can you upload it somewhere and send me the link?

from adop.

UPD: fixed it by passing path to original images directory to --image_dir.

I found this in output of colmap2adop:

Num Point3D 68762
Num         = [116]
Min,Max     = [-nan,-nan]
Mean,Median,Rms = [-nan,-nan,-nan]
sdev,var    = [-nan,-nan]
dynamic range: -nan

Is it ok?

Thanks for pointing that out. This was a bug and is now fixed in the latest commit.

from adop.

Link to my .ply

In meantime, in UnifiedMesh.cpp, in RemoveDoubles function I printed the size of vector, returned from RadiusSearch.
It is equal to 272339 sometimes, so I guess it causes O(n^2) complexity.
https://github.com/darglein/saiga/blob/0d6d033bf88a01ca674c15fb5907d6f5346fa12e/src/saiga/core/model/UnifiedMesh.cpp#L439-L484

from adop.

I made a work-around for this issue. I'd ask you to check that I haven't broken anything 👀

    std::vector<int> to_merge(NumVertices());
    std::vector<bool> processed(NumVertices(), false);

    std::vector<int> to_erase;
    std::vector<int> valid(NumVertices(), 0);
    
    KDTree<3, vec3> tree(position);
    for (int i = 0; i < position.size(); ++i)
    {
        if (processed[i]) {
            continue;
        }

        auto ps     = tree.RadiusSearch(position[i], distance);
        bool found  = false;
        to_merge[i] = i;
        for (auto pi : ps)
        {
            if (valid[pi])
            {
                to_erase.push_back(i);
                to_merge[i] = pi;
                found       = true;
                break;
            }
        }
        if (!found)
        {
            valid[i] = true;
            // all points in radius will find that point i is valid and do the same
            for (auto pi : ps)
            {
                if (!processed[pi] && pi != i) {
                    to_erase.push_back(pi);
                    to_merge[pi] = i;
                    processed[pi] = true;
                }
            }
        }
        processed[i] = true;
    }

It is quite strange that from 1955964 points I get 23659, but there really might be a lot of duplicates

from adop.

It looks like remove doubles is not broken.

The problem is that Assimp is not able to load your point cloud correctly. A simple workaround for now is to open the pointcloud with meshlab and select File -> export mesh -> Select Color, Normal, Binary Encoding -> Ok

After that Assimp is able to read it correctly.

I have added some debug checks in this commit f360201 to ensure the input point cloud is read correctly.

from adop.

Great thanks! It finally works, I managed to start training 😎

from adop.

@DenShlk Thanks for the instructions. I will add this those to the readme once all bugs are fixed.

I have just updated to use the latest saiga+assimp version a1f0219. @DenShlk could you please check if colmap2adop can process the raw COLMAP point cloud now?

(make sure to do a recursive submodule update after the pull)

scale_intrinsics is another way to increase training speed and reduce memory consumption. Let's say COLMAP was run on 2000x1000 images then the intrinsic parameters from colmap fit to this image size. If you set scale_intrinsics=0.5 then the intrinsic parameters will be scaled to fit images of 1000x500. Therefore you also have to downscale the images by a factor of 2 and then pass the downscaled images to colmap2adop. In some cases this is better than render_scale because the DataLoader has to read less bytes from disk and needs to do less processing.

from adop.

Hi! Sorry for delay.

@DenShlk could you please check if colmap2adop can process the raw COLMAP point cloud now?

I checked it, it works perfectly. Thanks!

I'd also asked you to consider using optimization for RemoveDuplicates function I suggested above, because at the moment computing time of it depends on how close are points relatively to distance parameter. (e.g. if you pass distance=1000 so almost all points should be deleted, it will do number_of_points^2 steps).

from adop.

Hi! I try to build my scenes with COLMAP,but when i run colmap2adop,it said "Assertion 'std::filesystem::exists(img_file)' failed!"
but i check the path,the image files are existence，and the path is also correct，how can i solve this problem?

from adop.

The converter did not find images.bin in the sparse directory. Please check if this file exists on your provided path.

from adop.

the images.bin is exsist in the sparse directory,but i copy the image directory and sparse directory from another computer,does this matter ?

from adop.

I find it was caused by a Wrong file organization，thank you ！

from adop.