Git Product home page Git Product logo

g2ltex's Introduction

G2LTex

This repository contains the implementation of "Texture Mapping for 3D Reconstruction with RGB-D Sensor (CVPR2018)" based on mvs-texturing. Due to the agreement with other company, some parts can only be released in the form of .so files. More information and the paper can be found on our group website and Qingan's homepage.

Publication

If you find this code useful for your research, please cite our work:

Yanping Fu, Qingan Yan, Long Yang, Jie Liao, Chunxia Xiao. Texture Mapping for 3D Reconstruction with RGB-D Sensor. In CVPR. 2018.

@inproceedings{fu2018texture,
  title={Texture Mapping for 3D Reconstruction with RGB-D Sensor},
  author={Fu, Yanping and Yan, Qingan and Yang, Long and Liao, Jie and Xiao, Chunxia},
  booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  pages={4645--4653},
  year={2018},
  organization={IEEE}
}

How to use

1. Run

To test our algorithm. run G2LTex in command line:

./bin/G2LTex [DIR] [PLY] 

Params explanation: -PLY: The reconstructed model for texture mapping. -DIR: The texture image directory, include rgb images, depth images, and camera trajectory.

The parameters of the camera and the system can be set in the config file.

Config/config.yml

How to install and run this code.

git clone https://github.com/fdp0525/G2LTex.git
cd G2LTex/bin
./G2LTex ../Data/bloster/textureimages ../Data/bloster/bloster.ply

We need to modify the configuration file config.yml before running the other datasets.

./G2LTex ../Data/apt0/apt0 ../Data/apt0/apt0.ply

2. Input Format

  • Color frames (color_XX.jpg): RGB, 24-bit, JPG.
  • Depth frames (depth_XX.png): depth (mm), 16-bit, PNG (invalid depth is set to 0).
  • Camera poses (color_XX.cam): world-to-camera [tx, ty, tz, R00, R01, R02, R10, R11, R12, R20, R21, R22].

3. Dependencies

The code has following prerequisites:

  • ubuntu 16.04
  • gcc (5.4.0)
  • OpenCV (2.4.10)
  • Eigen (>3.0)
  • png12
  • jpeg

4. Parameters

All the parameters can be set in the file Config/config.yml as follows:

%YAML:1.0
depth_fx: 540.69
depth_fy: 540.69
depth_cx: 479.75
depth_cy: 269.75
depth_width: 960
depth_height: 540

RGB_fx: 1081.37
RGB_fy: 1081.37
RGB_cx: 959.5
RGB_cy: 539.5
RGB_width: 1920
RGB_height: 1080
.
.
.

5. Results

Some precomputed results can be found in the folder results/.

g2ltex's People

Contributors

fdp0525 avatar yanqingan avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

g2ltex's Issues

hello,I have encountered a problem.

I used Poisson reconstruction for surface reconstruction of point clouds, but when I ran the program using the ply file, it reported the following error.If it completes point cloud loading, it will also report an error during iteration
微信截图_20230406163940
微信截图_20230406165434
Can anyone solve this problem? Thank you very much

question about camera pose file

Thanks for your work of G2LTex.
I am following this work. There is an issue about the data.
Would please tell me what the 12 figures in .cam file means? The first nine figures is the rotation matrix? Or the fourth, eighth, and twelfth figure combine as the translation vector?

I am looking forward for your reply. Thanks for your time.

libG2LTex.so error

I am trying to run the executable exactly as written in the Readme, but i get an error as follows:
./G2LTex: error while loading shared libraries: libG2LTex.so.1: cannot open shared object file: No such file or directory
Can you figure it out?
Thanks

OpenCV Version

Is it possible to run this project with OpenCV version 3. Is there any way to compile the project from scratch with the newer version of OpenCV or do I have to downgrade to OpenCV 2.4 for sure?
Has anybody tried making a build of this project on their own?

compile the code

Is it possible for you provide the additional makefile or cmake configuration for compilation?

Assertion `valid_pixel(p1) && valid_pixel(p2) && valid_pixel(p3)' failed. Aborted

Hi. Thank you for your great work. It's really interesting and helpful.
I am trying to run your program with your test data and I am getting the following error:
num_views:219
982349 total edges.
371352 faces have not been seen by a view.
Saving model... done.
loading face info ...
G2LTex: tex/texture_view.cpp:195: void tex::TextureView::get_face_info(const Vec3f&, const Vec3f&, const Vec3f&, tex::FaceProjectionInfo*, const tex::Settings&) const: Assertion `valid_pixel(p1) && valid_pixel (p2) && valid_pixel(p3)' failed.
Aborted
What should I do to make everything work smoothly?
изображение

The number of channels of depth map

Hi, thanks a lot for your great work! I am trying to run this code on my own recording and encountered the same error mentioned in #16 with

OpenCV Error: Assertion failed (scn == 3 || scn == 4) in cvtColor

I guess this is related to color space conversion in your library. I am wondering whether you enforce the 16bit depth to be saved in some specific shape, e.g. [H, W, C] which must be 3-dimensional even C=1?

Any suggestion will be appreciated! Thanks

Set texture resolution?

Hi, I managed to run G2LTex successfully on my data, but I have a few questions.
I end up with several textured meshes with one or more texture files per mesh, the biggest ones being 1024*1024.

  • Is there a way to generate bigger textures, e.g. 2048*2048? Is it a fixed dimension, or does it depend on input data?
  • Also, given the atlas computation is part of your method, I suppose it's not possible to provide my own texture atlas?
  • My mesh is only partly textured, some small parts are left black (not covered by the atlas maybe?), Do you know what might be causing this?
  • Finally, it is not clear to me what the different output meshes are (result_label, G2LTex, and orgTexSelection). Could you provide some explanation? In my case, orgTexSelection gives the best result.

Thank you for your help.

Invalid CAM File Error

Hi,
I am trying to run the executable exactly as written in the Readme, but i get invalid CAM file error as follows:

$ ./G2LTex ../Data/bloster/textureimages ../Data/bloster/bloster.ply
tok:../Data/bloster/textureimages
tok:../Data/bloster/bloster.ply
PLY Loader: comment VCGLIB generated
Reading PLY: 6972 verts... 12712 faces... done.
Generating texture views:
Invalid CAM file: color_01.cam

Camera poses problem?cam data problem?

Hi,
When our data is generating cam data, in the process of converting the world coordinate system to the camera coordinate system, each frame is converted for the data of the previous frame. How is the author's cam data generated? We use a TOF camera, and the space distance unit is mm. Yours is m. Will the resulting cam file be affected?

  Thank you very much and look forward to your answer.

Can I use this project with windows g++ compiler?

@yanqingan Hi,
Thanks for your excellent work. I don't have a linux system. I wonder if I can use this project with a wondows g++ compiler, like in MinGW. I'm new in this field. Will it have some potential errors I need to keep an eye on or any other suggestions?
Thank you.

Question about .cam file

Thanks for your work of G2LTex.
I am following this work. There is an issue about the data.
Would please tell me what the 12 figures in .cam file means? The first nine figures is the rotation matrix? Or the fourth, eighth, and twelfth figure combine as the translation vector?

I am looking forward for your reply. Thanks for your time.

Failed to open setting file (config.yml)

Hi,
I run the executable with your data without problems. Now I try with my new data of the same format. I also change certain lines in the 'config.yml'. When I run the executable again, I get the following:
"Failed to open settings file at:
depth fx:0 fy:0 cx:0 cy:0 width:0 height:0
RGB fx:0 fy:0 cx:0 cy:0 width:0 height:0
Global depth:0 RGB:0
Local RGB:0 edge:0 Reg:0
Global Iterations:0 Local Iterations:0"

Is there any place missed to change? Can you help me out?

Thanks :)

the result of my data seems very bad

Hi,
I took sfm and stereo matching steps to obtain the camera poses and the depth maps. The poses were calculated by openmvg and the depth maps were calculated by colmap. Some textures look really good but in general the result is bad: obvious texture misalignment, textures inconsistency in boundaries (see below). It seems some camera poses were over-modified and so leaves were mapped to building roofs. I think there are some reasons may explain this:
(1) the SSF reconstruction method you mentioned in paper is not fit for depth generated from image-based stereo method. There are still some noise sources from stereo method not handled in this method. So the reconstruction is very noisy.
(2)The original parameter settings mentioned in your paper were not right in my situation, I need to change them.
Anyway, I wanna know if I can further make this project work well on my situation by trying the above things or any other suggestions.
Thank you very much. Forgive me for I am new in this field.
PS: Here is my result, it did pick some high quality texture image than "let there be color" program.
屏幕截图 2020-09-18 194908

Compatibility with Ubuntu 18

Is the repo executable on Ubuntu 18?
Getting shared library errors on executing the code:
./G2LTex ../Data/bloster/textureimages ../Data/bloster/bloster.ply
Errors:

  1. ./G2LTex: error while loading shared libraries: libopencv_core.so.2.4: cannot open shared object file: No such file or directory
  2. ./G2LTex: error while loading shared libraries: libG2LTex.so.1: cannot open shared object file: No such file or directory
    Is this due to ubuntu version?

suitesparse's version

hi,
Sorry to bother again, but I wonder which version suitesparse you use in this project. I configure suitesparse5.1.2 in my system and get a libcholmod.so.3.0.11. Here the cmd points out I should use libcholmod.so.3.0.6:
image
Thank you~

ply format

Hi,

I have an obj file without any texture information. Does anyone know how to convert it to the ply format which is suitable for this project? I tried using blender and meshlab. However, they both tried to add texture information in the created ply file. I am not sure whether this was the reason that crash the program.
I am getting this error:
Screenshot from 2021-02-13 11-20-15

Optional arguments for the G2LTex executable file

Hello.

I am trying to exercise with models texturing and I would like to test your implementation of the algorithm.
The quality of the textures G2LTex provides seems quite good (given there are 15 texture atlases).
However, I cannot find the description of the optional arguments (and what arguments the program has) to control the texturing pipeline.

Could you please share the information about the optional arguments, please?
It would greatly help me with my research.

Depth-map 16-bit PNG

I want to use my own dataset include the mesh file and origin images, of cource I need the 16-bit depth_map. I want to obtain the depth-map from the dense_point_reconstruction stages and save the depth-image of png format in order to serve as the input of your program. How can I output the proper 16-bit png ? And 16 bit means the max depth is 65536 mm, But there may be larger than the max-thresold ....

libcholmod.so.3.0.6 issue

I have been facing this issue while running your code on Ubuntu 20.04.

./G2LTex: error while loading shared libraries: libcholmod.so.3.0.6: cannot open shared object file: No such file or directory

The solutions discussed in the existing issues doesn't help.

you are using an empty matrix"' failed

Hello,I meeting a problem as follows,I kept all the input formats consistent with the author. I don't know what could be a bug.I am new at it,if you give me some suggesions ,i will be appricate it .
(The example given by the author can run through.)

The command line information is as follows:
``Reading PLY: 854151 verts... 281965 faces...PLY Loader: Ignoring face with 5 vertices!
PLY Loader: Ignoring face with 5 vertices!
PLY Loader: Ignoring face with 5 vertices!
...
Warning: Zero-length normals detected: 15001 face normals, 45812 vertex normals
Generating texture views:
-----------tok ext size:12
depth fx:661.75 fy:661.75 cx:382.39 cy:277.98 width:768 height:576
RGB fx:661.75 fy:661.75 cx:382.39 cy:277.98 width:768 height:576
Global depth:100 RGB:1
Local RGB:1 edge:1 Reg:100
Global Iterations:300 Local Iterations:20
-----------tok ext size:12
-----------tok ext size:12
...
num_views:33
0 total edges.
260863 faces have not been seen by a view.
Saving model... done.
loading face info ...
Global iter:0/300
Global iter:50/300
...
local option iteration:18/20
local option iteration:19/20
----Global Texture Color Correction------
G2LTex: /usr/include/eigen3/Eigen/src/Core/Redux.h:411: typename Eigen::internal::traits::Scalar Eigen::DenseBase::redux(const Func&) const [with BinaryOp = Eigen::internal::scalar_sum_op; Derived = Eigen::Matrix<float, -1, 1>; typename Eigen::internal::traits::Scalar = float]: Assertion `this->rows()>0 && this->cols()>0 && "you are using an empty matrix"' failed.
Aborted (core dumped)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.