alyssaq / 3dreconstruction Goto Github PK

View Code? Open in Web Editor NEW

433.0 433.0 115.0 656 KB

3D reconstruction, sfm with Python3

Python 100.00%

3d-reconstruction numpy opencv3 python3 sfm

3dreconstruction's Introduction

3D reconstruction

3D reconstruction from 2D images pipeline

Steps:

Detect 2D points
Match 2D points across 2 images
Epipolar geometry
3a. If both intrinsic and extrinsic camera parameters are known, reconstruct with projection matrices.
3b. If only the intrinsic parameters are known, normalize coordinates and calculate the essential matrix.
3c. If neither intrinsic nor extrinsic parameters are known, calculate the fundamental matrix.
With fundamental or essential matrix, assume P1 = [I 0] and calulate parameters of camera 2.
Triangulate knowing that x1 = P1 * X and x2 = P2 * X.
Bundle adjustment to minimize reprojection errors and refine the 3D coordinates.

Note: Steps and code in this repo is my hobby / learning exercise. Ie, its probably not very efficient. If you wish to use a more production-ready library, check out OpenCV's SFM module. I have a docker environment for it at: https://github.com/alyssaq/reconstruction

Prerequisites

Python 3.5+
Install OpenCV: Mac installation steps
pip install -r requirements.txt

Example 3D cube reconstruction

$ python3 cube_reconstruction.py

Example Dino 3D reconstruction from 2D images

Download images from https://www.robots.ox.ac.uk/~vgg/data/mview/ and place into imgs/dinos

$ python3 example.py

Detected points and matched across 2 images.

3D reconstructed dino with essential matrix

3D to 2D Projection

$ python3 camera.py

3D points of model house from Oxford University VGG datasets.

Projected points

Datasets

Oxford University, Visual Geometry Group: http://www.robots.ox.ac.uk/~vgg/data/data-mview.html
EPFL computer vision lab: http://cvlabwww.epfl.ch/data/multiview/knownInternalsMVS.html

References

License

MIT

3dreconstruction's People

Contributors

Stargazers

Watchers

Forkers

bluemyocean baifengbai ai3dvision leisurelysage xshelton hzm8341 gelansheshed avi9839 tuiqdymy sajidullah xmye nikhil4595 konstantinklepikov schliffen mnabilarif keke18532 pandinosaurus jetapiero88 civftor ashprasad nnu-gisa smartcai loushangniao jinyujinyujinyu siddu1998 soooocitrus mm1860 cvclass-fun theforgerass ssgalitsky soubam mrlwh xuhuahaoren keyepoch simakovsp nts-wen sdi1982 nguyenchithien kongweizhe ljthink littlebaibear neumi suniachen yuanyingtao chenbla k7epifanio superchong1987 shaurya950 hejiao610 xiwj coderxlg sxstone etsilvia wenwenwendy ifhams98 yuantao15 freedreamer-crypto gyx97 chazard ahananq jake-zhi berooo faxose weijinghan mathias-mike hanrui56 aali17 walt105 huangsf96 simonggx kasreedhar itayg25 metavai ziyeshanwai purplebutterfly79 kingwingshome mcekwonu rowletqwq gitshohoku patilnabhi wu-byte riyathakore zhanghang1996 solomon-ma bourbon-whiskey lionelwoody sreerag-ibtl rashmi-a-singh arannnn sahil301290 chaohuam hp5564 shanecn lt-xiang zj-z ro360 xiaoqixiaowei jaiaaryan ticianogao huynhtnhut97

3dreconstruction's Issues

Current OpenCV version do not support SIFT

Current OpenCV versions do not support SIFT.

python3 example.py

returns

Traceback (most recent call last): File "example.py", line 90, in <module> points1, points2, intrinsic = dino() File "example.py", line 68, in dino pts1, pts2 = features.find_correspondence_points(img1, img2) File "/content/3Dreconstruction/features.py", line 6, in find_correspondence_points sift = cv2.xfeatures2d.SIFT_create() cv2.error: OpenCV(3.4.3) /io/opencv_contrib/modules/xfeatures2d/src/sift.cpp:1207: error: (-213:The function/feature is not implemented) This algorithm is patented and is excluded in this configuration; Set OPENCV_ENABLE_NONFREE CMake option and rebuild the library in function 'create'

Is it possible to get the real coordinates of the dino?

Since the result of the 3D reconstructed points of dino with essential matrix seems unlikely the real world coordinates, and the steps include assuming P1 = [I 0], I wonder is it possible to get the real coordinates by the code provided here? If not, what else do I require to reconstruct the accurate world coordinates by two images or even more(through sfm)?

NO it(a difference in data points normalization)

in structure.py
def scale_and_translate_points(points): …… x = points[0] y = points[1] center = points.mean(axis=1) # mean of each row cx = x - center[0] # center the points cy = y - center[1] dist = np.sqrt(np.power(cx, 2) + np.power(cy, 2)) scale = np.sqrt(2) / dist.mean()
Here, computation of scale is different from reference http://ece631web.groups.et.byu.net/Lectures/ECEn631%2013%20-%208%20Point%20Algorithm.pdf

I am sorry that I make a mistake

sample image

The image download site in the sample is closed, where else can I download it?

FileMissing

when i ran the example.py,an error ocurred

Traceback (most recent call last):
File "/home/ben/PyProjects/3Dreconstruction/example.py", line 9, in
import features
ImportError: No module named 'features'

it seems like the features.py is missing

True essential matrix: [[matrix([[0.]]) matrix([[-1.]]) matrix([[0.]])]
 [matrix([[-0.361547]]) matrix([[0.]]) matrix([[-3.14154162]])]
 [matrix([[0.]]) matrix([[3.]]) matrix([[0.]])]]
Computed essential matrix: [[-4.85388477e-05 -1.00000000e+00 -4.21762070e-04]
 [-3.62391216e-01 -3.38325541e-02 -3.14887717e+00]
 [ 1.45616543e-04  3.00000000e+00  1.26528621e-03]]
True fundamental matrix: [[matrix([[0.]]) matrix([[-1.11111111e-05]]) matrix([[0.00166667]])]
 [matrix([[-4.01718887e-06]]) matrix([[0.]]) matrix([[-0.00986923]])]
 [matrix([[0.00060258]]) matrix([[0.01166667]]) matrix([[-0.26961594]])]]
Traceback (most recent call last):
  File "cube_reconstruction.py", line 119, in <module>
    print('Computed fundamental matrix:', (F * true_F[2][2]))
  File "/home/zangetsu/proj/prometheus-core/demo/demo-20-3dreconstruction-flame/3Dreconstruction/venv3/lib/python3.6/site-packages/numpy/matrixlib/defmatrix.py", line 226, in __rmul__
    return N.dot(other, self)
ValueError: shapes (3,3) and (1,1) not aligned: 3 (dim 1) != 1 (dim 0)