Please see the Python notebook for the full README.
AR-Depth is MIT licensed, as found in the LICENSE file.
Fast Depth Densification for Occlusion-Aware Augmented Reality
License: Other
When I run the provided code, I met the error "key frame 7".
I guess this is caused by the omitted? or deleted data of images. (images.txt file starts from 000009.png file instead of 000001.png)
So, I slightly fixed my code and run again the code.
-> Solved the keyframe error.
However, the code couldn't compute the reference frame in this time. There is no difference between kf[0].Position() and kf[1].Position(). So I checked the translation info in the dictionary of views and found that they were all the same. [ 0.17589158 -0.09372958 0.2583495 ]
It seems that the code is not working with the current version.
Hello, there is something about the GetSparseDepthMap function that I don't understand. The following line appears unclear to me: depth_map[int(coord[1]), int(coord[0])] = depth with depth_map = np.zeros((camera.height, camera.width), dtype=np.float32)
This means that every 2D points that have an index superior to the camera image size (here 1080 and 1920) cause an error? Or they are actually not considered by the algorithm?
By the way, in my data, I have like 31000 points, so every points that have index below 31000 cause an error?
Please explain me how the algorithm read the images.txt file, and espacially the line with the 2D points.
The solver initialization, the sparse points, and the TemporalMedian function all need to reproject the depth points from another viewpoint. This component seems to be missing.
If rgb gradients all are positive, maximum get max gradient. But If rgb gradients all are negative, maximum get min gradient.
It seems that the code is not correct.
I am running the code with the sample data that ships with the repo. Speed issues aside, all of the depth outputs I have seen so far (~25 frames) are all the same empty image. Each pixel in the output PNG is [68, 1, 84, 255]
. Furthermore, I have to see the solver converge. Each frame runs 500 iterations and quits.
Am I doing something wrong? The intermediate images all look fine. The sparse depth images are a bit weak though. All I've done is run the code as is.
Here are the input, output and intermediate results for frame 39:
Hi, thanks for your wonderful work!
It requires the camera intrinsics to get the sparse points using SLAM systems. However, in many cases it is very difficult to satisfy this premise. Is there any possible solution if the came intrinsics is unknown?
Thanks.
Hello,
I tried to use your data with dso, but I can not repeat your result. As for me, I save 2d points and depth data of keyframe and project back to get 3d points, which is more than 1 million points. And I check the points you provide, it is only around 60 thousand points. How did you save the 3d points?
I download the whole rep and the sample data. Following the instructions, I run all the cells. However I met a keyerror. PS: I found that the first line in "images.txt" is 000009.png, not 000000.png.
As below:
Number of points: 62786
Number of frames: 691
Processing frame 000009.png
KeyError Traceback (most recent call last)
in ()
5 for frame in recon.ViewIds():
6 print "Processing frame " + recon.views[frame].name
----> 7 kfs = recon.GetReferenceFrames(frame)
8 if (len(kfs) == 0):
9 continue
in GetReferenceFrames(self, view_id)
24
25 def GetReferenceFrames(self, view_id):
---> 26 kf = self.GetNeighboringKeyframes(view_id)
27 dist = np.linalg.norm(self.views[kf[1]].Position() - self.views[kf[0]].Position()) / 2
28 pos = self.views[view_id].Position()
in GetNeighboringKeyframes(self, view_id)
12 next_keyframe = -1
13 for idx in range(view_id - 1, 0, -1):
---> 14 if self.views[idx].IsKeyframe():
15 previous_keyframe = idx
16 break
KeyError: 7
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.