Optimierung von Augmented Reality Anwendungen durch die Berücksichtigung von Tiefeninformationen mit Googles Project Tango

Optimization of AR applications considering the depth information with Googles project tango

• Final Prototoype
• Augmented Reality Demo
• Pointcloud Extraction
• Plane based Reconstruction
• PCL based native Reconstruction (Greedy Projection Triangulation)
• CHISEL based native reconstruction
• Unity Marching Cubes AR Demo
• OpenCV Depthmap Generation and Filtering
• OpenCV Depthmap Filtering for AR

• depthmap rendering with 3 different reconstruction mechanisms
  • pointcloud projection
  • tsdf reconstruction
  • plane reconstruction
• using depthmap as Z-Buffer (GL_DEPTH_COMPONENT) for augmented reality occlustion

• optional OpenCV guided filtering with settings possible

• AR Camera matching intrinsics
• motion tracking with correct gl positioning
• visualization of the current pointcloud
• ray-intersection for pointcloud interaction
• simple tower defense game with ray intersection
• simple direct pointcloud z-buffer occlusion

• collects points into a large pointcloud using an octree
• exports to pts file
• plane based reconstruction applicable
• marching cube reconstruction applicable (usualy to slow here!)

1. OctTree clustering of the global pointcloud
2. on incoming depth frame
	* update random n global clusters with points from depth frame
	* use RANSAC to detect 3 planes per cluster
	* project points from 3D space to 2D space (based on detected planes)
	* use Graham Scan to compute comvex hull for each plane
	* use Sweep‐line for triangulation with poly2tri
	* project polygon vertices back to 3D space (based on detected planes)

Video Demonstration

Old Screenshots

• VoxelGrid downsampling
• Greedy Triangulation with PCL
• Build PCL Library not included!

• online reconstruction using open_chisel library
• transformation and interfaces to native environment
• PLY exporter
• Build open_chisel Library not included!

• generate depthmap using this approach
• applying inpaint and guided filtering for depthmap improvements
• Build opencv+contrib Library not included!

Video Demonstration

• using the previous approach for a pointcloud reprojection

• Reconstruction using the MeshBuilder with an simple TSDF by Google
• Implementation of a Clipping Depth Shader
• simple ball moving interaction
• extension of shader for shadowing effects

Video Demonstration

Old Screenshots

• Rajawali
• tango-examples-java
• material-dialogs
• EventBus
• commons-math3
• jama
• Poly2Tri
• PCL
• Unity Engine
• OpenCV
• tango-examples-unity