Create layer compsitor for cardboard about cardboard HOT 5 OPEN

Thanks for the feature request. We will add this to our backlog. However, if you would like to add these changes, please read CONTRIBUTING.md and follow the process. Patches are welcome. The right place for adding this feature would be within the CardboardDistortionRenderer module, where a variant for _renderEyeToDisplay would be needed, that takes in a list of layer descriptions, each of them containing type, geometry and a texture. We are not familiar with VDDC and are uncertain if it makes sense to add it as a dependency.

from cardboard.

Hello jballoffet,
Let me first explain what dependencies / modifications would be needed for adding V.D.D.C to Cardboard:

1. The cardboard undistortion already distorts Vertices instead of Texture coordinates. However, its lens distortion model ( respectively the ViewportParams ) are calculated for a Full Screen OpenGL Viewport in contrast to V.D.D.C, which renders left and right eye in 2 different draw calls with a Viewport of Half the screen.
   Therefore, I had to make some changes to lens_distortion
   Most notably adding a new struct for calculating the undistortion ViewportParams in HSNDC ( Half screen normalized device coordinates )

2. V.D.D.C needs a PolynomialRadialInverse to calculate the inverse radial distortion more eficiently. I Have placed all this code in its own folder PolynomialRadialDistortion
   It adds a PolynomialRadialInverse together with the Math for caclulating it.
   Note: The old cardboard java library had the appropriate methods for calculating a PolynomialRadialInverse - I mostly only translated google java code to cpp there )

3. For rendering textured geometry with V.D.D.C I have a simple OpenGL shader that can both sample from a 'normal' and 'external' texture and optionally applies V.D.D.C to the vertices: GLProgramTexture
   You can find more details /helper for V.D.D.C in OpenGL shaders in this file:
   VDDC.h

Additionally, these Files come with some Header-only dependencies, most notably 'glm' for Matrices and some Helper
GLHelper
GLBuffer
Also they are compiled using c++17 ( I think Cardboard still uses c++11 )

Obviosly, Since some of these files have little in common with the original cardboard files ( For example, I could not get the protobuff library to compile-> therefore I decided to create a new struct MVrHeadsetParams that obtains all device / headset parameters from the gvr library )

However, I can help merging the code into cardboard if you decide to add V.D.D.C to either its core sdk or maybe as an extension.

from cardboard.

Benefits of adding V.D.D.C:

As already mentioned above, using V.D.D.C we can add all layer types that are for example already supported on oculus:
unity-ovroverlay
Later, depending on their adoption ( for example I expect a high demand for the Equirect layer aka 360* video layer ) we can add customized shaders for these specific layer types.

I already experimented with creating a VrCompositorRenderer.h
That holds some VRLayer's and draws all of them using V.D.D.C . I also created an example that contains both a 360° video layer and a 2D ui layer :
Renderer360Video

from cardboard.

Sorry for the delay here. Your proposed changes seem reasonable and valuable for the Cardboard SDK and we would like to take this to the PR phase. Taking into the account the magnitude of the required changes, another good option would be to propose a design first before moving forward with the required changes. Please read CONTRIBUTING.md and follow the process. Thanks for your proposal!

from cardboard.

Any update? @Consti10 @jballoffet

from cardboard.