godotvr / godot_oculus_mobile Goto Github PK

The current grip trigger axis is set to 3.
According to @BastiaanOlij it should be set to 4 to match the value for the Oculus desktop plugin.

Currently the GLES3 renderer seems to be setup to always perform a full frame copy after rendering even when no post processing is enabled (It then just does the Srgb conversion).
Below is a screenshot of a renderdoc trace that shows the frame copy for each eye that happens via a full screen quad.

On tile-based GPU architectures this is very expensive and according to this talk: https://www.youtube.com/watch?v=CQxkE_56xMU&feature=youtu.be&t=2426 it takes 2.5ms on an Oculus Quest (from the total of 13.8ms available for 72hz).

Ideally there would be an option to avoid post processing and the buffer copy completely even for a GLES3 renderer as in most cases it is way too expensive; but to solve this it would probably require change in the GLES3 render architecture.

A sideeffect of this fullscreen copy is that the 4x multisampling set here:

Screenshot of the GLES3 render doc trace (of two cubes rendered at the controller position)

I know this is about the GearVR, but as far as I understand it, Daydream VR should be pretty similar. Is it possible for this code base to support it as well or would it need a different approach?

Thank you in advance!

While the galaxy texture in the demo looks normal in the editor, upon running it on the oculus quest, the texture is squished into the middle beyond recognition.

https://developer.oculus.com/downloads/package/oculus-mobile-sdk/

The permission string for hands tracking access has been updated as well, so updates will need to be made as well to the Godot export logic to match the new string.

The plugin doesn't seem to properly set the eye transforms as the rendered 3D content is not stereoscopic.

While running the demo on the oculus quest, as you look around, there is a noticeable latency, causing motion sickness

I configured my Go to right handed and set the ARVRController to id 2. In this configuration the orientation of the controller is working perfectly but the position is at the origin. No matter where I position the transform of the controller when running on device the controller reverts back to the origin.

My current work-around to to set the position of the ARVRController every frame. It's not pretty but it's will work for the time being:

func _process(delta):
translation = Vector3(0.25,1.25,-0.25)

I am at the moment trying to implement a basic movement vignetting effect using a ColorRect with a custom shader.
When using a 2D Control node like for example a ColorRect on Desktop it will be rendered after the 3D scene into the view. On Oculus Quest it is not visible.

I looked into a Renderdoc trace and it appears that the colorrect is rendered twice as expected but not into the external texture like the 3D draw calls but into a new/separate texture that will not get displayed:

Normal 3D draw calls

2D Draws:

EID 177 and 332 are the ColorRect draws.

I just noticed that there is no check if the ovr mobile context is NULL at

and that only after bool OvrMobileSession::enter_vr_mode() is called (

The missing check leads to the call returning unexpected values; but I think this can be fixed by checking and returning an empty godot_variant (maybe with api->godot_variant_new_nil). Or there is a better way to return errors back to the user?

But this led to slighly more complex issue and there should also be a way for the user to check or know if calls are valid because for example at the moment it is not possible to call the Config API in the GDScript
func _ready():
after initializing the VR interface but only after the first time
void OvrMobileSession::process() is called. Maybe it would be valid to call bool OvrMobileSession::enter_vr_mode()already from the initialize() but I don't know the android application lifecycle.

In addition void OvrMobileSession::exit_vr_mode() is called when the application goes to sleep; when it wakes up again the first time
func _process(delta)
is executed it is again invalid to call the Config classes as the ovr mobile context is still NULL (for the second call it seems to be everything is well intialized again)

It's likely that we're not properly shutting down the godot engine when using Oculus 'Quit' interface. If using Android Studio's logcat, you can see that the app process is still running.

Force quitting the app (using the adb command, or Android Studio) allows the app to properly restart.

Repro steps:

1. Install and launch a Godot app on an Oculus device
2. Exit the app using the controller's home button, and close it by selecting 'Quit' on the Paused prompt.
3. Launch the app again.

Result:
The app will hang while trying to launch (black screen on Oculus Go, and black screen with loading dot on Oculus Quest).

Currently this project uses an Android make file to build, need to investigate either moving that into the root and completing it so the library end up in demo/addons/godot_gearvr or complete the scons build system for Android (my preference)

Hi,
first of all: thanks for the effort you guys put in all of this!

Then: is this supposed to work for Godot 3.2 on Mac OSX?

I can export the VR Starter Tutorial successfully to the Oculus Quest and it starts, I hear the sound but I only see three dots in the headset.

Although it compiles and gets executed on the the Quest, there is an error in the output that says:

 core/math/quick_hull.cpp:401 - Condition "O == E" is true. Continuing.
 Can't open dynamic library: /Applications/Godot.app/Contents/MacOS/../Frameworks/libgodot_openvr.dylib, error: dlopen(/Applications/Godot.app/Contents/MacOS/../Frameworks/libgodot_openvr.dylib, 2): Library not loaded: @loader_path/OpenVR.framework/Versions/A/OpenVR
  Referenced from: /Applications/Godot.app/Contents/Frameworks/libgodot_openvr.dylib
  Reason: image not found.
 modules/gdnative/gdnative.cpp:483 - No valid library handle, can't get symbol from GDNative object
 modules/gdnative/nativescript/nativescript.cpp:1506 - No nativescript_init in "res://addons/godot-openvr/bin/osx/libgodot_openvr.dylib" found

Manually moving libgodot_openvr.dylib to /Applications/Godot.app/Contents/Frameworks/ didn't help either.

What am I missing?
For the record - my machine is still on High Sierra.

Thanks for looking into this.

Best regards,
Dirk

I currently have the issue that when the app is paused and then resumed again that
gdscript _process() is called once before ovr is initialized again (via OvrMobileSession::enter_vr_mode()
This means that for one frame all calls to the gdns api functions are invalid.

Since ovr is recreated on app resume all settings need to be set again as they are initialized with the defaults. I would like to set them for example in a gdscript function that reacts on app resume:

func _notification(what):
	if (what == NOTIFICATION_APP_PAUSED):
		pass;
	if (what == NOTIFICATION_APP_RESUMED):
		set_all_app_settings(); # problem: ovr invalid at this point

A workaround would maybe be to set on NOTIFICATION_APP_RESUMED a counter; wait for >= frame in _process() and then set the settings; but then one frame would be rendered with "wrong" settings; and also all game logic would need to check if accessing gdns vrApi functions is valid.

Another Idea is maybe to call enter_vr for each exposed api function when not initialized?

When using the Time input in a Visual Shader, then the quality of the change in the Time input as well as the quality of textures altered with it deteriorates, making the changes stutter and lowering the resolution of the textures. The project I've provided shows three cubes, two with the same noise texture the right of which has an animated shader using the Time input, and one with a cycling opacity. The quality of the latter two should deteriorate in under 60 seconds.

time_deterioration.zip

OmniLights and SpotLights are invisible on the oculus quest, despite being visible in the editor. Example project attatched.
vr-playground.zip

When using the command line tools (CLT) the user will be confroted with a few problems.

• android SKD CLT doesn't come with cMake
• user has to use sdkmanager to install cmake
• sdk manager doesn't work with JAVA SE >10
• when trying to get Java SE 8 the user has to register at oracle (due to licensing)
• one way may be to use https://jdk.java.net/java-se-ri/10 but no idea how to install

When using Android Studio there are a few missing steps

• user needs to install cmake through sdkmanager
• on windows platform cmake is in C:\Users\ username \AppData\Local\Android\Sdk
• where is the CMakeLists.txt

GearVR requires access to certain platform dependent elements currently only accessible inside of the Godot Android source in the core. For now we've hacked this into GDNative to get things going but we need to implement proper API enhancements into GDNative in the Godot core.

When testing and debugging, I have code paths that are enabled when running on my PC and on my Android tablet, and different code paths that are enabled when running on the Quest. Is there a way to differentiate between the two platforms?

I am attempting to use the high level networking API on my Quest and I am getting this error:

• At: drivers/unix/net_socket_posix.cpp:440:poll() - Condition ' !is_open() ' is true. returned: ERR_UNCONFIGURED
• ERROR: Condition ' !is_open() ' is true. returned: ERR_UNCONFIGURED

I am fairly certain this is due to me running this on the Quest. The game I am running works perfectly fine on my laptop.

After reading this:
https://developer.oculus.com/documentation/platform/latest/concepts/dg-p2p/?locale=en_US

I believe that there may be a special way to make networking connections using the quest that doesn't work with the built in methods already in place.

Am I on the wrong track? Is this something more global or is this problem arising because of the Quest like I am thinking?

If you need anymore information please let me know! Thank you!

There is a perceivable lag in tracking that seems not related to rendering performance.

The easiest reproducer for me is to have a simple unlit cube attached to the ARVRCcontroller and wave it in front of the eye. Then change to oculus home and do the same with the controller; the oculus home controller matches much better the hand movement.
Both applications have swap interval and extra latency mode set to 1.
FPS is constant 72 according to the OVR Metrics tool.

reproducer_tracking_latency.zip

I will try to look into this more deeply but any idea, hint or trick that could help me trace and debug this would be great.

I have two theories at the moment that I would use as starting point:
A) It has to do sth. with the framebuffer swapping and there is maybe some additional buffering along the way to the OVR compositor
B) or it comes from how the tracking positions are handled on the way to godot for rendering that could introduce the visible lag.

https://developer.oculus.com/documentation/native/android/mobile-overlays/

With the godot PR that the readme says was necessary for this asset to work being merged into the master of Godot, and with no issues in this repository remaining, the README should be updated to reflect the current state of this project and remove the <wip> tags

Seems there is an issue with ovrTextureSwapChain :)

jni/ARVRInterface.cpp:396:86: error: subscript of pointer to incomplete type 'ovrTextureSwapChain'
        vrapi_GetTextureSwapChainLength(arvr_data->frameBuffer->colorTextureSwapChain[eye]);
                                        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
../../ovr_sdk_mobile_1.14.0/VrApi/Include/VrApi_Types.h:586:16: note: forward declaration of 'ovrTextureSwapChain'
typedef struct ovrTextureSwapChain ovrTextureSwapChain;
               ^
jni/ARVRInterface.cpp:398:54: error: subscript of pointer to incomplete type 'ovrTextureSwapChain'
        arvr_data->frameBuffer->colorTextureSwapChain[eye], colorTextureSwapChainIndex);
        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
../../ovr_sdk_mobile_1.14.0/VrApi/Include/VrApi_Types.h:586:16: note: forward declaration of 'ovrTextureSwapChain'
typedef struct ovrTextureSwapChain ovrTextureSwapChain;
               ^
jni/ARVRInterface.cpp:429:54: error: subscript of pointer to incomplete type 'ovrTextureSwapChain'
        arvr_data->frameBuffer->colorTextureSwapChain[eye];
        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
../../ovr_sdk_mobile_1.14.0/VrApi/Include/VrApi_Types.h:586:16: note: forward declaration of 'ovrTextureSwapChain'
typedef struct ovrTextureSwapChain ovrTextureSwapChain;

Controller tracking no longer works with the newest firmware update.

With the recent change in godotengine/godot#33444 turning on MSAA will result in fixed foveated rendeirng not working anymore as performance optimization.
It still gives the artifacts of FFR but not the performance benefit as godot first renders the scene to an internal framebuffer with full resolution and then does a full screen quad to display.
This means in addition a costly framebuffer copy is performed (similar to the problem
for GLES3 in #45).

The reason for the change was a fix in how MSAA works on desktop for GLES2 and to align the implementation to how godot handles MSAA.
The issue here for Oculus Mobile is at the moment only for information as it cannot be resolved inside the plugin. There were some potential solutions discussed with @BastiaanOlij but these would either be very hacky (by falling back only for android on the old behaviour for external textures) or require a breaking change in the ARVR api.
Another suggested solution by @akien-mga was to introduce an additional option called something like external MSAA in the render settings and then not use current the godot MSAA setting. This should also disable any post processing and sounds like a good solution to the problem.

Below are some performance measurements of the current master (the current state in the demo scene needs for 4xMSAA ~GPU level 3-4 to run while the old version was around GPU level 2-3):

MSAA turned off:

4x MSAA:

For reference here is a capture of the now reverted (hacky) fix from godotengine/godot@a3ac7a9

Just raising this for discussion purposes and recording info.

For GearVR our interface takes over output to screen from Godot. This means our interface is always used on the primary viewport and we simply provide our render buffers to GearVR.

More info to follow...

In Godot 3.2 Alpha 2, I can:

1. Build regular android apps.
2. Build and run the Oculus Mobile Plugin demo without issues.

In Godot 3.2 Mono Alpha 2, I can:

1. Build regular android apps with C#
2. Build the Oculus Mobile Plugin demo.

However, when I run the Oculus Mobile Plugin demo that was build on the Mono version, it crashes on launch.

I'd appreciate any help trying to track down the cause of this, but because I can build regular Android apps with C#, I think the problem is that there's some incompatibility between this plugin and C#.

I'm on Windows 10. I'd be happy to gather logs but I'm not sure how ... I mean I'm getting some logs but they don't seem to say anything about what's causing the crash, or even that there is a crash.

On an oculus quest, when the DOF toggle is changed from None to 6DOF or 3DOF and 6DOF, the panorama sky is broken, extremely warped in the y axis.

When the Oculus Quest is in Stationary mode (instead of room scale) and then accessing the guardian system via

onready var ovrGuardianSystem = preload("res://addons/godot_ovrmobile/OvrGuardianSystem.gdns").new()

func _ready():
	var arvr_interface = ARVRServer.find_interface("OVRMobile")
	if arvr_interface && arvr_interface.initialize():
		get_viewport().arvr = true

func _process(delta):
	var b = ovrGuardianSystem.get_boundary_oriented_bounding_box();
	print("GetBoundaryOrientedBoundingBox: ", b);

the output is

10-30 22:05:09.427 28836 28854 I godot   : GetBoundaryOrientedBoundingBox: []
10-30 22:05:09.534 28836 28854 I godot   : Game FPS: 1

and then the application crashes.
I would expect the output to be null in this case; but in any case it should not crash; The issue is probably in

Version 12.0 of the Oculus mobile SDK adds support for dynamic foveation, so the feature should be integrated and exposed in the Godot plugin.

https://developer.oculus.com/documentation/quest/latest/concepts/mobile-ffr/

Hello, I was following @BastiaanOlij tutorial, and when compiling the plugin the following problem occurs:

C:\Users\Lobinux-Lenovo\Desktop\Godot_Quest\godot_oculus_mobile\build>cmake -GNinja ..
CMake Error at C:/Users/Lobinux-Lenovo/AppData/Local/Android/Sdk/cmake/3.10.2.4988404/share/cmake-3.10/Modules/CMakeDetermineSystem.cmake:100 (message):
Could not find toolchain file:
C:\Users\Lobinux-Lenovo\AppData\Local\Android\Sdk\ndk/build/cmake/android.toolchain.cmake
Call Stack (most recent call first):
CMakeLists.txt:49 (project)

CMake Error: CMake was unable to find a build program corresponding to "Ninja". CMAKE_MAKE_PROGRAM is not set. You probably need to select a different build tool.
CMake Error: CMAKE_C_COMPILER not set, after EnableLanguage
CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage
-- Configuring incomplete, errors occurred!

and tried to unclog and reinstall Cmake, reviewing the environment variables, but I didn't find any problem, if someone can help me I would be very grateful.

CMakeCache.txt

In the meantime I've made a button testing project https://github.com/creikey/quest-controller-testing

We should add a clang format file into our project just to keep formatting consistent.

Updates the plugin api to notify when the headset is unmounted. This allows the game logic to pause until the user puts the headset back on.

The MSAA settings (Under Project Settings->Rendering->Quality->Msaa seem to have no effect on the actual rendering inside the Oculus Quest in GLES2.

Here is a comparison shot with MSAA off (left) vs MSAA 4x (right):

I tried to investigate briefly; My first suspicion is that in void RasterizerStorageGLES2::render_target_set_external_texture(RID p_render_target, unsigned int p_texture_id): https://github.com/godotengine/godot/blob/1d9233c3882afe888b9396f7f2aac917d4dcac4d/drivers/gles2/rasterizer_storage_gles2.cpp#L5063
no check if msaa is enabled (and no call to glRenderbufferStorageMultisample or glFramebufferTexture2DMultisample is performed) when the external eye target texture is attached to the framebuffer for rendering.

@NeoSpark314 @BastiaanOlij Now that the Android plugin re-architecture has been merged (godotengine/godot#33682 and godotengine/godot#36336), I'll be merging the 2.0 branch into master.

Can you take a look at the 2.0 branch and ensure that the migration to the new workflow won't be an issue.

Thanks!

I'm experimenting with Godot and the Oculus Quest. So far it's working very well, but I noticed that the normal maps on the PC are inverted on the Quest.

Bastiaan Olij - Sounds to me like an issue with how the normal map is converted, really weird.
Best raise an issue on the godotvr github page for the plugin.

Hey Paritosh,

Seeing your phone probably doesn't get this far, this is whats written to the logs:

06-29 20:44:00.538 10210 10227 I godot   : ==============================================================
06-29 20:44:00.538 10210 10227 I VrAppLoader: VrApi Loader Version: 1.1.14.0-0-0 Apr 16 2018 13:10:58
06-29 20:44:00.543 10210 10227 V VrDriver: Loading 32-bit VrDriver...
06-29 20:44:00.548 10210 10227 W System.err: java.lang.RuntimeException: Requested VrApi version (api=1.1.14.0, driver=0) is incompatible with the currently installed version.
06-29 20:44:00.548 10210 10227 W System.err:    at com.oculus.systemdriver.DriverLoader.loadInternal(DriverLoader.java:118)
06-29 20:44:00.548 10210 10227 W System.err:    at com.oculus.systemdriver.DriverLoader.load32(DriverLoader.java:239)
06-29 20:44:00.548 10210 10227 W System.err:    at org.godotengine.godot.GodotLib.step(Native Method)
06-29 20:44:00.548 10210 10227 W System.err:    at org.godotengine.godot.GodotView$Renderer.onDrawFrame(GodotView.java:666)
06-29 20:44:00.548 10210 10227 W System.err:    at android.opengl.GLSurfaceView$GLThread.guardedRun(GLSurfaceView.java:1649)
06-29 20:44:00.548 10210 10227 W System.err:    at android.opengl.GLSurfaceView$GLThread.run(GLSurfaceView.java:1354)
06-29 20:44:00.548 10210 10227 W VrAppLoader: vrapi_Initialize: Failed to load VrApi driver! Java Exception
06-29 20:44:00.548 10210 10227 W System.err: android.util.AndroidRuntimeException: Calling startActivity() from outside of an Activity  context requires the FLAG_ACTIVITY_NEW_TASK flag. Is this really what you want?
06-29 20:44:00.548 10210 10227 W System.err:    at android.app.ContextImpl.startActivity(ContextImpl.java:734)
06-29 20:44:00.548 10210 10227 W System.err:    at android.app.ContextImpl.startActivity(ContextImpl.java:721)
06-29 20:44:00.548 10210 10227 W System.err:    at android.content.ContextWrapper.startActivity(ContextWrapper.java:345)
06-29 20:44:00.548 10210 10227 W System.err:    at org.godotengine.godot.GodotLib.step(Native Method)
06-29 20:44:00.548 10210 10227 W System.err:    at org.godotengine.godot.GodotView$Renderer.onDrawFrame(GodotView.java:666)
06-29 20:44:00.548 10210 10227 W System.err:    at android.opengl.GLSurfaceView$GLThread.guardedRun(GLSurfaceView.java:1649)
06-29 20:44:00.548 10210 10227 W System.err:    at android.opengl.GLSurfaceView$GLThread.run(GLSurfaceView.java:1354)
06-29 20:44:00.548 10210 10227 E VrAppLoader: DisplayErrorAndExit: Failed to launch SystemActivities
06-29 20:44:00.548 10210 10227 I art     : System.exit called, status: 0
06-29 20:44:00.548 10210 10227 I AndroidRuntime: VM exiting with result code 0, cleanup skipped.

So the "================================" I add right before initialising, after that you can see that it calls VrAppLoader and getting feedback about the version, so it's getting pretty far.

It then crashes on an incompatible version error... Now that could very well be because my note 4 is running a pretty old version of gearvr.

All in all, I call that a breakthrough my friend.

When using the ARVRController in GDScript and querying the button state with
is_button_pressed not all buttons from the Oculus Touch Controller work; I tested the following ids that seem not to work even though they are set probably correctly set inside ovr_mobile_controller.cpp:

is_button_pressed(14)  # ovrButton_LThumb / ovrButton_RThumb
is_button_pressed(10)  # ovrTouch_ThumbUp
is_button_pressed(12)  # ovrTouch_IndexPointing

Button 11 is_button_pressed(11) # ovrTouch_IndexTrigger is toggled; but I'm not sure if it is the correct gesture reported for 11 or not.

Inside the oculus_quest_mobile plugin as far as I can tell everything seems to be set correctly; maybe the issue is somewhere in arvr_api->godot_arvr_set_controller_button

when starting the demo application while standing up, the game world's camera is intersecting with the ground, where it should be standing up as well.

With the current master and the demo application I get no lighting in the demo scene only black geometry (emissive material also seems not to work, but the panorama seems to be correctly rendered)

Add a set of api to enable customization of the Oculus mobile runtime parameters:

• cpu/gpu level
• fixed foveated rendering (FFR)
• render target size
• 72 / 60 Hz mode
• swap interval
• extra latency mode

When rendering a point light and a Spatial Material with low roughness values (for example 0.03) the resulting highlight becomes black instead of white.
The left image shows a roughness value slightly bigger; the right shows the issue.

In addition even when the highlight is not black as in the left there are disturbing flickering artifacts around the highlight.

On Desktop OpenGL everything renders as expected. So far I tested this only with SchlickGGX and GLES2; but Blinn seems to have similar problems with the flickering artifacts.

This issue was detected and mentioned originally by a user on discord. He shared the following Quest screenshot to show the problem:

My first guess for the cause of the issue is a floating point precision issue in the executed shader code that maybe leads to a division by zero.
If this is the case I would assume that it can be reproduced also on other Android devices that have a fragment shader float mantissa precision of 10bit for mediump (thats the value for Quest) and is thus not specific to the Oculus Quest.

Drawing to the compositor layer instead of the eye buffer is useful for a couple of things. An important use case is to render text more clearly. They already provide scripts for this in their Unity SDK called OVROverlay.

Version 12.0 of the Oculus mobile SDK adds support for hand tracking, so the feature should be included and exposed in the Godot plugin.

https://developer.oculus.com/documentation/quest/latest/concepts/mobile-vrapi-input-api/#hand-tracking

The demo project included with godot_oculus_mobile sets the physics refresh to 60 Hz, but the headset runs at 72 Hz. This creates noticeable jitter in the headset. Headset tracking appears to be affected by the physics refresh rate. The following warning seems to apply: https://docs.godotengine.org/en/3.1/tutorials/misc/jitter_stutter.html#jitter

For me, the fix is to change physics rate to 72 Hz Globals.set("physics/fixed_fps", 72) or using the GUI in Project Settings. The experience is generally very smooth after changing this rate. However, sometimes I can still reproduce jitter after launching the demo over and over for some reason. I was suspecting an issue with the "Physics Jitter Fix" option, but I'm not exactly sure what this does.

There is also jitter if headset rate can't keep up with the fixed physics rate. From what I understand, Godot may be supporting interpolation in the future that may fix this.

Is it possible for godot_oculus_mobile to automatically sync the physics rate with headset rate? Otherwise, maybe this should be mentioned in setup docs if it is an important consideration. Are there any other settings (such as Physics Jitter Fix) that might improve jitter? Is it possible to add interpolation that might improve headset tracking when physics rate is different from headset rate?

This is an alpha plugin with an alpha version of Godot, so it is entirely possible I'm just seeing issues related to this being alpha quality software.

EDIT: Edited this a few times to try to make it clear.

The Oculus mobile SDK powers the GearVR, Oculus Go, and upcoming Oculus Quest headsets.
As such, it'd be more appropriate to rename this repo to reflect that fact. The current name might confuse users and contributors as well.