Hi Remoting team. I was wondering if it is possible/makes sense to send the eye tracker from HL2 using a data channel in the custom player. The problem when I try to do this on the remote side (Unity) is that, the MRTK interfaces like EyeGazeProvider seem to be tied to the Unity's rendering frame rate and this causes missing lots of gaze samples depending on the fps variation. So I thought maybe using the UWP API Input.Spatial.SpatialPointerPose.Eyes on the client side can enable obtaining the data at a fixed frame rate 30 Hz regardless of the rendering load of the remote.

Describe the bug
Can not connect the remote player which retrun Disconnected with reason 12(TransportConnectionFailed) and Holographic Remoting: Disconnected - Reason: 12(XR_REMOTING_DISCONNECT_REASON_TRANSPORT_CONNECTION_FAILED_MSFT)
By the way I built the .sln in remote/uwp which can connect the player in holoens2.The another one which in remote/desktop can not.What is the difference between them.

Remote side (your Windows PC):

• OS version: 22000.856
• Are you using an engine (Unity/Unreal), if yes - which one and which version: [Unity 2022.1.1f1c1]
• Are you using OpenXR or Windows Holographic (or the respective plugins for your engine)
• Are you using your own render engine? If so - what version of the remoting SDK are you using? [2.8.0]
• Graphics card(s) / graphics driver version: [nVidia RTX 3060 laptop with driver version 512.95, AMD Radeon(TM) Graphics 30.0.13002.1001]

Hello,
I just want to ask, is there anyway to pick a file on PC while using Holographic Remoting? 
The problem is, the default FileExplorer is not rendered in Unity Scene, but outside of Unity Scene.  Therefore, PC cannot transfer its visual graphic to HoloLens. FileExplorer can only be seen on PC, not on Holographic of HoloLens. So, I can pick a file by using PC only, not on HoloLens itself. Is there any way to overcome this issue?

Hello!
I am new with unity and I have an Hololens 2 but I can not use Playmode. When I hit Play button, Holographic Remoting Player from Hololens shows "receiving" but nothing happens and unity shows just a black screen.

I met this problem both in the project 'remote' and 'player', but I have installed Microsoft.Holographic.Remoting and Microsoft.Windows.CppWinRT NuGet package.

Hello,

I'm not sure if it was best to open this issue here or on the Azure Spatial Anchors repository, so I'll link it here in case it's more related to the remoting library than the ASA library:
Azure/azure-spatial-anchors-samples#381

Basically it seems that the current OpenXR Remoting runtime 2.8.1 is lacking OpenXR extensions for ASA to work properly, and I was under the impression that Azure Spatial Anchors were meant to work with Remoting apps in the latest 2.8.1 version.
If that's the case, will there be some support for this in the future?

The system we had before was to manually download anchors and map files from the HoloLens portal and transfer them to the other HoloLens devices and I was thinking of using the portal REST APIs to automatically do those transfers but I guess using ASA would be much more practical in the long term, or at least having access to the SpatialAnchorTransferManager with the remoting runtime.

By the way the reason for this request is that we want to use the Remoting library not only for previewing and development in Unity Play Mode but also definitively for rendering our content with powerful graphic cards.

I built and ran the remote UWP sample project. It runs and connects to the Holographic Remoting Player (v2.3.1.0). The preview on my machine shows the cube and my hand movements & rays.

However nothing is visible through the HoloLens device. I ran into the same issue when trying to connect through unity. So I'm not sure if this is a bug with the current version of the MRTK or the device. Any help would much be appreciated (including being pointed to a better place to report this issue)

I wanted to run mixed reality capture with opt-in PV camera when the HL2 is running remote player. I inserted the following code into the initialization stage of the app (after the remote player context is initialized):

auto display = winrt::Windows::Graphics::Holographic::HolographicDisplay::GetDefault();
auto mrcViewConfiguration = display.TryGetViewConfiguration(winrt::Windows::Graphics::Holographic::HolographicViewConfigurationKind::PhotoVideoCamera);
if (mrcViewConfiguration != nullptr) {
        mrcViewConfiguration.IsEnabled(true);
}

However, the mrcViewConfiguration keeps being nullptr.
If I inserted this block of code before remote player context is initialized, the app will crash when MRC starts.
What is the correct way to opt-in PV camera MRC when HL2 is running remote player? Is it done on the PC side (the remote app)?

Hi,
we are developing a native windows desktop app using the nuget package Microsoft.Holographic.Remoting 2.8.1
Everything works fine on some pcs but on some others we get the connection error 5 (HandshakeConnectionFailed)
We suspect that the problem could be caused by the PerceptionDevice.dll.
We redist the PerceptionDevice dll with our app but we also found a copy of this dll inside the system32 folder.
There is a note about Version 2.3.1 (October 10, 2020)
Implemented PerceptionDeviceSetCreateFactoryOverride, which ensures that PerceptionDevice.dll shipped with Holographic Remoting doesn't interfere with the version shipped with Windows 10, but there isn't any documentation about that

Ivan

Good day.
I have a remote unity app (openxr) and a player with space marker recognition.

When I pass the position of the marker relative to the main camera of the Hololens 2, it is projected into the unit with some offset. As if an offset is added relative to the eyes and the main camera.

It turns out that the position of the main camera in the player is not equal to the position of the camera in Unity.

I tried using the built-in QR recognition function and it worked fine with the exact overlay in the remote app. But I can't see its implementation.

Here is how to synchronize the system for the player and the remote application. But it is not clear how to do this in Unity and is it possible to do this?

I've just stumbled across these OpenXR extensions and I have a few questions:

• Are they always going to be MSFT-specific or are there plans for making them cross-vendor/cross-runtime?
• Will they become part of the default OpenXR runtime shipped on devices? As far as I understood they live in a separate runtime implementation now, shipped inside a nuget package.

My Test Environment：
1.The remote app( running the .sln project under /remote/desktop)
2.The player(Holographic Remoting Player available in the store)
When running the remote app，the picture in Hololens2 is always frozen and indistinct，about every 5 seconds.
It looks like very unstable，Is there a way out?

I have attempted to get both samples working and have managed to get enough going to attempt a connection between the 2. Now I am using the HoloLens 2 simulator so I had to change the player sample to work for x64 instead of ARM. That is now working and showing the UI although I had to skip over a line of code (line 433 of SamplePlayerMain.cpp).

When I run the UWP host sample it crashes straight away so I tried the desktop version and managed to get something going. There wasn't much in terms of a tutorial so I assume that the host application having a plain white screen and then going to just a plain blue screen isn't how it's meant to go.

So when I hit the space bar in the desktop app it attempts to connect to the HoloLens player app and fails with this error message:
Disconnect: Transport connection was closed due to the requested video codec not being available.

Is there something I can do to get this working? If you need anymore information let me know!

Describe the bug
I run into an issue with trying run 2 holographic remote instances on the same PC.
The first instance connects successfully while the other cant establish the connection and got disconnected immediately. The order doesn’t matters.

My setup:
I have 2 hololens2 headsets and 1 PC,
Each one of the hololens2 runs the latest sample from GitHub - holographic remote player sample with the following configuration (first HoloLens2 with handshake port = 8265 & transport port = 8266, second HoloLens with handshake port = 8269 & transport port = 8270) – runs as listeners.
My PC runs 2 instances of the holographic remote with the related configuration (1 instance with remote port = 8265, 2 instance with remote port = 8269) – runs as clients.

Remote side - Laptop:

• OS version: WIN10 21H2, build 19044
• using Windows Holographic
• Nvidia RTX 3070 with driver version 30.0.15.1236

Player side - HoloLens 2

• OS version: 20348.1513.arm64fre.fe_release_svc_sydney_rel_prod.220802-1135

Do the holographic remote client support multiple instances on same PC?
Do you have any idea what can cause that problem?

Almog

Hi,

I was trying to use different video codec methods to test our network by remoteXR. I noticed there was 4 parameters we could choose from. Could you please tell me if I could transfer the video without compression? And what is the "XR_REMOTING_VIDEO_CODEC_MAX_ENUM_MSFT" method used for?

I want to use Holographic Remoting to stream holographic content to HoloLens 2 in real time. I follow this tutorial to preview my work with Holographic Remoting. But when I click the 'play' button in unity, I get this error in unity

Failed to enable remoting runtime.
UnityEngine.Debug:LogError (object)
Microsoft.MixedReality.OpenXR.Remoting.PlayModeRemotingPlugin:HookGetInstanceProcAddr (intptr) (at Library/PackageCache/com.microsoft.mixedreality.openxr@a4cc3ecd3f67/Runtime/FeaturePlugins/PlayModeRemotingPlugin.cs:77)
UnityEngine.XR.Management.XRGeneralSettings:AttemptInitializeXRSDKOnLoad () (at Library/PackageCache/[email protected]/Runtime/XRGeneralSettings.cs:148)

The Holographic Remoting App in Hololens2 had no response. It seems to be a problem in unity. This figure shows my setting and versions of unity (v2020.3.34), openXR plugin (v1.4..2), Mixed Reality OpenXR Plugin (v1.4.0). How can I solve it? Thanks for your help!

From the Holographic Remoting document, it writes that:

Holographic Remoting works by replacing the Windows Mixed Reality runtime which is part of Windows with a remoting specific runtime. This is done during the creation of the remote context. For that reason any call on any Windows Mixed Reality API before creating the remote context can result in unexpected behavior.

Does it mean that the default locator of the remote side is exactly same as the default locator on HL2, so that every spatial transform on the remote side matches the transform on HL2?

Describe the bug
I got 2 hololens2 headsets and 1 PC and run into an issue with try running 2 holographic remote (Sample remote for desktop) on same PC.

The link between the first pair of hololens2 <=> PC (remote) works well.
The 2nd one start to rendering frames and after about 1 second it throw the following exception:

[External Code]	
Microsoft.Holographic.AppRemoting.dll!00007ff91cb866b4()	Unknown
Microsoft.Holographic.AppRemoting.dll!00007ff91c68b0cf()	Unknown
Microsoft.Holographic.AppRemoting.dll!00007ff91c779f5c()	Unknown
Microsoft.Holographic.AppRemoting.dll!00007ff91c780b5a()	Unknown
SampleRemote.exe!winrt::impl::consume_Windows_Graphics_Holographic_IHolographicCameraPose<winrt::Windows::Graphics::Holographic::IHolographicCameraPose>::HolographicCamera() Line 127	C++
SampleRemote.exe!SampleRemoteApp::Render::__l2::<lambda>(std::map<unsigned int,std::unique_ptr<DXHelper::CameraResourcesD3D11Holographic,std::default_delete<DXHelper::CameraResourcesD3D11Holographic>>,std::less<unsigned int>,std::allocator<std::pair<unsigned int const ,std::unique_ptr<DXHelper::CameraResourcesD3D11Holographic,std::default_delete<DXHelper::CameraResourcesD3D11Holographic>>>>> & cameraResourceMap) Line 534	C++
SampleRemote.exe!DXHelper::DeviceResourcesD3D11Holographic::UseHolographicCameraResources<void <lambda>(std::map<unsigned int,std::unique_ptr<DXHelper::CameraResourcesD3D11Holographic,std::default_delete<DXHelper::CameraResourcesD3D11Holographic>>,std::less<unsigned int>,std::allocator<std::pair<unsigned int const ,std::unique_ptr<DXHelper::CameraResourcesD3D11Holographic,std::default_delete<DXHelper::CameraResourcesD3D11Holographic>>>>> &)>(const SampleRemoteApp::Render::__l2::void <lambda>(std::map<unsigned int,std::unique_ptr<DXHelper::CameraResourcesD3D11Holographic,std::default_delete<DXHelper::CameraResourcesD3D11Holographic>>,std::less<unsigned int>,std::allocator<std::pair<unsigned int const ,std::unique_ptr<DXHelper::CameraResourcesD3D11Holographic,std::default_delete<DXHelper::CameraResourcesD3D11Holographic>>>>> &) & callback) Line 100	C++
SampleRemote.exe!SampleRemoteApp::Render(winrt::Windows::Graphics::Holographic::HolographicFrame holographicFrame) Line 521	C++
SampleRemote.exe!SampleRemoteApp::Tick() Line 99	C++
SampleRemote.exe!wWinMain(HINSTANCE__ * hInstance, HINSTANCE__ * hPrevInstance, wchar_t * lpCmdLine, int nShowCmd) Line 239	C++
[External Code]	

To Reproduce
Steps to reproduce the behavior:

1. Run 1st Hololens2 with store player (v2.8.1.0)
2. Run 1st desktop executable instance and connect the hololens2 IP.
3. Run 2nd Hololens2 with store player (v2.8.1.0)
4. Run 2nd desktop executable instance and connect the hololens2 IP.

Screenshots / videos

Remote side (your Windows PC):

• OS version: WIN10 21H2, build 19044
• NVIDIA RTX 3000

Player side (e.g. your HoloLens 2)

• OS version: 20348.1513.arm64fre.fe_release_svc_sydney_rel_prod.220802-1135
• I am using the remoting player from the store [v2.8.1.0]

Hello,

I have recently added Azure SpatialAnchors support on top of remoting (2.7.1, C++/winrt) to our product.

After starting an Azure session (CloudSpatialAnchorSession::Start), I'm constantly getting:
"Failed to get availability status: RemotingHandlerCore_GetAnchorExportSufficiency_Failed : Remote call request failed".

This actually comes from the Azure callback logs as well as debug output but I already know those errors originate from the remoting library as remoting practically replaces all local interfaces. So I'm posting the issue here.

The error happens every 2-3 seconds and is quickly followed by a CloudSpatialAnchorSession::SessionUpdated callback with all metrics dropping to 0 - which sometimes pick up a few seconds later only to be 0 again after the next error.

Needless to say that having all metrics drop to 0 every two seconds practically renders Azure cloud anchors with remoting useless for us.. you can't properly save anchors until RecommendedForCreateProgress() >= 1.0f.
There is nothing special about the Azure session (4 lines of code), I create the session and call Start() only after successfully connecting to the remote headset.

Since in-between errors everything sorta works (I do get valid metrics for a few seconds before getting the error again and so on) - I was wondering if this is the expected (broken) behavior or I might be missing something?

Otherwise there seems to be a bug where Azure calls GetAnchorExportSufficiency internally and remoting 70% fails the call.

Much appreciated,
Daniel

I'm encountering the following error while building the player sample project:
OperationFailedException: Error HRESULT E_FAIL has been returned from a call to a COM component.
Could you please help fixing this?

Describe the bug
While reading /input/select/value and /input/squeeze/value inputs in a 1 - 1 test between native HoloLens 2 and using Holographic Remoting I get inconsistent values between the setups. While the values are read as a float in the remoting setup the returned values are always either 1 or 0. The value for /input/select/value also seems to be triggered less often then in the native setting. When running natively on a hololens I can get values between 0 and 1 for both inputs which does like the case when remoting.

To Reproduce

1. Enable XR_MSFT_hand_interaction.
2. Read /input/select/value and /input/squeeze/value as float.
3. Visualize output.
4. Repeat without remoting.

Screenshots / videos

Remote side:

• OS version: Windows 10 Pro, 21H2, build 19044.1766
• OpenXR Version 1.0.23
• Microsoft.Holographic.Remoting.OpenXr, version 2.8.0, NuGet Package
• NVIDIA GeForce GTX 1050 Ti Max-Q, driver 512.95 (30.0.15.1295)
• Intel UHD Graphics, driver 27.20.100.8729

Player side:

• HoloLens 2, Windows Holographic for Business, build 20348.1506
• Store Holographic Remoting Player version 2.8.0.0

Describe the bug
I've opened Azure account and enabling the spatial anchors service to build spatially aware mixed reality application.
Clone Azure Spatial Anchors github sample.
Can't run azure spatial anchors with holographic remoting (store edition), probably because lack of dll.

Error:
DllNotFoundException: AzureSpatialAnchors assembly: type: member:(null)
Microsoft.Azure.SpatialAnchors.AnchorLocateCriteria..ctor () (at Library/PackageCache/com.microsoft.azure.spatial-anchors-sdk.core@bd79930953e8/Runtime/Plugins/Standalone/AzureSpatialAnchorsBridge_Standalone.cs:1928)
Microsoft.Azure.SpatialAnchors.Unity.Examples.DemoScriptBase.Start () (at Assets/AzureSpatialAnchors.Examples/Scripts/DemoScriptBase.cs:112)
Microsoft.Azure.SpatialAnchors.Unity.Examples.AzureSpatialAnchorsSharedAnchorDemoScript.Start () (at Assets/AzureSpatialAnchors.Examples/Scripts/AzureSpatialAnchorsSharedAnchorDemoScript.cs:139)

It looks like Azure spatial anchors doesn't have the compatible .dll file for targeting WindowsStandalone build.

Remote side - Laptop:

• OS version: WIN10 21H2, build 19044
• Using Unity 2021.3.9f1 with MRTK holographic remote for play mode
• Nvidia RTX 3070 with driver version 30.0.15.1236

Player side - HoloLens 2

• OS version: 20348.1513.arm64fre.fe_release_svc_sydney_rel_prod.220802-1135
• Store player

Do you familiar with a way to achieve azure spatial anchor to work with holographic remote at all?
Do you familiar with other co-location sharing solution that can work with holographic remote & hololens2?

Almog

In order to facilitate local synchronization of multiple devices without the use of ASA, we would like to use the XR_MSFT_perception_anchor_interop extension to be able to use the SpatialAnchorTransferManager in WinRT to (de-)serialize the spatial anchors we get from OpenXR. It seems however that with holographic remoting, this extension is not available (atleast on my Hololens 2 device).

Is this correct and if so, would it be possible to implement a fix for that?
Thanks in advance.

Actually It's not the application in this hub produces this error. When I try to connect Unity (2019.4.1f1) with Holographic Remoting Player (V2.2.2.0）in Mircosoft Store, It comes up with Disconnect：Transport connection was closed due to the requested video format not being available in Hololens . As for unity , the error is below:
Disconnected with error 22
UnityEditor.EditorApplication:Internal_CallUpdateFunctions()
Any idea to fix that ? Thanks！

I was using UE4.27 before, it works perfect ,but a few days later,when I trys connecting the player,the Log shows :

LogTemp: Editor connecting to remote HoloLens2: 192.168.8.200
LogWmrHmd: WMRInterop: ConnectToRemoteHoloLens trying to connect to HoloLens2 192.168.8.200
LogWmrHmd: WMRInterop: RemotingDisconnectedEvent: Reason: 12 TransportConnectionFailed
LogHoloLensAR: Warning: HoloLens AR session disconnected from peer
LogWmrHmd: WMRInterop: DisconnectFromDevice: Disconnecting from wmr device.
LogWmrHmd: WMRInterop: DestroySpatialAnchorHelper

I don't understand what happend.I didn't do anything , just a few days!!!

Thanks for helping!

Describe the bug
When using Remoting (both audio and video) with the default Remoting Player in the headset, the audio is severely deformed and lags with pauses and ticks, and the pitch is below normal like the sound was slowed down. This happens only when the HoloLens 2 is paired with Bluetooth headphones. With the HoloLens 2 integrated speakers or when the PC itself is paired to the headphones, the audio is normal.

To Reproduce
Steps to reproduce the behavior:

1. Launch remoting player on the HoloLens
2. Connect from a basic OpenXR remoting Unity project
3. Play audio either from the Unity player or Windows with Bluetooth headphones paired to the HoloLens 2

Remote side (your Windows PC):

• OS version: tested on Windows 10 and 11
• Are you using an engine (Unity/Unreal), if yes - which one and which version: Unity 2020.3.42f1
• OpenXR 1.6.0 unity package
• Graphics card(s) / graphics driver version: RTX 2070 driver 526.98
• Unity audio engine: FMOD with FMOD Studio
• Unity platform: UWP
• Tested both in Editor mode and with a built Player

Player side (e.g. your HoloLens 2)

• Windows version 20348.1531.arm64fre.fe_release_svc_sydney_rel_prod.221204-1412
• OpenXR runtime version 112.2211.2002.0
• Remoting player from the store, 2.8.1

I tested with 2 different headphones, I also tested that the pitch was below normal when playing sounds from Windows directly.

Hi, thanks for the great App. I wtite this for that I found that when using the HoloRemoting APP to connect to the computer Unity, the plug-in cannot receive the depth camera data. Do you have any solutions?

I am writing a C++ application using OpenXR and holographic remoting and I have issues using the XrHandTrackerEXT extension. The runtime reports that the extension is supported, the system properties report that the hand tracking is supported, and the code runs without any OpenXR errors. However I always get zero results from xrLocateHandJointsEXT function.

The newest sample used the OverRenderingConfig. What does the 10% over render mean? What is the benefit of adding this?

Hi @NPohl-MSFT, bringing this up again as you've closed #20.

Unfortunately with 2.5.0 I'm still getting similar exceptions on disconnect and close.
It basically crashes the program when calling Disconnect.

If those exceptions are valid and the expected behavior is to disable the exceptions under the debugger (winrt::hresult_error), then it sounds pretty backwards to me.

IMO if everything worked until that point, exceptions on Disconnect/Close should be somehow handled internally or not thrown at all.

I understand the debugger by default breaks on winrt::hresult_error, but after calling Disconnect you can't really do anything about those exceptions and just want the thing to cleanly disconnect without throwing 200 exceptions or having to disable them and miss others :)

Please let me know if you want more exception logs - essentially very similar to #20 .
Much appreciated

This goes all the way to HL1 with remoting package 1.0, but happens quite often on HL2 specifically (multiple devices).

When remoting using the official holographic remoting app (2.3.1-2.4.0, matching package): the longer the app remains open on the device across/between sessions - remote speech recognition will become progressively worse until it doesn't work at all.

This happens even with a simple dictionary like {"hello world", "move", "change", "help"} etc.

C++/winrt Speech recognition code is exactly like in the samples.

Consistently, on the first run everything is recognized (with high confidence).
The longer the holographic remoting app is running and sessions connect/disconnect the worse recognition will get over time. (quiet environment).

There are two main reasons I think it has something to do with the official app:

1. Closing and reopening the holographic remoting app on the device magically fixes speech recognition every time.

2. Even when remote speech recognition doesn't work at all in the remote app, opening the start menu and speaking recognizes everything flawlessly.
   So if the system speech recognition still works great while the remote speech doesn't work at all, it clearly hints at a local issue.

We've been having this issue ever since HL1 but it seems to have gotten worse with HL2 and the new remoting versions.

I was wondering if this was a known issue and if you can please fix this?

Thanks!

Hello!

I am working on an OpenXR/Holographic Remoting backend for an application. While debugging my application all threads are frozen and the player (server) loses connection with the remote as communication has stopped. Are there any suggested strategies to mitigate this? The only "easy" way I can think off is to recreate the entire OpenXR backend, reconnect and refresh all handles.

I see that Unity has no issue with this problem but my guess is that Holographic Remoting/OpenXR code is separated from game code like MRTK for example and attaching the debugger only attaches to the game code and thus does not freeze the communication thread.

Thank you

Hello,

I checked the development pre-requisites, etc., and built the SamplePlayer and SampleRemote (desktop) solutions out-of-the-box. After connecting laptop and HoloLens2 over a local WiFi network, I spin up the HL2 app and launch the remote sample with the IP of the HoloLens as first command line argument. On hitting space bar, the client/server fail to connect. Running the remote end up in the debugger, I see the disconnect event is fired with code 'TransportConnectionFailed'.

First thing I tried was to disable the firewall (server side), but no change in behaviour. Maybe I've missed something obvious? Some tips for troubleshooting would be very useful... Is there a minimum HL2 OS version required, for example?

As an aside, I presume it would be possible to port the C++ Player, or at least the fundamental parts of it, to C#...? My existing HL2 app is written in C# (using SharpDX, etc.), and I'm hoping to integrate remoting, rather than do the port the other way around.

Thanks in advance for your help.

Kind regards,
Philip

Dr Philip J Pratt

Hi remoting team, I wonder if the remote and player has a way to synchronize timestamps? For example, when both sides query their own system relative time, is there a way to calculate the exact difference between them?
Does AppRemoting internally have it implemented, because the player needs to know the time that the remote side generate the images, so the player can re-project precisely?
Thanks!

good day to all,
I have been working with Holographic Remoting Player since I want to see in my HoloLens 2 view what I'm actually seeing on the game view of my unity project, but I'm having some problems with this application : when I connect it to my unity project giving the IP address of the HoloLens the game view of my project changes wrt the way the HoloLens are oriented but on the HoloLens view I can only see the text "Receiving" and not the actual game view. Anyone knows how to solve this problem?

There are important files that Microsoft projects should all have that are not present in this repository. A pull request has been opened to add the missing file(s). When the pr is merged this issue will be closed automatically.

Microsoft teams can learn more about this effort and share feedback within the open source guidance available internally.

Merge this pull request

PlayerContext::BlitRemoteFrame() still return Failed_NoRemoteFrameAvailable，“received frames ”has always been 0，“reused frames” has always been 60，and I can't see any remote renderings in hololens2.
I'm trying to upgrade microsoft.holographic.appremoting to 2.0.18. The problem remains.
But the host program m_remoteContext.OnSendFrame() can call back normally,and I can see everything on the host, including gestures, rays and squares, voice recognition and custom data channel can also be used normally.
How can I troubleshoot ? host or remoting player or network or something else?

Hi,
I need to get the depth camera data through remote rendering, but I encountered difficulties and asked for help.
How to use ResearchModeApi to remotely connect hololens2 player through PC application.

I want to use HolographicRemoting with an Android smartphone tethered to HoloLens2 in order to use our 5G network.
However the server initiates the conntection to the client which causes connection errors as the phone is acting as a second router, and won't forward unsolicited incoming traffic to devices behind it unless you forward ports on it.
I would therefore like to know which ports are used by client and server so I can forward them.

Thank you for your help!

Hi,
I'm trying to figure out how data is transmitted between the sample player and the remote OpenXR app. Like what does the OpenXR runtime use to send the remotely rendered frames? Is there a send queue for that? And how are they received by the sample player side?
I also would like to know if there are some default data channels for sending and receiving data like view and projection transform after the connection is established if I do not custom data channels.

Thank you so much!

Hello, thank you very much for providing such a convenient program. I have a problem now that I cannot align the point cloud (from HoloLens2ForCV) with the real object in Unity. I know that when the program starts, the current position of the device will be the starting position of Unity's main camera. But I'm not sure whether the "current position of the device" refers to the origin of the helmet (LF Camera coordinate system |Rig coordinate system) or other parts? ? ?

I tried to use the origin of the device as the initial position of the camera, and convert the point cloud to the device coordinate system (Rig coordinate system), and then use the point cloud as the local position of the sub-object of the main camera in Unity, but it seems that there is always some deviations.
I also learned that Unity’s coordinate system is different from hololens, such as the difference between the left and right-handed coordinate systems, but it doesn’t seem to be the alignment problem caused by this.

Hello, thank you for providing such a convenient program.
Unfortunately, I now encounter a very strange problem.
At first, I can successfully compile and deploy the project to my Hololens2, and it can run normally.
But now I don't know why, after the deployment is successful, it can't run normally in Hololens2.
The phenomenon is as follows: when you open the application, you can see the initial holographic image. After you click the "Run" triangle button, you can see the rendering of the newly created holographic image space, but after a while, it can't run normally, and it returns to The original square image without any errors.
I tried to change the Win SDK to the recommended version (Windows SDK 10.0.18362.0) in the project. I also tried to change the computer (also the VS2019 community version), reset Hololens2, and even replaced a Hololens2, but it didn't work. .
My computer environment is VS2019 Community Edition
Win SDK is the latest version

I've been trying to integrate ArUco and Vuforia image tracking into the custom player app, however everytime I'm trying to reference a third party dependency I'm receiving the following output while excecuting the program:

I've added my Vuforia dependencies by adding its include directories, lib directories and declaring VuforiaEngine.lib as an additional dependency. After a few seconds my program fails with the exception Exception thrown on 0x00007FF97039D5E8 (VuforiaEngine.dll) in SamplePlayer.exe: 0xC0000005: Access violation when reading in position 0x0000000000000000.

When using ArUco tracking I'm creating a C++/CX Windows Runtime Component within the same solution and adding it to my SamplePlayer by clicking right on References->Add Reference and then selecting HoloLensForCV
The output remains the same and the program crashes after a few seconds with pretty much the same exception Exception thrown on 0x00007FF96E1236D0 (HoloLensForCV.dll) in SamplePlayer.exe: 0xC0000005: Access violation when reading at position 0x0000000000000028. I've tried adding my reference to the Package.appxmanifest in the following way:

<Extension Category="windows.activatableClass.inProcessServer">
     <InProcessServer>
        <Path>HoloLensForCV.dll</Path>
		<ActivatableClass ActivatableClassId="HoloLensForCV.MediaFrameSourceGroup" ThreadingModel="both" />
		...
    </InProcessServer>
</Extension>

However this did not resolve my issue. Could someone please help me? Referencing these dependencies worked just fine in other projects!
As far as I know Holographic Remoting does not support the transmission of PhotoVideo camera frames so this feature needs to be implemented into the player app.

Hi,
I need to access the PV camera of HoloLens to do some vision job with Holographic remoting.
As I know I cannot access the PV camera from the PC side. I would like to know can I access the camera on the player side on the HoloLens?

Thanks.
YL

Thank you for providing such a great project. The code for player works perfect for me. But for the remote app using openXR, I followed every step and installed everything it needed but when I deployed it to the PC, I did not find any space for inputting the IP address of HoloLens. I've attached a pic which shows what it looks like. The same for using the Mixed Reality runtime. I even tried to use the 2.4.0 version of Holographic Remoting OpenXr but I got the same result. Could you please tell me how to fix it? Thanks a lot.

After the Project is retargeted to VS 2022 in the options. (We do not have VS 2019 and are not allowed to install older components in new code dev/builds.)

To Reproduce
Obtain latest code from repository and unzip to a folder.
Open VS 2022 (latest install) and open the remoteOpenXR solution.
Right-click the Project name and choose Properties from the context menu.
Under Configuration Properties/General, from the right pane's General Properties/Platform Toolset dropdown select Visual Studio 2022 to retarget the solution.
Under Configuration Properties/Debugging in the right pane's property Command Line Arguments enter the IP address of your HoloLens 2 device.
Build the app.

This code produces an error:
#ifdef _MSC_VER
template <typename X, typename Y>
constexpr const X& cast(const Y& value) {
static_assert(false, "Undefined cast from Y to type X");
}
#endif

If you change the code for VS 2022 you do not get the error because you have effectively removed the cast template.
//#ifdef _MSC_VER
#if _MSC_VER <= 1929
template <typename X, typename Y>
constexpr const X& cast(const Y& value) {
static_assert(false, "Undefined cast from Y to type X");
}
#endif

In any case this code does not work while the remote WMR version does work.

Screenshots / videos
If applicable, add screenshots or videos to help explain your problem.

Remote side (your Windows PC):

• OS version: [e.g. 21H2, build 22000.675]
• Are you using an engine (Unity/Unreal), if yes - which one and which version: [e.g. Unity 2022.1.3f1]
• Are you using OpenXR or Windows Holographic (or the respective plugins for your engine)
• Are you using your own render engine? If so - what version of the remoting SDK are you using? [e.g. 2.8.0]
• Graphics card(s) / graphics driver version: [e.g. nVidia RTX 3080 with driver version 511.23, also note if you have multiple GPUs in your system - a laptop with integrated and discrete graphics for example]

Player side (e.g. your HoloLens 2)

• OS version
• Are you using the remoting player from the store? If so which version is shown? [e.g. 2.8.0]
• If you use a custom built player and have problems, did you try with the stock store player?

When trying to connect to the HoloLens 2 while graphics debugging is on there is information that:
Disconnect: Transport connection was closed due to the requested video format not being available

It works like that for PreferredVideoCodec::Any and PreferredVideoCodec::H264 and by PreferredVideoCodec::H265 and error SampleRemote.exe: WinRT originate error - 0x80004005 : 'Cannot find an encoder for the requested video codec.'. is thrown.

Tried it with the 2.2.2. sample version.
It was possible to use graphics debugging with the HoloLens 1 and it was a useful feature.

I wonder if there is plan for enabling a C# player/remote sample?

I'm having issues with SpatialAnchorManager on remote sessions.

Using the latest Holographic Remoting player app, C++ nuget package 2.4.0 on winrt, HL2 latest OS version.
(also happens with 2.3.1, not sure about previous packages).

3 out of 4 times SpatialAnchorManager::RequestStoreAsync() never calls the Completed callback and is unavailable for the duration of the session.

SpatialSurfaceObserver::RequestAccessAsync and EyesPose::RequestAccessAsync always work for comparison.

I've tested several methods: calling RequestStoreAsync before connecting a session, on session connect, only when trying to save/load an anchor (after a while) and even polling in a dedicated worker thread.

The results are the same, on most of the sessions the callback never completes until I disconnect the session (causing all blocked callbacks to immediately return with an error).

When it does work and I get a valid SpatialAnchorStore saving does not persist even when SpatialAnchorStore::TrySave() returns true, it only persists until disconnecting.
The next 'valid' session (reconnecting / running the app again) most of the time returns an empty SpatialAnchorStore or none at all.

I did manage to get it to work intermediately by first clearing the anchor store and then saving the anchor 10 times in a row.
Disconnect then throws lots of "RemotingHandlerCore_AnchorStoreRemoveAnchor_Failed" but sometimes the new saved anchor persists. (I make sure to remove the anchor if it already exists before calling TrySave.)

It feels like something is blocked or blocking the data received/sent to SpatialAnchorStore which is only half flushed on Disconnect. Sometime only the SpatialAnchorStore::GetAllSavedAnchors Clear()/Remove persists ignoring the following TrySave.

Other callbacks (such as querying spatial surfaces meshes) work fine all the time.

Here is how I am requesting the store:

storeRequest = winrt::Windows::Perception::Spatial::SpatialAnchorManager::RequestStoreAsync();
storeRequest.Completed([this](winrt::Windows::Foundation::IAsyncOperationwinrt::Windows::Perception::Spatial::SpatialAnchorStore result, auto asyncStatus)
{
// most of the times never called or only called on Disconnect with an error
}

I am on a stable local wifi, remote tracking and creating local anchors (without save/load to store) all work fine.

Any ideas?
Much appreciated.

I get the following error when trying to build (with visual studio Professional)

The imported project "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Auxiliary\Build\14.26\Microsoft.VCToolsVersion.14.26.props" was not found. Confirm that the expression in the Import declaration "C:/Program Files (x86)/Microsoft Visual Studio/2019/Enterprise/VC/Auxiliary/Build/14.26/Microsoft.VCToolsVersion.14.26.props" is correct, and that the file exists on disk.

To workaround, I have to change this in the project file to point at the best approximation I have, which is Microsoft Visual Studio/2019/Professional/VC/Auxiliary/Build/Microsoft.VCToolsVersion.v142.default.props

Can this be set up so that it doesn't require the user to find and fix this?