Synchronizing Coordinate Systems in Holographic Remoting

[xyq3q]

Hi,

Your work is truly commendable, and I appreciate your efforts.

I have integrated hl2ss into the Holographic Remoting Sample Player and am using Unity for remote rendering. I am curious about how to synchronize Unity's coordinate system with that of hl2ss. I am aware of using UpdateCoordinateSystem() in UWP applications, but I'm unsure how to implement this in Holographic Remoting. I look forward to your guidance on this matter.

Best regards

[jdibenes]

Hi,

I cannot test this right now so I'm not sure it will work but:

1. Uncomment #define ENABLE_USER_COORDINATE_SYSTEM_SAMPLE in SamplePlayerMain.h
2. In SamplePlayerMain.cpp, in SamplePlayerMain::Initialize (lines 433-434) change m_userSpatialFrameOfReference = m_spatialLocator.CreateStationaryFrameOfReferenceAtCurrentLocation(float3(0.5f, 0.0f, -2.0f), quaternion(0, 0, 0, 1), 0.0); to m_userSpatialFrameOfReference = m_spatialLocator.CreateStationaryFrameOfReferenceAtCurrentLocation();
3. In SamplePlayerMain.cpp in SamplePlayerMain::SetWindow after InitializeStreams (hl2ss) add OverrideWorldCoordinateSystem(winrt::get_abi(m_userSpatialFrameOfReference.CoordinateSystem()))

This is based on https://learn.microsoft.com/en-us/windows/mixed-reality/develop/native/holographic-remoting-coordinate-system-synchronization-wmr
I'm hoping Unity internally calls m_remoteContext.GetUserSpatialFrameOfReference(); (or equivalent) when remoting, but I don't know if that's the case.