Senso VR glove support for ALVR

[alexeykachan]

Dear community, I'm developer of Senso VR Gloves.

At the moment I'm using "hook" in ovr_context.cpp for getting tracking info and directing it into our software, but it would be wonderful to have it in further official releases.

It works this way : ALVR is running without controllers, but Oculus Quest is sending information about tracking hands anyway, so our software is receiving it, and use as additional information for 3d position, for skeleton of the hand etc.

All we need are few strings of code - send the same into "inputSend(info);" in additional UDP port on the same server_ip. I did it in native code in ovr_context.cpp but it seems it's better to do in in Rust code.

What do you think ?

Goal is - to use Senso VR gloves ( senso.me ) with Oculus Quest + ALVR and do it properly.

[zmerp]

Which version of ALVR are you using? There is no ovr_context.cpp file in the current code. Also, for your use-case, you don't have to interface with ALVR at all. You have to disable controllers and then create a separate driver to send hand tracking data

[alexeykachan]

I'm using 18.2.2 because it's the latest version I was able to compile.
I've tried 19.0.0 but no luck, it has error ........\src\main\cpp\cpp_main.cpp:4:10: fatal error: 'alvr_client_core.h' file not found, and definitely I can't find it anywhere in sources.
Right now it works this way :

• I have a hook inside alvr for oculus quest which send me hand tracking data to the same server IP but different port.
• On the server side of ALVR hands controllers are off, but Oculus client side send it anyway - which is exactly what I need
• On the server side I have another driver which acts as OpenVR driver, hand controller, which collect data from both sources - haptic gloves IMU and from Oculus hand tracking, and provide it into SteamVR.
  So, ALVR hand tracking controllers are off, but instead of them I have all tracking data and our controllers which is what we need.

I would prefer - as you suggest - to have it from server side, because all I need, in fact, to get hand tracking data (which Oculus sends anyway to the server), and duplicate this stream in any form to other UDP port on the same server locally.

I started with client side because I was not sure if Oculus will send hand tracking data even with controllers switched off on server side.
I thought that in this case Oculus client part will not track hands (why they need, right ? save some resources) but I was wrong, it tracks hands anyway and send it on the server, and server just do not provide this data as controllers for SteamVR.

So, I would prefer to have this data on server side from server application.
Do you have suggestions how to get it ?
I can dig into the code, of course, but I don't really know how to do it permanently for further releases of your great software.

All I need is to have hand tracking data as binary UDP stream to the any port of the same server.
I considered just using sniffing like wireshark, but it's not the best practice and no guarantee of changing protocol in future.

And one more additional question - how to make it work with Pico 4 ?

The same client application is not working - obsiously - but I see that it has to work according to compatibility list.
I've tried to use this https://github.com/korejan/ALVR/releases/tag/v0.10.0 - it works with Pico 4 but it doesn't have hand tracking.

[zmerp]

ALVR now switched to OpenXR and support Quest, Focus 3 and Pico 4 headsets. Hand tracking works with Quest and Focus 3, i'm not sure about Pico 4 since i only have a Pico Neo 3 and it's not supported.

The reason of the error is because you didn't execute all commands in the Build from Source guide in the wiki.

For adding a stable API between releases, I cannot provide that unfortunately, it's too much of a hassle. Furthermore i'm using ALVR for another company for my job and i cannot help third parties as per contract.

[alexeykachan]

Thank you for clarification.
In this case I'll try to modify server side, because it seems unnecessary to duplicate stream of data from HMD, even though it's just 100 packets per second.
Frankly speaking, we don't need any real API, just to fork data packets to another destination, I'd consider it rather 'special debug data stream' than any API, but I understand your point.
It's good that hand tracking of Focus is supported, I have Pico 4 but don't have Focus yet to check.

[korejan]

@alexeykachan Hand-tracking is supported but currently disabled for Pico headsets in ALXR because Pico's OpenXR implementation of XR_EXT_hand_tracking is broken, I've made an issue ticket with videos for Pico, they've acknowledged it and will try to fix it hopefully soon. Please ignore the stable release builds for time being, it's quite an old build and very far behind from the nightly builds.

[alexeykachan]

Thank you. I'm working now on using hand tracking data + IMU data + NN to make most reliable and effective finger tracking with all available technologies, so as soon as Pico will have working hand tracking API I could use it as well.