NDI Source Discovery on Android Platform is Unstable

[xuzhaozheng]

Recently, I upgraded to KlakNDI 2.0.1 and found that NDI source discovery on Android platform is still unstable. In the same local network, NDI Studio Monitor on Windows 10 can find NDI streams, but KlakNDI Android apps on Meta Quest 3 takes several minutes for the program to find the NDI source after it is launched, or it may not be found at all.

Platform: Meta Quest 3
System: Android 12
KlakNDI Version: 2.1.1, with NDI Version 6.0.1
Unity Version: 2022.3.31

After some research, I found that this problem was caused by the mDNS service of the Android platform not working properly.

NDI Source Discovery Procedure

NDI's automatic discovery is based on mDNS. According to the NDI document and multicast specification, when an application wants to discover an NDI source, it should first send an IGMP message to join the mDNS multicast group (for example, 224.0.0.251), and then send an mDNS query message in the multicast group to query the _ndi._tcp.local service. The NDI source in the LAN will send an mDNS response message to provide the IP address to the queryer.

I used the Klak URP project with the NDI Sender deleted as an example, built and installed the application in Quest 3 through adb, and it was displayed as an unknown source app in the Meta library. I configured only Quest 3 (192.168.1.70) and NDI encoder (192.168.1.155) in the LAN. Use Wireshark to capture network packets.

Experiment 1: Unity 2022 + Quest 3 + unsigned KlakNDI application

Under the current configuration, I can't find the NDI source in the application. Wireshark shows that the app only sent IGMP messages, but no mDNS messages. Android Logcat has a Warn (yellow one) about the untrusted app's request being denied.

Later, I added a Windows PC host (192.168.1.247) to the network and opened NDI Studio Monitor. When the Windows host sent IGMP and mDNS multicast messages in the network, the KlakNDI app on Quest 3 immediately sent mDNS messages. After the NDI encoder responded, the KlakNDI app found the NDI source. This type of behavior can be reproduced 100%.

Experiment 2: Unity 2022 + Quest 3 + signed KlakNDI application

There are only Quest 3 and NDI encoder in the network. The signed application successfully multicasts mDNS messages and finds the NDI source. Logcat shows that after the Warn, there is an additional Has_NET_CAPABILITY_VALIDATED message. I found that the app can usually find the NDI source after this message appears. However, because I closed and started the KlakNDI app for testing (more than three times), the app made frequent requests in a short period of time, and the Meta Quest system seemed to limit this behavior and no longer prompted NET_CAPABILITY_VALIDATED, which resulted in the inability to find the NDI source. This limit will be reset a few minutes after closing the app.

Experiment 3: Unity 2021 + Quest 3 + unsigned KlakNDI 2.1.0 app

After multiple tests, this version of the app can successfully multicast IGMP and mDNS messages. The information prompted by Logcat is different from the Unity 2022 version. Unity 2021 starts the Android mdnsd service.

Experiment 4: Unity 2022 + OnePlus Android phone + unsigned KlakNDI app

In most cases, this configuration can successfully multicast IGMP and mDNS messages and find the NDI source. Sometimes it takes 50 seconds or even longer for the app to be started before sending the mDNS message, or it may not be sent at all. When the NDI Studio Monitor of another Windows PC host in the LAN is opened, the app on the phone will immediately send the mDNS message and find the NDI source.

Preliminary conclusion

1. The failure to successfully send the mDNS message and get a response is the reason why the NDI source was not found. The mDNS message should be sent actively without other hosts, rather than remaining silent until other hosts send it.
2. Based on Android receiver return black screen #166 and No streams found on Android target API level 32 #190 , mDNS multicast is restricted because Android 12 changes the relevant mDNS API and the security policy of the Meta Quest system.
3. Although the application does not actively send mDNS messages, the sending behavior can be triggered by the mDNS messages of other hosts, so that the NDI source can be found. When conducting relevant tests, make sure that the LAN does not contain other devices and software that may cause interference (for example, additional NDI Studio Monitor). This can easily lead to misjudgment of whether the software function is working properly.

Possible solutions

1. Although the NDI documentation claims that it is no longer necessary to manually initialize Android's nsdManager, re-adding the code to initialize nsdManager and adding the ability to actively request NDI services with mDNS messages may help solve this problem. As mentioned by No streams found on Android target API level 32 #190.( I got trouble using the plugin and ran into build error though)
2. The problem in Quest is that the application installed by adb belongs to an unknown source application of "untrusted". Maybe the application can be made a "trusted" application (I tried software signing and it didn't solve the problem). I don't know how to make it trusted. Maybe only those listed on the Meta Store will work, but not all applications are suitable for joining the Meta Store. At present, there is no way to know whether the trusted application will solve the problem.
3. Return to Unity 2021 and use the old version of KlakNDI to keep the Android Level before 32, but obviously this is not a long-term solution.