Android grpc with UDS is slow to connect sometimes

[Rawa]

Hello!

We are using gRPC with Unix Domain Socket (UDS) on android. We are using waitForReady() and are bring our gRPC server up when app is active. However, sometimes when opening the app the connection is slow. It seems like some back off is applied unnecessarily. When enabling logging we've come to see the messages below. Connecting can take anywhere from a few 100 ms to 30+ seconds, but usually 0-3 seconds.

We've tried using the channel.resetConnectBackoff() before trying to connect again, and also channel.enterIdle() before exiting the app w/o any improvement.

What version of gRPC-Java are you using?

Latest (1.65.1)

What is your environment?

Android

What did you expect to see?

Connecting and then Ready state.

What did you see instead?

Connecting, sometimes TransientFailure and then Ready.

We also see the following message, sometimes multiple times (=reconnecting with backoff?).

[Channel<1>: (dns:///localhost)] Failed to resolve name. status=Status{code=UNAVAILABLE, description=Unable to resolve host localhost, cause=java.lang.RuntimeException: java.net.UnknownHostException: Unable to resolve host "localhost": No address associated with hostname
	at io.grpc.internal.DnsNameResolver.resolveAddresses(DnsNameResolver.java:223)
	at io.grpc.internal.DnsNameResolver.doResolve(DnsNameResolver.java:282)
	at io.grpc.internal.DnsNameResolver$Resolve.run(DnsNameResolver.java:318)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:644)
	at java.lang.Thread.run(Thread.java:1012)
Caused by: java.net.UnknownHostException: Unable to resolve host "localhost": No address associated with hostname
	at java.net.Inet6AddressImpl.lookupHostByName(Inet6AddressImpl.java:156)
	at java.net.Inet6AddressImpl.lookupAllHostAddr(Inet6AddressImpl.java:103)
	at java.net.InetAddress.getAllByName(InetAddress.java:1152)
	at io.grpc.internal.DnsNameResolver$JdkAddressResolver.resolveAddress(DnsNameResolver.java:632)
	at io.grpc.internal.DnsNameResolver.resolveAddresses(DnsNameResolver.java:219)
	at io.grpc.internal.DnsNameResolver.doResolve(DnsNameResolver.java:282) 
	at io.grpc.internal.DnsNameResolver$Resolve.run(DnsNameResolver.java:318) 
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:644) 
	at java.lang.Thread.run(Thread.java:1012) 
Caused by: android.system.GaiException: android_getaddrinfo failed: EAI_NODATA (No address associated with hostname)
	at libcore.io.Linux.android_getaddrinfo(Native Method)
	at libcore.io.ForwardingOs.android_getaddrinfo(ForwardingOs.java:133)
	at libcore.io.BlockGuardOs.android_getaddrinfo(BlockGuardOs.java:222)
	at libcore.io.ForwardingOs.android_getaddrinfo(ForwardingOs.java:133)
	at java.net.Inet6AddressImpl.lookupHostByName(Inet6AddressImpl.java:135)
	at java.net.Inet6AddressImpl.lookupAllHostAddr(Inet6AddressImpl.java:103) 
	at java.net.InetAddress.getAllByName(InetAddress.java:1152) 
	at io.grpc.internal.DnsNameResolver$JdkAddressResolver.resolveAddress(DnsNameResolver.java:632) 
	at io.grpc.internal.DnsNameResolver.resolveAddresses(DnsNameResolver.java:219) 
	at io.grpc.internal.DnsNameResolver.doResolve(DnsNameResolver.java:282) 
	at io.grpc.internal.DnsNameResolver$Resolve.run(DnsNameResolver.java:318) 
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:644) 
	at java.lang.Thread.run(Thread.java:1012) 
}

Steps to reproduce the bug

Setup gRPC with Android using Unix Domain Socket and then open/close app.

[Rawa]

I believe we were able to resolve this issue by providing our own dummy DnsResolverFactory, seems a bit weird that the ManagedChannelImpl does a DNS request when using UDS socket, it should not be necessary.

[Rawa]

Our workaround for this issue: mullvad/mullvadvpn-app#6567

[Rawa]

We believe this issue might be because the default UdsChannelBuilder returns a url with a dns scheme:a instead of unix which results in the wrong name resolver being used.

[ejona86]

Localhost isn't resolved locally on Android? That's suspicious and surprising. But we could use dns:///127.0.0.1 or dns:///%5B::1%5D.

[Rawa]

@ejona86 Well it is kind of weird that a DNS request would happen at all right? We are using UDS, there is no DNS to resolve.

[ejona86]

The current approach is a hack until the unix address negotiation is implemented. Yes, long-term there would be a unix name resolver, and it'd return unix addresses, and that'd select the okhttp unix-supporting transport, and there'd be no DNS. Right now the result from the name resolver is thrown away. Doing "DNS" for "localhost" was supposed to be a local lookup that did no actual I/O; it calls get getaddrinfo but libc was not expected to do DNS.