turnc ERROR Fail to refresh permissions: all retransmissions failed for xxx

[SijmenHuizenga]

Our application that uses the livekit go server-sdk is spitting out a tsunami of Fail to refresh permissions: all retransmissions failed for error messages. At some point hundreds of these messages per hour. Note that this application is handling multiple multiple livekit rooms at the same time, each with 1-5 video streams & 2 audio streams.

The error seems to originate from the pion turn project, which is used by Pion Webrtc. From what I read in the the code of pion turn, it seems this error happens when the turn server does not respond to the turn createpermission request. Since were not seeing any failed to refresh allocation messages, my guess is that somehow the TURN connection is still alive? It's weird.

I cannot see anything weird in the actual video streams. But I'm worried that these errors are a sign of a leftover connection or something that's not cleaned up properly.

[SijmenHuizenga]

This is still an issue. At this time we are seeing this error continuously, about 13000 times per day.

[davidzhao]

sorry we have not had a chance to look into this one. thanks for the heads up.

[SijmenHuizenga]

We are still seeing about 19k of these errors every day.

Could anyone maybe point me into the direction of where to start debugging? I'm happy to add extra logs or do some experimentation. But I have no idea where to even start looking. Guidance would be much appreciated.

[SijmenHuizenga]

We are now at ~43k of these error messages every day. The growth may be related to our grows in number of rooms & tracks.

[kristian1108]

+1, not sure if this is impacting anything, but there are ton of these errors in the logs

[lukmandev]

'+' 1, we facing the same error