Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Choppy/Duplicate audio on LLM timeout #1238

Open
martin-purplefish opened this issue Dec 16, 2024 · 0 comments
Open

Choppy/Duplicate audio on LLM timeout #1238

martin-purplefish opened this issue Dec 16, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@martin-purplefish
Copy link
Contributor

martin-purplefish commented Dec 16, 2024

When the LLM times out after having streamed some tokens already, we end up with two audio streams playing at the same time, and all the text getting added to the same message.

It's basically:

  • The LLM starts streaming some tokens
  • Audio starts playing
  • The LLM times out so retry kicks off
  • Original audio keeps playing, and the new audio stream from retry also starts playing

The end result is a single agent message that's very confused (since it's coming from the same LLMStream), and two audio streams playing at the same time, leading to choppy audio that skips around.

In terms of how to fix this.

  1. We should cancel the speech if the LLM times out after some tokens have been streamed instead of retrying.
  2. If the LLM times out after some tokens have been played, treat it as an interruption, and retry the entire call to llm.chat() - including the call to before_llm_cb and before_tts_cb

This is what was really happening in #1077 - see that ticket for a detailed example.

Here's another person encountering it in the slack: https://livekit-users.slack.com/archives/C07FY8WHGPM/p1734179180491739

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant