Is Client-Side Recording, Server-Side Transcription Setup Possible?

[Hu-nie]

The current structure involves the client sending only a flag to initiate recording, while the server handles both recording and transcription.

Is it possible to change this setup so that the client handles recording using PyAudio, sends the audio data over a socket to the server, and then the server only processes the transcription?

[KoljaB]

Yes, that's possible. Please look into the RealtimeSTT_server folder. There's a Readme in there with details.
The stt-server can consume audiochunks and both the stt cli command as well as the AudioToTextRecorderClient class in the RealtimeSTT folder would record with pyaudio and send the chunks from the client to that stt-server.

[Hu-nie]

Oh, thanks! @KoljaB

But it seems that having multiple users connected at the same time is still difficult, right?

[KoljaB]

Yes, that's correct.

[Hu-nie]

I’m trying to develop multi-user connections, but can you tell me why it might not be working?
@KoljaB

[KoljaB]

1. faster_whisper doesn't allow concurrent transcription on a single GPU (it can do that for CPU only and on multiple GPU setups but that's not enough for RealtimeSTT)
2. Realtime transcription takes lots of resources. We'd need to load balance that.

These arent really that hard to solve problems. I didn't have the need personally to make it multi-user ready yet.

I currently offer (paid) consulting services to companies interested in implementing advanced real-time transcription and TTS solutions, including load balancing, multi-user support, and optimizing AI model deployment for resource efficiency.