Minimise latency

[JonnyTech]

What combination of settings would produce the lowest latency? Using Alsa with an external line-in input via roc across a LAN and outputting to a roc receiver and line-out produces a minimum of 100-250 milliseconds of latencey. How can this be improved to something comparable to jackd of about 4ms?

[gavv]

Hi! Good question.

First of all, so far we have not made any serious testing on latencies below 100ms, but this is definitely must be done. The short-term plans are: control protocols, more audio encodings, and then adaptive streaming & investigating and reducing latency. If you want to help with this, you're welcome.

The overall latency is decomposed into the following components:

• on sender:

  • input latency
  • pipeline latency

• on receiver:

  • pipeline latency
  • output latency

The input or output latency (io latency) depends on the audio system being used:

• A typical robust output latency for PulseAudio is about 40ms. In case of PulseAudio, the io latency includes PulseAudio latency, ALSA latency, and the hardware latency, i.e. it is the full latency between Roc and DAC.

• In case of ALSA, Roc tools will employ SoX backend, which doesn't allow to configure the io latency directly. IIRC SoX will suggest ALSA to use the latency of 8 * internal_frame_size / num_channels (but ALSA can choose a higher value). We have a task to implement custom ALSA backend (ALSA sink and source #231) that will allow to make the io latency on ALSA configurable directly.

The pipeline latency, off the top of my head, consists of:

• on sender: a single packet latency, equal to packet_length;

• on receiver: jitter/FEC buffer latency, a.k.a. "session latency"; if FEC is enabled, it should be equal to or greater than number_of_source_packets_in_fec_block * packet_length; if FEC is disabled, it should be equal to or greater than packet_length;

• on sender and receiver: resampler latency, equal to 2 * internal_frame_size / num_channels (if resampling is enabled); the internal frame size can't be smaller than the resampler window size.

There are also two important factors:

• Round trip time. RTT/2 is the lower bound for the session latency (jitter/FEC buffer).

• Pipeline processing time. It is the lower bound for the io latency. The major part of the processing time is resampling, which speed depends on the selected resampler profile.

Here are the command-line options of interest:

• roc-send (https://roc-project.github.io/roc/docs/manuals/roc_send.html)

  • --nbsrc, --nbrpr - Number of source and repair packets per FEC block.
  • --packet-length - Packet size, in time units.
  • --frame-size - Internal frame size, in samples (total for all channels).
  • --resampler-window - Resampler window size, in samples (per channel). Usually it's not configured directly but is determined by --resampler-profile.

• roc-recv (https://roc-project.github.io/roc/docs/manuals/roc_recv.html)

  • --sess-latency - Target session latency, in time units. This is the size of the jitter/FEC buffer.
  • --io-latency - Output (playback) latency, in time units. Currently is only supported for PulseAudio.
  • --frame-size - Internal frame size, in samples (total for all channels).
  • --resampler-window - Resampler window size, in samples (per channel). Usually it's not configured directly but is determined by --resampler-profile.

Most of these options are also available via the C API and PulseAudio modules parameters.

Currently, the defaults are:

• PulseAudio output latency is 60ms
• suggested ALSA output latency is about 17ms on 44100Hz (44100/320/8) (but ALSA will likely choose a higher value depending on hardware)
• session latency is 200ms
• FEC block size is 20x10 packets
• packet length is 7ms
• internal frame size is 640 samples (320 samples per channel)

So, to reduce the latency, you can try this (I'd suggest to do it step-by-step):

• configure sender to use smaller FEC block size or disable FEC if you don't need it;
• configure sender to use smaller packet length;
• after this you will be able to configure receiver to use smaller session latency;
• if you use PulseAudio, configure receiver to use smaller output latency;
• if you use ALSA, configure receiver to use smaller internal frame size so that SoX will suggest ALSA a lower latency;
• configure sender and receiver to use smaller internal frame size if you want to reduce resampler latency.

It would be interesting to see the results of your experiments. BTW, how did you measure the latency?

[kaosbeat]

Hi,
Trying to lower latency as well.
I'm using 0.1.5

following the step-by-step guide:

• What are the default --nbsrc, --nbrpr sizes? 20x10 packets? how do I write this in config style?
• I played with the --packet-length parameter with some success.
• Also lowering the --sess-latency worked quite well, until some glitches occurred.

I tried disabling resampling as this adds latency but...when using. --no-resampling I noticed strange behaviour.
What happens if source is 44100 and destination is 48000, sounded like it plays slower and drops the missing 3900 samples. I guess I should try and sync up the sound cards. But then still will the --no-resampling lead to the problems described here?

Overall a testing guide to testing the network to determine a good setting would be really helpfull. We're trying to set up a system for realtime use using 20 sources/destinations on the same network. Any input is appreciated. iperf style commands to determine the best settings...

Do I understand correctly that with default settings:
(input + pipeline + resampler + output)
7ms + 200ms + 14.7ms + 60ms = ~280ms

In the debug I can see
roc_audio: latency monitor: latency=2152 target=2205 fe=1.00000 trim_fe=1.00000 adj_fe=0.91875
But this monitors the pipeline latency I believe.
(44100/2205 = 1/20th of a second = 50ms = my --sess-latency)

How would I measure the total latency?
I used jack_iodelay before, but this requires jack, and a round trip setup.
best way I can think of is record two analog channels into DAW, one straight from the source, one from the receiving roc-device and compare.

[gavv]

Hi,

What are the default --nbsrc, --nbrpr sizes? 20x10 packets?

Yep.

how do I write this in config style?

What do you mean by "config style"?

What happens if source is 44100 and destination is 48000, sounded like it plays slower and drops the missing 3900 samples.

Yep, we use resampler both to convert from network rate to sound card rate (e.g. 44100 -> 48000) and to compensate clock jitter. If you disable resampling, I guess we're not performing the conversion.

If you're using backend like PulseAudio or ALSA with dmix, it should be able to perform conversion by itself. You can use --rate option to force recording or playback to 44100 (equal to network rate). If sound card rate is different, PulseAudio or dmix will automatically perform resampling in this case.

But then still will the --no-resampling lead to the problems described here?

Yes, so this is not recommended. I'm also thinking about adding an alternative clock drift compensation technique based on dropping/duplicating samples. The quality will suffer, but the latency and CPU usage will improve. This technique works well for speech, but not for music.

Overall a testing guide to testing the network to determine a good setting would be really helpfull. We're trying to set up a system for realtime use using 20 sources/destinations on the same network. Any input is appreciated.

Agree! This is in my TODO. Until it's done, feel free to ask specific questions..

But this monitors the pipeline latency I believe.

Yep.

How would I measure the total latency?

Maybe this tool could help you: https://github.com/gavv/signal-estimator

E.g. you can:

• create ALSA loopback device on sender
• signal-estimator on sender will write to loopback
• roc-send will read from loopback and send samples to receiver
• on receiver, roc-recv will read from network and write to an headphone jack
• you plug audio cable to headphone jack on receiver and connect it to a headphone microphone jack on sender
• signal-estimator will read from microphone jack and measure latency

In other words, you create a bug loop: signal-estimator -> alsa -> roc-send -> network -> roc-recv -> alsa -> audio cable -> alsa -> signal-estimator and measure its latency. You can also exclude roc and network from measurements to get "base" latency of the loop, and then subtract it from the results.

This is the hard way, of course.

I plan to add automatic measurements of the total latency using RTCP in future, but for now there is no builtin way to do it.

[kaosbeat]

What do you mean by "config style"?

how do I pass this on to roc-send? But I figured it out..
I was trying :
roc-send --nbsrc=50x10 -vv -s rtp+rs8m:192.168.1.152:10001 -r rs8m:192.168.1.152:10002

but it should be
roc-send --nbsrc=20 --nbrpr=10 -vv -s rtp+rs8m:192.168.1.152:10001 -r rs8m:192.168.1.152:10002

ready for testing now, just need a tool to measure latency

Maybe this tool could help you: https://github.com/gavv/signal-estimator

looks like you've been doing this before, I'll have a look

[davex25]

latency_examples.zip

Attached are some mp3 output examples of my setup, which consists of a wired connection between a source and sink, with ping times between hosts of .5 ms. Source is using sox alsa roc driver, which receives sounds via pulseaudio monitor of system output. Receiver uses wav output, frame size of 128 at varying session latencies, and resampling is not explicitly disabled (note my filenames have this wrong -- but doesn't make a noticeable difference).

Notice the artifacts when the repair port is not supplied (FEC disabled), happening on both latency examples. Also getting similar overrun counts at 20ms with and without repair port.

Let me know if I've missed something in this setup, or if I can provide any log data that could be helpful. For this test I've built the v0.1.3 tag.

[gavv]

@davex25 Thanks. Could you attach exact commands you've run and roc-recv output with -vv option (verbose logs)? roc-send output will be probably useful too. Also, what hardware (CPU) are you using?