Viewer should be able to stream in .rrd files whilst they're still being written to

[teh-cmc]

We can log to files in a streaming fashion already today:

import rerun as rr
import time

rr.init("rerun_example_points3d_simple")
rr.save("/tmp/stream.rrd")

for i in range(0, 100000):
    rr.set_time_sequence("frame", i)
    rr.log("points", rr.TextLog(f"frame {i}"))
    time.sleep(0.2)

That's great, but we cannot visualize file recordings in a streaming fashion:

$ rerun /tmp/stream.rrd  # will only show what it is there _right now_

[nikolausWest]

I just tried this with Rerun 0.18.2. The viewer can read from /tmp/stream.rrd while the script logs to it, but it only reads the data that was present when the viewer opened and doesn't continue to read new data as it comes in

[zehiko]

Quick summary of the issue:

• message Decoder that is used as an Iterator when reading the .rrd file is unaware of the fact file may still be written to, hence when it sees EoF from the reader, it concludes there's no more data and the Iterator will return None and reading and decoding stops.

Had a quick chat with Emil on the path forward and we discussed following steps:

1. add retrying for file-based reader (we can either make Decoder aware of the file or we can create a RetryableBuffReader and pass it to the Decoder. Retrying is just dumb loop with sleep and we retry indefinitely.
2. instead of sleep + retry, we rely on notify, so retrying is slightly smarter
3. we add a new message header that signifies "rrd file end" and is written by the FileSink and then we make Decoder aware of the header and we retry if we get EoF from the reader, but we still haven't seen the end file marker. Once we see end of file message / marker, decoder iterator is done. Header can be all-zeros header for example. Retry strategy in this case can be:

• stop after first time you see end of file marker message
• block and read forever
• try to see if there's more data after end of file marker message
  (these are useful for cases when the user concats multiple files together manually)

[zehiko]

Fixed as part of PR #7475.