Support a channel as a Sink
#270
Labels
acceptance: go ahead
Reviewed, implementation can start
area: result
Improvements in query result reporting
type: feature
New feature or request
Milestone
Is your feature request related to a problem? Please describe.
There is interest in using the engine as a pull-based iterator (#243). While that is hard to achieve, we can have a partial solution of supporting a channel between threads as a
Sink
. A user can then spin the engine in a separate thread and then use the other end of the channel as a source of results from which to pull.It might be beneficial to check if one of those impls should be used in
rq
itself. If the overhead is small, we might default to it on large files where writing the results to output may be a heavy operation. This might also be an option for online output.Describe the solution you'd like
We should have two implementations.
std::sync::mpsc
, the stdlib implementation of a channel.crossbeam_channel
, the most popular channel crate.The other should be behind a feature flag to enable the optional dependency – we won't force users to pay for it if they don't need it.
Describe alternatives you've considered
We could implement
Sink
on other primitives, like crossbeam's ArrayQueue, but that seems too specific. People trying to get a lower-level primitive to work with the engine probably would be better suited by a customSink
impl anyway.The text was updated successfully, but these errors were encountered: