-
Notifications
You must be signed in to change notification settings - Fork 20
Home
Alex Tan Hong Pin edited this page Apr 1, 2018
·
2 revisions
Patterns that can be achieved with go's channel
:
- take - take a subset of items from the channel
- generator - generate a pseudo-random or constant value and return the channel
-
shard/split - split the result from the
in
channel to multipleout
channels, with each channel receiving the same item -
worker - spawn multiple workers that streams data from the
in
channel that process the items concurrently before returning to a singleout
channel - backpressure - when the queue is filled, the channel will be blocked
- rate-limit - throttle the delivery of items to channel by limiting the items per duration, e.g. 100 req/s
- throttle/debounce - there are a few patterns to this, such as limiting the maximum number of in-flight messages (through buffered channels), or by filling a buffered channels and gradually release them at a fixed interval
- to-type - perform type conversion from interface to a determined typed
- repeat - takes an interface and repeatedly send the same item to the channel
- error - pattern for handling error in channel
-
or-done - slightly advance pattern of handling items in a channel, rather than the conventional
for i := range ch
- future - asynchronous processing of item and pending for the item to be returned
- fan-in/multiplex - similar to worker, split the processing across different channels/streams
- fan-out/demultiplex - merge multiple channels into a single channel
- cancellation - pattern to stop all goroutines when the context is cancelled
- deadline - pattern to stop all goroutines when a deadline is achieved
- timeout - pattern to stop all goroutines when a timeout is achieved
-
pause - pattern to stop all goroutine, can be achieved by
sync.Cond
- self-healing - pattern to stop unhealthy goroutines and restarting them
- ring buffer - dropping the messages when the message is full
- retry - retries an operation for a max number of tries, either through constant, linear (the time doubles on every retry), or exponential backoff
- pool - pattern to reuse a set of available objects rather than recreating them
Based on the book, Designing Distributed System.
- Copier takes a single stream of work items and duplicate it out into two or more identical streams.
- Filter reduces the stream of work items to smaller stream of work items by filtering out work items that does not meet specific criteria.
- Splitter evaluates some criteria - just like filter - but instead of eliminating input, the splitter sends different inputs to different queues based on that criteria.
- Sharder divides up a single queue into an evenly divided collection of work items based upon some sharding function.
- Merger is the opposite of copier - it takes two different work queues and turn them into a single work queue.
- the fastest you can process items in an event-driven processing is determined by your slowest channel
- handle errors by retrying them for a certain threshold - if fail, drop them
Look at elixir's OTP implementation on supervisor, and see how to implement them into the go's concurrency pattern, especially the stop-all, stop one etc.