Skip to content

Ordered-concurrently a library for concurrent processing with ordered output in Go. Process work concurrently and returns output in a channel in the order of input. It is useful in concurrently processing items in a queue, and get output in the order provided by the queue.

License

Notifications You must be signed in to change notification settings

tejzpr/ordered-concurrently

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

42 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Tests codecov Go Reference Gitpod ready-to-code Go Report Card

Ordered Concurrently

Parallel processing with ordered output in Go. This module processes work concurrently / in parallel and returns output in a channel in the order of input.

Usage

Get Module

go get github.com/tejzpr/ordered-concurrently

Import Module in your source code

import concurrently "github.com/tejzpr/ordered-concurrently" 

Create a work function

// The work that needs to be performed
func workFn(val interface{}) interface{} {
	time.Sleep(time.Millisecond * time.Duration(rand.Intn(100)))
	return val.(int) * 2
}

Run

func main() {
	max := 100
	// Can be a non blocking channel as well
	inputChan := make(chan *concurrently.OrderedInput)
	wg := &sync.WaitGroup{}

	outChan := concurrently.Process(inputChan, workFn, &concurrently.Options{PoolSize: 10})
	go func() {
		for out := range outChan {
			log.Println(out.Value)
			wg.Done()
		}
	}()

	// Create work and sent to input channel
	// Output will be in the order of input
	for work := 0; work < max; work++ {
		wg.Add(1)
		input := &concurrently.OrderedInput{work}
		inputChan <- input
	}
	close(inputChan)
	wg.Wait()
}

Credits

Thanks to u/justinisrael for inputs on improving resource usage.

About

Ordered-concurrently a library for concurrent processing with ordered output in Go. Process work concurrently and returns output in a channel in the order of input. It is useful in concurrently processing items in a queue, and get output in the order provided by the queue.

Topics

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Contributors 3

  •  
  •  
  •  

Languages