Create Multiple Supervised Tasks Using Elixir OTP

This is a short example of how to start concurrent workers to encode videos in this app.

Create a supervisor

Our tasks supervisor will create 12 supervised processes every time. We can change this number by passing the number to the args when we create it.

defmodule App.TasksSupervisor do
  use Supervisor

  def start_link(opts) do
    Supervisor.start_link(__MODULE__, opts)
  end

  def init(opts) do
    # Create 12 tasks
    children = for i <- 1..12 do
      # You have to add the id: i
      # The args are optionals
      args = [task_number: i]
      worker(App.SimpleTask,
        [Enum.concat(opts, args)], restart: :transient, id: i)
    end

    supervise(children, strategy: :one_for_one)
  end

  def start_child(pid, opts) do
    {:id, id} = Enum.at(opts, 0)
    Supervisor.start_child(pid, worker(App.SimpleTask,
        [opts], restart: :transient, id: id))
  end
end

Now we can start a tasks supervisor for every new video in our app. Each supervisor will start 12 tasks to support multiple resolutions.

Create a worker

For the encoding tasks will will use Task. Although most of the examples are about GenServer, We don't always have to use it. For simple tasks like these we should use Task.

Let's have a look at the encoding task:

defmodule App.SimpleTask do
  use Task

  def start_link(opts) do
    Task.start_link(__MODULE__, :work, [opts])
  end

  def work(opts) do
    # TODO: Execute 
    # encode ...
    :timer.sleep(2000)
  end
end

Conclusion

This task only encode one video in one resolution. This simple setup makes it easy to test and recover from errors. If few tasks fail, we will encode only those resolutions.