Skip to content

Commit

Permalink
Add more notes around distribution
Browse files Browse the repository at this point in the history
  • Loading branch information
josevalim committed Jun 15, 2024
1 parent 54e22a4 commit 791a63d
Showing 1 changed file with 5 additions and 3 deletions.
8 changes: 5 additions & 3 deletions nx/lib/nx/serving.ex
Original file line number Diff line number Diff line change
Expand Up @@ -216,12 +216,14 @@ defmodule Nx.Serving do
This can be done by passing either `Nx.backend_transfer/1` or `Nx.backend_copy/1`
as third argument:
Nx.Serving.batched_run(MyDistributedServing, input, &Nx.backend_copy/1)
Nx.Serving.batched_run(MyDistributedServing, input, &Nx.backend_copy(&1, Nx.BinaryBackend))
Use `backend_transfer/1` if you know the input will no longer be used.
Similarly, the serving has a `distributed_postprocessing` callback which can do
equivalent before sending the reply to the caller.
Similarly, the serving has a `distributed_postprocessing` callback which is
called on the remote machine before sending the reply to the caller. It can
be used to transfer resources to the binary backend before sending them over
the network.
The servings are dispatched using Erlang Distribution. You can use
`Node.connect/1` to manually connect nodes. In a production setup, this is
Expand Down

0 comments on commit 791a63d

Please sign in to comment.