Skip to content

Commit

Permalink
Switch to ProcessSession for Disco (#57)
Browse files Browse the repository at this point in the history
  • Loading branch information
masahi authored Nov 9, 2023
1 parent 338d6d2 commit c3c49b5
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion serve/mlc_serve/model/paged_cache_model.py
Original file line number Diff line number Diff line change
Expand Up @@ -303,7 +303,7 @@ def sample(logits, sampling_params, vocab_size):


def load_disco_module(artifact_path, lib_path, num_shards):
sess = di.ThreadedSession(num_workers=num_shards)
sess = di.ProcessSession(num_workers=num_shards)
devices = range(num_shards)
sess.init_ccl("nccl", *devices)
module = sess.load_vm_module(lib_path)
Expand Down

0 comments on commit c3c49b5

Please sign in to comment.