You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In working on a segment-anthing livebook, based upon this forum example, I am downloading the onnx model from hugging face. The benefit is that the Livebook can be readily shared with no dependencies.
Currently it needs to be saved to a file system before calling Ortext.load/1 since that function ultimately delegates to the following in model.rs:
let session = Session::builder()?
.with_execution_providers(&eps)?
.with_optimization_level(map_opt_level(opt))?
.with_model_from_file(model_path)?;
A version or an option which calls .with_model_from_memory(binary) would be a valuable, and hopefully a simple addition.
I think Livebooks is a likely a common use case so hopefully something you might consider. I would happily send a PR if I had any relevant skill, but in Rust I do not.
The text was updated successfully, but these errors were encountered:
Thanks for the idea! We've been planning to implement model loading from memory for a while now, but we're waiting on the upstream ort 2.0 changes to settle before doing so.
One other benefit that will unlock is loading directly from axon_onnx, so in the future we may be able to export Axon models to run with Ortex in memory.
In working on a segment-anthing livebook, based upon this forum example, I am downloading the onnx model from hugging face. The benefit is that the Livebook can be readily shared with no dependencies.
Currently it needs to be saved to a file system before calling
Ortext.load/1
since that function ultimately delegates to the following in model.rs:A version or an option which calls
.with_model_from_memory(binary)
would be a valuable, and hopefully a simple addition.I think Livebooks is a likely a common use case so hopefully something you might consider. I would happily send a PR if I had any relevant skill, but in Rust I do not.
The text was updated successfully, but these errors were encountered: