Onnx docs update (kserve#275)

* Updated Onnx example. Signed-off-by: Andrews Arokiam <[email protected]> * Reverting sklearn doc update as there is a separate PR Signed-off-by: andyi2it <[email protected]> * Added new schema in onnx example. Signed-off-by: Andrews Arokiam <[email protected]> * protocolVersion and old schema updated with onnx example. Signed-off-by: Andrews Arokiam <[email protected]> --------- Signed-off-by: Andrews Arokiam <[email protected]> Signed-off-by: andyi2it <[email protected]>
bmopuri · Jan 27, 2024 · 03c546c · 03c546c
1 parent aa186f9
commit 03c546c
Show file tree

Hide file tree

Showing 10 changed files with 2,086 additions and 1 deletion.
diff --git a/docs/modelserving/observability/prometheus_metrics.md b/docs/modelserving/observability/prometheus_metrics.md
@@ -28,7 +28,7 @@ spec:
   predictor:
     sklearn:
       protocolVersion: v2
-      storageUri: "gs://seldon-models/sklearn/mms/lr_model"
+      storageUri: "gs://seldon-models/sklearn/iris"
 ```
 
 The default values for `serving.kserve.io/enable-prometheus-scraping` can be set in the `inferenceservice-config` configmap. See [the docs](https://github.com/kserve/kserve/blob/master/qpext/README.md#configs) for more info.

diff --git a/docs/modelserving/v1beta1/onnx/README.md b/docs/modelserving/v1beta1/onnx/README.md
@@ -0,0 +1,67 @@
+
+# Deploy InferenceService with ONNX model
+## Setup
+1. Your ~/.kube/config should point to a cluster with [KServe installed](https://github.com/kserve/kserve#installation).
+2. Your cluster's Istio Ingress gateway must be [network accessible](https://istio.io/latest/docs/tasks/traffic-management/ingress/ingress-control/).
+
+## Create the InferenceService
+
+=== "New Schema"
+
+    ```yaml
+    apiVersion: "serving.kserve.io/v1beta1"
+    kind: "InferenceService"
+    metadata:
+      name: "style-sample"
+    spec:
+      predictor:
+        model:
+          protocolVersion: v2
+          modelFormat:
+            name: onnx
+          storageUri: "gs://kfserving-examples/models/onnx"
+    ```
+
+=== "Old Schema"
+
+    ```yaml
+    apiVersion: "serving.kserve.io/v1beta1"
+    kind: "InferenceService"
+    metadata:
+      name: "style-sample"
+    spec:
+      predictor:
+        onnx:
+          storageUri: "gs://kfserving-examples/models/onnx"
+    ```
+!!! Note
+    For the default kserve installation, While using new schema, you must specify **protocolVersion** as v2 for onnx. Otherwise, you will get a no runtime found error.
+Expected Output
+```
+$ inferenceservice.serving.kserve.io/style-sample configured
+```
+
+## Run a sample inference
+
+1. Setup env vars
+The first step is to [determine the ingress IP and ports](https://kserve.github.io/website/master/get_started/first_isvc/#4-determine-the-ingress-ip-and-ports) and set `INGRESS_HOST` and `INGRESS_PORT`
+
+```
+export ISVC_NAME=style-sample
+export SERVICE_HOSTNAME=$(kubectl get inferenceservice ${ISVC_NAME} -o jsonpath='{.status.url}' | cut -d "/" -f 3)
+```
+2. Verify the service is healthy
+```
+curl -v -H "Host:${SERVICE_HOSTNAME}" http://localhost:8080//v2/health/ready
+```
+3. Install dependencies
+```
+pip install -r requirements.txt
+```
+4. Run the [sample notebook](mosaic-onnx.ipynb) in jupyter
+```
+jupyter notebook
+```
+
+## Uploading your own model
+The sample model for the example in this readme is already uploaded and available for use. However if you would like to modify the example to use your own ONNX model, all you need to do is to upload your model as `model.onnx` to S3, GCS or an Azure Blob.
diff --git a/docs/modelserving/v1beta1/onnx/assets/onnx_ml_pb2.py b/docs/modelserving/v1beta1/onnx/assets/onnx_ml_pb2.py
diff --git a/docs/modelserving/v1beta1/onnx/assets/predict_pb2.py b/docs/modelserving/v1beta1/onnx/assets/predict_pb2.py
diff --git a/docs/modelserving/v1beta1/onnx/image.jpg b/docs/modelserving/v1beta1/onnx/image.jpg