Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update kubernetes_deployment.md #398

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Update kubernetes_deployment.md #398

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
c1c0171
Update kubernetes_deployment.md
Rajakavitha1 Sep 23, 2024
9dd3837
Update kubernetes_deployment.md
Rajakavitha1 Sep 23, 2024
bf1d4f9
Update kubernetes_deployment.md
Rajakavitha1 Sep 23, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
93 changes: 39 additions & 54 deletions docs/admin/kubernetes_deployment.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,52 +1,55 @@
# Kubernetes Deployment Installation Guide
KServe supports `RawDeployment` mode to enable `InferenceService` deployment with Kubernetes resources [`Deployment`](https://kubernetes.io/docs/concepts/workloads/controllers/deployment), [`Service`](https://kubernetes.io/docs/concepts/services-networking/service), [`Ingress`](https://kubernetes.io/docs/concepts/services-networking/ingress) and [`Horizontal Pod Autoscaler`](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale). Comparing to serverless deployment it unlocks Knative limitations such as mounting multiple volumes, on the other hand `Scale down and from Zero` is not supported in `RawDeployment` mode.

Kubernetes 1.22 is the minimally required version and please check the following recommended Istio versions for the corresponding
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure if we need to remove it.

Copy link
Contributor Author

@Rajakavitha1 Rajakavitha1 Sep 24, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My understanding was that Istio was not required The current installation guide for raw deployment mode is not accurate. For example, Istio is not needed if disableIngressCreation is enabled. See https://github.com/kserve/kserve/pull/3436

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, raw does not require it, but for the serverless mode the docs advises to use it, iirc.

Kubernetes version.

## Recommended Version Matrix
| Kubernetes Version | Recommended Istio Version |
|:-------------------|:--------------------------|
| 1.27 | 1.18, 1.19 |
| 1.28 | 1.19, 1.20 |
| 1.29 | 1.20, 1.21 |

## 1. Install Istio

The minimally required Istio version is 1.13 and you can refer to the [Istio install guide](https://istio.io/latest/docs/setup/install).

Once Istio is installed, create `IngressClass` resource for istio.
```yaml
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
name: istio
spec:
controller: istio.io/ingress-controller
```


!!! note
Istio ingress is recommended, but you can choose to install with other [Ingress controllers](https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/) and create `IngressClass` resource for your Ingress option.



## 2. Install Cert Manager
## 1. Install Cert Manager
The minimally required Cert Manager version is 1.9.0 and you can refer to [Cert Manager installation guide](https://cert-manager.io/docs/installation/).

!!! note
Cert manager is required to provision webhook certs for production grade installation, alternatively you can run self signed certs generation script.

## 3. Install KServe
## 2. Install KServe
!!! note
The default KServe deployment mode is `Serverless` which depends on Knative. The following step changes the default deployment mode to `RawDeployment` before installing KServe.

The default KServe deployment mode is `Serverless` which depends on Knative. The following step changes the default deployment mode to `RawDeployment` before installing KServe and the `RawDeployment` mode does not have any dependecy on Knative.

**i. Install KServe**
**i. Create a yaml file**
Open an editor of your choice and create `kustomization.yaml` file.
```yaml

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

namespace: kserve

resources:
- https://github.com/kserve/kserve/releases/download/v0.13.0/kserve.yaml

patches:
- target:
kind: ConfigMap
name: inferenceservice-config
patch: |
- path: /data/deploy
op: replace
value: |-
{
"defaultDeploymentMode": "RawDeployment"
}
- path: /data/ingress
op: replace
value: |-
{
"disableIstioVirtualHost": true,
"disableIngressCreation": true,
"ingressDomain": "svc.cluster.local",
"ingressGateway" : "disabled",
"ingressService" : "disabled"
Copy link

@spolti spolti Sep 23, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here we also need to set the default ingressClass otherwise you will need to do everything manually later.
If the reason is to decouple from Istio, most of k8s cluster have a default one, we need to give this info to the user e.g. minikube uses nginx.
Instead disabling the ingress would be better to explain both options.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please suggest what change needs to be made to the yaml file.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@spolti Can you provide some suggested verbiage here for @Rajakavitha1?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is described a few lines below, one of the removed.

btw @terrytangyuan, shouldn't we have a specific section for raw?
Instead of deleting the istio part, we should instead document both ways.
Istio is required for serverless and it is the default method.

You could change the beginning of the doc like keeping the requirements for the Serverless mode, and later, explain what is the RawDeployment (like it is done now) and describe the steps needed to install it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more. 

                "disableIstioVirtualHost": true,
                "disableIngressCreation": true,
                "ingressGateway" : "disabled",
                "ingressService" : "disabled"

I have a few questions regarding this configuration.

Isn't it true that the controller already handles these aspects? In other words, when using rawDeployment, these related calls may not even be triggered. Should we really be making changes in this area?

                "ingressDomain": "svc.cluster.local",

this would be needed.

"defaultDeploymentMode": "RawDeployment"

Additionally, I understand that this setting implies using rawDeployment as the default, but it doesn't mean that only rawDeployment will be used. I think this configuration might still be necessary when setting the serverlessMode in isvc.

Lastly, shouldn't the disableIngressCreation be set to false instead? does the ingress need not to be created?

}

**ii. Install KServe**

=== "kubectl"
```bash
kubectl apply -f https://github.com/kserve/kserve/releases/download/v0.13.0/kserve.yaml
kubectl apply -f kustomization.yaml
```

Install KServe default serving runtimes:
Expand All @@ -55,21 +58,3 @@ Install KServe default serving runtimes:
```bash
kubectl apply -f https://github.com/kserve/kserve/releases/download/v0.13.0/kserve-cluster-resources.yaml
```

**ii. Change default deployment mode and ingress option**

First in ConfigMap `inferenceservice-config` modify the `defaultDeploymentMode` in the `deploy` section,

=== "kubectl"
```bash
kubectl patch configmap/inferenceservice-config -n kserve --type=strategic -p '{"data": {"deploy": "{\"defaultDeploymentMode\": \"RawDeployment\"}"}}'
```

then modify the `ingressClassName` in `ingress` section to point to `IngressClass` name created in [step 1](#1-install-istio).
```yaml
ingress: |-
{
"ingressClassName" : "your-ingress-class",
}
```