feat: JARK stack update with Argo Workflows, Karpenter and KubeRay #487
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| # https://hub.docker.com/layers/rayproject/ray-ml/2.10.0-py310-gpu/images/sha256-4181ed53b0b25a758b155312ca6ab29a65cb78cd57296d42cfbe4806a2b77df4?context=explore | ||
| # docker buildx build --platform=linux/amd64 -t ray2.10.0-py310-gpu-stablediffusion:v1.0 -f Dockerfile . | ||
|
|
||
| # Use Ray base image | ||
| FROM rayproject/ray-ml:2.10.0-py310-gpu | ||
|
|
||
| # Maintainer label | ||
| LABEL maintainer="DoEKS" | ||
|
|
||
| # Set environment variables to non-interactive (this prevents some prompts) | ||
| ENV DEBIAN_FRONTEND=non-interactive | ||
|
|
||
| # Switch back to a non-root user for the subsequent commands | ||
| USER $USER | ||
|
|
||
| # Install Ray Serve and other Python packages with specific versions | ||
| RUN pip install --no-cache-dir requests torch "diffusers==0.12.1" "transformers=4.25.1" | ||
|
|
||
| # Set a working directory | ||
| WORKDIR /serve_app | ||
|
|
||
| # Copy your Ray Serve script into the container | ||
| COPY ray_serve_sd.py /serve_app/ray_serve_sd.py | ||
|
|
||
| # Set the PYTHONPATH environment variable | ||
| ENV PYTHONPATH=/serve_app:$PYTHONPATH |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| # Use Python base image | ||
| FROM --platform=linux/amd64 python:3.9-slim | ||
|
|
||
| # Set working directory in the container | ||
| WORKDIR /app | ||
|
|
||
| # Copy the Python script into the container | ||
| COPY gradio-app.py /app/gradio-app.py | ||
|
|
||
| RUN pip install --no-cache-dir gradio requests Pillow | ||
|
|
||
| # Command to run the Python script | ||
| ENTRYPOINT ["python", "gradio-app.py"] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,32 @@ | ||
| import gradio as gr | ||
| import requests | ||
| import os | ||
| from PIL import Image | ||
| from io import BytesIO | ||
|
|
||
| # Constants for model endpoint and service name | ||
| model_endpoint = os.environ.get("MODEL_ENDPOINT", "/imagine") | ||
| service_name = os.environ.get("SERVICE_NAME", "http://localhost:8000") | ||
|
|
||
| # Function to generate image based on prompt | ||
| def generate_image(prompt): | ||
|
|
||
| # Create the URL for the inference | ||
| url = f"{service_name}{model_endpoint}" | ||
|
|
||
| try: | ||
| # Send the request to the model service | ||
| response = requests.get(url, params={"prompt": prompt}, timeout=180) | ||
| response.raise_for_status() # Raise an exception for HTTP errors | ||
| i = Image.open(BytesIO(response.content)) | ||
| return i | ||
|
|
||
| except requests.exceptions.RequestException as e: | ||
| # Handle any request exceptions (e.g., connection errors) | ||
| # return f"AI: Error: {str(e)}" | ||
| return Image.new('RGB', (100, 100), color='red') | ||
|
|
||
| # Define the Gradio PromptInterface | ||
| demo = gr.Interface(fn=generate_image, | ||
| inputs = [gr.Textbox(label="Enter the Prompt")], | ||
| outputs = gr.Image(type='pil')).launch(server_name="0.0.0.0") |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,58 @@ | ||
| apiVersion: v1 | ||
| kind: Namespace | ||
| metadata: | ||
| name: gradio | ||
| --- | ||
| apiVersion: apps/v1 | ||
| kind: Deployment | ||
| metadata: | ||
| name: gradio-deployment | ||
| namespace: gradio | ||
| labels: | ||
| app: gradio | ||
| spec: | ||
| replicas: 1 | ||
| selector: | ||
| matchLabels: | ||
| app: gradio | ||
| template: | ||
| metadata: | ||
| labels: | ||
| app: gradio | ||
| spec: | ||
| containers: | ||
| - name: gradio | ||
| # Update this image to the Gradio app image you want to deploy | ||
| image: public.ecr.aws/data-on-eks/gradio-app:sd-v1.0 | ||
| imagePullPolicy: IfNotPresent | ||
| ports: | ||
| - containerPort: 7860 | ||
| resources: | ||
| requests: | ||
| cpu: "512m" | ||
| memory: "2048Mi" | ||
| limits: | ||
| cpu: "1" | ||
| memory: "4096Mi" | ||
| env: | ||
| - name: MODEL_ENDPOINT | ||
| value: "/imagine" | ||
| # Please note that the service name is currently hardcoded to match the Stable Diffusion service for this blueprint. | ||
| # If there are any updates or changes to the actual RayServe deployment, you'll need to update the service name in this code accordingly. | ||
| - name: SERVICE_NAME | ||
| value: "http://stablediffusion-service.stablediffusion.svc.cluster.local:8000" | ||
| --- | ||
| apiVersion: v1 | ||
| kind: Service | ||
| metadata: | ||
| name: gradio-service | ||
| namespace: gradio | ||
| spec: | ||
| selector: | ||
| app: gradio | ||
| ports: | ||
| - name: http | ||
| protocol: TCP | ||
| port: 7860 | ||
| targetPort: 7860 | ||
| type: ClusterIP |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
minor structure thing.
this PR moves the Nvidia plugin values inline at ~188, I added a similar toleration block for the efa device plugin in aws-efa-k8s-device-plugin-values.yaml. Is there a preference or style to follow?
I like having the smaller values blocks inline as it makes things a bit easier to read without having to jump between files. Though the values in a separate file is a bit easier syntax and more like helm.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We don't necessarily have a preference one way or the other. In general the guideline is small changes can go inline, large changes in a separate file.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1 to Apoorva. If it's a small config easy to embed into the code otherwise, create a dedicated yaml file