feat: Use Amazon S3 as Stable Diffusion Model Storage #616
Conversation
…om/lindarr915/data-on-eks into bottlerocket-cache-container-image
…ata-on-eks into cache-model-from-s3
There was a problem hiding this comment.
@lindarr915 thanks so much for the PR! Please address the comments. Also make sure you run pre-commit run -a and commit the changes made.
gen-ai/inference/stable-diffusion-rayserve-gpu/ray-serve-stablediffusion-from-s3.yaml
Show resolved
Hide resolved
| image: public.ecr.aws/data-on-eks/ray2.11.0-py310-gpu-stablediffusion:latest | ||
| imagePullPolicy: IfNotPresent # Ensure the image is always pulled when updated | ||
| image: public.ecr.aws/data-on-eks/ray-serve-gpu-stablediffusion:2.33.0-py311-gpu | ||
| imagePullPolicy: Always # Ensure the image is always pulled when updated |
There was a problem hiding this comment.
If the image public.ecr.aws/data-on-eks/ray-serve-gpu-stablediffusion:2.33.0-py311-gpu is not up-to-date, then kubelet will pull the image from ECR. Otherwise, it will use the cached image.
https://kubernetes.io/docs/concepts/containers/images/#image-pull-policy
Always
every time the kubelet launches a container, the kubelet queries the container image registry to resolve the name to an image digest. If the kubelet has a container image with that exact digest cached locally, the kubelet uses its cached image; otherwise, the kubelet pulls the image with the resolved digest, and uses that image to launch the container.
There was a problem hiding this comment.
Thus, if I update the image with the same tag, I can make sure it will use the latest version. If the SHA256 is the same, then kubelet will use the cached image on disk
|
|
||
| ## End-to-end Example | ||
|
|
||
| An end-to-end deployment example can be found in [Stable Diffusion on GPU](../gen-ai/inference/GPUs/stablediffusion-gpus). |
There was a problem hiding this comment.
Wrong architecture diagram.
There was a problem hiding this comment.
In the page there is no diagram. Would you let me know which architecture diagram you are referring to?
gen-ai/inference/stable-diffusion-rayserve-gpu/download_models.ipynb
Outdated
Show resolved
Hide resolved
- Added a new job to download and save models to Amazon S3. - Created a YAML file for the job configuration. - Defined a persistent volume and persistent volume claim for model storage. - Created a config map for the shell script used in the job. - Configured the job container with the necessary image and command. - Mounted the script and model storage directories in the container.
|
@askulkarni2 Please review the new commits for the PR. |
|
This PR has been automatically marked as stale because it has been open 30 days |
|
I am still working on this PR. |
fix: ray service yaml filename
|
This PR has been automatically marked as stale because it has been open 30 days |
What does this PR do?
Implement 448
🛑 Please open an issue first to discuss any significant work and flesh out details/direction - we would hate for your time to be wasted.
Consult the CONTRIBUTING guide for submitting pull-requests.
Motivation
More
website/docsorwebsite/blogsection for this featurepre-commit run -awith this PR. Link for installing pre-commit locallyFor Moderators
Additional Notes