feat: JARK stack update with Argo Workflows, Karpenter and KubeRay #487

vara-bonthu · 2024-04-05T15:13:44Z

What does this PR do?

🛑 Please open an issue first to discuss any significant work and flesh out details/direction - we would hate for your time to be wasted.
Consult the CONTRIBUTING guide for submitting pull-requests.

JARK stack updated with Argo workflows, Karpenter addons
KubeRay Operator upgraded
StableDiffusion example added with RayServe2.10 with autoscaling and servecinfigv2
EFA disabled by default
New folder structure created for the examples that shows training and inference

NOTE: Website Doc for this deployment will be raised as a Part2 PR

Motivation

More

Yes, I have tested the PR using my local account setup (Provide any test evidence report under Additional Notes)
Mandatory for new blueprints. Yes, I have added a example to support my blueprint PR
Mandatory for new blueprints. Yes, I have updated the website/docs or website/blog section for this feature
Yes, I ran pre-commit run -a with this PR. Link for installing pre-commit locally

For Moderators

E2E Test successfully complete before merge?

Additional Notes

alanty

I like the updates and think i follow the example files, we could really use docs on this stack.

alanty · 2024-04-05T16:41:41Z

ai-ml/jark-stack/terraform/addons.tf

-  enable_kuberay_operator = true
+  enable_nvidia_device_plugin = true
+  nvidia_device_plugin_helm_config = {
+    version = "v0.15.0-rc.2"


are there features we need in the RC release?

imo if we're using the release candidate we should ensure we update this again when the "full" 0.15.x release is out(or better, bump the addon default).

+1. We will need to keep this up to date on an ongoing basis based on the number of issues we keep encountering.

Agree. I had to change to the latest version to verify the NVIDIA Device plugin issue that we were facing. It turns out to be an EKS AMI issue but not the NVIDIA device plugin.

I will change this to stable version v0.14.5 for now

it's always better for us to lock the blueprints addons to a specific version to avoid failures with newer releases

I think we should use 0.14.5 major release for now instead of release candidate

alanty · 2024-04-05T16:51:39Z

ai-ml/jark-stack/terraform/addons.tf

+  nvidia_device_plugin_helm_config = {
+    version = "v0.15.0-rc.2"
+    name    = "nvidia-device-plugin"
+    values = [


minor structure thing.

this PR moves the Nvidia plugin values inline at ~188, I added a similar toleration block for the efa device plugin in aws-efa-k8s-device-plugin-values.yaml. Is there a preference or style to follow?

I like having the smaller values blocks inline as it makes things a bit easier to read without having to jump between files. Though the values in a separate file is a bit easier syntax and more like helm.

We don't necessarily have a preference one way or the other. In general the guideline is small changes can go inline, large changes in a separate file.

+1 to Apoorva. If it's a small config easy to embed into the code otherwise, create a dedicated yaml file

askulkarni2

Thank you for the awesome PR! LGTM!

askulkarni2 · 2024-04-05T17:23:21Z

ai-ml/jark-stack/terraform/addons.tf

-  enable_kuberay_operator = true
+  enable_nvidia_device_plugin = true
+  nvidia_device_plugin_helm_config = {
+    version = "v0.15.0-rc.2"


+1. We will need to keep this up to date on an ongoing basis based on the number of issues we keep encountering.

askulkarni2 · 2024-04-05T17:25:27Z

ai-ml/jark-stack/terraform/addons.tf

+  nvidia_device_plugin_helm_config = {
+    version = "v0.15.0-rc.2"
+    name    = "nvidia-device-plugin"
+    values = [


We don't necessarily have a preference one way or the other. In general the guideline is small changes can go inline, large changes in a separate file.

askulkarni2 · 2024-04-05T17:26:25Z

ai-ml/jark-stack/terraform/addons.tf

+        securityGroupSelectorTerms:
+          tags:
+            Name: ${module.eks.cluster_name}-node
+        instanceStorePolicy: RAID0


askulkarni2 · 2024-04-05T17:33:36Z

ai-ml/jark-stack/terraform/variables.tf

@@ -13,7 +13,7 @@ variable "region" {

 variable "eks_cluster_version" {
  description = "EKS Cluster version"
-  default     = "1.27"
+  default     = "1.29"


sanjeevrg89 · 2024-04-05T18:49:41Z

Overall, LGTM as we. Thank you @vara-bonthu

ratnopamc

LGTM!

sanjeevrg89

Looks great, approving the request

sanjeevrg89 · 2024-04-05T18:43:53Z

ai-ml/jark-stack/terraform/addons.tf

-  enable_kuberay_operator = true
+  enable_nvidia_device_plugin = true
+  nvidia_device_plugin_helm_config = {
+    version = "v0.15.0-rc.2"


I think we should use 0.14.5 major release for now instead of release candidate

vara-bonthu added 6 commits April 4, 2024 21:21

Stable Diffusion example added to JARK stack

37f7437

Karpenter and Argo Workflows added to the blueprint

85d62c0

Added karpenter and argo workflows addons

30b5cc2

made efa optional

e5dfd01

updated Karpenter resources

f5ae88b

Updated the docker images

3b825e5

vara-bonthu temporarily deployed to DoEKS Test April 5, 2024 15:13 — with GitHub Actions Inactive

vara-bonthu requested review from askulkarni2, ratnopamc, alanty, lusoal and sanjeevrg89 and removed request for askulkarni2 and ratnopamc April 5, 2024 15:15

alanty reviewed Apr 5, 2024

View reviewed changes

askulkarni2 approved these changes Apr 5, 2024

View reviewed changes

updpated nvidia deviceplugin version

7da6f3c

vara-bonthu temporarily deployed to DoEKS Test April 5, 2024 18:47 — with GitHub Actions Inactive

ratnopamc approved these changes Apr 5, 2024

View reviewed changes

sanjeevrg89 approved these changes Apr 5, 2024

View reviewed changes

vara-bonthu merged commit fcda963 into main Apr 5, 2024
55 of 56 checks passed

vara-bonthu deleted the feat-sd-jark branch April 5, 2024 18:52

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: JARK stack update with Argo Workflows, Karpenter and KubeRay #487

feat: JARK stack update with Argo Workflows, Karpenter and KubeRay #487

vara-bonthu commented Apr 5, 2024 •

edited

Loading

alanty left a comment

alanty Apr 5, 2024

askulkarni2 Apr 5, 2024

vara-bonthu Apr 5, 2024

sanjeevrg89 Apr 5, 2024

alanty Apr 5, 2024

askulkarni2 Apr 5, 2024

vara-bonthu Apr 5, 2024

askulkarni2 left a comment

askulkarni2 Apr 5, 2024

askulkarni2 Apr 5, 2024

askulkarni2 Apr 5, 2024

askulkarni2 Apr 5, 2024

sanjeevrg89 commented Apr 5, 2024

ratnopamc left a comment

sanjeevrg89 left a comment

sanjeevrg89 Apr 5, 2024

feat: JARK stack update with Argo Workflows, Karpenter and KubeRay #487

feat: JARK stack update with Argo Workflows, Karpenter and KubeRay #487

Conversation

vara-bonthu commented Apr 5, 2024 • edited Loading

What does this PR do?

Motivation

More

For Moderators

Additional Notes

alanty left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

askulkarni2 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sanjeevrg89 commented Apr 5, 2024

ratnopamc left a comment

Choose a reason for hiding this comment

sanjeevrg89 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vara-bonthu commented Apr 5, 2024 •

edited

Loading