Skip to content

Latest commit

 

History

History
45 lines (29 loc) · 1.55 KB

rapids-dask.md

File metadata and controls

45 lines (29 loc) · 1.55 KB

RAPIDS Dask

RAPIDS with Dask

Introduction

Dask allows distributed computation in Python. RAPIDS adds gpu acceleration to machine learning.

Dask has tight kubernetes integration that allows you to scale up/down your Dask cluster either from within your python code or using the kubectl utility.

Installation

Kubeflow

If Kubeflow has already been installed using the DeepOps Kubeflow Deployment Guide there are no additional K8S setup steps required.

When deploying through Kubeflow, it is necessary to ensure that a proper Docker image, entrypoint, and cmd have been specified; or Kubeflow will not properly start Jupyter and the service will immediately fail. See the Dask Kubernetes Dockerfile for an example.

Stand-alone

Deploy Kubernetes by following the DeepOps Kubernetes Deployment Guide

Deploy the LoadBalancer

Deploy Dask:

# Optionally, Modify chart configuration
vi config/helm/rapids-dask.yml

# Optionally, modify the K8S resources
vi config/k8s/rapids-dask-sa.yml

# Deploy
./scripts/k8s/deploy_rapids_dask.sh

For more configuration options, see: https://github.com/rmccorm4/charts/tree/update-stable-dask/stable/dask For more information about scaling dask in kubernetes see the included example notebooks.