Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added Dockerfile #738

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions Dockerfile

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am getting thr following error when i try to import FastLanguageModel which i do in a script which is inside a docker container which is setup same as your Dockerfile
image

Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@

ARG CUDA_VERSION=12.1.1
FROM nvcr.io/nvidia/cuda:${CUDA_VERSION}-cudnn8-devel-ubuntu22.04
ENV DEBIAN_FRONTEND=noninteractive
RUN echo 'tzdata tzdata/Areas select America' | debconf-set-selections \
&& echo 'tzdata tzdata/Zones/America select Los_Angeles' | debconf-set-selections \
&& apt-get update -y \
&& apt-get install -y software-properties-common python3 python3-pip python-is-python3 git curl sudo
RUN pip install --upgrade pip
RUN pip install --upgrade torch triton packaging
RUN pip install --upgrade --force-reinstall --no-cache-dir \
ninja einops flash-attn xformers trl peft accelerate bitsandbytes \
"unsloth[cu121-torch230] @ git+https://github.com/unslothai/unsloth.git"
11 changes: 11 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -170,6 +170,17 @@ pip install "unsloth[cu121-ampere-torch230] @ git+https://github.com/unslothai/u
nvcc
python -m xformers.info
python -m bitsandbytes
```

### Docker Installation
1. Edit Dockerfile to match required CUDA and Pytorch version. By default, CUDA 12.1.1 and Pytorch 2.3 are used, but you modify to use CUDA 11.8 and Pytorch 2.x instead as well as install optional dependencies by changing `unsloth[cu121-torch230]`.
2. Build docker image
```bash
docker build -t unsloth:latest .
```
3. Run unsloth docker image
```bash
docker run --gpus all -it --rm -v $PWD:/workspace --ulimit memlock=-1 --ulimit stack=67108864 --ipc host unsloth:latest
```

## 📜 Documentation
Expand Down