Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change to dashes in image names #2857

Closed
wants to merge 1 commit into from
Closed

Change to dashes in image names #2857

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 11 additions & 11 deletions docker/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ all dependencies for both NLP and Vision models. They are built on top of the
| Composer Version | CUDA Support | Docker Tag |
|--------------------|----------------|----------------------------------------------------------------|
| 0.17.2 | Yes | `mosaicml/composer:latest`, `mosaicml/composer:0.17.2` |
| 0.17.2 | No | `mosaicml/composer:latest_cpu`, `mosaicml/composer:0.17.2_cpu` |
| 0.17.2 | No | `mosaicml/composer:latest-cpu`, `mosaicml/composer:0.17.2-cpu` |
<!-- END_COMPOSER_BUILD_MATRIX -->

**Note**: For a lightweight installation, we recommended using a [MosaicML PyTorch Image](#pytorch-images) and manually
Expand All @@ -31,16 +31,16 @@ To install composer, once inside the image, run `pip install mosaicml`.
<!-- BEGIN_PYTORCH_BUILD_MATRIX -->
| Linux Distro | Flavor | PyTorch Version | CUDA Version | Python Version | Docker Tags |
|----------------|----------|-------------------|---------------------|------------------|------------------------------------------------------------------------------------------|
| Ubuntu 20.04 | Base | 2.3.0 | 12.1.0 (Infiniband) | 3.10 | `mosaicml/pytorch:2.3.0_cu121-nightly20240110-python3.10-ubuntu20.04` |
| Ubuntu 20.04 | Base | 2.1.2 | 12.1.0 (Infiniband) | 3.10 | `mosaicml/pytorch:latest`, `mosaicml/pytorch:2.1.2_cu121-python3.10-ubuntu20.04` |
| Ubuntu 20.04 | Base | 2.1.2 | 12.1.0 (EFA) | 3.10 | `mosaicml/pytorch:latest-aws`, `mosaicml/pytorch:2.1.2_cu121-python3.10-ubuntu20.04-aws` |
| Ubuntu 20.04 | Base | 2.1.2 | cpu | 3.10 | `mosaicml/pytorch:latest_cpu`, `mosaicml/pytorch:2.1.2_cpu-python3.10-ubuntu20.04` |
| Ubuntu 20.04 | Base | 2.0.1 | 11.8.0 (Infiniband) | 3.10 | `mosaicml/pytorch:2.0.1_cu118-python3.10-ubuntu20.04` |
| Ubuntu 20.04 | Base | 2.0.1 | 11.8.0 (EFA) | 3.10 | `mosaicml/pytorch:2.0.1_cu118-python3.10-ubuntu20.04-aws` |
| Ubuntu 20.04 | Base | 2.0.1 | cpu | 3.10 | `mosaicml/pytorch:2.0.1_cpu-python3.10-ubuntu20.04` |
| Ubuntu 20.04 | Base | 1.13.1 | 11.7.1 (Infiniband) | 3.10 | `mosaicml/pytorch:1.13.1_cu117-python3.10-ubuntu20.04` |
| Ubuntu 20.04 | Base | 1.13.1 | 11.7.1 (EFA) | 3.10 | `mosaicml/pytorch:1.13.1_cu117-python3.10-ubuntu20.04-aws` |
| Ubuntu 20.04 | Base | 1.13.1 | cpu | 3.10 | `mosaicml/pytorch:1.13.1_cpu-python3.10-ubuntu20.04` |
| Ubuntu 20.04 | Base | 2.3.0 | 12.1.0 (Infiniband) | 3.10 | `mosaicml/pytorch:2.3.0-cu121-nightly20240110-python3.10-ubuntu20.04` |
| Ubuntu 20.04 | Base | 2.1.2 | 12.1.0 (Infiniband) | 3.10 | `mosaicml/pytorch:latest`, `mosaicml/pytorch:2.1.2-cu121-python3.10-ubuntu20.04` |
| Ubuntu 20.04 | Base | 2.1.2 | 12.1.0 (EFA) | 3.10 | `mosaicml/pytorch:latest-aws`, `mosaicml/pytorch:2.1.2-cu121-python3.10-ubuntu20.04-aws` |
| Ubuntu 20.04 | Base | 2.1.2 | cpu | 3.10 | `mosaicml/pytorch:latest_cpu`, `mosaicml/pytorch:2.1.2-cpu-python3.10-ubuntu20.04` |
| Ubuntu 20.04 | Base | 2.0.1 | 11.8.0 (Infiniband) | 3.10 | `mosaicml/pytorch:2.0.1-cu118-python3.10-ubuntu20.04` |
| Ubuntu 20.04 | Base | 2.0.1 | 11.8.0 (EFA) | 3.10 | `mosaicml/pytorch:2.0.1-cu118-python3.10-ubuntu20.04-aws` |
| Ubuntu 20.04 | Base | 2.0.1 | cpu | 3.10 | `mosaicml/pytorch:2.0.1-cpu-python3.10-ubuntu20.04` |
| Ubuntu 20.04 | Base | 1.13.1 | 11.7.1 (Infiniband) | 3.10 | `mosaicml/pytorch:1.13.1-cu117-python3.10-ubuntu20.04` |
| Ubuntu 20.04 | Base | 1.13.1 | 11.7.1 (EFA) | 3.10 | `mosaicml/pytorch:1.13.1-cu117-python3.10-ubuntu20.04-aws` |
| Ubuntu 20.04 | Base | 1.13.1 | cpu | 3.10 | `mosaicml/pytorch:1.13.1-cpu-python3.10-ubuntu20.04` |
<!-- END_PYTORCH_BUILD_MATRIX -->

**Note**: The `mosaicml/pytorch:latest`, `mosaicml/pytorch:latest_cpu`, and `mosaicml/pytorch:latest-aws`
Expand Down
24 changes: 12 additions & 12 deletions docker/build_matrix.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
PYTORCH_NIGHTLY_VERSION: ''
PYTORCH_VERSION: 2.1.2
TAGS:
- mosaicml/pytorch:2.1.2_cu121-python3.10-ubuntu20.04
- mosaicml/pytorch:2.1.2-cu121-python3.10-ubuntu20.04
- mosaicml/pytorch:latest
TARGET: pytorch_stage
TORCHVISION_VERSION: 0.16.2
Expand Down Expand Up @@ -51,7 +51,7 @@
PYTORCH_NIGHTLY_VERSION: ''
PYTORCH_VERSION: 2.1.2
TAGS:
- mosaicml/pytorch:2.1.2_cu121-python3.10-ubuntu20.04-aws
- mosaicml/pytorch:2.1.2-cu121-python3.10-ubuntu20.04-aws
- mosaicml/pytorch:latest-aws
TARGET: pytorch_stage
TORCHVISION_VERSION: 0.16.2
Expand All @@ -66,7 +66,7 @@
PYTORCH_NIGHTLY_VERSION: ''
PYTORCH_VERSION: 2.1.2
TAGS:
- mosaicml/pytorch:2.1.2_cpu-python3.10-ubuntu20.04
- mosaicml/pytorch:2.1.2-cpu-python3.10-ubuntu20.04
- mosaicml/pytorch:latest_cpu
TARGET: pytorch_stage
TORCHVISION_VERSION: 0.16.2
Expand All @@ -86,7 +86,7 @@
PYTORCH_NIGHTLY_VERSION: ''
PYTORCH_VERSION: 2.0.1
TAGS:
- mosaicml/pytorch:2.0.1_cu118-python3.10-ubuntu20.04
- mosaicml/pytorch:2.0.1-cu118-python3.10-ubuntu20.04
TARGET: pytorch_stage
TORCHVISION_VERSION: 0.15.2
- AWS_OFI_NCCL_VERSION: v1.7.4-aws
Expand All @@ -105,7 +105,7 @@
PYTORCH_NIGHTLY_VERSION: ''
PYTORCH_VERSION: 2.0.1
TAGS:
- mosaicml/pytorch:2.0.1_cu118-python3.10-ubuntu20.04-aws
- mosaicml/pytorch:2.0.1-cu118-python3.10-ubuntu20.04-aws
TARGET: pytorch_stage
TORCHVISION_VERSION: 0.15.2
- AWS_OFI_NCCL_VERSION: ''
Expand All @@ -119,7 +119,7 @@
PYTORCH_NIGHTLY_VERSION: ''
PYTORCH_VERSION: 2.0.1
TAGS:
- mosaicml/pytorch:2.0.1_cpu-python3.10-ubuntu20.04
- mosaicml/pytorch:2.0.1-cpu-python3.10-ubuntu20.04
TARGET: pytorch_stage
TORCHVISION_VERSION: 0.15.2
- AWS_OFI_NCCL_VERSION: ''
Expand All @@ -133,7 +133,7 @@
PYTORCH_NIGHTLY_VERSION: ''
PYTORCH_VERSION: 1.13.1
TAGS:
- mosaicml/pytorch:1.13.1_cu117-python3.10-ubuntu20.04
- mosaicml/pytorch:1.13.1-cu117-python3.10-ubuntu20.04
TARGET: pytorch_stage
TORCHVISION_VERSION: 0.14.1
- AWS_OFI_NCCL_VERSION: v1.7.4-aws
Expand All @@ -147,7 +147,7 @@
PYTORCH_NIGHTLY_VERSION: ''
PYTORCH_VERSION: 1.13.1
TAGS:
- mosaicml/pytorch:1.13.1_cu117-python3.10-ubuntu20.04-aws
- mosaicml/pytorch:1.13.1-cu117-python3.10-ubuntu20.04-aws
TARGET: pytorch_stage
TORCHVISION_VERSION: 0.14.1
- AWS_OFI_NCCL_VERSION: ''
Expand All @@ -161,7 +161,7 @@
PYTORCH_NIGHTLY_VERSION: ''
PYTORCH_VERSION: 1.13.1
TAGS:
- mosaicml/pytorch:1.13.1_cpu-python3.10-ubuntu20.04
- mosaicml/pytorch:1.13.1-cpu-python3.10-ubuntu20.04
TARGET: pytorch_stage
TORCHVISION_VERSION: 0.14.1
- AWS_OFI_NCCL_VERSION: ''
Expand All @@ -188,7 +188,7 @@
PYTORCH_NIGHTLY_VERSION: dev20240110+cu121
PYTORCH_VERSION: 2.3.0
TAGS:
- mosaicml/pytorch:2.3.0_cu121-nightly20240110-python3.10-ubuntu20.04
- mosaicml/pytorch:2.3.0-cu121-nightly20240110-python3.10-ubuntu20.04
TARGET: pytorch_stage
TORCHVISION_VERSION: 0.18.0
- AWS_OFI_NCCL_VERSION: ''
Expand Down Expand Up @@ -232,7 +232,7 @@
PYTORCH_NIGHTLY_VERSION: ''
PYTORCH_VERSION: 2.1.2
TAGS:
- mosaicml/composer:0.17.2_cpu
- mosaicml/composer:latest_cpu
- mosaicml/composer:0.17.2-cpu
- mosaicml/composer:latest-cpu
TARGET: composer_stage
TORCHVISION_VERSION: 0.16.2
10 changes: 5 additions & 5 deletions docker/generate_build_matrix.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@ def _get_pytorch_tags(python_version: str, pytorch_version: str, cuda_version: s
else:
raise ValueError(f'Invalid stage: {stage}')
cuda_version_tag = _get_cuda_version_tag(cuda_version)
tags = [f'{base_image_name}:{pytorch_version}_{cuda_version_tag}-python{python_version}-ubuntu20.04']
tags = [f'{base_image_name}:{pytorch_version}-{cuda_version_tag}-python{python_version}-ubuntu20.04']

if python_version == LATEST_PYTHON_VERSION and pytorch_version == PRODUCTION_PYTORCH_VERSION:
if not cuda_version:
Expand All @@ -123,8 +123,8 @@ def _get_composer_tags(composer_version: str, use_cuda: bool):

tags = []
if not use_cuda:
tags.append(f'{base_image_name}:{composer_version}_cpu')
tags.append(f'{base_image_name}:latest_cpu')
tags.append(f'{base_image_name}:{composer_version}-cpu')
tags.append(f'{base_image_name}:latest-cpu')
else:
tags.append(f'{base_image_name}:{composer_version}')
tags.append(f'{base_image_name}:latest')
Expand Down Expand Up @@ -238,7 +238,7 @@ def _main():
'PYTORCH_VERSION': '2.3.0',
'PYTORCH_NIGHTLY_URL': 'https://download.pytorch.org/whl/nightly/cu121',
'PYTORCH_NIGHTLY_VERSION': 'dev20240110+cu121',
'TAGS': ['mosaicml/pytorch:2.3.0_cu121-nightly20240110-python3.10-ubuntu20.04'],
'TAGS': ['mosaicml/pytorch:2.3.0-cu121-nightly20240110-python3.10-ubuntu20.04'],
'TARGET': 'pytorch_stage',
'TORCHVISION_VERSION': '0.18.0'
}
Expand Down Expand Up @@ -321,7 +321,7 @@ def _main():
continue

table.append([
entry['TAGS'][0].split(':')[1].replace('_cpu', ''), # Composer version, or 'latest'
entry['TAGS'][0].split(':')[1].replace('-cpu', ''), # Composer version, or 'latest'
'No' if entry['BASE_IMAGE'].startswith('ubuntu:') else 'Yes', # Whether there is Cuda support
', '.join(reversed([f'`{x}`' for x in entry['TAGS']])), # Docker tags
])
Expand Down
Loading