From c01e9eeaae7b384a8a9e8dd0a86b4a7787fb8ae8 Mon Sep 17 00:00:00 2001 From: Steven Liu <59462357+stevhliu@users.noreply.github.com> Date: Fri, 15 Dec 2023 13:17:29 -0800 Subject: [PATCH] [docs] MPS (#28016) * mps docs * toctree --- docs/source/en/_toctree.yml | 2 +- docs/source/en/main_classes/trainer.md | 61 -------------------------- docs/source/en/perf_train_special.md | 47 ++++++++++++++++++-- 3 files changed, 44 insertions(+), 66 deletions(-) diff --git a/docs/source/en/_toctree.yml b/docs/source/en/_toctree.yml index b8413b2ebd5a79..57ab9769b60f80 100644 --- a/docs/source/en/_toctree.yml +++ b/docs/source/en/_toctree.yml @@ -151,7 +151,7 @@ - local: perf_train_tpu_tf title: Training on TPU with TensorFlow - local: perf_train_special - title: Training on Specialized Hardware + title: PyTorch training on Apple silicon - local: perf_hardware title: Custom hardware for training - local: hpo_train diff --git a/docs/source/en/main_classes/trainer.md b/docs/source/en/main_classes/trainer.md index beb3241e6232a3..2b2f5c3d5f8865 100644 --- a/docs/source/en/main_classes/trainer.md +++ b/docs/source/en/main_classes/trainer.md @@ -369,67 +369,6 @@ Pass `--fsdp "full shard"` along with following changes to be made in `--fsdp_co - For size based auto wrap policy, please add `min_num_params` in the config file. It specifies FSDP's minimum number of parameters for auto wrapping. - -### Using Trainer for accelerated PyTorch Training on Mac - -With PyTorch v1.12 release, developers and researchers can take advantage of Apple silicon GPUs for significantly faster model training. -This unlocks the ability to perform machine learning workflows like prototyping and fine-tuning locally, right on Mac. -Apple's Metal Performance Shaders (MPS) as a backend for PyTorch enables this and can be used via the new `"mps"` device. -This will map computational graphs and primitives on the MPS Graph framework and tuned kernels provided by MPS. -For more information please refer official documents [Introducing Accelerated PyTorch Training on Mac](https://pytorch.org/blog/introducing-accelerated-pytorch-training-on-mac/) -and [MPS BACKEND](https://pytorch.org/docs/stable/notes/mps.html). - - - -We strongly recommend to install PyTorch >= 1.13 (nightly version at the time of writing) on your MacOS machine. -It has major fixes related to model correctness and performance improvements for transformer based models. -Please refer to https://github.com/pytorch/pytorch/issues/82707 for more details. - - - -**Benefits of Training and Inference using Apple Silicon Chips** - -1. Enables users to train larger networks or batch sizes locally -2. Reduces data retrieval latency and provides the GPU with direct access to the full memory store due to unified memory architecture. -Therefore, improving end-to-end performance. -3. Reduces costs associated with cloud-based development or the need for additional local GPUs. - -**Pre-requisites**: To install torch with mps support, -please follow this nice medium article [GPU-Acceleration Comes to PyTorch on M1 Macs](https://medium.com/towards-data-science/gpu-acceleration-comes-to-pytorch-on-m1-macs-195c399efcc1). - -**Usage**: -`mps` device will be used by default if available similar to the way `cuda` device is used. -Therefore, no action from user is required. -For example, you can run the official Glue text classififcation task (from the root folder) using Apple Silicon GPU with below command: - -```bash -export TASK_NAME=mrpc - -python examples/pytorch/text-classification/run_glue.py \ - --model_name_or_path bert-base-cased \ - --task_name $TASK_NAME \ - --do_train \ - --do_eval \ - --max_seq_length 128 \ - --per_device_train_batch_size 32 \ - --learning_rate 2e-5 \ - --num_train_epochs 3 \ - --output_dir /tmp/$TASK_NAME/ \ - --overwrite_output_dir -``` - -**A few caveats to be aware of** - -1. Some PyTorch operations have not been implemented in mps and will throw an error. -One way to get around that is to set the environment variable `PYTORCH_ENABLE_MPS_FALLBACK=1`, -which will fallback to CPU for these operations. It still throws a UserWarning however. -2. Distributed setups `gloo` and `nccl` are not working with `mps` device. -This means that currently only single GPU of `mps` device type can be used. - -Finally, please, remember that, 🤗 `Trainer` only integrates MPS backend, therefore if you -have any problems or questions with regards to MPS backend usage, please, -file an issue with [PyTorch GitHub](https://github.com/pytorch/pytorch/issues). - Sections that were moved: [ DeepSpeed diff --git a/docs/source/en/perf_train_special.md b/docs/source/en/perf_train_special.md index 48727b24fef3e3..b9bbe32897dbd6 100644 --- a/docs/source/en/perf_train_special.md +++ b/docs/source/en/perf_train_special.md @@ -13,12 +13,51 @@ rendered properly in your Markdown viewer. --> -# Training on Specialized Hardware +# PyTorch training on Apple silicon - +Previously, training models on a Mac was limited to the CPU only. With the release of PyTorch v1.12, you can take advantage of training models with Apple's silicon GPUs for significantly faster performance and training. This is powered in PyTorch by integrating Apple's Metal Performance Shaders (MPS) as a backend. The [MPS backend](https://pytorch.org/docs/stable/notes/mps.html) implements PyTorch operations as custom Metal shaders and places these modules on a `mps` device. - Note: Most of the strategies introduced in the [single GPU section](perf_train_gpu_one) (such as mixed precision training or gradient accumulation) and [multi-GPU section](perf_train_gpu_many) are generic and apply to training models in general so make sure to have a look at it before diving into this section. + + +Some PyTorch operations are not implemented in MPS yet and will throw an error. To avoid this, you should set the environment variable `PYTORCH_ENABLE_MPS_FALLBACK=1` to use the CPU kernels instead (you'll still see a `UserWarning`). + +
+ +If you run into any other errors, please open an issue in the [PyTorch](https://github.com/pytorch/pytorch/issues) repository because the [`Trainer`] only integrates the MPS backend. -This document will be completed soon with information on how to train on specialized hardware. +With the `mps` device set, you can: + +* train larger networks or batch sizes locally +* reduce data retrieval latency because the GPU's unified memory architecture allows direct access to the full memory store +* reduce costs because you don't need to train on cloud-based GPUs or add additional local GPUs + +Get started by making sure you have PyTorch installed. MPS acceleration is supported on macOS 12.3+. + +```bash +pip install torch torchvision torchaudio +``` + +[`TrainingArguments`] uses the `mps` device by default if it's available which means you don't need to explicitly set the device. For example, you can run the [run_glue.py](https://github.com/huggingface/transformers/blob/main/examples/pytorch/text-classification/run_glue.py) script with the MPS backend automatically enabled without making any changes. + +```diff +export TASK_NAME=mrpc + +python examples/pytorch/text-classification/run_glue.py \ + --model_name_or_path bert-base-cased \ + --task_name $TASK_NAME \ +- --use_mps_device \ + --do_train \ + --do_eval \ + --max_seq_length 128 \ + --per_device_train_batch_size 32 \ + --learning_rate 2e-5 \ + --num_train_epochs 3 \ + --output_dir /tmp/$TASK_NAME/ \ + --overwrite_output_dir +``` + +Backends for [distributed setups](https://pytorch.org/docs/stable/distributed.html#backends) like `gloo` and `nccl` are not supported by the `mps` device which means you can only train on a single GPU with the MPS backend. + +You can learn more about the MPS backend in the [Introducing Accelerated PyTorch Training on Mac](https://pytorch.org/blog/introducing-accelerated-pytorch-training-on-mac/) blog post.