From c41029dd90851cc81bd6c562032f9aee3e925692 Mon Sep 17 00:00:00 2001 From: Keita Watanabe Date: Sun, 7 Apr 2024 10:11:16 +0900 Subject: [PATCH 1/2] Fix typo in 15.gpt-neox README --- 3.test_cases/15.gpt-neox/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/3.test_cases/15.gpt-neox/README.md b/3.test_cases/15.gpt-neox/README.md index 61b3fabf..6c077683 100644 --- a/3.test_cases/15.gpt-neox/README.md +++ b/3.test_cases/15.gpt-neox/README.md @@ -1,6 +1,6 @@ # Pythia GPT-NeoX Test Case -GPT-NeoX is an [EleutherAI](https://www.eleuther.ai)'s library for training large-scale language models on GPUs. This framework is based on NVIDIA's Megatron Language Model](https://github.com/NVIDIA/Megatron-LM) and has been augmented with techniques from [DeepSpeed](https://www.deepspeed.ai as well as some novel optimizations. This test case illustrates how to train [Pythia](https://arxiv.org/abs/2304.01373) model using GPT-Neox. +GPT-NeoX is an [EleutherAI](https://www.eleuther.ai)'s library for training large-scale language models on GPUs. This framework is based on [NVIDIA's Megatron Language Model](https://github.com/NVIDIA/Megatron-LM) and has been augmented with techniques from [DeepSpeed](https://www.deepspeed.ai as well as some novel optimizations. This test case illustrates how to train [Pythia](https://arxiv.org/abs/2304.01373) model using GPT-Neox. ## 1. Preparation From cbcae5d6031649592be91250e63d88704a33009b Mon Sep 17 00:00:00 2001 From: Keita Watanabe Date: Sun, 7 Apr 2024 10:12:24 +0900 Subject: [PATCH 2/2] Update README.md --- 3.test_cases/15.gpt-neox/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/3.test_cases/15.gpt-neox/README.md b/3.test_cases/15.gpt-neox/README.md index 6c077683..77b65d7a 100644 --- a/3.test_cases/15.gpt-neox/README.md +++ b/3.test_cases/15.gpt-neox/README.md @@ -1,6 +1,6 @@ # Pythia GPT-NeoX Test Case -GPT-NeoX is an [EleutherAI](https://www.eleuther.ai)'s library for training large-scale language models on GPUs. This framework is based on [NVIDIA's Megatron Language Model](https://github.com/NVIDIA/Megatron-LM) and has been augmented with techniques from [DeepSpeed](https://www.deepspeed.ai as well as some novel optimizations. This test case illustrates how to train [Pythia](https://arxiv.org/abs/2304.01373) model using GPT-Neox. +GPT-NeoX is an [EleutherAI](https://www.eleuther.ai)'s library for training large-scale language models on GPUs. This framework is based on [NVIDIA's Megatron Language Model](https://github.com/NVIDIA/Megatron-LM) and has been augmented with techniques from [DeepSpeed](https://www.deepspeed.ai) as well as some novel optimizations. This test case illustrates how to train [Pythia](https://arxiv.org/abs/2304.01373) model using GPT-Neox. ## 1. Preparation