From b7f814949a3adab242b0f26dac4d4b2d4ee27ff0 Mon Sep 17 00:00:00 2001 From: Yang Yong Date: Tue, 3 Sep 2024 02:52:50 +0800 Subject: [PATCH] Yongyang (#63) * update doc * update doc --- docs/en/source/advanced/model_test_v2.md | 8 ++++++-- docs/zh_cn/source/advanced/model_test_v2.md | 10 ++++++++-- 2 files changed, 14 insertions(+), 4 deletions(-) diff --git a/docs/en/source/advanced/model_test_v2.md b/docs/en/source/advanced/model_test_v2.md index 2f59f2b3..6e5bd417 100644 --- a/docs/en/source/advanced/model_test_v2.md +++ b/docs/en/source/advanced/model_test_v2.md @@ -72,6 +72,10 @@ According to the opencompass [documentation](https://opencompass.readthedocs.io/ Finally, you can load the above configuration and perform model compression and accuracy testing just like running a regular llmc program. -Note: If the model is too large to fit on a single GPU for evaluation, and multi-GPU evaluation is needed, we support using pipeline parallelism when running opencompass. +## Note: -What you need to do is identify which GPUs are available, add them to CUDA_VISIBLE_DEVICES at the beginning of your run script, and then modify the file pointed to by cfg_path under opencompass, setting the num_gpus to the desired number. +If the model is too large to fit on a single GPU for evaluation, and multi-GPU evaluation is needed, we support using pipeline parallelism when running opencompass. + +What you need to do is: +1. Identify which GPUs are available, add them to CUDA_VISIBLE_DEVICES at the beginning of your run script +2. Modify the file pointed to by cfg_path under opencompass, setting the num_gpus to the desired number. diff --git a/docs/zh_cn/source/advanced/model_test_v2.md b/docs/zh_cn/source/advanced/model_test_v2.md index 883ea3aa..b0f12b6a 100644 --- a/docs/zh_cn/source/advanced/model_test_v2.md +++ b/docs/zh_cn/source/advanced/model_test_v2.md @@ -72,6 +72,12 @@ pip install human-eval 最后你就可以像运行一个正常的llmc程序一样，载入上述的config，进行模型压缩和精度测试 -注意：如果模型太大，单卡评测放不下，需要使用多卡评测精度，我们支持在运行opencompass时使用pipeline parallel。 +## 注意： -你需要做的就是，确定哪些卡是可用的，在你的运行脚本最前面，添加到CUDA_VISIBLE_DEVICES中，然后修改opencompass下的cfg_path指向的文件，将里面的num_gpus设置成你需要的数量，仅此而已。 +如果模型太大，单卡评测放不下，需要使用多卡评测精度，我们支持在运行opencompass时使用pipeline parallel。 + +你需要做的仅仅就是： + +1. 确定哪些卡是可用的，在你的运行脚本最前面，添加到CUDA_VISIBLE_DEVICES中 + +2. 修改opencompass下的cfg_path指向的文件，将里面的num_gpus设置成你需要的数量