Skip to content

Commit

Permalink
Yongyang (#63)
Browse files Browse the repository at this point in the history
* update doc

* update doc
  • Loading branch information
helloyongyang authored Sep 2, 2024
1 parent 0924155 commit b7f8149
Show file tree
Hide file tree
Showing 2 changed files with 14 additions and 4 deletions.
8 changes: 6 additions & 2 deletions docs/en/source/advanced/model_test_v2.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,10 @@ According to the opencompass [documentation](https://opencompass.readthedocs.io/

Finally, you can load the above configuration and perform model compression and accuracy testing just like running a regular llmc program.

Note: If the model is too large to fit on a single GPU for evaluation, and multi-GPU evaluation is needed, we support using pipeline parallelism when running opencompass.
## Note:

What you need to do is identify which GPUs are available, add them to CUDA_VISIBLE_DEVICES at the beginning of your run script, and then modify the file pointed to by cfg_path under opencompass, setting the num_gpus to the desired number.
If the model is too large to fit on a single GPU for evaluation, and multi-GPU evaluation is needed, we support using pipeline parallelism when running opencompass.

What you need to do is:
1. Identify which GPUs are available, add them to CUDA_VISIBLE_DEVICES at the beginning of your run script
2. Modify the file pointed to by cfg_path under opencompass, setting the num_gpus to the desired number.
10 changes: 8 additions & 2 deletions docs/zh_cn/source/advanced/model_test_v2.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,12 @@ pip install human-eval

最后你就可以像运行一个正常的llmc程序一样,载入上述的config,进行模型压缩和精度测试

注意:如果模型太大,单卡评测放不下,需要使用多卡评测精度,我们支持在运行opencompass时使用pipeline parallel。
## 注意:

你需要做的就是,确定哪些卡是可用的,在你的运行脚本最前面,添加到CUDA_VISIBLE_DEVICES中,然后修改opencompass下的cfg_path指向的文件,将里面的num_gpus设置成你需要的数量,仅此而已。
如果模型太大,单卡评测放不下,需要使用多卡评测精度,我们支持在运行opencompass时使用pipeline parallel。

你需要做的仅仅就是:

1. 确定哪些卡是可用的,在你的运行脚本最前面,添加到CUDA_VISIBLE_DEVICES中

2. 修改opencompass下的cfg_path指向的文件,将里面的num_gpus设置成你需要的数量

0 comments on commit b7f8149

Please sign in to comment.