[npu] change device to accelerator api #5140

oahzxl · 2023-12-01T02:14:41Z

📌 Checklist before creating the PR

I have created an issue for this PR for traceability
The title follows the standard format: [doc/gemini/tensor/...]: A concise description
I have added relevant tags if possible for us to better distinguish different PRs

🚨 Issue number

Link this PR to your issue with words like fixed to automatically close the linked issue upon merge

e.g. fixed #1234, closed #1234, resolved #1234

📝 What does this PR do?

Summarize your work here.
if you have any plots/diagrams/screenshots/tables, please attach them here.

change device to accelerator api

💥 Checklist before requesting a review

I have linked my PR to an issue (instruction)
My issue clearly describes the problem/feature/proposal, with diagrams/charts/table/code if possible
I have performed a self-review of my code
I have added thorough tests.
I have added docstrings for all the functions/methods I implemented

⭐️ Do you enjoy contributing to Colossal-AI?

🌝 Yes, I do.
🌚 No, I don't.

Tell us more if you don't enjoy contributing to Colossal-AI.

github-actions · 2023-12-07T02:31:56Z

The code coverage for the changed files is 49%.

Click me to view the complete report

Name                                                                           Stmts   Miss  Cover
--------------------------------------------------------------------------------------------------
colossalai/__init__.py                                                             8      3    62%
colossalai/accelerator/__init__.py                                                 5      0   100%
colossalai/accelerator/api.py                                                     30      8    73%
colossalai/accelerator/base_accelerator.py                                       100      5    95%
colossalai/accelerator/cpu_accelerator.py                                         94     48    49%
colossalai/accelerator/cuda_accelerator.py                                        97     31    68%
colossalai/accelerator/npu_accelerator.py                                        103     51    50%
colossalai/amp/naive_amp/grad_scaler/base_grad_scaler.py                          31      8    74%
colossalai/amp/naive_amp/grad_scaler/dynamic_grad_scaler.py                       66     24    64%
colossalai/amp/naive_amp/mixed_precision_mixin/fp16.py                            48      0   100%
colossalai/auto_parallel/offload/amp_optimizer.py                                 99     99     0%
colossalai/auto_parallel/offload/solver.py                                       259    259     0%
colossalai/booster/mixed_precision/fp16_torch.py                                  47      2    96%
colossalai/booster/plugin/gemini_plugin.py                                       168     13    92%
colossalai/booster/plugin/hybrid_parallel_plugin.py                              443     49    89%
colossalai/booster/plugin/low_level_zero_plugin.py                               145      9    94%
colossalai/initialize.py                                                          48     23    52%
colossalai/kernel/cuda_native/mha/flash_attn_2.py                                 30      7    77%
colossalai/kernel/cuda_native/mha/mha.py                                          60      8    87%
colossalai/kernel/cuda_native/mha/utils.py                                        52     11    79%
colossalai/kernel/jit/option.py                                                   48     40    17%
colossalai/kernel/npu/__init__.py                                                  2      2     0%
colossalai/kernel/npu/mha/__init__.py                                              2      2     0%
colossalai/kernel/npu/mha/mha.py                                                  28     28     0%
colossalai/kernel/npu/mha/sdpa_attn.py                                             7      7     0%
colossalai/kernel/npu/mha/triangle_attn.py                                        54     54     0%
colossalai/legacy/amp/torch_amp/torch_amp.py                                      36     14    61%
colossalai/legacy/communication/p2p.py                                           131    107    18%
colossalai/legacy/communication/ring.py                                           20     15    25%
colossalai/legacy/communication/utils.py                                          64     51    20%
colossalai/legacy/engine/schedule/_base_schedule.py                               74     55    26%
colossalai/legacy/engine/schedule/_pipeline_schedule.py                          430    397     8%
colossalai/legacy/engine/schedule/_pipeline_schedule_v2.py                        78     78     0%
colossalai/legacy/initialize.py                                                  182    149    18%
colossalai/legacy/nn/layer/colossalai_layer/embedding.py                          30     11    63%
colossalai/legacy/nn/layer/colossalai_layer/normalization.py                      17      5    71%
colossalai/legacy/nn/layer/parallel_1d/layers.py                                 474    378    20%
colossalai/legacy/nn/layer/parallel_2d/_operation.py                             395    309    22%
colossalai/legacy/nn/layer/parallel_2d/layers.py                                 484    407    16%
colossalai/legacy/nn/layer/parallel_2p5d/_operation.py                           431    336    22%
colossalai/legacy/nn/layer/parallel_2p5d/layers.py                               477    401    16%
colossalai/legacy/nn/layer/parallel_3d/layers.py                                 513    431    16%
colossalai/legacy/nn/layer/parallel_sequence/_operation.py                        82     60    27%
colossalai/legacy/nn/layer/vanilla/layers.py                                     147    103    30%
colossalai/legacy/nn/loss/loss_2d.py                                              74     48    35%
colossalai/legacy/nn/loss/loss_2p5d.py                                            74     48    35%
colossalai/legacy/nn/loss/loss_3d.py                                              81     55    32%
colossalai/legacy/trainer/hooks/_metric_hook.py                                  223    223     0%
colossalai/legacy/utils/activation_checkpoint.py                                 151    151     0%
colossalai/legacy/utils/memory.py                                                 85     62    27%
colossalai/legacy/utils/profiler/legacy/comm_profiler.py                         204    204     0%
colossalai/legacy/zero/gemini/stateful_tensor_mgr.py                              67     48    28%
colossalai/legacy/zero/gemini/tensor_placement_policy.py                          82     53    35%
colossalai/legacy/zero/shard_utils/bucket_tensor_shard_strategy.py                32     23    28%
colossalai/legacy/zero/shard_utils/tensor_shard_strategy.py                       38     25    34%
colossalai/legacy/zero/sharded_model/sharded_model_v2.py                         299    239    20%
colossalai/legacy/zero/sharded_model/zero_hook.py                                 73     50    32%
colossalai/moe/routers.py                                                        179     34    81%
colossalai/moe/utils.py                                                          108     26    76%
colossalai/pipeline/schedule/generate.py                                         233    194    17%
colossalai/pipeline/schedule/interleaved_pp.py                                   172     11    94%
colossalai/pipeline/schedule/one_f_one_b.py                                      153     14    91%
colossalai/shardformer/layer/utils.py                                            123     16    87%
colossalai/shardformer/modeling/llama.py                                         223     71    68%
colossalai/testing/utils.py                                                      102     15    85%
colossalai/utils/__init__.py                                                       5      0   100%
colossalai/utils/timer.py                                                         71     46    35%
colossalai/zero/gemini/chunk/chunk.py                                            347     71    80%
colossalai/zero/gemini/chunk/manager.py                                          161     40    75%
colossalai/zero/gemini/gemini_ddp.py                                             455    104    77%
colossalai/zero/gemini/gemini_optimizer.py                                       440    106    76%
colossalai/zero/gemini/memory_tracer/chunk_memstats_collector.py                  17      2    88%
colossalai/zero/gemini/memory_tracer/memory_monitor.py                            72     35    51%
colossalai/zero/gemini/placement_policy.py                                       119     24    80%
colossalai/zero/gemini/utils.py                                                   59     37    37%
colossalai/zero/low_level/low_level_optim.py                                     492     33    93%
tests/test_auto_parallel/test_tensor_shard/test_compatibility_with_gemini.py      77     77     0%
tests/test_booster/test_plugin/test_low_level_zero_plugin.py                      62      6    90%
tests/test_moe/test_grad_handler.py                                               51      1    98%
tests/test_moe/test_kernel.py                                                     57      1    98%
tests/test_moe/test_moe_checkpoint.py                                            108      3    97%
tests/test_moe/test_moe_ep_tp.py                                                 141     21    85%
tests/test_moe/test_moe_group.py                                                  50      2    96%
tests/test_optimizer/test_adam_kernel.py                                          88      1    99%
tests/test_pipeline/test_p2p_communication.py                                     44      1    98%
tests/test_zero/test_gemini/test_chunkv2.py                                       87      1    99%
tests/test_zero/test_gemini/test_fwd_bwd.py                                       76      1    99%
tests/test_zero/test_gemini/test_grad_accum.py                                    99      1    99%
tests/test_zero/test_gemini/test_inference.py                                     88      1    99%
tests/test_zero/test_gemini/test_optim.py                                        116      2    98%
tests/test_zero/test_gemini/test_search.py                                        33      1    97%
tests/test_zero/test_low_level/test_grad_acc.py                                   89      1    99%
--------------------------------------------------------------------------------------------------
TOTAL                                                                          12419   6327    49%

.github/workflows/example_check_on_dispatch.yml

github-actions · 2023-12-08T17:14:32Z

The code coverage for the changed files is 49%.

Click me to view the complete report

Name                                                                           Stmts   Miss  Cover
--------------------------------------------------------------------------------------------------
colossalai/__init__.py                                                             8      3    62%
colossalai/accelerator/__init__.py                                                 5      0   100%
colossalai/accelerator/api.py                                                     30      8    73%
colossalai/accelerator/base_accelerator.py                                       100      5    95%
colossalai/accelerator/cpu_accelerator.py                                         95     48    49%
colossalai/accelerator/cuda_accelerator.py                                        97     31    68%
colossalai/accelerator/npu_accelerator.py                                        103     51    50%
colossalai/amp/naive_amp/grad_scaler/base_grad_scaler.py                          31      8    74%
colossalai/amp/naive_amp/grad_scaler/dynamic_grad_scaler.py                       66     24    64%
colossalai/amp/naive_amp/mixed_precision_mixin/fp16.py                            48      0   100%
colossalai/auto_parallel/offload/amp_optimizer.py                                 99     99     0%
colossalai/auto_parallel/offload/solver.py                                       259    259     0%
colossalai/booster/mixed_precision/fp16_torch.py                                  47      2    96%
colossalai/booster/plugin/gemini_plugin.py                                       168     13    92%
colossalai/booster/plugin/hybrid_parallel_plugin.py                              443     49    89%
colossalai/booster/plugin/low_level_zero_plugin.py                               145      9    94%
colossalai/initialize.py                                                          48     23    52%
colossalai/kernel/cuda_native/mha/flash_attn_2.py                                 30      7    77%
colossalai/kernel/cuda_native/mha/mha.py                                          60      8    87%
colossalai/kernel/cuda_native/mha/utils.py                                        52     11    79%
colossalai/kernel/jit/option.py                                                   48     40    17%
colossalai/kernel/npu/__init__.py                                                  2      2     0%
colossalai/kernel/npu/mha/__init__.py                                              2      2     0%
colossalai/kernel/npu/mha/mha.py                                                  28     28     0%
colossalai/kernel/npu/mha/sdpa_attn.py                                             7      7     0%
colossalai/kernel/npu/mha/triangle_attn.py                                        54     54     0%
colossalai/legacy/amp/torch_amp/torch_amp.py                                      36     14    61%
colossalai/legacy/communication/p2p.py                                           131    107    18%
colossalai/legacy/communication/ring.py                                           20     15    25%
colossalai/legacy/communication/utils.py                                          64     51    20%
colossalai/legacy/engine/schedule/_base_schedule.py                               74     55    26%
colossalai/legacy/engine/schedule/_pipeline_schedule.py                          430    397     8%
colossalai/legacy/engine/schedule/_pipeline_schedule_v2.py                        78     78     0%
colossalai/legacy/initialize.py                                                  182    149    18%
colossalai/legacy/nn/layer/colossalai_layer/embedding.py                          30     11    63%
colossalai/legacy/nn/layer/colossalai_layer/normalization.py                      17      5    71%
colossalai/legacy/nn/layer/parallel_1d/layers.py                                 474    378    20%
colossalai/legacy/nn/layer/parallel_2d/_operation.py                             395    309    22%
colossalai/legacy/nn/layer/parallel_2d/layers.py                                 484    407    16%
colossalai/legacy/nn/layer/parallel_2p5d/_operation.py                           431    336    22%
colossalai/legacy/nn/layer/parallel_2p5d/layers.py                               477    401    16%
colossalai/legacy/nn/layer/parallel_3d/layers.py                                 513    431    16%
colossalai/legacy/nn/layer/parallel_sequence/_operation.py                        82     60    27%
colossalai/legacy/nn/layer/vanilla/layers.py                                     147    103    30%
colossalai/legacy/nn/loss/loss_2d.py                                              74     48    35%
colossalai/legacy/nn/loss/loss_2p5d.py                                            74     48    35%
colossalai/legacy/nn/loss/loss_3d.py                                              81     55    32%
colossalai/legacy/trainer/hooks/_metric_hook.py                                  223    223     0%
colossalai/legacy/utils/activation_checkpoint.py                                 151    151     0%
colossalai/legacy/utils/memory.py                                                 85     62    27%
colossalai/legacy/utils/profiler/legacy/comm_profiler.py                         204    204     0%
colossalai/legacy/zero/gemini/stateful_tensor_mgr.py                              67     48    28%
colossalai/legacy/zero/gemini/tensor_placement_policy.py                          82     53    35%
colossalai/legacy/zero/shard_utils/bucket_tensor_shard_strategy.py                32     23    28%
colossalai/legacy/zero/shard_utils/tensor_shard_strategy.py                       38     25    34%
colossalai/legacy/zero/sharded_model/sharded_model_v2.py                         299    239    20%
colossalai/legacy/zero/sharded_model/zero_hook.py                                 73     50    32%
colossalai/moe/routers.py                                                        179     34    81%
colossalai/moe/utils.py                                                          108     26    76%
colossalai/pipeline/schedule/generate.py                                         233    233     0%
colossalai/pipeline/schedule/interleaved_pp.py                                   172     11    94%
colossalai/pipeline/schedule/one_f_one_b.py                                      153     14    91%
colossalai/shardformer/layer/utils.py                                            123     16    87%
colossalai/shardformer/modeling/llama.py                                         223     71    68%
colossalai/testing/utils.py                                                      102     15    85%
colossalai/utils/__init__.py                                                       5      0   100%
colossalai/utils/timer.py                                                         71     46    35%
colossalai/zero/gemini/chunk/chunk.py                                            347     71    80%
colossalai/zero/gemini/chunk/manager.py                                          161     40    75%
colossalai/zero/gemini/gemini_ddp.py                                             455    104    77%
colossalai/zero/gemini/gemini_optimizer.py                                       440    106    76%
colossalai/zero/gemini/memory_tracer/chunk_memstats_collector.py                  17      2    88%
colossalai/zero/gemini/memory_tracer/memory_monitor.py                            72     35    51%
colossalai/zero/gemini/placement_policy.py                                       119     24    80%
colossalai/zero/gemini/utils.py                                                   59     37    37%
colossalai/zero/low_level/low_level_optim.py                                     492     33    93%
tests/test_auto_parallel/test_tensor_shard/test_compatibility_with_gemini.py      77     77     0%
tests/test_booster/test_plugin/test_low_level_zero_plugin.py                      62      6    90%
tests/test_moe/test_grad_handler.py                                               51      1    98%
tests/test_moe/test_kernel.py                                                     57      1    98%
tests/test_moe/test_moe_checkpoint.py                                            108      3    97%
tests/test_moe/test_moe_ep_tp.py                                                 141     21    85%
tests/test_moe/test_moe_group.py                                                  50      2    96%
tests/test_optimizer/test_adam_kernel.py                                          88     10    89%
tests/test_pipeline/test_p2p_communication.py                                     44      1    98%
tests/test_zero/test_gemini/test_chunkv2.py                                       87      1    99%
tests/test_zero/test_gemini/test_fwd_bwd.py                                       76      1    99%
tests/test_zero/test_gemini/test_grad_accum.py                                    99      1    99%
tests/test_zero/test_gemini/test_inference.py                                     88      1    99%
tests/test_zero/test_gemini/test_optim.py                                        116      2    98%
tests/test_zero/test_gemini/test_search.py                                        33      1    97%
tests/test_zero/test_low_level/test_grad_acc.py                                   89      1    99%
--------------------------------------------------------------------------------------------------
TOTAL                                                                          12420   6375    49%

applications/Chat/coati/trainer/strategies/colossalai.py

applications/Colossal-LLaMA-2/train.py

colossalai/accelerator/__init__.py

colossalai/accelerator/cpu_accelerator.py

colossalai/accelerator/npu_accelerator.py

colossalai/initialize.py

ver217 · 2023-12-11T07:08:49Z

colossalai/legacy/communication/p2p.py

As you modify legacy code, please run test_legacy on your machine and attach screenshot of result here.

colossalai/zero/gemini/chunk/chunk.py

colossalai/zero/low_level/low_level_optim.py

colossalai/accelerator/base_accelerator.py

colossalai/accelerator/cpu_accelerator.py

colossalai/accelerator/cuda_accelerator.py

colossalai/initialize.py

ver217 · 2024-01-09T03:01:38Z

Merged in #5239

update accelerator

6c2d6aa

oahzxl requested a review from a team as a code owner December 1, 2023 02:14

oahzxl added the enhancement New feature or request label Dec 1, 2023

oahzxl and others added 11 commits December 1, 2023 10:59

fix timer

786a7f3

fix amp

4ce6f9d

update

6f4f897

fix

d780f24

update bug

5421bbf

add error raise

317f3bd

fix autocast

18ef3b7

fix set device

b689286

remove doc accelerator

8199ab1

update doc

ff84b95

update doc

9a94bff

oahzxl added 5 commits December 7, 2023 10:58

update doc

aa1a5ac

use nullcontext

9fb062c

update cpu

31184e9

update null context

fc29af2

change time limit for example

99277b5

oahzxl commented Dec 8, 2023

View reviewed changes

.github/workflows/example_check_on_dispatch.yml Show resolved Hide resolved

ver217 requested changes Dec 11, 2023

View reviewed changes

oahzxl added 4 commits December 11, 2023 17:49

udpate

62cbc74

update

1b28d5c

update

64b214f

update

4dc4852

ver217 requested changes Dec 12, 2023

View reviewed changes

ver217 changed the base branch from feature/npu to feature/npu-accelerator January 8, 2024 11:38

ver217 changed the base branch from feature/npu-accelerator to feature/npu January 8, 2024 11:39

ver217 mentioned this pull request Jan 8, 2024

[npu] change device to accelerator api #5239

Merged

10 tasks

ver217 closed this Jan 9, 2024

oahzxl deleted the acc branch January 22, 2024 08:39

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[npu] change device to accelerator api #5140

[npu] change device to accelerator api #5140

oahzxl commented Dec 1, 2023 •

edited

Loading

github-actions bot commented Dec 7, 2023

github-actions bot commented Dec 8, 2023

ver217 Dec 11, 2023

ver217 commented Jan 9, 2024

[npu] change device to accelerator api #5140

[npu] change device to accelerator api #5140

Conversation

oahzxl commented Dec 1, 2023 • edited Loading

📌 Checklist before creating the PR

🚨 Issue number

📝 What does this PR do?

💥 Checklist before requesting a review

⭐️ Do you enjoy contributing to Colossal-AI?

github-actions bot commented Dec 7, 2023

github-actions bot commented Dec 8, 2023

ver217 Dec 11, 2023

Choose a reason for hiding this comment

ver217 commented Jan 9, 2024

oahzxl commented Dec 1, 2023 •

edited

Loading