This repository has been archived by the owner on Oct 19, 2024. It is now read-only.

[BUG] Balance assignment of free equations in PipeshardParallel #549

Open

merrymercy opened this issue Jun 27, 2022 · 1 comment

Labels

good first issue known bug

Member

merrymercy commented Jun 27, 2022 •

edited

Loading

We should use a round-robin strategy here to assign these free equations. Otherwise, mesh 0 will run out of memory

alpa/alpa/pipeline_parallel/apply_grad.py

Lines 623 to 625 in 8d69815

    
           # all inputs are infered, all outputs are not assigned 
        
           sliced_eqns[0].append(eqn) 
        
           logger.debug(f'{eqn} are arbitrarily assigned')

Member Author

merrymercy commented Jun 27, 2022

ZYHowell added the known bug label

ZYHowell self-assigned this

merrymercy unassigned ZYHowell

merrymercy added the good first issue label

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.