Code

Improve neighbor allreduce#78

Code

Merged

hanbinhu merged 40 commits intomasterBluefog-Lib/bluefog:masterfrom improve_neighbor_allreduceBluefog-Lib/bluefog:improve_neighbor_allreduceCopy head branch name to clipboardApr 11, 2021

+1,233-508

Commits

Commits on Feb 8, 2021

• Fixed the self_weight under emtpy receiving case

  BichengYing

  committed
  61b472c

Commits on Feb 11, 2021

• Enable empty send neighbors and fix HalfTensor for recv_size==0

  

  Hanbin Hu

  committed
  7e8ea66

Commits on Feb 19, 2021

• Fixed the self_weight under emtpy receiving case

  

  BichengYing

  authored and

  Hanbin Hu

  committed
  1d43096

• Enable empty send neighbors and fix HalfTensor for recv_size==0

  

  Hanbin Hu

  committed
  c082ce9

• Merge branch 'improve_neighbor_allreduce' of https://github.com/ybc1991/bluefog into improve_neighbor_allreduce

  

  Hanbin Hu

  committed
  50933fb

• Rename neighbor_weights to src_weights, and send_neighbors to dst_weights for neighbor_allreduce

  

  Hanbin Hu

  committed
  a3365b3

• A script to test existing examples

  

  Hanbin Hu

  committed
  cf17875

• Accept dst_weights as Dict, and reorganize DoNeighborAllreduce

  

  Hanbin Hu

  committed
  923df09

• Reorganize CheckNeighborSendRecvPattern

  

  Hanbin Hu

  committed
  42d0af7

• Fix timeline_ptr for NCCL

  

  Hanbin Hu

  committed
  3184e17

• Fix timeline_ptr for NCCL

  

  Hanbin Hu

  committed
  c85cbd5

• Merge branch 'improve_neighbor_allreduce' of https://github.com/ybc1991/bluefog into improve_neighbor_allreduce

  

  Hanbin Hu

  committed
  8d9fe31

• Put dst_weights information into TensorTableEntry

  

  Hanbin Hu

  committed
  7d60585

Commits on Feb 27, 2021

• First Version of neighbor_allreduce dst_weight, existing problem: Fusion Not Implemented, CUDA data_weight problem

  

  Hanbin Hu

  committed
  79c57b3

• Add some delay after data_weight as a temporary solution

  

  Hanbin Hu

  committed
  29a239c

• CPU Fusion for dst_weighted added

  

  Hanbin Hu

  committed
  97517b3

Commits on Feb 28, 2021

• Merge branch 'master' into improve_neighbor_allreduce

  

  Hanbin Hu

  committed
  364f5fd

Commits on Mar 4, 2021

• Add ReadyEvent for dst_weight for single entry neighbor_allreduce

  

  Hanbin Hu

  committed
  cc9faf4

• Remove const identifier for tensor dtype as it is meaningless

  

  Hanbin Hu

  committed
  7f47ce4

• Add cuda source for scalebuffer

  

  Hanbin Hu

  committed
  d84138b

• Scale buffer to modify itself

  

  Hanbin Hu

  committed
  06c375d

Commits on Mar 5, 2021

• Add .o file to .gitignore

  

  Hanbin Hu

  committed
  e4db92f

• dst_weight using CUDA for fused entry & compile flow in Python setup.py

  

  Hanbin Hu

  committed
  ec2a3fc

• make clean *.o files generated by nvcc

  

  Hanbin Hu

  committed
  adaa191

Commits on Mar 15, 2021

• Add fix for NCCL single entry

  

  Hanbin Hu

  committed
  46e6431

• Make setup.py more robust

  

  Hanbin Hu

  committed
  ae57e75

Commits on Mar 17, 2021

• Add timeout and cuda check

  

  Hanbin Hu

  committed
  62dc99d

• Move test example

  

  Hanbin Hu

  committed
  a836b02

Commits on Mar 18, 2021

• Fix NCCL side dst_weight fusion bug

  

  Hanbin Hu

  committed
  e02a3d0

• Add agg to make matplotlib more stable

  

  Hanbin Hu

  committed
  8e42ca2

Commits on Mar 21, 2021

• Address comments for setup.py

  

  Hanbin Hu

  committed
  e5f8722

• Simpler logic for dst_weighting_enabled and weighted_average_computation

  

  Hanbin Hu

  committed
  9f2f55d

• Better consideration for weight buffer size

  

  Hanbin Hu

  committed
  d7a9310

Commits on Mar 26, 2021

• Merge branch 'master' into improve_neighbor_allreduce

  

  Hanbin Hu

  committed
  2fe9621

• Make src_weights as std::map, and simplify logic for PerformNeighborAllreduceCallback

  

  Hanbin Hu

  committed
  562b231

• Add TODO #80 and #81, and simplify the logic for dst_weight

  

  Hanbin Hu

  committed
  6764efa

• Wrap CheckNeighborSendRecvPattern again

  

  Hanbin Hu

  committed
  1e8db23

• Add two more TODOs

  

  Hanbin Hu

  committed
  3d90087

Commits on Mar 27, 2021

• Address review comments

  

  Hanbin Hu

  committed
  96887b5

Commits on Apr 11, 2021

• Add condition variable to control the loop (#88)

  Show description for 36c073a
  BichengYing

  authored
  36c073a