Skip to content
Benjamin Zaitlen edited this page Dec 4, 2019 · 8 revisions

Two GPUS

UCX

Merge benchmark
--------------------------
Chunk-size  | 100000000
Frac-match  | 0.3
Ignore-size | 1.05 MB
Protocol    | ucx
Device(s)   | 1,2
==========================
Total time  | 2.46 s

Worker-Worker Transfer Rates
==========================
(w1,w2)     | 25% 50% 75% (total nbytes)
--------------------------
(01,02)     | 13.33 GB/s 22.76 GB/s 24.38 GB/s (5.20 GB)
(02,01)     | 11.89 GB/s 19.80 GB/s 25.95 GB/s (5.20 GB)

Notice, peak is 50GB/s (two NVLINKs)

TCP

Merge benchmark
--------------------------
Chunk-size  | 100000000
Frac-match  | 0.3
Ignore-size | 1.05 MB
Protocol    | tcp
Device(s)   | 1,2
==========================
Total time  | 52.97 s
Worker-Worker Transfer Rates
==========================
(w1,w2)     | 25% 50% 75% (total nbytes)
--------------------------
(01,02)     | 100.58 MB/s 108.06 MB/s 113.09 MB/s (5.20 GB)
(02,01)     | 113.35 MB/s 122.18 MB/s 134.94 MB/s (5.20 GB)

Four GPUs on a DGX1

UCX

Merge benchmark
--------------------------
Chunk-size  | 100000000
Frac-match  | 0.3
Ignore-size | 1.05 MB
Protocol    | ucx
Device(s)   | 0,1,2,3
==========================
Total time  | 2.92 s
Worker-Worker Transfer Rates
==========================
(w1,w2)     | 25% 50% 75% (total nbytes)
--------------------------
(00,01)     | 2.07 GB/s 2.64 GB/s 3.22 GB/s (1.40 GB)
(00,02)     | 2.18 GB/s 2.93 GB/s 3.67 GB/s (1.40 GB)
(00,03)     | 2.84 GB/s 4.14 GB/s 17.25 GB/s (3.80 GB)
(01,00)     | 2.80 GB/s 4.12 GB/s 8.02 GB/s (3.80 GB)
(01,02)     | 2.69 GB/s 3.81 GB/s 4.94 GB/s (1.40 GB)
(01,03)     | 2.20 GB/s 2.93 GB/s 3.65 GB/s (1.40 GB)
(02,00)     | 2.14 GB/s 2.60 GB/s 3.06 GB/s (1.40 GB)
(02,01)     | 2.47 GB/s 3.34 GB/s 4.22 GB/s (1.40 GB)
(02,03)     | 2.37 GB/s 3.05 GB/s 3.72 GB/s (1.40 GB)
(03,00)     | 2.39 GB/s 3.19 GB/s 4.00 GB/s (1.40 GB)
(03,01)     | 2.85 GB/s 4.14 GB/s 11.11 GB/s (3.80 GB)
(03,02)     | 2.47 GB/s 3.37 GB/s 4.27 GB/s (1.40 GB)

TCP

Merge benchmark
--------------------------
Chunk-size  | 100000000
Frac-match  | 0.3
Ignore-size | 1.05 MB
Protocol    | tcp
Device(s)   | 0,1,2,3
==========================
Total time  | 69.61 s
Worker-Worker Transfer Rates
==========================
(w1,w2)     | 25% 50% 75% (total nbytes)
--------------------------
(00,01)     | 56.18 MB/s 63.24 MB/s 70.31 MB/s (1.40 GB)
(00,02)     | 50.41 MB/s 56.00 MB/s 69.10 MB/s (3.80 GB)
(00,03)     | 42.66 MB/s 44.67 MB/s 46.69 MB/s (1.40 GB)
(01,00)     | 40.70 MB/s 40.70 MB/s 40.70 MB/s (700.04 MB)
(01,02)     | 56.65 MB/s 60.60 MB/s 64.54 MB/s (1.40 GB)
(01,03)     | 79.28 MB/s 111.50 MB/s 114.57 MB/s (3.80 GB)
(02,00)     | 59.08 MB/s 82.82 MB/s 108.04 MB/s (4.50 GB)
(02,01)     | 67.86 MB/s 72.93 MB/s 101.29 MB/s (2.80 GB)
(02,03)     | 62.73 MB/s 79.31 MB/s 95.89 MB/s (1.40 GB)
(03,00)     | 47.00 MB/s 53.38 MB/s 59.76 MB/s (1.40 GB)
(03,01)     | 45.98 MB/s 46.76 MB/s 47.54 MB/s (1.40 GB)
(03,02)     | 43.86 MB/s 51.91 MB/s 81.22 MB/s (3.80 GB)

Eight GPUs

UCX

UCX_SOCKADDR_TLS_PRIORITY=sockcm UCX_TLS=tcp,cuda_copy,cuda_ipc,sockcm python local_cudf_merge.py -p ucx -d 0,1,2,3,4,5,6,7

Merge benchmark
--------------------------
Chunk-size  | 1000000
Frac-match  | 0.3
Ignore-size | 1.05 MB
Protocol    | ucx
Device(s)   | 0,1,2,3,4,5,6,7
==========================
Total time  | 1.87 s
==========================
Worker-Worker Transfer Rates
(w1,w2)     | 25% 50% 75% (total nbytes)
--------------------------
(00,01)     | 8.83 MB/s 8.83 MB/s 8.83 MB/s (7.01 MB)
(00,02)     | 166.70 MB/s 324.41 MB/s 482.13 MB/s (31.07 MB)
(00,03)     | 9.24 MB/s 9.24 MB/s 9.24 MB/s (7.02 MB)
(00,04)     | 9.05 MB/s 9.05 MB/s 9.05 MB/s (7.00 MB)
(00,05)     | 9.77 MB/s 9.77 MB/s 9.77 MB/s (6.99 MB)
(00,06)     | 12.10 MB/s 12.10 MB/s 12.10 MB/s (7.00 MB)
(00,07)     | 9.15 MB/s 9.15 MB/s 9.15 MB/s (7.01 MB)
(01,00)     | 9.21 MB/s 9.57 MB/s 9.94 MB/s (6.99 MB)
(01,02)     | 9.73 MB/s 11.63 MB/s 13.53 MB/s (6.98 MB)
(01,03)     | 11.04 MB/s 11.23 MB/s 11.42 MB/s (6.99 MB)
(01,04)     | 9.62 MB/s 11.47 MB/s 13.33 MB/s (7.01 MB)
(01,05)     | 8.98 MB/s 9.02 MB/s 9.05 MB/s (7.00 MB)
(01,06)     | 9.18 MB/s 9.35 MB/s 9.52 MB/s (7.00 MB)
(01,07)     | 9.47 MB/s 10.23 MB/s 46.52 MB/s (30.97 MB)
(02,00)     | 9.65 MB/s 9.65 MB/s 9.65 MB/s (7.00 MB)
(02,01)     | 9.65 MB/s 10.52 MB/s 11.39 MB/s (6.99 MB)
(02,03)     | 9.54 MB/s 10.39 MB/s 11.23 MB/s (6.99 MB)
(02,04)     | 8.94 MB/s 9.81 MB/s 10.68 MB/s (7.00 MB)
(02,05)     | 8.86 MB/s 9.60 MB/s 10.33 MB/s (7.01 MB)
(02,06)     | 10.34 MB/s 10.89 MB/s 139.64 MB/s (31.00 MB)
(02,07)     | 10.81 MB/s 11.18 MB/s 11.55 MB/s (7.01 MB)
(03,00)     | 10.36 MB/s 10.36 MB/s 10.36 MB/s (7.01 MB)
(03,01)     | 10.17 MB/s 10.17 MB/s 10.17 MB/s (7.00 MB)
(03,02)     | 10.47 MB/s 10.47 MB/s 10.47 MB/s (7.01 MB)
(03,04)     | 10.24 MB/s 10.24 MB/s 10.24 MB/s (6.99 MB)
(03,05)     | 11.20 MB/s 11.20 MB/s 11.20 MB/s (7.01 MB)
(03,06)     | 13.25 MB/s 13.25 MB/s 13.25 MB/s (7.01 MB)
(03,07)     | 9.41 MB/s 9.41 MB/s 9.41 MB/s (7.00 MB)
(04,00)     | 9.71 MB/s 10.36 MB/s 11.01 MB/s (6.98 MB)
(04,01)     | 11.57 MB/s 13.56 MB/s 15.56 MB/s (6.99 MB)
(04,02)     | 9.52 MB/s 11.42 MB/s 13.31 MB/s (7.00 MB)
(04,03)     | 12.43 MB/s 13.26 MB/s 14.09 MB/s (7.01 MB)
(04,05)     | 9.20 MB/s 9.93 MB/s 10.66 MB/s (6.99 MB)
(04,06)     | 9.74 MB/s 10.24 MB/s 10.73 MB/s (6.98 MB)
(04,07)     | 9.63 MB/s 10.26 MB/s 10.89 MB/s (6.98 MB)
(05,00)     | 8.53 MB/s 8.71 MB/s 8.88 MB/s (7.00 MB)
(05,01)     | 9.10 MB/s 9.56 MB/s 10.01 MB/s (7.00 MB)
(05,02)     | 9.08 MB/s 9.19 MB/s 9.29 MB/s (7.00 MB)
(05,03)     | 11.21 MB/s 12.16 MB/s 13.11 MB/s (7.00 MB)
(05,04)     | 8.87 MB/s 9.42 MB/s 9.98 MB/s (7.00 MB)
(05,06)     | 9.08 MB/s 9.13 MB/s 9.17 MB/s (7.01 MB)
(05,07)     | 9.08 MB/s 9.62 MB/s 10.16 MB/s (6.99 MB)
(06,00)     | 9.81 MB/s 10.57 MB/s 11.33 MB/s (7.00 MB)
(06,01)     | 9.47 MB/s 9.74 MB/s 136.37 MB/s (31.01 MB)
(06,02)     | 8.94 MB/s 9.38 MB/s 9.83 MB/s (6.99 MB)
(06,03)     | 10.73 MB/s 11.16 MB/s 11.59 MB/s (7.00 MB)
(06,04)     | 9.39 MB/s 10.17 MB/s 10.95 MB/s (7.01 MB)
(06,05)     | 9.15 MB/s 9.62 MB/s 10.08 MB/s (6.99 MB)
(06,07)     | 9.31 MB/s 9.32 MB/s 9.32 MB/s (7.01 MB)
(07,00)     | 9.39 MB/s 10.07 MB/s 77.19 MB/s (31.00 MB)
(07,01)     | 9.12 MB/s 9.34 MB/s 9.57 MB/s (7.00 MB)
(07,02)     | 8.90 MB/s 9.16 MB/s 9.41 MB/s (6.99 MB)
(07,03)     | 9.31 MB/s 9.53 MB/s 9.75 MB/s (6.99 MB)
(07,04)     | 8.97 MB/s 9.90 MB/s 10.83 MB/s (6.99 MB)
(07,05)     | 9.20 MB/s 9.43 MB/s 9.67 MB/s (6.99 MB)
(07,06)     | 9.19 MB/s 9.54 MB/s 9.90 MB/s (7.00 MB)
Clone this wiki locally