-
Notifications
You must be signed in to change notification settings - Fork 60
Benchmarks
Benjamin Zaitlen edited this page Dec 4, 2019
·
8 revisions
UCX
Merge benchmark
--------------------------
Chunk-size | 100000000
Frac-match | 0.3
Ignore-size | 1.05 MB
Protocol | ucx
Device(s) | 1,2
==========================
Total time | 2.46 s
Worker-Worker Transfer Rates
==========================
(w1,w2) | 25% 50% 75% (total nbytes)
--------------------------
(01,02) | 13.33 GB/s 22.76 GB/s 24.38 GB/s (5.20 GB)
(02,01) | 11.89 GB/s 19.80 GB/s 25.95 GB/s (5.20 GB)
Notice, peak is 50GB/s (two NVLINKs)
TCP
Merge benchmark
--------------------------
Chunk-size | 100000000
Frac-match | 0.3
Ignore-size | 1.05 MB
Protocol | tcp
Device(s) | 1,2
==========================
Total time | 52.97 s
Worker-Worker Transfer Rates
==========================
(w1,w2) | 25% 50% 75% (total nbytes)
--------------------------
(01,02) | 100.58 MB/s 108.06 MB/s 113.09 MB/s (5.20 GB)
(02,01) | 113.35 MB/s 122.18 MB/s 134.94 MB/s (5.20 GB)
UCX
Merge benchmark
--------------------------
Chunk-size | 100000000
Frac-match | 0.3
Ignore-size | 1.05 MB
Protocol | ucx
Device(s) | 0,1,2,3
==========================
Total time | 2.92 s
Worker-Worker Transfer Rates
==========================
(w1,w2) | 25% 50% 75% (total nbytes)
--------------------------
(00,01) | 2.07 GB/s 2.64 GB/s 3.22 GB/s (1.40 GB)
(00,02) | 2.18 GB/s 2.93 GB/s 3.67 GB/s (1.40 GB)
(00,03) | 2.84 GB/s 4.14 GB/s 17.25 GB/s (3.80 GB)
(01,00) | 2.80 GB/s 4.12 GB/s 8.02 GB/s (3.80 GB)
(01,02) | 2.69 GB/s 3.81 GB/s 4.94 GB/s (1.40 GB)
(01,03) | 2.20 GB/s 2.93 GB/s 3.65 GB/s (1.40 GB)
(02,00) | 2.14 GB/s 2.60 GB/s 3.06 GB/s (1.40 GB)
(02,01) | 2.47 GB/s 3.34 GB/s 4.22 GB/s (1.40 GB)
(02,03) | 2.37 GB/s 3.05 GB/s 3.72 GB/s (1.40 GB)
(03,00) | 2.39 GB/s 3.19 GB/s 4.00 GB/s (1.40 GB)
(03,01) | 2.85 GB/s 4.14 GB/s 11.11 GB/s (3.80 GB)
(03,02) | 2.47 GB/s 3.37 GB/s 4.27 GB/s (1.40 GB)
TCP
Merge benchmark
--------------------------
Chunk-size | 100000000
Frac-match | 0.3
Ignore-size | 1.05 MB
Protocol | tcp
Device(s) | 0,1,2,3
==========================
Total time | 69.61 s
Worker-Worker Transfer Rates
==========================
(w1,w2) | 25% 50% 75% (total nbytes)
--------------------------
(00,01) | 56.18 MB/s 63.24 MB/s 70.31 MB/s (1.40 GB)
(00,02) | 50.41 MB/s 56.00 MB/s 69.10 MB/s (3.80 GB)
(00,03) | 42.66 MB/s 44.67 MB/s 46.69 MB/s (1.40 GB)
(01,00) | 40.70 MB/s 40.70 MB/s 40.70 MB/s (700.04 MB)
(01,02) | 56.65 MB/s 60.60 MB/s 64.54 MB/s (1.40 GB)
(01,03) | 79.28 MB/s 111.50 MB/s 114.57 MB/s (3.80 GB)
(02,00) | 59.08 MB/s 82.82 MB/s 108.04 MB/s (4.50 GB)
(02,01) | 67.86 MB/s 72.93 MB/s 101.29 MB/s (2.80 GB)
(02,03) | 62.73 MB/s 79.31 MB/s 95.89 MB/s (1.40 GB)
(03,00) | 47.00 MB/s 53.38 MB/s 59.76 MB/s (1.40 GB)
(03,01) | 45.98 MB/s 46.76 MB/s 47.54 MB/s (1.40 GB)
(03,02) | 43.86 MB/s 51.91 MB/s 81.22 MB/s (3.80 GB)
UCX_SOCKADDR_TLS_PRIORITY=sockcm UCX_TLS=tcp,cuda_copy,cuda_ipc,sockcm python local_cudf_merge.py -p ucx -d 0,1,2,3,4,5,6,7
Merge benchmark
--------------------------
Chunk-size | 1000000
Frac-match | 0.3
Ignore-size | 1.05 MB
Protocol | ucx
Device(s) | 0,1,2,3,4,5,6,7
==========================
Total time | 1.87 s
==========================
Worker-Worker Transfer Rates
(w1,w2) | 25% 50% 75% (total nbytes)
--------------------------
(00,01) | 8.83 MB/s 8.83 MB/s 8.83 MB/s (7.01 MB)
(00,02) | 166.70 MB/s 324.41 MB/s 482.13 MB/s (31.07 MB)
(00,03) | 9.24 MB/s 9.24 MB/s 9.24 MB/s (7.02 MB)
(00,04) | 9.05 MB/s 9.05 MB/s 9.05 MB/s (7.00 MB)
(00,05) | 9.77 MB/s 9.77 MB/s 9.77 MB/s (6.99 MB)
(00,06) | 12.10 MB/s 12.10 MB/s 12.10 MB/s (7.00 MB)
(00,07) | 9.15 MB/s 9.15 MB/s 9.15 MB/s (7.01 MB)
(01,00) | 9.21 MB/s 9.57 MB/s 9.94 MB/s (6.99 MB)
(01,02) | 9.73 MB/s 11.63 MB/s 13.53 MB/s (6.98 MB)
(01,03) | 11.04 MB/s 11.23 MB/s 11.42 MB/s (6.99 MB)
(01,04) | 9.62 MB/s 11.47 MB/s 13.33 MB/s (7.01 MB)
(01,05) | 8.98 MB/s 9.02 MB/s 9.05 MB/s (7.00 MB)
(01,06) | 9.18 MB/s 9.35 MB/s 9.52 MB/s (7.00 MB)
(01,07) | 9.47 MB/s 10.23 MB/s 46.52 MB/s (30.97 MB)
(02,00) | 9.65 MB/s 9.65 MB/s 9.65 MB/s (7.00 MB)
(02,01) | 9.65 MB/s 10.52 MB/s 11.39 MB/s (6.99 MB)
(02,03) | 9.54 MB/s 10.39 MB/s 11.23 MB/s (6.99 MB)
(02,04) | 8.94 MB/s 9.81 MB/s 10.68 MB/s (7.00 MB)
(02,05) | 8.86 MB/s 9.60 MB/s 10.33 MB/s (7.01 MB)
(02,06) | 10.34 MB/s 10.89 MB/s 139.64 MB/s (31.00 MB)
(02,07) | 10.81 MB/s 11.18 MB/s 11.55 MB/s (7.01 MB)
(03,00) | 10.36 MB/s 10.36 MB/s 10.36 MB/s (7.01 MB)
(03,01) | 10.17 MB/s 10.17 MB/s 10.17 MB/s (7.00 MB)
(03,02) | 10.47 MB/s 10.47 MB/s 10.47 MB/s (7.01 MB)
(03,04) | 10.24 MB/s 10.24 MB/s 10.24 MB/s (6.99 MB)
(03,05) | 11.20 MB/s 11.20 MB/s 11.20 MB/s (7.01 MB)
(03,06) | 13.25 MB/s 13.25 MB/s 13.25 MB/s (7.01 MB)
(03,07) | 9.41 MB/s 9.41 MB/s 9.41 MB/s (7.00 MB)
(04,00) | 9.71 MB/s 10.36 MB/s 11.01 MB/s (6.98 MB)
(04,01) | 11.57 MB/s 13.56 MB/s 15.56 MB/s (6.99 MB)
(04,02) | 9.52 MB/s 11.42 MB/s 13.31 MB/s (7.00 MB)
(04,03) | 12.43 MB/s 13.26 MB/s 14.09 MB/s (7.01 MB)
(04,05) | 9.20 MB/s 9.93 MB/s 10.66 MB/s (6.99 MB)
(04,06) | 9.74 MB/s 10.24 MB/s 10.73 MB/s (6.98 MB)
(04,07) | 9.63 MB/s 10.26 MB/s 10.89 MB/s (6.98 MB)
(05,00) | 8.53 MB/s 8.71 MB/s 8.88 MB/s (7.00 MB)
(05,01) | 9.10 MB/s 9.56 MB/s 10.01 MB/s (7.00 MB)
(05,02) | 9.08 MB/s 9.19 MB/s 9.29 MB/s (7.00 MB)
(05,03) | 11.21 MB/s 12.16 MB/s 13.11 MB/s (7.00 MB)
(05,04) | 8.87 MB/s 9.42 MB/s 9.98 MB/s (7.00 MB)
(05,06) | 9.08 MB/s 9.13 MB/s 9.17 MB/s (7.01 MB)
(05,07) | 9.08 MB/s 9.62 MB/s 10.16 MB/s (6.99 MB)
(06,00) | 9.81 MB/s 10.57 MB/s 11.33 MB/s (7.00 MB)
(06,01) | 9.47 MB/s 9.74 MB/s 136.37 MB/s (31.01 MB)
(06,02) | 8.94 MB/s 9.38 MB/s 9.83 MB/s (6.99 MB)
(06,03) | 10.73 MB/s 11.16 MB/s 11.59 MB/s (7.00 MB)
(06,04) | 9.39 MB/s 10.17 MB/s 10.95 MB/s (7.01 MB)
(06,05) | 9.15 MB/s 9.62 MB/s 10.08 MB/s (6.99 MB)
(06,07) | 9.31 MB/s 9.32 MB/s 9.32 MB/s (7.01 MB)
(07,00) | 9.39 MB/s 10.07 MB/s 77.19 MB/s (31.00 MB)
(07,01) | 9.12 MB/s 9.34 MB/s 9.57 MB/s (7.00 MB)
(07,02) | 8.90 MB/s 9.16 MB/s 9.41 MB/s (6.99 MB)
(07,03) | 9.31 MB/s 9.53 MB/s 9.75 MB/s (6.99 MB)
(07,04) | 8.97 MB/s 9.90 MB/s 10.83 MB/s (6.99 MB)
(07,05) | 9.20 MB/s 9.43 MB/s 9.67 MB/s (6.99 MB)
(07,06) | 9.19 MB/s 9.54 MB/s 9.90 MB/s (7.00 MB)