Add markdown documentation

hhu-bsinfo · Jan 4, 2021 · e1c21c1 · e1c21c1
1 parent 7ac8b5f
commit e1c21c1
Show file tree

Hide file tree

Showing 21 changed files with 1,569 additions and 2 deletions.
diff --git a/README.md b/README.md
@@ -23,6 +23,38 @@ It is intended to benchmark fundamental point-to-point connections by measuring
 
 Observatory provides a single lean interface, that combines messaging, as well as RDMA operations, so that implementations for different libaries can easily be developed. To configure a set of benchmark runs, a JSON file can be used.
 
+## Build and Run Instructions
+
+Observatory offers a [Java](https://github.com/hhu-bsinfo/observatory/tree/development/java) and a [C++](https://github.com/hhu-bsinfo/observatory/tree/development/cpp) implementation. For instructions on how to build and run each implemenation, see the README file in the corresponding subdirectory.
+
+## Configuration
+
+Observatory uses a simple server-client architecture to setup a connection between two nodes. It can be configured with the following command line arguments:
+
+ - `--server, -s`: Configure this instance as a server and wait for a client instance to connect.
+ - `--remote, -r <address>:<port>`: Connect this instance to a remote server.
+ - `--address, -a <address>:<port>`: Use a specific address to bind this instance to (Default: `0.0.0.0:2998`).
+ - `--retries, -t <number>`: Specify the amount of retries after a failed connection attempt (Default: 10).
+ - `--output, -o <path>`: Specify the path to where the result files shall be written (Default: `./result/`).
+ - `--config, -c <path>`: Specify configuration file to be used.
+
+Observatory uses a JSON-file to configure benchmark runs, which specifies the following parameters. For more details and examples, see the [examples foder](https://github.com/hhu-bsinfo/observatory/blob/master/example/config).
+
+ - `resultName`: The name to use in the result files.
+ - `parameters`: An array of key-value tuples, that are defined by each binding. See the README file in each binding's subdirectory for further details.
+ - `detector`: Configuration parameters for the `jDetector` libraries, which measures raw data throughput by reading the HCA's hardware counters. When running the Java version of Observatory, make sure to have the native library `libdetectorJNI.so` installed, before enabling this feature.
+   * `enabled`: Boolean switch to enable/disable the use of `jDetector`.
+   * `deviceNumber`: The device number to use for reading the hardware counters (in most cases, `0` should work fine).
+   * `mode`: Either `mad` to utilize the `ibmad`-library (needs root privileges), or `compat` to read the hardware counters from the local filesystem. If you are unsure, use `compat`, since it will work fine for Observatory's use case. For more details, see the [jDetector repository](https://github.com/hhu-bsinfo/jdetector).
+ - `operations`:An array of objects, describing the benchmark operations to  execute. An operation is defined as a set of iterations, where each iteration describes a single benchmark run.
+   * `name`: The name of the benchmark operation. Valid names are `MessagingThroughput`, `MessagingLatency`, `MessagingPingPong`, `RdmaWriteThroughput`, `RdmaWriteLatency`, `RdmaReadThroughput`, `RdmaReadLatency`.
+   * `modes`: An array containing the benchmark modes to execute. Valid modes are `unidirectional` and `bidirectional`. Each benchmark operation will be executed once for each mode.
+   * `repetitions`: The amount of times each iteration shall be repeated. This useful, if one plans on calculating average values and deviations afterwards.
+   * `iterations`: An array of objects describing the benchmark iterations.
+     - `size`: The buffer size to be used (e.g. message size).
+     - `count`: The amount of times the buffer shall be transferred.
+     - `warmUp`: The amount of warm up iterations to execute. A warm up iteration is exactly the same as a normal iteration without measuring time. If you do not want to execute warm up iterations, just set this value to `0`.
+
 ## Architecture
 
 <p align="center">

diff --git a/cpp/README.md b/cpp/README.md
@@ -0,0 +1,38 @@
+# Observatory C++ implementation
+
+This is the C++ implementation of Observatory, written in C++-11 and using CMake as build system.
+
+## Build instructions
+
+Observatory depends on [nlohmann_json](https://github.com/nlohmann/json), [log4cpp](http://log4cpp.sourceforge.net/). Make sure to have these installed, before building Observatory.
+
+Each Observatory binding can be build into a standalone application. The bindings are located in the subdirectory `bindings`. To build a binding you can run the following command from the project's root directory:
+
+```
+./build.sh <binding-name>
+```
+
+For example, to build the `socket-binding` you can run:
+
+```
+./build.sh socket-binding
+```
+
+It is also possible to build multiple implementations at once, by providing multiple names, seperated by spaces.
+
+## Run instructions
+
+After building a binding you need to change into the build output directory:
+
+```
+cd  build/<binding-subdirectory>/
+```
+
+After that, simply run `./bin/observatory/`.
+
+## Bindings
+
+The following bindings are implemented for the C++ version of Observatory:
+ - [Dummy Binding](https://github.com/hhu-bsinfo/observatory/tree/development/cpp/src/bindings/dummy-binding): An example binding, that can be used as a starting point for new implementations.
+ - [Socket Binding](https://github.com/hhu-bsinfo/observatory/tree/development/cpp/src/bindings/socket-binding): This implementation uses plain sockets and does **not** support RDMA.
+ - [Verbs Binding](https://github.com/hhu-bsinfo/observatory/tree/development/cpp/src/bindings/verbs-binding): This implementation uses the standard native ibverbs library access InfiniBand hardware.
diff --git a/cpp/src/bindings/dummy-binding/README.md b/cpp/src/bindings/dummy-binding/README.md
@@ -0,0 +1,20 @@
+# Dummy binding for Observatory (C++)
+
+This binding shall serve as an example and a starting point for new implementations. It is not functional, as each implemented method just returns `Status::NOT_IMPLEMENTED`.
+
+## Build instructions
+
+To build Observatory using this binding, run the following command from the `cpp` directory:
+
+```
+./build.sh dummy-binding
+```
+
+## Run instructions
+
+To run Observatory using this binding, change into the build output directory and run the `observatory` executable:
+
+```
+cd build/dummy-binding/
+./bin/observatory
+```
diff --git a/cpp/src/bindings/socket-binding/README.md b/cpp/src/bindings/socket-binding/README.md
@@ -0,0 +1,24 @@
+# Socket binding for Observatory (C++)
+
+This binding does not access InfiniBand hardware directly, but uses plain sockets. Its purpose is to be used with InfiniBand solutions, that intercept socket traffic and redirect it over an InfiBand HCA. It does not support RDMA benchmarks.
+
+The socket binding has been tested with the following InfiniBand solutions:
+ - [IP over InfiniBand](https://www.ietf.org/rfc/rfc4392.txt)
+ - [libvma](https://github.com/Mellanox/libvma/) (by Mellanox)
+
+## Build instructions
+
+To build Observatory using this binding, run the following command from the `cpp` directory:
+
+```
+./build.sh socket-binding
+```
+
+## Run instructions
+
+To run Observatory using this binding, change into the build output directory and run the `observatory` executable:
+
+```
+cd build/socket-binding/
+./bin/observatory
+```
diff --git a/cpp/src/bindings/verbs-binding/README.md b/cpp/src/bindings/verbs-binding/README.md
@@ -0,0 +1,20 @@
+# Socket binding for Observatory (C++)
+
+This binding uses the standard native ibverbs library (availabe in the [rdma-core](https://github.com/linux-rdma/rdma-core) package) to access InfiniBand hardware. Its purpose is to get baseline results, that other implementations can be compared against.
+
+## Build instructions
+
+To build Observatory using this binding, run the following command from the `cpp` directory:
+
+```
+./build.sh verbs-binding
+```
+
+## Run instructions
+
+To run Observatory using this binding, change into the build output directory and run the `observatory` executable:
+
+```
+cd build/verbs-binding/
+./bin/observatory
+```
diff --git a/example/config/cpp/ibverbs/socket.json b/example/config/cpp/ibverbs/socket.json
@@ -0,0 +1,212 @@
+{
+  "className": "Verbs::Benchmark",
+  "resultName": "ibverbs",
+  "parameters": [
+    { "key":  "deviceNumber", "value":  "0" },
+    { "key":  "portNumber", "value":  "1" },
+    { "key": "queueSize", "value": "200" }
+  ],
+  "detector": {
+    "enabled": true,
+    "deviceNumber": 0,
+    "mode": "compat"
+  },
+  "operations": [
+    {
+      "name": "MessagingThroughput",
+      "modes": [ "unidirectional", "bidirectional" ],
+      "repetitions": 3,
+      "iterations": [
+        { "size": 1, "count": 100000000, "warmUp": 1000000 },
+        { "size": 2, "count": 100000000, "warmUp": 1000000 },
+        { "size": 4, "count": 100000000, "warmUp": 1000000 },
+        { "size": 8, "count": 100000000, "warmUp": 1000000 },
+        { "size": 16, "count": 100000000, "warmUp": 1000000 },
+        { "size": 32, "count": 100000000, "warmUp": 1000000 },
+        { "size": 64, "count": 100000000, "warmUp": 1000000 },
+        { "size": 128, "count": 100000000, "warmUp": 1000000 },
+        { "size": 256, "count": 100000000, "warmUp": 1000000 },
+        { "size": 512, "count": 100000000, "warmUp": 1000000 },
+        { "size": 1024, "count": 100000000, "warmUp": 1000000 },
+        { "size": 2048, "count": 100000000, "warmUp": 1000000 },
+        { "size": 4096, "count": 100000000, "warmUp": 1000000 },
+        { "size": 8192, "count": 50000000, "warmUp": 500000 },
+        { "size": 16384, "count": 25000000, "warmUp": 500000 },
+        { "size": 32768, "count": 12500000, "warmUp": 500000 },
+        { "size": 65536, "count": 6250000, "warmUp": 250000 },
+        { "size": 131072, "count": 3125000, "warmUp": 250000 },
+        { "size": 262144, "count": 1562500, "warmUp": 250000 },
+        { "size": 524288, "count": 781250, "warmUp": 125000 },
+        { "size": 1048576, "count": 390625, "warmUp": 125000 }
+      ]
+    },
+    {
+      "name": "RdmaWriteThroughput",
+      "modes": [ "unidirectional", "bidirectional" ],
+      "repetitions": 3,
+      "iterations": [
+        { "size": 1, "count": 100000000, "warmUp": 1000000 },
+        { "size": 2, "count": 100000000, "warmUp": 1000000 },
+        { "size": 4, "count": 100000000, "warmUp": 1000000 },
+        { "size": 8, "count": 100000000, "warmUp": 1000000 },
+        { "size": 16, "count": 100000000, "warmUp": 1000000 },
+        { "size": 32, "count": 100000000, "warmUp": 1000000 },
+        { "size": 64, "count": 100000000, "warmUp": 1000000 },
+        { "size": 128, "count": 100000000, "warmUp": 1000000 },
+        { "size": 256, "count": 100000000, "warmUp": 1000000 },
+        { "size": 512, "count": 100000000, "warmUp": 1000000 },
+        { "size": 1024, "count": 100000000, "warmUp": 1000000 },
+        { "size": 2048, "count": 100000000, "warmUp": 1000000 },
+        { "size": 4096, "count": 100000000, "warmUp": 1000000 },
+        { "size": 8192, "count": 50000000, "warmUp": 500000 },
+        { "size": 16384, "count": 25000000, "warmUp": 500000 },
+        { "size": 32768, "count": 12500000, "warmUp": 500000 },
+        { "size": 65536, "count": 6250000, "warmUp": 250000 },
+        { "size": 131072, "count": 3125000, "warmUp": 250000 },
+        { "size": 262144, "count": 1562500, "warmUp": 250000 },
+        { "size": 524288, "count": 781250, "warmUp": 125000 },
+        { "size": 1048576, "count": 390625, "warmUp": 125000 }
+      ]
+    },
+    {
+      "name": "RdmaReadThroughput",
+      "modes": [ "unidirectional", "bidirectional" ],
+      "repetitions": 3,
+      "iterations": [
+        { "size": 1, "count": 100000000, "warmUp": 1000000 },
+        { "size": 2, "count": 100000000, "warmUp": 1000000 },
+        { "size": 4, "count": 100000000, "warmUp": 1000000 },
+        { "size": 8, "count": 100000000, "warmUp": 1000000 },
+        { "size": 16, "count": 100000000, "warmUp": 1000000 },
+        { "size": 32, "count": 100000000, "warmUp": 1000000 },
+        { "size": 64, "count": 100000000, "warmUp": 1000000 },
+        { "size": 128, "count": 100000000, "warmUp": 1000000 },
+        { "size": 256, "count": 100000000, "warmUp": 1000000 },
+        { "size": 512, "count": 100000000, "warmUp": 1000000 },
+        { "size": 1024, "count": 100000000, "warmUp": 1000000 },
+        { "size": 2048, "count": 100000000, "warmUp": 1000000 },
+        { "size": 4096, "count": 100000000, "warmUp": 1000000 },
+        { "size": 8192, "count": 50000000, "warmUp": 500000 },
+        { "size": 16384, "count": 25000000, "warmUp": 500000 },
+        { "size": 32768, "count": 12500000, "warmUp": 500000 },
+        { "size": 65536, "count": 6250000, "warmUp": 250000 },
+        { "size": 131072, "count": 3125000, "warmUp": 250000 },
+        { "size": 262144, "count": 1562500, "warmUp": 250000 },
+        { "size": 524288, "count": 781250, "warmUp": 125000 },
+        { "size": 1048576, "count": 390625, "warmUp": 125000 }
+      ]
+    },
+    {
+      "name": "MessagingLatency",
+      "modes": [ "unidirectional" ],
+      "repetitions": 3,
+      "iterations": [
+        { "size": 1, "count": 10000000, "warmUp": 100000 },
+        { "size": 2, "count": 10000000, "warmUp": 100000 },
+        { "size": 4, "count": 10000000, "warmUp": 100000 },
+        { "size": 8, "count": 10000000, "warmUp": 100000 },
+        { "size": 16, "count": 10000000, "warmUp": 100000 },
+        { "size": 32, "count": 10000000, "warmUp": 100000 },
+        { "size": 64, "count": 10000000, "warmUp": 100000 },
+        { "size": 128, "count": 10000000, "warmUp": 100000 },
+        { "size": 256, "count": 10000000, "warmUp": 100000 },
+        { "size": 512, "count": 10000000, "warmUp": 100000 },
+        { "size": 1024, "count": 10000000, "warmUp": 100000 },
+        { "size": 2048, "count": 10000000, "warmUp": 100000 },
+        { "size": 4096, "count": 10000000, "warmUp": 100000 },
+        { "size": 8192, "count": 5000000, "warmUp": 50000 },
+        { "size": 16384, "count": 2500000, "warmUp": 50000 },
+        { "size": 32768, "count": 1250000, "warmUp": 50000 },
+        { "size": 65536, "count": 625000, "warmUp": 25000 },
+        { "size": 131072, "count": 312500, "warmUp": 25000 },
+        { "size": 262144, "count": 156250, "warmUp": 25000 },
+        { "size": 524288, "count": 781250, "warmUp": 12500 },
+        { "size": 1048576, "count": 39062, "warmUp": 12500 }
+      ]
+    },
+    {
+      "name": "RdmaWriteLatency",
+      "modes": [ "unidirectional" ],
+      "repetitions": 3,
+      "iterations": [
+        { "size": 1, "count": 10000000, "warmUp": 100000 },
+        { "size": 2, "count": 10000000, "warmUp": 100000 },
+        { "size": 4, "count": 10000000, "warmUp": 100000 },
+        { "size": 8, "count": 10000000, "warmUp": 100000 },
+        { "size": 16, "count": 10000000, "warmUp": 100000 },
+        { "size": 32, "count": 10000000, "warmUp": 100000 },
+        { "size": 64, "count": 10000000, "warmUp": 100000 },
+        { "size": 128, "count": 10000000, "warmUp": 100000 },
+        { "size": 256, "count": 10000000, "warmUp": 100000 },
+        { "size": 512, "count": 10000000, "warmUp": 100000 },
+        { "size": 1024, "count": 10000000, "warmUp": 100000 },
+        { "size": 2048, "count": 10000000, "warmUp": 100000 },
+        { "size": 4096, "count": 10000000, "warmUp": 100000 },
+        { "size": 8192, "count": 5000000, "warmUp": 50000 },
+        { "size": 16384, "count": 2500000, "warmUp": 50000 },
+        { "size": 32768, "count": 1250000, "warmUp": 50000 },
+        { "size": 65536, "count": 625000, "warmUp": 25000 },
+        { "size": 131072, "count": 312500, "warmUp": 25000 },
+        { "size": 262144, "count": 156250, "warmUp": 25000 },
+        { "size": 524288, "count": 781250, "warmUp": 12500 },
+        { "size": 1048576, "count": 39062, "warmUp": 12500 }
+      ]
+    },
+    {
+      "name": "RdmaReadLatency",
+      "modes": [ "unidirectional" ],
+      "repetitions": 3,
+      "iterations": [
+        { "size": 1, "count": 10000000, "warmUp": 100000 },
+        { "size": 2, "count": 10000000, "warmUp": 100000 },
+        { "size": 4, "count": 10000000, "warmUp": 100000 },
+        { "size": 8, "count": 10000000, "warmUp": 100000 },
+        { "size": 16, "count": 10000000, "warmUp": 100000 },
+        { "size": 32, "count": 10000000, "warmUp": 100000 },
+        { "size": 64, "count": 10000000, "warmUp": 100000 },
+        { "size": 128, "count": 10000000, "warmUp": 100000 },
+        { "size": 256, "count": 10000000, "warmUp": 100000 },
+        { "size": 512, "count": 10000000, "warmUp": 100000 },
+        { "size": 1024, "count": 10000000, "warmUp": 100000 },
+        { "size": 2048, "count": 10000000, "warmUp": 100000 },
+        { "size": 4096, "count": 10000000, "warmUp": 100000 },
+        { "size": 8192, "count": 5000000, "warmUp": 50000 },
+        { "size": 16384, "count": 2500000, "warmUp": 50000 },
+        { "size": 32768, "count": 1250000, "warmUp": 50000 },
+        { "size": 65536, "count": 625000, "warmUp": 25000 },
+        { "size": 131072, "count": 312500, "warmUp": 25000 },
+        { "size": 262144, "count": 156250, "warmUp": 25000 },
+        { "size": 524288, "count": 781250, "warmUp": 12500 },
+        { "size": 1048576, "count": 39062, "warmUp": 12500 }
+      ]
+    },
+    {
+      "name": "MessagingPingPong",
+      "modes": [ "unidirectional" ],
+      "repetitions": 3,
+      "iterations": [
+        { "size": 1, "count": 10000000, "warmUp": 100000 },
+        { "size": 2, "count": 10000000, "warmUp": 100000 },
+        { "size": 4, "count": 10000000, "warmUp": 100000 },
+        { "size": 8, "count": 10000000, "warmUp": 100000 },
+        { "size": 16, "count": 10000000, "warmUp": 100000 },
+        { "size": 32, "count": 10000000, "warmUp": 100000 },
+        { "size": 64, "count": 10000000, "warmUp": 100000 },
+        { "size": 128, "count": 10000000, "warmUp": 100000 },
+        { "size": 256, "count": 10000000, "warmUp": 100000 },
+        { "size": 512, "count": 10000000, "warmUp": 100000 },
+        { "size": 1024, "count": 10000000, "warmUp": 100000 },
+        { "size": 2048, "count": 10000000, "warmUp": 100000 },
+        { "size": 4096, "count": 10000000, "warmUp": 100000 },
+        { "size": 8192, "count": 5000000, "warmUp": 50000 },
+        { "size": 16384, "count": 2500000, "warmUp": 50000 },
+        { "size": 32768, "count": 1250000, "warmUp": 50000 },
+        { "size": 65536, "count": 625000, "warmUp": 25000 },
+        { "size": 131072, "count": 312500, "warmUp": 25000 },
+        { "size": 262144, "count": 156250, "warmUp": 25000 },
+        { "size": 524288, "count": 781250, "warmUp": 12500 },
+        { "size": 1048576, "count": 39062, "warmUp": 12500 }
+      ]
+    }
+  ]
+}