Scalability Benchmark Write Up

Device Performance Ranking

Tests were done with 1500 fake devices, each with 18 points. Only the "depth first all" for each device was published. Instructions for how to setup and run this benchmark are in scripts/scalability-testing/README.md.

This test is meant to push devices way beyond what is normally expected of the platform. Poor performance on this test does not necessarily mean that the device is unsuited for use with the platform. For instance, all listed devices are more than adequate for retrieving data from 20 real devices with 20 data point and forwarding the results to another platform.

Time is measured from the start of the first device publish to the completion of the last device publish. Data was published once per minute. DNF means that the device could not complete the task before the next publish started.

All values are in seconds.

Device	Storage	Publish Only	Publish One listener	Historian V2 Catch Up
Xeon CPU W3503 (2.4GHz 2 cores)	Hard Drive	1.76	3.5	~6
6 Core VM on Xeon CPU E3-2603 (1.8GHZ 8 cores)	SSD Drive	5.8	9.3	~5
Odroid XU4	eMMC	5.8	8.02	~5
Intel Compute Stick	eMMC	8.6	12.78	~7.5
Minnowboard Max	SD UHS-3	16.14	28.7	~14
Banana Pi	SD UHS-3	17.76	36.16	~25
RaspberryPi 2 Model B	SD UHS-3	18.03	22.29	~25
SMC TWT controller	SD	36.69	DNF	DNF
BeagleBone2	SD	47.99	DNF	DNF
BeagleBoneBlack	SD	49.87	DNF	DNF