Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DO NOT MERGE - Measure performance of pre-order implementations #2467

Draft
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

hatyo
Copy link
Contributor

@hatyo hatyo commented Jan 25, 2024

This is on top of #2464 and should not be merged. It adds a JMH Benchmark that compares the performance of current pre-order implementations in TreeLike.

To run the benchmark, go to PreOrderPerformance and run main function. The benchmark generates a random TreeLike tree comprising 100 nodes with maximum of 20 nodes in a level, it then compares the two implementations of pre-order traversal:

  1. The old implementation, which is done in TreeLike<>#inPreOrder()
  2. The new implementation exposed through the Iterable#iterator() interface that TreeLike<> now extends from.

The benchmark creates 1 million pre-ordered lists once using the old implementation and another time using the new implementation, and compares, among other things the memory consumption.

Excerpt from the test results:

Benchmark                                             Mode  Cnt            Score         Error   Units
PreOrderPerformance.measureNew                        avgt    5       496852,082 ±   22302,728   us/op
PreOrderPerformance.measureNew:·gc.alloc.rate         avgt    5          469,146 ±      20,819  MB/sec
PreOrderPerformance.measureNew:·gc.alloc.rate.norm    avgt    5    244400053,833 ±       6,387    B/op
PreOrderPerformance.measureNew:·gc.count              avgt    5            8,000                counts
PreOrderPerformance.measureNew:·gc.time               avgt    5           36,000                    ms
PreOrderPerformance.measureOld                        avgt    5      4122177,704 ± 2476618,195   us/op
PreOrderPerformance.measureOld:·gc.alloc.rate         avgt    5         2688,688 ±    1422,556  MB/sec
PreOrderPerformance.measureOld:·gc.alloc.rate.norm    avgt    5  11428000294,400 ±     400,408    B/op
PreOrderPerformance.measureOld:·gc.count              avgt    5          106,000                counts
PreOrderPerformance.measureOld:·gc.time               avgt    5        21614,000                    ms

I think the important bit here is gc.alloc.rate which shows the rate in which the garbage collector is invoked. In the old implementation 1422,556 MB/sec compared to 20,819 MB/sec, meaning that the new implementation is reducing the allocation rate by ~98,5%.

@foundationdb-ci

This comment was marked as off-topic.

@foundationdb-ci
Copy link
Contributor

Result of fdb-record-layer-pr on Linux CentOS 7

  • Commit ID: 67efc8f
  • Duration 0:20:09
  • Result: ❌ FAILED
  • Error: Error while executing command: ./gradlew --no-daemon --console=plain -b ./build.gradle build destructiveTest sonarqube -PcoreNotStrict -PreleaseBuild=false -PpublishBuild=false -PspotbugsEnableHtmlReport. Reason: exit status 1
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Contributor

Result of fdb-record-layer-pr on Linux CentOS 7

  • Commit ID: 67efc8f
  • Duration 0:19:54
  • Result: ❌ FAILED
  • Error: Error while executing command: ./gradlew --no-daemon --console=plain -b ./build.gradle build destructiveTest -PcoreNotStrict -PreleaseBuild=false -PpublishBuild=false -PspotbugsEnableHtmlReport. Reason: exit status 1
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants