deploy: 6ba513f

changelog.html

@@ -174,6 +174,10 @@ <h1 class="menu-title">running-ng Documentation</h1>
                         <h1 id="changelog"><a class="header" href="#changelog">Changelog</a></h1>
 <h2 id="unreleased"><a class="header" href="#unreleased">Unreleased</a></h2>
 <h3 id="added"><a class="header" href="#added">Added</a></h3>
+<h4 id="benchmark-suites"><a class="header" href="#benchmark-suites">Benchmark Suites</a></h4>
+<ul>
+<li><code>DaCapo</code>: add 23.11-Chopin release and minheap values.</li>
+</ul>
 <h3 id="changed"><a class="header" href="#changed">Changed</a></h3>
 <h3 id="deprecated"><a class="header" href="#deprecated">Deprecated</a></h3>
 <h3 id="removed"><a class="header" href="#removed">Removed</a></h3>
@@ -208,7 +212,7 @@ <h4 id="runtimes-1"><a class="header" href="#runtimes-1">Runtimes</a></h4>
 <li><code>JuliaMMTK</code></li>
 <li><code>JuliaStock</code></li>
 </ul>
-<h4 id="benchmark-suites"><a class="header" href="#benchmark-suites">Benchmark Suites</a></h4>
+<h4 id="benchmark-suites-1"><a class="header" href="#benchmark-suites-1">Benchmark Suites</a></h4>
 <ul>
 <li><code>JuliaGCBenchmarks</code></li>
 </ul>
@@ -225,13 +229,13 @@ <h3 id="fixed-3"><a class="header" href="#fixed-3">Fixed</a></h3>
 <ul>
 <li>Gracifully handle empty modifiers strings in configs, such as <code>openjdk7||foobar</code>.</li>
 </ul>
-<h4 id="benchmark-suites-1"><a class="header" href="#benchmark-suites-1">Benchmark Suites</a></h4>
+<h4 id="benchmark-suites-2"><a class="header" href="#benchmark-suites-2">Benchmark Suites</a></h4>
 <ul>
 <li>DaCapo specific workarounds are now handled by the <code>DaCapo</code> class rather than the <code>JavaBenchmark</code> class to avoid confusions.</li>
 </ul>
 <h2 id="v039-2023-08-02"><a class="header" href="#v039-2023-08-02"><a href="https://github.com/anupli/running-ng/releases/tag/v0.3.9"><code>v0.3.9</code> (2023-08-02)</a></a></h2>
 <h3 id="fixed-4"><a class="header" href="#fixed-4">Fixed</a></h3>
-<h4 id="benchmark-suites-2"><a class="header" href="#benchmark-suites-2">Benchmark Suites</a></h4>
+<h4 id="benchmark-suites-3"><a class="header" href="#benchmark-suites-3">Benchmark Suites</a></h4>
 <ul>
 <li><code>DaCapo</code>: don't explicitly pass <code>-s default</code> to DaCapo unless the user requests so by setting the <code>size</code> key of <code>DaCapo</code> or overriding the sizes for individual benchmarks using the <a href="./references/suite.html#benchmark-specification">benchmark specification syntax</a>. This is so that users can override the size via <code>ProgramArg</code>.</li>
 </ul>
@@ -292,7 +296,7 @@ <h4 id="base-syntax"><a class="header" href="#base-syntax">Base Syntax</a></h4>
 <ul>
 <li>Use the <code>$RUNNING_NG_PACKAGE_DATA</code> environment variable to refer to base configurations shipped with running-ng, such as <code>$RUNNING_NG_PACKAGE_DATA/base/runbms.yml</code>, regardless how you installed runnin-ng.</li>
 </ul>
-<h4 id="benchmark-suites-3"><a class="header" href="#benchmark-suites-3">Benchmark Suites</a></h4>
+<h4 id="benchmark-suites-4"><a class="header" href="#benchmark-suites-4">Benchmark Suites</a></h4>
 <ul>
 <li><code>DaCapo</code> gains an extra key <code>companion</code> to facilitate eBPF tracing programs.</li>
 </ul>
@@ -316,7 +320,7 @@ <h4 id="modifiers-3"><a class="header" href="#modifiers-3">Modifiers</a></h4>
 <li><code>JVMClasspathAppend</code></li>
 <li><code>JVMClasspathPrepend</code></li>
 </ul>
-<h4 id="benchmark-suites-4"><a class="header" href="#benchmark-suites-4">Benchmark Suites</a></h4>
+<h4 id="benchmark-suites-5"><a class="header" href="#benchmark-suites-5">Benchmark Suites</a></h4>
 <ul>
 <li><code>SPECjvm98</code></li>
 </ul>
@@ -339,7 +343,7 @@ <h3 id="removed-2"><a class="header" href="#removed-2">Removed</a></h3>
 </ul>
 <h2 id="v022-2022-03-07"><a class="header" href="#v022-2022-03-07"><a href="https://github.com/anupli/running-ng/releases/tag/v0.2.2"><code>v0.2.2</code> (2022-03-07)</a></a></h2>
 <h3 id="fixed-9"><a class="header" href="#fixed-9">Fixed</a></h3>
-<h4 id="benchmark-suites-5"><a class="header" href="#benchmark-suites-5">Benchmark Suites</a></h4>
+<h4 id="benchmark-suites-6"><a class="header" href="#benchmark-suites-6">Benchmark Suites</a></h4>
 <ul>
 <li><code>JavaBenchmarkSuite</code>: Some DaCapo benchmarks refers to internal classes (e.g., under <code>jdk.internal.ref</code>), and DaCapo implemented a workaround for this behaviour in the jar. However, since we are invoking DaCapo using <code>-cp</code> and the name of the main class, that workaround is bypassed. That workaround is now reimplemented in running-ng through an extra JVM argument <code>--add-exports</code>.</li>
 </ul>
@@ -350,7 +354,7 @@ <h4 id="commands-8"><a class="header" href="#commands-8">Commands</a></h4>
 <li><code>runbms</code> now skips printing CPU frequencies if the system doesn't support it, e.g., when using Docker Desktop on Mac.</li>
 </ul>
 <h3 id="fixed-10"><a class="header" href="#fixed-10">Fixed</a></h3>
-<h4 id="benchmark-suites-6"><a class="header" href="#benchmark-suites-6">Benchmark Suites</a></h4>
+<h4 id="benchmark-suites-7"><a class="header" href="#benchmark-suites-7">Benchmark Suites</a></h4>
 <ul>
 <li><code>BinaryBenchmarkSuite</code>: fixes missing parameter when constructing <code>BinaryBenchmark</code> due to a bug in previous refactoring</li>
 </ul>
@@ -390,15 +394,15 @@ <h4 id="runtimes-2"><a class="header" href="#runtimes-2">Runtimes</a></h4>
 <li><code>JavaScriptCore</code></li>
 <li><code>JVM</code> now detects OOM generated in the form of Rust panic from <code>mmtk-core</code>.</li>
 </ul>
-<h4 id="benchmark-suites-7"><a class="header" href="#benchmark-suites-7">Benchmark Suites</a></h4>
+<h4 id="benchmark-suites-8"><a class="header" href="#benchmark-suites-8">Benchmark Suites</a></h4>
 <ul>
 <li><code>DaCapo</code> gains an extra key <code>size</code>, which is used to specify the size of the input.</li>
 <li><code>DaCapo</code> now allows individual benchmark to override the timing iteration, input size, and timeout of the suite.</li>
 <li><code>SPECjbb2015</code>: basic support for running SPECjbb 2015 in composite mode.</li>
 <li><code>Octane</code>: basic support for running Octane using Wenyu's wrapper script.</li>
 </ul>
 <h3 id="changed-9"><a class="header" href="#changed-9">Changed</a></h3>
-<h4 id="benchmark-suites-8"><a class="header" href="#benchmark-suites-8">Benchmark Suites</a></h4>
+<h4 id="benchmark-suites-9"><a class="header" href="#benchmark-suites-9">Benchmark Suites</a></h4>
 <ul>
 <li>The <code>minheap</code> key of <code>DaCapo</code> changes from a dictionary to a string. The string is used to look up <code>minheap_values</code>, which are collections of minheap values. This makes it easier to store multiple sets of minheap values for the same benchmark suite measured using different runtimes.</li>
 </ul>
@@ -408,7 +412,7 @@ <h4 id="base-syntax-1"><a class="header" href="#base-syntax-1">Base Syntax</a></
 </ul>
 <h4 id="commands-10"><a class="header" href="#commands-10">Commands</a></h4>
 <ul>
-<li>The <code>--slice</code> argument of <code>runbms</code> now accepts multiple comma-separated floating point numbers. </li>
+<li>The <code>--slice</code> argument of <code>runbms</code> now accepts multiple comma-separated floating point numbers.</li>
 </ul>
 <h3 id="removed-3"><a class="header" href="#removed-3">Removed</a></h3>
 <h4 id="base-configurations-4"><a class="header" href="#base-configurations-4">Base Configurations</a></h4>
@@ -445,7 +449,7 @@ <h4 id="runtimes-3"><a class="header" href="#runtimes-3">Runtimes</a></h4>
 <li><code>OpenJDK</code></li>
 <li><code>JikesRVM</code></li>
 </ul>
-<h4 id="benchmark-suites-9"><a class="header" href="#benchmark-suites-9">Benchmark Suites</a></h4>
+<h4 id="benchmark-suites-10"><a class="header" href="#benchmark-suites-10">Benchmark Suites</a></h4>
 <ul>
 <li><code>BinaryBenchmarkSuite</code></li>
 <li><code>DaCapo</code></li>

print.html

@@ -1072,6 +1072,10 @@ <h1 id="machine-specific-known-problems"><a class="header" href="#machine-specif
 <div style="break-before: page; page-break-before: always;"></div><h1 id="changelog"><a class="header" href="#changelog">Changelog</a></h1>
 <h2 id="unreleased"><a class="header" href="#unreleased">Unreleased</a></h2>
 <h3 id="added"><a class="header" href="#added">Added</a></h3>
+<h4 id="benchmark-suites"><a class="header" href="#benchmark-suites">Benchmark Suites</a></h4>
+<ul>
+<li><code>DaCapo</code>: add 23.11-Chopin release and minheap values.</li>
+</ul>
 <h3 id="changed"><a class="header" href="#changed">Changed</a></h3>
 <h3 id="deprecated"><a class="header" href="#deprecated">Deprecated</a></h3>
 <h3 id="removed"><a class="header" href="#removed">Removed</a></h3>
@@ -1106,7 +1110,7 @@ <h4 id="runtimes-2"><a class="header" href="#runtimes-2">Runtimes</a></h4>
 <li><code>JuliaMMTK</code></li>
 <li><code>JuliaStock</code></li>
 </ul>
-<h4 id="benchmark-suites"><a class="header" href="#benchmark-suites">Benchmark Suites</a></h4>
+<h4 id="benchmark-suites-1"><a class="header" href="#benchmark-suites-1">Benchmark Suites</a></h4>
 <ul>
 <li><code>JuliaGCBenchmarks</code></li>
 </ul>
@@ -1123,13 +1127,13 @@ <h3 id="fixed-3"><a class="header" href="#fixed-3">Fixed</a></h3>
 <ul>
 <li>Gracifully handle empty modifiers strings in configs, such as <code>openjdk7||foobar</code>.</li>
 </ul>
-<h4 id="benchmark-suites-1"><a class="header" href="#benchmark-suites-1">Benchmark Suites</a></h4>
+<h4 id="benchmark-suites-2"><a class="header" href="#benchmark-suites-2">Benchmark Suites</a></h4>
 <ul>
 <li>DaCapo specific workarounds are now handled by the <code>DaCapo</code> class rather than the <code>JavaBenchmark</code> class to avoid confusions.</li>
 </ul>
 <h2 id="v039-2023-08-02"><a class="header" href="#v039-2023-08-02"><a href="https://github.com/anupli/running-ng/releases/tag/v0.3.9"><code>v0.3.9</code> (2023-08-02)</a></a></h2>
 <h3 id="fixed-4"><a class="header" href="#fixed-4">Fixed</a></h3>
-<h4 id="benchmark-suites-2"><a class="header" href="#benchmark-suites-2">Benchmark Suites</a></h4>
+<h4 id="benchmark-suites-3"><a class="header" href="#benchmark-suites-3">Benchmark Suites</a></h4>
 <ul>
 <li><code>DaCapo</code>: don't explicitly pass <code>-s default</code> to DaCapo unless the user requests so by setting the <code>size</code> key of <code>DaCapo</code> or overriding the sizes for individual benchmarks using the <a href="./references/suite.html#benchmark-specification">benchmark specification syntax</a>. This is so that users can override the size via <code>ProgramArg</code>.</li>
 </ul>
@@ -1190,7 +1194,7 @@ <h4 id="base-syntax"><a class="header" href="#base-syntax">Base Syntax</a></h4>
 <ul>
 <li>Use the <code>$RUNNING_NG_PACKAGE_DATA</code> environment variable to refer to base configurations shipped with running-ng, such as <code>$RUNNING_NG_PACKAGE_DATA/base/runbms.yml</code>, regardless how you installed runnin-ng.</li>
 </ul>
-<h4 id="benchmark-suites-3"><a class="header" href="#benchmark-suites-3">Benchmark Suites</a></h4>
+<h4 id="benchmark-suites-4"><a class="header" href="#benchmark-suites-4">Benchmark Suites</a></h4>
 <ul>
 <li><code>DaCapo</code> gains an extra key <code>companion</code> to facilitate eBPF tracing programs.</li>
 </ul>
@@ -1214,7 +1218,7 @@ <h4 id="modifiers-4"><a class="header" href="#modifiers-4">Modifiers</a></h4>
 <li><code>JVMClasspathAppend</code></li>
 <li><code>JVMClasspathPrepend</code></li>
 </ul>
-<h4 id="benchmark-suites-4"><a class="header" href="#benchmark-suites-4">Benchmark Suites</a></h4>
+<h4 id="benchmark-suites-5"><a class="header" href="#benchmark-suites-5">Benchmark Suites</a></h4>
 <ul>
 <li><code>SPECjvm98</code></li>
 </ul>
@@ -1237,7 +1241,7 @@ <h3 id="removed-2"><a class="header" href="#removed-2">Removed</a></h3>
 </ul>
 <h2 id="v022-2022-03-07"><a class="header" href="#v022-2022-03-07"><a href="https://github.com/anupli/running-ng/releases/tag/v0.2.2"><code>v0.2.2</code> (2022-03-07)</a></a></h2>
 <h3 id="fixed-9"><a class="header" href="#fixed-9">Fixed</a></h3>
-<h4 id="benchmark-suites-5"><a class="header" href="#benchmark-suites-5">Benchmark Suites</a></h4>
+<h4 id="benchmark-suites-6"><a class="header" href="#benchmark-suites-6">Benchmark Suites</a></h4>
 <ul>
 <li><code>JavaBenchmarkSuite</code>: Some DaCapo benchmarks refers to internal classes (e.g., under <code>jdk.internal.ref</code>), and DaCapo implemented a workaround for this behaviour in the jar. However, since we are invoking DaCapo using <code>-cp</code> and the name of the main class, that workaround is bypassed. That workaround is now reimplemented in running-ng through an extra JVM argument <code>--add-exports</code>.</li>
 </ul>
@@ -1248,7 +1252,7 @@ <h4 id="commands-8"><a class="header" href="#commands-8">Commands</a></h4>
 <li><code>runbms</code> now skips printing CPU frequencies if the system doesn't support it, e.g., when using Docker Desktop on Mac.</li>
 </ul>
 <h3 id="fixed-10"><a class="header" href="#fixed-10">Fixed</a></h3>
-<h4 id="benchmark-suites-6"><a class="header" href="#benchmark-suites-6">Benchmark Suites</a></h4>
+<h4 id="benchmark-suites-7"><a class="header" href="#benchmark-suites-7">Benchmark Suites</a></h4>
 <ul>
 <li><code>BinaryBenchmarkSuite</code>: fixes missing parameter when constructing <code>BinaryBenchmark</code> due to a bug in previous refactoring</li>
 </ul>
@@ -1288,15 +1292,15 @@ <h4 id="runtimes-3"><a class="header" href="#runtimes-3">Runtimes</a></h4>
 <li><code>JavaScriptCore</code></li>
 <li><code>JVM</code> now detects OOM generated in the form of Rust panic from <code>mmtk-core</code>.</li>
 </ul>
-<h4 id="benchmark-suites-7"><a class="header" href="#benchmark-suites-7">Benchmark Suites</a></h4>
+<h4 id="benchmark-suites-8"><a class="header" href="#benchmark-suites-8">Benchmark Suites</a></h4>
 <ul>
 <li><code>DaCapo</code> gains an extra key <code>size</code>, which is used to specify the size of the input.</li>
 <li><code>DaCapo</code> now allows individual benchmark to override the timing iteration, input size, and timeout of the suite.</li>
 <li><code>SPECjbb2015</code>: basic support for running SPECjbb 2015 in composite mode.</li>
 <li><code>Octane</code>: basic support for running Octane using Wenyu's wrapper script.</li>
 </ul>
 <h3 id="changed-9"><a class="header" href="#changed-9">Changed</a></h3>
-<h4 id="benchmark-suites-8"><a class="header" href="#benchmark-suites-8">Benchmark Suites</a></h4>
+<h4 id="benchmark-suites-9"><a class="header" href="#benchmark-suites-9">Benchmark Suites</a></h4>
 <ul>
 <li>The <code>minheap</code> key of <code>DaCapo</code> changes from a dictionary to a string. The string is used to look up <code>minheap_values</code>, which are collections of minheap values. This makes it easier to store multiple sets of minheap values for the same benchmark suite measured using different runtimes.</li>
 </ul>
@@ -1306,7 +1310,7 @@ <h4 id="base-syntax-1"><a class="header" href="#base-syntax-1">Base Syntax</a></
 </ul>
 <h4 id="commands-10"><a class="header" href="#commands-10">Commands</a></h4>
 <ul>
-<li>The <code>--slice</code> argument of <code>runbms</code> now accepts multiple comma-separated floating point numbers. </li>
+<li>The <code>--slice</code> argument of <code>runbms</code> now accepts multiple comma-separated floating point numbers.</li>
 </ul>
 <h3 id="removed-3"><a class="header" href="#removed-3">Removed</a></h3>
 <h4 id="base-configurations-4"><a class="header" href="#base-configurations-4">Base Configurations</a></h4>
@@ -1343,7 +1347,7 @@ <h4 id="runtimes-4"><a class="header" href="#runtimes-4">Runtimes</a></h4>
 <li><code>OpenJDK</code></li>
 <li><code>JikesRVM</code></li>
 </ul>
-<h4 id="benchmark-suites-9"><a class="header" href="#benchmark-suites-9">Benchmark Suites</a></h4>
+<h4 id="benchmark-suites-10"><a class="header" href="#benchmark-suites-10">Benchmark Suites</a></h4>
 <ul>
 <li><code>BinaryBenchmarkSuite</code></li>
 <li><code>DaCapo</code></li>