Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add performance tests for 6502 implementations #42

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

iflan
Copy link
Contributor

@iflan iflan commented Nov 8, 2020

There are currently 5 CPU implementations:

  • jscpu6502.js: The original JS implementation of cpu6502.
  • tscpu6502.ts: The initial Typescript conversion of the JS implementation using bind.
  • tscpu6502v2.ts: A Typescript version that uses call instead of bind.
  • tscpu6502v5.ts: A Typescript version that uses a giant switch statement and this instead of an array. (Only emulates the 6502.)
  • tscpu6502v6.ts: A Typescript version using class variables from PR Use class fields instead of binding #40.

The performance tests added by this change use the following methodology:

  • Tests are compiled and run using the production tool chain, including Webpack.
  • Performance is measured by running the 6502 and 65C02 variations against the same ROMs as the unit tests.
  • A configurable number of "warm-up" runs are performed before the actual measurement runs. This should cause the browser to optimize all of the methods it's going to optimize.
  • A configurable number of measurement runs are performed.

Results include:

  • Minimum time.
  • Maximum time.
  • Mean (average) time.
  • Median time.
  • Number of runs.

The CPU implementations live in test/perf/impl. Tests for the implementations live in test (though not all implementations have tests).

To run the tests using Node's http-server:

npm run build
http-server ./

then navigate to http://localhost:8080/test/perf/cpu_benchmark.html.

Note that there is plenty of room for improvement in both the benchmark implementation and the measurements.

Here are the results for an example run in Chrome 86 comparing the original JS version, the Typescript version using bind and the version from PR #40:

  cpu6502.js cpu6502.js cpu6502.ts cpu6502.ts cpu6502v6.ts cpu6502v6.ts
  6502 65C02 6502 65C02 6502 65C02
min 1850.06 2227.38 1380.41 1613.08 1219.65 1547.14
max 2043.47 2303.67 1445.7 1697.81 1268.03 1624.13
mean 1881.92 2259.27 1403.78 1651.35 1237.17 1577.89
median 1874.9 2260.42 1397.07 1647.96 1233.75 1575.78
runs 50 50 50 50 50 50

iflan added 2 commits November 8, 2020 11:44
This creates a very simple test harness for CPU implementations that
can be run in the browser. It is not dependent on any other libraries.
The harness supports a number of warmup runs and a number of timing
runs. It records the min, max, mean, and median times for the test
runs.

This works by creating a new entrypoint, `cpu_benchmark`, which is
loaded by the `test/perf/cpu_benchmark.html` file. You can fire up a
Node server to serve this by running:

```shell
node http-server ./
```

This change also adds the `bin2js` utility that can create JS files
out of binary ROM images.
Running the performance tests shows that the previous version was
actually slightly faster.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant