Running tests

Configuring the browsers

First step is configuring the supported browsers. Create the "test/resources/browser_manifests/browser_manifest.json" file. The samples of the file can be found in the "test/resources/browser_manifests" folder.

Running the reference tests

The tests that are known to pass are listed in the "test/test_manifest.json" file. The following command runs these tests:

make reftest

This command requires reference images that are created by the make makeref (see section below). Normal practice is to create/update reference images before the change (baseline) and then to verify the changes by running the reference tests.

By default, the tests are run using the sources files located in the "src/" folder. However it is possible to run the test on the files generated by make build-bundle in the "build/bundle/" folder -- use TEST_FLAGS=--bundle make makeref.

Making new reference images

To create new references images locally, run the make makeref command.

Adding new tests

Edit the "test/test_manifest.json" file to add a new test. There are two types of tests: frame image comparison ("eq"), API ("sanity") and scriptable ("stas"). All tests shall have "id" and "type" attributes defined. The "id" attribute must be unique.

The frame image comparison tests shall have the following format:

    {  "id": "button-ref",
       "frames": [10],
       "swf": "swfs/button1.swf",
       "type": "eq"
    },

where "frames" is a list of the frame indices where snapshots will be taken, "swf" - is a path to a SWF file.

The scriptable tests shall have the following format:

    {  "id": "basic",
       "stas": "swfs/trace.stas",
       "filenames": [
         "swfs/local2global.swf"
       ],
       "type": "stas"
    },

where "stas" is a path to a script file (see example), "filenames" is SWF test files paths.

The API/sanity tests shall have the following format:

    {  "id": "sanity-test",
       "filenames": [
         "sanity/default.js"
       ],
       "type": "sanity"
    },

where "filenames" is JavaScript test files that perform basic API checks. A test fails when console.error is called or an exception raised (see example).

Running AVM1 unit tests

See Running swfdec tests

Running AVM2 unit tests

The following command runs tests for the AVM2:

make test-avm2

The tests are located in the "src/avm2/tests/regress/correctness/pass/" folder.

Running jshint

The jshint (linting tool) can be run using:

make lint

See also Style Guide.