Running tests

Configuring the browsers

First step is configuring the supported browsers. Create the "test/resources/browser_manifests/browser_manifest.json" file. The samples of the file can be found in the "test/resources/browser_manifests" folder.

Running the reference tests

The tests that are known to pass are listed in the "test/test_manifest.json" file. The following command runs these tests:

make reftest

This command requires reference images that are created by the make makeref (see section below). Normal practice is to create/update reference images before the change (baseline) and then to verify the changes by running the reference tests.

Making new reference images

To create new references images locally, run the make makeref command.

Adding new tests

Edit the "test/test_manifest.json" file to add a new test. There are two types of tests: frame image comparison ("eq"), API ("sanity") and scriptable ("stas"). All tests shall have "id" and "type" attributes defined. The "id" attribute must be unique.

The frame image comparison tests shall have the following format:

    {  "id": "button-ref",
       "frames": [10],
       "swf": "swfs/button1.swf",
       "type": "eq"
    },

where "frames" is a list of the frame indices where snapshots will be taken, "swf" - is a path to a SWF file.

The scriptable tests shall have the following format:

    {  "id": "basic",
       "stas": "swfs/trace.stas",
       "filenames": [
         "swfs/local2global.swf"
       ],
       "type": "stas"
    },

where "stas" is a path to a script file (see example), "filenames" is SWF test files paths.

The API/sanity tests shall have the following format:

    {  "id": "sanity-test",
       "filenames": [
         "sanity/default.js"
       ],
       "type": "sanity"
    },

where "filenames" is JavaScript test files that perform basic API checks. A test fails when console.error is called or an exception raised (see example).

Running AVM1/AVM2 unit tests

The following command runs tests for the AVM1 and AVM2:

make test

See also Running swfdec tests