This project assists in testing (SOAP) web services and web applications by providing an application to define and run tests. To this end it contains a baseline installation of FitNesse (an acceptance testing wiki framework) and some FitNesse fixture (base) classes.
The fixtures provided aim to assist in testing (SOAP) web services and web applications (using Selenium) minimizing the amount of (custom) Java code needed to define tests.
The baseline FitNesse installation offers the following features:
- Ability to easily create a standalone (no JDK or Maven required) FitNesse environment.
- Run FitNesse tests on a build server, reporting the results in both JUnit XML format and HTML.
- HSAC's fitnesse-plugin to add additional Wiki features (random values, calculating relative dates, Slim scenarios without need to specify all parameters, Slim scripts that take a screenshot after each step).
- Praegus' toolchain-plugin, improving the wiki's look and feel and page editing features, combining:
- FitNesse installation for test/fixture developers containing:
- the fixture base classes (and Selenium drivers for Chrome, Internet Explorer, Edge and Firefox),
- easy fixture debugging.
The fastest way to get started: just download the 'standalone.zip' from the Releases, extract, run it (you'll just need a Java runtime) and explore the example tests and maybe add a couple of your own.
When you want to use the project's baseline to create and maintain your own test suite, we recommend creating your own Maven project based on the project. This will allow you to run and maintain your test set, in version control, without the need to keep your own copies of dependencies (neither Selenium WebDrivers nor Java libraries).
To create a Maven project to run your own tests, based on this project's baseline (assuming you already have a working Maven installation:
- Go to the directory below which you want to create the project.
- Use the archetype,
nl.hsac:fitnesse-project, to generate the project:- (Maven should find the latest version of the archetype automatically, but it doesn't always.) Check the latest version number on the releases page, and use this instead of
<latest-version>in the next command. - On the commandline execute:
mvn archetype:generate -DarchetypeGroupId=nl.hsac -DarchetypeArtifactId=fitnesse-project -DarchetypeVersion=<latest-version>(or use your IDE's ability to create a project from an archetype). - Answer the prompts for 'groupId', 'artifactId' and 'package' for the project.
- (Maven should find the latest version of the archetype automatically, but it doesn't always.) Check the latest version number on the releases page, and use this instead of
- Alternatively you could also manually copy and update a sample project:
- Clone or download the sample project to a new directory. (This is the approach used in the Installation Guide.)
- Update the 'groupId' and 'artifactId' in the
pom.xmldownloaded as part of the sample to reflect your own organisation and project name. - Move the file for Java class
FixtureDebugTestto a package/directory of your choice.
- Start the wiki, as described in the generated project's
README.md. - Start writing tests (and custom fixtures if needed)...
Execute mvn clean package -DskipTests, the standalone installation is present in the wiki
directory and as ...-standalone.zip file in the target directory. It can be distributed by just copying the wiki directory or by copying and extracting the zip file to a location without spaces in its own name, or in its parent's names).
This standalone installation can be started using java -jar fitnesse-standalone.jar from the wiki directory (or directory where the standalone.zip was extracted).
A zip file containing released versions of this project can be downloaded from the Releases or Maven Central. A similar zip file containing the latest snapshot (i.e. not released but based on the most recent code) version is published as part of the automated build of this project at https://fhoeben.github.io/hsac-fitnesse-fixtures-test-results/hsac-fitnesse-fixtures-snapshot-standalone.zip.
Tests can be run on a build (or Continuous Integration) server in two ways. One can either gather the dependencies, compile and run tests using a build tool such as Maven or Gradle. Or one can use a docker container which contains a compiled version of the project, add the wiki page to that and have it run tests.
Have the build server checkout the project and execute mvn clean test-compile failsafe:integration-test. Append failsafe:verify to the command if you want the build to fail in case of test failures.
The result in JUnit XML results can be found in: target/failsafe-reports (most build servers will pick these up automatically)
The HTML results can be found in: target/fitnesse-results/index.html
The FitNesse suite to run can be specified by changing the value of the @Suite annotation in nl.hsac.fitnesse.fixture.FixtureDebugTest, or (preferably) by adding a system property, called fitnesseSuiteToRun, specifying the suite to run to the build server's mvn execution.
By using the @SuiteFilter and @ExcludeSuiteFilter annotations, or (preferably) by adding suiteFilter and/or excludeSuiteFilter system properties, one can provide tags to in- or exclude and further filter the tests to run within the specified suite. Provide multiple tags by comma-separating them.
When multiple test runs are done (for instance in parallel to reduce total execution times) one should configure the the HTML output directory for each run to be a subdirectory of a common root (e.g. target/fitnesse-results/run1 and target/fitnesse-results/run2) and then use nl.hsac.fitnesse.junit.reportmerge.HtmlReportIndexGenerator to generate a combined overview of all runs.
The Selenium configuration (e.g. what browser on what platform) to use when testing websites can be overridden by using system properties (i.e. seleniumGridUrl and either seleniumBrowser or seleniumCapabilities).
This allows different configurations on the build server to test with different browsers, without requiring different Wiki content, but only requiring a different build configuration.
Using docker one can start a container (i.e. virtual machine) containing a compiled version of the project and have it execute the tests defined. The main advantage of this approach is there is no need to gather dependencies and compile on each test run. This can be quite advantages when running builds on fully virtualized infrastructure where no state is maintained between runs, and all dependencies have to be downloaded afresh each run.
The way the actual tests are run is quite similar inside a docker container to what was described above for 'Running using Maven' in that a jUnit test is executed to run a suite of wiki pages. Which suite is executed, filters to include and exclude, and Selenium can be configured without modifing the images provided.
Two images are provided on Docker Hub to run tests (or to be used as base for custom images, of course):
- hsac/fitnesse-fixtures-test-jre8 which contains a Java runtime and this project, which can be used to run tests that either do not need Selenium (i.e. do not test using BrowserTest) or use an existing/external Selenium Grid.
- hsac/fitnesse-fixtures-test-jre8-chrome which adds this project to a Selenium standalone Chrome image and can be used to run tests with a Chrome browser (without the need to have/maintain a Selenium Grid).
An additional image is provided to combine multiple test run's results: hsac/fitnesse-fixtures-combine.
Detailed instructions on how to use the images are provided in each image's description.
Example reports for Windows using a Sauce Labs Selenium driver (https://fhoeben.github.io/hsac-fitnesse-fixtures-test-results/examples-results/) and Linux with Chrome Headless (https://fhoeben.github.io/hsac-fitnesse-fixtures-test-results/acceptance-test-results/) are generated in the automated build process of this project.
Import this project in your favorite Java IDE (with Maven support).
To start FitNesse: have the IDE execute mvn compile dependency:copy-dependencies exec:exec. The port used by FitNesse can be controlled by changing the fitnesse.port property's value in pom.xml.
FitNesse will be available at http://localhost:<fitnesse.port>/, example usage of the symbols and fixtures can be seen in http://localhost:<fitnesse.port>/HsacExamples.
To debug a fixture used in a FitNesse page: change the @Suite annotation's value in nl.hsac.fitnesse.fixture.FixtureDebugTest to contain the page name, then just debug this test.
More information about this project can be found on its GitHub Wiki