Skip to content

Commit

Permalink
test: fix and refactor e2e tests (#506)
Browse files Browse the repository at this point in the history
* test: fix and refactor e2e tests

* test: introduce npm workspaces and fix tests

* fix: add install-chromium script and update e2e tests
  • Loading branch information
shadowusr authored Sep 18, 2023
1 parent 41bb37c commit 6d638d8
Show file tree
Hide file tree
Showing 92 changed files with 3,113 additions and 1,908 deletions.
31 changes: 24 additions & 7 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
@@ -1,26 +1,43 @@
version: 2
jobs:
build:
working_directory: ~/html-reporter
docker:
- image: cimg/node:12.18-browsers
- image: cimg/node:16.20-browsers
environment:
CHROME_VERSION: 116
SERVER_HOST: localhost

steps:

- checkout

- run: npm install
- run: npm ci
- run:
name: Build html-reporter

command: npm run build
- run:
name: Download Chromium
command: >-
cd test/func/docker/browser-utils &&
npm ci &&
npm run install-chromium
- run:
name: Download Selenium
command: sudo npm install selenium-standalone@6.17.0 -g
command: npm install selenium-standalone@9.1.1 -g
- run:
name: Start Selenium
command: selenium-standalone install && selenium-standalone start
command: >-
selenium-standalone install --drivers.chrome.version=$CHROME_VERSION &&
selenium-standalone start --drivers.chrome.version=$CHROME_VERSION
background: true

- run:
name: Functional tests
command: npm run test-func
command: npm run e2e

- store_artifacts:
path: hermione-report/
destination: /hermione-report
path: test/func/tests/reports/
destination: /reports
5 changes: 3 additions & 2 deletions .eslintignore
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ hot
/lib/static/*.min.js
/lib/static/data.js
/lib/static/sql-wasm.js
/test/func/fixtures/report
/test/func/html-reporter-plugins/*/plugin.js
/test/func/**/report
/test/func/**/reports
/test/func/packages/*/plugin.js
/hermione-report
4 changes: 3 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -22,4 +22,6 @@ sqlite.db
.nyc_output

hermione-report
/test/func/fixtures/report
test/func/**/report
test/func/**/reports
test/func/packages/*/plugin.js
1 change: 0 additions & 1 deletion .hermione.conf.js

This file was deleted.

40 changes: 40 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# Contribution guide

> This document is currently a work in progress and is not yet comprehensive.
> Additional info will be added over time.
### Running e2e tests

End-to-end testing of html-reporter consists of two stages: generating fixture reports using different tools and tests,
then running hermione tests on these reports.

In order to make e2e/screenshot tests stable and reproducible across different environments,
you need to launch browsers inside a Docker container.

1. Make sure you have Docker installed.
<details><summary>How to?</summary>
1. If you want to make a personal open-source contribution, you may use Docker free of charge and follow the [official guide](https://docs.docker.com/get-docker/).
2. If you are acting on behalf of a company, you may not have access to Docker Desktop. In this case:
- On Linux, you may follow the official installation guide.
- On Mac, you may use [colima](https://github.com/abiosoft/colima) as a replacement for Docker Desktop.
- On Windows, you may use Windows Subsystem for Linux to run the Docker CLI without the Desktop application.
</details>

2. Build and start an image with browsers:
```
npm run e2e:build-browsers && npm run e2e:launch-browsers
```
3. Run e2e tests:
```bash
npm run e2e
```
End-to-end tests are divided into multiple workspaces:
- `test/func/packages` — packages needed for generating fixture reports, e.g. test html-reporter plugins
- `test/func/fixtures` — packages to generate fixture reports
- `test/func/tests` — hermione tests that run on fixture reports
If you want a finer-grained control over the process, the following commands may be useful:
- `npm run e2e:generate-fixtures` — generate fixture reports to run tests on
- `npm run --workspace=test/func/tests gui:plugins` — launch hermione GUI for the `plugins` tests set
- `npm run e2e:test` — run e2e tests only, without building packages or generating fixtures
2 changes: 1 addition & 1 deletion lib/static/components/controls/menu-bar.jsx
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ class MenuBar extends Component {
}

return (
<div className="menu-bar">
<div className="menu-bar" data-test-id='menu-bar'>
<Dropdown item icon="bars" simple direction="left">
<Dropdown.Menu>
<ExtensionPoint name={MENU_BAR}>
Expand Down
4 changes: 2 additions & 2 deletions lib/static/components/controls/report-info.jsx
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,13 @@ class ReportInfo extends Component {

return (
<div className="report-info">
<Label className="control-label">
<Label className="control-label" data-test-id='version-label'>
Version
<Label.Detail>
{version}
</Label.Detail>
</Label>
{!gui && <Label className="control-label">
{!gui && <Label className="control-label" data-test-id='created-at-label'>
Created at
<Label.Detail>
{date}
Expand Down
1 change: 1 addition & 0 deletions lib/static/components/controls/selects/group-tests.jsx
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,7 @@ class GroupTestsSelect extends Component {
onChange={this._groupTestsByKey}
placeholder="select key"
clearable={Boolean(selectedGroupKey)}
data-test-id='group-by-dropdown'
>
<Dropdown.Menu>
{Object.values(SECTIONS).map((sectionName) => this._renderSection(sectionName))}
Expand Down
Loading

0 comments on commit 6d638d8

Please sign in to comment.