Skip to content

Commit

Permalink
try no retries to see if last run passes
Browse files Browse the repository at this point in the history
  • Loading branch information
alexfinnarn committed Mar 28, 2024
1 parent 3874466 commit 8fcf11e
Show file tree
Hide file tree
Showing 3 changed files with 27 additions and 8 deletions.
8 changes: 4 additions & 4 deletions .github/workflows/a11y.yml
Original file line number Diff line number Diff line change
Expand Up @@ -62,10 +62,10 @@ jobs:
- name: Run Playwright tests
run: |
if [ ${{ matrix.shardIndex }} -eq ${{ matrix.shardTotal }} ]; then
echo "Exiting early because this is the last shard"
exit 0
fi
# if [ ${{ matrix.shardIndex }} -eq ${{ matrix.shardTotal }} ]; then
# echo "Exiting early because this is the last shard"
# exit 0
# fi

TOTAL_SEGMENTS=${{ matrix.shardTotal }} SEGMENT_INDEX=${{ matrix.shardIndex }} yarn playwright test --project=a11y

Expand Down
25 changes: 21 additions & 4 deletions READMEs/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,15 +75,32 @@ This project can be tested for a11y compliance in several ways:
expect(accessibilityScanResults.violations).toEqual([])
```
- A full site scan of all urls known to next-build (generated in the sitemap) using Playwright: `yarn test:playwright:a11y`
- A full site scan of all urls known to next-build (generated in the sitemap) using Playwright:
The full scan will run daily in CI using GitHub Actions and multiple runners.
### Full Site Scan
###
The full scan will run daily in CI using a GitHub Workflow and multiple
runners, but you can also run it locally for testing and debugging purposes.
You can run it manually after generating the sitemap with a few steps:
- Workflow file: `.github/workflows/a11y.yml`
- Yarn command: `yarn test:playwright:a11y`
#### Setup
We designed the scan to run against the production va.gov/sitemap.xml to
ensure accessibility issues facing actual users are caught. Testing locally
and in lower environments is great, but to get the most bang for your buck,
running it on the actual output ensures that any fixes carry through to
production.
You don't have to build a site locally, but if you want to test against a
fresh next-build instancecan
run it manually after
generating the sitemap
with a few steps:
1. `yarn export` to generate the static pages for the site
2. `yarn build:sitemap` to generate the sitemap for pages from step 1.
3. `yarn export:serve` to host the static pages locally
4. `yarn test:playwright:a11y` to run the scan. This runs `playwright/tests/a11y.spec.js` which loops over the sitemap and tests each page individually using `@axe-core/playwright`.
2 changes: 2 additions & 0 deletions playwright.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,8 @@ export default defineConfig({
{
name: 'a11y',
testMatch: /.a11y.spec.js/,
// No retries so last GH runner doesn't fail.
retries: 0,
use: {
browserName:
(process.env.PW_BROWSER_VALUE as 'chromium' | 'firefox' | 'webkit') ||
Expand Down

0 comments on commit 8fcf11e

Please sign in to comment.