Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update sitemap a11y scan #463

Merged
merged 90 commits into from
Apr 11, 2024
Merged

Update sitemap a11y scan #463

merged 90 commits into from
Apr 11, 2024

Conversation

tjheffner
Copy link
Contributor

@tjheffner tjheffner commented Mar 20, 2024

Description

Relates to #16117.

Testing done

Running test locally with all the options via CLI, e.g. BASE_URL=https://va.gov USE_PROXY=false PW_WIDTH_VALUE=768 PW_HEIGHT_VALUE=720 PW_BROWSER_VALUE=firefox SEGMENT_INDEX=0 yarn playwright test --project=a11y

Screenshots

Screenshot 2024-04-05 at 11 18 40 AM

QA steps

  1. Go to the a11y workflow section on GH: https://github.com/department-of-veterans-affairs/next-build/actions/workflows/a11y.yml
  2. Look at the latest run to see it is green.
  3. Download the a11y-scan-1 artifact and open it to confirm violations are listed by URL with a valid JSON syntax. The browser name and viewport size should be included with each URL listed.
  4. Re-run the latest job and see it passes. The run should take around 20 minutes.
  5. Repeat steps 2 and 3 to see they still pass.

@va-cms-bot va-cms-bot temporarily deployed to Tugboat March 20, 2024 17:19 Destroyed
@va-cms-bot va-cms-bot temporarily deployed to Tugboat March 20, 2024 17:29 Destroyed
@alexfinnarn
Copy link
Contributor

This branch runs locally with BASE_URL=https://va.gov USE_PROXY=false yarn test:playwright:a11y and completes five segments.

Tests running:
Screenshot 2024-03-20 at 1 36 11 PM

HTML report home page:
Screenshot 2024-03-20 at 1 40 23 PM

HTML report for specific test:
Screenshot 2024-03-20 at 1 41 21 PM

@alexfinnarn alexfinnarn self-requested a review March 20, 2024 17:51
alexfinnarn
alexfinnarn previously approved these changes Mar 20, 2024
Copy link
Contributor

@alexfinnarn alexfinnarn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! I agree with not processing the sitemap on each request but this works locally now and can be modified later.

@va-cms-bot va-cms-bot temporarily deployed to Tugboat March 20, 2024 17:59 Destroyed
@va-cms-bot va-cms-bot temporarily deployed to Tugboat March 20, 2024 20:49 Destroyed
@va-cms-bot va-cms-bot temporarily deployed to Tugboat March 21, 2024 15:15 Destroyed
@va-cms-bot va-cms-bot temporarily deployed to Tugboat March 21, 2024 15:29 Destroyed
@va-cms-bot va-cms-bot temporarily deployed to Tugboat March 21, 2024 16:42 Destroyed
@va-cms-bot va-cms-bot temporarily deployed to Tugboat March 21, 2024 16:52 Destroyed
@va-cms-bot va-cms-bot temporarily deployed to Tugboat March 21, 2024 17:05 Destroyed
@va-cms-bot va-cms-bot temporarily deployed to Tugboat March 21, 2024 17:21 Destroyed
@va-cms-bot va-cms-bot temporarily deployed to Tugboat March 21, 2024 17:44 Destroyed
@va-cms-bot va-cms-bot temporarily deployed to Tugboat March 21, 2024 17:50 Destroyed
@va-cms-bot va-cms-bot temporarily deployed to Tugboat March 21, 2024 17:55 Destroyed
@va-cms-bot va-cms-bot temporarily deployed to Tugboat April 4, 2024 19:31 Destroyed
@va-cms-bot va-cms-bot temporarily deployed to Tugboat April 4, 2024 19:57 Destroyed
.github/workflows/a11y.yml Show resolved Hide resolved
.github/workflows/a11y.yml Show resolved Hide resolved
playwright/tests/a11y.spec.js Outdated Show resolved Hide resolved
playwright/tests/a11y.spec.js Outdated Show resolved Hide resolved
playwright/tests/a11y.spec.js Outdated Show resolved Hide resolved
@@ -0,0 +1,158 @@
name: A11y Tests
on:
pull_request:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

once we have all the kinks worked out, let's update this to run on a schedule like thebroken-links-check.yml workflow

Something like:

on:
  # Manual trigger
  workflow_dispatch:
  # Schedule; currently once nightly, at 3:20 UTC.
  schedule:
    - cron: 20 3 * * *

(I've adjusted the hours from the broken link schedule to offset load on the runners)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alexfinnarn everything LGTM, let's bump this to be scheduled and we can merge

.github/workflows/a11y.yml Outdated Show resolved Hide resolved
READMEs/testing.md Show resolved Hide resolved
playwright/tests/a11y.spec.js Outdated Show resolved Hide resolved
@va-cms-bot va-cms-bot temporarily deployed to Tugboat April 5, 2024 19:01 Destroyed
.github/workflows/a11y.yml Outdated Show resolved Hide resolved
playwright/tests/a11y.spec.js Show resolved Hide resolved
playwright/tests/a11y.spec.js Outdated Show resolved Hide resolved
@va-cms-bot va-cms-bot temporarily deployed to Tugboat April 9, 2024 18:08 Destroyed
@va-cms-bot va-cms-bot temporarily deployed to Tugboat April 9, 2024 18:14 Destroyed
@va-cms-bot va-cms-bot temporarily deployed to Tugboat April 9, 2024 19:28 Destroyed
@va-cms-bot va-cms-bot temporarily deployed to Tugboat April 9, 2024 20:09 Destroyed
@tjheffner tjheffner requested a review from alexfinnarn April 11, 2024 15:53
Copy link
Contributor Author

@tjheffner tjheffner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there's one false positive with a 502 in the latest run, but otherwise everything LGTM. i'd like to get this merged and we can iterate as needed on the actual results. great work on this!

@@ -0,0 +1,158 @@
name: A11y Tests
on:
pull_request:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alexfinnarn everything LGTM, let's bump this to be scheduled and we can merge

@va-cms-bot va-cms-bot temporarily deployed to Tugboat April 11, 2024 17:01 Destroyed
@va-cms-bot va-cms-bot temporarily deployed to Tugboat April 11, 2024 17:06 Destroyed
@va-cms-bot va-cms-bot temporarily deployed to Tugboat April 11, 2024 17:14 Destroyed
@va-cms-bot va-cms-bot temporarily deployed to Tugboat April 11, 2024 17:23 Destroyed
@tjheffner tjheffner merged commit 9c3218a into main Apr 11, 2024
10 checks passed
@tjheffner tjheffner deleted the VACMS-16117-playwright-a11y-tests branch April 11, 2024 17:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Create a11y check workflow for Next Build output
3 participants