playwright pr comment

[fungairino]

What does this PR do?

• Adds a step to the e2e ci job that will comment on the PR with the result of the test run.

Discussion

• Distinguish mv2 and mv3 report?

Checklist

• Add jest or playwright tests and/or storybook stories
• Designate a primary reviewer @grahamlangford

[github-actions]

Playwright test results

 38 passed

Details

 38 tests across 15 suites
 9 minutes, 49 seconds
 2b74482

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 73.51%. Comparing base (53f84ce) to head (036eec6).
Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #8370      +/-   ##
==========================================
+ Coverage   73.50%   73.51%   +0.01%     
==========================================
  Files        1332     1332              
  Lines       41200    41200              
  Branches     7665     7665              
==========================================
+ Hits        30284    30290       +6     
+ Misses      10916    10910       -6     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[github-actions]

Playwright test results - MV2

 32 passed
 8 skipped

Details

 40 tests across 15 suites
 7 minutes, 26 seconds
 036eec6

Skipped tests

chrome › tests/extensionConsoleActivation.spec.ts › can activate a mod with built-in integration
edge › tests/extensionConsoleActivation.spec.ts › can activate a mod with built-in integration
chrome › tests/runtime/sidebarController.spec.ts › sidebar controller › shows focus dialog in top-level frame
edge › tests/runtime/sidebarController.spec.ts › sidebar controller › shows focus dialog in top-level frame
chrome › tests/runtime/sidebarNavigation.spec.ts › sidebar mod panels are persistent during navigation
chrome › tests/runtime/sidebarNavigation.spec.ts › sidebar form and temporary panels are unavailable after navigation
edge › tests/runtime/sidebarNavigation.spec.ts › sidebar mod panels are persistent during navigation
edge › tests/runtime/sidebarNavigation.spec.ts › sidebar form and temporary panels are unavailable after navigation

[github-actions]

Playwright test results - MV3

 40 passed

Details

 40 tests across 15 suites
 10 minutes, 24 seconds
 036eec6

[fungairino]

The PR ID isn't available unless I include this pull_request trigger for the playwright summary. Though I'm a bit worried this may not work in a non-PR context

[grahamlangford]

This runs when we version main and when we cut release branches, neither of which are pull requests

[fungairino]

Copied convo from slack:

Eduardo
1:26 PM
Theoretically it should be fine to add the additional trigger pull_request since every kind of that event is also a push, right? Just with extra information

1:26
Though I did notice that pull_request will checkout the merged version of all the commits in the PR rather than the tip of the branch

Graham
1:27 PM
What happens when it's a push without a pull_request though?
1:28
Ex: https://github.com/pixiebrix/pixiebrix-extension/actions/runs/8837003915

Eduardo
1:29 PM
Theoretically it won't pass.
I can add a check for when not to run a step:
if: github.event_name != 'pull_request'

[mnholtz]

It's not clear to me what was decided here - could you clarify @fungairino ? I do see the github.event_name != 'pull_request' clause below.

[mnholtz]

Approving to unblock, because it seems like the conversation about adding the pull_request trigger is resolved?

[github-actions]

No loom links were found in the first post. Please add one there if you'd like to it to appear on Slack.

Do not edit this comment manually.