Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve snapshot testing for Optimization Detective #1791

Merged
merged 8 commits into from
Jan 13, 2025

Conversation

westonruter
Copy link
Member

@westonruter westonruter commented Jan 12, 2025

Writing snapshot tests for Optimization Detective has been tedious since the buffer HTML and expected HTML have been embedded in PHP files. This has made adding new tests tedious by needing to ensure the HTML is properly escaped. The HTML in the PHP files was also indented additional tabs for readability, and when attempting to paste updates to the expected output into the PHP, this would often cause problems with auto-formatting.

Previously test cases were contained in single PHP files which returned an array with set_up, buffer, and expected keys. With this PR, now tests are in separate directories with set-up.php, buffer.html, and expected.html files. Whenever the buffer.html and expected.html files do not match during tests, then the test will emit an actual.html (which is ignored from git) which the developer can then simply rename to expected.html to update the test. This is also the case when creating a new test, where you just add a set-up.php and buffer.html and the test will then issue an "test incomplete" warning while writing out the actual.html for the developer to then examine and rename to expected.html if it is all as expected.

This PR also removes duplicated test set up logic between Optimization Detective, Image Prioritizer, and Embed Optimizer.

This PR is massive, but the changes in plugins/**/test-cases/* are just splitting up the PHP files into the set-up.php, buffer.html, and expected.html files. So these do not need to be reviewed. The only exception may be the set_up logic in video-with-large-poster-and-desktop-url-metrics-missing and video-with-large-poster-and-desktop-url-metrics-collected which change/improve how the image URLs are inserted into the buffer using placeholder replacements.

Note that phpunit-snapshot-assertions cannot be used because it requires PHP 8.1.

Fixes #1418

@westonruter westonruter added [Type] Enhancement A suggestion for improvement of an existing feature no milestone PRs that do not have a defined milestone for release labels Jan 12, 2025
Copy link

github-actions bot commented Jan 12, 2025

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: westonruter <[email protected]>
Co-authored-by: swissspidy <[email protected]>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

@westonruter westonruter added [Plugin] Optimization Detective Issues for the Optimization Detective plugin [Plugin] Embed Optimizer Issues for the Embed Optimizer plugin (formerly Auto Sizes) [Plugin] Image Prioritizer Issues for the Image Prioritizer plugin (dependent on Optimization Detective) labels Jan 12, 2025
Copy link

codecov bot commented Jan 12, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 57.43%. Comparing base (4c1aa71) to head (b059cbf).
Report is 262 commits behind head on trunk.

Additional details and impacted files
@@           Coverage Diff           @@
##            trunk    #1791   +/-   ##
=======================================
  Coverage   57.43%   57.43%           
=======================================
  Files          84       84           
  Lines        6508     6508           
=======================================
  Hits         3738     3738           
  Misses       2770     2770           
Flag Coverage Δ
multisite 57.43% <ø> (ø)
single 34.55% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Member

@swissspidy swissspidy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice evolution on how tests for Optimization Detective are written! This will definitely speed up development going forward.

Base automatically changed from update/od-auth-conditions to trunk January 13, 2025 14:32
@westonruter westonruter merged commit 81112a9 into trunk Jan 13, 2025
18 checks passed
@westonruter westonruter deleted the update/od-snapshot-testing branch January 13, 2025 14:34
@westonruter westonruter added skip changelog PRs that should not be mentioned in changelogs and removed no milestone PRs that do not have a defined milestone for release labels Jan 25, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
[Plugin] Embed Optimizer Issues for the Embed Optimizer plugin (formerly Auto Sizes) [Plugin] Image Prioritizer Issues for the Image Prioritizer plugin (dependent on Optimization Detective) [Plugin] Optimization Detective Issues for the Optimization Detective plugin skip changelog PRs that should not be mentioned in changelogs [Type] Enhancement A suggestion for improvement of an existing feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Snapshot testing for Optimization Detective
2 participants