-
Notifications
You must be signed in to change notification settings - Fork 206
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add: playwright sharding for parallel test execution #2504
Add: playwright sharding for parallel test execution #2504
Conversation
WalkthroughThis pull request introduces comprehensive modifications to the testing infrastructure for a WordPress plugin, focusing on enhancing test workflows, reporting mechanisms, and test configuration. The changes span multiple files in the test suite, including GitHub Actions workflow updates, Playwright configuration adjustments, and the introduction of new utility scripts for merging test reports and coverage summaries. The modifications aim to improve test execution, parallel processing, and result reporting across end-to-end (E2E) and API testing environments. Changes
Sequence DiagramsequenceDiagram
participant GA as GitHub Actions
participant E2E as E2E Tests
participant API as API Tests
participant Reporter as Report Merger
GA->>E2E: Trigger Sharded Tests
GA->>API: Trigger Sharded Tests
E2E-->>Reporter: Generate Test Report
API-->>Reporter: Generate Test Report
Reporter->>Reporter: Merge Reports
Reporter->>GA: Upload Consolidated Report
Possibly related PRs
Poem
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 4
🧹 Nitpick comments (6)
tests/pw/utils/mergeSummaryReport.ts (1)
7-7
: Remove debug console.log.Debug logging should be removed before committing the code.
tests/pw/e2e.config.ts (1)
53-54
: LGTM! Good environment-specific reporter configuration.The reporter configuration is well-optimized for different environments:
- CI: Using blob reporter for better CI integration
- Non-CI: Using HTML reporter for better local development experience
Consider adding a comment explaining the rationale for different reporters in different environments:
reporter: CI ? [ // ['github'], + // Use blob reporter in CI for better integration with CI tools // ['html', { open: 'never', outputFolder: 'playwright-report/e2e/html-report' }], ['blob', { open: 'outputDir', outputDir: 'playwright-report/e2e/blob-report' }], ['list', { printSteps: true }], ['./utils/summaryReporter.ts', { outputFile: 'playwright-report/e2e/summary-report/results.json' }], ] : [ + // Use HTML reporter in non-CI for better local development experience // ['blob', { open: 'outputDir', outputDir: 'playwright-report/e2e/blob-report' }], ['html', { open: 'never', outputFolder: 'playwright-report/e2e/html-report' }],Also applies to: 59-61
tests/pw/pages/vendorPage.ts (1)
52-52
: Improve the flakiness comment documentation.While the fix for visibility is good, the comment could be more descriptive about the specific issue being addressed.
Consider expanding the comment to explain the root cause:
- await this.removeAttribute(productsVendor.rowActions(productName), 'class'); // forcing the row actions to be visible, to avoid flakiness + // Remove class attribute that controls hover visibility to prevent race conditions + // where the click action fails because the element is not visible during hover state transition + await this.removeAttribute(productsVendor.rowActions(productName), 'class');tests/pw/tests/e2e/productsDetails.spec.ts (1)
441-442
: Address or document remaining TODOs.Several TODOs remain in the file that should be addressed:
- EU compliance tests are skipped
- Advertising, rank math SEO, and variation options tests are pending
Would you like me to help create GitHub issues to track these pending test implementations? I can help generate the test code for these scenarios.
Also applies to: 534-536
tests/pw/pages/productsPage.ts (1)
615-622
: Enhance error handling in nonce extraction.The nonce extraction logic could be improved to provide more specific error messages.
- if (!nonce) throw new Error('Nonce not found'); + if (!url) throw new Error('Product edit URL not found'); + if (!nonce) throw new Error('Product edit nonce not found in URL');.github/workflows/e2e_api_tests.yml (1)
340-417
: Add error handling for report merging process.While the merge-reports job is well-structured, it could benefit from better error handling:
- Consider adding retry mechanism for artifact downloads
- Add validation for report file existence before merging
- Add error logging for debugging failures
Example improvement for validation:
- name: Generate merged summary report id: merge-summary-report if: steps.download-test-artifacts.outcome == 'success' working-directory: tests/pw run: | + # Validate report files exist + for file in ./all-reports/test-artifact-*/playwright-report/e2e/summary-report/results.json; do + if [ ! -f "$file" ]; then + echo "Error: Missing report file: $file" + exit 1 + fi + done REPORT_TYPE=e2e npx ts-node ./utils/mergeSummaryReport.ts
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
tests/pw/package-lock.json
is excluded by!**/package-lock.json
📒 Files selected for processing (15)
.github/workflows/e2e_api_tests.yml
(5 hunks)tests/pw/e2e.config.ts
(1 hunks)tests/pw/package.json
(3 hunks)tests/pw/pages/productsPage.ts
(40 hunks)tests/pw/pages/vendorPage.ts
(1 hunks)tests/pw/tests/api/calculation.spec.ts
(1 hunks)tests/pw/tests/e2e/_auth.setup.ts
(3 hunks)tests/pw/tests/e2e/productsDetails.spec.ts
(4 hunks)tests/pw/tests/e2e/shipstation.spec.ts
(0 hunks)tests/pw/tests/e2e/storeAppearance.spec.ts
(1 hunks)tests/pw/types/environment.d.ts
(2 hunks)tests/pw/utils/mergeCoverageSummary.ts
(1 hunks)tests/pw/utils/mergeSummaryReport.ts
(1 hunks)tests/pw/utils/summaryReporter.ts
(1 hunks)tests/pw/utils/testData.ts
(1 hunks)
💤 Files with no reviewable changes (1)
- tests/pw/tests/e2e/shipstation.spec.ts
⏰ Context from checks skipped due to timeout of 90000ms (4)
- GitHub Check: e2e tests (3, 3)
- GitHub Check: e2e tests (2, 3)
- GitHub Check: e2e tests (1, 3)
- GitHub Check: api tests (1, 1)
🔇 Additional comments (17)
tests/pw/types/environment.d.ts (1)
21-21
: LGTM! Environment variables properly declared.The new environment variables
PRODUCT_EDIT_NONCE
andREPORT_TYPE
are correctly typed and well-placed within the interface.Also applies to: 80-80
tests/pw/tests/e2e/_auth.setup.ts (1)
3-3
: LGTM! Product edit nonce properly retrieved during vendor authentication.The implementation follows the established pattern and ensures the nonce is available for subsequent product editing operations.
Also applies to: 80-82
tests/pw/utils/mergeSummaryReport.ts (1)
26-32
: LGTM! Well-implemented duration formatting.The helper function for formatting duration is clear, concise, and handles all cases correctly.
tests/pw/tests/e2e/storeAppearance.spec.ts (1)
Line range hint
22-26
: LGTM on the widget initialization, but let's clarify the commented code.The initialization of empty sidebar widgets helps ensure test isolation. However, there are several commented-out widget configurations that should either be:
- Implemented if they're required for future test cases
- Removed if they're no longer needed
- Documented with a TODO comment explaining the intent
Let's check if any tests depend on specific widget configurations:
✅ Verification successful
Add a TODO comment to document widget configurations
The commented widget configurations are actively used in other test files (privacyPolicy.spec.ts, singleStore.spec.ts) and should be documented for future test cases. Add a TODO comment explaining their purpose:
await dbUtils.setOptionValue('sidebars_widgets', dbData.emptySideBarsWidgets); // TODO: Keep these widget configurations for future test cases // These are used in privacy policy and single store tests for: // - Store contact form (vendor communication) // - Store location (vendor details) // - Store open/close times (business hours) // await dbUtils.updateOptionValue(dbData.dokanWidgets.names.storeLocation, ...); // await dbUtils.updateOptionValue(dbData.dokanWidgets.names.storeOpenClose, ...); // await dbUtils.updateOptionValue(dbData.dokanWidgets.names.storeContactForm, ...); // await dbUtils.updateOptionValue('sidebars_widgets', ...);🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Search for tests that might depend on widget configurations rg -A 5 "sidebars_widgets|storeLocation|storeOpenClose|storeContactForm" --type tsLength of output: 16299
tests/pw/pages/vendorPage.ts (1)
54-54
: Verify timeout implications of stricter wait condition.The change to wait for
networkIdle
is more robust but could potentially increase test execution time or cause timeouts in slow network conditions.Let's check for potential timeout issues in the CI logs:
✅ Verification successful
Network idle wait condition is appropriate for product edit page load
The wait condition is used specifically for the product edit page load where ensuring complete dynamic content loading is crucial for test reliability. Playwright's built-in network idle implementation has reasonable defaults and the subsequent assertion validates the page load.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Search for timeout errors related to networkIdle rg "Timeout.*networkIdle|TimeoutError.*networkIdle" --type logLength of output: 213
Script:
#!/bin/bash # Find implementation of clickAndWaitForResponseAndLoadStateUntilNetworkIdle ast-grep --pattern 'clickAndWaitForResponseAndLoadStateUntilNetworkIdle($$$) { $$$ }' # Find similar usage patterns rg "waitForLoadState.*networkIdle|waitForResponse.*networkIdle" --type ts # Look for timeout configurations rg "timeout.*networkIdle" --type tsLength of output: 203
Script:
#!/bin/bash # Find potential base page class fd "page.ts" tests/pw --exec grep -l "class.*Page" # Look for playwright config fd "playwright.config.ts" # Broader search for networkIdle usage rg "networkIdle" --type ts # Find class definition in vendorPage.ts cat tests/pw/pages/vendorPage.tsLength of output: 23166
tests/pw/tests/e2e/productsDetails.spec.ts (1)
18-19
: LGTM! Good naming standardization using IDs.The change from
productName
toproductId
improves code clarity by using more reliable identifiers.tests/pw/pages/productsPage.ts (2)
624-631
: LGTM! Robust validation and fallback.The method properly validates the product ID and includes a fallback mechanism.
Line range hint
816-1661
: Verify the impact of nonce usage across the codebase.The switch to using nonces improves security but requires verification of all product edit flows.
✅ Verification successful
Nonce implementation for product edit URLs is properly secured
The verification confirms that:
- Product edit URLs are properly defined with nonce parameter in testData.ts
- All product edit operations use the secure URL pattern through proper imports
- Click handlers implement proper response and load state waiting
- No direct URL manipulation or insecure patterns were found
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Check for any remaining direct product edit URL usage that might bypass nonce validation # Search for product edit URL patterns without nonce rg -A 2 "dashboard/products/\?product_id=" --glob "!{productsPage.ts}" # Search for old method usage rg "goToProductEdit\(" --glob "!{productsPage.ts}"Length of output: 816
Script:
#!/bin/bash # Check implementation of goToProductEdit methods echo "=== commissionPage.ts implementation ===" ast-grep --pattern 'async goToProductEdit($_, $_) { $$$ }' tests/pw/pages/commissionPage.ts echo -e "\n=== vendorPage.ts implementation ===" ast-grep --pattern 'async goToProductEdit($_, $_) { $$$ }' tests/pw/pages/vendorPage.tsLength of output: 367
Script:
#!/bin/bash echo "=== commissionPage.ts implementation ===" rg -B 2 -A 10 "async goToProductEdit" tests/pw/pages/commissionPage.ts echo -e "\n=== vendorPage.ts implementation ===" rg -B 2 -A 10 "async goToProductEdit" tests/pw/pages/vendorPage.ts echo -e "\n=== Checking imports ===" rg "import.*testData" tests/pw/pages/{commission,vendor}Page.tsLength of output: 2089
Script:
#!/bin/bash # Check productsVendor implementation and usage echo "=== Checking productsVendor import ===" rg "import.*productsVendor" tests/pw/pages/{commission,vendor}Page.ts echo -e "\n=== Checking productsVendor definition ===" rg -B 2 -A 5 "editProduct.*=.*" tests/pw/pages/ tests/pw/utils/Length of output: 2339
tests/pw/utils/testData.ts (1)
1176-1176
: LGTM! Consistent URL construction.The method properly constructs the product edit URL with required parameters.
tests/pw/package.json (2)
10-14
: LGTM! Enhanced browser installation options.The script commands now provide more flexibility for installing browsers and dependencies.
56-58
: Verify compatibility with updated dependencies.The dependency updates include TypeScript-related packages which may affect type checking.
Also applies to: 72-72
✅ Verification successful
Dependencies are compatible with the project's TypeScript configuration
The updated TypeScript-related dependencies are compatible with the project's strict TypeScript configuration. The test suite uses modern TypeScript features and strict type checking, which are well-supported by the updated versions of @types/node and @typescript-eslint packages.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Check for any type errors with updated dependencies # Run TypeScript compiler in noEmit mode npm run type # Check for any TypeScript-specific lint errors npm run lint -- --ext .tsLength of output: 571
Script:
#!/bin/bash # Check for TypeScript configuration and usage echo "=== TypeScript Config ===" fd tsconfig.json echo -e "\n=== TypeScript Files ===" fd -e ts -e tsx echo -e "\n=== Package Scripts ===" cat tests/pw/package.json | jq '.scripts'Length of output: 11312
Script:
#!/bin/bash # Check TypeScript configuration cat tests/pw/tsconfig.json # Check if there are any TypeScript errors in the current state cd tests/pw && npx tsc --noEmitLength of output: 5087
tests/pw/utils/summaryReporter.ts (2)
82-83
: LGTM! Improved test tracking documentation.The comment clearly explains the rationale for including skipped tests in the total count.
91-91
: Verify the impact on test reporting accuracy.The changes to test counting logic may affect reporting statistics.
Also applies to: 96-96, 100-100
✅ Verification successful
The changes to test counting logic are safe and accurate
The modification improves consistency in test tracking while maintaining accurate statistics. The core test counting logic (passed, failed, flaky, skipped) remains unchanged as it uses a separate mechanism based on test outcomes rather than the summary.tests array.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify test reporting accuracy with new counting logic # Run tests and check summary report format npm run test:e2e -- --reporter=list cat playwright-report/summary.json | jq '.total_tests, .skipped, .passed, .failed, .flaky'Length of output: 418
Script:
#!/bin/bash # Get the full context of summaryReporter.ts cat tests/pw/utils/summaryReporter.ts # Search for files that might depend on summary.tests rg -l "summary\.tests" --type tsLength of output: 4655
.github/workflows/e2e_api_tests.yml (3)
68-76
: LGTM! Consider monitoring the timeout value.The sharding configuration with 3 shards is well-structured and the fail-fast strategy is correctly disabled. The 20-minute timeout seems reasonable but should be monitored in production to ensure it's sufficient for all test scenarios.
207-215
: LGTM! API test configuration is well-structured.The API test configuration with a single shard is appropriate, and the 10-minute timeout should be sufficient for API tests.
160-167
: Verify if 15-minute timeout is sufficient for sharded tests.The timeout has been significantly reduced from 40 to 15 minutes. While sharding should make tests faster, this is a substantial reduction that should be validated.
Run this script to check historical test execution times:
✅ Verification successful
15-minute timeout is appropriate for current test execution patterns
Recent workflow runs consistently complete within 14-15 minutes, with the longest observed run taking 18m10s. The current timeout provides an appropriate balance between allowing tests to complete and preventing hung jobs.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Check historical test execution times from GitHub Actions logs gh run list --workflow=e2e_api_tests.yml --json durationMs,conclusion --jq '.[] | select(.conclusion=="success") | .durationMs' | sort -nLength of output: 375
Script:
#!/bin/bash # Get git history of the workflow file to see when sharding was introduced git log -p .github/workflows/e2e_api_tests.yml | grep -A 5 "matrix.shard" # Get recent workflow runs with their timing information gh run list --workflow=e2e_api_tests.yml --json conclusion,startedAt,updatedAt --limit 20 \ | jq -r '.[] | select(.conclusion=="success") | (.startedAt + " to " + .updatedAt)'Length of output: 715
tests/pw/tests/api/calculation.spec.ts (1)
29-33
: LGTM! Appropriate removal of debug test.The removal of the skipped debug test case is appropriate as it was not part of the main test suite.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 0
🔭 Outside diff range comments (1)
tests/pw/tests/e2e/payments.spec.ts (1)
Line range hint
44-60
: Improve test cleanup implementation.There are commented-out cleanup operations that should be reviewed:
// await apiUtils.setStoreSettings(payloads.defaultStoreSettings, payloads.vendorAuth);
Incomplete cleanup could affect subsequent test runs and parallel execution. Please either:
- Remove the commented code if it's no longer needed
- Implement the cleanup if it's required
- Add a comment explaining why it's intentionally disabled
🧹 Nitpick comments (4)
tests/pw/tests/e2e/storelisting.spec.ts (1)
Line range hint
5-62
: Consider reorganizing tests for optimal parallel execution.Since this PR implements Playwright sharding, consider these improvements for better parallel execution:
- Split the test suite into smaller, focused describe blocks based on functionality (e.g., basic listing, filtering, navigation)
- Ensure test independence by avoiding shared state
- Consider adding test priorities or dependencies if certain tests must run in a specific order
Example structure:
test.describe('Basic Store Listing', () => { // Basic tests like view, sort, layout }); test.describe('Store Filtering', () => { // All filter-related tests }); test.describe('Store Navigation', () => { // Tests for map view and single store navigation });tests/pw/pages/menuManagerPage.ts (2)
28-28
: LGTM! Enhanced navigation reliability.The change to
gotoUntilNetworkidle
improves test stability by ensuring the page is fully loaded before proceeding.Consider adding a timeout parameter to handle slow network conditions:
- await this.gotoUntilNetworkidle(data.subUrls.backend.dokan.settings, { waitUntil: 'networkidle' }, true); + await this.gotoUntilNetworkidle(data.subUrls.backend.dokan.settings, { waitUntil: 'networkidle', timeout: 30000 }, true);
Line range hint
1-156
: Consider consistent application of network idle waiting.Other methods in the file still use
clickAndWaitForResponseAndLoadState
. For consistency and reliability, consider applying the same network idle waiting pattern to:
renameMenu
methodreorderMenu
methodresetMenuManagerSettings
methodExample for
renameMenu
:- await this.clickAndWaitForResponseAndLoadState(data.subUrls.ajax, settingsAdmin.saveChanges); + await this.clickAndWaitForResponseAndLoadStateUntilNetworkIdle(data.subUrls.ajax, settingsAdmin.saveChanges);tests/pw/tests/e2e/payments.spec.ts (1)
Line range hint
20-42
: Review test setup for parallel execution compatibility.The current test setup might face issues with parallel execution due to:
- Global state modifications through API calls (currency settings, module activation)
- Shared database access without proper isolation
- Dependencies between test cases
Consider the following improvements for better parallel execution support:
- Move shared setup to a global setup file
- Implement test isolation using unique data per test worker
- Use dynamic test data generation instead of fixed data
Example approach for test isolation:
// In a global setup file async function globalSetup(config: FullConfig) { // Perform one-time setup like module activation const apiUtils = new ApiUtils(await request.newContext()); if (DOKAN_PRO) { await Promise.all([ apiUtils.activateModules(payloads.moduleIds.mangopay, payloads.adminAuth), apiUtils.activateModules(payloads.moduleIds.paypalMarketplace, payloads.adminAuth), // ... other modules ]); } } // In the test file test.beforeEach(async ({ page }, testInfo) => { // Use test worker id for isolation const workerId = testInfo.worker.workerIndex; // Generate unique data per test const testData = generateUniqueTestData(workerId); });
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (4)
tests/pw/package.json
(4 hunks)tests/pw/pages/menuManagerPage.ts
(2 hunks)tests/pw/tests/e2e/payments.spec.ts
(1 hunks)tests/pw/tests/e2e/storelisting.spec.ts
(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- tests/pw/package.json
⏰ Context from checks skipped due to timeout of 90000ms (4)
- GitHub Check: e2e tests (3, 3)
- GitHub Check: e2e tests (2, 3)
- GitHub Check: e2e tests (1, 3)
- GitHub Check: api tests (1, 1)
🔇 Additional comments (3)
tests/pw/tests/e2e/storelisting.spec.ts (1)
53-55
: Please clarify why this test is being skipped.Given that this PR focuses on implementing Playwright sharding for parallel test execution, it would be helpful to understand:
- Is this skip temporary during the sharding implementation?
- Are there specific issues with this test when running in parallel?
- Could you add a comment explaining why the test is skipped?
This test appears to cover important functionality for filtering stores that are currently open, and skipping it might impact our test coverage.
Let's check if there are any related issues or discussions:
tests/pw/pages/menuManagerPage.ts (1)
34-34
: LGTM! Improved state management reliability.The change to
clickAndWaitForResponseAndLoadStateUntilNetworkIdle
ensures that save operations complete fully before proceeding with assertions.Please verify error handling for network timeouts. Consider running the following script to check for similar patterns in other test files:
Also applies to: 45-45
✅ Verification successful
Verified: Network idle wait is intentionally selective
The change to
clickAndWaitForResponseAndLoadStateUntilNetworkIdle
is correctly applied only to critical state-changing operations like save changes and product edits, where ensuring complete state synchronization is essential. The selective usage pattern across the codebase confirms this is an intentional design choice rather than an inconsistency.🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Check for consistent usage of network idle waiting patterns across test files # and identify potential places where error handling might be needed. # Search for similar save operations in other test files rg -t typescript "clickAndWaitForResponseAndLoadState[^U]" tests/pw/pages/ rg -t typescript "clickAndWaitForResponseAndLoadStateUntilNetworkIdle" tests/pw/pages/ # Check for try-catch blocks around network operations ast-grep --pattern 'try { $$$ await $_.clickAndWaitForResponseAndLoadStateUntilNetworkIdle($$$) $$$ } catch' tests/pw/pages/Length of output: 366
Script:
#!/bin/bash # Search for all occurrences of both method variants without file type restriction rg "clickAndWaitForResponseAndLoadState[^U]" tests/pw/pages/ rg "clickAndWaitForResponseAndLoadStateUntilNetworkIdle" tests/pw/pages/ # Look for method definitions ast-grep --pattern 'async clickAndWaitForResponseAndLoadState($_) { $$$ }' # Look for method definitions with network idle ast-grep --pattern 'async clickAndWaitForResponseAndLoadStateUntilNetworkIdle($_) { $$$ }' # Check for any existing error handling patterns in similar methods rg "try.*catch.*clickAndWaitForResponse" tests/pw/pages/Length of output: 31555
tests/pw/tests/e2e/payments.spec.ts (1)
Line range hint
95-99
: Document the rationale for skipping the Mangopay test.The test for adding Mangopay payment method has been skipped without explanation. Since this is part of a PR implementing test sharding, we need to understand:
- Is this a temporary skip to facilitate sharding implementation?
- Are there known issues with this test in parallel execution?
- If this is a permanent skip, what are the underlying reasons?
Please add a comment explaining why this test is being skipped to help maintain test suite documentation.
Let's verify if this test has any history of failures or issues in the CI pipeline:
* Add product Edit nonce & product edit navigation function * add comment * Add: playwright sharding for parallel test execution (getdokan#2504) * Add: product edit nonce to for direct navigation * Add: playwright sharding for parallel job * Update packages * Fix: flaky tests
All Submissions:
Changes proposed in this Pull Request:
Related Pull Request(s)
Closes
How to test the changes in this Pull Request:
Changelog entry
Title
Detailed Description of the pull request. What was previous behaviour
and what will be changed in this PR.
Before Changes
Describe the issue before changes with screenshots(s).
After Changes
Describe the issue after changes with screenshot(s).
Feature Video (optional)
Link of detailed video if this PR is for a feature.
PR Self Review Checklist:
FOR PR REVIEWER ONLY:
Summary by CodeRabbit
New Features
Chores
Bug Fixes
Tests