Ignore timeouts in the malicious site test suite #3646
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This allows us to still detect failures when the suite doesn't time out.
samsymons
commented
Dec 7, 2024
|
|
||
| switch result { | ||
| case .completed: break | ||
| case .timedOut: throw XCTSkip("Test timed out") |
There was a problem hiding this comment.
This is the key change of the PR, when we receive a time out we should skip it. This allows us to still check the conditions as usual when the timeouts aren't happening, but avoids failing the whole run otherwise. It'd be better to improve the test resilience properly, but for now I just want to get rid of the failures.
ayoy
approved these changes
Dec 9, 2024
There was a problem hiding this comment.
LGTM, I like skipping on external service timeout 👏 thanks @samsymons!
samsymons
deleted the
sam/update-malicious-sites-test-to-have-timeout-resilience
branch
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This allows us to still detect failures when the suite doesn't time out.
Task/Issue URL: https://app.asana.com/0/1199230911884351/1208926848453614/f
Tech Design URL:
CC:
Description:
This PR updates the malicious site tests to ignore time outs. These tests rely on an outside dependency which is not reliably loading; rather than disable them entirely, let's run them as expected in cases where the timeout doesn't occur.
Optional E2E tests:
Check this to run the Personal Information Removal end to end tests. If updating CCF, or any PIR related code, tick this.
Steps to test this PR:
Definition of Done:
Internal references:
Pull Request Review Checklist
Software Engineering Expectations
Technical Design Template
Pull Request Documentation