-
Notifications
You must be signed in to change notification settings - Fork 349
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
iOS end to end workflow changes related to iOS 17 and iPad #6445
Conversation
84f90cd
to
abc5768
Compare
9798586
to
b341018
Compare
1de01e9
to
1047d7d
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 14 of 15 files at r1, 2 of 2 files at r6, all commit messages.
Reviewable status: all files reviewed, 3 unresolved discussions (waiting on @niklasberglund)
.github/workflows/ios.yml
line 59 at r6 (raw file):
with: limit-access-to-actor: true detached: true
I don't think we need this in prod.
ios/MullvadVPNUITests/README.md
line 51 at r6 (raw file):
* `MullvadVPNUITestsVerifyDNSSettingsChanged` - Verify custom DNS settings still changed * `MullvadVPNUITestsChangeSettings` - Change all settings except custom DNS setting * `MullvadVPNUITestsVerifySettingsChanged` - Verify all settings except custom DNS setting still changed
I need that trailing newline.
ios/MullvadVPNUITests/tests.json
line 18 at r6 (raw file):
] } }
This could also do with a newline at the end.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 1 of 15 files at r1.
Reviewable status: 14 of 16 files reviewed, 3 unresolved discussions (waiting on @niklasberglund and @pinkisemils)
99cd75d
to
372cf99
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: 1 of 16 files reviewed, 3 unresolved discussions (waiting on @buggmagnet, @niklasberglund, and @pinkisemils)
372cf99
to
c60fbe6
Compare
This PR changes how iOS end to end test workflows are set up because we ran into some issues. The test iPhone was changed. First it was changed to an iPad and then to another iPhone. The new iPhone is running iOS 17 which don't reinstall the app when launched from tests. So the workflow instead invoke multiple test jobs which run one test suite each. This way we have more control of what we want to do in between test suite runs in CI(uninstall app).
This PR also adds support for running tests on iPad, some minor changes were needed. There's a new configuration property
TEST_DEVICE_IS_IPAD
which should be set inUITests.xcconfig
. Also it can now be configured inUITests.xcconfig
whether app should be uninstalled in test suite teardown. This is by default true(1
) for running locally, set to false(0
) in CI.The workflow file
ios-end-to-end-tests.yml
is now a reusable workflow which is reused by other thin workflows.More information about this in
ios/MullvadVPNUITests/README.md
.There are some things that should probably still be improved on:
TEST_DEVICE_UDID
inios-end-to-end-tests.yml
andios-end-to-end-tests-settings-migration.yml
. This should be more dynamic to be easier to set up runner on other computers and to run tests on multiple devices.The changes in this PR can be tested by running the iOS end-to-end tests workflow https://github.com/mullvad/mullvadvpn-app/actions/workflows/ios-end-to-end-tests.yml and specifying the PR branch. The PR also introduces other workflows but unfortunately they won't show up in GitHub until they've been merged to
main
. It might be easiest to ask Niklas for a demo?The
ios/MullvadVPNUITests/README.md
has been updated to reflect changes introduced in this PR and better describe the end to end tests, but all README changes are in a separate PR #6446This change is