Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Prod Testing Workflows #10061

Draft
wants to merge 44 commits into
base: main
Choose a base branch
from
Draft

WIP: Prod Testing Workflows #10061

wants to merge 44 commits into from

Conversation

oskirby
Copy link
Collaborator

@oskirby oskirby commented Nov 21, 2024

Description

This is all broken - don't look. Well, I'll be damned! It works.

Since we have a lot of builds happening in the VPN, across a lot of platforms, it's a bit of a problem that we don't really have a solid connection between the builds that we ship to users and the thing that gets tested by our CI. This PR attempts to resolve that discrepancy by downloading the build artifacts directly off taskcluster and running the functional test suite against them.

To get there, we did a bunch of things:

  • Write some Github action magic to resolve the taskcluster jobs that are associated with a given workflow run.
  • Once the build tasks are known, we then have to wait for the builds to finish and then download the artifacts off taskcluster.
  • We do a bit of refactoring the functional tests into a reusable workflow and feed it with the artifacts from Taskcluster.
  • Then we go completely wild and use a matrix strategy to run the tests on basically every kind of runner that github will give us.

As a downside, there are a bunch of tests that appear to be permabroken when run from the production artifacts.

Reference

i.e Jira or Github issue URL

Checklist

  • My code follows the style guidelines for this project
  • I have not added any packages that contain high risk or unknown licenses (GPL, LGPL, MPL, etc. consult with DevOps if in question)
  • I have performed a self review of my own code
  • I have commented my code PARTICULARLY in hard to understand areas
  • I have added thorough tests where needed

@oskirby oskirby force-pushed the naomi-release-tests branch 3 times, most recently from cee5d52 to 3087f07 Compare November 21, 2024 21:42
@oskirby oskirby changed the title WIP: Release Testing Workflows WIP: Prod Testing Workflows Nov 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant