Improve performance and simplify WorkspaceCreate view

[mikerkelly]

The WorkspaceCreate view (/{project_name}/new-workspace/) takes 5-10s to load, both on initial visit and when submitting the form. This is poor UX and will worsen over time as more repos and branches are created in the opensafely GitHub organization. It would be good to improve this if not too expensive. In mitigation, new workspaces are only occasionally created (there are ~500 in prod over 3.5 years).

Analysis

Almost all of the execution time comes from github.get_repos_with_branches. The form does need the list of repos to populate the repo selector. And the branches for the repo that is selected. But it doesn't need to know all of the branches. It should be quicker to fetch just the repos list initially, then the branches for a single repo as needed.

Ad hoc testing with curl should it takes about a second to get the list of repos, and less than 0.5s to get one branch:

time curl -o /dev/null  -H "Authorization: token $TOKEN"  -H "Accept: application/vnd.github.v3+json"  "https://api.github.com/orgs/opensafely/repos"
real 0m1.015s user 0m0.064s sys	0m0.009s


time curl -o /dev/null  -H "Authorization: token $TOKEN"  -H "Accept: application/vnd.github.v3+json"  "https://api.github.com/repos/opensafely/amr-uom-brit/branches"
real 0m0.332s user 0m0.079s sys 0m0.011s

Wizard solution

A "wizard" refers to a multi-step form process that guides users through a series of steps or stages to complete a task. We've not used these so far but they could perform better and be simpler to maintain than the current approach.

The repo selector could be on a separate page. Then the initial form needs the repo list, and the separate page needs the branches for the selected repo.

Could use Django's built-in FormView and manage state between steps e.g in the session, or django-formtools library's WizardView. Or HTMX for a single-page multi-part wizard?

Single-page solution

Sketch:

• The view or form calls github.get_repos to get the list of repos in the opensafely organization and populate the repos ChoiceField. Cache it for the clean.
• The unbound form has no branches populated initially.
• A bound form calls github.get_branches to populate the initial branches field. Or pseudo-cache it in the POST data payload. Cache it for the clean.
• Expose github.get_branches as a JSON data endpoint.
  • Possibly needs some secret shared with the form to avoid spam, as it makes an expensive call.
• workspace_create.js makes an AJAX request to the new endpoint when the repo selector changes, instead of looking up in the repos_with_branches currently embedded in the HTML page.
• The form clean uses the cached repos and branches for validation, or calls github.get_branches if it has changed.
• Remove github.get_repos_with_branches.

Possibly this would be better with HTMX?

Other options

• Improve the performance of github.get_repos_with_branches. I haven't fully understood why it is slow. It's written with GraphQL with performance in mind. Not clear there are improvements to make. The remote GitHub API may not provide a more efficient way to get all the data.
• We could cache the repo info in the DB. This seems like overkill for this one form.
• The form could POST all the repo and branch data back to the view as a form of pseudo-caching. Then the bound form wouldn't need to re-query the API. A hard-to-reason-about abuse of the form payload. And it wouldn't speed the initial unbound form, which is by far the most common case (bound form is only seen on validation error).