The state of these workflows

[brian-rose]

Overview

I thought it would be useful to write down an explicit list of tasks that we want these workflows to accomplish, with a checklist to keep track of what still needs work.

What basic functionality are we trying to achieve? There's a matrix of different use cases that we are trying to cover, which fall into two basic groups: building (i.e. executing notebooks and rendering the html of the book pages), and publishing (pushing the built html to GitHub Pages)

Building

We are trying to support notebook execution on GitHub Actions as well as on a Binder, with and without credentials (or secrets) needed for the build, and for both Pull Requests and builds that originate from the main branch.

Use cases without credentials (or secrets)

For Cookbooks that don't need any credentials for data access, we are supporting:

• Scheduled nightly builds of the main branch, with notebook execution on GitHub Actions
• Scheduled nightly builds of the main branch, with notebook execution on a Binder
• Book build from Pull Request, with notebook execution on GitHub Actions
• Book build from Pull Request, with notebook execution on Binder

Use cases where secrets are needed

Supporting passing secrets through to notebook execution (such as the Radar Cookbook requires) makes things trickier. The list of use cases is the same as above, but the implementation is different:

• Scheduled nightly builds of the main branch, with notebook execution on GitHub Actions -- secrets needed
• Scheduled nightly builds of the main branch, with notebook execution on a Binder -- secrets needed
• Book build from Pull Request, with notebook execution on GitHub Actions -- secrets needed
• Book build from Pull Request, with notebook execution on Binder -- secrets needed

All these build cases (with and without secrets) must be able to handle changes in the environment file appropriately. That is the last sticking point. For the case with secrets and on Binder we still don't have a solution to point the Binder toward the correct environment file.

Publishing

Publishing here means deploying the built and rendered html to GitHub Pages. Use cases we need to support:

• Displaying previews of Pull Requests that are built without secrets
• Displaying previews of Pull Requests built with secrets
• Displaying the preview link and/or notifying the user when preview is ready (avoid displaying non-functional links)
• Publishing the built book upon push to main branch
• Publishing the built book after successful nightly build (Nightly Build Does Not Push to Github Pages #62)

I'm checking off things that I feel are adequately supported with our current code.

[brian-rose]

Maybe the text above can serve as the beginnings of some documentation for this elaborate build-and-publish machine that we've been building.