Use Github Actions to build and deploy container

[arisp99]

Github Actions could be a useful tool for us to test, document, and even build our container.

While there is a lot of actions we could eventually incorporate, as of now, the one that most catches my eye is the ability to build our container each time we push. The https://github.com/singularityhub/github-ci repo provides three example actions that we could experiment with.

With these actions, we can build and test the container. We can also use the Github Package repository to store our built containers. This can make it easier for users to download the built container, rather than needing to build the container on their own machine...

Before I go ahead and try this out, @AshlinHarris what do you think about implementing this action and Github Actions in general?

[arisp99]

I have been able to successfully configure a GHA workflow that does the following:

1. Sets up Singularity
2. Builds the container
3. Deploys the container to Github Packages with a tag
   1. The tag is dev if it is the development version.
   2. The tag is v?.?.? if there has been a release.
   3. The tag can be manually set.

I have tested the workflow in the https://github.com/bailey-lab/deploy-singularity-testing repo. Installing the container at this point can be done using the Singularity CLI as follows:

# N.B. I am using the example repo and the dev tag of the container.
singularity pull oras://ghcr.io/bailey-lab/deploy-singularity-testing:dev

Note that as of now, I am only able to pull the container on Oscar, and not on Seekdeep. I believe this is because of differences in the version number of singularity. Oscar is running version 3.8.0-1.el7, whereas Seekdeep is running version 3.6.4.

Additional notes:

• The workflow will attempt to build the container on both pushes and on PRs. However, it will not deploy the container if it is triggered by a PR.
• You can also manually trigger the workflow and set a custom tag.
• Currently, the action is triggered for several cases:
  • Push to main/master
  • PR
  • Release
  • Manually
• The triggers do not check to see if the definition file has been altered. It may be useful to trigger the GHA using more stringent constraints. For instance, we may want to avoid building the container if we make changes to a certain folder (e.g. /docs) that is not important in the building of the container.

[arisp99]

The workflow runs all terminate with the following error message:

g++-9: fatal error: Killed signal terminated program cc1plus
compilation terminated.
make: *** [Makefile:90: build/MIPWranglerBuild/src/mipster/filtering/filtering.o] Error 1
make: *** Waiting for unfinished jobs....
g++-9: fatal error: Killed signal terminated program cc1plus
compilation terminated.
make: *** [Makefile:90: build/MIPWranglerBuild/src/mipster/objects/MipMapResult.o] Error 1
g++-9: fatal error: Killed signal terminated program cc1plus
compilation terminated.
make: *** [Makefile:90: build/MIPWranglerBuild/src/mipster/objects/MippedRead.o] Error 1
FATAL:   While performing build: while running engine: exit status 2
Error: Process completed with exit code 255.

After some investigation, it appears that this may be due to hardware limitations of the Github runners. In particular, it may be due to a memory issue. Github-hosted runners are given 7 GB of RAM memory.

Potential Solutions

Solution: Reduce the size of the container, thereby reducing memory usage when building the container.
Limitations: None that I can think of. I think this is the preferred long term solution, as some processes are slowly being moved over to the R side with https://github.com/bailey-lab/miplicorn.

Solution: Use a self-hosted runner to build the container on GHA.
Limitations: There are security threats with hosting your own runner and using it in a public repository. See Github's warnings: https://docs.github.com/en/actions/hosting-your-own-runners/adding-self-hosted-runners.

Solution: Manually build and upload the container to Github packages. For this, we can use Seekdeep, which is able to build the image.
Limitations: We need to manually build the container rather than automating the process.