Skip to content

Commit

Permalink
add two exercises
Browse files Browse the repository at this point in the history
  • Loading branch information
bast committed Oct 7, 2024
1 parent 224011b commit 3cfc6a2
Show file tree
Hide file tree
Showing 3 changed files with 160 additions and 4 deletions.
92 changes: 88 additions & 4 deletions content/sharing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -160,7 +160,91 @@ Use version control and public registries
- Link the repo to the public registry


Exercise
--------

(work in progress - Radovan working on this part)
Exercises
---------

.. exercise:: Exercise Sharing-1: Time-travel with containers

Imagine the following situation: A researcher has written and published their research code which
requires a number of libraries and system dependencies. They ran their code
on a Linux computer (Ubuntu). One very nice thing they did was to publish
also a container image with all dependencies included, as well as the
definition file (below) to create the container image.

Now we travel 3 years into the future and want to reuse their work and adapt
it for our data. The container registry where they uploaded the container
image however no longer exists. But luckily (!) we still have the definition
file (below). From this we should be able to create a new container image.

- Can you anticipate problems using the definition file here 3 years after its
creation? Which possible problems can you point out?
- Discuss possible take-aways for creating more reusable containers.

.. tabs::

.. tab:: Python project using virtual environment

.. literalinclude:: sharing/bad-example-python.def
:language: singularity
:linenos:

.. solution::

- Line 2: "ubuntu:latest" will mean something different 3 years in future.
- Lines 11-12: The compiler gcc and the library libgomp1 will have evolved.
- Line 30: The container uses requirements.txt to build the virtual environment but we don't see
here what libraries the code depends on.
- Line 33: Data is copied in from the hard disk of the person who created it. Hopefully we can find the data somewhere.
- Line 35: The library fancylib has been built outside the container and copied in but we don't see here how it was done.
- Python version will be different then and hopefully the code still runs then.
- Singularity/Apptainer will have also evolved by then. Hopefully this definition file then still works.
- No help text.
- No contact address to ask more questions about this file.
- (Can you find more? Please contribute more points.)

.. literalinclude:: sharing/bad-example-python.def
:language: singularity
:linenos:
:emphasize-lines: 2, 11-12, 30, 33, 35

.. tab:: C++ project

This definition files has potential problems 3 years later. Further
down on this page we show a better and real version.

.. literalinclude:: sharing/bad-example-cxx.def
:language: singularity
:linenos:

.. solution::

- Line 2: "ubuntu:latest" will mean something different 3 years in future.
- Lines 9: The libraries will have evolved.
- Line 11: We clone a Git repository recursively and that repository might evolve until we build the container image the next time.
here what libraries the code depends on.
- Line 18: The library fancylib has been built outside the container and copied in but we don't see here how it was done.
- Singularity/Apptainer will have also evolved by then. Hopefully this definition file then still works.
- No help text.
- No contact address to ask more questions about this file.
- (Can you find more? Please contribute more points.)

.. literalinclude:: sharing/bad-example-cxx.def
:language: singularity
:linenos:
:emphasize-lines: 2, 9, 11, 18


.. exercise:: Exercise Sharing-2: Building a container on GitHub

You can build a container on GitHub (using GitHub Actions) or GitLab (using
GitLab CI) and host the image it on GitHub/GitLab. This has the following
advantages:
- You don't need to host it yourself.

Check failure on line 242 in content/sharing.rst

View workflow job for this annotation

GitHub Actions / Build and gh-pages

Unexpected indentation.
- But the image stays close to its sources and is not on a different service.
- Anybody can inspect the recipe and how it was built.
- Every time you make a change to the recipe, it builds a new image.

If you want to try this out:
- Take `this repository <https://github.com/bast/html2pdf>`_ as starting point and inspiration.
- Don't focus too much on what this container does, but rather `how it is built <https://github.com/bast/html2pdf/tree/main/.github/workflows>`_.
- To build a new version, one needs to send a pull request which updates ``VERSION`` and modifies the definition file.
26 changes: 26 additions & 0 deletions content/sharing/bad-example-cxx.def
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
Bootstrap: docker
From: ubuntu:latest

%post
export DEBIAN_FRONTEND=noninteractive
apt-get update -y

apt install -y git build-essential pkg-config
apt install -y libz-dev libbz2-dev liblzma-dev libcurl4-openssl-dev libssl-dev libgsl-dev

git clone --recursive https://github.com/someuser/sometool.git
cd sometool

make

%files
# Workaround to fix dependency on fancylib
/home/myself/fancylib /usr/lib/fancylib

%environment
export LC_ALL=C

%runscript
export PATH=/sometool:$PATH

$@
46 changes: 46 additions & 0 deletions content/sharing/bad-example-python.def
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
Bootstrap: docker
From: ubuntu:latest

%post
# Set environment variables
export VIRTUAL_ENV=/app/venv

# Install system dependencies and Python 3
apt-get update && \
apt-get install -y --no-install-recommends \
gcc \
libgomp1 \
python3 \
python3-venv \
python3-distutils \
python3-pip && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*

# Set up the virtual environment
python3 -m venv $VIRTUAL_ENV
. $VIRTUAL_ENV/bin/activate

# Install Python libraries
pip install --no-cache-dir --upgrade pip && \
pip install --no-cache-dir -r /app/requirements.txt

%files
# Copy project files
./requirements.txt /app/requirements.txt
./app.py /app/app.py
# Copy data
/home/myself/data /app/data
# Workaround to fix dependency on fancylib
/home/myself/fancylib /usr/lib/fancylib

%environment
# Set the environment variables
export LANG=C.UTF-8 LC_ALL=C.UTF-8
export VIRTUAL_ENV=/app/venv

%runscript
# Activate the virtual environment
. $VIRTUAL_ENV/bin/activate
# Run the application
python /app/app.py

0 comments on commit 3cfc6a2

Please sign in to comment.