Add documentation rendering of package versions

This can be enhanced as needed, but the basic versions and tooling is
there.

Jinja2 is installed in the images, so we re-use the images themselves to
render the template.

Dockerfile

@@ -16,8 +16,9 @@ RUN mkdir /workspace
 WORKDIR /workspace
 
 ARG MAJOR_VERSION
+ARG BASE
 # ACTION_EXEC sets the default executable for the entrypoint in the base-docker image
-ENV ACTION_EXEC=python MAJOR_VERSION=${MAJOR_VERSION}
+ENV ACTION_EXEC=python MAJOR_VERSION=${MAJOR_VERSION} BASE=${BASE}
 
 COPY ${MAJOR_VERSION}/dependencies.txt /root/dependencies.txt
 # use space efficient utility from base image

justfile

@@ -19,6 +19,12 @@ test version *args="tests -v": (build version)
 # run pip-compile to add new dependencies, or update existing ones with --upgrade
 update version *args="": (build version)
     docker-compose --env-file {{ version }}/env run --rm -v $PWD:/workspace base pip-compile {{ args }} {{ version }}/requirements.in -o {{ version }}/requirements.txt
+    {{ just_executable() }} render {{ version }}
+
+
+# render package version information
+render version *args:
+    docker-compose --env-file {{ version }}/env run --rm -v $PWD:/workspace python ./scripts/render.py {{ args }} > {{ version }}/packages.md
 
 
 # run linters

scripts/packages.j2.md

@@ -0,0 +1,12 @@
+# Package Versions for {{ MAJOR_VERSION }}
+
+This python:{{ MAJOR_VERSION }} OpenSAFELY image is based on Ubuntu {{ BASE }} with Python {{ PYTHON_VERSION }}.
+
+## Packages
+
+It comes pre-installed with a standard set of python packages.
+
+{% for pkg in PACKAGES -%}
+ - [{{ pkg | replace("==", ": ")}}](https://pypi.org/project/{{pkg.name}}/{{pkg.specs[0][1]}}/)
+{% endfor -%}
+

scripts/render.py

@@ -0,0 +1,24 @@
+#!/usr/bin/env -S python3 -W ignore
+from pathlib import Path
+import os
+import sys
+import pkg_resources
+
+from jinja2 import Environment, FileSystemLoader
+env = Environment(loader=FileSystemLoader("scripts"))
+
+version = os.environ["MAJOR_VERSION"]
+requirements = Path(version) / "requirements.txt"
+
+context = {
+    "MAJOR_VERSION": version,
+    "BASE": os.environ["BASE"],
+    "PYTHON_VERSION": "{}.{}.{}".format(*sys.version_info),
+}
+
+with requirements.open() as r:
+    context["PACKAGES"] = list(pkg_resources.parse_requirements(r))
+
+template = env.get_template("packages.j2.md")
+
+print(template.render(**context))

v1/packages.md

@@ -0,0 +1,148 @@
+# Package Versions for v1
+
+> [!NOTE]  
+> This file is auto-generated - do not edit.
+
+This python:v1 OpenSAFELY image is based on Ubuntu 20.04 with Python 3.8.10.
+
+## Packages
+
+It comes pre-installed with a standard set of python packages.
+
+- [astor: 0.8.1](https://pypi.org/project/astor/0.8.1/)
+- [attrs: 19.3.0](https://pypi.org/project/attrs/19.3.0/)
+- [autograd: 1.3](https://pypi.org/project/autograd/1.3/)
+- [autograd-gamma: 0.5.0](https://pypi.org/project/autograd-gamma/0.5.0/)
+- [backcall: 0.1.0](https://pypi.org/project/backcall/0.1.0/)
+- [bash-kernel: 0.7.2](https://pypi.org/project/bash-kernel/0.7.2/)
+- [bleach: 3.1.2](https://pypi.org/project/bleach/3.1.2/)
+- [cachetools: 4.0.0](https://pypi.org/project/cachetools/4.0.0/)
+- [cairocffi: 1.4.0](https://pypi.org/project/cairocffi/1.4.0/)
+- [cairosvg: 2.5.2](https://pypi.org/project/cairosvg/2.5.2/)
+- [certifi: 2019.11.28](https://pypi.org/project/certifi/2019.11.28/)
+- [cffi: 1.15.1](https://pypi.org/project/cffi/1.15.1/)
+- [chardet: 3.0.4](https://pypi.org/project/chardet/3.0.4/)
+- [click: 7.0](https://pypi.org/project/click/7.0/)
+- [click-plugins: 1.1.1](https://pypi.org/project/click-plugins/1.1.1/)
+- [cligj: 0.5.0](https://pypi.org/project/cligj/0.5.0/)
+- [coverage: 4.5.4](https://pypi.org/project/coverage/4.5.4/)
+- [cssselect2: 0.7.0](https://pypi.org/project/cssselect2/0.7.0/)
+- [cycler: 0.10.0](https://pypi.org/project/cycler/0.10.0/)
+- [decorator: 4.4.1](https://pypi.org/project/decorator/4.4.1/)
+- [defusedxml: 0.6.0](https://pypi.org/project/defusedxml/0.6.0/)
+- [descartes: 1.1.0](https://pypi.org/project/descartes/1.1.0/)
+- [ebmdatalab: 0.0.30](https://pypi.org/project/ebmdatalab/0.0.30/)
+- [entrypoints: 0.3](https://pypi.org/project/entrypoints/0.3/)
+- [fiona: 1.8.13](https://pypi.org/project/fiona/1.8.13/)
+- [formulaic: 0.2.4](https://pypi.org/project/formulaic/0.2.4/)
+- [future: 0.18.2](https://pypi.org/project/future/0.18.2/)
+- [geopandas: 0.6.3](https://pypi.org/project/geopandas/0.6.3/)
+- [google-api-core: 1.16.0](https://pypi.org/project/google-api-core/1.16.0/)
+- [google-auth: 1.11.0](https://pypi.org/project/google-auth/1.11.0/)
+- [google-auth-oauthlib: 0.4.1](https://pypi.org/project/google-auth-oauthlib/0.4.1/)
+- [google-cloud-bigquery: 1.24.0](https://pypi.org/project/google-cloud-bigquery/1.24.0/)
+- [google-cloud-core: 1.3.0](https://pypi.org/project/google-cloud-core/1.3.0/)
+- [google-resumable-media: 0.5.0](https://pypi.org/project/google-resumable-media/0.5.0/)
+- [googleapis-common-protos: 1.51.0](https://pypi.org/project/googleapis-common-protos/1.51.0/)
+- [idna: 2.8](https://pypi.org/project/idna/2.8/)
+- [interface-meta: 1.2.4](https://pypi.org/project/interface-meta/1.2.4/)
+- [ipykernel: 5.1.4](https://pypi.org/project/ipykernel/5.1.4/)
+- [ipython: 7.12.0](https://pypi.org/project/ipython/7.12.0/)
+- [ipython-genutils: 0.2.0](https://pypi.org/project/ipython-genutils/0.2.0/)
+- [ipywidgets: 7.5.1](https://pypi.org/project/ipywidgets/7.5.1/)
+- [jedi: 0.16.0](https://pypi.org/project/jedi/0.16.0/)
+- [jinja2: 2.11.1](https://pypi.org/project/jinja2/2.11.1/)
+- [joblib: 1.0.1](https://pypi.org/project/joblib/1.0.1/)
+- [json5: 0.9.0](https://pypi.org/project/json5/0.9.0/)
+- [jsonschema: 3.2.0](https://pypi.org/project/jsonschema/3.2.0/)
+- [jupyter: 1.0.0](https://pypi.org/project/jupyter/1.0.0/)
+- [jupyter-client: 5.3.4](https://pypi.org/project/jupyter-client/5.3.4/)
+- [jupyter-console: 6.1.0](https://pypi.org/project/jupyter-console/6.1.0/)
+- [jupyter-core: 4.6.1](https://pypi.org/project/jupyter-core/4.6.1/)
+- [jupyterlab: 1.2.6](https://pypi.org/project/jupyterlab/1.2.6/)
+- [jupyterlab-server: 1.0.6](https://pypi.org/project/jupyterlab-server/1.0.6/)
+- [jupytext: 1.3.3](https://pypi.org/project/jupytext/1.3.3/)
+- [kaleido: 0.2.1](https://pypi.org/project/kaleido/0.2.1/)
+- [kiwisolver: 1.1.0](https://pypi.org/project/kiwisolver/1.1.0/)
+- [lifelines: 0.26.4](https://pypi.org/project/lifelines/0.26.4/)
+- [llvmlite: 0.34.0](https://pypi.org/project/llvmlite/0.34.0/)
+- [lz4: 3.1.3](https://pypi.org/project/lz4/3.1.3/)
+- [markupsafe: 1.1.1](https://pypi.org/project/markupsafe/1.1.1/)
+- [matplotlib: 3.1.3](https://pypi.org/project/matplotlib/3.1.3/)
+- [mistune: 0.8.4](https://pypi.org/project/mistune/0.8.4/)
+- [more-itertools: 8.2.0](https://pypi.org/project/more-itertools/8.2.0/)
+- [munch: 2.5.0](https://pypi.org/project/munch/2.5.0/)
+- [nbconvert: 5.6.1](https://pypi.org/project/nbconvert/5.6.1/)
+- [nbformat: 5.0.4](https://pypi.org/project/nbformat/5.0.4/)
+- [nbval: 0.9.4](https://pypi.org/project/nbval/0.9.4/)
+- [notebook: 6.0.3](https://pypi.org/project/notebook/6.0.3/)
+- [numba: 0.51.2](https://pypi.org/project/numba/0.51.2/)
+- [numpy: 1.18.1](https://pypi.org/project/numpy/1.18.1/)
+- [oauthlib: 3.1.0](https://pypi.org/project/oauthlib/3.1.0/)
+- [opensafely-cohort-extractor: 1.88.0](https://pypi.org/project/opensafely-cohort-extractor/1.88.0/)
+- [opensafely-matching: 0.2.0](https://pypi.org/project/opensafely-matching/0.2.0/)
+- [packaging: 20.1](https://pypi.org/project/packaging/20.1/)
+- [pandas: 1.0.1](https://pypi.org/project/pandas/1.0.1/)
+- [pandas-gbq: 0.13.0](https://pypi.org/project/pandas-gbq/0.13.0/)
+- [pandocfilters: 1.4.2](https://pypi.org/project/pandocfilters/1.4.2/)
+- [parso: 0.6.1](https://pypi.org/project/parso/0.6.1/)
+- [patsy: 0.5.1](https://pypi.org/project/patsy/0.5.1/)
+- [pep517: 0.10.0](https://pypi.org/project/pep517/0.10.0/)
+- [pexpect: 4.8.0](https://pypi.org/project/pexpect/4.8.0/)
+- [pickleshare: 0.7.5](https://pypi.org/project/pickleshare/0.7.5/)
+- [pillow: 8.1.0](https://pypi.org/project/pillow/8.1.0/)
+- [pip-tools: 6.2.0](https://pypi.org/project/pip-tools/6.2.0/)
+- [plotly: 4.5.0](https://pypi.org/project/plotly/4.5.0/)
+- [pluggy: 0.13.1](https://pypi.org/project/pluggy/0.13.1/)
+- [prometheus-client: 0.7.1](https://pypi.org/project/prometheus-client/0.7.1/)
+- [prompt-toolkit: 3.0.3](https://pypi.org/project/prompt-toolkit/3.0.3/)
+- [protobuf: 3.11.3](https://pypi.org/project/protobuf/3.11.3/)
+- [ptyprocess: 0.6.0](https://pypi.org/project/ptyprocess/0.6.0/)
+- [py: 1.8.1](https://pypi.org/project/py/1.8.1/)
+- [pyarrow: 3.0.0](https://pypi.org/project/pyarrow/3.0.0/)
+- [pyasn1: 0.4.8](https://pypi.org/project/pyasn1/0.4.8/)
+- [pyasn1-modules: 0.2.8](https://pypi.org/project/pyasn1-modules/0.2.8/)
+- [pycparser: 2.21](https://pypi.org/project/pycparser/2.21/)
+- [pydata-google-auth: 0.3.0](https://pypi.org/project/pydata-google-auth/0.3.0/)
+- [pygments: 2.5.2](https://pypi.org/project/pygments/2.5.2/)
+- [pyparsing: 2.4.6](https://pypi.org/project/pyparsing/2.4.6/)
+- [pyproj: 2.4.2.post1](https://pypi.org/project/pyproj/2.4.2.post1/)
+- [pyrsistent: 0.15.7](https://pypi.org/project/pyrsistent/0.15.7/)
+- [pytest: 5.3.5](https://pypi.org/project/pytest/5.3.5/)
+- [python-dateutil: 2.8.1](https://pypi.org/project/python-dateutil/2.8.1/)
+- [pytz: 2019.3](https://pypi.org/project/pytz/2019.3/)
+- [pyyaml: 5.3](https://pypi.org/project/pyyaml/5.3/)
+- [pyzmq: 18.1.1](https://pypi.org/project/pyzmq/18.1.1/)
+- [qtconsole: 4.6.0](https://pypi.org/project/qtconsole/4.6.0/)
+- [requests: 2.22.0](https://pypi.org/project/requests/2.22.0/)
+- [requests-oauthlib: 1.3.0](https://pypi.org/project/requests-oauthlib/1.3.0/)
+- [retry: 0.9.2](https://pypi.org/project/retry/0.9.2/)
+- [retrying: 1.3.3](https://pypi.org/project/retrying/1.3.3/)
+- [rsa: 4.0](https://pypi.org/project/rsa/4.0/)
+- [scikit-learn: 0.24.1](https://pypi.org/project/scikit-learn/0.24.1/)
+- [scipy: 1.4.1](https://pypi.org/project/scipy/1.4.1/)
+- [seaborn: 0.10.0](https://pypi.org/project/seaborn/0.10.0/)
+- [send2trash: 1.5.0](https://pypi.org/project/send2trash/1.5.0/)
+- [shapely: 1.7.0](https://pypi.org/project/shapely/1.7.0/)
+- [six: 1.14.0](https://pypi.org/project/six/1.14.0/)
+- [sqlparse: 0.4.1](https://pypi.org/project/sqlparse/0.4.1/)
+- [statsmodels: 0.11.0](https://pypi.org/project/statsmodels/0.11.0/)
+- [structlog: 20.2.0](https://pypi.org/project/structlog/20.2.0/)
+- [tabulate: 0.8.7](https://pypi.org/project/tabulate/0.8.7/)
+- [terminado: 0.8.3](https://pypi.org/project/terminado/0.8.3/)
+- [testpath: 0.4.4](https://pypi.org/project/testpath/0.4.4/)
+- [threadpoolctl: 2.1.0](https://pypi.org/project/threadpoolctl/2.1.0/)
+- [tinycss2: 1.2.1](https://pypi.org/project/tinycss2/1.2.1/)
+- [toml: 0.10.2](https://pypi.org/project/toml/0.10.2/)
+- [tornado: 6.0.3](https://pypi.org/project/tornado/6.0.3/)
+- [tqdm: 4.42.1](https://pypi.org/project/tqdm/4.42.1/)
+- [traitlets: 4.3.3](https://pypi.org/project/traitlets/4.3.3/)
+- [upsetplot: 0.6.1](https://pypi.org/project/upsetplot/0.6.1/)
+- [urllib3: 1.25.8](https://pypi.org/project/urllib3/1.25.8/)
+- [venn: 0.1.3](https://pypi.org/project/venn/0.1.3/)
+- [wcwidth: 0.1.8](https://pypi.org/project/wcwidth/0.1.8/)
+- [webencodings: 0.5.1](https://pypi.org/project/webencodings/0.5.1/)
+- [wheel: 0.36.2](https://pypi.org/project/wheel/0.36.2/)
+- [widgetsnbextension: 3.5.1](https://pypi.org/project/widgetsnbextension/3.5.1/)
+- [wrapt: 1.13.3](https://pypi.org/project/wrapt/1.13.3/)
+