From 159ddda2498b2cf618bc2dd1f99bf77a82bd68da Mon Sep 17 00:00:00 2001 From: alee-ntap <142503442+alee-ntap@users.noreply.github.com> Date: Thu, 16 Nov 2023 11:13:18 +0100 Subject: [PATCH 1/7] Create README.md --- .github/workflows/README.md | 90 +++++++++++++++++++++++++++++++++++++ 1 file changed, 90 insertions(+) create mode 100644 .github/workflows/README.md diff --git a/.github/workflows/README.md b/.github/workflows/README.md new file mode 100644 index 0000000..e1670d6 --- /dev/null +++ b/.github/workflows/README.md @@ -0,0 +1,90 @@ +# Gardenlinux build-package workflows + +This repository contains the Gardenlinux workflows for building and deploying containers and provide a resuable workflow for other packages to build and depolying. + +## Workflows + +The following workflows are defined in this repository: + +- `build_container.yml`: This workflow builds container images based on the provided .containererfile and pushes it to a container registry. +- `build_pkg.yml`: This is reusable workflow builds a package based on the provided input configurations and pushes it to a package repository. + +## Workflow Details + +### Build Container + +The `build_container.yml` workflow is responsible for building container images and pushing it to a container registry. + +#### Workflow Configuration + +The `container/` folder in this project contains all it's configurations. The `build_container.yml` workflow is triggered on every push to the repository. + +#### Job Details + +The `build` job within the `build_container.yml` workflow performs the following steps: +- Checks out the repository. +- Sets up the binfmt using a privileged Podman container. +- Builds the container images based on the provided containerfiles, considering the `host` and `target` matrix values that builds amd64:amd64, arm64v8:amd64, amd64:arm64v8 and arm64v8:arm64v8 images. +- Publishes the built container images to the container registry. + +### Build Package + +The `build_pkg.yml` workflow is a reusable workflow that can be called by other packages. It builds a package based on the provided input configurations and pushes it to a package repository. + +#### Workflow Configuration + +The `build_pkg.yml` workflow can be triggered by calling it with specific input parameters. + +Input Parameters available in `build_pkg.yml`: +- `repository` (string, default: ${{ github.repository }}): The repository to build the package from. +- `ref` (string, default: ${{ github.sha }}): The ref (commit or branch) to build the package from. +- `build_container` (string, default: ghcr.io/gardenlinux/package-build): The container image used for building the package. +- `dependencies` (string, default: ""): Comma-separated list of repositories and tags to fetch dependencies from. +- `debemail` (string, default: "contact@gardenlinux.io"): The Debian package maintainer email. +- `debfullname` (string, default: "Garden Linux Builder"): The Debian package maintainer full name. +- `distribution` (string, default: "gardenlinux"): The target distribution for the package. +- `message` (string, default: "Rebuild for Garden Linux."): The changelog entry message for the package build. +- `build_options` (string): Additional build options for the package. +- `source_name` (string, default: "${{ github.workflow }}"): The name of the source package. +- `source_version` (string): The version of the source package. +- `version_suffix` (string, default: "gl0"): The version suffix for the package. + +#### Job Details + +The `build_pkg.yml` workflow consists of the following jobs: +- `source`: This job retrieves the source package, sets up the build environment, and builds the source package. +- `packages`: This job builds the binary packages for the specified architecture. +- `publish`: This job publishes the drafted release. +- `cleanup`: This job deletes the drafted release in case of failure. + +#### Usage + +To use the `build_pkg.yml` workflow defined in this repository, follow these steps: + +1. Create a new workflow file (e.g., `build.yaml`) in the `.github/workflows/` folder of your package-project. +2. In the new workflow file, include the following content, eg: htop: + +``` + name: project + on: push + jobs: + htop: + uses: gardenlinux/package-build/.github/workflows/build_pkg.yaml@branchname +``` +Replace `project` to other package name of your package-project. Replace `branchname` with the specific branch or tag of the build_pkg.yaml workflow you want to use. The default is `main` branch. + +3. Call the gardenlinux/package-build/.github/workflows/build_pkg.yml@branchname workflow with the specific input parameters under `with:` specific to your project. such as `debemail`, `debfullname`, `distribution`, `message` and others mentioned in the Workflow Configuration section. The default values will be use if wihout provide any input parameters. +``` + uses: gardenlinux/package-build/.github/workflows/build_pkg.yaml@branchname + with: + debemail: your@address + debfullname: "Your Name" +``` +4. Commit and push the new workflow file to your project's repository. +5. The `build_pkg.yml` workflow will be triggered based on the provided inputs. + +Note: Make sure to adjust and use the matches branch name on your package-project in the uses field to match the desired branchname of the build_pkg.yaml@branchname workflow. + +## Contributing + +Contributions to this repository are welcome. If you encounter any issues or have suggestions for improvements, please open an issue or submit a pull request. From 65507f5ddaec79f3cb1a478724bc13325735fa06 Mon Sep 17 00:00:00 2001 From: alee-ntap <142503442+alee-ntap@users.noreply.github.com> Date: Thu, 16 Nov 2023 11:17:19 +0100 Subject: [PATCH 2/7] Fix typo in examples. --- .github/workflows/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/README.md b/.github/workflows/README.md index e1670d6..60fd860 100644 --- a/.github/workflows/README.md +++ b/.github/workflows/README.md @@ -69,13 +69,13 @@ To use the `build_pkg.yml` workflow defined in this repository, follow these ste on: push jobs: htop: - uses: gardenlinux/package-build/.github/workflows/build_pkg.yaml@branchname + uses: gardenlinux/package-build/.github/workflows/build_pkg.yml@branchname ``` Replace `project` to other package name of your package-project. Replace `branchname` with the specific branch or tag of the build_pkg.yaml workflow you want to use. The default is `main` branch. 3. Call the gardenlinux/package-build/.github/workflows/build_pkg.yml@branchname workflow with the specific input parameters under `with:` specific to your project. such as `debemail`, `debfullname`, `distribution`, `message` and others mentioned in the Workflow Configuration section. The default values will be use if wihout provide any input parameters. ``` - uses: gardenlinux/package-build/.github/workflows/build_pkg.yaml@branchname + uses: gardenlinux/package-build/.github/workflows/build_pkg.yml@branchname with: debemail: your@address debfullname: "Your Name" From 245536e7225d4a35d649058f04c0c7c0d838a4ef Mon Sep 17 00:00:00 2001 From: Michael Sprengel Date: Thu, 30 Nov 2023 09:53:31 +0100 Subject: [PATCH 3/7] Move workflow documentation to dedicated docs/ dir --- .github/workflows/README.md => docs/workflow.md | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename .github/workflows/README.md => docs/workflow.md (100%) diff --git a/.github/workflows/README.md b/docs/workflow.md similarity index 100% rename from .github/workflows/README.md rename to docs/workflow.md From 8371672d593f38594f6c797a06ed2fdabd7b9619 Mon Sep 17 00:00:00 2001 From: Michael Sprengel Date: Thu, 30 Nov 2023 15:04:19 +0100 Subject: [PATCH 4/7] Updated workflow documentation --- docs/workflow.md | 33 ++++++++++++++++++--------------- 1 file changed, 18 insertions(+), 15 deletions(-) diff --git a/docs/workflow.md b/docs/workflow.md index 60fd860..80ccafb 100644 --- a/docs/workflow.md +++ b/docs/workflow.md @@ -36,18 +36,21 @@ The `build_pkg.yml` workflow is a reusable workflow that can be called by other The `build_pkg.yml` workflow can be triggered by calling it with specific input parameters. Input Parameters available in `build_pkg.yml`: -- `repository` (string, default: ${{ github.repository }}): The repository to build the package from. -- `ref` (string, default: ${{ github.sha }}): The ref (commit or branch) to build the package from. -- `build_container` (string, default: ghcr.io/gardenlinux/package-build): The container image used for building the package. -- `dependencies` (string, default: ""): Comma-separated list of repositories and tags to fetch dependencies from. -- `debemail` (string, default: "contact@gardenlinux.io"): The Debian package maintainer email. -- `debfullname` (string, default: "Garden Linux Builder"): The Debian package maintainer full name. -- `distribution` (string, default: "gardenlinux"): The target distribution for the package. -- `message` (string, default: "Rebuild for Garden Linux."): The changelog entry message for the package build. -- `build_options` (string): Additional build options for the package. -- `source_name` (string, default: "${{ github.workflow }}"): The name of the source package. -- `source_version` (string): The version of the source package. -- `version_suffix` (string, default: "gl0"): The version suffix for the package. +| Parameter | Type | Description | +| --------- | ---- | ------------| +|`repository`|**Type:** string
**Default:** `${{ github.repository }}`| The repository to build the package from.| +|`ref`|**Type:** string
**Default:** `${{ github.sha }}`| The ref (commit or branch) to build the package from.| +|`build_container`|**Type:** string
**Default:** `ghcr.io/gardenlinux/package-build`| The container image used for building the package.| +|`dependencies`|**Type:** string| Comma-separated list of repositories and tags to fetch dependencies from.| +|`source`|**Type:** string| The source name of the package. There are three values that one can choose from:
- Debian Source Package: `{SOURCE PACKAGE NAME}`
- Git Source: `git+{GIT URL}`
- Native Build: `native`| +|`debian_source`|**Type:** string| Defines from which source the `debian/` directory should be used. There are three values that one can choose from:
- Debian Source Package: `{SOURCE PACKAGE NAME}`
- Git Source: `git+{GIT URL}`
- Native Build: `native`| +|`email`|**Type:** string
**Default:** `contact@gardenlinux.io`| The Debian package maintainer email.| +|`maintainer`|**Type:** string
**Default:** `Garden Linux Builder`| The Debian package maintainer full name.| +|`distribution`|**Type:** string
**Default:** `gardenlinux`| The target distribution for the package.| +|`message`|**Type:** string
**Default:** `Rebuild for Garden Linux.`| The changelog entry message for the package build.| +|`build_option`|**Type:** string| Additional build options for the package build. Build option `terse` is always set.| +|`build_profiles`|**Type:** string| Additional build profiles for the package build.| + #### Job Details @@ -73,12 +76,12 @@ To use the `build_pkg.yml` workflow defined in this repository, follow these ste ``` Replace `project` to other package name of your package-project. Replace `branchname` with the specific branch or tag of the build_pkg.yaml workflow you want to use. The default is `main` branch. -3. Call the gardenlinux/package-build/.github/workflows/build_pkg.yml@branchname workflow with the specific input parameters under `with:` specific to your project. such as `debemail`, `debfullname`, `distribution`, `message` and others mentioned in the Workflow Configuration section. The default values will be use if wihout provide any input parameters. +3. Call the gardenlinux/package-build/.github/workflows/build_pkg.yml@branchname workflow with the specific input parameters under `with:` specific to your project. such as `email`, `maintainer`, `distribution`, `message` and others mentioned in the Workflow Configuration section. The default values will be use if wihout provide any input parameters. ``` uses: gardenlinux/package-build/.github/workflows/build_pkg.yml@branchname with: - debemail: your@address - debfullname: "Your Name" + email: your@address + maintainer: "Your Name" ``` 4. Commit and push the new workflow file to your project's repository. 5. The `build_pkg.yml` workflow will be triggered based on the provided inputs. From 0d9ea480119530e570faa75e7a6b3cf0a859c426 Mon Sep 17 00:00:00 2001 From: Michael Sprengel Date: Thu, 30 Nov 2023 15:10:38 +0100 Subject: [PATCH 5/7] Adjust documentation of local development and the versioning --- README.md | 55 +++++++++------------------------------------ docs/local_build.md | 47 ++++++++++++++++++++++++++++++++++++++ docs/versioning.md | 1 + docs/workflow.md | 2 +- 4 files changed, 60 insertions(+), 45 deletions(-) create mode 100644 docs/local_build.md create mode 100644 docs/versioning.md diff --git a/README.md b/README.md index e7f3573..fa6aa08 100644 --- a/README.md +++ b/README.md @@ -7,56 +7,23 @@ It contains: - `container/`: This folder contains container-related files that define images and configurations. - `*.containerfile`: These files define container images. - `apt/`: This folder contains apt configuration files. - - `bin/`: This folder contains executable scripts installs into the container. Note: Also available via bind mount in the workflow for enable the possibility of workflow development tests. + - `bin/`: This folder contains executable scripts installs into the container. + > Note: Also available via bind mount in the workflow for enable the possibility of workflow development tests. - `conf/`: This folder contains containers configurations on installed packages list and native container setup. - `misc/`: This folder contains miscellaneous files related to the project. - `scripts/`: This folder contains scripts for it's github actions workflow. -## Local Development - -By following these steps, you will be able to build the container image for your desired architecture and generate packages using the containerized environment. This enables efficient local development, allowing you to iterate on your code, test packages, and validate changes effectively. - -### Build container - -To build the container, follow the steps below: - -Install `podman` on your system: https://podman.io/docs/installation - -Update unqualified search in `/etc/containers/registries.conf`: +## Versioning -``` -unqualified-search-registries = ["docker.io"] -``` +In the following documentation, you can find more information about how the versioning of Garden Linux packages works: +* [docs/versioning.md](docs/versioning.md) -On amd64 architecture: -``` -podman build --build-arg arch=amd64 -f container/build.containerfile -t build:amd64 . -``` - -On arm64 architecture: -``` -podman build --build-arg arch=arm64v8 -f container/build.containerfile -t build:arm64v8 . -``` - -Note: You do not need to use `sudo` as the build should work just fine with unprivileged containers. - -### Build a package - -To build a package, perform the following steps inside the package repository: - -Create a .build directory: -``` -mkdir .build -``` +## Local Development -On amd64 architecture: -``` -podman run --rm -v $PWD:/input -v $PWD/.build:/output build:amd64 build -``` +In the following documentation, you can find more information about how to build Garden Linux packages locally on your machine: +* [docs/local_build.md](docs/local_build.md) -On arm64 architecture: -``` -podman run --rm -v $PWD:/input -v $PWD/.build:/output build:arm64v8 build -``` +## Workflow -Alternatively, you can use pre-built containers from the GitHub Container Registry (ghcr) by accessing the following link: https://github.com/gardenlinux/package-build/pkgs/container/package-build. These pre-built containers can be used as an alternative to building the container locally. +In the following documentation, you can find more information about the Github Action and workflows used to build Garden Linux packages. +* [docs/workflow.md](docs/workflow.md) diff --git a/docs/local_build.md b/docs/local_build.md new file mode 100644 index 0000000..b14081e --- /dev/null +++ b/docs/local_build.md @@ -0,0 +1,47 @@ +# Local Development +By following these steps, you will be able to build the container image for your desired architecture and generate packages using the containerized environment. This enables efficient local development, allowing you to iterate on your code, test packages, and validate changes effectively. + +## Build container + +To build the container, follow the steps below: + +Install `podman` on your system: https://podman.io/docs/installation + +Update unqualified search in `/etc/containers/registries.conf`: + +``` +unqualified-search-registries = ["docker.io"] +``` + +On amd64 architecture: +``` +podman build --build-arg arch=amd64 -f container/build.containerfile -t build:amd64 . +``` + +On arm64 architecture: +``` +podman build --build-arg arch=arm64v8 -f container/build.containerfile -t build:arm64v8 . +``` + +Note: You do not need to use `sudo` as the build should work just fine with unprivileged containers. + +## Build a package + +To build a package, perform the following steps inside the package repository: + +Create a .build directory: +``` +mkdir .build +``` + +On amd64 architecture: +``` +podman run --rm -v $PWD:/input -v $PWD/.build:/output build:amd64 build +``` + +On arm64 architecture: +``` +podman run --rm -v $PWD:/input -v $PWD/.build:/output build:arm64v8 build +``` + +Alternatively, you can use pre-built containers from the GitHub Container Registry (ghcr) by accessing the following link: https://github.com/gardenlinux/package-build/pkgs/container/package-build. These pre-built containers can be used as an alternative to building the container locally. \ No newline at end of file diff --git a/docs/versioning.md b/docs/versioning.md new file mode 100644 index 0000000..1ca61ba --- /dev/null +++ b/docs/versioning.md @@ -0,0 +1 @@ +# Versioning diff --git a/docs/workflow.md b/docs/workflow.md index 80ccafb..ed36386 100644 --- a/docs/workflow.md +++ b/docs/workflow.md @@ -86,7 +86,7 @@ Replace `project` to other package name of your package-project. Replace `branch 4. Commit and push the new workflow file to your project's repository. 5. The `build_pkg.yml` workflow will be triggered based on the provided inputs. -Note: Make sure to adjust and use the matches branch name on your package-project in the uses field to match the desired branchname of the build_pkg.yaml@branchname workflow. +> Note: Make sure to adjust and use the matches branch name on your package-project in the uses field to match the desired branch name of the `build_pkg.yaml@branchname`` workflow. ## Contributing From 3618de3e65de043ca91cbf3f112eaf353a2c9c68 Mon Sep 17 00:00:00 2001 From: Michael Sprengel Date: Thu, 30 Nov 2023 15:19:58 +0100 Subject: [PATCH 6/7] Add table of content to workflow documentation --- docs/workflow.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/docs/workflow.md b/docs/workflow.md index ed36386..7427f21 100644 --- a/docs/workflow.md +++ b/docs/workflow.md @@ -2,6 +2,17 @@ This repository contains the Gardenlinux workflows for building and deploying containers and provide a resuable workflow for other packages to build and depolying. +- [Workflow](#workflows) +- [Workflow Detail](#workflow-details) + - [Build Container](#build-container) + - [Workflow Configuration](#workflow-configuration) + - [Job Details](#job-details) + - [Build Package](#build-package) + - [Workflow Configuration](#workflow-configuration-1) + - [Job Details](#job-details-1) + - [Usage](#usage) +- [Contributing](#contributing) + ## Workflows The following workflows are defined in this repository: From 496933ac13c63fba93642546c4d866036ed37c8e Mon Sep 17 00:00:00 2001 From: Michael Sprengel Date: Thu, 30 Nov 2023 15:52:32 +0100 Subject: [PATCH 7/7] Add version documentation --- docs/versioning.md | 37 +++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) diff --git a/docs/versioning.md b/docs/versioning.md index 1ca61ba..1d3f30d 100644 --- a/docs/versioning.md +++ b/docs/versioning.md @@ -1 +1,38 @@ # Versioning + +This chapter describes how the versioning of Garden Linux packages work that have not been mirrored by other sources. Thereby, there are three types of version schemas used: + +- [Commit Hash Version](#commit-hash-version) +- [First Upstream Version](#first-upstream-version) +- [Git Tag Version](#git-tag-version) + +## Commit Hash Version +**Example:** `gardenlinux/3.2.2-2gardenlinux~3618de3e65de043ca91cbf3f112eaf353a2c9c68` +**Published:** `false` + +Whenever there is a build triggered in a given package repo that is not based on a provided Git Tag, the pipeline tries to evaluate if there was already a released build for the given upstream version of the corresponding package by checking the Git tags of the corresponding package repo. + +If there was already such a build, this versioning schema applies to the package build then. + +This way, a developer can test changes to the package repo like providing new patches for it. Once the resulting package fulfills the expectation, a developer could then tag its latest commit, so that the latest version is properly versioned and could be published to Github releases as described in chapter [Git Tag Version](#git-tag-version). + +Packages that are using this kind of versioning are only built but not published. In order to access those packages, the corresponding package build (Github Action) must be opened. It contains the source and binary package of the given built package in its artifact section. + +## First Upstream Version +**Example:** `gardenlinux/3.2.2-2gardenlinux0` +**Published:** `true` + +If the package pipeline is triggered without providing a Git Tag and it turns out that there was not already a released build for the given upstream version, then the pipeline will automatically add a Git Tag and publish the resulting package to the Github releases. Thereby, the pipeline checks the Git tags of the given package repo to determine if such a version has already been released or not. + +In order to provide a proper version schema, the pipeline will add the version suffix `gardenlinux0` to such packages. This way, a developer can see that the package was automatically built and updated by the packaging pipeline. + +This mechanism allows to automatically update Garden Linux packages whenever there is an update in the corresponding upstream version without requiring a developer to add a Git tag. This reduces the general maintenance efforts because a developer doesn't need to manually tag a package to get the latest upstream version into the Garden Linux releases. + +## Git Tag Version +**Example:** `gardenlinux/3.2.2-2gardenlinux1`, `gardenlinux/3.2.2-2gardenlinux2`, `gardenlinux/3.2.2-2gardenlinux3`, ... +**Published:** `true` + +If there is already a released built for the given upstream version in the Garden Linux package releases but a developer needs to provide updates to the package itself (e.g. via new patches), the developer can release such an update by tagging the last functional Git commit with the following schema: +* `{UPSTREAM VERSION}gardenlinux{1|2|3|...}` + +As long as the developer does not tag such a package, the versioning from chapter [Commit Hash Version](#commit-hash-version) applies to the built package. Such packages are considered to be used for testing only whereas tagged versions are published to the Garden Linux package releases. \ No newline at end of file