Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: Add initial CI #22

Merged
merged 68 commits into from
Jul 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
68 commits
Select commit Hold shift + click to select a range
7923fbf
ci: Add initial CI
gabyx Jun 25, 2024
cad61c2
chore: Move workflow file
gabyx Jun 25, 2024
981015e
ci: Update Github checkout step
gabyx Jun 25, 2024
f71221a
ci: Fix shitty github actions...
gabyx Jun 25, 2024
1f318f1
ci: Fix Github workflows
gabyx Jun 25, 2024
3707104
fix: WIP
gabyx Jun 26, 2024
0948d07
feat: Improve Github workflow
gabyx Jun 27, 2024
18c5201
feat: WIP pacakge with Nix
gabyx Jun 27, 2024
9e31201
feat: Add Nix packaging and more CI
gabyx Jun 28, 2024
416e4f3
fix: CI
gabyx Jun 28, 2024
77e9a8d
fix: CI
gabyx Jun 28, 2024
7a8534b
fix: ci
gabyx Jun 28, 2024
fd73922
fix: Ci
gabyx Jun 28, 2024
3727efb
fix: ci
gabyx Jun 28, 2024
d8a3e00
fix: ci
gabyx Jun 28, 2024
abbf88d
fix: ci
gabyx Jun 28, 2024
4a3ff69
fix: ci
gabyx Jun 28, 2024
8b6dec1
fix: ci
gabyx Jun 28, 2024
7a2cfeb
fix: ci
gabyx Jun 28, 2024
84496f7
fix: lint
gabyx Jun 28, 2024
21a3058
fix: CI
gabyx Jun 28, 2024
3e93415
fix: ci
gabyx Jun 28, 2024
b21ea25
fix: Build image too
gabyx Jun 28, 2024
168083e
fix: correct direnv sourcing
gabyx Jun 29, 2024
d9a40a7
fix: improve CI with release flow
gabyx Jun 29, 2024
3179fa7
fix: cleanup all unused shit
gabyx Jul 3, 2024
f5ca334
fix: review changes
gabyx Jul 4, 2024
a01ffcd
fix: remove all unnecessary images
gabyx Jul 5, 2024
4884f63
fix: CI
gabyx Jul 5, 2024
7dbf8b4
fix: Correct release script
gabyx Jul 5, 2024
448320f
fix: no concurrency for prepare tagging
gabyx Jul 5, 2024
6d5e553
fix: permissions and readme
gabyx Jul 5, 2024
a26f21c
fix: setting Git user/email address
gabyx Jul 5, 2024
0955303
fix: remove before script
gabyx Jul 5, 2024
6edb881
fix: git troubles
gabyx Jul 5, 2024
3e50ef3
fix: pre-cache nix store
gabyx Jul 5, 2024
4bacf87
fix: path/string issues in Nix
gabyx Jul 5, 2024
d49246f
fix: CI
gabyx Jul 5, 2024
97a8a86
fix: use relative paths (weird Nix 2.23??)
gabyx Jul 5, 2024
db3354e
fix: try to reproduce Nix weird behaviour
gabyx Jul 5, 2024
2e86370
fix: reproduce what is wrong with `just nix-package`
gabyx Jul 5, 2024
a86e8fe
fix: try to build image too...
gabyx Jul 5, 2024
5372833
fix: TODO
gabyx Jul 5, 2024
e2a92c9
fix: correct weird bug in `nix build` when in `devShell`
gabyx Jul 6, 2024
5446818
fix: build image
gabyx Jul 6, 2024
8235344
fix: skopeo cannot deal with `:` in image names
gabyx Jul 6, 2024
9410494
fix: CI and deployment
gabyx Jul 6, 2024
652eae9
fix: CI
gabyx Jul 6, 2024
cc8ee40
fix: review comments
gabyx Jul 9, 2024
41eff3a
fix: WIP
gabyx Jul 9, 2024
6b61bf2
fix: add cleanup at end
gabyx Jul 9, 2024
ca10b38
fix: script typo
gabyx Jul 9, 2024
25dac36
fix: CI
gabyx Jul 9, 2024
4c8315d
fix: CI
gabyx Jul 9, 2024
2ff14aa
fix: CI
gabyx Jul 9, 2024
f5a03fd
fix: typo
gabyx Jul 9, 2024
bff0153
fix: typo
gabyx Jul 9, 2024
c8c8175
fix: add opencontainers label on image build
gabyx Jul 9, 2024
21d4632
fix: remove obsolete lint ub
gabyx Jul 9, 2024
30df3a0
fix: stupid token error
gabyx Jul 9, 2024
4a1d29e
fix: add Github release step
gabyx Jul 9, 2024
87e281b
fix: typo
gabyx Jul 9, 2024
455702b
fix: dependencies
gabyx Jul 9, 2024
791e37a
fix: checkout
gabyx Jul 9, 2024
4bc9bab
fix: make deploy depending on `test` only
gabyx Jul 9, 2024
ae11d4c
fix: typo in release name
gabyx Jul 9, 2024
03e8734
fix: typo
gabyx Jul 9, 2024
29cff2b
fix: typo in release name
gabyx Jul 10, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .envrc
Original file line number Diff line number Diff line change
@@ -1 +1 @@
use flake ./nix#default
use flake ./tools/nix#default
24 changes: 24 additions & 0 deletions .github/workflows/main-and-pr.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
name: Main and PR Pipeline

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cmdoret marked this conversation as resolved.
Show resolved Hide resolved
cancel-in-progress: true

defaults:
run:
shell: bash

on:
push:
branches:
- main

pull_request:
branches:
- main

jobs:
normal:
uses: ./.github/workflows/normal.yaml
with:
is_release: false
212 changes: 212 additions & 0 deletions .github/workflows/normal.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,212 @@
# yaml-language-server: $schema=https://json.schemastore.org/github-workflow.json
#
# This a demonstration pipeline which uses Nix in a container to drive the toolchain.
#
# - The Nix container (with cached toolchain) are quite big (~ 2Gb) and with no caching makes the
# pull up to 2min at the start.
# - Without caching they result in ~500mb which is better but the toolchain needs to be installed
# during the `nix develop` call.
#
# Remedies: Either to use own runners with proper image caching or do some Github trickery
# (not sure if they work), use the action/cache to cache docker layers and in the next step use
# a step with `uses: docker://...` but then `run:` does not work, how stupid ...
# (need to write an own action in the repo, 💩)
#
name: Normal Pipeline

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

permissions: read-all

defaults:
run:
shell: bash

on:
workflow_call:
inputs:
is_release:
required: true
type: boolean

jobs:
format:
if: ${{ ! inputs.is_release }}
runs-on: ubuntu-latest
container:
image: ghcr.io/sdsc-ordes/rdf-protect:ci-nix-1.0.0
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 1

- name: Setup Git & Cache Nix
run: |
./tools/ci/setup-git.sh
just nix-develop-ci echo "Built cache."

- name: Format
run: |
just nix-develop-ci just format

lint:
if: ${{ ! inputs.is_release }}
runs-on: ubuntu-latest
container:
image: ghcr.io/sdsc-ordes/rdf-protect:ci-nix-1.0.0
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 1

- name: Setup Git & Cache Nix
run: |
./tools/ci/setup-git.sh
just nix-develop-ci echo "Built cache."

- name: Lint
run: |
just nix-develop-ci just lint

build:
if: ${{ ! inputs.is_release }}
runs-on: ubuntu-latest
container:
image: ghcr.io/sdsc-ordes/rdf-protect:ci-nix-1.0.0
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 1

- name: Setup Git & Cache Nix
run: |
./tools/ci/setup-git.sh
just nix-develop-ci echo "Built cache."

- name: Build
run: |
just nix-develop-ci just build

test:
runs-on: ubuntu-latest
container:
image: ghcr.io/sdsc-ordes/rdf-protect:ci-nix-1.0.0
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 1

- name: Setup Git & Cache Nix
run: |
./tools/ci/setup-git.sh
just nix-develop-ci git --version

- name: Test
run: |
just nix-develop-ci just test

package:
runs-on: ubuntu-latest
container:
image: ghcr.io/sdsc-ordes/rdf-protect:ci-nix-1.0.0

if: ${{ ! inputs.is_release }}

steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 1

- name: Setup Git
run: |
./tools/ci/setup-git.sh
just nix-develop-ci echo "Built cache."

- name: Build Package (nix)
run: |
just nix-develop-ci just nix-package

- name: Build Container Image (nix)
run: |
just nix-develop-ci just nix-image

deploy:
if: ${{ inputs.is_release }}
needs: [test]

runs-on: ubuntu-latest
container:
image: ghcr.io/sdsc-ordes/rdf-protect:ci-nix-1.0.0

permissions:
contents: write
packages: write

env:
CI_IS_RELEASE: true

steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 1

- name: Setup Git & Nix Cache
run: |
./tools/ci/setup-git.sh
just nix-develop-ci echo "Built cache."

- name: Create Version Tag
run: |
just nix-develop-ci ./tools/ci/assert-tag.sh \
create-and-check "$GITHUB_REF"

- name: Build Container Image (nix)
run: |
just nix-develop-ci just nix-image

- name: Push Image
env:
REGISTRY_USERNAME: ${{ github.actor }}
REGISTRY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}
run: |
just nix-develop-ci tools/ci/upload-image.sh

- name: Push Tag
run: |
just nix-develop-ci ./tools/ci/assert-tag.sh push "$GITHUB_REF"

- name: Cleanup
if: always()
run: |
just nix-develop-ci tools/ci/assert-tag.sh cleanup "$GITHUB_REF"

release:
if: ${{ inputs.is_release }}
needs: ["deploy"]

runs-on: ubuntu-latest

permissions:
contents: write

steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 1

- name: Create Github Release
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
./tools/ci/create-github-release.sh \
"${GITHUB_REF#refs/tags/}" \
"$GITHUB_REPOSITORY"
22 changes: 22 additions & 0 deletions .github/workflows/release.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
name: Main and PR Pipeline

concurrency:
group: ${{ github.ref }}
cancel-in-progress: true

defaults:
run:
shell: bash

on:
push:
tags:
# This is not a real version tag, its just used to trigger
# the release build. Glob pattern:
- "prepare-v[0-9]+.[0-9]+.[0-9]+*"

jobs:
release:
uses: ./.github/workflows/normal.yaml
with:
is_release: true
9 changes: 8 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,9 @@
/target
# Tooling
.direnv

# Rust
/target

# Nix
result
build
2 changes: 1 addition & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

39 changes: 20 additions & 19 deletions Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,22 +1,23 @@
[package]
name = "rdf-protect"
version = "0.1.0"
edition = "2021"
[dependencies]
bitflags = '2.5.0'
io-enum = '1.1.3'
rio_api = '0.8.4'
rio_turtle = '0.8.4'
serde_yml = '0.0.10'
slog = '2.7.0'
slog-async = '2.8.0'
slog-term = '2.9.0'
tempfile = '3.10.1'

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies.clap]
features = ['derive']
version = '4.5.7'

[dependencies]
# Good logging library.
slog = "2.7.0"
slog-term = "2.9.0"
slog-async = "2.8.0"
[dependencies.serde]
features = ['derive']
version = '1.0'

# Popular serialization library.
serde = { version = "1.0", features = ['derive']}
clap = { version = "4.5.7", features = ["derive"] }
rio_turtle = "0.8.4"
rio_api = "0.8.4"
bitflags = "2.5.0"
io-enum = "1.1.3"
serde_yml = "0.0.10"
tempfile = "3.10.1"
[package]
edition = '2021'
name = 'rdf-protect'
version = '0.0.1'
Loading