Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

github cron job to update release branch snap_installation.yaml #83

Merged
merged 15 commits into from
May 3, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion .github/workflows/promote-charms.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,12 @@ jobs:
BRANCH=${BRANCH#refs/heads/} # strip off refs/heads/ if it exists
if [[ "${BRANCH}" == "main" ]]; then
echo "track=latest" >> "$GITHUB_OUTPUT"
elif [[ "${BRANCH}" =~ "^release-[0-9]+\.[0-9]+$" ]]; then
elif [[ "${BRANCH}" =~ ^release-[0-9]+\.[0-9]+$ ]]; then
echo "track=${BRANCH:8}" >> "$GITHUB_OUTPUT"
else
echo "::error Failed to determine track from branch ${BRANCH}"
exit 1
fi
echo "Promote from $track/${{github.event.inputs.origin-risk}} to $track/${{github.event.inputs.destination-risk}}"
select-charms:
runs-on: ubuntu-latest
Expand Down
5 changes: 4 additions & 1 deletion .github/workflows/publish-charms.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,12 @@ jobs:
if [[ "${BRANCH}" == "main" ]]; then
echo "track=latest" >> "$GITHUB_OUTPUT"
echo "risk=edge" >> "$GITHUB_OUTPUT"
elif [[ "${BRANCH}" =~ "^release-[0-9]+\.[0-9]+$" ]]; then
elif [[ "${BRANCH}" =~ ^release-[0-9]+\.[0-9]+$ ]]; then
echo "track=${BRANCH:8}" >> "$GITHUB_OUTPUT"
echo "risk=beta" >> "$GITHUB_OUTPUT"
else
echo "::error Failed to determine track/risk from branch ${BRANCH}"
exit 1
fi
publish-to-edge:
needs: [configure-channel]
Expand Down
79 changes: 79 additions & 0 deletions .github/workflows/update-snap-revision.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
#!/usr/bin/env python3
# Copyright 2024 Canonical Ltd.
# See LICENSE file for licensing details.


import os
import json
import logging
import sys
from pathlib import Path
from urllib.request import Request, urlopen
import yaml

logging.basicConfig(level=logging.INFO)
log = logging.getLogger("update-snap-revision")

ROOT = Path(__file__).parent / ".." / ".."
INSTALLATION = ROOT / "charms/worker/k8s/templates/snap_installation.yaml"
LICENSE = Path(__file__).read_text().splitlines(keepends=True)[1:4]

def find_current_revision(arch: str) -> None | str:
content = yaml.safe_load(INSTALLATION.read_text())
if arch_spec := content.get(arch):
for value in arch_spec:
if value.get("name") == "k8s":
rev = value.get("revision")
log.info("Currently arch=%s revision=%s", arch, rev)
return rev


def find_snapstore_revision(track: str, arch: str, risk: str) -> str:
URL = f"https://api.snapcraft.io/v2/snaps/info/k8s?architecture={arch}&fields=revision"
HEADER = {"Snap-Device-Series": 16}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The client must send the Snap-Device-Series and User-Agent mandatory headers. For the Snap-Device-Series header, the only supported value so far is '16'.

Heh, funny magic numbers. Do you have any other information on why this field is required?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://api.snapcraft.io/docs/info.html

The client must send the Snap-Device-Series and User-Agent mandatory headers. For the Snap-Device-Series header, the only supported value so far is '16'.

I'm guessing it's some kind of API versioning number? I have no clue

req = Request(URL, headers=HEADER)
with urlopen(req) as response:
snap_resp = json.loads(response.read())

for mapping in snap_resp["channel-map"]:
if (channel := mapping.get("channel")) and (
channel.get("architecture") == arch
and (channel.get("risk") == risk if risk else True)
and track in channel.get("track")
):
rev = mapping.get("revision")
log.info("SnapStore arch=%s revision=%s track=%s%s", arch, rev, track, f" risk={risk}" if risk else "")
return rev
log.warning("SnapStore arch=%s revision=%s track=%s%s", arch, "N/A", track, f" risk={risk}" if risk else "")


def update_current_revision(arch: str, rev: str):
content = yaml.safe_load(INSTALLATION.read_text())
if arch_spec := content.get(arch):
for value in arch_spec:
if value.get("name") == "k8s":
value["revision"] = rev
log.info("Updating arch=%s revision=%s", arch, rev)
with INSTALLATION.open("w") as f:
f.writelines(LICENSE)
f.write(yaml.safe_dump(content))


def update_github_env(variable:str, value:str):
if github_output := os.environ.get("GITHUB_OUTPUT", None):
with Path(github_output).open(mode="a+") as f:
f.write(f"{variable}={value}")

if __name__ == "__main__":
arch, track, risk = sys.argv[1:]
current_rev = find_current_revision(arch)
snapstore_rev = find_snapstore_revision(track, arch, risk)
if (
snapstore_rev and
current_rev and
current_rev != snapstore_rev
):
update_current_revision(arch, snapstore_rev)
update_github_env("result", snapstore_rev)
else:
log.info("No change arch=%s current=%s snapstore=%s", arch, current_rev, snapstore_rev)
72 changes: 72 additions & 0 deletions .github/workflows/update-snap-revision.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
name: Update Snap Revisions

on:
schedule:
- cron: "0 */5 * * *" # every 5 hours


jobs:
stable-branches:
runs-on: ubuntu-latest
outputs:
branches: ${{ steps.release-branches.outputs.data }}
steps:
- uses: octokit/[email protected]
id: list-branches
with:
route: GET /repos/${{ github.repository }}/branches
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- id: release-branches
run: |-
DATA='${{ steps.list-branches.outputs.data }}'
NAMES=$(jq -r '.[] | .name' <<< $DATA)
RELEASES=()
for BRANCH in ${NAMES}; do
if [[ "${BRANCH}" =~ ^release-[0-9]+\.[0-9]+$ ]]; then
RELEASES+=($BRANCH)
fi
done
echo data=$(printf '%s\n' "${RELEASES[@]}" | jq -R . | jq -s .) >> ${GITHUB_OUTPUT}

update-branches:
runs-on: ubuntu-latest
needs: [stable-branches]
strategy:
matrix:
branch: ${{ fromJSON(needs.stable-branches.outputs.branches) }}
arch: ["amd64", "arm64"]
steps:
- name: Prepare Track
run: |-
BRANCH="${{matrix.branch}}"
echo "TRACK=${BRANCH:8}" | tee -a "$GITHUB_ENV"

- name: Checkout ${{ matrix.branch }}
uses: actions/checkout@v4
with:
repository: ${{ github.repository }}
ref: ${{ matrix.branch }}

- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.12'

- name: Update Revision
id: update-revision
run: |
python3 .github/workflows/update-snap-revision.py ${{ matrix.arch }} ${{ env.TRACK }} stable
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just one question. Where is this TRACK envvar set?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wow, nice call. Updated and spotted another bug. Thanks @mateoflorido

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

track env is now from here


- name: Create pull request
uses: peter-evans/create-pull-request@v6
if: ${{ steps.update-revision.outputs.result != '' }}
with:
commit-message: Update K8s ${{ env.TRACK }}/stable revision to ${{ steps.update-revision.outputs.result }} on ${{ matrix.arch }}
title: "Update K8s ${{ env.TRACK }}/stable revision on ${{ matrix.arch }}"
body: |-
Updates K8s version for ${{ env.TRACK }}/stable
* revision=${{ steps.update-revision.outputs.result }}
* arch = ${{ matrix.arch }}
branch: revision-update-job/${{ env.TRACK }}/${{matrix.arch}}/${{ steps.update-revision.outputs.result }}
base: ${{ matrix.branch }}
Loading