--> tinkerbell/charts: introduce showcase chart

- `showcase` is a chart that, based on values.yaml dictionaries: - generates Tinkerbell CRs (Hardware/Template/Workflow) for both standard (UEFI) & exotic (supported by Armbian) devices - generates download/process jobs for multiple Hook flavors (see tinkerbell/hook#205) - generates download/process jobs for a few OS images (Ubuntu Cloud Images, Armbian, etc) - should be independent of how one deployed Tinkerbell itself (stack chart, individual components, etc) - A few features: - validates values.yaml for common mistakes; arch must match, etc. - validates & handles rootDisk differences (re-invents "formatPartition()" a bit) - avoids re-downloading Hooks and Images that are already on disk, even if Job re-runs - allows easy way to use - custom Hooks - custom Kernel cmdline parameters at both the Hook & device level - for example `acpi=off` at Hook level and `console=ttyS0` at board level - custom OS images for deployment - reboot or kexec to finish deployment - different partition numbers for OS image's rootfs (some images have ESP, some have a separate `/boot`, etc) - control if growpart and/or ssh/user setup is done during provisioning or not - conversion of OS images (`qemu-to-raw-gzip` and `xz-to-gz`) - has a "merge" mechanism with a common way to set parameters like net gateway, UEFI, etc (also easy to override per-device) - default values have everything `enabled: false` thus showcase should produce nothing by default. - Hooks & Images can be forced `enabled: true` in values.yaml, or - `enabled: true` Devices automatically enable their Hook & Image - Probably missing: - More validations - Currently pointing to my Tinkerbell Actions, which I haven't PR'ed yet - How to use: - Clone it, edit the values.yaml to your liking, and deploy. Signed-off-by: Ricardo Pardini <[email protected]> Squashed commits: - showcase: fix insecure-registries empty bug; add odroidm1; bump hooks - showcase: not all images are ubuntu - showcase: introduce insecureRegistries/tinkWorkerImage/tinkWorkerImagePrefix - tinkWorkerImage defaults to quay.io/tinkerbellrpardini/tink-worker:v0.11.0-alpha-rpardini3 - showcase: introduce grpcTLS/grpcInsecureTLS - showcase: generateJobs default to true, allow disabling Jobs - showcase: introduce image.doRestoreGRUBNormalcy for GRUB+EFI entry juggling and enable it for fatso images - showcase: introduce image.doAddEFIBootEntry for efibootmgr juggling and enable it for fatso images - showcase: WiP: nanopct6 (ipxe won't load, even with DTB) - showcase: use rpardini's fork of waitdaemon at quay.io - showcase: bump Hook to 20240604-0609 (further mdev by-id fixes) - showcase: bump Hook to 20240603-0515 (mdev by-id fixes); bump fatso images to v1008 - showcase: bump Hooks; bump images; add rpi4b armbian cloud image; add odroidhc4 example - showcase: fix default userdata script - showcase: bump all armsurvivors Armbian images to 24.05.20-armsurvivors-184; add t95z-cloud-edge image; use qcow2-to-img conversion where relevant - showcase: add support for `xz-qcow2-to-img-gz` image conversion - showcase: bump Hook to 20240520-0941 (finally working armbian's again!) - showcase: bump Hook to 20240520-0729 - showcase: bump Hook to 20240519-1935 (new scheme) - showcase: switch to 20240519-1405 Hook, with new (old?) naming convention - showcase: add demo t95z (meson64) - showcase: bump fatso - showcase: bump images - showcase: add ntp stuff - showcase: bump images, reorg - showcase: add image's 'doInjectHegelCloudInit'; set device's `userData` on the Hardware - add configuration hegelURL - some examples - showcase: bump Hook version; add `latest-lts-xxx` Hooks; add more examples - showcase: bump fatso images - showcase: allow to override hook's defaults via `hookOverride: {}` on the device - showcase: bump hook versions; fix example - showcase: bump hook versions; add examples for peg/pegk and skipDownload - showcase: hook: allow skipping hook download via `skipDownload` - showcase: devices: show `imgstat` before `boot` - showcase: bump fatso images version to 1001; add local variant examples - showcase: HACK: change default Hook amd64 bootMode to reboot (this needs override per-device) - showcase: example 'fatso' baremetal images with download-only and local example - showcase: bump orangepi3b image - showcase: introduce 'download-only' and 'local' non-conversions - showcase: reduce WAIT_SECONDS to zero - showcase: introduce doFixResolvConf at image level - showcase: install pkgs with DEBIAN_FRONTEND=noniteractive, no suggests, no recommends; 180s timeout - showcase: default netplan to device `e*` (not `en*`) so it matches `eth0` too - showcase: add `rock-5b-edge-trixie-uefi-dtb` image for Rock-5b - showcase: add armbian cloud-k8s images for uefi arm64 and amd64 - showcase: bump hook to rpardini's 20240404-2216; add `armbian-rk3588-edge` hook definition (but no board example yet) - showcase: VM examples - showcase: add R58X 3588 example & UEFI cloud image - showcase: validate hookRef not unset - showcase: 0.0.2: disable kexec on arm64, it doesn't really work (yet) - showcase: add README
rpardini · Jul 20, 2024 · 3265b16 · 3265b16
1 parent 95df5bc
commit 3265b16
Show file tree

Hide file tree

Showing 6 changed files with 1,270 additions and 0 deletions.
diff --git a/tinkerbell/showcase/Chart.yaml b/tinkerbell/showcase/Chart.yaml
@@ -0,0 +1,6 @@
+apiVersion: v2
+name: showcase
+description: Generates Tinkerbell CR's for a plethora of standard and exotic hardware; downloads & prepares Hook and OS images for provisioning
+type: application
+version: "0.0.2"
+
diff --git a/tinkerbell/showcase/README.md b/tinkerbell/showcase/README.md
@@ -0,0 +1,31 @@
+#### tinkerbell/charts: introduce `showcase` chart
+
+- `showcase` is a chart that, based on values.yaml dictionaries:
+    - generates Tinkerbell CRs (Hardware/Template/Workflow) for both standard (UEFI) & exotic (supported by Armbian) devices
+    - generates download/process jobs for multiple Hook flavors (see https://github.com/tinkerbell/hook/pull/205)
+    - generates download/process jobs for a few OS images (Ubuntu Cloud Images, Armbian, etc)
+    - should be independent of how one deployed Tinkerbell itself (stack chart, individual components, etc)
+- A few features:
+    - validates values.yaml for common mistakes; arch must match, etc.
+    - validates & handles rootDisk differences (re-invents "formatPartition()" a bit)
+    - avoids re-downloading Hooks and Images that are already on disk, even if Job re-runs
+    - allows easy way to use
+        - custom Hooks
+        - custom Kernel cmdline parameters at both the Hook & device level
+            - for example `acpi=off` at Hook level and `console=ttyS0` at board level
+        - custom OS images for deployment
+        - reboot or kexec to finish deployment
+        - different partition numbers for OS image's rootfs (some images have ESP, some have a separate `/boot`, etc)
+        - control if growpart and/or ssh/user setup is done during provisioning or not
+        - conversion of OS images (`qemu-to-raw-gzip` and `xz-to-gz`)
+    - has a "merge" mechanism with a common way to set parameters like net gateway, UEFI, etc (also easy to override per-device)
+    - default values have everything `enabled: false` thus showcase should produce nothing by default.
+        - Hooks & Images can be forced `enabled: true` in values.yaml, or
+            - `enabled: true` Devices automatically enable their Hook & Image
+- Probably missing:
+    - More validations
+    - Currently pointing to my Tinkerbell Actions, which I haven't PR'ed yet
+- How to use:
+    - Clone it, edit the values.yaml to your liking, and deploy.
+
+Signed-off-by: Ricardo Pardini <[email protected]>
diff --git a/tinkerbell/showcase/templates/devices.yaml b/tinkerbell/showcase/templates/devices.yaml
@@ -0,0 +1,344 @@
+{{- range $deviceId, $dev := .Values.hardware.devices }}
+
+{{- if not $dev.enabled }}
+  # Device not enabled: {{$deviceId}}
+{{- else }}
+  # Device enabled: {{$deviceId}}
+{{- $common := $.Values.hardware.common -}}
+{{- $mergedDevice := merge $dev $common }}
+  # Getting hook from hookRef: '{{$mergedDevice.hookRef}}'
+{{- if not $mergedDevice.hookRef }}{{- fail (printf "Device '%s', hookRef '%s': %s" $deviceId $mergedDevice.hookRef "hookRef unset?") }}{{- end }}
+
+{{- $hookObj := index $.Values.provision.hook $mergedDevice.hookRef }}
+{{- if not $hookObj }}{{- fail (printf "Device '%s', hookRef '%s': %s" $deviceId $mergedDevice.hookRef "hookRef not found") }}{{- end }}
+  # if the device has a hookOverride property, merge it on top of the hookObj; pre-merge: {{$hookObj.bootMode}}
+{{ if $mergedDevice.hookOverride }}
+  # YES!!!!!!!!!! has hookOverride {{$deviceId}}
+{{ $hookObj = merge $mergedDevice.hookOverride $hookObj }}
+{{ else }}
+  # NO!!!!!!!!!! no hookOverride {{$deviceId}}
+{{ end }}
+  # Reboot mode after merge: {{$hookObj.bootMode}}
+
+{{- $imageObj := index $.Values.provision.images $mergedDevice.imageRef }}
+{{- if not $imageObj }}{{- (printf "Device '%s', imageRef '%s': %s" $deviceId $mergedDevice.imageRef "imageRef not found") }}{{- end }}
+  # Check sanity of arch across device / image / hook - they all must match
+  # Device arch: {{$dev.arch}} Image arch: {{$imageObj.arch}} Hook arch: {{$hookObj.arch}}
+{{- if ne $imageObj.arch $hookObj.arch }}{{- fail (printf "Device '%s': '%s'" $deviceId "image and hook arch mismatch") }}{{- end }}
+{{- if ne $dev.arch $imageObj.arch }}{{- fail (printf "Device '%s': '%s'" $deviceId "device and image arch mismatch") }}{{- end }}
+{{- if ne $dev.arch $hookObj.arch }}{{- fail (printf "Device '%s': '%s'" $deviceId "device and hook arch mismatch") }}{{- end }}
+{{- $rootDiskDevice := required (printf "Device '%s' - %s" $deviceId "rootDisk is required") $mergedDevice.rootDisk }}
+{{- if not (hasPrefix "/dev" $rootDiskDevice) }}{{- fail (printf "Device '%s' (rootDisk '%s'): '%s'" $deviceId $rootDiskDevice "rootDisk does not begin with /dev") }}{{- end }}
+{{- $rootDiskRootfsPartitionNumber := printf "%s" $imageObj.rootfsPartitionNumber }}
+  # rootDiskRootfsPartitionNumber is {{$rootDiskRootfsPartitionNumber}}
+{{- $rootDiskRootfsPartitionDevice := "unknown" }}
+{{- $espPartitionDevice := "unknown" }}
+{{- $espPartitionNumber := "1" }}
+
+{{- if hasPrefix "/dev/disk/" $rootDiskDevice }}
+  # YES! {{$rootDiskDevice}} begins with /dev/disk
+{{- $rootDiskRootfsPartitionDevice = printf "%s-part%s" $rootDiskDevice $rootDiskRootfsPartitionNumber  }}
+{{- $espPartitionDevice = printf "%s-part%s" $rootDiskDevice $espPartitionNumber  }}
+{{- else }}
+  # NO! {{$rootDiskDevice}} does not begin with /dev/disk
+{{- if regexMatch "[0-9]$" $rootDiskDevice }}
+  # YES! {{$rootDiskDevice}} ends with a digit - REGEX MATCH
+{{- $rootDiskRootfsPartitionDevice = printf "%sp%s" $rootDiskDevice $rootDiskRootfsPartitionNumber  }}
+{{- $espPartitionDevice = printf "%sp%s" $rootDiskDevice $espPartitionNumber  }}
+{{- else }}
+  # NO! {{$rootDiskDevice}} does not end with a digit - REGEX NOT MATCH
+{{- $rootDiskRootfsPartitionDevice = printf "%s%s" $rootDiskDevice $rootDiskRootfsPartitionNumber  }}
+{{- $espPartitionDevice = printf "%s%s" $rootDiskDevice $espPartitionNumber  }}
+{{- end }}
+{{- end }}
+  # Thus at the end of the day, $rootDiskRootfsPartitionDevice is {{$rootDiskRootfsPartitionDevice}} and ESP is {{$espPartitionDevice}}
+---
+apiVersion: "tinkerbell.org/v1alpha1"
+kind: Hardware
+metadata:
+  name: "{{ $deviceId }}-hardware"
+  labels:
+    "app.kubernetes.io/instance": "{{ $.Release.Name }}"
+    "app.kubernetes.io/part-of": "tinkerbell-showcase"
+spec:
+  disks:
+    - device: "{{ $rootDiskDevice }}"
+  userData: |
+    {{- $mergedDevice.userData | default "#!/bin/bash\necho 'tinkerbell-showcase: userData not configured for device' >&2;" | nindent 4 }}
+  metadata:
+    facility:
+      facility_code: tinkerbell-showcase
+      plan_slug: "{{$deviceId}}"
+    instance:
+      hostname: "{{ $deviceId }}"
+      id: "{{ $dev.mac }}"
+      tags:
+        - "tinkerbell-showcase"
+      operating_system:
+        distro: "ubuntu" # @TODO
+        image_tag: "latest"
+        slug: "ubuntuslug"
+        os_slug: "ubuntu_24_04"
+        version: "24.04"
+  interfaces:
+    - dhcp:
+        arch: "{{ $dev.arch }}"
+        hostname: "{{ $deviceId }}"
+        ip:
+          address: "{{ $dev.ipv4.address }}"
+          netmask: "{{ $dev.ipv4.netmask }}"
+          gateway: "{{ $dev.ipv4.gateway }}"
+        lease_time: 86400
+        mac: "{{ $dev.mac }}"
+        name_servers:
+          {{- range $mergedDevice.ipv4.dns }}
+          - {{ . | quote }}
+          {{- end }}
+        uefi: {{ $dev.uefi }}
+        time_servers:
+          {{- range $mergedDevice.ipv4.time_servers }}
+          - {{ . | quote }}
+          {{- end }}
+      netboot:
+        allowPXE: true
+        allowWorkflow: true
+        ipxe: # @TODO
+          contents: |
+            echo Showcase starting for {{$deviceId}} with hook {{$mergedDevice.hookRef}} and image {{$mergedDevice.imageRef}}...
+            set download-url {{ $.Values.tinkerbell.hookURL }}
+            set kernel-params tink_worker_image={{ $.Values.tinkerbell.tinkWorkerImagePrefix }}{{ $.Values.tinkerbell.tinkWorkerImage }} facility= syslog_host={{ $.Values.tinkerbell.syslogHost }} grpc_authority={{ $.Values.tinkerbell.grpcAuthority }} tinkerbell_tls={{ $.Values.tinkerbell.grpcTLS }} tinkerbell_insecure_tls={{ $.Values.tinkerbell.grpcInsecureTLS }} worker_id={{$dev.mac}} hw_addr={{$dev.mac}} {{ if $.Values.tinkerbell.insecureRegistries.enabled }}insecure_registries={{ $.Values.tinkerbell.insecureRegistries.registries }} {{end}} modules=loop,squashfs,sd-mod,usb-storage initrd={{ $hookObj.initrd }} {{$hookObj.kernelCommandLine}} {{ $dev.extraKernelCommandLine }}
+            echo Kernel image: ${download-url}/{{ $hookObj.kernel }}
+            echo Kernel initrd: ${download-url}/{{ $hookObj.initrd }}
+            echo Kernel cmdline: ${kernel-params}
+            kernel ${download-url}/{{ $hookObj.kernel }} ${kernel-params}
+            initrd ${download-url}/{{ $hookObj.initrd }}
+            imgstat
+            boot
+---
+apiVersion: "tinkerbell.org/v1alpha1"
+kind: Template
+metadata:
+  name: "{{ $deviceId }}-template"
+  labels:
+    "app.kubernetes.io/instance": "{{ $.Release.Name }}"
+    "app.kubernetes.io/part-of": "tinkerbell-showcase"
+spec:
+  data: |
+    version: "0.1"
+    name: "{{ $deviceId }}-template"
+    global_timeout: 1800
+    tasks:
+      - name: "os-installation-{{ $deviceId }}"
+        worker: "{{ $dev.mac }}"
+        volumes:
+          - /dev:/dev
+          - /dev/console:/dev/console
+          - /lib/firmware:/lib/firmware:ro
+
+        actions:
+          - name: "stream-image-{{ $deviceId }}"
+            image: {{ $.Values.actions.repository }}/image2disk:{{ $.Values.actions.version }}
+            timeout: 600
+            environment:
+              DEST_DISK: "{{ $rootDiskDevice }}"
+              IMG_URL: "{{ $.Values.tinkerbell.imagesURL }}/{{ $imageObj.image }}"
+              COMPRESSED: true
+
+          {{- if $imageObj.doGrowPart }}
+          - name: "grow-partition-{{ $deviceId }}"
+            image: {{ $.Values.actions.repository }}/cexec:{{ $.Values.actions.version }}
+            timeout: 90
+            environment:
+              BLOCK_DEVICE: {{ $rootDiskRootfsPartitionDevice }}
+              FS_TYPE: ext4
+              CHROOT: y
+              DEFAULT_INTERPRETER: "/bin/sh -c"
+              CMD_LINE: "growpart {{ $rootDiskDevice }} {{$rootDiskRootfsPartitionNumber}} && resize2fs {{$rootDiskRootfsPartitionDevice}}"
+          {{- end }}
+
+          {{- if ($imageObj.doFixResolvConf)  }}
+          - name: "fix-resolv-{{ $deviceId }}"
+            image: {{ $.Values.actions.repository }}/cexec:{{ $.Values.actions.version }}
+            timeout: 90
+            environment:
+              BLOCK_DEVICE: {{$rootDiskRootfsPartitionDevice}}
+              FS_TYPE: ext4
+              CHROOT: y
+              DEFAULT_INTERPRETER: "/bin/sh -c"
+              CMD_LINE: "echo 'list /etc/resolv.conf: '; ls -la /etc/resolv.conf; echo 'cat /etc/resolv.conf'; cat /etc/resolv.conf; echo 'moving...'; mv -v /etc/resolv.conf /etc/resolv.conf.orig.tink; echo 'nameserver {{ index $mergedDevice.ipv4.dns 0 }} ' > /etc/resolv.conf; echo 'new resolf.conf:' ; cat /etc/resolv.conf"
+          {{- end }}
+
+
+          {{- if $imageObj.doUserAndSshSetup }}
+          - name: "install-packages-{{ $deviceId }}"
+            image: {{ $.Values.actions.repository }}/cexec:{{ $.Values.actions.version }}
+            timeout: 180 # 3 minutes
+            environment:
+              BLOCK_DEVICE: {{$rootDiskRootfsPartitionDevice}}
+              FS_TYPE: ext4
+              CHROOT: y
+              DEFAULT_INTERPRETER: "/bin/sh -c"
+              CMD_LINE: "apt -y update && DEBIAN_FRONTEND=noninteractive apt -y install openssl neofetch --no-install-recommends --no-install-suggests"
+          - name: "create-user-{{ $deviceId }}"
+            image: {{ $.Values.actions.repository }}/cexec:{{ $.Values.actions.version }}
+            timeout: 90
+            environment:
+              BLOCK_DEVICE: {{$rootDiskRootfsPartitionDevice}}
+              FS_TYPE: ext4
+              CHROOT: y
+              DEFAULT_INTERPRETER: "/bin/sh -c"
+              CMD_LINE: "useradd -p $(openssl passwd -1 tink) -s /bin/bash -d /home/tink/ -m -G sudo tink"
+          - name: "enable-ssh-{{ $deviceId }}"
+            image: {{ $.Values.actions.repository }}/cexec:{{ $.Values.actions.version }}
+            timeout: 90
+            environment:
+              BLOCK_DEVICE: {{$rootDiskRootfsPartitionDevice}}
+              FS_TYPE: ext4
+              CHROOT: y
+              DEFAULT_INTERPRETER: "/bin/sh -c"
+              CMD_LINE: "ssh-keygen -A; systemctl enable ssh.service; echo 'PasswordAuthentication yes' > /etc/ssh/sshd_config.d/60-cloudimg-settings.conf"
+          - name: "disable-apparmor-{{ $deviceId }}"
+            image: {{ $.Values.actions.repository }}/cexec:{{ $.Values.actions.version }}
+            timeout: 90
+            environment:
+              BLOCK_DEVICE: {{$rootDiskRootfsPartitionDevice}}
+              FS_TYPE: ext4
+              CHROOT: y
+              DEFAULT_INTERPRETER: "/bin/sh -c"
+              CMD_LINE: "systemctl disable apparmor; systemctl disable snapd"
+          - name: "write-netplan-{{ $deviceId }}"
+            image: {{ $.Values.actions.repository }}/writefile:{{ $.Values.actions.version }}
+            timeout: 90
+            environment:
+              DEST_DISK: {{$rootDiskRootfsPartitionDevice}}
+              FS_TYPE: ext4
+              DEST_PATH: /etc/netplan/config.yaml
+              CONTENTS: |
+                network:
+                  version: 2
+                  renderer: networkd
+                  ethernets:
+                    id0:
+                      match:
+                        name: e*
+                      dhcp4: true
+              UID: 0
+              GID: 0
+              MODE: 0644
+              DIRMODE: 0755
+          {{- end }}
+
+          {{- if ($imageObj.doInjectHegelCloudInit )  }}
+          # Based on https://tinkerbell.org/docs/integrations/cloudinit/#setup-cloud-init-to-use-hegel
+          - name: "inject-cloud-init-hegel-cfg-{{ $deviceId }}"
+            image: {{ $.Values.actions.repository }}/writefile:{{ $.Values.actions.version }}
+            timeout: 90
+            environment:
+              DEST_PATH: /etc/cloud/cloud.cfg.d/10_tinkerbell.cfg
+              CONTENTS: |
+                datasource:
+                  Ec2:
+                    metadata_urls: ["{{ $.Values.tinkerbell.hegelURL }}"]
+                    strict_id: false
+                manage_etc_hosts: localhost
+                warnings:
+                  dsid_missing_source: off      
+              DEST_DISK: {{$rootDiskRootfsPartitionDevice}}
+              FS_TYPE: ext4
+              UID: 0
+              GID: 0
+              MODE: 0600
+              DIRMODE: 0700
+
+          - name: "inject-cloud-init-hegel-ds-{{ $deviceId }}"
+            image: {{ $.Values.actions.repository }}/writefile:{{ $.Values.actions.version }}
+            timeout: 90
+            environment:
+              DEST_PATH: /etc/cloud/ds-identify.cfg
+              CONTENTS: |
+                datasource: Ec2
+              DEST_DISK: {{$rootDiskRootfsPartitionDevice}}
+              FS_TYPE: ext4
+              UID: 0
+              GID: 0
+              MODE: 0600
+              DIRMODE: 0700
+          {{- end }}
+
+          {{- if ($imageObj.doFixResolvConf )  }}
+          - name: "revert-fix-resolv-{{ $deviceId }}"
+            image: {{ $.Values.actions.repository }}/cexec:{{ $.Values.actions.version }}
+            timeout: 90
+            environment:
+              BLOCK_DEVICE: {{$rootDiskRootfsPartitionDevice}}
+              FS_TYPE: ext4
+              CHROOT: y
+              DEFAULT_INTERPRETER: "/bin/sh -c"
+              CMD_LINE: "rm -v /etc/resolv.conf; mv -v /etc/resolv.conf.orig.tink /etc/resolv.conf"
+          {{- end }}
+
+          {{- if ($imageObj.doAddEFIBootEntry )  }}
+          - name: "add-efi-boot-entry-{{ $deviceId }}"
+            image: {{ $.Values.actions.repository }}/cexec:{{ $.Values.actions.version }}
+            timeout: 90
+            environment:
+              BLOCK_DEVICE: {{$rootDiskRootfsPartitionDevice}}
+              FS_TYPE: ext4
+              CHROOT: y
+              DEFAULT_INTERPRETER: "/bin/bash -c"
+              CMD_LINE: "set -x; mount; lsblk; blkid; mount -o remount,rw /sys; mount -t efivarfs none /sys/firmware/efi/efivars; mount; efibootmgr --verbose; efibootmgr --create --disk '{{ $rootDiskDevice }}' --label PROViSiONED; efibootmgr --verbose; umount /sys/firmware/efi/efivars; sync"
+          {{- end }}
+
+          {{- if ($imageObj.doRestoreGRUBNormalcy )  }}
+          - name: "restore-grub-normalcy-{{ $deviceId }}"
+            image: {{ $.Values.actions.repository }}/cexec:{{ $.Values.actions.version }}
+            timeout: 90
+            environment:
+              BLOCK_DEVICE: {{$rootDiskRootfsPartitionDevice}}
+              FS_TYPE: ext4
+              CHROOT: y
+              DEFAULT_INTERPRETER: "/bin/bash -c"
+              CMD_LINE: "set -x; mount; lsblk; blkid; mount -o remount,rw /sys; mount -t efivarfs none /sys/firmware/efi/efivars; mount '{{ $espPartitionDevice }}' /boot; mount; rm -rfv /boot/ubuntu /boot/loader /boot/EFI; mkdir -pv /boot/EFI; update-initramfs -k all -c; echo 'GRUB_CMDLINE_LINUX_DEFAULT=\"${GRUB_CMDLINE_LINUX_DEFAULT} {{$mergedDevice.extraKernelCommandLine}}\"' > /etc/default/grub.d/60-provisioned.cfg; cat /etc/default/grub.d/60-provisioned.cfg; mv -v /etc/os-release /etc/os-release.hold; echo 'NAME={{$mergedDevice.imageRef}}' > /etc/os-release; grub-install --efi-directory /boot '--bootloader-id={{$mergedDevice.imageRef}}'; update-grub; tree /boot; cat /boot/grub/grub.cfg; efibootmgr --verbose; umount /sys/firmware/efi/efivars; umount /boot; mv -v /etc/os-release.hold /etc/os-release; sync"
+          {{- end }}
+
+          {{- if eq $hookObj.bootMode "kexec" }}
+          - name: "kexec-{{ $deviceId }}"
+            image: {{ $.Values.actions.repository }}/waitdaemon:{{ $.Values.actions.version }}
+            timeout: 90
+            pid: host
+            environment:
+              BLOCK_DEVICE: {{$rootDiskRootfsPartitionDevice}}
+              FS_TYPE: ext4
+              IMAGE: {{ $.Values.actions.repository }}/kexec:{{ $.Values.actions.version }}
+              WAIT_SECONDS: 0
+            volumes:
+              - /var/run/docker.sock:/var/run/docker.sock
+          {{- end }}
+
+          {{- if eq $hookObj.bootMode "reboot" }}
+          - name: "reboot-{{ $deviceId }}"
+            image: {{ $.Values.actions.repository }}/waitdaemon:{{ $.Values.actions.version }}
+            timeout: 90
+            pid: host
+            command: ["reboot"]
+            environment:
+              IMAGE: alpine
+              WAIT_SECONDS: 0
+            volumes:
+              - /var/run/docker.sock:/var/run/docker.sock
+          {{- end }}
+---
+apiVersion: "tinkerbell.org/v1alpha1"
+kind: Workflow
+metadata:
+  name: "{{ $deviceId }}-workflow"
+  labels:
+    "app.kubernetes.io/instance": "{{ $.Release.Name }}"
+    "app.kubernetes.io/part-of": "tinkerbell-showcase"
+spec:
+  templateRef: "{{ $deviceId }}-template"
+  hardwareRef: "{{ $deviceId }}-hardware"
+  hardwareMap:
+    device_1: "{{ $dev.mac }}"
+{{- end }}
+{{- end }}