Merge pull request #2 from KTE/dex-os-videolooper

dexOS with videolooper

.gitmodules

@@ -10,3 +10,6 @@
 [submodule "packages/example-content"]
 	path = packages/example-content
 	url = https://github.com/KTE/dex-example-content
+[submodule "packages/pi_video_looper"]
+	path = packages/pi_video_looper
+	url = https://github.com/adafruit/pi_video_looper

README.md

@@ -6,6 +6,29 @@ packages:
 * [example content](./packages/example-content/README.md)
 * [branding](./packages/branding/README.md)
 
+## getting started
+
+Build a dex player by flashing the image to an SD card,
+using the official [Raspberry Pi Imager](https://www.raspberrypi.org/software/).
+
+Then boot the Raspberry Pi with the SD card.
+It it worked, it will show a 2-second demo video loop.
+
+### advanced
+
+For customizing the player/operating system,
+ssh access needs to be enabled.
+
+This can be done using the "customization" feature of the Raspberry Pi Imager,
+choosing "Enable SSH" in the "Advanced Options". It is recommended to use key-based authentication.
+The user name in the image is `dex` should not be changed, the default password is also `dex` and should be changed if SSH login is enabled and password authentication is used.
+
+Then, after booting the Raspberry Pi, ssh into it:
+
+```sh
+ssh dex@dexpi # or another hostname if you changed it in the customization
+```
+
 ## development
 
 ### creating patches
@@ -24,10 +47,19 @@ Good tutorials on using `quilt`:
 quilt new "99-name-of-my-patch"
 quilt add ./packages/some-upstream-code/some-file
 # edit ./packages/some-upstream-code/some-file
-quilt refresh # pathfile is added to ./patches and name added to ./patches/series 
+quilt refresh # patchfile is added to ./patches and patch name is added to ./patches/series 
 quilt rename "99-better-name-of-my-patch"
 ```
 
+editing existing patches:
+
+```sh
+PATCH_NAME="project/99-name-of-my-patch"
+quilt add -P "$PATCH_NAME" ./packages/some-upstream-code/some-file
+# edit ./packages/some-upstream-code/some-file
+quilt refresh "$PATCH_NAME" # patchfile is updated in ./patches
+```
+
 ## housekeeping
 
 ### update pi-gen repo

packages/dex-os/README.md

@@ -13,7 +13,7 @@ Install
 
 ## build
 
-Builds a custom Raspbian image, based on `2024-03-12-raspios-bullseye-arm64-lite.img`.
+Builds a custom Raspbian/Raspberry Pi OS image, based on Debian `buster`.
 
 1. set up customized pi-gen
 
@@ -26,6 +26,7 @@ Builds a custom Raspbian image, based on `2024-03-12-raspios-bullseye-arm64-lite
    ```sh
    time ./build.sh
    ```
+
 ## development
 
 for incremental build, first run:
@@ -40,6 +41,55 @@ and then
 time PRESERVE_CONTAINER=1 CONTINUE=1 ./build.sh
 ```
 
+debug the container:
+
+```sh
+./dev-debug-container.sh
+# gives root shell in the build container
+cd pi-gen/work/…
+# for example, delete our stage for faster rebuild
+rm -rf stage-dex
+```
+
+starting from scratch:
+
+```sh
+./dev-reset.sh
+```
+
+## notes
+
+### data partition
+
+The image has a third partition, formatted as FAT, for storing media and configuration files.
+Its mounted under `/dexdata` on the Raspberry Pi.
+It can be managed by mounting the SD card on a computer (macOS, Windows, Linux) and copying files to it.
+
+Creating and handling this partition differs from the default Raspberry Pi OS behavior:
+* Raspbian/Raspberry Pi OS uses a single partition `rootfs` partition for the operating system and user data.
+  * On first boot, the `rootfs` *partition* is expanded to fill the SD card.
+    * This is done by settings the `init` (in `/boot/cmdline.txt`) to `init=/usr/lib/raspi-config/init_resize.sh`
+    * After this script ran, it removes the `init` line from `/boot/cmdline.txt` so the system will boot regularly from then on.
+    * Script source: <https://github.com/RPi-Distro/raspi-config/blob/de70c08c7629b2370d683193a62587ca30051e36/usr/lib/raspi-config/init_resize.sh>
+  * On the second boot, the `rootfs` *filesystem* is resized to fill the now expanded partition.
+    * This is done by the `resize2fs_once` script in `/etc/init.d`
+    * This script will also remove itself after its done, so it only runs once.
+    * There is not reboot necessary after this step (it runs early enough in the boot process).
+  * Note: those steps are necessary because modifying the `rootfs` partition, where the operating system is running from, is not possible while it is mounted. 
+
+* dexOS handles it differently:
+  * The `rootfs` partition is not expanded on first boot (the special `init` is already removed `/boot/cmdline.txt`).
+  * The `resize2fs_once` script therefore runs on first boot, and is customized to handle the data partition only
+    * It resizes the `dexdata` *partition* to fill the whole SD card.
+    * It *recreates* the `dexdata` *filesystem* to fill the partition.
+      * This is necessary because the `resize2fs` command not resize FAT partitions, and `fatresize` did not work reliably.
+      * The data in this partition is kept by backing it up to the `rootfs`, which means it must be rather small initially so there is enough space (currently 10MB).
+  * NOTE: right now this is a compromise. After flashing the image, it could immediately be mounted on the same computer, so the media can be copied directly to the SD card.
+    However, 10MB is only enough for the very small example video. 
+      * Only after booting the SD card with an Raspberry Pi, the data partition will be expanded to the full size of the SD card.
+        * That means the data partition could also be ignored in the build step and only be created on first boot, but it already works.
+      * On the other hand, a script could be used to resize the partition on the same computer that was used to flash the images, but then its not as easy to use a just using the official Raspberry Pi Imager.
+
 ## alternatives
 
 * <https://github.com/guysoft/CustomPiOS>

packages/dex-os/build.sh

@@ -80,18 +80,14 @@ fi
 # add our config and stage to pi-gen
 cp ./pi-gen-config.env "${PI_GEN_DIR}/config"
 rm -rf "$PI_GEN_DIR/stage-dex"
-cp -r ./pi-gen/stage-dex "$PI_GEN_DIR"
+rsync --archive --cvs-exclude --copy-links ./pi-gen/stage-dex "$PI_GEN_DIR"
 
 echo "STAGE_LIST=\"$STAGE_LIST\"" >> "${PI_GEN_DIR}/config"
 echo "USE_QCOW2=\"$USE_QCOW2\"" >> "${PI_GEN_DIR}/config"
 
 cd "$PI_GEN_DIR"
 cat ./config
 
-# get git hash from monorepo
-GIT_HASH="$(git rev-parse HEAD)"
-export GIT_HASH
-
 ./build-docker.sh
 
 mv ./deploy/* "${DIST_DIR}/"
@@ -100,5 +96,7 @@ if [ "${CONTINUE:-0}" != "1" ]; then
   clear
 fi
 
-echo "🎉 output in ${DIST_DIR}:"
+echo; echo; echo "🎉 output in ${DIST_DIR}:"
 ls -lah "${DIST_DIR}"
+
+echo "[build] OK"

packages/dex-os/dev-debug-container.sh

@@ -0,0 +1,7 @@
+#!/bin/bash
+set -euo pipefail
+WD="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+cd "$WD"
+
+docker start pigen_work || true
+docker exec -it pigen_work /bin/bash

packages/dex-os/dev-reset.sh

@@ -0,0 +1,12 @@
+#!/bin/bash
+set -euo pipefail
+WD="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+cd "$WD"
+
+docker stop dex-os-apt-cacher-ng || true
+docker rm dex-os-apt-cacher-ng || true
+
+docker stop pigen_work || true
+docker rm pigen_work || true
+
+echo "[dev-reset] OK"

packages/dex-os/dist.sh

@@ -0,0 +1,10 @@
+#!/bin/bash
+set -euo pipefail
+WD="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+cd "$WD"
+
+./dev-reset.sh
+./prepare.sh
+./build.sh
+
+echo "[dist] OK"

packages/dex-os/pi-gen/stage-dex/00-data-partition/00-packages

@@ -0,0 +1,3 @@
+parted
+dosfstools
+mtools

packages/dex-os/pi-gen/stage-dex/00-data-partition/01-run.sh

@@ -0,0 +1,7 @@
+# TODO: convert those to proper patches? Makes the repo less readable, but catches unexpected changes from upstream
+
+# disable the default init_resize.sh script (we dont need to resize the root partition)
+sed -i 's| init=/usr/lib/raspi-config/init_resize.sh||' "${ROOTFS_DIR}/boot/cmdline.txt"
+
+# install our resize2fs_once script, which resizes the data partition on first boot
+install -m 755 files/resize2fs_once "${ROOTFS_DIR}/etc/init.d/"

packages/dex-os/pi-gen/stage-dex/00-data-partition/files/resize2fs_once

@@ -0,0 +1,51 @@
+#!/bin/sh
+### BEGIN INIT INFO
+# Provides:           resize2fs_once
+# Required-Start:
+# Required-Stop:
+# Default-Start:      3
+# Default-Stop:
+# Short-Description:  Resize /dexdata partition to fill the SD card
+# Description:
+### END INIT INFO
+. /lib/lsb/init-functions
+case "$1" in
+  start)
+    log_daemon_msg "Starting resize2fs_once"
+    set -exu
+
+    DATA_MOUNT="/dexdata/"
+
+    # find device and partition
+    mount "$DATA_MOUNT" || mount "$DATA_MOUNT" -o remount
+    DATA_PART="$(findmnt "$DATA_MOUNT" -n -o SOURCE -r)"
+    DATA_PART_DEV="$(echo "$DATA_PART" | sed 's/p.*//')"
+
+    # backup files
+    TMP_DIR="$(mktemp -d)${DATA_MOUNT}/"
+    rsync -avPh "$DATA_MOUNT" "$TMP_DIR"
+
+    # resize partition and recreate filesystem
+    umount "$DATA_MOUNT"
+    parted "$DATA_PART_DEV" --align optimal --script 'resizepart 3 100%'
+    mkfs.vfat -F 32 -n "DEXDATA" "$DATA_PART"
+    dosfsck -w -l -a -v -f -y "$DATA_PART"
+
+    # restore files
+    mount "$DATA_MOUNT"
+    rsync -avPh "$TMP_DIR" "$DATA_MOUNT"
+
+    # remove run-once script
+    update-rc.d resize2fs_once remove
+    rm /etc/init.d/resize2fs_once
+
+    log_daemon_msg "resize2fs_once finished"
+
+    log_end_msg $?
+
+    ;;
+  *)
+    echo "Usage: $0 start" >&2
+    exit 3
+    ;;
+esac

packages/dex-os/pi-gen/stage-dex/01-videolooper/01-run.sh

@@ -0,0 +1,20 @@
+# shellcheck shell=sh
+
+# install adafruit pi_video_looper using their install script, omitting omxplayer
+
+rm -rf "/home/${FIRST_USER_NAME}/pi_video_looper"
+cp -r files/pi_video_looper "${ROOTFS_DIR}/home/${FIRST_USER_NAME}/"
+
+on_chroot << EOF
+
+cd "/home/${FIRST_USER_NAME}/pi_video_looper"
+
+# remove 'omxplayer' from the list of packages to install (we only want hello_video)
+sed -i 's/omxplayer//g' ./install.sh
+
+./install.sh
+
+EOF
+
+# add our custom video_looper.ini configuration file
+install -m 644 files/video_looper.ini "${ROOTFS_DIR}/boot/video_looper.ini"

packages/dex-os/pi-gen/stage-dex/01-videolooper/files/pi_video_looper

@@ -0,0 +1 @@
+../../../../../pi_video_looper