Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bookworm] cannot unlock LUKS2 volume from initramfs #362

Open
bklop opened this issue Jan 24, 2024 · 23 comments
Open

[Bookworm] cannot unlock LUKS2 volume from initramfs #362

bklop opened this issue Jan 24, 2024 · 23 comments

Comments

@bklop
Copy link

bklop commented Jan 24, 2024

With RBPI OS Lite 64 based on Debian 11 (2023-05-03-raspios-bullseye-arm64-lite.img.xz) running on a RBPI CM4 I can decrypt a LUKS2 volume encrypted with Adiantum from initramfs. The procedure I follow for building initramfs:

sudo apt install busybox cryptsetup initramfs-tools

Check that the algorithms are available:
cryptsetup benchmark -c xchacha20,aes-adiantum-plain64

/etc/initramfs-tools/hooks/luks_hooks

#!/bin/sh -e
PREREQS=""
case $1 in
        prereqs) echo "${PREREQS}"; exit 0;;
esac

. /usr/share/initramfs-tools/hook-functions

copy_exec /sbin/resize2fs /sbin
copy_exec /sbin/fdisk /sbin
copy_exec /sbin/cryptsetup /sbin

sudo chmod +x /etc/initramfs-tools/hooks/luks_hooks

/etc/initramfs-tools/modules

algif_skcipher
xchacha20
adiantum
aes_arm
sha256
nhpoly1305
dm-crypt

Finally, build a new initramfs:
sudo -E CRYPTSETUP=y mkinitramfs -o /boot/initramfs.gz

I then update /boot/cmdline.txt etc. And during reboot I get bumped into a shell where I can unlock a volume:
cryptsetup luksOpen /dev/XYZ crypted

This works fine with Debian 11. When I followed the same procedure with Bookworm (2023-12-11-raspios-bookworm-arm64-lite.img.xz) everything works fine until I boot into initramfs and try to unlock the volume, where I got this error:

libgcc_s.so.1 must be installed for pthread_exit to work

I realized it was most likely this issue: https://bugs.launchpad.net/ubuntu/+source/cryptsetup/+bug/1960080

So I added libpthread and libgcc to my luks_hooks script:

#!/bin/sh -e
PREREQS=""
case $1 in
        prereqs) echo "${PREREQS}"; exit 0;;
esac

. /usr/share/initramfs-tools/hook-functions

copy_exec /usr/lib/aarch64-linux-gnu/libgcc_s.so.1 /usr/lib/aarch64-linux-gnu
copy_exec /usr/lib/aarch64-linux-gnu/libpthread.so.0 /usr/lib/aarch64-linux-gnu
copy_exec /sbin/resize2fs /sbin
copy_exec /sbin/fdisk /sbin
copy_exec /sbin/cryptsetup /sbin

After building a new initramfs I then checked the archive for the presence of the modules:

lsinitramfs /boot/initramfs.gz | grep -P "libgcc"
lsinitramfs /boot/initramfs.gz | grep -P "sbin/(cryptsetup|resize2fs|fdisk)"
lsinitramfs /boot/initramfs.gz | grep -P "(algif_skcipher|chacha|adiantum|aes-arm|sha256|nhpoly1305|dm-crypt)"

However, after booting into initramfs I still cannot use cryptfs to unlock the volume:
Cannot use xchacha20,aes-adiantum-plain64 cipher

@XECDesign
Copy link
Member

@tdewey-rpi, since you're looking at something similar, is this something you've stumbled upon?

@tdewey-rpi
Copy link

ACK.

I'm going to try and reproduce something along these lines now.

@tdewey-rpi
Copy link

tdewey-rpi commented Jan 29, 2024

Thanks for the report, @blkop.

I've been able to reproduce part of your failure and methodology. I can confirm I'm seeing files explicitly named in the hooks file be transferred to the resultant initramfs, but it appears that the documentation for mkinitramfs isn't aligning with observed behaviour on bookworm. Specifically, it says:

mkinitramfs will automatically detect which libraries the executable depends on and copy them to the initramfs

And sure enough, the adiantum module isn't being copied into the initramfs, while it's present on the installed system:

initramfs:
usr/lib/modules/6.1.0-rpi8-rpi-2712/kernel/arch/arm64/crypto/chacha-neon.ko usr/lib/modules/6.1.0-rpi8-rpi-2712/kernel/arch/arm64/crypto/poly1305-neon.ko usr/lib/modules/6.1.0-rpi8-rpi-2712/kernel/arch/arm64/lib/xor-neon.ko usr/lib/modules/6.1.0-rpi8-rpi-2712/kernel/crypto/blake2b_generic.ko usr/lib/modules/6.1.0-rpi8-rpi-2712/kernel/crypto/xor.ko usr/lib/modules/6.1.0-rpi8-rpi-2712/kernel/lib/crypto/libarc4.ko usr/lib/modules/6.1.0-rpi8-rpi-2712/kernel/lib/crypto/libchacha.ko usr/lib/modules/6.1.0-rpi8-rpi-2712/kernel/lib/crypto/libchacha20poly1305.ko usr/lib/modules/6.1.0-rpi8-rpi-2712/kernel/lib/crypto/libcurve25519-generic.ko

Installed system:
tdewey@tdewey-pi:/usr/lib/modules/6.1.0-rpi8-rpi-2712/kernel/arch/arm64/crypto $ ls aes-arm64.ko.xz aes-ce-ccm.ko.xz aes-neon-blk.ko.xz chacha-neon.ko.xz poly1305-neon.ko.xz sha256-arm64.ko.xz sha3-ce.ko.xz sha512-ce.ko.xz sm4-ce-cipher.ko.xz aes-ce-blk.ko.xz aes-ce-cipher.ko.xz aes-neon-bs.ko.xz ghash-ce.ko.xz sha1-ce.ko.xz sha2-ce.ko.xz sha512-arm64.ko.xz sm3-ce.ko.xz tdewey@tdewey-pi:/usr/lib/modules/6.1.0-rpi8-rpi-2712/kernel/arch/arm64/crypto $ ls /usr/lib/modules/6.1.0-rpi8-rpi-2712/kernel/crypto adiantum.ko.xz algif_hash.ko.xz authencesn.ko.xz cast_common.ko.xz chacha_generic.ko.xz ctr.ko.xz echainiv.ko.xz ghash-generic.ko.xz md4.ko.xz poly1305_generic.ko.xz sm4.ko.xz xcbc.ko.xz aes_generic.ko.xz algif_rng.ko.xz authenc.ko.xz cbc.ko.xz cmac.ko.xz drbg.ko.xz essiv.ko.xz hmac.ko.xz md5.ko.xz seqiv.ko.xz twofish_common.ko.xz xor.ko.xz af_alg.ko.xz algif_skcipher.ko.xz blake2b_generic.ko.xz ccm.ko.xz cryptd.ko.xz ecc.ko.xz gcm.ko.xz jitterentropy_rng.ko.xz michael_mic.ko.xz sha3_generic.ko.xz twofish_generic.ko.xz xxhash_generic.ko.xz algif_aead.ko.xz async_tx cast5_generic.ko.xz chacha20poly1305.ko.xz crypto_user.ko.xz ecdh_generic.ko.xz gf128mul.ko.xz lz4.ko.xz nhpoly1305.ko.xz sm3.ko.xz wp512.ko.xz zstd.ko.xz tdewey@tdewey-pi:/usr/lib/modules/6.1.0-rpi8-rpi-2712/kernel/arch/arm64/crypto $ ls /usr/lib/modules/6.1.0-rpi8-rpi-2712/kernel/lib/crypto libaes.ko.xz libarc4.ko.xz libchacha20poly1305.ko.xz libchacha.ko.xz libcurve25519-generic.ko.xz libcurve25519.ko.xz libpoly1305.ko.xz

Have you tried adding the kernel modules to /etc/initramfs-tools/modules? Having done so on my build, I now see those objects where I'd expect.

Just saw your modules file - the only way I can investigate further is to spin up another device, so I'll schedule this digging.

@tdewey-rpi
Copy link

Can you confirm you have libpoly1305.ko in your initramfs?

@bklop
Copy link
Author

bklop commented Jan 31, 2024

Yes libpoly1305.ko was copied:

pi@raspberrypi:~ $ lsinitramfs /boot/initramfs.gz | grep -P "(cryptsetup|adiantum|libpoly1305)"
usr/lib/aarch64-linux-gnu/libcryptsetup.so.12
usr/lib/aarch64-linux-gnu/libcryptsetup.so.12.9.0
usr/lib/modules/6.1.0-rpi7-rpi-v8/kernel/crypto/adiantum.ko
usr/lib/modules/6.1.0-rpi7-rpi-v8/kernel/lib/crypto/libpoly1305.ko

@bklop
Copy link
Author

bklop commented Apr 8, 2024

@tdewey-rpi did you get a chance to look into this?

@AutumnSpark1226
Copy link

I followed the same (I guess; this one? https://rr-developer.github.io/LUKS-on-Raspberry-Pi/) tutorial and got similar errors.
Firstly, I noticed that everthing that was in /boot in the tutorial must now be in /boot/firmware. You can move /boot/initramfs.gz to /boot/firmware/initramfs.gz
Secondly, on 64bit RasPi OS installations the aes_arm module doesn't exist. Instead you should use the aes_arm64 module.
Manually adding libgcc_s.so.1 and libpthread.so.0 is still required.

@marsante
Copy link

marsante commented Aug 1, 2024

@bklop
According to /etc/initramfs-tools/modules https://alioth-lists.debian.net/pipermail/pkg-cryptsetup-devel/2023-January/009588.html
you can add

aes_generic
chacha_generic
nhpoly1350
xts

in /etc/initramfs-tools/modules to fix:

However, after booting into initramfs, I still cannot use cryptfs to unlock the volume:
Cannot use xchacha20, aes-adiantum-plain64 encryption

Despite all the fixes I've made, when i unlocked luks on initramfs and typed exit initramfs opens with ALERT! root=/dev/mapper/sdcard does not exist. Switching to a shell. ls /dev/mapper/sdcard shows it is present oddly.

@exander77
Copy link

exander77 commented Aug 31, 2024

I am having exactly the same issue.

I don't understand how can it write that /dev/mapper/XXX is not present if I see it is there. What kind of nonsensical error message is that?

@exander77
Copy link

exander77 commented Aug 31, 2024

Seems culprit is this function:

# Resolve device node from a name.  This expands any LABEL or UUID.
# $1=name
# Resolved name is echoed.
resolve_device() {
        DEV="$1"

        case "$DEV" in
        LABEL=* | UUID=* | PARTLABEL=* | PARTUUID=*)
                DEV="$(blkid -l -t "$DEV" -o device)" || return 1
                ;;
        esac
        [ -e "$DEV" ] && echo "$DEV"
}

It doesn't handle case where device is already resolved, like:

root=/dev/mapper/XXX

@exander77
Copy link

Fix:

resolve_device() {
    DEV="$1"

    case "$DEV" in
    /dev/*)
        # If the input starts with /dev/, we don't need to do anything else
        ;;
    LABEL=* | UUID=* | PARTLABEL=* | PARTUUID=*)
        DEV="$(blkid -l -t "$DEV" -o device)" || return 1
        ;;
    esac
    
    [ -e "$DEV" ] && echo "$DEV"
}

@marsante
Copy link

marsante commented Sep 1, 2024

Thanks for sharing a fix. It's located in /usr/share/initramfs-tools/scripts/functions ? I’ll be able to test it in a week.

@exander77
Copy link

Thanks for sharing a fix. It's located in /usr/share/initramfs-tools/scripts/functions ? I’ll be able to test it in a week.

Yes. For practical reasons I do this:

sudo sed -i '/^resolve_device()/,/^}/c\resolve_device() {\n    DEV="\$1"\n\n    case "\$DEV" in\n    /dev/*)\n        ;;\n    LABEL=* | UUID=* | PARTLABEL=* | PARTUUID=*)\n        DEV="\$(blkid -l -t "\$DEV" -o device)" || return 1\n        ;;\n    esac\n\n    [ -e "\$DEV" ] && echo "\$DEV"\n}' /mnt/root/usr/share/initramfs-tools/scripts/functions

Replacing entire function, /mnt/root is the location where I have root file system mounted.

@exander77
Copy link

@marsante Btw, what are you doing? I am currently making script that creates mdadm raid1, and luks on top of that and makes it a raspberry pi root. I have 2x m.2 on Raspberry Pi. seems to work fine after I fixed this.

@marsante
Copy link

marsante commented Sep 2, 2024

@exander77 much simpler than you, just encrypting the SD card.

@marsante
Copy link

marsante commented Sep 8, 2024

I can test, it works, but every time I reboot, I have to reopen luks via initramfs, maybe I missed something ?

Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
Begin: Running /script/local-premount ... done.
Begin: Waiting for root file system ... Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
...

@exander77
Copy link

I can test, it works, but every time I reboot, I have to reopen luks via initramfs, maybe I missed something ?

Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
Begin: Running /script/local-premount ... done.
Begin: Waiting for root file system ... Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
...

I am not sure what you mean by this.

@marsante
Copy link

Sorry if it's not clear .
I apply your fix

Fix:

resolve_device() {
    DEV="$1"

    case "$DEV" in
    /dev/*)
        # If the input starts with /dev/, we don't need to do anything else
        ;;
    LABEL=* | UUID=* | PARTLABEL=* | PARTUUID=*)
        DEV="$(blkid -l -t "$DEV" -o device)" || return 1
        ;;
    esac
    
    [ -e "$DEV" ] && echo "$DEV"
}

After that I can finish all the procedures in initramfs, exit it and get into the OS.
But if I reboot, the above error message appears and I get into initramfs. I can run

cryptsetup luksOpen /dev/mmcblk0p2 sdcard
exit

to boot into the OS.

@exander77
Copy link

cryptsetup luksOpen /dev/mmcblk0p2 sdcard
exit

to boot into the OS.

That would be expected if nothing else is calling luksOpen. Do you have some script in initramfs to open luks?
If you don't want to write it yourself, you can install cryptsetup-initramfs, that installs such script into initramfs.

@exander77
Copy link

If you list the contents, you can see what it does:

$ apt-file list cryptsetup-initramfs
cryptsetup-initramfs: /etc/cryptsetup-initramfs/conf-hook
cryptsetup-initramfs: /usr/share/bug/cryptsetup-initramfs
cryptsetup-initramfs: /usr/share/cryptsetup/initramfs/bin/cryptroot-unlock
cryptsetup-initramfs: /usr/share/doc/cryptsetup-initramfs/README.initramfs.gz
cryptsetup-initramfs: /usr/share/doc/cryptsetup-initramfs/changelog.Debian.gz
cryptsetup-initramfs: /usr/share/doc/cryptsetup-initramfs/copyright
cryptsetup-initramfs: /usr/share/initramfs-tools/conf-hooks.d/cryptsetup
cryptsetup-initramfs: /usr/share/initramfs-tools/hooks/cryptgnupg
cryptsetup-initramfs: /usr/share/initramfs-tools/hooks/cryptgnupg-sc
cryptsetup-initramfs: /usr/share/initramfs-tools/hooks/cryptkeyctl
cryptsetup-initramfs: /usr/share/initramfs-tools/hooks/cryptopensc
cryptsetup-initramfs: /usr/share/initramfs-tools/hooks/cryptpassdev
cryptsetup-initramfs: /usr/share/initramfs-tools/hooks/cryptroot
cryptsetup-initramfs: /usr/share/initramfs-tools/hooks/cryptroot-unlock
cryptsetup-initramfs: /usr/share/initramfs-tools/scripts/local-block/cryptroot
cryptsetup-initramfs: /usr/share/initramfs-tools/scripts/local-bottom/cryptgnupg-sc
cryptsetup-initramfs: /usr/share/initramfs-tools/scripts/local-bottom/cryptopensc
cryptsetup-initramfs: /usr/share/initramfs-tools/scripts/local-bottom/cryptroot
cryptsetup-initramfs: /usr/share/initramfs-tools/scripts/local-top/cryptopensc
cryptsetup-initramfs: /usr/share/initramfs-tools/scripts/local-top/cryptroot
cryptsetup-initramfs: /usr/share/lintian/overrides/cryptsetup-initramfs

Basically, it installs a hook and unlock script that calls luksOpen.

@marsante
Copy link

Oh thanks, I thought it was a recommended package with cryptsetup, but I have to install it manually. I'll be able to search in the installed files, because it unfortunately doesn't change the behavior at boot.

@marsante
Copy link

Ok I think I got it. First time I made.

sudo update-initramfs -u
sudo -E CRYPTSETUP=y mkinitramfs -o /boot/firmware/initramfs.gz

This time I make :

sudo update-initramfs -c -k $(uname -r)
sudo -E CRYPTSETUP=y mkinitramfs -o /boot/firmware/initramfs.gz

I don't know if sudo -E CRYPTSETUP=y mkinitramfs -o /boot/firmware/initramfs.gz command was necessary in addition to sudo update-initramfs -c -k $(uname -r) but at the end it works.

Thank again for your help @exander77

@exander77
Copy link

Ok I think I got it. First time I made.

sudo update-initramfs -u
sudo -E CRYPTSETUP=y mkinitramfs -o /boot/firmware/initramfs.gz

This time I make :

sudo update-initramfs -c -k $(uname -r)
sudo -E CRYPTSETUP=y mkinitramfs -o /boot/firmware/initramfs.gz

I don't know if sudo -E CRYPTSETUP=y mkinitramfs -o /boot/firmware/initramfs.gz command was necessary in addition to sudo update-initramfs -c -k $(uname -r) but at the end it works.

Thank again for your help @exander77

Btw, I don't think you have to set up CRYPTSETUP=y on command line. You may be missing it in configuration if you actually need it. You definitely need to rebuild initramfs with update-initramfs tools.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants