FATAL: kernel too old

[otakutyrant]

I would like to provide a detailed report but I am away from my PC and posting this issue on my phone instead. Actually I think it is needless to explicate it so far because today the new version of glibic is released so that thereafter everyone who have once updated their system will, maybe, encounter the same issue. I think it may be time to update the kernel of ArchWSL and release it.

Update: The specific issue is that when you upgrade the system, you will encounter "FATAL: kernel too old" after the update of glibc and some programs (maybe because they are depended on it?) and every prompt in a Shell.

[claudiocabral]

had the same issue, at the end "solved" by updating the kernel with these instructions and switching to WSL 2

[Enter-tainer]

had the same issue, at the end "solved" by updating the kernel with these instructions and switching to WSL 2

Is it possible for me to stay on wsl1? Since wsl2 has terrible IO performance on Windows disks, a simple git status can take more 10 seconds.

[Esgariot]

It's most likely due to glibc 2.33-4 update which happened today.

[typr7]

I have the same issue, arch wsl conldn't work after glibc update.

[claudiocabral]

just a work around if you want to use WSL 1:
I switched to WSL 2, changed my pacman mirror to point to the 31st of January archive
Server = https://archive.archlinux.org/repos/2021/01/31/$repo/os/$arch
and reverted all of my packages to that point with pacman -Syyuu.
I bricked my WSL install a couple of times in the process, so try at your own risk.

[otakutyrant]

had the same issue, at the end "solved" by updating the kernel with these instructions and switching to WSL 2

I studied those instructions and found out installing a preview build of Windows 10 is required but I do not want to put my working computer at risk of using an unstable system. It can't be helped.

I googled how to upgrade the kernel of WSL but no useful information was found.

[krnets]

Set your distribution version to WSL 2:

wsl -l -v

  NAME    STATE           VERSION
* Arch    Stopped         1


wsl --set-version Arch 2

If version still shows 1, run the following and reboot:

dism.exe /online /enable-feature /featurename:VirtualMachinePlatform /all /norestart

Run wsl --set-version Arch 2 again.

Conversion in progress, this may take a few minutes...

If update successful, wsl -l -v will list as version 2:

wsl -l -v

  NAME    STATE           VERSION
* Arch    Stopped         2

[dynxer]

For those who want to stay at WSL1 and want to fix the problem, a simple (also risky) solution is to overwrite the WSL installation directly with the old glibc package.

Update: Please follow the #190 (comment) for more detailed steps.
In detail, use the /usr part from glibc-2.33-3 to overwrite your WSL installation, then you can get into it and downgrade glibc-2.33-4 to glibc-2.33-3. Remember to modify pacman.conf to ignore subsequent upgrades of glibc.

It is important to reiterate that there are risks associated with this operation, such as the permissions of the replaced files being reset. Although in my case these file permissions would be reset to 777, making it possible to go into WSL and reinstall glibc. After reinstallation these permissions will be corrected.

According to this document, permissions are stored in NTFS extended attributes. Using some specific way to overwrite (e.g. dd) can preserve NTFS extended attributes, which is also safer.

[devtryongit]

For those who want to stay at WSL1 and want to fix the problem, a simple (also risky) solution is to overwrite the WSL installation directly with the old glibc package.

In detail, use the /usr part from glibc-2.33-3 to overwrite your WSL installation, then you can get into it and downgrade glibc-2.33-4 to glibc-2.33-3. Remember to modify pacman.conf to ignore subsequent upgrades of glibc.

It is important to reiterate that there are risks associated with this operation, such as the permissions of the replaced files being reset. Although in my case these file permissions would be reset to 777, making it possible to go into WSL and reinstall glibc. After reinstallation these permissions will be corrected.

According to this document, permissions are stored in NTFS extended attributes. Using some specific way to overwrite (e.g. dd) can preserve NTFS extended attributes, which is also safer.

I had tried this myself, but by overwriting all the files of glibc package, and unfortunately it broke the installation even further. I now tried to overwrite all files from glibc-2.33-4 (to restore a "clean" state) and then /usr files only from glibc-2.33-3, but it is still broken (shell immediately closes).

In the end I had to install a new instance of Arch and set the server to the archive as suggested by @claudiocabral. Packages from the 14th of February are still ok.

Server = https://archive.archlinux.org/repos/2021/02/14/$repo/os/$arch

Very annoying I had to reinstall/reconfigure everything though.

[liyiliuxingyu]

我的也失败了，就等着作者更新了。

[dynxer]

After more testing, I have refined that solution. Here are the steps.

1. Make a another WSL1 installation, no matter what distro is. Get into it when you finshied the install.
2. Remount the drive with featrue metadata where your broken ArchWSL installation is located. If it is on drive C: it should be:
   sudo umount /mnt/c
sudo mount -t drvfs C: /mnt/c -o metadata
3. Get glibc-2.33-3 and extract it with tar. Be careful to keep the permissions correct. For example:
   sudo tar --zstd -xf glibc-2.33-3-x86_64.pkg.tar.zst
4. Overwrite the /usr part with the extracted package. Remember to change to your specific location.
   sudo cp -r --preserve=all --force <Extracted Pakage>/usr/ <Broken ArchWSL Location>/rootfs/
5. Restart the service LxssManager. This can force a file system refresh, which may be delayed in some cases. Please note that this will stop all WSL instances.
6. You should be able to get into the broken ArchWSL.

The whole point of the process is to make sure that the replaced files have proper permissions. While this may be the safest steps, please note that it still carries risks. You can also try upgrading to WSL2 first, downgrading glibc and then downgrade to WSL1. This should work too.
There is also a glibc-linux4 from AUR.

[dynxer]

For those who want to stay at WSL1 and want to fix the problem, a simple (also risky) solution is to overwrite the WSL installation directly with the old glibc package.
In detail, use the /usr part from glibc-2.33-3 to overwrite your WSL installation, then you can get into it and downgrade glibc-2.33-4 to glibc-2.33-3. Remember to modify pacman.conf to ignore subsequent upgrades of glibc.
It is important to reiterate that there are risks associated with this operation, such as the permissions of the replaced files being reset. Although in my case these file permissions would be reset to 777, making it possible to go into WSL and reinstall glibc. After reinstallation these permissions will be corrected.
According to this document, permissions are stored in NTFS extended attributes. Using some specific way to overwrite (e.g. dd) can preserve NTFS extended attributes, which is also safer.

I had tried this myself, but by overwriting all the files of glibc package, and unfortunately it broke the installation even further. I now tried to overwrite all files from glibc-2.33-4 (to restore a "clean" state) and then /usr files only from glibc-2.33-3, but it is still broken (shell immediately closes).

In the end I had to install a new instance of Arch and set the server to the archive as suggested by @claudiocabral. Packages from the 14th of February are still ok.

Server = https://archive.archlinux.org/repos/2021/02/14/$repo/os/$arch

Very annoying I had to reinstall/reconfigure everything though.

I also tested that the shell immediately closes may be because the overwritten file permissions have been reset to unreadable or unexecutable.
If you still want to try to fix it, consider referring to the new comment I just posted. Following that should ensure that the overwritten files have the correct permissions from the start. You may need to overwrite with the entire package since you've done that before.

[devtryongit]

@dynxer thanks, will try for sure in the next days

[otakutyrant]

@dynxer

4. Overwrite the /usr part with the extracted package. Remember to change to your specific location.
   sudo cp -r --preserve=all --force <Extracted Pakage>/usr/ <Broken ArchWSL Location>/rootfs/

I checked my location C:\Users\86139\AppData\Local\Packages but I am not sure what directory is the ArchWSL location. No directory contains "Arch". Ony that Canical.Unbuntu contains rootfs when I searched the latter. Any idea?

[otakutyrant]

I have a sneaking suspiction that my ArchWSL is overwrited by Ubuntu which I have just installed in order to follow @dynxer steps. However it still returns the FATAL error when I open Powershell and execute wsl.

[dynxer]

@dynxer

1. Overwrite the /usr part with the extracted package. Remember to change to your specific location.
   sudo cp -r --preserve=all --force <Extracted Pakage>/usr/ <Broken ArchWSL Location>/rootfs/

I checked my location C:\Users\86139\AppData\Local\Packages but I am not sure what directory is the ArchWSL location. No directory contains "Arch". Ony that Canical.Unbuntu contains rootfs when I searched the latter. Any idea?

The rootfs location should under the folder where you extract the ArchWSL installer zip.

I have a sneaking suspiction that my ArchWSL is overwrited by Ubuntu which I have just installed in order to follow @dynxer steps. However it still returns the FATAL error when I open Powershell and execute wsl.

If you still get FATAL, this means that ArchWSL has not been overwrited. You can check your all WSL installation with wsl -l

[otakutyrant]

The rootfs location should under the folder where you extract the ArchWSL installer zip.

Okay, I found my ArchWSL location. Let me try again.

If you still get FATAL, this means that ArchWSL has not been overwrited. You can check your all WSL installation with wsl -l

You are right. My wsl -l shows that there are ArchWSL, regarded as default, and Ubuntu.

[otakutyrant]

My ArchWSL reborned successfully! I thought arch-chroot but I didn't know where to chroot into. Now @dynxer dispensed a viable way. Thanks a lot!

[hzhangxyz]

It seems microsoft never update WSL1 kernel version, so if we do not want to use WSL2, we need keep glibc to IgnorePkg in /etc/pacman.conf. Since glibc-2.33-3 works well, but glibc-2.33-4 break the system, maybe next version glibc will not complain kernel version too old? just a hope...

To fix the problem temporarily, just download a pacman-static and ./pacman-static -U /var/cache/pacman/pkg/glibc-2.33-3-x86_64.pkg.tar.zst

[hzhangxyz]

I really do not like WSL2, which memory usage is a hell, I am really upset now.

[hzhangxyz]

ArchLinux will never have a plan to solve it, see here
Archlinux on WSL is not Archlinux? I think Archlinux on WSL is just an Archlinux with one package replaced, which names is linux.

While microsoft update is very slow, and it seems WSL1 kernel is not updated for a long time. And more, see here, Recent release notes are all about WSL2! although microsoft says "WSL1 is not abandoned",

[xctan]

just a work around if you want to use WSL 1:
I switched to WSL 2, changed my pacman mirror to point to the 31st of January archive
Server = https://archive.archlinux.org/repos/2021/01/31/$repo/os/$arch
and reverted all of my packages to that point with pacman -Syyuu.
I bricked my WSL install a couple of times in the process, so try at your own risk.

Thanks very much! This did work. For China users, TUNA has provided an Arch Linux Rollback Machine. I successfully recovered my access to Archlinux on WSL1 using the mirror below.
Server = https://arch-archive.tuna.tsinghua.edu.cn/2021/02-08/$repo/os/$arch

[apisarenco]

In case you don't want to deal with separate files and risk breaking your machine more than it already is, while it no longer starts up, here's what you can do:

1. In PowerShell list your distributions wsl -l -v
2. Then run the export command wsl --export <distro name from above command> Arch.tar
3. Copy the .tar file to ANY linux machine. I happened to have a VirtualBox running another Arch instance so I went there.
4. In the Linux machine, create a new directory, go there, and run this command sudo tar --same-owner -xvpf /path/to/Arch.tar
5. Now that you have your full wsl machine in a directory, chroot to it: sudo chroot . /bin/bash -i
6. Download the working glibc: wget https://archive.archlinux.org/packages/g/glibc/glibc-2.33-3-x86_64.pkg.tar.zst
7. Install the working glibc: pacman -U glibc-2.33-3-x86_64.pkg.tar.zst
8. Test if 2.33-3 is installed: pacman -Qi glibc
9. Edit /etc/pacman.conf and add glibc to IgnorePkg, for example: IgnorePkg = fakeroot glibc
10. Clean up rm glibc-2.33-3-x86_64.pkg.tar.zst
11. re-tar your directory sudo tar --same-owner -cvpf ../Arch2.tar .
12. Copy the new tar file to the Windows machine with wsl troubles
13. Unregister the old distribution if you want to re-use the name wsl --unregister <distro name>
14. Import the "fixed" distribution (you also get to choose where it's storing the rootfs and temp directories): wsl --import <distro name you want to use> M:\wsl\Arch C:\Path\To\Arch2.tar
15. Your wsl is ready and working. But you it's like a new WSL, so you need to apply any old configuration you have, for example: Arch.exe config --default-user {username}

$ pacman -Qi glibc
Name            : glibc
Version         : 2.33-3

$ sudo pacman -Syuu
:: Synchronizing package databases...
 core is up to date
 extra is up to date
 community is up to date
:: Starting full system upgrade...
warning: fakeroot: ignoring package upgrade (1.25.3-1 => 1.25.3-2)
warning: glibc: ignoring package upgrade (2.33-3 => 2.33-4)
 there is nothing to do

[viruscamp]

Updated for both new WslFS and old LxFS

There is another method which do not need another Linux or WSL distro.

0. If you have update to glibc-2.34 or greater, do not below, upgrade to WSL2 or reinstall
1. Download https://archive.archlinux.org/packages/g/glibc/glibc-2.33-3-x86_64.pkg.tar.zst
2. Use https://github.com/mcmilk/7-Zip-zstd/ to extract files in glibc*.zst/usr/lib/ below to a temp dir d:\usrlib for example

ld-2.33.so
libc-2.33.so
libdl-2.33.so
libm-2.33.so
libpthread-2.33.so
libresolv-2.33.so
librt-2.33.so
libutil-2.33.so

3. Use https://github.com/viruscamp/lxssattr/ in cmd to copy correct posix attrib to copied files

> cd <Your WSL Distro Folder>\rootfs\usr\lib
# found a valid source file , make sure the file is Ownership(Uid: (0 / ), Gid: (0 / )) and Access((0755) -rwxr-xr-x)
> lxssattr libdl-2.33.so
# output for new WslFS
$LXUID:                    Uid: (0 / )
$LXGID:                    Gid: (0 / )
$LXMOD:                    Mode: 100755 (octal)
Access:                    (0755) -rwxr-xr-x

# output for old LxFS
LXATTRB:
Flags:                     0
Version:                   1
Mode:                      100755 (octal)
Ownership:                 Uid: (0 / ), Gid: (0 / )
Access:                    (0755) -rwxr-xr-x
Last status change:        2019-11-19 18:29:52.102270300 +0800
Last file access:          2019-11-19 18:29:52.000000000 +0800
Last file modification:    2019-11-14 01:57:46.000000000 +0800

# copy and set files listed in step 2
> copy d:\usrlib\ld-2.33.so .\
> lxssattr libdl-2.33.so ld-2.33.so
> copy d:\usrlib\libc-2.33.so .\
> lxssattr libdl-2.33.so libc-2.33.so

# last one, change a source
> copy d:\usrlib\libdl-2.33.so .\
> lxssattr ld-2.33.so libdl-2.33.so

4. Restart Windows service LxssManager
5. Now you can open bash of ArchWSL. Let's follow @apisarenco 's advices FATAL: kernel too old  #190 (comment)
6. Install the working glibc: pacman -U glibc-2.33-3-x86_64.pkg.tar.zst
7. Test if 2.33-3 is installed: pacman -Qi glibc
8. Edit /etc/pacman.conf and add glibc to IgnorePkg, for example: IgnorePkg = fakeroot glibc
9. Done. It's OK to do: sudo pacman -Syuu

[specter119]

If you use an accessible clean archwsl with wsl1.
After https://wsldl-pg.github.io/ArchW-docs/How-to-Setup/#initialize-keyring
you can add the archlinux-cn mirror and then.

pacman -Syu --nodeps glibc-linux4

[TomoeMami]

https://github.com/archlinuxcn/mirrorlist-mirrors

archlinux-cn mirrorlist🤗

[SdtElectronics]

There is another method which do not need another Linux or WSL distro.

1. Download https://archive.archlinux.org/packages/g/glibc/glibc-2.33-3-x86_64.pkg.tar.zst
2. Use https://github.com/mcmilk/7-Zip-zstd/ to extract files below to /rootfs/usr/lib

/usr/lib/ld-2.33.so
/usr/lib/libc-2.33.so

Make sure the files are overwritten, otherwise you have to fix the symbol links.

1. Use https://github.com/viruscamp/lxssattr/ to copy correct posix attrib to copied files

> cd <Your WSL Distro Folder>\rootfs\usr\lib
> lxssattr libdl-2.33.so

Flags:                     0
Version:                   1
Mode:                      100755 (octal)
Ownership:                 Uid: (0 / ), Gid: (0 / )
Access:                    (0755) -rwxr-xr-x
Last status change:        2021-02-27 12:29:53.349915600 +0800
Last file access:          2021-02-27 12:29:53.000000000 +0800
Last file modification:    2021-02-06 08:09:35.000000000 +0800
> lxssattr libdl-2.33.so ld-2.33.so
> lxssattr libdl-2.33.so libc-2.33.so

Make sure the source file has the same Ownership(Uid: (0 / ), Gid: (0 / )) and Access((0755) -rwxr-xr-x)

1. Restart Windows service LxssManager
2. Now you can open bash of ArchWSL. Let's follow @apisarenco 's advices #190 (comment)
3. Install the working glibc: pacman -U glibc-2.33-3-x86_64.pkg.tar.zst
4. Test if 2.33-3 is installed: pacman -Qi glibc
5. Edit /etc/pacman.conf and add glibc to IgnorePkg, for example: IgnorePkg = fakeroot glibc
6. Done. It's OK to do: sudo pacman -Syuu

This saved my WSL. However, I encountered errors when using lxssattr

[ERROR] NtQueryEaFile: 0xc0000052 

But the operation seems succeeded anyway, so if anyone have met the same issue just don't worry. After some searching I learned what that lxssattr command do is copying the file attributes including the permissions so that Permission denied won't be thrown during loading libraries. Thus, when you lxssattr ld-2.33.so and lxssattr libc-2.33.so and it reports permission 755 , it means the operations succeeded.

[sl1pkn07]

Server = https://archive.archlinux.org/repos/2021/02/14/$repo/os/$arch

this snapshot install glibc-2.33-4, byt the day 13 not

[viruscamp]

ArchWSL installed as WSL1 recently used the new WslFS file system, for which old version of lxssattr is invalid.

ArchWSL installed long ago is still using the old LxFS file system, for which old version of lxssattr is valid. You may use wslconfig /upgrade <distro> change to WslFS.

As microsoft/WSL#1744 (comment) said:

Before the upgrade LxFs files used LXATTRB extended attribute to store the permissions but now both use LXGID LXMOD and LXUID.

[viruscamp]

I have update lxssattr to https://github.com/viruscamp/lxssattr/releases/tag/2.0 which can handle both old LxFS and new WslFS.

[liyiliuxingyu]

I have update lxssattr to https://github.com/viruscamp/lxssattr/releases/tag/2.0 which can handle both old LxFS and new WslFS.

I try it. Can't solve this problem. Can you tell me how can I use it?