Project

General

Profile

kmod-31-1 causes problems with linux-libre

bdunahu - 7 months ago -

I believe had my first failed system update today.

When booting, I am dropped into what appears to be an emergency maintenance shell created by systemd-udevd after systemd fails to mount /boot/efi. The relevant errors from the journal seem to be:

mount[257]: mount: /boot/efi: unknown filesystem type 'vfat'.
mount[257]:        dmesg(1) may have more information after failed mount system call.
systemd[1]: boot-efi.mount: Mount process exited, code=exited, status=32/n/a
systemd[1]: boot-efi.mount: Failed with result 'exit-code'.
systemd[1]: Failed to mount /boot/efi.
systemd[1]: Dependency failed for Local File Systems.
systemd[1]: local-fs.target: Job local-fs.target/start failed with result 'dependency'.
systemd[1]: local-fs.target: Triggering OnFailure= dependencies.
systemd[1]: systemd-ask-password-console.path: Deactivated successfully.

The last time I had a successful update was Thursday. There are no abnormal messages in the pacman log, but I do remember having some issues during the time of update with the pcr database having no servers responding, but I chose to push the update through anyways (I am vague here because I have no log of exactly what occurred). Being able to boot into my environment outside of the install media would help me troubleshoot this issue further. I have tried to revert all of the packages upgraded today to their previous versions, but have not had success using the cache (I never clear it, but most of the earlier versions of packages are absent from my pacman cache, for whatever reason).

Note I am writing this by hand, but can properly export these files if need be. Packages updated today were:

removed kguiaddons
upgraded bluez-libs
upgraded brotli
upgraded cairo
upgraded libsigc++
upgraded cairomm
upgraded coreutils
upgraded feh
upgraded gnutls
upgraded libcups
upgraded mesa
upgraded kmod
upgraded qt5-base
upgraded qt5wayland
installed kguiaddons5
upgraded flameshot
upgraded gperftools
upgraded gtk-update-icon-cache
upgraded imagemagick
upgraded libixion
upgraded libnvme
upgraded liborcus
upgraded pinentry
upgraded libreoffice-still
upgraded libsynctex
upgraded libvpx
upgraded man-db
upgraded meson
upgraded mpg123
upgraded openexr
upgraded parabola-keyring

This forum also seems relevant:
https://bbs.archlinux.org/viewtopic.php?id=178358

Here is what is displayed when I query:

$ uname -r
6.3.3-gnu-1
$ pacman -Q linux
linux-libre 6.3.3-1

To my knowledge, the versions are the same. I also did not update the kernel today, so my issue seems to be something different.

Any help understanding this is appreciated, as well as any advice as to what I should safely try next. Thank you.


Replies (11)

RE: Advice for failed system update - Necklace - 7 months ago -

I think I've seen that error message before after an install.
To fix it I did as it says in the note here: https://wiki.archlinux.org/title/EFI_system_partition#Mount_the_partition

Add a file in /etc/modules-load.d/vfat.conf

with contents:

vfat
nls_cp437
nls_ascii

Pretty sure that solved my issue, have you tried that?

RE: Advice for failed system update - bdunahu - 7 months ago -

I had not tried that, but it unfortunately did not solve my issue. When it tries to load that module early, the journal outputs the same:

systemd-modules-load191: Failed to insert module 'vfat': No such file or directory

Even when I am using the installation media and attempting to reconfigure grub, I get an "unknown file-system" error.

I am confused because I have not updated my kernel in the past couple weeks, which is when most people seem to have similar issues.

Commenting out this mount in /etc/fstab does allow my computer to boot into my usual environment, however, which is quite a help for the time being.

RE: Advice for failed system update - Necklace - 7 months ago -

Well, you could try temporarily downgrading the kernel I suppose.

When you reconfigure grub, what command are you using?
By the way, I'm assuming you actually do have an EFI partition and that it is mounted when you try to reconfigure grub - otherwise I think it might fail.

Also, I suggest maybe running grub-install again, I remember I had to do that a while ago, something like this:

grub-install --target=x86_64-efi --efi-directory=/efi --bootloader-id=GRUB --removable --recheck /dev/sdaX

Obviously correct /efi and /dev/sdaX to your own paths.

RE: Advice for failed system update - bdunahu - 7 months ago -

I was running grub-install, but I do not think grub is the issue--the installation media is the only system capable of mounting my ESP. When I try to configure grub when I am arch-chrooted into my standard root directory, it complains it is unable to even read the filesystem. I do not think the update I did yesterday regenerated my initramfs files yesterday--so I am guessing one of the packages installed yesterday failed... I could try to look up what each one does, and reinstall whichever ones I suspect, though I am not sure if that would be successful.

It is strange to me that my pacman cache only contains .zst files for the package versions currently installed on my system. I have never had to downgrade packages, though I had thought it preserved a large history of versions:

pacman -Q feh
feh 3.10.1-2

ls -la /var/cache/pacman/pkg/*feh*
-rw-r--r-- 1 root root 189057 Oct 2 18:14 /var/cache/pacman/pkg/feh-3.10.1-2-x86_64.pkg.tar.zst
-rw-r--r-- 1 root root    566 Oct 2 18:14 /var/cache/pacman/pkg/feh-3.10.1-2-x86_64.pkg.tar.zst.sig

Is this expected?

RE: Advice for failed system update - Necklace - 7 months ago -

Is what expected? That your cache only has the current version?
No I'd say that is not the default - do you have a hook that removes older versions?
Something like a /etc/pacman.d/hooks/paccache.hook file ?

Maybe there's an archived version somewhere on the net, or maybe you can try linux-libre-lts ?

Speaking of which have you tried checking /etc/mkinitcpio.conf ?
Maybe it was modified, I'm sure you've tried mkinitcpio -p linux-libre already.
Though that really shouldn't matter.
Maybe it actually is something with the new kernel version after all..

RE: Advice for failed system update - bdunahu - 7 months ago -

I think I am addressing the wrong issue now. I have noticed that my wireless, ethernet, and mousepad do not function in my environment, after skipping the problem with the vfat module. There are some new failures in my journal file when iwd.service and dhcpcd fail to start:

iwd[291]: No HMAC(SHA1) support found
iwd[291]: No HMAC(MD5) support found
iwd[291]: No CMAC(AES) support found
iwd[291]: No HMAC(SHA256) support found
iwd[291]: No HMAC(SHA512) support found, certain TLS connections might fail
iwd[291]: DES support not found
iwd[291]: AES support not found
iwd[291]: No CBD(DES3_EDE) support found, certain TLS connections might fail
...
iwd[291]: The following options are missing in the kernel:
iwd[291]:         CONFIG_CRYPTO_USER_API_HASH
iwd[291]:         CONFIG_CRYPTO_USER_API_SKCIPHER
iwd[291]:         CONFIG_CRYPTO_USER_API_ECB
iwd[291]:         CONFIG_CRYPTO_USER_API_MD5
iwd[291]:         CONFIG_CRYPTO_USER_API_CBC
iwd[291]:         CONFIG_CRYPTO_USER_API_SHA256
...

I do not know where to look for anything related to ethernet or the mousepad, however. I am not sure if this is the usual output for this kernel (likely not, because I use these services normally).

I also have no hook files in that directory. I have not had a problem with space--so I did not manually try to configure pacman to remove cached versions. I have only really configured my mirrorlist.

RE: Advice for failed system update - cshep - 7 months ago -

I am having this same issue after a system upgrade. The only packages that were upgraded for me were kmod and parabola keyring.

Could this have something to do with kmod? It's a dependency of linux-libre and it's description says "Linux kernel module management tools and library"

My last kernel upgrade was May 23 to 6.3.3-1. My fstab looks normal. UUID's and mountpoints are correct.

I am have the exact same issue as OP.

RE: Missing kernal modules 'vfat' and others after update - bdunahu - 7 months ago -

Did you fix this problem? If not, when did you install the update that broke your system?

You should try reverting those updates if you have the older versions still in your cache. I am unable to, but am curious if that would fix it.

RE: Missing kernal modules 'vfat' and others after update - bdunahu - 7 months ago -

Wow, thank you cshep, I overlooked that package. Reinstalling kmod 30-3 from the archlinux archive fixed this issue for me, at least until I need to re-update.

RE: kmod-31-1 causes problems with linux-libre - Necklace - 7 months ago -

That makes sense, the next thing I was going to ask for was to grep for vfat in /lib/modules/6.x.y/*

And now I see there's actually an issue for kmod too, might want to confirm this then: https://labs.parabola.nu/issues/3539

RE: kmod-31-1 causes problems with linux-libre - bdunahu - 7 months ago -

You're right. I suppose I should check there sooner if I have another issue like this. Thank you for your help, Necklace

    (1-11/11)