Fixes for mixing the kernel and userland from different distributions

Introduction

In “Install Slackware in a chroot from any distribution” we saw how to use the kernel of Fedora Linux in an installation of Slackware when Secure Boot is enabled. We noticed an issue with the device driver of the audio card, but the same kernel and driver were working in Fedora, so it must be a fixable issue.

The same issue happened when we installed Alpine Linux with Fedora’s kernel (see Alpine Linux on a laptop) on the same laptop.

When we tried Alpine Linux with Slackware kernel on another device, we did not find any problem.

How to fix the audio card issue

To investigate the issue, try to list the available cards with aplay -L (available in the package alsa-utils). In my case there was only the null device and a dummy device. So running dmesg was the next logical step:

$ dmesg | grep -i -e fail -e error
[   54.081103] snd_sof_pci_intel_tgl 0000:03:00.5: failed to enable device (error -1)

Looks like a kernel module is failing to load. Trying to reload the module with modprobe doesn’t work. The issue is probably caused by a hidden dependency between modules or by a lack of tuning of the default values for the parameters of the module.

From the running Fedora (where the audio card works well) we took the list of loaded modules, saved in a file and then checked the difference:

# on Fedora
lsmod > lsmod_fedora
cat lsmod_fedora | cut -d ' ' -f 1 | tail -n +2 | sort > moudles_fedora

# on the Slackware/Alpine installation with audio issues
lsmod  | cut -d ' ' -f 1 | tail -n +2 | sort > current_modules

# print the modules in modules_fedora, but not in current_modules
comm -23 modules_fedora current_modules

We removed from the list everything that did not seem related to sounds.

We also took advantage of the column “Used by” displayed by lsmod in Fedora to understand the dependencies. So far we identified 3 modules and their relationships:

snd_ctl_led -> snd_sof_probes -> snd_sof_pci_intel_tgl

At this point we tried to remove these modules from the kernel with rmmod and added again in the order with modprobe.

This test was successful, so it is evident that this a missing module from the list of modules that the OS is supposed to load during the bootstrap, and that the order of loading is important.

modprobe is supposed to load the module and its dependencies[1], however, funny enough, using modprobe only on snd_sof_pci_intel_tgl (after the removal of the other two modules) does not load also snd_ctl_led and the audio card remains not working.

So, this is the final steps we identified to fix our situation:

# remove modules from the running kernel
/sbin/rmmod snd_ctl_led
/sbin/rmmod snd_sof_probes
/sbin/rmmod snd_sof_pci_intel_tgl

# reload modules in the correct order
/sbin/modprobe snd_ctl_led
/sbin/modprobe snd_sof_pci_intel_tgl

The same sequence of operations fixes the issue both in Alpine and Slackware.

To make the change permanent:

In Alpine Linux you can create a .start script in /etc/local.d/[2][3] or edit the function start() of the script /etc/init.d/modules
In Slackware Linux put the commands in /etc/rc.d/rc.local[4][5]

The solution that we found works, but it would be more elegant to understand why the loading of the modules during startup did not work. There is at least a couple of strategies[6]:

the kernel modules are loaded from a static list saved in a file
the daemon udevd load automatically based on some rules

Slackware uses eudev[7], but Alpine relies on /etc/modules unless eudev is explicitly installed or it is required as a dependency (common case for desktop installations).

Since there is no straightforward intuitive configuration of udev, we are happy with our solution.

Failed to mount USB key formatted in exfat

The second small issue we found is a failure when trying to mount a USB storage drive.

The mount command fails to automatically detect and load the proper module for the filesystem. The typical filesystem used in a USD drive is FAT32[9] or ExFAT[8] and some kernels compile these filesystems as separate modules. Check with blkid what is the correct one.

In our case it’s ExFAT and after loading the device driver by running modprobe exfat, the issues were solved. To make the fix permanent might be useful to add the module’s name in /etc/modules (Alpine) or /etc/rc.d/rc.modules (Slackware).

Conclusions

Our experiments show that it is perfectly possible to run a distribution using kernel, kernel modules and firmware compiled for another distribution. Some small issues can happen during module loading, but once the fix is identified, it is a matter of putting the proper commands in the right script that runs during the bootstrap. We were able to identify the proper sequence of commands to fix both the audio card and the pen drive issues. We made the fix permanent by editing an init script. Now the issues are solved in both Alpine and Slackware.

References

[1] https://www.man7.org/linux/man-pages/man8/modprobe.8.html

[2] https://wiki.alpinelinux.org/wiki/OpenRC#Local_service

[3] https://github.com/OpenRC/openrc/blob/master/local.d/README

[4] https://docs.slackware.com/howtos:slackware_admin:runit

[5] The content of the file /etc/rc.d/rc.local which is:

#!/bin/bash
#
# /etc/rc.d/rc.local:  Local system initialization script.
#
# Put any local startup commands in here.  Also, if you have
# anything that needs to be run at shutdown time you can
# make an /etc/rc.d/rc.local_shutdown script and put those
# commands in there.