Bug#1019425: dkms 3.0.6-2 not signing modules

[Holger Schröder]

Package: dkms
Version: 3.0.6-2
Severity: important

Dear Maintainer,

With dkms 3.0.6-2 the modules are no longer signed. This means that secure-boot
no longer works.

Back to 3.0.3-4 and and signing works again.

Cheers Holger...


-- System Information:
Debian Release: bookworm/sid
APT prefers unstable
APT policy: (500, 'unstable'), (100, 'experimental')
merged-usr: no
Architecture: amd64 (x86_64)

Kernel: Linux 5.19.0-1-amd64 (SMP w/16 CPU threads; PREEMPT)
Kernel taint flags: TAINT_PROPRIETARY_MODULE, TAINT_OOT_MODULE
Locale: LANG=de_DE.UTF-8, LC_CTYPE=de_DE.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages dkms depends on:
ii build-essential 12.9
ii dctrl-tools 2.24-3+b1
ii dh-dkms 3.0.6-2
ii dpkg-dev 1.21.9
ii gcc [c-compiler] 4:12.2.0-1
ii gcc-10 [c-compiler] 10.4.0-5
ii gcc-11 [c-compiler] 11.3.0-6
ii gcc-12 [c-compiler] 12.2.0-1
ii gcc-9 [c-compiler] 9.5.0-2
ii kmod 30+20220630-3
ii lsb-release 11.2
ii make 4.3-4.1
ii patch 2.7.6-7

Versions of packages dkms recommends:
ii fakeroot 1.29-1
ii linux-headers-amd64 [linux-headers-generic] 5.19.6-1
ii sudo 1.9.11p3-1

Versions of packages dkms suggests:
ii e2fsprogs 1.46.5-2
ii menu 2.1.49

-- Configuration Files:
/etc/modprobe.d/dkms.conf changed:


-- no debconf information

[Vincent Danjean]

Package: dkms
Version: 3.0.6-2
Followup-For: Bug #1019425
Control: tags -1 patch

The dkms script has several flaw that forbid module signing:
- Debian, contrary to ubuntu, does not have kmodsign
sign-file from the kernel should be directly used
- the script logic was wrong (if [[ -x "$(command -v XXX) ]]; then XXX missing ; fi => this is the reverse)
- debian update-secureboot-policy does not accept/use the --new-key and --enroll-key options (contrary to ubuntu?)

So, here is the patch I applied to dkms on my system in order to get module signing back.

Note that:
- the part of the patch to generate and enroll the key should be carefully checked
(I already have keys so I do not test this part of the patch)
Perhaps, "mokutil --import KEY" should be run after checking that the key is not already enrolled
- on upgrade, if a user previously make module signing with its own sign_tool/sign_helper.sh,
the key is not necessarly at the default expected place (/var/lib/dkms)
- perhaps, it would be better in Debian to put the key by default in
/etc/dkms/keys/ instead of /var/lib/dkms/ (the current default set in the dkms script)

Regards
Vincent

-- System Information:
Debian Release: bookworm/sid

APT prefers stable-updates
APT policy: (500, 'stable-updates'), (500, 'stable-security'), (500, 'oldstable-updates'), (500, 'unstable'), (500, 'testing'), (500, 'stable'), (500, 'oldstable'), (1, 'experimental')

merged-usr: no
Architecture: amd64 (x86_64)

Foreign Architectures: i386, armel, mipsel

Kernel: Linux 5.18.0-4-amd64 (SMP w/4 CPU threads; PREEMPT)
Kernel taint flags: TAINT_OOT_MODULE
Locale: LANG=fr_FR.utf8, LC_CTYPE=fr_FR.utf8 (charmap=UTF-8), LANGUAGE not set

Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages dkms depends on:
ii build-essential 12.9

ii clang-11 [c-compiler] 1:11.1.0-6+b2
ii clang-13 [c-compiler] 1:13.0.1-7
ii clang-14 [c-compiler] 1:14.0.6-2
ii clang-9 [c-compiler] 1:9.0.1-20+b1

ii dctrl-tools 2.24-3+b1
ii dh-dkms 3.0.6-2
ii dpkg-dev 1.21.9
ii gcc [c-compiler] 4:12.2.0-1
ii gcc-10 [c-compiler] 10.4.0-5
ii gcc-11 [c-compiler] 11.3.0-6

ii gcc-12 [c-compiler] 12.2.0-2

ii gcc-9 [c-compiler] 9.5.0-2
ii kmod 30+20220630-3
ii lsb-release 11.2
ii make 4.3-4.1
ii patch 2.7.6-7

Versions of packages dkms recommends:
ii fakeroot 1.29-1
ii linux-headers-amd64 [linux-headers-generic] 5.19.6-1
ii sudo 1.9.11p3-1

Versions of packages dkms suggests:
ii e2fsprogs 1.46.5-2
ii menu 2.1.49

-- no debconf information

[Chris Putnam]

I would argue the severity of this bug should be raised to grave. This is system-breaking for those that must keep secure boot enabled and are using necessary 3rd party drivers such as NVIDIA or out-of-tree modules like ZFS.

As a side note I discovered that despite secure boot being enabled, Debian's default kernel config is not enforcing signature checks in the first place. Those subscribed to this bug may find that interesting!

[Holger Schröder]

The patch does not work for me. The modules are signed again with the
patch but at boot time they are not accepted and I end up at the
initramfs prompt. zfs module not loaded in my case.


Cheers

[Tomas Janousek]

Hi,

Note that there have been several changes committed upstream related to signing modules on Debian/Ubuntu: https://github.com/dell/dkms/compare/v3.0.6...master

Specifically, https://github.com/dell/dkms/commit/8d60956f6dcda0679066954215eb8be4045413b4 and https://github.com/dell/dkms/commit/3ca52f8769bdf7ebdc83735355fff7c5c0664152 look relevant here. Might be worth testing that in preference to the patch posted here.

--

Tomáš "liskin" ("Pivník") Janoušek, https://lisk.in/

[Holger Schröder]

With a git-snashot it looks no different.


...

[Nye Liu]

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=939392

[Nicolas Cavallari]

Because patching /usr/sbin/dkms after every update quickly gets old, i
found another workaround by putting this into /etc/dkms/framework.conf:

====

do_signing=1

check_the_mok_key() {
case "${KBUILD_SIGN_PIN-}" in
[Nn][oO])
return 0;;
esac
KBUILD_SIGN_PIN="${KBUILD_SIGN_PIN-}" \
openssl rsa -in "$mok_signing_key" \
-passin env:KBUILD_SIGN_PIN -check -noout || return
}

ask_for_mok_password() {
until check_the_mok_key; do
stty -echo
printf "\nEnter passphrase for %s (type 'no' to cancel):" \
"$mok_signing_key"
IFS='' read -r KBUILD_SIGN_PIN || KBUILD_SIGN_PIN=no
stty echo
done
}

kmodsign() {
ask_for_mok_password < /dev/tty > /dev/tty 2>&1

KBUILD_SIGN_PIN="${KBUILD_SIGN_PIN-}" "$sign_file" "$@"
}

====

Hopefully this will be resolved in the future and this workaround will
no longer be necessary.

[Holger Schröder]

Modules signed with the new dkms-3.0.6-4 are rejected by the kernel and
I end up at the prompt of initramfs. In my case zfs-modul. The modules
are signed but not accepted. Neither with my own keys nor the ones
created by dkms.


Cheers Holger

[Holger Schröder]

Hi.

I have found the problem why it does not work. It is because of this line:


eval '"$sign_file" sha512 "$mok_signing_key" "$mok_certificate"
"$built_module"'


Here I get hash_algo sha512 signed which doesn't work on my box and my
laptop. If I change this to sha256 secure-boot works as desired. Why
this is so I can not say, this is beyond my knowledge. Also if this is a
bug or not I can't say.

Sorry for my crap english :-)


Cheers Holger...

[Holger Schröder]

https://github.com/dell/dkms/issues/266


...

[Siddh Raman Pant]

On Wed, 21 Sep 2022 10:49:46 +0200 Maurizio Avogadro <mav...@gmail.com> wrote:
> ... and support for MOK keys with a passphrase seem to have disappeared all at
> once...
>
> Furthermore, I'm unable to find any documentation on changes regarding module
> signing, and there are no useful comments on config files.

We can use the sign_helper.sh which was used in sign_tool config prior to v3 in
the sign_file config option in /etc/dkms/framework.conf.

Courtesy: https://github.com/dell/dkms/issues/273.

I use it with keyring in the following way:
https://gist.github.com/siddhpant/19c07b07d912811f5a4b2893ca706c99

Thanks,
Siddh

[Maurizio Avogadro]

Il 27/10/22 17:38, Siddh Raman Pant ha scritto:

>
> We can use the sign_helper.sh which was used in sign_tool config prior to v3 in
> the sign_file config option in /etc/dkms/framework.conf.
>
> Courtesy: https://github.com/dell/dkms/issues/273.
>
> I use it with keyring in the following way:
> https://gist.github.com/siddhpant/19c07b07d912811f5a4b2893ca706c99
>
> Thanks,
> Siddh

Hi Siddh

since dkms is supposed to run as root, it's not clear to me which keyring is
the signing key passphrase going to be stored to. I definitely wouldn't
recommend running Gnome/KDE keyrings as root, and user keyrings seem to be
unreachable via D-bus in a console where root has been gained with su -.

BTW, by adding a configuration snippet in /etc/dkms/framework.conf.d/ containing

export KBUILD_SIGN_PIN='my_mok_key_pin'

(permissions 600 recommended) seems to work (given the kernel you are running
is configured to use the SHA512 hash algorithm; official Debian kernels use
SHA256 [1]).

We also have to face the problem that only official Debian kernels store the
sign-file executable in

/usr/lib/linux-kbuild-${kernelver%.*}/scripts/

where dkms looks in Debian: Xanmod and Liquorix for example store this file in

/usr/src/linux-headers-${kernelver}/scripts/

but this path can't be easily changed in current dkms version (v3.0.6-4)
because the $kernelver variable isn't available when sourcing the configuration
files; dkms in current master tree fixed this issue by adding a fallback to

/lib/modules/${kernelver}/build/scripts/

in distro detection code, which makes signing work with any kernel I could test
till now.


[1] https://github.com/dell/dkms/issues/266

[Siddh Raman Pant]

On Thu, 27 Oct 2022 20:31:47 +0200, Maurizio Avogadro <mav...@gmail.com> wrote:> Hi Siddh

>
> since dkms is supposed to run as root, it's not clear to me which keyring is
> the signing key passphrase going to be stored to. I definitely wouldn't
> recommend running Gnome/KDE keyrings as root, and user keyrings seem to be
> unreachable via D-bus in a console where root has been gained with su

I used libsecret (which is used as backend by the Python program named keyring),
with the keyring being stored as root (isn't visible outside root in seahorse).
Can you tell why that may be a bad idea?

Anyways, the main point was that one can use a bash script. The keyring usage
was just a personal config choice for convenience, one can use other methods
for getting the passphrase from the user.

> BTW, by adding a configuration snippet in /etc/dkms/framework.conf.d/ containing
>
> export KBUILD_SIGN_PIN='my_mok_key_pin'
>
> (permissions 600 recommended) seems to work

I won't really want to store password in plaintext...

> (given the kernel you are running
> is configured to use the SHA512 hash algorithm; official Debian kernels use
> SHA256 [1]).

I didn't know that, thanks.

> We also have to face the problem that only official Debian kernels store the
> sign-file executable in
>
> /usr/lib/linux-kbuild-${kernelver%.*}/scripts/
>
> where dkms looks in Debian: Xanmod and Liquorix for example store this file in
>
> /usr/src/linux-headers-${kernelver}/scripts/
>
> but this path can't be easily changed in current dkms version (v3.0.6-4)
> because the $kernelver variable isn't available when sourcing the configuration
> files; dkms in current master tree fixed this issue by adding a fallback to
>
> /lib/modules/${kernelver}/build/scripts/
>
> in distro detection code, which makes signing work with any kernel I could test
> till now.

What I pointed out was setting sign_file to a bash script. It would solve this
issue too, as the correct path to the actual sign_file binary could be used in
the bash script / handler. dkms would call the bash script with its arguments,
and the script will call the correct binary (passing the arguments to it), as
$kernelver variable would be available to the script.

Also, v3.0.6-4 seems to have the fallback you mentioned. Refer:
https://salsa.debian.org/debian/dkms/-/blob/debian/3.0.6-4/dkms.in#L870

Thanks,
Siddh

[Maurizio Avogadro]

Il 27/10/22 21:56, Siddh Raman Pant ha scritto:

> I used libsecret (which is used as backend by the Python program named keyring),
> with the keyring being stored as root (isn't visible outside root in seahorse).
> Can you tell why that may be a bad idea?

It can be a nice solution but I admit I totally ignore how this works and many
questions arose on my mind: how is the keyring first created? Is the keyring
password protected? How is the keyring password asked? Does this work in a
su-ed/sudo-ed text only console? What happens when issuing this command in a
root console, and what would be the output:

# keyring get uefi mok

When talking of Gnome/KDE keyrings my head goes to X11/Wayland, and I prefer
not to run graphical applications as root to avoid making the whole process
more complex and therefore potentially more prone to security issues.

> [...]

> I won't really want to store password in plaintext...

Agreed, but remember that current dkms is designed to create new MOK key
_without_ a password, it doesn't currently even support using a password
protected MOK key: this is even less secure.
The whole secure boot support is still looking to get a stable shape and I
wouldn't deploy it in a production environment yet; in the meanwhile, while
experimenting it, I can live with a root only readable file containing the
clear-text password.

> [...]

> What I pointed out was setting sign_file to a bash script. It would solve this
> issue too, as the correct path to the actual sign_file binary could be used in
> the bash script / handler. dkms would call the bash script with its arguments,
> and the script will call the correct binary (passing the arguments to it), as
> $kernelver variable would be available to the script.

Yours is indeed a possible solution until the issues will be fixed on the dkms
side IMO: I used a bash sign-file script too at first; but this is actually a
workaround, and looking abstractly at the issue I think that if dkms is
supposed to manage module signing, a fix have to be introduced there.

> Also, v3.0.6-4 seems to have the fallback you mentioned. Refer:
> https://salsa.debian.org/debian/dkms/-/blob/debian/3.0.6-4/dkms.in#L870

That's not a fallback: when running a Debian distro the * case will never be
executed. In current master tree the * case has been removed and a check has
been added after the case block to set a fallback path if the sign-file
executable wasn't found [1].

> [...]

Attached here the patch to dkms I'm currently using to make module signing work.

[1] https://github.com/dell/dkms/blob/master/dkms.in#L893

[Siddh Raman Pant]

On Fri, 28 Oct 2022 10:46:54 +0200 Maurizio Avogadro <mav...@gmail.com> wrote:
> Il 27/10/22 21:56, Siddh Raman Pant ha scritto:
> > I used libsecret (which is used as backend by the Python program named keyring),
> > with the keyring being stored as root (isn't visible outside root in seahorse).
> > Can you tell why that may be a bad idea?
>
> It can be a nice solution but I admit I totally ignore how this works and many
> questions arose on my mind: how is the keyring first created? Is the keyring
> password protected? How is the keyring password asked? Does this work in a
> su-ed/sudo-ed text only console? What happens when issuing this command in a
> root console, and what would be the output:
>
> # keyring get uefi mok
>
> When talking of Gnome/KDE keyrings my head goes to X11/Wayland, and I prefer
> not to run graphical applications as root to avoid making the whole process
> more complex and therefore potentially more prone to security issues.
>

Yeah now I see, it's not a good one for headless environment. OOTB it currently
shows a password prompt in GUI for unlocking the keyring (it is protected).

You can have a look at the following for headless behaviour:
https://github.com/jaraco/keyring/#using-keyring-on-headless-linux-systems

Anyways, keyring here is immaterial. You could `read -s` the MOK key passphrase.
It was just a convenience.

> > [...]
> > I won't really want to store password in plaintext...
>
> Agreed, but remember that current dkms is designed to create new MOK key
> _without_ a password, it doesn't currently even support using a password
> protected MOK key: this is even less secure.
> The whole secure boot support is still looking to get a stable shape and I
> wouldn't deploy it in a production environment yet; in the meanwhile, while
> experimenting it, I can live with a root only readable file containing the
> clear-text password.

I agree, it is not what ideally should have been there.

Regardless, if you want, you can store your password in the bash script.

>
> > [...]
> > What I pointed out was setting sign_file to a bash script. It would solve this
> > issue too, as the correct path to the actual sign_file binary could be used in
> > the bash script / handler. dkms would call the bash script with its arguments,
> > and the script will call the correct binary (passing the arguments to it), as
> > $kernelver variable would be available to the script.
>
> Yours is indeed a possible solution until the issues will be fixed on the dkms
> side IMO: I used a bash sign-file script too at first; but this is actually a
> workaround, and looking abstractly at the issue I think that if dkms is
> supposed to manage module signing, a fix have to be introduced there.

Agreed fully.

I just pointed out a way if someone is stuck. I was stuck too, and I personally was
also browsing the internet (esp. forums and mailing list like this one) in order to
fix my machine and get drivers like nvidia to be loaded.

Since I managed to get something up, even though hacky, I decided I should also post
it up on the places I was finding/browsing for a solution, so it may help others for
the time being before it is actually fixed.

It also serves as another data point that dkms currently doesn't have support for
KBUILD_SIGN_PIN.

> > Also, v3.0.6-4 seems to have the fallback you mentioned. Refer:
> > https://salsa.debian.org/debian/dkms/-/blob/debian/3.0.6-4/dkms.in#L870
>
> That's not a fallback: when running a Debian distro the * case will never be
> executed. In current master tree the * case has been removed and a check has
> been added after the case block to set a fallback path if the sign-file
> executable wasn't found [1].

Noted. Thanks for correcting me.

>
> > [...]
>
> Attached here the patch to dkms I'm currently using to make module signing work.

Thanks,
Siddh

[Holger Schröder]

Upstream has fixed this issue.


https://github.com/dell/dkms


...

[Holger Schröder]

fixed version available:

https://github.com/dell/dkms/releases/tag/v3.0.9


...