Yubikey in challenge/response mode to unlock LUKS on boot

[the2nd]

Hi,

i switched to Qubes OS 3.2 on my notebook some weeks ago. Besides some issues i had it works very well.

One problem was to get the installer to install qubes on LVM-on-LUKS. I preferred this over the default LUKS-on-LVM setup because you dont have to encrypt any LV separately.

After fiddling around some other issues i wanted to use my yubikey to unlock the luks partition on boot like i did it before with my ubuntu installation (https://github.com/cornelinux/yubikey-luks).

After trying this:
https://github.com/bpereto/ykfde/blob/master/README-dracut.md

Which did not work and besides this does manage some IMHO useless (someone may correct me if i am wrong) extra challenges within the initramfs.

And reading this:
https://groups.google.com/forum/#!searchin/qubes-users/yubikey$20luks%7Csort:relevance/qubes-users/7pIS_grFZ4s/AlCoPuf-BwAJ

and this:
https://github.com/QubesOS/qubes-issues/issues/2712

I came to the conclusion that there is no working solution yet. So i tried to write my own dracut module. The main problem with this was to find the best hook in the boot process to send the user password to the yubikey and unlock the luks partition. After some testing i got a version which works for my purposes.

You can find the module and some install instructions at: https://github.com/the2nd/ykluks

Please note that the current version will probably not work with a default qubes LUKS-on-LVM installation. But if some experienced user is willing to help testing i'll try to come up with a version that supports this too.

Besides the yubikey/luks stuff the module handles the rd.qubes.hide_all_usb stuff via its own rd.ykluks.hide_all_usb command line parameter because the yubikey is connected via USB and needs to be accessable until we got the challenge from it. i am still unsure if this is the best method to implement this. So if anyone with a deeper knowledge of qubes/dracut does have a better/more secure solution i happy about any help.

Regards
the2nd

[joev...@gmail.com]

This is working great for me.
A few questions though:

1) The default Qubes 3.2 install seems to be LVM-on-LUKS where there is only one LUKS encryption and root/swap LVMs within that. So your instructions work with the default install.

2) It is not clear what can be done if you forget your Yubikey one day and want to use the really strong LUKS passphrase from another slot.
Is "Something went wrong" section in which you specify an older initramfs, the only way? Do I need to periodically update this backup "org" initramfs? And it doesn't mention anything about uncommentting the commented crypttab entry from the install instructions?

3) It does seem to hang after timing out. It will accept the password, but will not continue booting. I can't turn the system on, and come back later to use the yubikey. It seems like it is set to timeout in a minute or so.

Thank you.

[Ron Hunter-Duvar]

On 10/02/2017 08:34 PM, joev...@gmail.com wrote:
> On Saturday, 5 August 2017 11:20:27 UTC-4, the2nd wrote:
>> Hi,
>>
>> i switched to Qubes OS 3.2 on my notebook some weeks ago. Besides some issues i had it works very well.
>>
>> One problem was to get the installer to install qubes on LVM-on-LUKS. I preferred this over the default LUKS-on-LVM setup because you dont have to encrypt any LV separately.

>> ...

>> Please note that the current version will probably not work with a default qubes LUKS-on-LVM installation. But if some experienced user is willing to help testing i'll try to come up with a version that supports this too.
>>
>> Besides the yubikey/luks stuff the module handles the rd.qubes.hide_all_usb stuff via its own rd.ykluks.hide_all_usb command line parameter because the yubikey is connected via USB and needs to be accessable until we got the challenge from it. i am still unsure if this is the best method to implement this. So if anyone with a deeper knowledge of qubes/dracut does have a better/more secure solution i happy about any help.
>>
>> Regards
>> the2nd
> This is working great for me.
> A few questions though:
>
> 1) The default Qubes 3.2 install seems to be LVM-on-LUKS where there is only one LUKS encryption and root/swap LVMs within that. So your instructions work with the default install.
>

> ...
I'd have to say that the2nd is right. I didn't notice on my first Qubes
3.2 install, because I only had one encrypted partition on my OS drive
(skipped a swap partition, despite the installer's whining). Second time
around I gave in and created one.

lsblk shows sda2 with a luks-encrypted / within it, and sda3 with a
luks-encrypted swap. If it were LVM-on-LUKS, it would be a single
luks-encrypted partition two logical volumes within it.

Ron

PS: I'm a Qubes-noob, but long-time Linux user.

[__ __]

Hello,

sorry for the long delay. Didnt had time to answer.

If some of you is willing to help with testing LUKS-on-LVM could you please provide the output of the commands below?

sudo su -
. /usr/lib/dracut/modules.d/99base/dracut-lib.sh
getarg rd.ykluks.uuid

If you have not modified your grub config for the ykluks dracut module yet use this getarg command:
getarg rd.luks.uuid

Regarding the other questions/problems.

2) If you want to unlock the luks device without yubikey you can use the steps from the "Something went wrong :(" section, skipping step 4. This should disable the ykluks module and re-enable normal luks handling for one boot.

3) I do have two notebooks with Qubes 3.2 and yubikey for luks unlock Both do a re-prompt on wrong password. Can you please describe in detail what steps could be used to reproduce?

Thanks

the2nd

--
You received this message because you are subscribed to a topic in the Google Groups "qubes-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/qubes-users/hB0XaquzBAg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to qubes-users+unsubscribe@googlegroups.com.
To post to this group, send email to qubes...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-users/814cee70-0b5c-12a4-ee3e-bdb1f5479f3e%40shaw.ca.

For more options, visit https://groups.google.com/d/optout.

[Ron Qubed]

On Sunday, October 22, 2017 at 6:56:55 AM UTC-6, the2nd wrote:
> Hello,
>
> sorry for the long delay. Didnt had time to answer.
>
> If some of you is willing to help with testing LUKS-on-LVM could you please provide the output of the commands below?
>
> sudo su -
> . /usr/lib/dracut/modules.d/99base/dracut-lib.sh
> getarg rd.ykluks.uuid
>
> If you have not modified your grub config for the ykluks dracut module yet use this getarg command:
> getarg rd.luks.uuid

...
> Thanks
> the2nd

getarg rd.ykluks.uuid outputs nothing for me. But then, I'm not using a Yubikey.

getarg rd.luks.uuid outputs "luks-<uuid>", where lsblk shows that partition name to be a "crypt [SWAP]" on sda3 (sda1 being my /boot/efi/, and sda2 containing "crypt /".

Not sure if/how any of this helps, but there it is.

Ron

[__ __]

Is there only one line? Or one line per uuid? Can you provide the complete Output?

Regards

the2nd

--
You received this message because you are subscribed to a topic in the Google Groups "qubes-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/qubes-users/hB0XaquzBAg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to qubes-users+unsubscribe@googlegroups.com.
To post to this group, send email to qubes...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-users/3fe76359-4792-4177-b6a6-014426c8024b%40googlegroups.com.

[joev...@gmail.com]

On Monday, 23 October 2017 23:42:56 UTC-4, the2nd wrote:
> Is there only one line? Or one line per uuid? Can you provide the complete Output?
>
>
> Regards
> the2nd
>
>
>
> Am 24.10.2017 5:31 vorm. schrieb "Ron Qubed" <ronq...@gmail.com>:
>
> On Sunday, October 22, 2017 at 6:56:55 AM UTC-6, the2nd wrote:
>
> > Hello,
>
> >
>
> > sorry for the long delay. Didnt had time to answer.
>
> >
>
> > If some of you is willing to help with testing LUKS-on-LVM could you please provide the output of the commands below?
>
> >
>
> > sudo su -
>
> > . /usr/lib/dracut/modules.d/99base/dracut-lib.sh
>
> > getarg rd.ykluks.uuid
>
> >
>
> > If you have not modified your grub config for the ykluks dracut module yet use this getarg command:
>
> > getarg rd.luks.uuid
>
> ...
>
> > Thanks
>
> > the2nd
>
>
>
> getarg rd.ykluks.uuid outputs nothing for me. But then, I'm not using a Yubikey.
>
>
>
> getarg rd.luks.uuid outputs "luks-<uuid>", where lsblk shows that partition name to be a "crypt [SWAP]" on sda3 (sda1 being my /boot/efi/, and sda2 containing "crypt /".
>
>
>
> Not sure if/how any of this helps, but there it is.
>
>
>
> Ron
>
>
>
>
>
>
> --
>
> You received this message because you are subscribed to a topic in the Google Groups "qubes-users" group.
>
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/qubes-users/hB0XaquzBAg/unsubscribe.
>

> To unsubscribe from this group and all its topics, send an email to qubes-users...@googlegroups.com.

>
> To post to this group, send email to qubes...@googlegroups.com.
>
> To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-users/3fe76359-4792-4177-b6a6-014426c8024b%40googlegroups.com.
>
>
> For more options, visit https://groups.google.com/d/optout.

for me,... it is a single line:
[root@dom0 ~]# getarg rd.ykluks.uuid
luks-96fcb441-0f4c-4856-bcb7-1c76ab31ad73

[root@dom0 ~]# lsblk
...
sdc 8:32 0 3.7T 0 disk
├─sdc2 8:34 0 500M 0 part /boot
├─sdc3 8:35 0 3.7T 0 part
│ └─luks-96fcb441-0f4c-4856-bcb7-1c76ab31ad73
│ 253:0 0 3.7T 0 crypt
│ ├─qubes_dom0-root 253:1 0 3.6T 0 lvm /
│ └─qubes_dom0-swap 253:2 0 7.7G 0 lvm [SWAP]
└─sdc1 8:33 0 1M 0 part
...

[joev...@gmail.com]

On Sunday, 22 October 2017 08:56:55 UTC-4, the2nd wrote:
> Regarding the other questions/problems.
>
> 2) If you want to unlock the luks device without yubikey you can use the steps from the "Something went wrong :(" section, skipping step 4. This should disable the ykluks module and re-enable normal luks handling for one boot.
>

Thanks.

> 3) I do have two notebooks with Qubes 3.2 and yubikey for luks unlock Both do a re-prompt on wrong password. Can you please describe in detail what steps could be used to reproduce?

I actually meant to write originally, that it is not a problem with wrong password. But rather a timeout if waiting for a while. Entering the password after a few minutes results in an error and I must reboot.

>
> Thanks
> the2nd
>
>
>
>
>
>
>
>
>
> On Tue, Oct 3, 2017 at 5:11 AM, Ron Hunter-Duvar <ro...@shaw.ca> wrote:
> On 10/02/2017 08:34 PM, joev...@gmail.com wrote:
>
>
> On Saturday, 5 August 2017 11:20:27 UTC-4, the2nd  wrote:
>
>
> Hi,
>
>
>
> i switched to Qubes OS 3.2 on my notebook some weeks ago. Besides some issues i had it works very well.
>
>
>
> One problem was to get the installer to install qubes on LVM-on-LUKS. I preferred this over the default LUKS-on-LVM setup because you dont have to encrypt any LV separately.
>
> ...
>
> Please note that the current version will probably not work with a default qubes LUKS-on-LVM installation. But if some experienced user is willing to help testing i'll try to come up with a version that supports this too.
>
>
>
> Besides the yubikey/luks stuff the module handles the rd.qubes.hide_all_usb stuff via its own rd.ykluks.hide_all_usb command line parameter because the yubikey is connected via USB and needs to be accessable until we got the challenge from it. i am still unsure if this is the best method to implement this. So if anyone with a deeper knowledge of qubes/dracut does have a better/more secure solution i happy about any help.
>
>
>
> Regards
>
> the2nd
>
>
> This is working great for me.
>
> A few questions though:
>
>
>
> 1)  The default Qubes 3.2 install seems to be LVM-on-LUKS where there is only one LUKS encryption and root/swap LVMs within that.  So your instructions work with the default install.
>
>
>
> ...
>
>
> I'd have to say that the2nd is right. I didn't notice on my first Qubes 3.2 install, because I only had one encrypted partition on my OS drive (skipped a swap partition, despite the installer's whining). Second time around I gave in and created one.
>
>
>
> lsblk shows sda2 with a luks-encrypted / within it, and sda3 with a luks-encrypted swap. If it were LVM-on-LUKS, it would be a single luks-encrypted partition two logical volumes within it.
>
>
>
> Ron
>
>
>
> PS: I'm a Qubes-noob, but long-time Linux user.
>
>
>
> --
>
> You received this message because you are subscribed to a topic in the Google Groups "qubes-users" group.
>
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/qubes-users/hB0XaquzBAg/unsubscribe.
>

> To unsubscribe from this group and all its topics, send an email to qubes-users...@googlegroups.com.

[joev...@gmail.com]

> Please note that the current version will probably not work with a default qubes LUKS-on-LVM installation. But if some experienced user is willing to help testing i'll try to come up with a version that supports this too.
>
> Besides the yubikey/luks stuff the module handles the rd.qubes.hide_all_usb stuff via its own rd.ykluks.hide_all_usb command line parameter because the yubikey is connected via USB and needs to be accessable until we got the challenge from it. i am still unsure if this is the best method to implement this. So if anyone with a deeper knowledge of qubes/dracut does have a better/more secure solution i happy about any help.
>
> Regards
> the2nd

So I've screwed up... when I filled up my LVM, I added a disk to the Volume Group and expanded the pool.

But I didn't encrypt the new drive, thinking I had LVM on LUKS. But I have this now.
[root@dom0]# lsblk | grep -v "\-\-"
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sdb 8:16 0 3.7T 0 disk
└─sdb1 8:17 0 3.7T 0 part
├─qubes_dom0-pool00_tmeta 253:1 0 2.1G 0 lvm
│ └─qubes_dom0-pool00-tpool 253:3 0 1T 0 lvm
│ ├─qubes_dom0-pool00 253:6 0 1T 0 lvm
│ ├─qubes_dom0-root 253:4 0 192.6G 0 lvm /
├─qubes_dom0-pool00_meta0 253:63 0 2.1G 0 lvm
└─qubes_dom0-pool00_tdata 253:2 0 1T 0 lvm
└─qubes_dom0-pool00-tpool 253:3 0 1T 0 lvm
├─qubes_dom0-pool00 253:6 0 1T 0 lvm
├─qubes_dom0-root 253:4 0 192.6G 0 lvm /
sr0 11:0 1 1024M 0 rom
loop0 7:0 0 500M 0 loop
sda 8:0 0 232.9G 0 disk
└─sda1 8:1 0 232.9G 0 part
nvme0n1 259:0 0 232.9G 0 disk
├─nvme0n1p1 259:1 0 1G 0 part /boot
└─nvme0n1p2 259:2 0 231.9G 0 part
└─luks-bfcca13a-213d-46ec-b156-53df348dba30 253:0 0 231.9G 0 crypt
├─qubes_dom0-pool00_tdata 253:2 0 1T 0 lvm
│ └─qubes_dom0-pool00-tpool 253:3 0 1T 0 lvm
│ ├─qubes_dom0-pool00 253:6 0 1T 0 lvm
│ ├─qubes_dom0-root 253:4 0 192.6G 0 lvm /
└─qubes_dom0-swap 253:5 0 23.3G 0 lvm [SWAP]


With this LVM on LUKS setup, extending the thin pool onto a new disk that was added to the volume group... winds up leaving plain text data on the new disk.


Here's what I think my setup will have to be:

nvme0n1 (2 drives in hw RAID 0)
├─nvme0n1p1 part /boot
└─nvme0n1p2 part
└─luks (same key) crypt
├─qubes_dom0-pool00_tmeta lvm
├─qubes_dom0-pool00_tdata lvm
│ └─qubes_dom0-pool00-tpool lvm
│ ├─qubes_dom0-pool00 lvm
│ ├─qubes_dom0-root lvm /
│ └─ ... vm lvm
└─qubes_dom0-swap lvm [SWAP]

sda (2 drives in hw RAID 0)
└─sda1 part
└─luks (same key) crypt
└─qubes_dom0-pool00_tdata lvm
└─qubes_dom0-pool00-tpool lvm
├─qubes_dom0-pool00 lvm
├─qubes_dom0-root lvm /
└─ ... vm lvm

With your ykluks dracut module:
> The default Qubes OS installation is a LVM-on-LUKS setup which will not work yet. Patches for LVM-on-LUKS are welcome as well as experienced testers because a dont have a LVM-on-LUKS installation to test with.

I will be a tester for this.

Thanks

[__ __]

Hi,

i've modified the module to support multiple LUKS devices (UUIDs). It works with my setup which has only one LUKS device but it should work with more than one.

You have to add the UUIDs of you luks devices separated by comma (e.g. rd.ykluks.uuid=UUD1,UUID2,UUID3).

Hope this works and happy to get any feedback.

Regards

the2nd

--
You received this message because you are subscribed to a topic in the Google Groups "qubes-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/qubes-users/hB0XaquzBAg/unsubscribe.

To unsubscribe from this group and all its topics, send an email to qubes-users+unsubscribe@googlegroups.com.

To post to this group, send email to qubes...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-users/08e39e6c-97f6-456b-b0c6-c09a86a8a856%40googlegroups.com.

[joev...@gmail.com]

Thanks. I'll try it.
What's the best to add the UUID? I assume edit the grub.cfg directly. But will kernel updates overwrite? Do I need to edit something else and run dracut -f?

[__ __]

Hmm, thats strange because it is working for me and it was working for you before.

I've updated the github version to explicit install cryptsetup.

Please let me know if this fixes the problem.....

On Wed, Aug 15, 2018 at 9:45 PM, Joeviocoe Gmail <joev...@gmail.com> wrote:

Thanks. Something messed up though.

I added a single comma, and the uuid for the new Crypt_luks... To that line in etc/default/grub.

I ran mkgrub and dracut -f as per the normal installation.

Got an error saying it could not find the device, then I realized the only recently made the updates to the GitHub.

Downloaded and installed the new git.  the changes seem pretty straightforward, and shouldn't cause a problem.

But now, I have an error from dracut-initqueue saying cryptsetup command not found on line 66 of ykluks.sh

Also, the yubikey prompt to insert, does not show up.  Just a blank screen until I insert the key, then it does prompt for the passphrase.

I reinstalled the old version I had, removed the second uuid from default grub, and reran the mkgrub & dracut -f... It is prompting me to insert the yubikey again, but I still have the error of command not found for cryptsetup.

I have two entries in etc/crypttab, for each uuid, but those are both commented out.

I don't know why dracut cannot find the command.

Now I have to use the full passphrase by removing the yk as shown in the recovery steps.

On Wed, Aug 15, 2018, 12:43 PM __ __ <hei...@gmail.com> wrote:

You can add it to the GRUB_CMDLINE_LINUX in /etc/default/grub

On Wed, Aug 15, 2018 at 6:38 PM, <joev...@gmail.com> wrote:

Thanks.  I'll try it.
What's the best to add the UUID?  I assume edit the grub.cfg directly.  But will kernel updates overwrite?  Do I need to edit something else and run dracut -f?

--
You received this message because you are subscribed to a topic in the Google Groups "qubes-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/qubes-users/hB0XaquzBAg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to qubes-users+unsubscribe@googlegroups.com.
To post to this group, send email to qubes...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-users/f09343a5-6ff7-4283-b8e2-d1df0e3a1b95%40googlegroups.com.

[__ __]

I've added a fallback option now that, if enabled, will prompt for the LUKS passphrase if no yubikey was found within the configured time.

You can also specify the yubikey slot to use in the config now. And i've improved the message sending functions.

Regards

the2nd

On Wed, Aug 15, 2018 at 10:29 PM, __ __ <hei...@gmail.com> wrote:

Good to know that it works now. 😊

Maybe i should add an option to make my module work without the yubikey in case the yubikey is lost or otherwise not available. This should not be hard to implement......

Joeviocoe Gmail <joev...@gmail.com> schrieb am Mi., 15. Aug. 2018, 22:23:

Yep, that's simply fixed it. It is strange that it needs to be explicit now when it had not before.

also, I see that you are using the same display message function for everything.  1 second sleep time before it hides was too short, so I changed to 60 seconds.

Thank you for updating you are awesome module. Both Luks volumes open at boot time now, so I can try extending my LVM to the new drive without leaving the data unencrypted.

In case of recovery, I'm not sure how easy it will be to stop using your module with two encrypted volumes that need to be unlocked before the lvm.  I don't think the native rd.luks.uuid Will allow comma separated values.

I will let you know how well it boots after I extend the lvm to the new drive so you can update your documentation regarding LUKS on LVM.

Thanks again.

[joev...@gmail.com]

I love the new options. It works great to open 3 luks volumes on boot now. 2 of which have an LVM volume group for qubes, the 3rd just an extra ext4 volume.

Two questions:
1) Can you execute multiple cryptsetup commands at the same time? It has to wait a few seconds for each one in sequence, which lengthens the overall boot time. Or would there be a problem if the script exits before all required luks volumes are open? Maybe run cryptsetup commands with &, then finish by checking if all commands are complete.

2) I would like a stealth mode where the default prompt is for the luks passphrase, just like it would be without your module. In the background, looking for the yubikey. When found, change the prompt to ask for the yubikey password. But then systemd-ask-password would need to be something that can be cancelled/replaced by script, is that possible? The other option would be to not change the prompt at all, and just run the ykchalresp command if the yubikey is detected, and skip it if not.

Let me know what you think.
And thank you again for the hard work.

[joev...@gmail.com]

Something unrelated completely corrupted my system. dom0 got hosed and I was not able to recover. So I have reinstalled qubes from scratch, but this time I am using a software raid on 2 nvme pcie drives.

Qubes 4 set up does allow for an encrypted raid the graphical setup. It does not create an lvm. I am using a separate drive with luks and an lvm thin pool.

So now I have 3 luks partitions opened on boot. / (root), swap, and secondary drive that isn't important to the OS.

The way grub is set up by default now, is to have multiple "rd.luks.uuid=" parameters, one for each. Also, after each luks parameter, if one of the raid volumes, there is a "rd.md.uuid=" parameter.
This works using a single luks passphrase at boot time.

Command line: placeholder root=UUID=9f9879f9-b275-4313-abef-1d99ecff7810 ro rd.luks.uuid=luks-4a69493c-62a7-4c2b-8f4b-a90133d925f5 rd.luks.uuid=luks-d4d18b89-907e-47a2-bdc1-7da5096fc437 rd.luks.uuid=luks-1dfee293-9d48-470b-8b53-d10ad9b13b0b rd.md.uuid=2d63c5de:209df367:6cc0fc7e:e96b1484 rd.md.uuid=0a9b3000:21ca14f0:eea9dcd4:0fa1b693 i915.alpha_support=1 rhgb quiet rd.ykluks.hide_all_usb

So now I am thinking about your setup instructions, in this scenario.

From what I've tested, multiple "rd.ykluks.uuid" and entries on the grub line, tries to invoke multiple instances, and boot fails. I then tried a single rd.ykluks.uuid parameter with the comma separated uuids. And keep the existing "rd.md.uuid" parameters after that.

It doesn't work. I just get a blinking cursor, no prompts or messages.
I've tried removing the "luks-" prefix on the UUIDs, but still fails.

If I remove the "rd.md.uuid" parameters... I do get prompted for yubikey password and it does begin to decrypt the volumes as expected. But without the raid mounting "md" parameters... it doesn't boot from there.

My experience with dracut modules is very limited, but I want to test this RAID use case so your module is more robust. What should I try next?

Thanks.

[simonda...@googlemail.com]

Is this module working on Qubes 4.0?

[Joe]

On Wednesday, 26 September 2018 03:28:21 UTC-4, simonda...@googlemail.com wrote:
> Is this module working on Qubes 4.0?

Yes, it is working for me on Qubes 4.0 and I have used it with LVM and Raid configurations.