Encrypted containers & the Debian installer.

[Diagonal Arg]

On my first tries with the Debian installer, I am struggling with the limited resources for installing to encrypted disks. I am using the same technique I have used with Ubuntu, but failing at the last step:

I create my luks disk(s) before-hand, then run the installer. I find I have to anna-install cryptsetup-udeb, as there is no such choice in "Load Installer Modules". Dropping to a shell, opening the disk, and re-detecting hard drives allows me to carry out the installation (as long as there's a filesystem in the mapped device), but on reboot I'm at an initramfs without cryptsetup. So I use a debian-live to pivot into the system to create a crypttab. I find I also have to install cryptsetup. Then I run update-initramfs. Here is where I'm stuck. The new initramfs still does not include cryptsetup. Why is it not recognizing the crypttab?

I have tried other approaches eg, during installation doing adding an apt-install cryptsetup (after "Select and Install Software") and then editing crypttab, but to no avail.

/D

PS. I pivot like this, in case I'm missing something:

mount root & boot devices in /target
for f in dev dev/pts sys proc; do sudo mount -B /$f /target/$f; done
chroot /target

[David Wright]

On Tue 15 May 2018 at 23:05:10 (-0700), Diagonal Arg wrote:
> On my first tries with the Debian installer, I am struggling with the limited resources for installing to encrypted disks. I am using the same technique I have used with Ubuntu, but failing at the last step:
>
> I create my luks disk(s) before-hand, then run the installer. I find I have to anna-install cryptsetup-udeb, as there is no such choice in "Load Installer Modules". Dropping to a shell, opening the disk, and re-detecting hard drives allows me to carry out the installation (as long as there's a filesystem in the mapped device), but on reboot I'm at an initramfs without cryptsetup. So I use a debian-live to pivot into the system to create a crypttab. I find I also have to install cryptsetup. Then I run update-initramfs. Here is where I'm stuck. The new initramfs still does not include cryptsetup. Why is it not recognizing the crypttab?

Are you not seeing the last line in this screen excerpt?



┌────────────────────┤ [?] Load installer components from CD ├────────────────────┐
│ │
│ All components of the installer needed to complete the install will be loaded │
│ automatically and are not listed here. Some other (optional) installer │
│ components are shown below. They are probably not necessary, but may be │
│ interesting to some users. │
│ │
│ Note that if you select a component that requires others, those components │
│ will also be loaded. │
│ │
│ Installer components to load: │
│ │
│ [ ] choose-mirror: Choose mirror to install from (menu item) ↑ │
│ [ ] crypto-dm-modules-4.9.0-2-686-di: devicemapper crypto module ▮ │


Cheers,
David.

[Diagonal Arg]

---- On Wed, 16 May 2018 07:42:42 -0700 David Wright <deb...@lionunicorn.co.uk> wrote ----

David - thanks, but crypto-dm-modules does not include cryptsetup.

And, even when I anna-install it, it doesn't help with the other issues I mentioned above.

> Cheers,
> David.

/D

[David Wright]

On Thu 17 May 2018 at 22:24:01 (-0700), Diagonal Arg wrote:
>
>
>
> ---- On Wed, 16 May 2018 07:42:42 -0700 David Wright <deb...@lionunicorn.co.uk> wrote ----
> > On Tue 15 May 2018 at 23:05:10 (-0700), Diagonal Arg wrote:
> > > On my first tries with the Debian installer, I am struggling with the limited resources for installing to encrypted disks. I am using the same technique I have used with Ubuntu, but failing at the last step:
> > >
> > > I create my luks disk(s) before-hand, then run the installer. I find I have to anna-install cryptsetup-udeb, as there is no such choice in "Load Installer Modules". Dropping to a shell, opening the disk, and re-detecting hard drives allows me to carry out the installation (as long as there's a filesystem in the mapped device), but on reboot I'm at an initramfs without cryptsetup. So I use a debian-live to pivot into the system to create a crypttab. I find I also have to install cryptsetup. Then I run update-initramfs. Here is where I'm stuck. The new initramfs still does not include cryptsetup. Why is it not recognizing the crypttab?
> >

> > │ [ ] crypto-dm-modules-4.9.0-2-686-di: devicemapper crypto module ▮ │
>
> David - thanks, but crypto-dm-modules does not include cryptsetup.
>
> And, even when I anna-install it, it doesn't help with the other issues I mentioned above.

OK, well I haven't got time to check this out but here's my guess of
what's going on. I've never used anna-install to play tricks behind
the installer's back. If you take upon yourself the unlocking of
encrypted disks and then use the d-i to build a system, the d-i may
be unaware that there are any encrypted partitions in the installation.

I can also see from other people's comments elsewhere that you might
be within a hair's breadth of wiping your encrypted partition(s)
during this process. However, just to get the necessary software
installed by the d-i in the final product, a workaround to try
might be to create during installation an encrypted partition on,
say, a nonce stick for a nonce mountpoint.

As I say, I haven't tried it out. Risks are that using the d-i's
partitioner to encrypt the stick does something simultaneously to
the original partitions you're trying to preserve, and also the
/etc/crypttab that gets written will want the stick to be present
at first boot (before you rewrite it).

Sometime I will try this out on a scratch PC. Mine all have room
for two systems, so I can install A, encrypted in the usual manner
and then try installing on the B root partition, keeping the
shared encrypted /home partition. (I haven't used LVM so can't
see how that would interact with things.)

Cheers,
David.

[Diagonal Arg]

On 05/18/2018 08:35 AM, David Wright wrote:
> On Thu 17 May 2018 at 22:24:01 (-0700), Diagonal Arg wrote:
>> ---- On Wed, 16 May 2018 07:42:42 -0700 David Wright <deb...@lionunicorn.co.uk> wrote ----
>> > On Tue 15 May 2018 at 23:05:10 (-0700), Diagonal Arg wrote:
>> > >
>> > > On my first tries with the Debian installer, I am struggling with the limited resources for installing to encrypted disks. I am using the same technique I have used with Ubuntu, but failing at the last step:
>> > >
>> > > I create my luks disk(s) before-hand, then run the installer. I find I have to anna-install cryptsetup-udeb, as there is no such choice in "Load Installer Modules". Dropping to a shell, opening the disk, and re-detecting hard drives allows me to carry out the installation (as long as there's a filesystem in the mapped device), but on reboot I'm at an initramfs without cryptsetup. So I use a debian-live to pivot into the system to create a crypttab. I find I also have to install cryptsetup. Then I run update-initramfs. Here is where I'm stuck. The new initramfs still does not include cryptsetup. Why is it not recognizing the crypttab?
>> >
>> > │ [ ] crypto-dm-modules-4.9.0-2-686-di: devicemapper crypto module ▮ │
>>
>> David - thanks, but crypto-dm-modules does not include cryptsetup.
>>
>> And, even when I anna-install it, it doesn't help with the other issues I mentioned above.
>
> OK, well I haven't got time to check this out but here's my guess of
> what's going on. I've never used anna-install to play tricks behind
> the installer's back.

I understand, though it is odd that the installer installs
crypto-dm-modules, but waits to install cryptsetup until such time as
the partitioner is used to make an encrypted device. Why would that be?

> If you take upon yourself the unlocking of
> encrypted disks and then use the d-i to build a system, the d-i may
> be unaware that there are any encrypted partitions in the installation.

That is indeed the case with Ubuntu, so I assumed it would be the same
here. What is different is that after install, when I pivot into my new
system, fixing cryptab, installing cryptsetup, and updating initramfs
does not end up including cryptsetup in initramfs.

> I can also see from other people's comments elsewhere that you might
> be within a hair's breadth of wiping your encrypted partition(s)
> during this process.

Well, they certainly /seem/ to be fine. I am for example, able to do an
apt-get update && apt-get install after I pivot in.

> However, just to get the necessary software
> installed by the d-i in the final product, a workaround to try
> might be to create during installation an encrypted partition on,
> say, a nonce stick for a nonce mountpoint.

Ok, I'm not sure what a nonce stick/mountpoint is.

I'm sure this is unrelated, by I am putting /boot on a USB stick and /
on the luks container which takes the the whole hard drive.

> As I say, I haven't tried it out. Risks are that using the d-i's
> partitioner to encrypt the stick does something simultaneously to
> the original partitions you're trying to preserve, and also the
> /etc/crypttab that gets written will want the stick to be present
> at first boot (before you rewrite it).
>
> Sometime I will try this out on a scratch PC.

I have run a number of experiments in a VM. That seems to be the
easiest approach.

> Mine all have room
> for two systems, so I can install A, encrypted in the usual manner
> and then try installing on the B root partition, keeping the
> shared encrypted /home partition. (I haven't used LVM so can't
> see how that would interact with things.)

I'm not using LVM.

> Cheers,
> David.

Thanks,
/D

[Diagonal Arg]

On 05/18/2018 08:35 AM, David Wright wrote:

> On Thu 17 May 2018 at 22:24:01 (-0700), Diagonal Arg wrote:
>> ---- On Wed, 16 May 2018 07:42:42 -0700 David Wright <deb...@lionunicorn.co.uk> wrote ----
>> > On Tue 15 May 2018 at 23:05:10 (-0700), Diagonal Arg wrote:
>> > >
>> > > On my first tries with the Debian installer, I am struggling with the limited resources for installing to encrypted disks. I am using the same technique I have used with Ubuntu, but failing at the last step:
>> > >
>> > > I create my luks disk(s) before-hand, then run the installer. I find I have to anna-install cryptsetup-udeb, as there is no such choice in "Load Installer Modules". Dropping to a shell, opening the disk, and re-detecting hard drives allows me to carry out the installation (as long as there's a filesystem in the mapped device), but on reboot I'm at an initramfs without cryptsetup. So I use a debian-live to pivot into the system to create a crypttab. I find I also have to install cryptsetup. Then I run update-initramfs. Here is where I'm stuck. The new initramfs still does not include cryptsetup. Why is it not recognizing the crypttab?
>> >
>> > │ [ ] crypto-dm-modules-4.9.0-2-686-di: devicemapper crypto module ▮ │
>>
>> David - thanks, but crypto-dm-modules does not include cryptsetup.
>>
>> And, even when I anna-install it, it doesn't help with the other issues I mentioned above.
>
> OK, well I haven't got time to check this out but here's my guess of
> what's going on. I've never used anna-install to play tricks behind
> the installer's back.

I understand, though it is odd that the installer installs
crypto-dm-modules, but waits to install cryptsetup until such time as
the partitioner is used to make an encrypted device. Why would that be?

> If you take upon yourself the unlocking of
> encrypted disks and then use the d-i to build a system, the d-i may
> be unaware that there are any encrypted partitions in the installation.

That is indeed the case with Ubuntu, so I assumed it would be the same
here. What is different is that after install, when I pivot into my new
system, fixing cryptab, installing cryptsetup, and updating initramfs
does not end up including cryptsetup in initramfs.

> I can also see from other people's comments elsewhere that you might
> be within a hair's breadth of wiping your encrypted partition(s)
> during this process.

Well, they certainly /seem/ to be fine. I am for example, able to do an
apt-get update && apt-get install after I pivot in.

> However, just to get the necessary software
> installed by the d-i in the final product, a workaround to try
> might be to create during installation an encrypted partition on,
> say, a nonce stick for a nonce mountpoint.

Ok, I'm not sure what a nonce stick/mountpoint is.

I'm sure this is unrelated, by I am putting /boot on a USB stick and /
on the luks container which takes the the whole hard drive.

> As I say, I haven't tried it out. Risks are that using the d-i's
> partitioner to encrypt the stick does something simultaneously to
> the original partitions you're trying to preserve, and also the
> /etc/crypttab that gets written will want the stick to be present
> at first boot (before you rewrite it).
>
> Sometime I will try this out on a scratch PC.

I have run a number of experiments in a VM. That seems to be the
easiest approach.

> Mine all have room
> for two systems, so I can install A, encrypted in the usual manner
> and then try installing on the B root partition, keeping the
> shared encrypted /home partition. (I haven't used LVM so can't
> see how that would interact with things.)

[Dev Rana]

On 05/18/2018 08:35 AM, David Wright wrote:

> On Thu 17 May 2018 at 22:24:01 (-0700), Diagonal Arg wrote:
>> ---- On Wed, 16 May 2018 07:42:42 -0700 David Wright <deb...@lionunicorn.co.uk> wrote ----
>> > On Tue 15 May 2018 at 23:05:10 (-0700), Diagonal Arg wrote:
>> > >
>> > > On my first tries with the Debian installer, I am struggling with the limited resources for installing to encrypted disks. I am using the same technique I have used with Ubuntu, but failing at the last step:
>> > >
>> > > I create my luks disk(s) before-hand, then run the installer. I find I have to anna-install cryptsetup-udeb, as there is no such choice in "Load Installer Modules". Dropping to a shell, opening the disk, and re-detecting hard drives allows me to carry out the installation (as long as there's a filesystem in the mapped device), but on reboot I'm at an initramfs without cryptsetup. So I use a debian-live to pivot into the system to create a crypttab. I find I also have to install cryptsetup. Then I run update-initramfs. Here is where I'm stuck. The new initramfs still does not include cryptsetup. Why is it not recognizing the crypttab?
>> >
>> > │ [ ] crypto-dm-modules-4.9.0-2-686-di: devicemapper crypto module ▮ │
>>
>> David - thanks, but crypto-dm-modules does not include cryptsetup.
>>
>> And, even when I anna-install it, it doesn't help with the other issues I mentioned above.
>
> OK, well I haven't got time to check this out but here's my guess of
> what's going on. I've never used anna-install to play tricks behind
> the installer's back.

I understand, though it is odd that the installer installes

crypto-dm-modules, but waits to install cryptsetup until such time as
the partitioner is used to make an encrypted device. Why would that be?

> If you take upon yourself the unlocking of
> encrypted disks and then use the d-i to build a system, the d-i may
> be unaware that there are any encrypted partitions in the installation.

That is indeed the case with Ubuntu, so I assumed it would be the same
here. What is different is that after install, when I pivot into my new
system, fixing cryptab, installing cryptsetup, and updating initramfs
does not end up including cryptsetup in initramfs.

> I can also see from other people's comments elsewhere that you might
> be within a hair's breadth of wiping your encrypted partition(s)
> during this process.

Well, they certainly /seem/ to be fine. I am for example, able to do an
apt-get update && apt-get install after I pivot in.

> However, just to get the necessary software
> installed by the d-i in the final product, a workaround to try
> might be to create during installation an encrypted partition on,
> say, a nonce stick for a nonce mountpoint.

Ok, I'm not sure what a nonce stick/mountpoint is.

I'm sure this is unrelated, by I am putting /boot on a USB stick and /
on the luks container which takes the the whole hard drive.

> As I say, I haven't tried it out. Risks are that using the d-i's
> partitioner to encrypt the stick does something simultaneously to
> the original partitions you're trying to preserve, and also the
> /etc/crypttab that gets written will want the stick to be present
> at first boot (before you rewrite it).
>
> Sometime I will try this out on a scratch PC.

I have run a number of experiments in a VM. That seems to be the
easiest approach.

> Mine all have room
> for two systems, so I can install A, encrypted in the usual manner
> and then try installing on the B root partition, keeping the
> shared encrypted /home partition. (I haven't used LVM so can't
> see how that would interact with things.)

I'm not using LVM.

> Cheers,
> David.

/D

[21n...@gmail.com]

Hello,


In the file “/etc/cryptsetup-initramfs/conf-hook”, there is a line
“CRYPTSETUP” which is commented and/or has the default value “n”. If
this is the case, replace the line with “CRYPTSETUP=y”. So, the next use
of the command “update-initramfs” should solve your problem if I
understood it correctly.

I think “CRYPTSETUP=y” is automatically set if you create an encrypted
partition by following the installer’s instructions, but not when you do
it outside these instructions.

[David Wright]

On Tue 22 May 2018 at 00:32:14 (-0700), Diagonal Arg wrote:
> On 05/18/2018 08:35 AM, David Wright wrote:
> > On Thu 17 May 2018 at 22:24:01 (-0700), Diagonal Arg wrote:
> >> ---- On Wed, 16 May 2018 07:42:42 -0700 David Wright <deb...@lionunicorn.co.uk> wrote ----
> >> > On Tue 15 May 2018 at 23:05:10 (-0700), Diagonal Arg wrote:
> >> > >
> >> > > On my first tries with the Debian installer, I am struggling with the limited resources for installing to encrypted disks. I am using the same technique I have used with Ubuntu, but failing at the last step:
> >> > >
> >> > > I create my luks disk(s) before-hand, then run the installer. I find I have to anna-install cryptsetup-udeb, as there is no such choice in "Load Installer Modules". Dropping to a shell, opening the disk, and re-detecting hard drives allows me to carry out the installation (as long as there's a filesystem in the mapped device), but on reboot I'm at an initramfs without cryptsetup. So I use a debian-live to pivot into the system to create a crypttab. I find I also have to install cryptsetup. Then I run update-initramfs. Here is where I'm stuck. The new initramfs still does not include cryptsetup. Why is it not recognizing the crypttab?
> >> >
> >> > │ [ ] crypto-dm-modules-4.9.0-2-686-di: devicemapper crypto module ▮ │
> >>
> >> David - thanks, but crypto-dm-modules does not include cryptsetup.
> >>
> >> And, even when I anna-install it, it doesn't help with the other issues I mentioned above.
> >
> > OK, well I haven't got time to check this out but here's my guess of
> > what's going on. I've never used anna-install to play tricks behind
> > the installer's back.
>
> I understand, though it is odd that the installer installs
> crypto-dm-modules, but waits to install cryptsetup until such time as
> the partitioner is used to make an encrypted device. Why would that be?

AIUI if you use the installer as intended, then everything required
gets installed in time for it to be used at the appropriate times.
That said, it does appear that the d-i can't really handle installing
a system into an already encrypted partition. I don't know why this
is, whether it's on technical or policy grounds, or just an oversight.

> > If you take upon yourself the unlocking of
> > encrypted disks and then use the d-i to build a system, the d-i may
> > be unaware that there are any encrypted partitions in the installation.
>
> That is indeed the case with Ubuntu, so I assumed it would be the same
> here. What is different is that after install, when I pivot into my new
> system, fixing cryptab, installing cryptsetup, and updating initramfs
> does not end up including cryptsetup in initramfs.
>
> > I can also see from other people's comments elsewhere that you might
> > be within a hair's breadth of wiping your encrypted partition(s)
> > during this process.
>
> Well, they certainly /seem/ to be fine. I am for example, able to do an
> apt-get update && apt-get install after I pivot in.

I'm glad to hear that. I've avoided the problem on my PCs by
installing the second (B) system with no separate /home partition.
Then subsequently I scrap that directory's contents (IOW it was
a nonce /home) and it becomes just a mountpoint for the encrypted
/home partition which is being shared with the first (A) system.

> > However, just to get the necessary software
> > installed by the d-i in the final product, a workaround to try
> > might be to create during installation an encrypted partition on,
> > say, a nonce stick for a nonce mountpoint.
>
> Ok, I'm not sure what a nonce stick/mountpoint is.

A nonce is anything you create and use for a one-off occasion.
https://www.merriam-webster.com/dictionary/nonce

> I'm sure this is unrelated, by I am putting /boot on a USB stick and /
> on the luks container which takes the the whole hard drive.
>
> > As I say, I haven't tried it out. Risks are that using the d-i's
> > partitioner to encrypt the stick does something simultaneously to
> > the original partitions you're trying to preserve, and also the
> > /etc/crypttab that gets written will want the stick to be present
> > at first boot (before you rewrite it).
> >
> > Sometime I will try this out on a scratch PC.
>
> I have run a number of experiments in a VM. That seems to be the
> easiest approach.

You're ahead of me then. I've not played with VMs.

> > Mine all have room
> > for two systems, so I can install A, encrypted in the usual manner
> > and then try installing on the B root partition, keeping the
> > shared encrypted /home partition. (I haven't used LVM so can't
> > see how that would interact with things.)
>
> I'm not using LVM.

I think others will have to comment to progress things further.
My own interest in encryption is finding the best way to solve a
different problem:

PC boots (unattended) with a dummy /home that's really just a mount
point. (This presupposes they don't have cron jobs set up.) When the
first user logs in, either at a VC or by ssh, the real encrypted
/home is mounted after said user types the passphrase to unlock it.
(No need to relock it when users log out.)
I'm not there yet.

Cheers,
David.

[Diagonal Arg]

> [...]

Thank you! Cryptsetup is now in my initramfs. I am close, but there
still remains one issue. When updating the initramfs, I get:

cryptsetup: WARNING: root target luks.root uses a key file, skipped

But it doesn't use a key file. My cryptsetup is:
luks.root UUID=xxx pwd luks

And my fstab contains:
/dev/mapper/luks.root / btrfs defaults 0 0

When I boot the system, I drop into initramfs. There, I can open the
root disk myself and continue the boot process.

/D

[Diagonal Arg]

Oops! It's not "pwd" in the crypttab, it's "none"!

Works now. Yay, yay!! :)

Thanks so much.

/D

[21n...@gmail.com]

Sorry for the huge delay.

You’re welcome :)