Recovering TemplateVM from boot failure caused by incorrect /etc/fstab

[Fabio]

I made an accidental edit to /etc/fstab which is causing my TemplateVM to timeout on boot with error Cannot connect to qrexec agent for 60 seconds....

Is there a way to edit a particular VM's etc/fstab without booting it up? Can I perhaps mount it's root filesystem?

Attempted solutions

1. I've tried adding init=/bin/bash to the kernel opts, starting the templateVM and then running sudo xl console TEMPLATE_NAME from Dom0 but I did not reach a command-prompt before fstab was read and executed.

2. I tried mounting the root filesystem for the TemplateVM from Dom0 with: sudo mount /dev/qubes_dom0/vm-fedora-26-root mnt but got error wrong fs type, bad option, bad superblock...

Any suggestions on how I could edit /etc/fstab for my TemplateVM before it gets executed?

Steps to reproduce the behavior:

1. Edit your /etc/fstab to be incorrectly formatted
   
2. Run qvm-start YOUR_FAULTY_TEMPLATE
   

Related issue: https://unix.stackexchange.com/questions/44027/how-to-fix-boot-failure-due-to-incorrect-fstab

[Ivan Mitev]

On 11/26/18 1:46 PM, 'Fabio' via qubes-users wrote:
> I made an accidental edit to /etc/fstab which is causing my TemplateVM to timeout on boot with error Cannot connect to qrexec agent for 60 seconds....
>
> Is there a way to edit a particular VM's etc/fstab without booting it up? Can I perhaps mount it's root filesystem?

for R4:

https://groups.google.com/forum/#!searchin/qubes-users/losetup|sort:date/qubes-users/rWRNIt0NOc4/VaFHp42rBgAJ

Alternatively the following command will configure a loopback device
automatically:

qvm-start --hdisk dom0:/dev/qubes_dom0/vm-...-root vmname


R3.2 uses files by default instead of LVM volumes, I can't remember
where they were located (/var/lib/qubes/ or something like that) but the
procedure should be more or less the same.

> Attempted solutions
>
> 1. I've tried adding init=/bin/bash to the kernel opts, starting the templateVM and then running sudo xl console TEMPLATE_NAME from Dom0 but I did not reach a command-prompt before fstab was read and executed.
>
> 2. I tried mounting the root filesystem for the TemplateVM from Dom0 with: sudo mount /dev/qubes_dom0/vm-fedora-26-root mnt but got error wrong fs type, bad option, bad superblock...

It's not recommended to access VMs' files in dom0. Once you set up a
loopback device and the volume pops up in the device tray, attach it to
a trusted VM and fix fstab there.

[a...@it-minds.dk]

Hello!

Please this issue: https://groups.google.com/forum/#!topic/qubes-users/rWRNIt0NOc4

Hope this helps :)

[Fabio]

Thanks for the reply Ivan.

I was able to mount `dom0:/dev/qubes_dom0/vm-...-private vmname` successfully, however when trying to mount `dom0:/dev/qubes_dom0/vm-...-root vmname`, I still get the `wrong fs type, bad option, bad superblock...` error.

I can confirm that I get this same error message when trying to mount any `--root` suffixed volume nested under `/dev/qubes_dom0`.

To confirm, I am running Qubes v4.0.

Any reason why the `--root` volumes in particular aren't mounting? Those are where `/etc/fstab` would be located, right?

Thanks again for your help!


Fabio


‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐

> ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> You received this message because you are subscribed to the Google Groups "qubes-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to qubes-users...@googlegroups.com.
> To post to this group, send email to qubes...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-users/a303a7f7-6b8f-5ff3-7e46-13154b8edda4%40maa.bz.
> For more options, visit https://groups.google.com/d/optout.

[Ivan Mitev]

Hi Fabio,

On 11/27/18 2:33 AM, 'Fabio' via qubes-users wrote:
> Thanks for the reply Ivan.
>
> I was able to mount `dom0:/dev/qubes_dom0/vm-...-private vmname` successfully, however when trying to mount `dom0:/dev/qubes_dom0/vm-...-root vmname`, I still get the `wrong fs type, bad option, bad superblock...` error.

You're probably mounting the wrong partition or you are trying to mount
the whole disk.

The following steps work for me:

in dom0:

sudo losetup /dev/loop3 /dev/qubes_dom0/vm-fedora-26-medium-bkp-root

then, attach the device to some VM like you did. In that VM run `sudo
dmesg` and you should see a line like

[timestamp_here] xvdi: xvdi1 xvdi2 xvdi3

Note: it could also be xvdj, xvdk, ... if you have other devices
attached to the VM.

There are thus 3 partitions in the device/volume you attached. You can
see what they are with `sudo sfdisk -l /dev/xvdi`:

Device Start End Sectors Size Type
/dev/xvdi1 2048 411647 409600 200M EFI System
/dev/xvdi2 411648 415743 4096 2M BIOS boot
/dev/xvdi3 415744 20971486 20555743 9.8G Linux filesystem

Trying to mount the first two partitions will return a "bad superblock"
error because there isn't any recognised fs on those. Trying to mount
the whole device (/dev/xvdi) will fail for the same reason.

Mount the third partition:

sudo mount /dev/xvdi3 /mnt

And you should be able to fix fstab there:

ll /mnt/etc/fstab

-rw-r--r-- 1 root root 851 May 11 2018 /mnt/etc/fstab

Then, reverse the instructions: sudo unmount /mnt, detach the device in
the tray and remove the loopback device in dom0 (losetup -d /dev/loop3).

Hope this helps...

[Fabio]

Hi Ivan,

Thanks a lot for the clarification! That was indeed it, I was trying to mount the whole device.

I was able to fix the `/etc/fstab` and get my VM back up! (celebrations!)

Really appreciate all your help with this!


Fabio


‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐

> > > You received this message because you are subscribed to the Google Groups "qubes-users" group.
> > > To unsubscribe from this group and stop receiving emails from it, send an email to qubes-users...@googlegroups.com.
> > > To post to this group, send email to qubes...@googlegroups.com.
> > > To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-users/a303a7f7-6b8f-5ff3-7e46-13154b8edda4%40maa.bz.
> > > For more options, visit https://groups.google.com/d/optout.
>
> --
>
> You received this message because you are subscribed to the Google Groups "qubes-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to qubes-users...@googlegroups.com.
> To post to this group, send email to qubes...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-users/9770a421-70a9-6f8e-ce03-361cb8afdee9%40maa.bz.