Fixing a VM's botched grub install

[tetra...@danwin1210.me]

As a result of some issues in a standaloneVM (debian-9), I ended up
using the Xen hypervisor serial console to migrade to debian-9-testing --
part way.

Sadly the hypervisor console does NOT support curses-style modal dialog
boxes, and the process of updating GRUB involves navigating one of
these. I did my best, but I do not think it worked.

I have included a copy of the console output at the end of this message
for reference.

Now when I do `sudo file -s /dev/xvda[1..3]` I do not see any grub
installation, and running `grub-emu` also suggests that the system will
not recover from being rebooted: it is all "error: no such device" and
"error: can't find command `linux`" and "error: can't find command
`initrd`"...

Unfortunately the Qubes documentation does not really describe what kind
of bootloader Qubes expects to find on VM images, nor where it expects
to find them.

Any suggestions?



Console output:

.[1;23r.[4l.)0.[m.(B.[1;23r.[H.[J.[1;1HPackage configuration.[6;20H.[1K ┌───────┤ Configuring grub-pc ├───────┐.[7;20H.[1K │ GRUB install devices:.[15C│.[8;20H.[1K │.[37C│.[9;20H.[1K │ [.[7m .[m.(B] /dev/xvda (21223 MB; ???) │.[10;20H.[1K │ [ ] /dev/xvdb (4349 MB; ???).[5C│.[11;20H.[1K │ [ ] /dev/xvdc (10737 MB; ???) │.[12;20H.[1K │ [ ] /dev/xvdd (524 MB; ???).[6C│.[13;20H.[1K │ [ ] /dev/xvda3 (21010 MB; ???) │.[14;20H.[1K │.[37C│.[15;20H.[1K │.[37C│.[16;20H.[1K │.[15C<Ok>.[18C│.[17;20H.[1K │.[37C│.[18;20H.[1K └─────────────────────────────────────┘.[9;30H.[9;27H .[10;27H.[7m .[10;30H.[10;27H.[m.(B .[11;27H.[7m .[11;30H.[11;27H.[m.(B .[12;27H.[7m .[12;30H.[12;27H.[m.(B .[13;27H.[7m .[13;3.[1;23r.[4l.)0.[m.(B.[1;23r.[H.[J.[1;1HPackage configuration.[3;2H┌──────────────────────────┤ Configuring grub-pc ├──────────────────────────┐.[4;2H│.[75C│.[5;2H│ You chose not to install GRUB to any devices. If you continue, the boot │.[6;2H│ loader may not be properly configured, and when this computer next.[8C│.[7;2H│ starts up it will use whatever was previously in the boot sector. If.[6C│.[8;2H│ there is an earlier version of GRUB 2 in the boot sector, it may be.[7C│.[9;2H│ unable to load modules or handle the current configuration file..[10C│.[10;2H│.[75C│.[11;2H│ If you are already using a different boot loader and want to carry on.[5C│.[12;2H│ doing so, or if this is a special environment where you do not need a.[5C│.[13;2H│ boot loader, then you should continue anyway. Otherwise, you should.[7C│.[14;2H│ install GRUB somewhere..[51C│.[15;2H│.[75C│.[16;2H│ Continue without installing GRUB?.[41C│.[17;2H│.[75C│.[18;2H│.[20C<Yes>.[23C.[7m<No>.[23C.[m.(B│.[19;2H│.[75C│.[20;2H└───────────────────────────────────────────────────────────────────────────┘.[18;52H.[18;23H.[7m<Yes>.[23C.[m.(B<No>.[23C.[18;24<Yes>.[23C.[7m<No>.[23C.[18;52H.[18;23H<Yes>.[23C.[m.(B<No>.[23C.[18;24H.[2Generating grub configuration file ...

[awokd]

tetrahedra via qubes-users:

> As a result of some issues in a standaloneVM (debian-9), I ended up
> using the Xen hypervisor serial console to migrade to debian-9-testing --
> part way.

> Now when I do `sudo file -s /dev/xvda[1..3]` I do not see any grub
> installation, and running `grub-emu` also suggests that the system will
> not recover from being rebooted: it is all "error: no such device" and
> "error: can't find command `linux`" and "error: can't find command
> `initrd`"...

In Qube Settings, is the kernel set to None? If not, it boots from a
kernel provided by Qubes anyways so you don't need grub in that case. It
might be a good idea to qvm-copy out anything you need while you still
can, in case you can't get it to boot up again. Also try "sudo
grub-install /dev/xvda" if your kernel is set to none.

--
- don't top post
Mailing list etiquette:
- trim quoted reply to only relevant portions
- when possible, copy and paste text instead of screenshots

[tetra...@danwin1210.me]

On Thu, Oct 17, 2019 at 04:58:00AM +0000, 'awokd' via qubes-users wrote:
>In Qube Settings, is the kernel set to None? If not, it boots from a
>kernel provided by Qubes anyways so you don't need grub in that case. It
>might be a good idea to qvm-copy out anything you need while you still
>can, in case you can't get it to boot up again. Also try "sudo
>grub-install /dev/xvda" if your kernel is set to none.

No, kernel is set to the default value. Looks like I'm safe then!
Backups have already been made :)

Thanks!

[Jin-oh Kang]

This is what I see from your output:

https://asciinema.org/a/2sMvgiISVELkjTxAjDlfoNP5Z

[Jin-oh Kang]

The escape sequence crippling is caused by https://github.com/QubesOS/qubes-vmm-xen/blob/xen-4.8/patch-tools-xenconsole-replace-ESC-char-on-xenconsole-outp.patch , which is reasonable given the Qubes security model.

For interactive console you could use `qvm-console-dispvm <vmname>` provided by the qubes-core-admin package.

[tetra...@danwin1210.me]

On Thu, Oct 17, 2019 at 01:06:24PM -0700, Jin-oh Kang wrote:
>This is what I see from your output:
>
>https://asciinema.org/a/2sMvgiISVELkjTxAjDlfoNP5Z

That's really cool!

[tetra...@danwin1210.me]

On Thu, Oct 17, 2019 at 01:24:00PM -0700, Jin-oh Kang wrote:
>The escape sequence crippling is caused by https://github.com/QubesOS/qubes-vmm-xen/blob/xen-4.8/patch-tools-xenconsole-replace-ESC-char-on-xenconsole-outp.patch , which is reasonable given the Qubes security model.
>
>For interactive console you could use `qvm-console-dispvm <vmname>` provided by the qubes-core-admin package.

I wish I had seen that tool earlier! Here's a PR for the community docs:

https://github.com/Qubes-Community/Contents/pull/74

[awokd]

tetrahedra via qubes-users:

Likewise; that is new to me too.

[Jin-oh Kang]

Thanks for your contribution!

That said, I think I might have caused some misunderstandings here. The reason I used asciinema.org was just to animate what the console output originally was, and the service by itself doesn't have the ability to automatically substitute escape sequences or whatsoever. What I actually did was just substitute all dots followed by '[', ']', '(' or ')' to the ASCII ESC (0x1b, ' ') character, a method that could have false positives. Since the patch I mentioned also adds the '-r' flag which reverts its escape filtering feature, you can try it out with `xl console -r <Domain ID>` (although highly not recommended).

I'll submit the correction PR when I get to have some free time for it. I apologize for not having made this clear. Meanwhile if you'd like to do it instead -- since you're also the author of the article -- then I'd appreciate it ;)

[awokd]

Jin-oh Kang:

> I'll submit the correction PR when I get to have some free time for it. I
> apologize for not having made this clear. Meanwhile if you'd like to do it
> instead -- since you're also the author of the article -- then I'd
> appreciate it ;)

I got it (I think).

[tetra...@danwin1210.me]

On Sun, Oct 20, 2019 at 01:28:29PM +0900, Jin-oh Kang wrote:
>I'll submit the correction PR when I get to have some free time for it. I
>apologize for not having made this clear. Meanwhile if you'd like to do it
>instead -- since you're also the author of the article -- then I'd
>appreciate it ;)

awokd beat us to it. But please do submit a more in-depth explanation
when you get a moment!