Bhyve and Rocky Linux startup error

[Frank Leonhardt]

Hi All,

I'm not a fan of byhve or virtualisation in general, and not a fan of
Linux, but they've come together in my life and I'm trying to run Rocky
(the successor to CentOS) under FreeBSD.

When I kick it off I get the following:
---------------
BdsDxe: starting Boot0001 "UEFI BHYVE SATA DVD ROM BHYVE-DE5C-3F46-0368"
from PciRoot(0x0)/Pci(0x3,0x0)/Sata(0x0,0xFFFF,0x0)
!!!! X64 Exception Type - 06(#UD - Invalid Opcode)  CPU Apic ID - 0000000

< snip register dump>


FXSAVE_STATE - 000000003FF421D0
!!!! Find image based on IP(0x3D9E61F0) (No PDB)
(ImageBase=000000003D9D3000, EntryPoint=000000003D9D4000) !!!!

---------------

It remains running and I have to kill it on another terminal.

This is on FreeBSD 14.2-RELEASE, amd64 generic kernel.

I configure the environment with:

kldload nmdm
kldload vmm
ifconfig tap0 create
sysctl net.link.tap.up_on_open=1
ifconfig bridge0 create
ifconfig bridge0 name bge0bridge
ifconfig bge0bridge addm bge0 tap0
ifconfig bge0bridge up

bhyve -c 1 -m 1G -w -H \
-s 0,hostbridge \
-s 3,ahci-cd,Rocky-10.0-x86_64-dvd1.iso \
-s 4,virtio-blk,/home/fjl/rockyvm.hdd \
-s 5,virtio-net,tap0 \
-s 29,fbuf,tcp=0.0.0.0:5900,w=800,h=600,wait \
-s 30,xhci,tablet \
-s 31,lpc -l com1,stdio \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
rockyvm

All the files exist (rockyvm.hdd  is 16G of nulls) although I note the
uefi firmware is no longer maintained. I can connect to it using a VNC
client and watch it crash.

Any ideas? May be simple, may be complicated. I just don't know with bhyve.

Thanks, Frank.

[Frank Leonhardt]

Further - if I use a Debian 13 ISO and change nothing else, it seems to
work. I suspect the CD-ROM image being funny but don't know enough about
why. I tried more than one Rocky ISO file.

(Methinks I may have got the networking wrong, but that's another question).

Thanks, Frank.

[fatty.merc...@aceecat.org]

On Fri, Aug 15, 2025 at 02:23:11PM +0100, Frank Leonhardt wrote:

> Further - if I use a Debian 13 ISO and change nothing else, it seems
> to work. I suspect the CD-ROM image being funny but don't know
> enough about why. I tried more than one Rocky ISO file.

First, I know nothing about byhve.

One thing about Debian ISO images is that they're prepared with
`isohybrid`. Which means they're both a valid ISO image and a valid
MBR paritioned disk image. That's why you can flash them to thumb
drives and then boot from them.

Other distributions' ISOs may not be treated that way.

--
Ian

[void]

On Fri, 15 Aug 2025, at 13:16, Frank Leonhardt wrote:
(snip)

> -s 3,ahci-cd,Rocky-10.0-x86_64-dvd1.iso \

Just a thought - Rocky 10 linux has x86_64-v3
as a hard minimum requirement. What's your cpu and
what are its capabilities?

https://docs.rockylinux.org/gemstones/test_cpu_compat/
--

[Frank Leonhardt]

Interesting! And thanks also to Ian for tip about Debian media. I had
tried 9 on it too, with the same results, but not 8.

I've since tried running Rocky on Hyper-V and it bombs on that too, but
in a different way. As I'm actually looking at ways of running Docker
containers under FreeBSD I may stick with Debian for the time being, but
when I get time I'll try this. Pretty naff if they (and presumably Red
Hat) are releasing an installer that crashes and burns instead of
printing a "This CPU isn't supported, sorry" message and halting.

Regards, Frank.

[Oliver Peter]

On Fri, Aug 15, 2025 at 01:16:42PM +0100, Frank Leonhardt wrote:
> bhyve -c 1 -m 1G -w -H \

...
It's not FreeBSD, 1G isn't just enough:
https://docs.rockylinux.org/guides/minimum_hardware_requirements/#memory-ram

--
Oliver Peter oli...@gfuzz.de 0x456D688F

[Frank Leonhardt]

On 18/08/2025 16:18, Oliver Peter wrote:

On Fri, Aug 15, 2025 at 01:16:42PM +0100, Frank Leonhardt wrote:

bhyve -c 1 -m 1G -w -H \

...
It's not FreeBSD, 1G isn't just enough:
    https://docs.rockylinux.org/guides/minimum_hardware_requirements/#memory-ram

Good spot! Thanks.

"2 GB minimum (text-mode install)"

What the hell? That's systemd for you ;-)

I suspect this was someone being lazy and trying it on the smallest machine they had, and calling that the minimum. It will be interesting to find out when I can stop it faulting. I remember upgrading to 32Mb to run FreeBSD 4 and thinking it was getting out of hand. IIRC Unix Version 7 ran in 64K (although 128K was better).

But seriously, this is crashing out on the bootloader as far as I can tell so unless the bootloader is written in Python or something, that's probably not the reason for the crash. It also borks on Hyper-V and I was speaking to some penguin enthusiast today and reckons his company has had trouble installing newer releases of Rocky. I may have to burn a CD-ROM and see if I can find a server with an actual CD drive. I think I have a few left.

Regards, Frank.

[Liam Proven]

On 18/08/2025 6:25 pm, Frank Leonhardt wrote:
> I remember upgrading to 32Mb to run FreeBSD 4 and thinking it was
> getting out of hand.

32 *MB*, surely?

;-)

--
Liam Proven ~ lpr...@sitpub.com ~ https://www.theregister.com/
Linux and FOSS Reporter, the Register
Isle of Man tel: +44 7624 227612 ~ UK tel: +44 7939 087884 (*not* 24x7)
Czech tel: +420 702 829 053 (also Whatsapp/Telegram/Signal)