crosvm run fails on ARM64 guest with “vcpu hit unknown error: Bad address (os error 14)”

[Simone Montella]

Hi everyone, thanks in advance for your support.

I'm an embedded system student and i'm currently studying about pKVM.

I’m encountering a consistent issue when trying to launch a Crosvm guest inside an ARM64 virtual machine.
Every attempt results in the following error:

ERROR crosvm::crosvm::sys::linux::vcpu] vcpu hit unknown error: Bad address (os error 14)
INFO  crosvm::crosvm::sys::linux] vcpu crashed
ERROR crosvm::crosvm::sys::linux::vcpu] failed to send VcpuControl: sending on a closed channel

System setup

Physical host:

• Ubuntu 24.04 (x86_64, AMD CPU with KVM enabled)

• QEMU used to launch an ARM64 guest with a pKVM patched kernel

• KVM and nested virtualization enabled

First level VM (ARM64, Debian Bullseye):

• Running Crosvm built from source (latest main branch)

• /dev/kvm exists and kvm-ok reports KVM acceleration can be used (it is the patched pKVM)

• Goal: run a Crosvm VM inside this ARM64 guest 

Second level ARM64 VM to run with crosvm from the first level VM:

• Image created via debootstrap (Debian Bullseye arm64)

• Files available:

  • debian-bullseye-arm64.qcow2 (converted to raw → rootfs.raw)

  • initrd.img-5.10.0-26-arm64

  • vmlinuz-5.10.0-26-arm64

My attempts:

• sudo crosvm run \
    --block ./rootfs.raw \
    --initrd ./initrd.img-5.10.0-26-arm64 \
    ./vmlinuz-5.10.0-26-arm64 \
    -p "root=/dev/vda rw console=ttyS0" \
    --serial type=stdout \
    --disable-sandbox
• Tried to add ",root" to the block parameter, tried multiple root devices to attack (dev/vda1, sda, etc..)
• Tried different arm64 kernels and disks, built with debootstrap or downloaded from online repository/official sites/tools (like apt linux-image-arm64)
• Tried with and without --disable-sandbox
• Tried a minimal crosvm run with just the kernel, without initrd and disk, like mentioned in the Advanced Usage guide, trying to isolate the error 

Any of these tries always gave me the error
ERROR crosvm::crosvm::sys::linux::vcpu] vcpu hit unknown error: Bad address (os error 14) 

[Frederick Mayle]

I think it isn't possible to run the pKVM hypervisor inside a VM unfortunately.

Supposing it was possible, I think pKVM is disabled by default and you need something like "kvm-arm.mode=protected" in the kernel cmdline params of the first level VM. Then, when you launch crosvm, it will still launch a non-protected VM by default, so you need to provide the `--protected` flag to the `crosvm run` command.

CC @Android KVM 

--
You received this message because you are subscribed to the Google Groups "crosvm-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to crosvm-dev+...@chromium.org.
To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/crosvm-dev/7f044059-c292-4faa-b65e-d32f355683aen%40chromium.org.

[Simone Montella]

Hi @Frederick, thanks for your answer.
I'm noot sure at all about the uncompatibility of pKVM and virtual machines because i've heard (i don't want to be wrong about it) of my colleagues that already made it.

I also already ran the VM with the kvm-arm.mode=protected but i forgot to mention it into my first post, btw pKVM seems enabled since i can read it from the boot logs ( Protected nVHE mode initialized successfully and other similar messages )

Maybe from the KVM team we could get some other infos about pKVM but i thought that the problem was with crosvm (the Bad Address/os error 14) 

[Quentin Perret]

Hi Simone,

Which kernel version are you running on your aarch64 "host" (that is,
the emulated arm64 debian, the first level VM) ?

We've fixed bugs recently that could lead to getting -EFAULT back from
VCPU_RUN, so I'm wondering if you could be hitting that. All the
supported pKVM kernels (with the aforementioned bugs fixed) are in
Android Common Kernel: https://android.googlesource.com/kernel/common/

The android16-6.12 branch is the latest, so I'd suggest trying that.
We also have a port on a more recent kernel, but it's experimental.

Thanks,
Quentin

On Tue, Oct 7, 2025 at 11:11 AM Simone Montella
<simonem...@gmail.com> wrote:
>
> Hi @Frederick, thanks for your answer.
> I'm noot sure at all about the uncompatibility of pKVM and virtual machines because i've heard (i don't want to be wrong about it) of my colleagues that already made it.
> I also already ran the VM with the kvm-arm.mode=protected but i forgot to mention it into my first post, btw pKVM seems enabled since i can read it from the boot logs ( Protected nVHE mode initialized successfully and other similar messages )
>
> Maybe from the KVM team we could get some other infos about pKVM but i thought that the problem was with crosvm (the Bad Address/os error 14)
>