IOMMU problem using QEMU/KVM on AMD hardware

[Claudio Scordino]

Dear all,

when running

sudo jailhouse enable configs/qemu-vm.cell

I get

JAILHOUSE_ENABLE: Invalid argument

The tracing console says

hypervisor/arch/x86/amd_iommu.c:329: returning error -EINVAL

meaning that the following condition fails:

327 /* Protect against accidental VT-d configs. */
328 if (!iommu->amd_bdf)

Therefore, apparently I have misconfigured some setting about AMD IOMMU.
However, looking at the documentation I have not found any AMD-specific setting.

Here is some further information about my configuration:

HOST:
- Linux distribution: Ubuntu 16.04.1 LTS
- Linux kernel: 4.4.0-36 x86_64
- Qemu version: 2.7.0

GUEST:
- Linux distribution: Linux Mint 18 Sarah
- Linux kernel: 4.4.0-36 x86_64
- Kernel boot param: memmap=66M$0x3b000000
- Run through the following command: qemu-system-x86_64 -machine q35 -m 1G -enable-kvm -smp 4 -cpu host,-kvm_pv_eoi,-kvm_steal_time,-kvm_asyncpf,-kvmclock,+x2apic -drive file="Mint.vdi",id=disk,if=none -device ide-hd,drive=disk -serial stdio -serial vc -s -net user,hostfwd=tcp::22222-:22 -net nic

Many thanks and best regards,

Claudio

[Jan Kiszka]

First of all, there is no IOMMU configured in your qemu command line.

But then the question is if you re running on an AMD host processor. The
qemu-vm.cell is pre-configured for an Intel VM with Intel IOMMU (emulation).

AMD is a different beast. Patches for IOMMU emulation are on their way
upstream but not yet complete for use with Jailhouse. Moreover, KVM has
some bug (or more) when in comes to running Jailhouse as a guest. Fixing
this is long on my to-do list, but there is too much right now.

Long story short: better try on an Intel host when you want it inside
KVM. Running Jailhouse natively on AMD should be fine again (no known
issues ATM).

Jan

--
Siemens AG, Corporate Technology, CT RDA ITP SES-DE
Corporate Competence Center Embedded Linux

[Valentine Sinitsyn]

Hi Claudio,

Most likely, this error means that a) you running on an AMD host, and b) your config has some IOMMUs defined, which are not AMD ones.

Commenting out iommus in the config may help, but as Jan pointed out, there could be some other issues with nested SVM virtualization.

Valentine

--
You received this message because you are subscribed to the Google Groups "Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jailhouse-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Claudio Scordino]

Dear Jan, Valentine,

I see. BTW, can you please tell me which specific Qemu argument I was missing ?

 

But then the question is if you re running on an AMD host processor. The
qemu-vm.cell is pre-configured for an Intel VM with Intel IOMMU (emulation).

AMD is a different beast. Patches for IOMMU emulation are on their way
upstream but not yet complete for use with Jailhouse. Moreover, KVM has
some bug (or more) when in comes to running Jailhouse as a guest. Fixing
this is long on my to-do list, but there is too much right now.

Long story short: better try on an Intel host when you want it inside
KVM. Running Jailhouse natively on AMD should be fine again (no known
issues ATM).

I understand. Therefore, I'm going to use Intel.

However, let me point out that the documentation at https://github.com/siemens/jailhouse is measleading as it says "The included system configuration qemu-vm.c can be used to run Jailhouse in QEMU/KVM virtual machine on x86 hosts (Intel and AMD are supported).".

Many thanks,

            Claudio 

[Jan Kiszka]

On 2016-09-19 08:45, Claudio Scordino wrote:
> Dear Jan, Valentine,
>
> 2016-09-16 19:57 GMT+02:00 Jan Kiszka <jan.k...@siemens.com

> <mailto:jan.k...@siemens.com>>:

-device intel-iommu - see README.

>
>
>
>
> But then the question is if you re running on an AMD host processor. The
> qemu-vm.cell is pre-configured for an Intel VM with Intel IOMMU
> (emulation).
>
> AMD is a different beast. Patches for IOMMU emulation are on their way
> upstream but not yet complete for use with Jailhouse. Moreover, KVM has
> some bug (or more) when in comes to running Jailhouse as a guest. Fixing
> this is long on my to-do list, but there is too much right now.
>
> Long story short: better try on an Intel host when you want it inside
> KVM. Running Jailhouse natively on AMD should be fine again (no known
> issues ATM).
>
>
> I understand. Therefore, I'm going to use Intel.
>
> However, let me point out that the documentation
> at https://github.com/siemens/jailhouse is measleading as it says "The

> included system configuration *qemu-vm.c* can be used to run Jailhouse
> in QEMU/KVM virtual machine on x86 hosts (*Intel and AMD are supported*).".

Yeah, indeed. We'll have to fix it, one way or the other.

[Valentine Sinitsyn]

Hi everyone,

I feel, we should at least split Qemu configs for Intel and AMD-based
hosts. It also makes sense to add some notes regrading issues related to
nested virtualization to the README.

Valentine

>
> Jan
>

[Jan Kiszka]

Agreed. Patches welcome.