Another issue with running JH on x86

[Yijun Zhu]

Hi Jan & all,

I have enable the root cell on a X86 machine successfully with old jailhouse code and kernel v4.1.3, but failed with the newest jailhouse version.

Hypervisor log as below:

...
Page pool usage after late setup: mem 395/16346, remap 65820/131072
FATAL: Invalid MMIO/RAM read, addr: 0x0000000080008100
RIP: 0xffffffff81512133 RSP: 0xffff880a56ec3be0 FLAGS: 10086
RAX: 0xffffc90020000000 RBX: 0xffffc90020008100 RCX: 0x0000000000000000
RDX: 0x00000000000000ff RSI: 0x0000000000000000 RDI: 0x0000000000000000
CS: 10 BASE: 0x0000000000000000 AR-BYTES: a09b EFER.LMA 1
CR0: 0x0000000080050033 CR3: 0x0000000001953000 CR4: 0x00000000000427a0
EFER: 0x0000000000000d01
Parking CPU 3 (Cell: "RootCell")

Address 0x0000000080008100 is not belong to any pci device in my config file. It's strange that root cell kernel access it.

Then I git bisect the code, the result as below:

e17d52525dfd0bfddaa33d4e9eeaf057483ff164 is the first bad commit
commit e17d52525dfd0bfddaa33d4e9eeaf057483ff164
Author: Jan Kiszka <jan.k...@siemens.com>
Date: Wed Aug 5 11:16:32 2015 +0200

core: pci: Migrate MMCONFIG access handling to the new MMIO dispatcher

Register the MMCONFIG memory region, if available, with the generic MMIO
dispatcher and drop the related handler invocation from
pci_mmio_access_handler.

Signed-off-by: Jan Kiszka <jan.k...@siemens.com>

:040000 040000 42e5a818e28163216e045ce0a7d627e785e522e5 896f8a65d9f9198519c50a629301d67005fa4c9c M hypervisor

Could anybody help to check this patch. Thanks!

Zhu Yijun

[Jan Kiszka]

On 2015-09-18 10:05, Yijun Zhu wrote:
> Hi Jan & all,
>
> I have enable the root cell on a X86 machine successfully with old jailhouse code and kernel v4.1.3, but failed with the newest jailhouse version.
>
> Hypervisor log as below:
>
> ...
> Page pool usage after late setup: mem 395/16346, remap 65820/131072
> FATAL: Invalid MMIO/RAM read, addr: 0x0000000080008100
> RIP: 0xffffffff81512133 RSP: 0xffff880a56ec3be0 FLAGS: 10086
> RAX: 0xffffc90020000000 RBX: 0xffffc90020008100 RCX: 0x0000000000000000
> RDX: 0x00000000000000ff RSI: 0x0000000000000000 RDI: 0x0000000000000000
> CS: 10 BASE: 0x0000000000000000 AR-BYTES: a09b EFER.LMA 1
> CR0: 0x0000000080050033 CR3: 0x0000000001953000 CR4: 0x00000000000427a0
> EFER: 0x0000000000000d01
> Parking CPU 3 (Cell: "RootCell")
>
> Address 0x0000000080008100 is not belong to any pci device in my config file. It's strange that root cell kernel access it.

If you check your /proc/iomem, I bet that region is MMCONFIG. Then the
last 12 bits are the config space address of the PCI device 00:01.0
(BDF). Probably it's an extended PCI capability that is accessed here
and that we do not allow by default (someone would have to analyse the
implications of the access). For a quick fix, you then need to model the
capability in the config, permitting the access (see existing configs
for examples).

>
> Then I git bisect the code, the result as below:
>
> e17d52525dfd0bfddaa33d4e9eeaf057483ff164 is the first bad commit
> commit e17d52525dfd0bfddaa33d4e9eeaf057483ff164
> Author: Jan Kiszka <jan.k...@siemens.com>
> Date: Wed Aug 5 11:16:32 2015 +0200
>
> core: pci: Migrate MMCONFIG access handling to the new MMIO dispatcher
>
> Register the MMCONFIG memory region, if available, with the generic MMIO
> dispatcher and drop the related handler invocation from
> pci_mmio_access_handler.
>
> Signed-off-by: Jan Kiszka <jan.k...@siemens.com>

Yeah, I suspect that change causes misreporting of rejected PCI config
space accesses. I will look into this (in fact, I ran into it recently
as well but was in a hurry and resolved the access problem).

Jan