[RFCv4 00/24] Jailhouse initial AArch64 support for ARMv8 processors

[Antonios Motakis]

This patch series is an RFC towards AArch64 support in the Jailhouse
hypervisor. RFCs versions 1-3 where internal, this RFCv4 is our first public
call for comments on this project :)

Some parts are still prototypical in nature, and I'm already fixing them in my
local copy, however feedback on the overall direction of the port is more than
welcome.

The patch series has a few distinct parts:

[A] Patches 01 - 05: missing includes from various files in Jailhouse.
- We bumped into those while hacking on the AArch64 port :)

[B] Patches 06 - 08: preparatory patches on Jailhouse and the existing
AArch32 port
- Patch 06 affects the Linux driver that loads the firmware: on the existing
targets we make the assumption that Jailhouse can be built to execute from
the same VA region that Linux will map it on. However this assumption is not
very friendly with AArch64, due to a lack of an TTBR1_EL2 register :)
- Patch 07 - 08 should be more obvious preparatory changes.

[C] Patches 09 - 16: headers and helpers for AArch64
- Here we just stick a number of defines and inline helpers that we wish to
use in the AArch64 target. In the case of structs, we attempt to define the
minimum we need to build a stub firmware binary.

[D] Patches 17 and 18: build a stub firmware, and load it on an AArch64 target
- We build a stub firmware that doesn't really do anything yet. Also include
a valid root cell config for the Foundation ARMv8 model, so we have a valid
AArch64 target to work with.

[E] Patches 19 - 24: initial implementation for the AArch64 port
- We start gradually adding the functionality needed to have a working
hypervisor binary on AArch64, starting with the EL2 entry code. Applying
each one of these patches in sequence gets you a little further in the
Hypervisor initialization, before it finally fails :)

We are already working on restoring the root cell, with the Stage 2 MMU
translation switched on, and proper GIC support.

Coming in RFCv5:
- Restore the root cell
- MMU fixes and hopefully GICv2 support
- Make the calculation of the entry code offset properly, instead of the
current hack
- Add warnings to all the stub functions that do not return an error. Make
sure we can always notice when a stub is being used while we work on this
- Improve abort handling and unhandled aborts reporting

More TODOs:
- Hypervisor disable support
- Guest cells support
- Fix TLB flushes and memory consistency operations, barriers, etc
- Enable the caches and implement all the flushes needed
- SMP (PSCI) support
- Support the 4th page table level in AArch64, bigger granule sizes

Antonios Motakis (20):
hypervisor: printk: include asm/bitops.h directly
hypervisor: arm: add missing include to gic-common.c
hypervisor: add missing include to irqchip.h
hypervisor: add missing include to control.h
hypervisor: add missing include to setup.c
driver: ioremap the hypervisor firmware to any kernel address
hypervisor: arm: hide TLB flush behind a macro
hypervisor: arm64: add sysregs helper macros
hypervisor: arm64: add asm/processor.h header for AArch64
hypervisor: arm64: add definitions for the Aarch64 page table format
hypervisor: arm64: spinlock implementation
hypervisor: arm64: add percpu.h header file
hypervisor: arm64: add cell.h header file
hypervisor: arm64: minimum stubs to allow building on aarch64
Add configuration for the ARMv8 Foundation model
hypervisor: arm64: implement entry code to the hypervisor firmware
hypervisor: arm64: initial exception handling and catch EL2 aborts
hypervisor: arm64: add hypervisor mmu code
hypervisor: arm64: add stubs for handling mmio accesses
hypervisor: arm64: build the GICv2 code from the AArch32 port

Claudio Fontana (3):
hypervisor: lib: make generic memory ops weak
arm64: implement bitops
hypervisor: arm64: implement efficient memory operations

Dmitry Voytik (1):
hypervisor: arm64: add types.h

ci/jailhouse-config-foundation-v8.h | 4 +
configs/foundation-v8.c | 119 +++++
driver/main.c | 9 +-
hypervisor/Makefile | 4 +
hypervisor/arch/arm/gic-common.c | 1 +
hypervisor/arch/arm/include/asm/irqchip.h | 1 +
.../arch/arm/include/asm/jailhouse_hypercall.h | 1 +
hypervisor/arch/arm/include/asm/processor.h | 2 +
hypervisor/arch/arm/lib.c | 12 -
hypervisor/arch/arm/mmu_cell.c | 2 +-
hypervisor/arch/arm64/Makefile | 26 +
hypervisor/arch/arm64/asm-defines.c | 19 +
hypervisor/arch/arm64/caches.c | 21 +
hypervisor/arch/arm64/control.c | 147 ++++++
hypervisor/arch/arm64/entry.S | 174 +++++++
hypervisor/arch/arm64/exception.S | 81 +++
hypervisor/arch/arm64/include/asm/bitops.h | 120 +++++
hypervisor/arch/arm64/include/asm/cell.h | 36 ++
hypervisor/arch/arm64/include/asm/head.h | 16 +
.../arch/arm64/include/asm/jailhouse_hypercall.h | 55 +++
hypervisor/arch/arm64/include/asm/paging.h | 184 +++++++
hypervisor/arch/arm64/include/asm/percpu.h | 86 ++++
hypervisor/arch/arm64/include/asm/platform.h | 51 ++
hypervisor/arch/arm64/include/asm/processor.h | 111 +++++
hypervisor/arch/arm64/include/asm/setup.h | 29 ++
hypervisor/arch/arm64/include/asm/spinlock.h | 66 +++
hypervisor/arch/arm64/include/asm/sysregs.h | 26 +
hypervisor/arch/arm64/include/asm/traps.h | 32 ++
hypervisor/arch/arm64/include/asm/types.h | 46 ++
hypervisor/arch/arm64/lib.c | 19 +
hypervisor/arch/arm64/mmio.c | 53 ++
hypervisor/arch/arm64/setup.c | 57 +++
hypervisor/arch/arm64/string.S | 545 +++++++++++++++++++++
hypervisor/arch/arm64/traps.c | 33 ++
.../arch/x86/include/asm/jailhouse_hypercall.h | 1 +
hypervisor/include/jailhouse/control.h | 1 +
hypervisor/lib.c | 15 +
hypervisor/printk.c | 1 +
hypervisor/setup.c | 3 +-
inmates/demos/arm64/Makefile | 0
inmates/lib/arm64/Makefile | 0
inmates/tools/arm64/Makefile | 0
42 files changed, 2193 insertions(+), 16 deletions(-)
create mode 100644 ci/jailhouse-config-foundation-v8.h
create mode 100644 configs/foundation-v8.c
create mode 100644 hypervisor/arch/arm64/Makefile
create mode 100644 hypervisor/arch/arm64/asm-defines.c
create mode 100644 hypervisor/arch/arm64/caches.c
create mode 100644 hypervisor/arch/arm64/control.c
create mode 100644 hypervisor/arch/arm64/entry.S
create mode 100644 hypervisor/arch/arm64/exception.S
create mode 100644 hypervisor/arch/arm64/include/asm/bitops.h
create mode 100644 hypervisor/arch/arm64/include/asm/cell.h
create mode 100644 hypervisor/arch/arm64/include/asm/head.h
create mode 100644 hypervisor/arch/arm64/include/asm/jailhouse_hypercall.h
create mode 100644 hypervisor/arch/arm64/include/asm/paging.h
create mode 100644 hypervisor/arch/arm64/include/asm/percpu.h
create mode 100644 hypervisor/arch/arm64/include/asm/platform.h
create mode 100644 hypervisor/arch/arm64/include/asm/processor.h
create mode 100644 hypervisor/arch/arm64/include/asm/setup.h
create mode 100644 hypervisor/arch/arm64/include/asm/spinlock.h
create mode 100644 hypervisor/arch/arm64/include/asm/sysregs.h
create mode 100644 hypervisor/arch/arm64/include/asm/traps.h
create mode 100644 hypervisor/arch/arm64/include/asm/types.h
create mode 100644 hypervisor/arch/arm64/lib.c
create mode 100644 hypervisor/arch/arm64/mmio.c
create mode 100644 hypervisor/arch/arm64/setup.c
create mode 100644 hypervisor/arch/arm64/string.S
create mode 100644 hypervisor/arch/arm64/traps.c
create mode 100644 inmates/demos/arm64/Makefile
create mode 100644 inmates/lib/arm64/Makefile
create mode 100644 inmates/tools/arm64/Makefile

--
2.4.3.368.g7974889

[Antonios Motakis]

Implement the entry point we will jump to after the Linux driver loads
the firmware to memory. Here we set up a stack for the hypervisor to use.

Unlike Aarch32, we jump to EL2 as soon as we enter the hypervisor binary.
To do this, we also set up temporary MMU mappings. We use just two pages,
to statically configure the MMU for identity mapping; we need this in
order to perform unaligned accesses from the hypervisor binary during
early initialization.

To generate the early page tables at build time, we need to know of the
Jailhouse physical address, and the physical address of the debug
UART. We introduce this change of behaviour in Aarch64, because in
this architecture we do not have the option of using the page tables set
up previously by Linux.

Signed-off-by: Antonios Motakis <antonios...@huawei.com>
---
hypervisor/arch/arm64/entry.S | 153 +++++++++++++++++++++++++++++
hypervisor/arch/arm64/include/asm/percpu.h | 6 ++
2 files changed, 159 insertions(+)

diff --git a/hypervisor/arch/arm64/entry.S b/hypervisor/arch/arm64/entry.S
index 23ef549..918f227 100644
--- a/hypervisor/arch/arm64/entry.S
+++ b/hypervisor/arch/arm64/entry.S
@@ -10,8 +10,161 @@
* the COPYING file in the top-level directory.
*/

+#include <asm/head.h>
+#include <asm/percpu.h>
+#include <asm/platform.h>
+
/* Entry point for Linux loader module on JAILHOUSE_ENABLE */
.text
.globl arch_entry
arch_entry:
+ /* x0: cpuid */
+ stp x29, x30, [sp, #-16]! /* push fp, lr */
+ stp x19, x20, [sp, #-16]! /* push callee saved registers */
+ stp x21, x22, [sp, #-16]!
+ stp x23, x24, [sp, #-16]!
+ stp x25, x26, [sp, #-16]!
+ stp x27, x28, [sp, #-16]!
+
+ mov x19, x0 /* keep cpuid. we need it */
+
+ /* get physical address of bootstrap_vectors */
+ ldr x0, =bootstrap_vectors
+ hvc #0 /* linux stub installs bootstrap_vectors */
+ hvc #0 /* bootstrap vector enters EL2 */
+
+ /* the bootstrap vector returns us here in physical addressing */
+el2_return:
+ mrs x1, esr_el2
+ lsr x1, x1, #26
+ cmp x1, #0x16
+ b.ne . /* not hvc */
+
+ /* enable temporary mmu mappigns for early initialization */
+ ldr x0, =bootstrap_pt_l1
+ bl enable_mmu_el2
+
+ mov x0, x19 /* restore cpuid */
+
+ ldr x1, =__page_pool
+ mov x2, #(1 << PERCPU_SIZE_SHIFT)
+ /*
+ * percpu data = pool + cpuid * shift
+ * AARCH64_TODO: handle affinities
+ */
+ madd x1, x2, x0, x1
+ add x19, x1, #PERCPU_LINUX_SP
+ msr tpidr_el2, x1
+
+ /*
+ * Save SP, LR, and PSTATE
+ * x19 is used so that they can be easily retrieved on failure.
+ */
+ mrs x2, sp_el1
+ str x2, [x19], #8
+ str x30, [x19], #8
+ mrs x2, daif
+ str x2, [x19]
+
+ /* set SP for hypervisor */
+ add x2, x1, #PERCPU_STACK_END
+ mov sp, x2
+
+ /* setup frame pointer */
+ stp xzr, xzr, [sp, #-16]!
+ sub x29, x2, #16
+
+ /* Call entry(cpuid, struct per_cpu*) */
+ bl entry
+
+ /* AARCH64_TODO: proper error / vmreturn handling */
b .
+
+ .globl enable_mmu_el2
+enable_mmu_el2:
+ /*
+ * x0: u64 ttbr0_el2
+ */
+
+ /* setup the MMU for EL2 hypervisor mappings */
+ ldr x1, =DEFAULT_MAIR_EL2
+ msr mair_el2, x1
+
+ /* AARCH64_TODO: look into inner and outer shareable domains on our
+ * target, and how to handle properly here */
+ ldr x1, =(T0SZ | (TCR_RGN_WB_WA << TCR_IRGN0_SHIFT) \
+ | (TCR_RGN_WB_WA << TCR_ORGN0_SHIFT) \
+ | (TCR_INNER_SHAREABLE << TCR_SH0_SHIFT) \
+ | (TCR_PS_40B << TCR_PS_SHIFT) \
+ | TCR_EL2_RES1)
+ msr tcr_el2, x1
+
+ msr ttbr0_el2, x0
+
+ tlbi alle2
+ dsb nsh
+
+ ldr x1, =(SCTLR_I_BIT | SCTLR_C_BIT | SCTLR_M_BIT | SCTLR_EL2_RES1)
+ msr sctlr_el2, x1
+ isb
+
+ ret
+
+/*
+ * Using two pages, we can economically identity map the whole address space
+ * of the machine (that is accessible with mappings starting from L1 with a 4KB
+ * translation granule), with the 2MB block that includes the UART marked as
+ * device memory. This allows us to start initializing the hypervisor before
+ * we set up the final EL2 page tables.
+ */
+.align 12
+bootstrap_pt_l1:
+ addr = 0
+ blk_sz = 1 << 30
+ .rept 512
+ .if (addr ^ UART_BASE) >> 30
+ .quad addr | PAGE_DEFAULT_FLAGS
+ .else
+ .quad bootstrap_pt_l2 + PTE_TABLE_FLAGS
+ .endif
+ addr = addr + blk_sz
+ .endr
+bootstrap_pt_l2:
+ addr = UART_BASE & ~((1 << 30) - 1)
+ blk_sz = 1 << 21
+ .rept 512
+ .if (addr ^ UART_BASE) >> 21
+ .quad addr | PAGE_DEFAULT_FLAGS
+ .else
+ .quad addr | PAGE_DEFAULT_FLAGS | PAGE_FLAG_DEVICE
+ .endif
+ addr = addr + blk_sz
+ .endr
+
+.macro ventry label
+ .align 7
+ b \label
+.endm
+
+ .globl bootstrap_vectors
+ .align 11
+bootstrap_vectors:
+ ventry .
+ ventry .
+ ventry .
+ ventry .
+
+ ventry .
+ ventry .
+ ventry .
+ ventry .
+
+ ventry el2_return
+ ventry .
+ ventry .
+ ventry .
+
+ ventry .
+ ventry .
+ ventry .
+ ventry .
diff --git a/hypervisor/arch/arm64/include/asm/percpu.h b/hypervisor/arch/arm64/include/asm/percpu.h
index a9eeb8f..36125eb 100644
--- a/hypervisor/arch/arm64/include/asm/percpu.h
+++ b/hypervisor/arch/arm64/include/asm/percpu.h
@@ -29,6 +29,12 @@
#include <asm/spinlock.h>

struct per_cpu {
+ u8 stack[PAGE_SIZE];
+ unsigned long linux_sp;
+ unsigned long linux_ret;
+ unsigned long linux_flags;
+ unsigned long linux_reg[NUM_ENTRY_REGS];
+
/* common fields */
unsigned int cpu_id;
struct cell *cell;
--
2.4.3.368.g7974889