[PATCH v8 00/28] Jailhouse port to ARM AArch64 processors

[antonios...@huawei.com]

From: Antonios Motakis <antonios...@huawei.com>

This patch series includes the port of Jailhouse on ARM AArch64.
This is the first version of the series, where we drop the RFC
tag, as the port has matured a lot since the first release.

In this version of the series, we have splited some of the
functionality and included in two separate series; one with
any preparatory changes, and a separate series with the inmate
demos for AArch64.

These apply in succession (preparatory -> main -> inmates series),
on the latest next branch from upstream, and they can also be
pulled from https://github.com/tvelocity/jailhouse.git (branch arm64_v8)

Changes from RFCv7:
- The early bootstrap page tables are now being generated
dynamically, during entry. This change will make it easier
to make Jailhouse a position independent executable in the
future.
- ARMv7 (should be) working again!
- Fixed a few issues during hypervisor disable
- The hypervisor should work on a wider range of PARanges now
- Further small fixes, clean ups, as always

Changes from RFCv6:
- Probably too many to list here!
- Initial support for MPID affinity levels (as needed by PSCI)
- Working inmates
- Linux inmate support, by Dmitry Voytik!
- Improved /fixed cache coherency handling by Dmitry Voytik
- Support for the 4th level of page tables, allowing for a PARange of 40-48
- Many fixes that were discovered by running Jailhouse on the AMD Seattle

Changes from RFCv5:
- PSCI support
- Hypercalls to the hypervisor
- Hypervisor disable, and also return to Linux properly when
initialization fails
- More clean ups, clean ups, fixes
Contributions by Dmitry Voytik:
- Implement cache flushes, maintenance of the memory system
- Refactored a lot of trap handling code and other mmio bits
- Dump cell registers support for AArch64.

Changes from RFCv4:
- Stubs now use trace_error, or block, to make it more obvious when
we run into a missing stub during development.
- Working root cell! Thanks to working MMU mappings, and working
GICv2 handling.
- MMU mappings are being set up for the hypervisor (EL2), and for
the root cell (Stage 2 EL1).
- Reworked the JAILHOUSE_IOMAP_ADDR decoupling from JAILHOUSE_BASE
- Clean ups, clean ups, fixes


Antonios Motakis (22):
hypervisor: arm64: add sysregs helper macros
hypervisor: arm64: add asm/processor.h header for AArch64
hypervisor: arm64: add definitions for the AArch64 page table format
hypervisor: arm64: spinlock implementation
hypervisor: arm64: add percpu.h header file
hypervisor: arm64: add cell.h header file
hypervisor: arm64: add jailhouse_hypercall.h header file
hypervisor: arm64: minimum stubs to allow building on AArch64
core: add root cell configuration for the ARMv8 Foundation model
hypervisor: arm64: root cell configuration for the AMD Seattle
hypervisor: arm64: implement data cache flush
hypervisor: arm64: initialize EL2 for hypervisor entry
hypervisor: arm64: initialize the hypervisor for entry
hypervisor: arm64: initial exception handling and catch EL2 aborts
hypervisor: arm64: plug the hypervisor mmu code
hypervisor: arm64: handle accesses to emulated mmio regions
hypervisor: arm64: plug the irqchip and GICv2 code from AArch32
hypervisor: arm64: PSCI support for SMP on AArch64
hypervisor: arm64: reanimate the root cell back from the dead
hypervisor: arm64: hande hypercalls from the cells
hypervisor: arm64: hypervisor disable support
hypervisor: arm64: implement cell control infrastructure

Claudio Fontana (1):
arm64: implement bitops

Dmitry Voytik (4):
hypervisor: arm64: add control.h header file
hypervisor: arm64: add types.h
hypervisor: arm64: dump stack on unhandled exceptions
tools: arm64: add exception dump parser

Jan Kiszka (1):
ci: Add support for arm64

.travis.yml | 2 +-
ci/build-all-configs.sh | 6 +-
ci/gen-kernel-build.sh | 1 +
ci/jailhouse-config-amd-seattle.h | 5 +
ci/kernel-config-amd-seattle | 3091 ++++++++++++++++++++
configs/amd-seattle.c | 162 +
configs/foundation-v8.c | 120 +
hypervisor/Makefile | 4 +
hypervisor/arch/arm/include/asm/psci.h | 2 +-
hypervisor/arch/arm/include/asm/uart_pl011.h | 2 +
hypervisor/arch/arm/psci.c | 4 +-
hypervisor/arch/arm64/Makefile | 26 +
hypervisor/arch/arm64/asm-defines.c | 19 +
hypervisor/arch/arm64/caches.S | 48 +
hypervisor/arch/arm64/control.c | 317 ++
hypervisor/arch/arm64/entry.S | 330 +++
hypervisor/arch/arm64/exception.S | 96 +
hypervisor/arch/arm64/include/asm/bitops.h | 141 +
hypervisor/arch/arm64/include/asm/cell.h | 37 +
hypervisor/arch/arm64/include/asm/control.h | 43 +
hypervisor/arch/arm64/include/asm/head.h | 16 +
.../arch/arm64/include/asm/jailhouse_hypercall.h | 93 +
hypervisor/arch/arm64/include/asm/paging.h | 243 ++
hypervisor/arch/arm64/include/asm/percpu.h | 126 +
hypervisor/arch/arm64/include/asm/platform.h | 69 +
hypervisor/arch/arm64/include/asm/processor.h | 191 ++
hypervisor/arch/arm64/include/asm/sections.h | 29 +
hypervisor/arch/arm64/include/asm/setup.h | 29 +
hypervisor/arch/arm64/include/asm/spinlock.h | 71 +
hypervisor/arch/arm64/include/asm/sysregs.h | 26 +
hypervisor/arch/arm64/include/asm/traps.h | 37 +
hypervisor/arch/arm64/include/asm/types.h | 46 +
hypervisor/arch/arm64/mmio.c | 109 +
hypervisor/arch/arm64/psci_low.S | 65 +
hypervisor/arch/arm64/setup.c | 126 +
hypervisor/arch/arm64/traps.c | 203 ++
inmates/demos/arm64/Makefile | 0
inmates/lib/arm64/Makefile | 0
inmates/tools/arm64/Makefile | 0
scripts/arm64-parsedump.py | 167 ++
40 files changed, 6098 insertions(+), 4 deletions(-)
create mode 100644 ci/jailhouse-config-amd-seattle.h
create mode 100644 ci/kernel-config-amd-seattle
create mode 100644 configs/amd-seattle.c
create mode 100644 configs/foundation-v8.c
create mode 100644 hypervisor/arch/arm64/Makefile
create mode 100644 hypervisor/arch/arm64/asm-defines.c
create mode 100644 hypervisor/arch/arm64/caches.S
create mode 100644 hypervisor/arch/arm64/control.c
create mode 100644 hypervisor/arch/arm64/entry.S
create mode 100644 hypervisor/arch/arm64/exception.S
create mode 100644 hypervisor/arch/arm64/include/asm/bitops.h
create mode 100644 hypervisor/arch/arm64/include/asm/cell.h
create mode 100644 hypervisor/arch/arm64/include/asm/control.h
create mode 100644 hypervisor/arch/arm64/include/asm/head.h
create mode 100644 hypervisor/arch/arm64/include/asm/jailhouse_hypercall.h
create mode 100644 hypervisor/arch/arm64/include/asm/paging.h
create mode 100644 hypervisor/arch/arm64/include/asm/percpu.h
create mode 100644 hypervisor/arch/arm64/include/asm/platform.h
create mode 100644 hypervisor/arch/arm64/include/asm/processor.h
create mode 100644 hypervisor/arch/arm64/include/asm/sections.h
create mode 100644 hypervisor/arch/arm64/include/asm/setup.h
create mode 100644 hypervisor/arch/arm64/include/asm/spinlock.h
create mode 100644 hypervisor/arch/arm64/include/asm/sysregs.h
create mode 100644 hypervisor/arch/arm64/include/asm/traps.h
create mode 100644 hypervisor/arch/arm64/include/asm/types.h
create mode 100644 hypervisor/arch/arm64/mmio.c
create mode 100644 hypervisor/arch/arm64/psci_low.S
create mode 100644 hypervisor/arch/arm64/setup.c
create mode 100644 hypervisor/arch/arm64/traps.c
create mode 100644 inmates/demos/arm64/Makefile
create mode 100644 inmates/lib/arm64/Makefile
create mode 100644 inmates/tools/arm64/Makefile
create mode 100755 scripts/arm64-parsedump.py

--
2.7.0

[antonios...@huawei.com]

From: Antonios Motakis <antonios...@huawei.com>

The AArch64 page table format is very similar, almost identical to
the AArch32 page table format. Add a header file for the AArch64 page
table format, based on the AArch32 implementation.

AArch64 introduces an extra level of page tables, for a total of
four, and support for different translation granule sizes.
Sticking to a granule size of 4Kb, we end up with an identical page
table format to AArch32. Using either 3 or the full 4 supported
levels of page tables, we can reach a physical address range of
48 bits.

Signed-off-by: Antonios Motakis <antonios...@huawei.com>
---
hypervisor/arch/arm64/include/asm/paging.h | 243 +++++++++++++++++++++++++++++
1 file changed, 243 insertions(+)
create mode 100644 hypervisor/arch/arm64/include/asm/paging.h

diff --git a/hypervisor/arch/arm64/include/asm/paging.h b/hypervisor/arch/arm64/include/asm/paging.h
new file mode 100644
index 0000000..872af42
--- /dev/null
+++ b/hypervisor/arch/arm64/include/asm/paging.h
@@ -0,0 +1,243 @@
+/*
+ * Jailhouse AArch64 support
+ *
+ * Copyright (C) 2015-2016 Huawei Technologies Duesseldorf GmbH
+ *
+ * Authors:
+ * Antonios Motakis <antonios...@huawei.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2. See
+ * the COPYING file in the top-level directory.
+ */
+
+#ifndef _JAILHOUSE_ASM_PAGING_H
+#define _JAILHOUSE_ASM_PAGING_H
+
+#include <jailhouse/types.h>
+#include <jailhouse/utils.h>
+#include <asm/processor.h>
+#include <asm/sysregs.h>
+
+/*
+ * This file is based on hypervisor/arch/arm/include/asm/paging.h for AArch32.
+ * However, there are some differences. AArch64 supports different granule
+ * sizes for pages (4Kb, 16Kb, and 64Kb), while AArch32 supports only a 4Kb
+ * native page size. AArch64 also supports 4 levels of page tables, numbered
+ * L0-3, while AArch32 supports only 3 levels numbered L1-3.
+ *
+ * We currently only implement 4Kb granule size for the page tables.
+ * We support physical address ranges of up to 48 bits.
+ */
+
+#define PAGE_SHIFT 12
+#define PAGE_SIZE (1 << PAGE_SHIFT)
+#define PAGE_MASK ~(PAGE_SIZE - 1)
+#define PAGE_OFFS_MASK (PAGE_SIZE - 1)
+
+#define MAX_PAGE_TABLE_LEVELS 4
+
+#define L0_VADDR_MASK BIT_MASK(47, 39)
+#define L1_VADDR_MASK BIT_MASK(38, 30)
+#define L2_VADDR_MASK BIT_MASK(29, 21)
+
+#define L3_VADDR_MASK BIT_MASK(20, 12)
+
+/*
+ * Stage-1 and Stage-2 lower attributes.
+ * The contiguous bit is a hint that allows the PE to store blocks of 16 pages
+ * in the TLB. This may be a useful optimisation.
+ */
+#define PTE_ACCESS_FLAG (0x1 << 10)
+/*
+ * When combining shareability attributes, the stage-1 ones prevail. So we can
+ * safely leave everything non-shareable at stage 2.
+ */
+#define PTE_NON_SHAREABLE (0x0 << 8)
+#define PTE_OUTER_SHAREABLE (0x2 << 8)
+#define PTE_INNER_SHAREABLE (0x3 << 8)
+
+#define PTE_MEMATTR(val) ((val) << 2)
+#define PTE_FLAG_TERMINAL (0x1 << 1)
+#define PTE_FLAG_VALID (0x1 << 0)
+
+/* These bits differ in stage 1 and 2 translations */
+#define S1_PTE_NG (0x1 << 11)
+#define S1_PTE_ACCESS_RW (0x0 << 7)
+#define S1_PTE_ACCESS_RO (0x1 << 7)
+/* Res1 for EL2 stage-1 tables */
+#define S1_PTE_ACCESS_EL0 (0x1 << 6)
+
+#define S2_PTE_ACCESS_RO (0x1 << 6)
+#define S2_PTE_ACCESS_WO (0x2 << 6)
+#define S2_PTE_ACCESS_RW (0x3 << 6)
+
+/*
+ * Descriptor pointing to a page table
+ * (only for L1 and L2. L3 uses this encoding for terminal entries...)
+ */
+#define PTE_TABLE_FLAGS 0x3
+
+#define PTE_L0_BLOCK_ADDR_MASK BIT_MASK(47, 39)
+#define PTE_L1_BLOCK_ADDR_MASK BIT_MASK(47, 30)
+#define PTE_L2_BLOCK_ADDR_MASK BIT_MASK(47, 21)
+#define PTE_TABLE_ADDR_MASK BIT_MASK(47, 12)
+#define PTE_PAGE_ADDR_MASK BIT_MASK(47, 12)
+
+#define BLOCK_512G_VADDR_MASK BIT_MASK(38, 0)
+#define BLOCK_1G_VADDR_MASK BIT_MASK(29, 0)
+#define BLOCK_2M_VADDR_MASK BIT_MASK(20, 0)
+
+/*
+ * AARCH64_TODO: the way TTBR_MASK is handled is almost certainly wrong. The
+ * low bits of the TTBR should be zero, however this is an alignment requirement
+ * as well for the actual location of the page table root. We get around the
+ * buggy behaviour in the AArch32 code we share, by setting the mask to the
+ * de facto alignment employed by the arch independent code: one page.
+ */
+#define TTBR_MASK BIT_MASK(47, 12)
+#define VTTBR_VMID_SHIFT 48
+
+#define TCR_EL2_RES1 ((1 << 31) | (1 << 23))
+#define VTCR_RES1 ((1 << 31))
+#define T0SZ(parange) (64 - parange)
+#define SL0_L0 2
+#define SL0_L1 1
+#define SL0_L2 0
+#define TCR_PS_32B 0x0
+#define TCR_PS_36B 0x1
+#define TCR_PS_40B 0x2
+#define TCR_PS_42B 0x3
+#define TCR_PS_44B 0x4
+#define TCR_PS_48B 0x5
+#define TCR_RGN_NON_CACHEABLE 0x0
+#define TCR_RGN_WB_WA 0x1
+#define TCR_RGN_WT 0x2
+#define TCR_RGN_WB 0x3
+#define TCR_NON_SHAREABLE 0x0
+#define TCR_OUTER_SHAREABLE 0x2
+#define TCR_INNER_SHAREABLE 0x3
+
+#define TCR_PS_SHIFT 16
+#define TCR_SH0_SHIFT 12
+#define TCR_ORGN0_SHIFT 10
+#define TCR_IRGN0_SHIFT 8
+#define TCR_SL0_SHIFT 6
+#define TCR_S_SHIFT 4
+
+/*
+ * Hypervisor memory attribute indexes:
+ * 0: normal WB, RA, WA, non-transient
+ * 1: device
+ * 2: normal non-cacheable
+ * 3-7: unused
+ */
+#define DEFAULT_MAIR_EL2 0x00000000004404ff
+#define MAIR_IDX_WBRAWA 0
+#define MAIR_IDX_DEV 1
+#define MAIR_IDX_NC 2
+
+/* Stage 2 memory attributes (MemAttr[3:0]) */
+#define S2_MEMATTR_OWBIWB 0xf
+#define S2_MEMATTR_DEV 0x1
+
+#define S1_PTE_FLAG_NORMAL PTE_MEMATTR(MAIR_IDX_WBRAWA)
+#define S1_PTE_FLAG_DEVICE PTE_MEMATTR(MAIR_IDX_DEV)
+#define S1_PTE_FLAG_UNCACHED PTE_MEMATTR(MAIR_IDX_NC)
+
+#define S2_PTE_FLAG_NORMAL PTE_MEMATTR(S2_MEMATTR_OWBIWB)
+#define S2_PTE_FLAG_DEVICE PTE_MEMATTR(S2_MEMATTR_DEV)
+
+#define S1_DEFAULT_FLAGS (PTE_FLAG_VALID | PTE_ACCESS_FLAG \
+ | S1_PTE_FLAG_NORMAL | PTE_INNER_SHAREABLE\
+ | S1_PTE_ACCESS_EL0)
+
+/* Macros used by the core, only for the EL2 stage-1 mappings */
+#define PAGE_FLAG_DEVICE S1_PTE_FLAG_DEVICE
+#define PAGE_DEFAULT_FLAGS (S1_DEFAULT_FLAGS | S1_PTE_ACCESS_RW)
+#define PAGE_READONLY_FLAGS (S1_DEFAULT_FLAGS | S1_PTE_ACCESS_RO)
+#define PAGE_PRESENT_FLAGS PTE_FLAG_VALID
+#define PAGE_NONPRESENT_FLAGS 0
+
+#define INVALID_PHYS_ADDR (~0UL)
+
+#define REMAP_BASE 0x00100000UL
+#define NUM_REMAP_BITMAP_PAGES 1
+
+#define NUM_TEMPORARY_PAGES 16
+
+#ifndef __ASSEMBLY__
+
+typedef u64 *pt_entry_t;
+
+extern unsigned int cpu_parange;
+
+/* cpu_parange initialized in arch_paging_init */
+static inline unsigned int get_cpu_parange(void)
+{
+ unsigned long id_aa64mmfr0;
+
+ arm_read_sysreg(ID_AA64MMFR0_EL1, id_aa64mmfr0);
+
+ switch (id_aa64mmfr0 & 0xf) {
+ case TCR_PS_32B:
+ return 32;
+ case TCR_PS_36B:
+ return 36;
+ case TCR_PS_40B:
+ return 40;
+ case TCR_PS_42B:
+ return 42;
+ case TCR_PS_44B:
+ return 44;
+ case TCR_PS_48B:
+ return 48;
+ default:
+ return 0;
+ }
+}
+
+/* The size of the cpu_parange, determines from which level we can
+ * start from the S2 translations, and the size of the first level
+ * page table */
+#define T0SZ_CELL T0SZ(cpu_parange)
+#define SL0_CELL ((cpu_parange >= 44) ? SL0_L0 : SL0_L1)
+#define ARM_CELL_ROOT_PT_SZ \
+ ({ unsigned int ret = 1; \
+ if (cpu_parange > 39 && cpu_parange < 44) \
+ ret = 1 << (cpu_parange - 39); \
+ ret; })
+
+/* Just match the host's PARange */
+#define TCR_PS_CELL \
+ ({ unsigned int ret = 0; \
+ switch (cpu_parange) { \
+ case 32: ret = TCR_PS_32B; break; \
+ case 36: ret = TCR_PS_36B; break; \
+ case 40: ret = TCR_PS_40B; break; \
+ case 42: ret = TCR_PS_42B; break; \
+ case 44: ret = TCR_PS_44B; break; \
+ case 48: ret = TCR_PS_48B; break; \
+ } \
+ ret; })
+
+#define VTCR_CELL (T0SZ_CELL | (SL0_CELL << TCR_SL0_SHIFT)\
+ | (TCR_RGN_WB_WA << TCR_IRGN0_SHIFT) \
+ | (TCR_RGN_WB_WA << TCR_ORGN0_SHIFT) \
+ | (TCR_INNER_SHAREABLE << TCR_SH0_SHIFT)\
+ | (TCR_PS_CELL << TCR_PS_SHIFT) \
+ | VTCR_RES1)
+
+
+/* Only executed on hypervisor paging struct changes */
+static inline void arch_paging_flush_page_tlbs(unsigned long page_addr)
+{
+ asm volatile("tlbi vae2, %0\n"
+ : : "r" (page_addr >> PAGE_SHIFT));
+}
+
+/* Used to clean the PAGE_MAP_COHERENT page table changes */
+extern void arch_paging_flush_cpu_caches(void *addr, long size);
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* !_JAILHOUSE_ASM_PAGING_H */
--
2.7.0

[antonios...@huawei.com]

From: Antonios Motakis <antonios...@huawei.com>

Add the initial cell.h header file needed to build on AArch64.

Signed-off-by: Antonios Motakis <antonios...@huawei.com>
---

hypervisor/arch/arm64/include/asm/cell.h | 34 ++++++++++++++++++++++++++++++++
1 file changed, 34 insertions(+)
create mode 100644 hypervisor/arch/arm64/include/asm/cell.h

diff --git a/hypervisor/arch/arm64/include/asm/cell.h b/hypervisor/arch/arm64/include/asm/cell.h
new file mode 100644
index 0000000..4ba8224
--- /dev/null
+++ b/hypervisor/arch/arm64/include/asm/cell.h
@@ -0,0 +1,34 @@

+/*
+ * Jailhouse AArch64 support
+ *

+ * Copyright (C) 2015 Huawei Technologies Duesseldorf GmbH

+ *
+ * Authors:
+ * Antonios Motakis <antonios...@huawei.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2. See
+ * the COPYING file in the top-level directory.
+ */
+

+#ifndef _JAILHOUSE_ASM_CELL_H
+#define _JAILHOUSE_ASM_CELL_H
+
+#include <jailhouse/types.h>
+#include <asm/spinlock.h>
+
+#ifndef __ASSEMBLY__
+
+#include <jailhouse/cell-config.h>
+#include <jailhouse/hypercall.h>
+#include <jailhouse/paging.h>
+
+struct arch_cell {
+ struct paging_structures mm;
+ spinlock_t caches_lock;
+ bool needs_flush;
+};
+
+extern struct cell root_cell;

+
+#endif /* !__ASSEMBLY__ */

+#endif /* !_JAILHOUSE_ASM_CELL_H */
--
2.7.0

[antonios...@huawei.com]

From: Antonios Motakis <antonios...@huawei.com>

Add the root cell configuration and necessary headers to build
and run Jailhouse on the AMD Seattle development board.

Signed-off-by: Antonios Motakis <antonios...@huawei.com>
---

ci/jailhouse-config-amd-seattle.h | 5 +
configs/amd-seattle.c | 162 +++++++++++++++++++++++++++
hypervisor/arch/arm/include/asm/uart_pl011.h | 2 +
hypervisor/arch/arm64/include/asm/platform.h | 19 ++++
4 files changed, 188 insertions(+)
create mode 100644 ci/jailhouse-config-amd-seattle.h
create mode 100644 configs/amd-seattle.c

diff --git a/ci/jailhouse-config-amd-seattle.h b/ci/jailhouse-config-amd-seattle.h
new file mode 100644
index 0000000..c721a46
--- /dev/null
+++ b/ci/jailhouse-config-amd-seattle.h
@@ -0,0 +1,5 @@
+#define CONFIG_TRACE_ERROR 1
+#define CONFIG_ARM_GIC 1
+#define CONFIG_MACH_AMD_SEATTLE 1
+#define CONFIG_SERIAL_AMBA_PL011 1
+#define JAILHOUSE_BASE 0x82fc000000
diff --git a/configs/amd-seattle.c b/configs/amd-seattle.c
new file mode 100644
index 0000000..2bc27eb
--- /dev/null
+++ b/configs/amd-seattle.c
@@ -0,0 +1,162 @@

+/*
+ * Jailhouse AArch64 support
+ *
+ * Copyright (C) 2015 Huawei Technologies Duesseldorf GmbH
+ *
+ * Authors:
+ * Antonios Motakis <antonios...@huawei.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2. See
+ * the COPYING file in the top-level directory.
+ */
+

+#include <linux/types.h>
+#include <jailhouse/cell-config.h>
+
+#define ARRAY_SIZE(a) (sizeof(a) / sizeof(a[0]))
+
+struct {
+ struct jailhouse_system header;
+ __u64 cpus[1];
+ struct jailhouse_memory mem_regions[15];
+ struct jailhouse_irqchip irqchips[1];
+} __attribute__((packed)) config = {
+ .header = {
+ .signature = JAILHOUSE_SYSTEM_SIGNATURE,
+ .hypervisor_memory = {
+ .phys_start = 0x82fc000000,
+ .size = 0x4000000,
+ },
+ .debug_console = {
+ .phys_start = 0xe1010000,
+ .size = 0x1000,
+ .flags = JAILHOUSE_MEM_IO,
+ },
+ .root_cell = {
+ .name = "AMD Seattle",
+
+ .cpu_set_size = sizeof(config.cpus),
+ .num_memory_regions = ARRAY_SIZE(config.mem_regions),
+ .num_irqchips = 1,
+ },
+ },
+
+ .cpus = {
+ 0xff,
+ },
+
+ .mem_regions = {
+ /* gpio */ {
+ .phys_start = 0xe0030000,
+ .virt_start = 0xe0030000,
+ .size = 0x1000,
+ .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
+ JAILHOUSE_MEM_IO,
+ },
+ /* gpio */ {
+ .phys_start = 0xe0080000,
+ .virt_start = 0xe0080000,
+ .size = 0x1000,
+ .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
+ JAILHOUSE_MEM_IO,
+ },
+ /* gpio */ {
+ .phys_start = 0xe1050000,
+ .virt_start = 0xe1050000,
+ .size = 0x1000,
+ .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
+ JAILHOUSE_MEM_IO,
+ },
+ /* sata */ {
+ .phys_start = 0xe0300000,
+ .virt_start = 0xe0300000,
+ .size = 0x10000,
+ .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
+ JAILHOUSE_MEM_IO,
+ },
+ /* xgmac */ {
+ .phys_start = 0xe0700000,
+ .virt_start = 0xe0700000,
+ .size = 0x100000,
+ .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
+ JAILHOUSE_MEM_IO,
+ },
+ /* xgmac */ {
+ .phys_start = 0xe0900000,
+ .virt_start = 0xe0900000,
+ .size = 0x100000,
+ .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
+ JAILHOUSE_MEM_IO,
+ },
+ /* smmu */ {
+ .phys_start = 0xe0600000,
+ .virt_start = 0xe0600000,
+ .size = 0x10000,
+ .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
+ JAILHOUSE_MEM_IO,
+ },
+ /* smmu */ {
+ .phys_start = 0xe0800000,
+ .virt_start = 0xe0800000,
+ .size = 0x10000,
+ .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
+ JAILHOUSE_MEM_IO,
+ },
+ /* serial */ {
+ .phys_start = 0xe1010000,
+ .virt_start = 0xe1010000,
+ .size = 0x1000,
+ .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
+ JAILHOUSE_MEM_IO,
+ },
+ /* ssp */ {
+ .phys_start = 0xe1020000,
+ .virt_start = 0xe1020000,
+ .size = 0x1000,
+ .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
+ JAILHOUSE_MEM_IO,
+ },
+ /* ssp */ {
+ .phys_start = 0xe1030000,
+ .virt_start = 0xe1030000,
+ .size = 0x1000,
+ .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
+ JAILHOUSE_MEM_IO,
+ },
+ /* phy */ {
+ .phys_start = 0xe1240000,
+ .virt_start = 0xe1240000,
+ .size = 0x1000,
+ .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
+ JAILHOUSE_MEM_IO,
+ },
+ /* phy */ {
+ .phys_start = 0xe1250000,
+ .virt_start = 0xe1250000,
+ .size = 0x1000,
+ .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
+ JAILHOUSE_MEM_IO,
+ },
+ /* ccn */ {
+ .phys_start = 0xe8000000,
+ .virt_start = 0xe8000000,
+ .size = 0x1000000,
+ .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
+ JAILHOUSE_MEM_IO,
+ },
+ /* RAM */ {
+ .phys_start = 0x8000000000,
+ .virt_start = 0x8000000000,
+ .size = 0x400000000,
+ .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
+ JAILHOUSE_MEM_EXECUTE,
+ },
+ },
+ .irqchips = {
+ /* GIC */ {
+ .address = 0xe1100000,
+ .pin_bitmap = 0xffffffffffffffff,
+ },
+ },
+
+};
diff --git a/hypervisor/arch/arm/include/asm/uart_pl011.h b/hypervisor/arch/arm/include/asm/uart_pl011.h
index 8548c86..9505161 100644
--- a/hypervisor/arch/arm/include/asm/uart_pl011.h
+++ b/hypervisor/arch/arm/include/asm/uart_pl011.h
@@ -70,6 +70,7 @@

static void uart_init(struct uart_chip *chip)
{
+#ifndef CONFIG_MACH_AMD_SEATTLE
/* 115200 8N1 */
/* FIXME: Can be improved with an implementation of __aeabi_uidiv */
u32 bauddiv = UART_CLK / (16 * 115200);
@@ -83,6 +84,7 @@ static void uart_init(struct uart_chip *chip)
mmio_write16(base + UARTIBRD, bauddiv);
mmio_write16(base + UARTCR, (UARTCR_EN | UARTCR_TXE | UARTCR_RXE |
UARTCR_Out1 | UARTCR_Out2));
+#endif
}

static void uart_wait(struct uart_chip *chip)
diff --git a/hypervisor/arch/arm64/include/asm/platform.h b/hypervisor/arch/arm64/include/asm/platform.h
index f8d4d91..e97b0f9 100644
--- a/hypervisor/arch/arm64/include/asm/platform.h
+++ b/hypervisor/arch/arm64/include/asm/platform.h
@@ -47,4 +47,23 @@

#endif /* CONFIG_MACH_FOUNDATION_V8 */

+#ifdef CONFIG_MACH_AMD_SEATTLE
+
+/* the device tree shipped with the kernel is wrong;
+ * these are the corrected values */
+# define GICD_BASE ((void *)0xe1110000)
+# define GICD_SIZE 0x1000
+# define GICC_BASE ((void *)0xe112f000)
+# define GICC_SIZE 0x2000
+# define GICH_BASE ((void *)0xe1140000)
+# define GICH_SIZE 0x10000
+# define GICV_BASE ((void *)0xe116f000)
+# define GICV_SIZE 0x2000
+
+# include <asm/gic_v2.h>
+# define MAINTENANCE_IRQ 25
+# define UART_BASE 0xe1010000
+
+#endif /* CONFIG_MACH_AMD_SEATTLE */
+
#endif /* !_JAILHOUSE_ASM_PLATFORM_H */
--
2.7.0

[antonios...@huawei.com]

From: Antonios Motakis <antonios...@huawei.com>

Add under config/foundation-v8.c a root cell configuration for the
ARMv8 Foundation model, so we can in use this target with Jailhouse.
We also add the neccessary parameters in asm/platform.h for this
model.

Signed-off-by: Antonios Motakis <antonios...@huawei.com>
---

configs/foundation-v8.c | 120 +++++++++++++++++++++++++++
hypervisor/arch/arm64/include/asm/platform.h | 32 +++++++
2 files changed, 152 insertions(+)
create mode 100644 configs/foundation-v8.c

diff --git a/configs/foundation-v8.c b/configs/foundation-v8.c
new file mode 100644
index 0000000..c930659
--- /dev/null
+++ b/configs/foundation-v8.c
@@ -0,0 +1,120 @@

+/*
+ * Jailhouse AArch64 support
+ *
+ * Copyright (C) 2015 Huawei Technologies Duesseldorf GmbH
+ *
+ * Authors:
+ * Antonios Motakis <antonios...@huawei.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2. See
+ * the COPYING file in the top-level directory.
+ */
+
+#include <linux/types.h>
+#include <jailhouse/cell-config.h>
+
+#define ARRAY_SIZE(a) (sizeof(a) / sizeof(a[0]))
+
+struct {
+ struct jailhouse_system header;
+ __u64 cpus[1];

+ struct jailhouse_memory mem_regions[9];

+ struct jailhouse_irqchip irqchips[1];
+} __attribute__((packed)) config = {
+ .header = {
+ .signature = JAILHOUSE_SYSTEM_SIGNATURE,
+ .hypervisor_memory = {

+ .phys_start = 0xfc000000,

+ .size = 0x4000000,
+ },
+ .debug_console = {

+ .phys_start = 0x1c090000,

+ .size = 0x1000,
+ .flags = JAILHOUSE_MEM_IO,
+ },
+ .root_cell = {

+ .name = "Foundation ARMv8",

+
+ .cpu_set_size = sizeof(config.cpus),
+ .num_memory_regions = ARRAY_SIZE(config.mem_regions),
+ .num_irqchips = 1,
+ },
+ },
+
+ .cpus = {

+ 0xf,

+ },
+
+ .mem_regions = {

+ /* ethernet */ {
+ .phys_start = 0x1a000000,
+ .virt_start = 0x1a000000,
+ .size = 0x00010000,

+ .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
+ JAILHOUSE_MEM_IO,
+ },

+ /* sysreg */ {
+ .phys_start = 0x1c010000,
+ .virt_start = 0x1c010000,
+ .size = 0x00001000,

+ .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
+ JAILHOUSE_MEM_IO,
+ },

+ /* uart0 */ {
+ .phys_start = 0x1c090000,
+ .virt_start = 0x1c090000,
+ .size = 0x00001000,

+ .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
+ JAILHOUSE_MEM_IO,
+ },

+ /* uart1 */ {
+ .phys_start = 0x1c0a0000,
+ .virt_start = 0x1c0a0000,
+ .size = 0x00001000,

+ .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
+ JAILHOUSE_MEM_IO,
+ },

+ /* uart2 */ {
+ .phys_start = 0x1c0b0000,
+ .virt_start = 0x1c0b0000,
+ .size = 0x00001000,

+ .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
+ JAILHOUSE_MEM_IO,
+ },

+ /* uart3 */ {
+ .phys_start = 0x1c0c0000,
+ .virt_start = 0x1c0c0000,
+ .size = 0x00001000,

+ .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
+ JAILHOUSE_MEM_IO,
+ },

+ /* virtio_block */ {
+ .phys_start = 0x1c130000,
+ .virt_start = 0x1c130000,
+ .size = 0x00001000,

+ .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
+ JAILHOUSE_MEM_IO,
+ },
+ /* RAM */ {

+ .phys_start = 0x80000000,
+ .virt_start = 0x80000000,
+ .size = 0x7c000000,

+ .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
+ JAILHOUSE_MEM_EXECUTE,
+ },

+ /* RAM */ {
+ .phys_start = 0x880000000,
+ .virt_start = 0x880000000,
+ .size = 0x80000000,

+ .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
+ JAILHOUSE_MEM_EXECUTE,
+ },
+ },
+ .irqchips = {
+ /* GIC */ {

+ .address = 0x2c001000,

+ .pin_bitmap = 0xffffffffffffffff,
+ },
+ },
+
+};

diff --git a/hypervisor/arch/arm64/include/asm/platform.h b/hypervisor/arch/arm64/include/asm/platform.h
index afd7e72..f8d4d91 100644
--- a/hypervisor/arch/arm64/include/asm/platform.h
+++ b/hypervisor/arch/arm64/include/asm/platform.h
@@ -15,4 +15,36 @@

#include <jailhouse/config.h>

+#ifdef CONFIG_MACH_FOUNDATION_V8
+
+# ifdef CONFIG_ARM_GIC_V3
+# define GICD_BASE ((void *)0x2f000000)
+# define GICD_SIZE 0x10000
+# define GICR_BASE ((void *)0x2f100000)
+# define GICR_SIZE 0x100000
+
+# include <asm/gic_v3.h>
+# else /* GICv2 */
+# define GICD_BASE ((void *)0x2c001000)
+# define GICD_SIZE 0x1000
+# define GICC_BASE ((void *)0x2c002000)
+/*
+ * WARN: most device trees are broken and report only one page for the GICC.
+ * It will brake the handle_irq code, since the GICC_DIR register is located at
+ * offset 0x1000...
+ */
+# define GICC_SIZE 0x2000
+# define GICH_BASE ((void *)0x2c004000)
+# define GICH_SIZE 0x2000
+# define GICV_BASE ((void *)0x2c006000)

+# define GICV_SIZE 0x2000
+
+# include <asm/gic_v2.h>

+# endif /* GIC */
+
+# define MAINTENANCE_IRQ 25
+# define UART_BASE 0x1c090000
+
+#endif /* CONFIG_MACH_FOUNDATION_V8 */

[antonios...@huawei.com]

From: Antonios Motakis <antonios...@huawei.com>

Add the jailhouse_hypercall.h header file for AArch64. We will need
this also from the Linux side, in order to load Jailhouse in memory
and to issue hypercalls to an already loaded instance of the
hypervisor.

Signed-off-by: Antonios Motakis <antonios...@huawei.com>
---

.../arch/arm64/include/asm/jailhouse_hypercall.h | 93 ++++++++++++++++++++++
1 file changed, 93 insertions(+)
create mode 100644 hypervisor/arch/arm64/include/asm/jailhouse_hypercall.h

diff --git a/hypervisor/arch/arm64/include/asm/jailhouse_hypercall.h b/hypervisor/arch/arm64/include/asm/jailhouse_hypercall.h
new file mode 100644
index 0000000..662b2b1
--- /dev/null
+++ b/hypervisor/arch/arm64/include/asm/jailhouse_hypercall.h
@@ -0,0 +1,93 @@

+/*
+ * Jailhouse AArch64 support
+ *
+ * Copyright (C) 2015 Huawei Technologies Duesseldorf GmbH
+ *
+ * Authors:
+ * Antonios Motakis <antonios...@huawei.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2. See
+ * the COPYING file in the top-level directory.
+ */
+

+#include <jailhouse/config.h>
+
+#define JAILHOUSE_BORROW_ROOT_PT 0
+
+#define JAILHOUSE_CALL_INS "hvc #0x4a48"
+#define JAILHOUSE_CALL_NUM_RESULT "x0"
+#define JAILHOUSE_CALL_ARG1 "x1"
+#define JAILHOUSE_CALL_ARG2 "x2"
+
+/* CPU statistics */
+#define JAILHOUSE_CPU_STAT_VMEXITS_MAINTENANCE JAILHOUSE_GENERIC_CPU_STATS
+#define JAILHOUSE_CPU_STAT_VMEXITS_VIRQ JAILHOUSE_GENERIC_CPU_STATS + 1
+#define JAILHOUSE_CPU_STAT_VMEXITS_VSGI JAILHOUSE_GENERIC_CPU_STATS + 2
+#define JAILHOUSE_NUM_CPU_STATS JAILHOUSE_GENERIC_CPU_STATS + 3
+
+#ifndef __ASSEMBLY__
+
+struct jailhouse_comm_region {
+ COMM_REGION_GENERIC_HEADER;
+};
+
+static inline __u64 jailhouse_call(__u64 num)
+{
+ register __u64 num_result asm(JAILHOUSE_CALL_NUM_RESULT) = num;
+
+ asm volatile(
+ JAILHOUSE_CALL_INS
+ : "=r" (num_result)
+ : "r" (num_result)
+ : "memory");
+ return num_result;
+}
+
+static inline __u64 jailhouse_call_arg1(__u64 num, __u64 arg1)
+{
+ register __u64 num_result asm(JAILHOUSE_CALL_NUM_RESULT) = num;
+ register __u64 __arg1 asm(JAILHOUSE_CALL_ARG1) = arg1;
+
+ asm volatile(
+ JAILHOUSE_CALL_INS
+ : "=r" (num_result)
+ : "r" (num_result), "r" (__arg1)
+ : "memory");
+ return num_result;
+}
+
+static inline __u64 jailhouse_call_arg2(__u64 num, __u64 arg1, __u64 arg2)
+{
+ register __u64 num_result asm(JAILHOUSE_CALL_NUM_RESULT) = num;
+ register __u64 __arg1 asm(JAILHOUSE_CALL_ARG1) = arg1;
+ register __u64 __arg2 asm(JAILHOUSE_CALL_ARG2) = arg2;
+
+ asm volatile(
+ JAILHOUSE_CALL_INS
+ : "=r" (num_result)
+ : "r" (num_result), "r" (__arg1), "r" (__arg2)
+ : "memory");
+ return num_result;
+}
+
+static inline void
+jailhouse_send_msg_to_cell(struct jailhouse_comm_region *comm_region,
+ __u64 msg)
+{
+ comm_region->reply_from_cell = JAILHOUSE_MSG_NONE;
+ /* ensure reply was cleared before sending new message */
+ asm volatile("dmb ishst" : : : "memory");
+ comm_region->msg_to_cell = msg;
+}
+
+static inline void
+jailhouse_send_reply_from_cell(struct jailhouse_comm_region *comm_region,
+ __u64 reply)
+{
+ comm_region->msg_to_cell = JAILHOUSE_MSG_NONE;
+ /* ensure message was cleared before sending reply */
+ asm volatile("dmb ishst" : : : "memory");
+ comm_region->reply_from_cell = reply;
+}

+
+#endif /* !__ASSEMBLY__ */

--
2.7.0

[antonios...@huawei.com]

From: Antonios Motakis <antonios...@huawei.com>

Add the percpu.h header file for the AArch64 implementation. This is
the bare bones version of the header needed to compile a stub
hypervisor binary on AArch64. A lot of these fields could probably
be moved to an arch independent header.

Signed-off-by: Antonios Motakis <antonios...@huawei.com>
---

hypervisor/arch/arm64/include/asm/percpu.h | 72 ++++++++++++++++++++++++++++++
1 file changed, 72 insertions(+)
create mode 100644 hypervisor/arch/arm64/include/asm/percpu.h

diff --git a/hypervisor/arch/arm64/include/asm/percpu.h b/hypervisor/arch/arm64/include/asm/percpu.h
new file mode 100644
index 0000000..7c4e186
--- /dev/null
+++ b/hypervisor/arch/arm64/include/asm/percpu.h
@@ -0,0 +1,72 @@

+/*
+ * Jailhouse AArch64 support
+ *
+ * Copyright (C) 2015 Huawei Technologies Duesseldorf GmbH
+ *
+ * Authors:
+ * Antonios Motakis <antonios...@huawei.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2. See
+ * the COPYING file in the top-level directory.
+ */
+

+#ifndef _JAILHOUSE_ASM_PERCPU_H
+#define _JAILHOUSE_ASM_PERCPU_H
+
+#include <jailhouse/types.h>
+#include <asm/paging.h>
+
+#ifndef __ASSEMBLY__
+
+#include <asm/cell.h>
+#include <asm/spinlock.h>
+
+struct per_cpu {
+ /* common fields */
+ unsigned int cpu_id;
+ struct cell *cell;
+ u32 stats[JAILHOUSE_NUM_CPU_STATS];
+ int shutdown_state;
+ bool failed;
+
+ bool flush_vcpu_caches;
+} __attribute__((aligned(PAGE_SIZE)));
+
+static inline struct per_cpu *this_cpu_data(void)
+{
+ while (1);
+ return NULL;
+}
+
+#define DEFINE_PER_CPU_ACCESSOR(field) \
+static inline typeof(((struct per_cpu *)0)->field) this_##field(void) \
+{ \
+ return this_cpu_data()->field; \
+}
+
+DEFINE_PER_CPU_ACCESSOR(cpu_id)
+DEFINE_PER_CPU_ACCESSOR(cell)
+
+static inline struct per_cpu *per_cpu(unsigned int cpu)
+{
+ while (1);
+ return NULL;
+}
+
+static inline unsigned int arm_cpu_phys2virt(unsigned int cpu_id)
+{
+ return per_cpu(cpu_id)->virt_id;
+}
+
+unsigned int arm_cpu_virt2phys(struct cell *cell, unsigned int virt_id);
+
+/* Validate defines */
+#define CHECK_ASSUMPTION(assume) ((void)sizeof(char[1 - 2*!(assume)]))
+
+static inline void __check_assumptions(void)
+{
+ CHECK_ASSUMPTION(sizeof(unsigned long) == (8));

+}
+#endif /* !__ASSEMBLY__ */

+
+#endif /* !_JAILHOUSE_ASM_PERCPU_H */
--
2.7.0

[antonios...@huawei.com]

From: Antonios Motakis <antonios...@huawei.com>

Add the minimum stub functions expected by the rest of the codebase
to enable building on AArch64. We may implement the missing AArch64
functionality from here.

Signed-off-by: Antonios Motakis <antonios...@huawei.com>
---

hypervisor/Makefile | 4 ++
hypervisor/arch/arm64/Makefile | 22 ++++++++
hypervisor/arch/arm64/asm-defines.c | 19 +++++++
hypervisor/arch/arm64/caches.S | 16 ++++++
hypervisor/arch/arm64/control.c | 83 ++++++++++++++++++++++++++++
hypervisor/arch/arm64/entry.S | 19 +++++++
hypervisor/arch/arm64/include/asm/control.h | 1 +
hypervisor/arch/arm64/include/asm/head.h | 16 ++++++
hypervisor/arch/arm64/include/asm/percpu.h | 13 +++++
hypervisor/arch/arm64/include/asm/platform.h | 18 ++++++
hypervisor/arch/arm64/mmio.c | 27 +++++++++
hypervisor/arch/arm64/setup.c | 41 ++++++++++++++

inmates/demos/arm64/Makefile | 0
inmates/lib/arm64/Makefile | 0
inmates/tools/arm64/Makefile | 0

15 files changed, 279 insertions(+)

create mode 100644 hypervisor/arch/arm64/Makefile
create mode 100644 hypervisor/arch/arm64/asm-defines.c
create mode 100644 hypervisor/arch/arm64/caches.S
create mode 100644 hypervisor/arch/arm64/control.c
create mode 100644 hypervisor/arch/arm64/entry.S

create mode 100644 hypervisor/arch/arm64/include/asm/head.h
create mode 100644 hypervisor/arch/arm64/include/asm/platform.h
create mode 100644 hypervisor/arch/arm64/mmio.c
create mode 100644 hypervisor/arch/arm64/setup.c

create mode 100644 inmates/demos/arm64/Makefile
create mode 100644 inmates/lib/arm64/Makefile
create mode 100644 inmates/tools/arm64/Makefile

diff --git a/hypervisor/Makefile b/hypervisor/Makefile
index 0532e4e..c037ed0 100644
--- a/hypervisor/Makefile
+++ b/hypervisor/Makefile
@@ -33,6 +33,10 @@ ifeq ($(SRCARCH),arm)
KBUILD_CFLAGS += -marm
endif

+ifeq ($(SRCARCH),arm64)
+LINUXINCLUDE += -I$(src)/arch/arm/include
+endif
+
ifneq ($(wildcard $(obj)/include/jailhouse/config.h),)
KBUILD_CFLAGS += -include $(obj)/include/jailhouse/config.h
endif
diff --git a/hypervisor/arch/arm64/Makefile b/hypervisor/arch/arm64/Makefile
new file mode 100644
index 0000000..0911ff3
--- /dev/null
+++ b/hypervisor/arch/arm64/Makefile
@@ -0,0 +1,22 @@
+#
+# Jailhouse AArch64 support
+#
+# Copyright (C) 2015 Huawei Technologies Duesseldorf GmbH
+#
+# Authors:
+# Antonios Motakis <antonios...@huawei.com>
+#
+# This work is licensed under the terms of the GNU GPL, version 2. See
+# the COPYING file in the top-level directory.
+#
+
+include $(CONFIG_MK)
+
+KBUILD_AFLAGS := $(subst -include asm/unified.h,,$(KBUILD_AFLAGS))
+
+always := built-in.o
+
+obj-y := entry.o setup.o control.o mmio.o caches.o
+obj-y += ../arm/mmu_cell.o ../arm/paging.o ../arm/dbg-write.o ../arm/lib.o
+
+obj-$(CONFIG_SERIAL_AMBA_PL011) += ../arm/dbg-write-pl011.o
diff --git a/hypervisor/arch/arm64/asm-defines.c b/hypervisor/arch/arm64/asm-defines.c
new file mode 100644
index 0000000..c026a3c
--- /dev/null
+++ b/hypervisor/arch/arm64/asm-defines.c
@@ -0,0 +1,19 @@

+/*
+ * Jailhouse AArch64 support
+ *
+ * Copyright (C) 2015 Huawei Technologies Duesseldorf GmbH
+ *
+ * Authors:
+ * Antonios Motakis <antonios...@huawei.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2. See
+ * the COPYING file in the top-level directory.
+ */
+

+#include <jailhouse/gen-defines.h>
+
+void common(void);
+
+void common(void)
+{
+}
diff --git a/hypervisor/arch/arm64/caches.S b/hypervisor/arch/arm64/caches.S
new file mode 100644
index 0000000..4859032
--- /dev/null
+++ b/hypervisor/arch/arm64/caches.S
@@ -0,0 +1,16 @@

+/*
+ * Jailhouse AArch64 support
+ *

+ * Copyright (C) 2016 Huawei Technologies Duesseldorf GmbH

+ *
+ * Authors:
+ * Antonios Motakis <antonios...@huawei.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2. See
+ * the COPYING file in the top-level directory.
+ *

+ */
+
+ .global arch_paging_flush_cpu_caches
+arch_paging_flush_cpu_caches:
+ b .
diff --git a/hypervisor/arch/arm64/control.c b/hypervisor/arch/arm64/control.c
new file mode 100644
index 0000000..a1c4774
--- /dev/null
+++ b/hypervisor/arch/arm64/control.c
@@ -0,0 +1,83 @@

+/*
+ * Jailhouse AArch64 support
+ *
+ * Copyright (C) 2015 Huawei Technologies Duesseldorf GmbH
+ *
+ * Authors:
+ * Antonios Motakis <antonios...@huawei.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2. See
+ * the COPYING file in the top-level directory.
+ */
+

+#include <jailhouse/control.h>
+#include <jailhouse/printk.h>
+
+int arch_cell_create(struct cell *cell)
+{
+ return trace_error(-EINVAL);
+}
+
+void arch_flush_cell_vcpu_caches(struct cell *cell)
+{
+ /* AARCH64_TODO */
+ trace_error(-EINVAL);
+}
+
+void arch_cell_destroy(struct cell *cell)
+{
+ trace_error(-EINVAL);
+ while (1);
+}
+
+void arch_config_commit(struct cell *cell_added_removed)
+{
+}
+
+void arch_shutdown(void)
+{
+ trace_error(-EINVAL);
+ while (1);
+}
+
+void arch_suspend_cpu(unsigned int cpu_id)
+{
+ trace_error(-EINVAL);
+ while (1);
+}
+
+void arch_resume_cpu(unsigned int cpu_id)
+{
+ trace_error(-EINVAL);
+ while (1);
+}
+
+void arch_reset_cpu(unsigned int cpu_id)
+{
+ trace_error(-EINVAL);
+ while (1);
+}
+
+void arch_park_cpu(unsigned int cpu_id)
+{
+ trace_error(-EINVAL);
+ while (1);
+}
+
+void arch_shutdown_cpu(unsigned int cpu_id)
+{
+ trace_error(-EINVAL);
+ while (1);
+}
+
+void __attribute__((noreturn)) arch_panic_stop(void)
+{
+ trace_error(-EINVAL);
+ while (1);
+}
+
+void arch_panic_park(void)
+{
+ trace_error(-EINVAL);
+ while (1);
+}
diff --git a/hypervisor/arch/arm64/entry.S b/hypervisor/arch/arm64/entry.S
new file mode 100644
index 0000000..a7c0f2c
--- /dev/null
+++ b/hypervisor/arch/arm64/entry.S
@@ -0,0 +1,19 @@

+/*
+ * Jailhouse AArch64 support
+ *

+ * Copyright (C) 2015-2016 Huawei Technologies Duesseldorf GmbH

+ *
+ * Authors:
+ * Antonios Motakis <antonios...@huawei.com>

+ * Dmitry Voytik <dmitry...@huawei.com>

+ *
+ * This work is licensed under the terms of the GNU GPL, version 2. See
+ * the COPYING file in the top-level directory.
+ */
+

+/* Entry point for Linux loader module on JAILHOUSE_ENABLE */
+ .text
+ .globl arch_entry
+arch_entry:
+ mov x0, -22
+ ret
diff --git a/hypervisor/arch/arm64/include/asm/control.h b/hypervisor/arch/arm64/include/asm/control.h
index 1957d55..6db6bd0 100644
--- a/hypervisor/arch/arm64/include/asm/control.h
+++ b/hypervisor/arch/arm64/include/asm/control.h
@@ -35,6 +35,7 @@ struct registers* arch_handle_exit(struct per_cpu *cpu_data,
bool arch_handle_phys_irq(struct per_cpu *cpu_data, u32 irqn);
void arch_reset_self(struct per_cpu *cpu_data);
void arch_shutdown_self(struct per_cpu *cpu_data);
+unsigned int arm_cpu_by_mpid(struct cell *cell, unsigned long mpid);

void __attribute__((noreturn)) vmreturn(struct registers *guest_regs);
void __attribute__((noreturn)) arch_shutdown_mmu(struct per_cpu *cpu_data);
diff --git a/hypervisor/arch/arm64/include/asm/head.h b/hypervisor/arch/arm64/include/asm/head.h
new file mode 100644
index 0000000..53dd26a
--- /dev/null
+++ b/hypervisor/arch/arm64/include/asm/head.h
@@ -0,0 +1,16 @@

+/*
+ * Jailhouse AArch64 support
+ *
+ * Copyright (C) 2015 Huawei Technologies Duesseldorf GmbH
+ *
+ * Authors:
+ * Antonios Motakis <antonios...@huawei.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2. See
+ * the COPYING file in the top-level directory.
+ */
+

+#ifndef _JAILHOUSE_ASM_HEAD_H
+#define _JAILHOUSE_ASM_HEAD_H_
+
+#endif /* !_JAILHOUSE_ASM_HEAD_H */
diff --git a/hypervisor/arch/arm64/include/asm/percpu.h b/hypervisor/arch/arm64/include/asm/percpu.h
index 7c4e186..b9b1b96 100644
--- a/hypervisor/arch/arm64/include/asm/percpu.h
+++ b/hypervisor/arch/arm64/include/asm/percpu.h
@@ -21,6 +21,18 @@
#include <asm/cell.h>
#include <asm/spinlock.h>

+union mpidr {
+ u64 val;
+ struct {
+ u8 aff0;
+ u8 aff1;
+ u8 aff2;
+ u8 pad1;
+ u8 aff3;
+ u8 pad2[3];
+ } f;
+};
+
struct per_cpu {
/* common fields */
unsigned int cpu_id;
@@ -30,6 +42,7 @@ struct per_cpu {
bool failed;

bool flush_vcpu_caches;
+ union mpidr mpidr;
} __attribute__((aligned(PAGE_SIZE)));

static inline struct per_cpu *this_cpu_data(void)
diff --git a/hypervisor/arch/arm64/include/asm/platform.h b/hypervisor/arch/arm64/include/asm/platform.h
new file mode 100644
index 0000000..afd7e72
--- /dev/null
+++ b/hypervisor/arch/arm64/include/asm/platform.h
@@ -0,0 +1,18 @@

+/*
+ * Jailhouse AArch64 support
+ *
+ * Copyright (C) 2015 Huawei Technologies Duesseldorf GmbH
+ *
+ * Authors:
+ * Antonios Motakis <antonios...@huawei.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2. See
+ * the COPYING file in the top-level directory.
+ */
+

+#ifndef _JAILHOUSE_ASM_PLATFORM_H
+#define _JAILHOUSE_ASM_PLATFORM_H
+
+#include <jailhouse/config.h>
+
+#endif /* !_JAILHOUSE_ASM_PLATFORM_H */
diff --git a/hypervisor/arch/arm64/mmio.c b/hypervisor/arch/arm64/mmio.c
new file mode 100644
index 0000000..37745d7
--- /dev/null
+++ b/hypervisor/arch/arm64/mmio.c
@@ -0,0 +1,27 @@

+/*
+ * Jailhouse AArch64 support
+ *
+ * Copyright (C) 2015 Huawei Technologies Duesseldorf GmbH
+ *
+ * Authors:
+ * Antonios Motakis <antonios...@huawei.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2. See
+ * the COPYING file in the top-level directory.
+ */
+

+#include <jailhouse/entry.h>
+#include <jailhouse/mmio.h>
+#include <jailhouse/printk.h>
+
+unsigned int arch_mmio_count_regions(struct cell *cell)
+{
+ /* not entirely a lie :) */

+ return 0;
+}
+

+void arm_mmio_perform_access(unsigned long base, struct mmio_access *mmio)
+{
+ trace_error(-EINVAL);
+ while (1);
+}
diff --git a/hypervisor/arch/arm64/setup.c b/hypervisor/arch/arm64/setup.c
new file mode 100644
index 0000000..ca83940
--- /dev/null
+++ b/hypervisor/arch/arm64/setup.c
@@ -0,0 +1,41 @@

+/*
+ * Jailhouse AArch64 support
+ *
+ * Copyright (C) 2015 Huawei Technologies Duesseldorf GmbH
+ *
+ * Authors:
+ * Antonios Motakis <antonios...@huawei.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2. See
+ * the COPYING file in the top-level directory.
+ */
+

+#include <jailhouse/entry.h>
+#include <jailhouse/printk.h>
+
+int arch_init_early(void)
+{
+ return trace_error(-EINVAL);
+}
+
+int arch_cpu_init(struct per_cpu *cpu_data)
+{
+ return trace_error(-EINVAL);
+}
+
+int arch_init_late(void)
+{
+ return trace_error(-EINVAL);
+}
+
+void __attribute__((noreturn)) arch_cpu_activate_vmm(struct per_cpu *cpu_data)
+{
+ trace_error(-EINVAL);
+ while (1);
+}
+
+void arch_cpu_restore(struct per_cpu *cpu_data, int return_code)
+{
+ trace_error(-EINVAL);
+ while (1);
+}
diff --git a/inmates/demos/arm64/Makefile b/inmates/demos/arm64/Makefile
new file mode 100644
index 0000000..e69de29
diff --git a/inmates/lib/arm64/Makefile b/inmates/lib/arm64/Makefile
new file mode 100644
index 0000000..e69de29
diff --git a/inmates/tools/arm64/Makefile b/inmates/tools/arm64/Makefile
new file mode 100644
index 0000000..e69de29
--
2.7.0

[antonios...@huawei.com]

From: Antonios Motakis <antonios...@huawei.com>

Enable the MMU mappings for the hypervisor running in EL2, and add
functions to map device regions to the hypervisor address space.

Signed-off-by: Antonios Motakis <antonios...@huawei.com>
---

hypervisor/arch/arm64/include/asm/setup.h | 29 +++++++++++++++++++++++++++++
hypervisor/arch/arm64/setup.c | 25 ++++++++++++++++++++++---
2 files changed, 51 insertions(+), 3 deletions(-)
create mode 100644 hypervisor/arch/arm64/include/asm/setup.h

diff --git a/hypervisor/arch/arm64/include/asm/setup.h b/hypervisor/arch/arm64/include/asm/setup.h
new file mode 100644
index 0000000..a2d1930
--- /dev/null
+++ b/hypervisor/arch/arm64/include/asm/setup.h
@@ -0,0 +1,29 @@

+/*
+ * Jailhouse AArch64 support
+ *
+ * Copyright (C) 2015 Huawei Technologies Duesseldorf GmbH
+ *
+ * Authors:
+ * Antonios Motakis <antonios...@huawei.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2. See
+ * the COPYING file in the top-level directory.
+ */
+

+#ifndef _JAILHOUSE_ASM_SETUP_H
+#define _JAILHOUSE_ASM_SETUP_H
+
+#include <asm/head.h>
+#include <asm/percpu.h>
+
+#ifndef __ASSEMBLY__
+
+#include <jailhouse/string.h>
+
+void enable_mmu_el2(page_table_t ttbr0_el2);
+
+int arch_map_device(void *paddr, void *vaddr, unsigned long size);
+int arch_unmap_device(void *addr, unsigned long size);

+
+#endif /* !__ASSEMBLY__ */

+#endif /* !_JAILHOUSE_ASM_SETUP_H */
diff --git a/hypervisor/arch/arm64/setup.c b/hypervisor/arch/arm64/setup.c
index ca83940..13e6387 100644
--- a/hypervisor/arch/arm64/setup.c
+++ b/hypervisor/arch/arm64/setup.c
@@ -12,20 +12,25 @@

#include <jailhouse/entry.h>
#include <jailhouse/printk.h>
+#include <asm/control.h>
+#include <asm/setup.h>

int arch_init_early(void)
{
- return trace_error(-EINVAL);
+ return arch_mmu_cell_init(&root_cell);
}

int arch_cpu_init(struct per_cpu *cpu_data)
{
- return trace_error(-EINVAL);
+ /* switch to the permanent page tables */
+ enable_mmu_el2(hv_paging_structs.root_table);
+
+ return arch_mmu_cpu_cell_init(cpu_data);
}

int arch_init_late(void)
{
- return trace_error(-EINVAL);
+ return map_root_memory_regions();
}

void __attribute__((noreturn)) arch_cpu_activate_vmm(struct per_cpu *cpu_data)
@@ -34,6 +39,20 @@ void __attribute__((noreturn)) arch_cpu_activate_vmm(struct per_cpu *cpu_data)
while (1);
}

+int arch_map_device(void *paddr, void *vaddr, unsigned long size)
+{
+ return paging_create(&hv_paging_structs, (unsigned long)paddr, size,
+ (unsigned long)vaddr,
+ PAGE_DEFAULT_FLAGS | S1_PTE_FLAG_DEVICE,
+ PAGING_NON_COHERENT);
+}
+
+int arch_unmap_device(void *vaddr, unsigned long size)
+{
+ return paging_destroy(&hv_paging_structs, (unsigned long)vaddr, size,
+ PAGING_NON_COHERENT);
+}
+

void arch_cpu_restore(struct per_cpu *cpu_data, int return_code)

{
trace_error(-EINVAL);
--
2.7.0

[antonios...@huawei.com]

From: Antonios Motakis <antonios...@huawei.com>

Initialize the hypervisor firmware, so we can finally enter
the arch independent entry() function. This includes setting
up a stack, and saving the state of the processor before
entering the hypervisor.

Signed-off-by: Antonios Motakis <antonios...@huawei.com>
---

hypervisor/arch/arm64/entry.S | 46 ++++++++++++++++++++++++++++++
hypervisor/arch/arm64/include/asm/percpu.h | 32 ++++++++++++++++++---
2 files changed, 74 insertions(+), 4 deletions(-)

diff --git a/hypervisor/arch/arm64/entry.S b/hypervisor/arch/arm64/entry.S
index d9570cf..346e91a 100644
--- a/hypervisor/arch/arm64/entry.S
+++ b/hypervisor/arch/arm64/entry.S
@@ -23,7 +23,14 @@ arch_entry:
/*
* x0: cpuid
*
+ * We don't have access to our own address space yet, so we will
+ * abuse some caller saved registers to preserve accross calls:
+ * x16: saved hyp vectors
+ * x17: cpuid
+ * x18: caller lr
*/
+ mov x17, x0
+ mov x18, x30

/* Note 1: After turning MMU off the CPU can start bypassing caches.
* But cached before data is kept in caches either until the CPU turns
@@ -38,6 +45,11 @@ arch_entry:
* Invalidate is safe in guests.
*/

+ /* keep the linux stub EL2 vectors for later */
+ mov x0, xzr
+ hvc #0
+ mov x16, x0
+
/* install bootstrap_vectors */
ldr x0, =bootstrap_vectors
hvc #0
@@ -58,6 +70,40 @@ el2_entry:
adr x0, bootstrap_pt_l0
bl enable_mmu_el2

+ mov x0, x17 /* preserved cpuid, will be passed to entry */
+ adrp x1, __page_pool
+ mov x2, #(1 << PERCPU_SIZE_SHIFT)
+ /*
+ * percpu data = pool + cpuid * shift
+ */
+ madd x1, x2, x0, x1
+ msr tpidr_el2, x1
+
+ /* set up the stack and push the root cell's callee saved registers */
+ add sp, x1, #PERCPU_STACK_END
+ stp x29, x18, [sp, #-16]! /* note: our caller lr is in x18 */
+ stp x27, x28, [sp, #-16]!
+ stp x25, x26, [sp, #-16]!
+ stp x23, x24, [sp, #-16]!
+ stp x21, x22, [sp, #-16]!
+ stp x19, x20, [sp, #-16]!
+ /*
+ * We pad the stack, so we can consistently access the guest
+ * registers from either the initialization, or the exception
+ * handling code paths. 19 caller saved registers plus the
+ * exit_reason, which we don't use on entry.
+ */
+ sub sp, sp, 20 * 8
+
+ mov x29, xzr /* reset fp,lr */
+ mov x30, xzr
+
+ /* save the Linux stub vectors we kept earlier */
+ add x2, x1, #PERCPU_LINUX_SAVED_VECTORS
+ str x16, [x2]
+
+ /* Call entry(cpuid, struct per_cpu*). Should not return. */
+ bl entry
b .

.globl enable_mmu_el2
diff --git a/hypervisor/arch/arm64/include/asm/percpu.h b/hypervisor/arch/arm64/include/asm/percpu.h
index b9b1b96..3763d8f 100644
--- a/hypervisor/arch/arm64/include/asm/percpu.h
+++ b/hypervisor/arch/arm64/include/asm/percpu.h
@@ -16,6 +16,11 @@
#include <jailhouse/types.h>
#include <asm/paging.h>

+/* Keep in sync with struct per_cpu! */
+#define PERCPU_SIZE_SHIFT 13
+#define PERCPU_STACK_END PAGE_SIZE
+#define PERCPU_LINUX_SAVED_VECTORS PERCPU_STACK_END
+
#ifndef __ASSEMBLY__

#include <asm/cell.h>
@@ -34,6 +39,9 @@ union mpidr {
};

struct per_cpu {
+ u8 stack[PAGE_SIZE];
+ unsigned long saved_vectors;
+

/* common fields */
unsigned int cpu_id;

struct cell *cell;
@@ -47,8 +55,10 @@ struct per_cpu {

static inline struct per_cpu *this_cpu_data(void)

{
- while (1);
- return NULL;
+ struct per_cpu *cpu_data;
+
+ arm_read_sysreg(TPIDR_EL2, cpu_data);
+ return cpu_data;
}

#define DEFINE_PER_CPU_ACCESSOR(field) \
@@ -62,8 +72,16 @@ DEFINE_PER_CPU_ACCESSOR(cell)

static inline struct per_cpu *per_cpu(unsigned int cpu)

{
- while (1);
- return NULL;
+ extern u8 __page_pool[];
+
+ return (struct per_cpu *)(__page_pool + (cpu << PERCPU_SIZE_SHIFT));
+}
+
+static inline struct registers *guest_regs(struct per_cpu *cpu_data)
+{
+ /* assumes that the cell registers are at the beginning of the stack */
+ return (struct registers *)(cpu_data->stack + PERCPU_STACK_END
+ - sizeof(struct registers));

}

static inline unsigned int arm_cpu_phys2virt(unsigned int cpu_id)

@@ -78,7 +96,13 @@ unsigned int arm_cpu_virt2phys(struct cell *cell, unsigned int virt_id);

static inline void __check_assumptions(void)
{
+ struct per_cpu cpu_data;

+
CHECK_ASSUMPTION(sizeof(unsigned long) == (8));

+ CHECK_ASSUMPTION(sizeof(struct per_cpu) == (1 << PERCPU_SIZE_SHIFT));
+ CHECK_ASSUMPTION(sizeof(cpu_data.stack) == PERCPU_STACK_END);
+ CHECK_ASSUMPTION(__builtin_offsetof(struct per_cpu, saved_vectors) ==
+ PERCPU_LINUX_SAVED_VECTORS);

[antonios...@huawei.com]

From: Antonios Motakis <antonios...@huawei.com>

Unlike AArch32, we jump to EL2 as soon as we enter the hypervisor
binary.

In order to do that, we also need to switch on the MMU, so we
generate early bootstrap page tables.
We need these in order to perform unaligned accesses from the
hypervisor binary during early initialization, and to avoid having
to maintain the caches during initialization.

Signed-off-by: Antonios Motakis <antonios...@huawei.com>
Signed-off-by: Dmitry Voytik <dmitry...@huawei.com>
---
hypervisor/arch/arm64/entry.S | 230 ++++++++++++++++++++++++++-
hypervisor/arch/arm64/include/asm/sections.h | 29 ++++
2 files changed, 258 insertions(+), 1 deletion(-)
create mode 100644 hypervisor/arch/arm64/include/asm/sections.h

diff --git a/hypervisor/arch/arm64/entry.S b/hypervisor/arch/arm64/entry.S
index a7c0f2c..d9570cf 100644
--- a/hypervisor/arch/arm64/entry.S
+++ b/hypervisor/arch/arm64/entry.S
@@ -11,9 +11,237 @@

* the COPYING file in the top-level directory.

*/

+#include <asm/head.h>
+#include <asm/percpu.h>
+#include <asm/platform.h>
+#include <asm/jailhouse_hypercall.h>
+

/* Entry point for Linux loader module on JAILHOUSE_ENABLE */

.text
.globl arch_entry
arch_entry:
- mov x0, -22
+ /*
+ * x0: cpuid

+ *
+ */
+

+ /* Note 1: After turning MMU off the CPU can start bypassing caches.
+ * But cached before data is kept in caches either until the CPU turns
+ * MMU on again or other coherent agents move cached data out. That's
+ * why there is no need to clean D-cache before turning MMU off.
+ *
+ * Note 2: We don't have to clean D-cache to protect against malicious
+ * guests, which can execute 'dc isw' (data or unified Cache line
+ * Invalidate by Set/Way) because when virtualization is enabled
+ * (HCR_EL2.VM == 1) then HW automatically upgrade 'dc isw' to
+ * 'dc cisw' (Clean + Invallidate). Executing Clean operation before
+ * Invalidate is safe in guests.
+ */
+

+ /* install bootstrap_vectors */

+ ldr x0, =bootstrap_vectors
+ hvc #0
+ hvc #0 /* bootstrap vectors enter EL2 at el2_entry */
+ b . /* we don't expect to return here */
+
+ /* the bootstrap vector returns us here in physical addressing */
+el2_entry:
+ mrs x1, esr_el2
+ lsr x1, x1, #26
+ cmp x1, #0x16
+ b.ne . /* not hvc */
+
+ /* init bootstrap page tables */
+ bl init_bootstrap_pt
+
+ /* enable temporary mmu mapings for early initialization */
+ adr x0, bootstrap_pt_l0
+ bl enable_mmu_el2
+
+ b .
+
+ .globl enable_mmu_el2
+enable_mmu_el2:
+ /*
+ * x0: u64 ttbr0_el2
+ */
+
+ /* setup the MMU for EL2 hypervisor mappings */
+ ldr x1, =DEFAULT_MAIR_EL2
+ msr mair_el2, x1
+
+ /* AARCH64_TODO: ARM architecture supports CPU clusters which could be
+ * in separate inner shareable domains. At the same time: "The Inner
+ * Shareable domain is expected to be the set of PEs controlled by
+ * a single hypervisor or operating system." (see p. 93 of ARM ARM)
+ * We should think what hw configuration we support by one instance of
+ * the hypervisor and choose Inner or Outter sharable domain.
+ */
+ ldr x1, =(T0SZ(48) | (TCR_RGN_WB_WA << TCR_IRGN0_SHIFT) \

+ | (TCR_RGN_WB_WA << TCR_ORGN0_SHIFT) \
+ | (TCR_INNER_SHAREABLE << TCR_SH0_SHIFT)\

+ | (TCR_PS_48B << TCR_PS_SHIFT) \
+ | TCR_EL2_RES1)
+ msr tcr_el2, x1
+
+ msr ttbr0_el2, x0
+
+ isb
+ tlbi alle2
+ dsb nsh
+
+ /* Enable MMU, allow cacheability for instructions and data */
+ ldr x1, =(SCTLR_I_BIT | SCTLR_C_BIT | SCTLR_M_BIT | SCTLR_EL2_RES1)
+ msr sctlr_el2, x1
+
+ isb
+ tlbi alle2
+ dsb nsh
+
ret
+
+/*
+ * macros used by init_bootstrap_pt
+ */
+
+/* clobbers x8,x9 */
+.macro set_pte table, xidx, xval, flags
+ add x8, \xval, #(\flags)
+ adr x9, \table
+ add x9, x9, \xidx, lsl #3
+ str x8, [x9]
+.endm
+
+/* clobbers x8,x9 */
+.macro set_block table, index, addr, lvl
+ and x8, \addr, \
+ #(((1 << ((\lvl + 1) * 9)) - 1) << (12 + (3 - \lvl) * 9))
+ set_pte \table, \index, x8, PAGE_DEFAULT_FLAGS
+.endm
+
+/* clobbers x8,x9 */
+.macro set_block_dev table, index, addr, lvl
+ and x8, \addr, \
+ #(((1 << ((\lvl + 1) * 9)) - 1) << (12 + (3 - \lvl) * 9))
+ set_pte \table, \index, x8, (PAGE_DEFAULT_FLAGS|PAGE_FLAG_DEVICE)
+.endm
+
+/* clobbers x8,x9 */
+.macro set_table parent, index, child
+ adr x8, \child
+ set_pte \parent, \index, x8, PTE_TABLE_FLAGS
+.endm
+
+.macro get_index idx, addr, lvl
+ ubfx \idx, \addr, #(12 + (3 - \lvl) * 9), 9
+.endm
+
+init_bootstrap_pt:
+ /*
+ * Initialize early page tables to bootstrap the
+ * initialization process. These tables will be replaced
+ * during hypervisor initialization.
+ *
+ * x0: physical address of hypervisor binary (2mb block)
+ * x1: physical address of uart to map (2mb block)
+ *
+ * These are referenced statically for now.
+ * AARCH64_TODO: remove the build time dependency, and take
+ * these values as input from the system configuration.
+ *
+ * Clobbers x0-x4,x8,x9
+ */
+ ldr x0, =JAILHOUSE_BASE
+ ldr x1, =UART_BASE
+
+ /* l0 pt index for firmware and uart */
+ get_index x2, x0, 0
+ get_index x3, x1, 0
+
+ /* map the l1 table that includes the firmware */
+ set_table bootstrap_pt_l0, x2, bootstrap_pt_l1
+
+ cmp x2, x3
+ b.eq 1f
+
+ /*
+ * Case 1: firmware and uart reside on sepparate l0 entries
+ * (512gb regions). The wildcard table is used as an
+ * l1 table for the uart.
+ */
+ get_index x2, x0, 1
+ set_block bootstrap_pt_l1, x2, x0, 1 /* 1gb block for firmware */
+
+ /* 512gb blocks are not supported by the hardware. Use the
+ * wildcard table to map a 1gb block for the uart */
+ set_table bootstrap_pt_l0, x3, bootstrap_pt_wildcard
+ get_index x3, x1, 1
+ set_block_dev bootstrap_pt_wildcard, x3, x1, 1
+
+ b flush
+
+1: get_index x2, x0, 1
+ get_index x3, x1, 1
+ cmp x2, x3
+ b.eq 1f
+
+ /*
+ * Case 2: firwmare and uart reside on sepparate l1 entries.
+ * Just map 1gb blocks, we don't need the wildcard.
+ */
+ set_block bootstrap_pt_l1, x2, x0, 1
+ set_block_dev bootstrap_pt_l1, x3, x1, 1
+
+ b flush
+
+ /* l1 granularity not enough; attempt to map on l2 blocks (2mb) */
+1: set_table bootstrap_pt_l1, x2, bootstrap_pt_wildcard
+ get_index x2, x0, 2
+ get_index x3, x1, 2
+ cmp x2, x3
+ b.eq 1f
+
+ /*
+ * Case 3: firmware and uart reside on sepparate l2 entries,
+ * we can still salvage the situation (2mb blocks).
+ * We use the wildcard table for the l2 table for
+ * the firmware and the uart.
+ */
+ set_block bootstrap_pt_wildcard, x2, x0, 2
+ set_block_dev bootstrap_pt_wildcard, x3, x1, 2
+ b flush
+
+ /* uart and firmware within same 2MB block; cry now */
+1: b .
+
+flush: adr x0, bootstrap_pt_l0
+ mov x1, PAGE_SIZE * 3
+ b arch_paging_flush_cpu_caches // will ret to caller
+
+.macro ventry label
+ .align 7
+ b \label
+.endm
+
+ .globl bootstrap_vectors
+ .align 11
+bootstrap_vectors:
+ ventry .
+ ventry .
+ ventry .
+ ventry .
+
+ ventry .
+ ventry .
+ ventry .
+ ventry .
+
+ ventry el2_entry
+ ventry .
+ ventry .
+ ventry .
+
+ ventry .
+ ventry .
+ ventry .
+ ventry .
diff --git a/hypervisor/arch/arm64/include/asm/sections.h b/hypervisor/arch/arm64/include/asm/sections.h
new file mode 100644
index 0000000..3f2e18e
--- /dev/null
+++ b/hypervisor/arch/arm64/include/asm/sections.h

@@ -0,0 +1,29 @@
+/*
+ * Jailhouse AArch64 support
+ *

+ * Copyright (C) 2015-2016 Huawei Technologies Duesseldorf GmbH

+ *
+ * Authors:
+ * Antonios Motakis <antonios...@huawei.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2. See
+ * the COPYING file in the top-level directory.
+ */
+

+/* We have no memory management during early init; three pages is the
+ * minimum we can get away with to switch on the MMU with identity
+ * mapping for they hypervisor firmware and the UART.
+ *
+ * TODO: find a way to avoid having these three empty pages in the
+ * Jailhouse binary!
+ */
+#define ARCH_SECTIONS \
+ . = ALIGN(PAGE_SIZE); \
+ .bootstrap_page_tables : { \
+ bootstrap_pt_l0 = .; \
+ . = . + PAGE_SIZE; \
+ bootstrap_pt_l1 = .; \
+ . = . + PAGE_SIZE; \
+ bootstrap_pt_wildcard = .; \
+ . = . + PAGE_SIZE; \
+ }
--
2.7.0

[antonios...@huawei.com]

From: Antonios Motakis <antonios...@huawei.com>

The arch_paging_flush_cpu_caches function is needed early on
during initialization, in order to flush the early bootstrap
page tables for they hypervisor.

Signed-off-by: Antonios Motakis <antonios...@huawei.com>
---

hypervisor/arch/arm64/caches.S | 34 +++++++++++++++++++++++++++++++++-
1 file changed, 33 insertions(+), 1 deletion(-)

diff --git a/hypervisor/arch/arm64/caches.S b/hypervisor/arch/arm64/caches.S
index 4859032..d70b530 100644
--- a/hypervisor/arch/arm64/caches.S
+++ b/hypervisor/arch/arm64/caches.S
@@ -9,8 +9,40 @@

* This work is licensed under the terms of the GNU GPL, version 2. See

* the COPYING file in the top-level directory.
*

+ * Implementation copied from Linux source files:
+ * - arch/arm64/mm/cache.S
+ * - arch/arm64/mm/proc-macros.S
*/

+/*
+ * dcache_line_size - get the minimum D-cache line size from the CTR register.
+ */
+ .macro dcache_line_size, reg, tmp
+ mrs \tmp, ctr_el0 // read CTR
+ ubfm \tmp, \tmp, #16, #19 // cache line size encoding
+ mov \reg, #4 // bytes per word
+ lsl \reg, \reg, \tmp // actual cache line size
+ .endm
+
+/*
+ * arch_paging_flush_cpu_caches(addr, size)
+ *
+ * Ensure that the data held in the page addr is written back to the
+ * page in question.
+ *
+ * - addr - address
+ * - size - size in question
+ */
.global arch_paging_flush_cpu_caches
arch_paging_flush_cpu_caches:
- b .
+ dcache_line_size x2, x3
+ add x1, x0, x1
+ sub x3, x2, #1
+ bic x0, x0, x3
+1: dc civac, x0 // clean & invalidate D line
+ // unified line
+ add x0, x0, x2
+ cmp x0, x1
+ b.lo 1b
+ dsb sy
+ ret
--
2.7.0

[antonios...@huawei.com]

From: Antonios Motakis <antonios...@huawei.com>

Catch accesses to the mmio regions that we want to handle from the
hypervisor. These are used also by the GIC code.

Signed-off-by: Antonios Motakis <antonios...@huawei.com>
Signed-off-by: Dmitry Voytik <dmitry...@huawei.com>
---

hypervisor/arch/arm64/include/asm/traps.h | 2 +
hypervisor/arch/arm64/mmio.c | 84 ++++++++++++++++++++++++++++++-
hypervisor/arch/arm64/traps.c | 10 +++-
3 files changed, 93 insertions(+), 3 deletions(-)

diff --git a/hypervisor/arch/arm64/include/asm/traps.h b/hypervisor/arch/arm64/include/asm/traps.h
index 3a60e30..2f2e0f6 100644
--- a/hypervisor/arch/arm64/include/asm/traps.h
+++ b/hypervisor/arch/arm64/include/asm/traps.h
@@ -31,5 +31,7 @@ struct trap_context {

void arch_skip_instruction(struct trap_context *ctx);

+int arch_handle_dabt(struct trap_context *ctx);
+
#endif /* !__ASSEMBLY__ */
#endif /* !_JAILHOUSE_ASM_TRAPS_H */
diff --git a/hypervisor/arch/arm64/mmio.c b/hypervisor/arch/arm64/mmio.c
index 37745d7..0c43365 100644
--- a/hypervisor/arch/arm64/mmio.c
+++ b/hypervisor/arch/arm64/mmio.c
@@ -2,10 +2,14 @@
* Jailhouse AArch64 support
*
* Copyright (C) 2015 Huawei Technologies Duesseldorf GmbH
+ * Copyright (C) 2014 ARM Limited
*
* Authors:
* Antonios Motakis <antonios...@huawei.com>
*
+ * Part of the fuctionality is derived from the AArch32 implementation, under
+ * hypervisor/arch/arm/mmio.c by Jean-Philippe Brucker.
+ *

* This work is licensed under the terms of the GNU GPL, version 2. See
* the COPYING file in the top-level directory.

*/
@@ -13,6 +17,12 @@
#include <jailhouse/entry.h>
#include <jailhouse/mmio.h>
#include <jailhouse/printk.h>
+#include <asm/bitops.h>
+#include <asm/percpu.h>
+#include <asm/sysregs.h>
+#include <asm/traps.h>
+
+/* AARCH64_TODO: consider merging this with the AArch32 version */

unsigned int arch_mmio_count_regions(struct cell *cell)
{
@@ -20,8 +30,80 @@ unsigned int arch_mmio_count_regions(struct cell *cell)
return 0;
}

-void arm_mmio_perform_access(unsigned long base, struct mmio_access *mmio)
+static void arch_inject_dabt(struct trap_context *ctx, unsigned long addr)
{
trace_error(-EINVAL);
while (1);
}
+
+int arch_handle_dabt(struct trap_context *ctx)
+{
+ enum mmio_result mmio_result;
+ struct mmio_access mmio;
+ unsigned long hpfar;
+ unsigned long hdfar;
+ /* Decode the syndrome fields */
+ u32 iss = ESR_ISS(ctx->esr);
+ u32 isv = iss >> 24;
+ u32 sas = iss >> 22 & 0x3;
+ u32 sse = iss >> 21 & 0x1;
+ u32 srt = iss >> 16 & 0x1f;
+ u32 ea = iss >> 9 & 0x1;
+ u32 cm = iss >> 8 & 0x1;
+ u32 s1ptw = iss >> 7 & 0x1;
+ u32 is_write = iss >> 6 & 0x1;
+ u32 size = 1 << sas;
+
+ arm_read_sysreg(HPFAR_EL2, hpfar);
+ arm_read_sysreg(FAR_EL2, hdfar);
+ mmio.address = hpfar << 8;
+ mmio.address |= hdfar & 0xfff;
+
+ this_cpu_data()->stats[JAILHOUSE_CPU_STAT_VMEXITS_MMIO]++;
+
+ /*
+ * Invalid instruction syndrome means multiple access or writeback,
+ * there is nothing we can do.
+ */
+ if (!isv)
+ goto error_unhandled;
+
+ /* Re-inject abort during page walk, cache maintenance or external */
+ if (s1ptw || ea || cm) {
+ arch_inject_dabt(ctx, hdfar);
+ return TRAP_HANDLED;
+ }
+
+ if (is_write) {
+ /* Load the value to write from the src register */
+ mmio.value = (srt == 31) ? 0 : ctx->regs[srt];
+ if (sse)
+ mmio.value = sign_extend(mmio.value, 8 * size);
+ } else {
+ mmio.value = 0;
+ }
+ mmio.is_write = is_write;
+ mmio.size = size;
+
+ mmio_result = mmio_handle_access(&mmio);
+ if (mmio_result == MMIO_ERROR)
+ return TRAP_FORBIDDEN;
+ if (mmio_result == MMIO_UNHANDLED)
+ goto error_unhandled;
+
+ /* Put the read value into the dest register */
+ if (!is_write && (srt != 31)) {
+ if (sse)
+ mmio.value = sign_extend(mmio.value, 8 * size);
+ ctx->regs[srt] = mmio.value;
+ }
+
+ arch_skip_instruction(ctx);
+ return TRAP_HANDLED;
+
+error_unhandled:
+ panic_printk("Unhandled data %s at 0x%x(%d)\n",
+ (is_write ? "write" : "read"), mmio.address, size);
+
+ return TRAP_UNHANDLED;
+}
diff --git a/hypervisor/arch/arm64/traps.c b/hypervisor/arch/arm64/traps.c
index 199b497..cc6fe6c 100644
--- a/hypervisor/arch/arm64/traps.c
+++ b/hypervisor/arch/arm64/traps.c
@@ -15,6 +15,7 @@
#include <jailhouse/printk.h>
#include <asm/control.h>
#include <asm/gic_common.h>
+#include <asm/mmio.h>
#include <asm/platform.h>
#include <asm/psci.h>
#include <asm/sysregs.h>
@@ -23,8 +24,9 @@

void arch_skip_instruction(struct trap_context *ctx)
{
- trace_error(-EINVAL);
- while(1);
+ u32 instruction_length = ESR_IL(ctx->esr);
+
+ ctx->pc += (instruction_length ? 4 : 2);
}

static void dump_regs(struct trap_context *ctx)
@@ -105,6 +107,10 @@ static void arch_handle_trap(struct per_cpu *cpu_data,

/* exception class */
switch (ESR_EC(ctx.esr)) {
+ case ESR_EC_DABT_LOW:
+ ret = arch_handle_dabt(&ctx);
+ break;
+
default:
ret = TRAP_UNHANDLED;
}
--
2.7.0

[antonios...@huawei.com]

From: Antonios Motakis <antonios...@huawei.com>

Catch exceptions on the AArch64 target of Jailhouse. Catch and aborts
from EL2 that might be caused by the hypervisor.

Signed-off-by: Antonios Motakis <antonios...@huawei.com>
Signed-off-by: Dmitry Voytik <dmitry...@huawei.com>
---

hypervisor/arch/arm64/Makefile | 1 +
hypervisor/arch/arm64/entry.S | 4 +
hypervisor/arch/arm64/exception.S | 96 ++++++++++++++++++++++++
hypervisor/arch/arm64/include/asm/traps.h | 35 +++++++++
hypervisor/arch/arm64/traps.c | 119 ++++++++++++++++++++++++++++++
5 files changed, 255 insertions(+)
create mode 100644 hypervisor/arch/arm64/exception.S
create mode 100644 hypervisor/arch/arm64/include/asm/traps.h
create mode 100644 hypervisor/arch/arm64/traps.c

diff --git a/hypervisor/arch/arm64/Makefile b/hypervisor/arch/arm64/Makefile
index 0911ff3..603056e 100644
--- a/hypervisor/arch/arm64/Makefile
+++ b/hypervisor/arch/arm64/Makefile
@@ -18,5 +18,6 @@ always := built-in.o

obj-y := entry.o setup.o control.o mmio.o caches.o

obj-y += ../arm/mmu_cell.o ../arm/paging.o ../arm/dbg-write.o ../arm/lib.o

+obj-y += exception.o traps.o

obj-$(CONFIG_SERIAL_AMBA_PL011) += ../arm/dbg-write-pl011.o
diff --git a/hypervisor/arch/arm64/entry.S b/hypervisor/arch/arm64/entry.S
index 346e91a..fda7164 100644
--- a/hypervisor/arch/arm64/entry.S
+++ b/hypervisor/arch/arm64/entry.S
@@ -63,6 +63,10 @@ el2_entry:
cmp x1, #0x16

b.ne . /* not hvc */

+ /* install jailhouse vectors */
+ adr x1, hyp_vectors
+ msr vbar_el2, x1

+
/* init bootstrap page tables */

bl init_bootstrap_pt

diff --git a/hypervisor/arch/arm64/exception.S b/hypervisor/arch/arm64/exception.S
new file mode 100644
index 0000000..943be9b
--- /dev/null
+++ b/hypervisor/arch/arm64/exception.S
@@ -0,0 +1,96 @@

+/*
+ * Jailhouse AArch64 support
+ *

+ * Copyright (C) 2015 Huawei Technologies Duesseldorf GmbH
+ *

+ * Authors:
+ * Antonios Motakis <antonios...@huawei.com>
+ *

+ * This work is licensed under the terms of the GNU GPL, version 2. See
+ * the COPYING file in the top-level directory.
+ */
+
+#include <asm/head.h>
+#include <asm/processor.h>
+#include <asm/sysregs.h>

+
+.macro ventry label
+ .align 7
+ b \label
+.endm
+

+.macro handle_vmexit exit_reason
+ .align 7
+ /* Fill the struct registers. Should comply with NUM_USR_REGS */
+ stp x29, x30, [sp, #-16]!

+ stp x27, x28, [sp, #-16]!
+ stp x25, x26, [sp, #-16]!
+ stp x23, x24, [sp, #-16]!
+ stp x21, x22, [sp, #-16]!
+ stp x19, x20, [sp, #-16]!

+ stp x17, x18, [sp, #-16]!
+ stp x15, x16, [sp, #-16]!
+ stp x13, x14, [sp, #-16]!
+ stp x11, x12, [sp, #-16]!
+ stp x9, x10, [sp, #-16]!
+ stp x7, x8, [sp, #-16]!
+ stp x5, x6, [sp, #-16]!
+ stp x3, x4, [sp, #-16]!
+ stp x1, x2, [sp, #-16]!
+
+ mov x1, #\exit_reason
+ stp x1, x0, [sp, #-16]!

+
+ mov x29, xzr /* reset fp,lr */
+ mov x30, xzr

+ mrs x0, tpidr_el2
+ mov x1, sp
+ bl arch_handle_exit
+ b .
+.endm
+
+ .text
+ .globl hyp_vectors
+ .align 11
+hyp_vectors:

+ ventry .
+ ventry .
+ ventry .
+ ventry .
+

+ handle_vmexit EXIT_REASON_EL2_ABORT

+ ventry .
+ ventry .
+ ventry .
+

+ handle_vmexit EXIT_REASON_EL1_ABORT
+ handle_vmexit EXIT_REASON_EL1_IRQ

+ ventry .
+ ventry .
+
+ ventry .
+ ventry .
+ ventry .
+ ventry .
+

+ .globl vmreturn
+vmreturn:
+ /* x0: struct registers* */
+ mov sp, x0
+ ldp x1, x0, [sp], #16 /* x1 is the exit_reason */
+ ldp x1, x2, [sp], #16
+ ldp x3, x4, [sp], #16
+ ldp x5, x6, [sp], #16
+ ldp x7, x8, [sp], #16
+ ldp x9, x10, [sp], #16
+ ldp x11, x12, [sp], #16
+ ldp x13, x14, [sp], #16
+ ldp x15, x16, [sp], #16
+ ldp x17, x18, [sp], #16
+ ldp x19, x20, [sp], #16
+ ldp x21, x22, [sp], #16
+ ldp x23, x24, [sp], #16
+ ldp x25, x26, [sp], #16
+ ldp x27, x28, [sp], #16
+ ldp x29, x30, [sp], #16
+ eret
diff --git a/hypervisor/arch/arm64/include/asm/traps.h b/hypervisor/arch/arm64/include/asm/traps.h
new file mode 100644
index 0000000..3a60e30
--- /dev/null
+++ b/hypervisor/arch/arm64/include/asm/traps.h
@@ -0,0 +1,35 @@

+/*
+ * Jailhouse AArch64 support
+ *

+ * Copyright (C) 2015 Huawei Technologies Duesseldorf GmbH
+ *

+ * Authors:
+ * Antonios Motakis <antonios...@huawei.com>
+ *

+ * This work is licensed under the terms of the GNU GPL, version 2. See
+ * the COPYING file in the top-level directory.
+ */
+
+#ifndef _JAILHOUSE_ASM_TRAPS_H
+#define _JAILHOUSE_ASM_TRAPS_H
+
+#ifndef __ASSEMBLY__
+
+enum trap_return {
+ TRAP_HANDLED = 1,
+ TRAP_UNHANDLED = 0,
+ TRAP_FORBIDDEN = -1,
+};
+
+struct trap_context {
+ unsigned long *regs;
+ u64 esr;
+ u64 spsr;
+ u64 pc;
+ u64 sp;
+};
+
+void arch_skip_instruction(struct trap_context *ctx);

+
+#endif /* !__ASSEMBLY__ */

+#endif /* !_JAILHOUSE_ASM_TRAPS_H */
diff --git a/hypervisor/arch/arm64/traps.c b/hypervisor/arch/arm64/traps.c
new file mode 100644
index 0000000..b27bb2e
--- /dev/null
+++ b/hypervisor/arch/arm64/traps.c
@@ -0,0 +1,119 @@

+/*
+ * Jailhouse AArch64 support
+ *

+ * Copyright (C) 2015 Huawei Technologies Duesseldorf GmbH
+ *

+ * Authors:
+ * Antonios Motakis <antonios...@huawei.com>

+ * Dmitry Voytik <dmitry...@huawei.com>
+ *

+ * This work is licensed under the terms of the GNU GPL, version 2. See
+ * the COPYING file in the top-level directory.
+ */
+
+#include <jailhouse/control.h>
+#include <jailhouse/printk.h>

+#include <asm/control.h>
+#include <asm/gic_common.h>
+#include <asm/platform.h>
+#include <asm/psci.h>
+#include <asm/sysregs.h>
+#include <asm/traps.h>
+#include <asm/processor.h>
+
+void arch_skip_instruction(struct trap_context *ctx)
+{
+ trace_error(-EINVAL);
+ while(1);
+}
+
+static void dump_regs(struct trap_context *ctx)
+{
+ unsigned char i;
+
+ panic_printk(" pc: %016lx lr: %016lx spsr: %08lx EL%1d\n"
+ " sp: %016lx esr: %02x %01x %07lx\n",
+ ctx->pc, ctx->regs[30], ctx->spsr, SPSR_EL(ctx->spsr),
+ ctx->sp, ESR_EC(ctx->esr), ESR_IL(ctx->esr),
+ ESR_ISS(ctx->esr));
+ for (i = 0; i < NUM_USR_REGS - 1; i++)
+ panic_printk("%sx%d: %016lx%s", i < 10 ? " " : "", i,
+ ctx->regs[i], i % 3 == 2 ? "\n" : " ");
+ panic_printk("\n");
+}
+
+static void fill_trap_context(struct trap_context *ctx, struct registers *regs)
+{
+ arm_read_sysreg(ELR_EL2, ctx->pc);
+ arm_read_sysreg(SPSR_EL2, ctx->spsr);
+ switch (SPSR_EL(ctx->spsr)) { /* exception level */
+ case 0:
+ arm_read_sysreg(SP_EL0, ctx->sp); break;
+ case 1:
+ arm_read_sysreg(SP_EL1, ctx->sp); break;
+ case 2:
+ arm_read_sysreg(SP_EL2, ctx->sp); break;
+ default:
+ ctx->sp = 0; break; /* should never happen */
+ }
+ arm_read_sysreg(ESR_EL2, ctx->esr);
+ ctx->regs = regs->usr;
+}
+
+static void arch_handle_trap(struct per_cpu *cpu_data,
+ struct registers *guest_regs)
+{
+ struct trap_context ctx;
+ int ret;
+
+ fill_trap_context(&ctx, guest_regs);
+
+ /* exception class */
+ switch (ESR_EC(ctx.esr)) {
+ default:
+ ret = TRAP_UNHANDLED;
+ }
+
+ if (ret == TRAP_UNHANDLED || ret == TRAP_FORBIDDEN) {
+ panic_printk("\nFATAL: exception %s\n", (ret == TRAP_UNHANDLED ?
+ "unhandled trap" :
+ "forbidden access"));
+ panic_printk("Cell state before exception:\n");
+ dump_regs(&ctx);
+ panic_park();
+ }
+
+ arm_write_sysreg(ELR_EL2, ctx.pc);
+}
+
+static void arch_dump_exit(struct registers *regs, const char *reason)
+{
+ struct trap_context ctx;
+
+ fill_trap_context(&ctx, regs);
+ panic_printk("\nFATAL: Unhandled HYP exception: %s\n", reason);
+ dump_regs(&ctx);
+}
+
+struct registers *arch_handle_exit(struct per_cpu *cpu_data,
+ struct registers *regs)
+{
+ cpu_data->stats[JAILHOUSE_CPU_STAT_VMEXITS_TOTAL]++;
+
+ switch (regs->exit_reason) {
+ case EXIT_REASON_EL1_ABORT:
+ arch_handle_trap(cpu_data, regs);
+ break;
+
+ case EXIT_REASON_EL2_ABORT:
+ arch_dump_exit(regs, "synchronous abort from EL2");
+ panic_stop();
+ break;
+
+ default:
+ arch_dump_exit(regs, "unexpected");
+ panic_stop();
+ }
+
+ vmreturn(regs);
+}
--
2.7.0

[antonios...@huawei.com]

From: Antonios Motakis <antonios...@huawei.com>

Add hypervisor disable support to the Jailhouse firmware. Handle
Jailhouse disable calls from the root cell, and also disable the
hypervisor in case of an error during initialization.

Signed-off-by: Antonios Motakis <antonios...@huawei.com>
---

hypervisor/arch/arm64/control.c | 8 +++++--
hypervisor/arch/arm64/entry.S | 33 +++++++++++++++++++++++++++
hypervisor/arch/arm64/include/asm/percpu.h | 1 +
hypervisor/arch/arm64/setup.c | 36 ++++++++++++++++++++++++++++--
hypervisor/arch/arm64/traps.c | 4 ++++
5 files changed, 78 insertions(+), 4 deletions(-)

diff --git a/hypervisor/arch/arm64/control.c b/hypervisor/arch/arm64/control.c
index 7f11701..023787f 100644
--- a/hypervisor/arch/arm64/control.c
+++ b/hypervisor/arch/arm64/control.c
@@ -124,8 +124,12 @@ void arch_config_commit(struct cell *cell_added_removed)

void arch_shutdown(void)
{
- trace_error(-EINVAL);
- while (1);
+ unsigned int cpu;
+
+ /* turn off the hypervisor when we return from the exit handler */
+ if (root_cell.cpu_set)
+ for_each_cpu(cpu, root_cell.cpu_set)
+ per_cpu(cpu)->shutdown = true;
}

void arch_suspend_cpu(unsigned int cpu_id)
diff --git a/hypervisor/arch/arm64/entry.S b/hypervisor/arch/arm64/entry.S
index fda7164..b1bcd9c 100644
--- a/hypervisor/arch/arm64/entry.S
+++ b/hypervisor/arch/arm64/entry.S
@@ -110,6 +110,39 @@ el2_entry:
bl entry
b .

+ .globl arch_shutdown_mmu
+arch_shutdown_mmu:
+ /* x0: struct percpu* */
+ mov x19, x0
+
+ /* Note: no memory accesses must be done after turning MMU off. There
+ * is non-zero probability that cached data can be not syncronized with
+ * system memory. CPU can access data bypassing D-cache when MMU is off.
+ */
+
+ /* hand over control of EL2 back to Linux */
+ add x1, x19, #PERCPU_LINUX_SAVED_VECTORS
+ ldr x2, [x1]
+ msr vbar_el2, x2
+
+ /* disable the hypervisor MMU */
+ mrs x1, sctlr_el2
+ ldr x2, =(SCTLR_M_BIT | SCTLR_C_BIT | SCTLR_I_BIT)
+ bic x1, x1, x2
+ msr sctlr_el2, x1
+ isb
+
+ msr mair_el2, xzr
+ msr ttbr0_el2, xzr
+ msr tcr_el2, xzr
+ isb
+
+ msr tpidr_el2, xzr
+
+ /* Call vmreturn(guest_registers) */
+ add x0, x19, #(PERCPU_STACK_END - 32 * 8)
+ b vmreturn
+
.globl enable_mmu_el2
enable_mmu_el2:
/*
diff --git a/hypervisor/arch/arm64/include/asm/percpu.h b/hypervisor/arch/arm64/include/asm/percpu.h
index b30be6f..667cf8d 100644
--- a/hypervisor/arch/arm64/include/asm/percpu.h
+++ b/hypervisor/arch/arm64/include/asm/percpu.h
@@ -67,6 +67,7 @@ struct per_cpu {

unsigned int virt_id;
union mpidr mpidr;
+ bool shutdown;
} __attribute__((aligned(PAGE_SIZE)));

static inline struct per_cpu *this_cpu_data(void)

diff --git a/hypervisor/arch/arm64/setup.c b/hypervisor/arch/arm64/setup.c
index a6b35ab..95fe23f 100644
--- a/hypervisor/arch/arm64/setup.c
+++ b/hypervisor/arch/arm64/setup.c
@@ -87,8 +87,40 @@ int arch_unmap_device(void *vaddr, unsigned long size)
PAGING_NON_COHERENT);
}

+/* disable the hypervisor on the current CPU */
+void arch_shutdown_self(struct per_cpu *cpu_data)
+{
+ irqchip_cpu_shutdown(cpu_data);
+
+ /* Free the guest */
+ arm_write_sysreg(HCR_EL2, HCR_RW_BIT);
+ arm_write_sysreg(VTCR_EL2, VTCR_RES1);
+
+ /* Remove stage-2 mappings */
+ arch_cpu_tlb_flush(cpu_data);
+
+ /* TLB flush needs the cell's VMID */
+ isb();
+ arm_write_sysreg(VTTBR_EL2, 0);
+
+ /* we will restore the root cell state with the MMU turned off,
+ * so we need to make sure it has been commited to memory */
+ arch_paging_flush_cpu_caches(guest_regs(cpu_data),
+ sizeof(struct registers));
+ dsb(ish);
+
+ /* Return to EL1 */
+ arch_shutdown_mmu(cpu_data);

+}
+
void arch_cpu_restore(struct per_cpu *cpu_data, int return_code)
{

- trace_error(-EINVAL);
- while (1);
+ struct registers *regs = guest_regs(cpu_data);
+
+ /* Jailhouse initialization failed; return to the caller in EL1 */
+ arm_write_sysreg(ELR_EL2, regs->usr[30]);
+
+ regs->usr[0] = return_code;
+
+ arch_shutdown_self(cpu_data);
}
diff --git a/hypervisor/arch/arm64/traps.c b/hypervisor/arch/arm64/traps.c
index 8797165..afedab8 100644
--- a/hypervisor/arch/arm64/traps.c
+++ b/hypervisor/arch/arm64/traps.c
@@ -195,5 +195,9 @@ struct registers *arch_handle_exit(struct per_cpu *cpu_data,
panic_stop();
}

+ if (cpu_data->shutdown)
+ /* Won't return here. */
+ arch_shutdown_self(cpu_data);
+
vmreturn(regs);
}
--
2.7.0

[antonios...@huawei.com]

From: Antonios Motakis <antonios...@huawei.com>

This patch mostly implements the functionality needed to create
and control new cells. The functionality is very similar to the
one from AArch32, and there is potential to unify some code in
the future between the architectures.

Signed-off-by: Antonios Motakis <antonios...@huawei.com>
---

hypervisor/arch/arm64/control.c | 161 ++++++++++++++++++++++++++-----
hypervisor/arch/arm64/include/asm/cell.h | 1 +
2 files changed, 136 insertions(+), 26 deletions(-)

diff --git a/hypervisor/arch/arm64/control.c b/hypervisor/arch/arm64/control.c
index 023787f..88ef44f 100644
--- a/hypervisor/arch/arm64/control.c
+++ b/hypervisor/arch/arm64/control.c
@@ -72,50 +72,114 @@ static void arch_reset_el1(struct registers *regs)
void arch_reset_self(struct per_cpu *cpu_data)
{
int err = 0;
- unsigned long reset_address;
+ unsigned long reset_address = 0;
struct cell *cell = cpu_data->cell;

struct registers *regs = guest_regs(cpu_data);

+ bool is_shutdown = cpu_data->shutdown;

- if (cell != &root_cell) {
- trace_error(-EINVAL);
- panic_stop();
- }
+ if (!is_shutdown)
+ err = arch_mmu_cpu_cell_init(cpu_data);
+ if (err)
+ printk("MMU setup failed\n");

/*
* Note: D-cache cleaning and I-cache invalidation is done on driver
* level after image is loaded.
*/

- err = irqchip_cpu_reset(cpu_data);
- if (err)
- printk("IRQ setup failed\n");
+ /*
+ * We come from the IRQ handler, but we won't return there, so the IPI
+ * is deactivated here.
+ */
+ irqchip_eoi_irq(SGI_CPU_OFF, true);
+
+ if (is_shutdown) {
+ if (cell != &root_cell) {
+ irqchip_cpu_shutdown(cpu_data);
+
+ smc(PSCI_CPU_OFF, 0, 0, 0);
+ panic_printk("FATAL: PSCI_CPU_OFF failed\n");
+ panic_stop();
+ }
+ /* arch_shutdown_self resets the GIC on all remaining CPUs. */
+ } else {
+ err = irqchip_cpu_reset(cpu_data);
+ if (err)
+ printk("IRQ setup failed\n");
+ }

/* Wait for the driver to call cpu_up */
- reset_address = psci_emulate_spin(cpu_data);
+ if (cpu_data->virt_id != 0)
+ reset_address = psci_emulate_spin(cpu_data);

/* Restore an empty context */
arch_reset_el1(regs);

arm_write_sysreg(ELR_EL2, reset_address);

+ if (is_shutdown)

+ /* Won't return here. */
+ arch_shutdown_self(cpu_data);
+
vmreturn(regs);
}

int arch_cell_create(struct cell *cell)
{
- return trace_error(-EINVAL);
+ int err;
+ unsigned int cpu;
+ unsigned int virt_id = 0;
+
+ err = arch_mmu_cell_init(cell);
+ if (err)
+ return err;
+
+ /*
+ * Generate a virtual CPU id according to the position of each CPU in
+ * the cell set
+ */
+ for_each_cpu(cpu, cell->cpu_set) {
+ per_cpu(cpu)->virt_id = virt_id;
+ virt_id++;
+ }
+ cell->arch.last_virt_id = virt_id - 1;
+
+ err = irqchip_cell_init(cell);
+ if (err) {
+ arch_mmu_cell_destroy(cell);
+ return err;
+ }
+ irqchip_root_cell_shrink(cell);
+
+ return 0;
}

void arch_flush_cell_vcpu_caches(struct cell *cell)
{
- /* AARCH64_TODO */
- trace_error(-EINVAL);

+ unsigned int cpu;
+

+ for_each_cpu(cpu, cell->cpu_set)
+ if (cpu == this_cpu_id())
+ arch_cpu_tlb_flush(per_cpu(cpu));
+ else
+ per_cpu(cpu)->flush_vcpu_caches = true;
}

void arch_cell_destroy(struct cell *cell)

{
- trace_error(-EINVAL);
- while (1);
+ unsigned int cpu;

+ struct per_cpu *percpu;
+
+ for_each_cpu(cpu, cell->cpu_set) {
+ percpu = per_cpu(cpu);
+ /* Re-assign the physical IDs for the root cell */
+ percpu->virt_id = percpu->cpu_id;
+ arch_reset_cpu(cpu);
+ }
+
+ irqchip_cell_exit(cell);
+
+ arch_mmu_cell_destroy(cell);
}

void arch_config_commit(struct cell *cell_added_removed)
@@ -134,38 +198,72 @@ void arch_shutdown(void)

void arch_suspend_cpu(unsigned int cpu_id)

{
- trace_error(-EINVAL);
- while (1);

+ struct sgi sgi;
+
+ if (psci_cpu_stopped(cpu_id))
+ return;
+
+ sgi.routing_mode = 0;
+ sgi.aff1 = 0;
+ sgi.aff2 = 0;
+ sgi.aff3 = 0;
+ sgi.targets = 1 << cpu_id;
+ sgi.id = SGI_CPU_OFF;
+
+ irqchip_send_sgi(&sgi);
+
+ psci_wait_cpu_stopped(cpu_id);
}

void arch_resume_cpu(unsigned int cpu_id)

{
- trace_error(-EINVAL);
- while (1);

+ /*
+ * Simply get out of the spin loop by returning to handle_sgi
+ * If the CPU is being reset, it already has left the PSCI idle loop.
+ */
+ if (psci_cpu_stopped(cpu_id))
+ psci_resume(cpu_id);
}

void arch_reset_cpu(unsigned int cpu_id)

{
- trace_error(-EINVAL);
- while (1);

+ unsigned long cpu_data = (unsigned long)per_cpu(cpu_id);
+
+ if (psci_cpu_on(cpu_id, (unsigned long)arch_reset_self, cpu_data))
+ printk("ERROR: unable to reset CPU%d (was running)\n", cpu_id);
}

void arch_park_cpu(unsigned int cpu_id)

{
- trace_error(-EINVAL);
- while (1);

+ struct per_cpu *cpu_data = per_cpu(cpu_id);
+
+ /*
+ * Reset always follows park_cpu, so we just need to make sure that the
+ * CPU is suspended
+ */
+ if (psci_wait_cpu_stopped(cpu_id) != 0)
+ printk("ERROR: CPU%d is supposed to be stopped\n", cpu_id);
+ else
+ cpu_data->cell->arch.needs_flush = true;
}

void arch_shutdown_cpu(unsigned int cpu_id)

{
- trace_error(-EINVAL);
- while (1);

+ struct per_cpu *cpu_data = per_cpu(cpu_id);
+
+ cpu_data->virt_id = cpu_id;
+ cpu_data->shutdown = true;
+
+ if (psci_wait_cpu_stopped(cpu_id))
+ printk("FATAL: unable to stop CPU%d\n", cpu_id);
+
+ arch_reset_cpu(cpu_id);
}

void __attribute__((noreturn)) arch_panic_stop(void)

{
- trace_error(-EINVAL);
- while (1);

+ psci_cpu_off(this_cpu_data());
+ __builtin_unreachable();
}

void arch_panic_park(void)
@@ -174,6 +272,14 @@ void arch_panic_park(void)
while (1);
}

+static void arch_suspend_self(struct per_cpu *cpu_data)
+{
+ psci_suspend(cpu_data);
+
+ if (cpu_data->flush_vcpu_caches)
+ arch_cpu_tlb_flush(cpu_data);
+}
+
void arch_handle_sgi(struct per_cpu *cpu_data, u32 irqn)
{
cpu_data->stats[JAILHOUSE_CPU_STAT_VMEXITS_MANAGEMENT]++;
@@ -182,6 +288,9 @@ void arch_handle_sgi(struct per_cpu *cpu_data, u32 irqn)
case SGI_INJECT:
irqchip_inject_pending(cpu_data);
break;
+ case SGI_CPU_OFF:
+ arch_suspend_self(cpu_data);
+ break;
default:
printk("WARN: unknown SGI received %d\n", irqn);
}
diff --git a/hypervisor/arch/arm64/include/asm/cell.h b/hypervisor/arch/arm64/include/asm/cell.h
index 9a9689e..dbdc7b8 100644
--- a/hypervisor/arch/arm64/include/asm/cell.h
+++ b/hypervisor/arch/arm64/include/asm/cell.h
@@ -28,6 +28,7 @@ struct arch_cell {
bool needs_flush;

u64 spis;
+ unsigned int last_virt_id;
};

extern struct cell root_cell;
--
2.7.0

[antonios...@huawei.com]

From: Antonios Motakis <antonios...@huawei.com>

We have now enough implemented functionality to return to the root
cell. We just need to enable guest traps, which will be handled by
the MMU, MMIO, and GIC code we already plugged to the port. Finally,
we restore the state of the root cell that we previously stored in
the stack.

Signed-off-by: Antonios Motakis <antonios...@huawei.com>
---

hypervisor/arch/arm64/setup.c | 13 +++++++++++--
1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/hypervisor/arch/arm64/setup.c b/hypervisor/arch/arm64/setup.c
index 8dd1095..a6b35ab 100644
--- a/hypervisor/arch/arm64/setup.c
+++ b/hypervisor/arch/arm64/setup.c
@@ -31,6 +31,8 @@ int arch_init_early(void)
int arch_cpu_init(struct per_cpu *cpu_data)
{
int err = 0;
+ unsigned long hcr = HCR_VM_BIT | HCR_IMO_BIT | HCR_FMO_BIT
+ | HCR_TSC_BIT | HCR_TAC_BIT | HCR_RW_BIT;

/* switch to the permanent page tables */

enable_mmu_el2(hv_paging_structs.root_table);
@@ -39,6 +41,9 @@ int arch_cpu_init(struct per_cpu *cpu_data)
cpu_data->virt_id = cpu_data->cpu_id;
arm_read_sysreg(MPIDR_EL1, cpu_data->mpidr.val);

+ /* Setup guest traps */
+ arm_write_sysreg(HCR_EL2, hcr);
+
err = arch_mmu_cpu_cell_init(cpu_data);
if (err)
return err;
@@ -60,8 +65,12 @@ int arch_init_late(void)

void __attribute__((noreturn)) arch_cpu_activate_vmm(struct per_cpu *cpu_data)

{
- trace_error(-EINVAL);
- while (1);

+ struct registers *regs = guest_regs(cpu_data);
+
+ /* return to the caller in Linux */

+ arm_write_sysreg(ELR_EL2, regs->usr[30]);
+

+ vmreturn(regs);

}

int arch_map_device(void *paddr, void *vaddr, unsigned long size)

--
2.7.0

[antonios...@huawei.com]

From: Antonios Motakis <antonios...@huawei.com>

Plug in the core handler for hypercalls, so we can start implementing
the more interesting stuff.

Signed-off-by: Antonios Motakis <antonios...@huawei.com>
---

hypervisor/arch/arm64/traps.c | 9 ++++-----
1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/hypervisor/arch/arm64/traps.c b/hypervisor/arch/arm64/traps.c
index 7a0b24c..8797165 100644
--- a/hypervisor/arch/arm64/traps.c
+++ b/hypervisor/arch/arm64/traps.c
@@ -47,11 +47,10 @@ static int arch_handle_hvc(struct trap_context *ctx)
{
unsigned long *regs = ctx->regs;

- if (!IS_PSCI_FN(regs[0]))
- return TRAP_UNHANDLED;
-
- regs[0] = psci_dispatch(ctx);
- arch_skip_instruction(ctx);
+ if (IS_PSCI_FN(regs[0]))
+ regs[0] = psci_dispatch(ctx);
+ else
+ regs[0] = hypercall(regs[0], regs[1], regs[2]);

return TRAP_HANDLED;
}
--
2.7.0

[antonios...@huawei.com]

From: Antonios Motakis <antonios...@huawei.com>

On AArch64 we pretty much rely on PSCI being present for SMP
support (turning multiple cores on and off). This patch implements
the helpers needed for SMP and plugs in the PSCI code from AArch32.

On AArch64 PSCI calls can be issued via SVC64 hypercalls as well,
contrary to AArch32 which uses SVC32 calls only. We add the changes
necessary to support the hypercalls that are used by a Linux root
cell. CPU hotplug now is still working after we enable Jailhouse.

Signed-off-by: Antonios Motakis <antonios...@huawei.com>
---

hypervisor/arch/arm/include/asm/psci.h | 2 +-
hypervisor/arch/arm/psci.c | 4 +-
hypervisor/arch/arm64/Makefile | 1 +
hypervisor/arch/arm64/control.c | 84 ++++++++++++++++++++++++++++++
hypervisor/arch/arm64/include/asm/percpu.h | 6 +++
hypervisor/arch/arm64/psci_low.S | 65 +++++++++++++++++++++++
hypervisor/arch/arm64/setup.c | 5 ++
hypervisor/arch/arm64/traps.c | 34 ++++++++++++
8 files changed, 199 insertions(+), 2 deletions(-)
create mode 100644 hypervisor/arch/arm64/psci_low.S

diff --git a/hypervisor/arch/arm/include/asm/psci.h b/hypervisor/arch/arm/include/asm/psci.h
index ba0adac..4e4a6c9 100644
--- a/hypervisor/arch/arm/include/asm/psci.h
+++ b/hypervisor/arch/arm/include/asm/psci.h
@@ -48,7 +48,7 @@

#define IS_PSCI_FN(hvc) ((((hvc) >> 24) | 0x40) == 0xc4)

-#define PSCI_INVALID_ADDRESS 0xffffffff
+#define PSCI_INVALID_ADDRESS (-1)

#ifndef __ASSEMBLY__

diff --git a/hypervisor/arch/arm/psci.c b/hypervisor/arch/arm/psci.c
index bc1297d..8d9cca3 100644
--- a/hypervisor/arch/arm/psci.c
+++ b/hypervisor/arch/arm/psci.c
@@ -78,7 +78,7 @@ int psci_wait_cpu_stopped(unsigned int cpu_id)
static long psci_emulate_cpu_on(struct per_cpu *cpu_data,
struct trap_context *ctx)
{
- unsigned int cpu;
+ unsigned long cpu;
struct psci_mbox *mbox;

cpu = arm_cpu_by_mpid(cpu_data->cell, ctx->regs[1]);
@@ -153,10 +153,12 @@ long psci_dispatch(struct trap_context *ctx)
return 0;

case PSCI_CPU_ON_32:
+ case PSCI_CPU_ON_64:
case PSCI_CPU_ON_V0_1_UBOOT:
return psci_emulate_cpu_on(cpu_data, ctx);

case PSCI_AFFINITY_INFO_32:
+ case PSCI_AFFINITY_INFO_64:
return psci_emulate_affinity_info(cpu_data, ctx);

default:
diff --git a/hypervisor/arch/arm64/Makefile b/hypervisor/arch/arm64/Makefile
index 540f7e3..f3a9e57 100644
--- a/hypervisor/arch/arm64/Makefile
+++ b/hypervisor/arch/arm64/Makefile
@@ -20,6 +20,7 @@ obj-y := entry.o setup.o control.o mmio.o caches.o

obj-y += ../arm/mmu_cell.o ../arm/paging.o ../arm/dbg-write.o ../arm/lib.o

obj-y += exception.o traps.o
obj-y += ../arm/irqchip.o ../arm/gic-common.o
+obj-y += ../arm/psci.o psci_low.o

obj-$(CONFIG_SERIAL_AMBA_PL011) += ../arm/dbg-write-pl011.o
obj-$(CONFIG_ARM_GIC) += ../arm/gic-v2.o
diff --git a/hypervisor/arch/arm64/control.c b/hypervisor/arch/arm64/control.c
index edb57a7..7f11701 100644
--- a/hypervisor/arch/arm64/control.c
+++ b/hypervisor/arch/arm64/control.c
@@ -12,11 +12,95 @@

#include <jailhouse/control.h>
#include <jailhouse/printk.h>
+#include <jailhouse/string.h>
#include <asm/control.h>
#include <asm/irqchip.h>
#include <asm/platform.h>
#include <asm/traps.h>

+static void arch_reset_el1(struct registers *regs)
+{
+ /* put the cpu in a reset state */
+ /* AARCH64_TODO: handle big endian support */
+ arm_write_sysreg(SPSR_EL2, RESET_PSR);
+ arm_write_sysreg(SCTLR_EL1, SCTLR_EL1_RES1);
+ arm_write_sysreg(CNTKCTL_EL1, 0);
+ arm_write_sysreg(PMCR_EL0, 0);
+
+ /* wipe any other state to avoid leaking information accross cells */
+ memset(regs, 0, sizeof(struct registers));
+
+ /* AARCH64_TODO: wipe floating point registers */
+
+ /* wipe special registers */
+ arm_write_sysreg(SP_EL0, 0);
+ arm_write_sysreg(SP_EL1, 0);
+ arm_write_sysreg(SPSR_EL1, 0);
+
+ /* wipe the system registers */
+ arm_write_sysreg(AFSR0_EL1, 0);
+ arm_write_sysreg(AFSR1_EL1, 0);
+ arm_write_sysreg(AMAIR_EL1, 0);
+ arm_write_sysreg(CONTEXTIDR_EL1, 0);
+ arm_write_sysreg(CPACR_EL1, 0);
+ arm_write_sysreg(CSSELR_EL1, 0);
+ arm_write_sysreg(ESR_EL1, 0);
+ arm_write_sysreg(FAR_EL1, 0);
+ arm_write_sysreg(MAIR_EL1, 0);
+ arm_write_sysreg(PAR_EL1, 0);
+ arm_write_sysreg(TCR_EL1, 0);
+ arm_write_sysreg(TPIDRRO_EL0, 0);
+ arm_write_sysreg(TPIDR_EL0, 0);
+ arm_write_sysreg(TPIDR_EL1, 0);
+ arm_write_sysreg(TTBR0_EL1, 0);
+ arm_write_sysreg(TTBR1_EL1, 0);
+ arm_write_sysreg(VBAR_EL1, 0);
+
+ /* wipe timer registers */
+ arm_write_sysreg(CNTP_CTL_EL0, 0);
+ arm_write_sysreg(CNTP_CVAL_EL0, 0);
+ arm_write_sysreg(CNTP_TVAL_EL0, 0);
+ arm_write_sysreg(CNTV_CTL_EL0, 0);
+ arm_write_sysreg(CNTV_CVAL_EL0, 0);
+ arm_write_sysreg(CNTV_TVAL_EL0, 0);
+
+ /* AARCH64_TODO: handle PMU registers */
+ /* AARCH64_TODO: handle debug registers */
+ /* AARCH64_TODO: handle system registers for AArch32 state */
+}
+
+void arch_reset_self(struct per_cpu *cpu_data)
+{
+ int err = 0;
+ unsigned long reset_address;
+ struct cell *cell = cpu_data->cell;

+ struct registers *regs = guest_regs(cpu_data);
+

+ if (cell != &root_cell) {

+ trace_error(-EINVAL);
+ panic_stop();
+ }
+
+ /*
+ * Note: D-cache cleaning and I-cache invalidation is done on driver
+ * level after image is loaded.
+ */
+

+ err = irqchip_cpu_reset(cpu_data);
+ if (err)
+ printk("IRQ setup failed\n");
+
+ /* Wait for the driver to call cpu_up */

+ reset_address = psci_emulate_spin(cpu_data);
+
+ /* Restore an empty context */
+ arch_reset_el1(regs);
+
+ arm_write_sysreg(ELR_EL2, reset_address);
+
+ vmreturn(regs);
+}
+
int arch_cell_create(struct cell *cell)
{
return trace_error(-EINVAL);
diff --git a/hypervisor/arch/arm64/include/asm/percpu.h b/hypervisor/arch/arm64/include/asm/percpu.h
index 642e2f3..b30be6f 100644
--- a/hypervisor/arch/arm64/include/asm/percpu.h
+++ b/hypervisor/arch/arm64/include/asm/percpu.h
@@ -25,6 +25,7 @@

#include <jailhouse/printk.h>
#include <asm/cell.h>
+#include <asm/psci.h>
#include <asm/spinlock.h>

struct pending_irq;
@@ -60,6 +61,11 @@ struct per_cpu {
void *gicr_base;

bool flush_vcpu_caches;
+
+ __attribute__((aligned(16))) struct psci_mbox psci_mbox;
+ struct psci_mbox guest_mbox;
+
+ unsigned int virt_id;
union mpidr mpidr;
} __attribute__((aligned(PAGE_SIZE)));

diff --git a/hypervisor/arch/arm64/psci_low.S b/hypervisor/arch/arm64/psci_low.S
new file mode 100644
index 0000000..f1ee1fc
--- /dev/null
+++ b/hypervisor/arch/arm64/psci_low.S
@@ -0,0 +1,65 @@

+/*
+ * Jailhouse AArch64 support
+ *
+ * Copyright (C) 2015 Huawei Technologies Duesseldorf GmbH
+ *
+ * Authors:
+ * Antonios Motakis <antonios...@huawei.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2. See
+ * the COPYING file in the top-level directory.
+ */
+
+#include <asm/head.h>

+#include <asm/psci.h>
+
+ .globl smc
+ /*
+ * Since we trap all SMC instructions, it may be useful to forward them
+ * when it isn't a PSCI call. The shutdown code will also have to issue
+ * a real PSCI_OFF call on secondary CPUs.
+ */
+smc:
+ smc #0
+ ret
+
+ .global _psci_cpu_off
+ /* x0: struct psci_mbox* */
+_psci_cpu_off:
+ ldr x2, =PSCI_INVALID_ADDRESS
+ /* Clear mbox */
+ str x2, [x0]
+
+ /* Wait for a CPU_ON call that updates the mbox */
+1: wfe
+ ldr x3, [x0]
+ cmp x3, #PSCI_INVALID_ADDRESS
+ b.eq 1b
+
+ /* Jump to the requested entry, with a parameter */
+ ldr x0, [x0, #8]
+ br x3
+ ret
+
+ .global _psci_cpu_on
+ /* x0: struct psci_mbox*, x1: entry, x2: context */
+_psci_cpu_on:
+1: ldxp x4, x5, [x0]
+ cmp x4, #PSCI_INVALID_ADDRESS
+ b.ne store_failed
+ stxp w7, x1, x2, [x0]
+ cbnz w7, 1b
+
+ dsb ishst
+ sev
+
+ mov x0, #0
+ ret
+
+store_failed:
+ mov x0, #PSCI_ALREADY_ON
+ ret
+
+ .global _psci_suspend_return
+_psci_suspend_return:
+ ret
diff --git a/hypervisor/arch/arm64/setup.c b/hypervisor/arch/arm64/setup.c
index 838c541..8dd1095 100644
--- a/hypervisor/arch/arm64/setup.c
+++ b/hypervisor/arch/arm64/setup.c
@@ -15,6 +15,7 @@
#include <asm/control.h>
#include <asm/irqchip.h>
#include <asm/setup.h>
+#include <asm/smp.h>

int arch_init_early(void)
{
@@ -34,6 +35,10 @@ int arch_cpu_init(struct per_cpu *cpu_data)

/* switch to the permanent page tables */
enable_mmu_el2(hv_paging_structs.root_table);

+ cpu_data->psci_mbox.entry = 0;
+ cpu_data->virt_id = cpu_data->cpu_id;
+ arm_read_sysreg(MPIDR_EL1, cpu_data->mpidr.val);

+
err = arch_mmu_cpu_cell_init(cpu_data);
if (err)
return err;

diff --git a/hypervisor/arch/arm64/traps.c b/hypervisor/arch/arm64/traps.c
index fec057d..7a0b24c 100644
--- a/hypervisor/arch/arm64/traps.c
+++ b/hypervisor/arch/arm64/traps.c
@@ -30,6 +30,32 @@ void arch_skip_instruction(struct trap_context *ctx)

ctx->pc += (instruction_length ? 4 : 2);
}

+static int arch_handle_smc(struct trap_context *ctx)
+{
+ unsigned long *regs = ctx->regs;
+
+ if (!IS_PSCI_FN(regs[0]))
+ return TRAP_UNHANDLED;
+

+ regs[0] = psci_dispatch(ctx);

+ arch_skip_instruction(ctx);
+

+ return TRAP_HANDLED;
+}
+

+static int arch_handle_hvc(struct trap_context *ctx)
+{
+ unsigned long *regs = ctx->regs;
+
+ if (!IS_PSCI_FN(regs[0]))
+ return TRAP_UNHANDLED;
+

+ regs[0] = psci_dispatch(ctx);

+ arch_skip_instruction(ctx);
+

+ return TRAP_HANDLED;
+}
+

static void dump_regs(struct trap_context *ctx)

{
unsigned char i;
@@ -112,6 +138,14 @@ static void arch_handle_trap(struct per_cpu *cpu_data,
ret = arch_handle_dabt(&ctx);
break;

+ case ESR_EC_SMC64:
+ ret = arch_handle_smc(&ctx);
+ break;
+
+ case ESR_EC_HVC64:
+ ret = arch_handle_hvc(&ctx);

[antonios...@huawei.com]

From: Antonios Motakis <antonios...@huawei.com>

We plug the irqchip handling code, and the GICv2 implementation from
AArch32.

GICv3 is slightly trickier; it makes heavier use of the sysregs, so
we will need to review more carefully that the sysregs macros do
the right thing.

Signed-off-by: Antonios Motakis <antonios...@huawei.com>
---

hypervisor/arch/arm64/Makefile | 2 ++
hypervisor/arch/arm64/control.c | 37 ++++++++++++++++++++++++++++++
hypervisor/arch/arm64/include/asm/cell.h | 2 ++
hypervisor/arch/arm64/include/asm/percpu.h | 10 ++++++++
hypervisor/arch/arm64/mmio.c | 4 ++--
hypervisor/arch/arm64/setup.c | 24 +++++++++++++++++--
hypervisor/arch/arm64/traps.c | 5 ++++
7 files changed, 80 insertions(+), 4 deletions(-)

diff --git a/hypervisor/arch/arm64/Makefile b/hypervisor/arch/arm64/Makefile
index 603056e..540f7e3 100644
--- a/hypervisor/arch/arm64/Makefile
+++ b/hypervisor/arch/arm64/Makefile
@@ -19,5 +19,7 @@ always := built-in.o

obj-y := entry.o setup.o control.o mmio.o caches.o
obj-y += ../arm/mmu_cell.o ../arm/paging.o ../arm/dbg-write.o ../arm/lib.o
obj-y += exception.o traps.o

+obj-y += ../arm/irqchip.o ../arm/gic-common.o

obj-$(CONFIG_SERIAL_AMBA_PL011) += ../arm/dbg-write-pl011.o
+obj-$(CONFIG_ARM_GIC) += ../arm/gic-v2.o
diff --git a/hypervisor/arch/arm64/control.c b/hypervisor/arch/arm64/control.c
index a1c4774..edb57a7 100644
--- a/hypervisor/arch/arm64/control.c
+++ b/hypervisor/arch/arm64/control.c
@@ -12,6 +12,10 @@

#include <jailhouse/control.h>
#include <jailhouse/printk.h>
+#include <asm/control.h>
+#include <asm/irqchip.h>
+#include <asm/platform.h>
+#include <asm/traps.h>

int arch_cell_create(struct cell *cell)
{
@@ -81,3 +85,36 @@ void arch_panic_park(void)
trace_error(-EINVAL);
while (1);
}
+
+void arch_handle_sgi(struct per_cpu *cpu_data, u32 irqn)
+{
+ cpu_data->stats[JAILHOUSE_CPU_STAT_VMEXITS_MANAGEMENT]++;
+
+ switch (irqn) {
+ case SGI_INJECT:
+ irqchip_inject_pending(cpu_data);
+ break;
+ default:
+ printk("WARN: unknown SGI received %d\n", irqn);
+ }
+}
+
+/*
+ * Handle the maintenance interrupt, the rest is injected into the cell.
+ * Return true when the IRQ has been handled by the hyp.
+ */
+bool arch_handle_phys_irq(struct per_cpu *cpu_data, u32 irqn)
+{
+ if (irqn == MAINTENANCE_IRQ) {
+ cpu_data->stats[JAILHOUSE_CPU_STAT_VMEXITS_MAINTENANCE]++;
+
+ irqchip_inject_pending(cpu_data);
+ return true;
+ }
+
+ cpu_data->stats[JAILHOUSE_CPU_STAT_VMEXITS_VIRQ]++;
+
+ irqchip_set_pending(cpu_data, irqn, true);
+
+ return false;
+}
diff --git a/hypervisor/arch/arm64/include/asm/cell.h b/hypervisor/arch/arm64/include/asm/cell.h
index 4ba8224..9a9689e 100644
--- a/hypervisor/arch/arm64/include/asm/cell.h
+++ b/hypervisor/arch/arm64/include/asm/cell.h
@@ -26,6 +26,8 @@ struct arch_cell {
struct paging_structures mm;
spinlock_t caches_lock;
bool needs_flush;
+
+ u64 spis;
};

extern struct cell root_cell;
diff --git a/hypervisor/arch/arm64/include/asm/percpu.h b/hypervisor/arch/arm64/include/asm/percpu.h
index 3763d8f..642e2f3 100644
--- a/hypervisor/arch/arm64/include/asm/percpu.h
+++ b/hypervisor/arch/arm64/include/asm/percpu.h
@@ -23,9 +23,12 @@

#ifndef __ASSEMBLY__

+#include <jailhouse/printk.h>
#include <asm/cell.h>
#include <asm/spinlock.h>

+struct pending_irq;
+
union mpidr {
u64 val;
struct {
@@ -49,6 +52,13 @@ struct per_cpu {
int shutdown_state;
bool failed;

+ /* Other CPUs can insert sgis into the pending array */
+ spinlock_t gic_lock;
+ struct pending_irq *pending_irqs;
+ struct pending_irq *first_pending;
+ /* Only GICv3: redistributor base */
+ void *gicr_base;
+
bool flush_vcpu_caches;
union mpidr mpidr;
} __attribute__((aligned(PAGE_SIZE)));
diff --git a/hypervisor/arch/arm64/mmio.c b/hypervisor/arch/arm64/mmio.c
index 0c43365..8888251 100644
--- a/hypervisor/arch/arm64/mmio.c
+++ b/hypervisor/arch/arm64/mmio.c
@@ -18,6 +18,7 @@
#include <jailhouse/mmio.h>
#include <jailhouse/printk.h>
#include <asm/bitops.h>
+#include <asm/irqchip.h>
#include <asm/percpu.h>
#include <asm/sysregs.h>
#include <asm/traps.h>
@@ -26,8 +27,7 @@

unsigned int arch_mmio_count_regions(struct cell *cell)

{
- /* not entirely a lie :) */
- return 0;
+ return irqchip_mmio_count_regions(cell);

}

static void arch_inject_dabt(struct trap_context *ctx, unsigned long addr)

diff --git a/hypervisor/arch/arm64/setup.c b/hypervisor/arch/arm64/setup.c
index 13e6387..838c541 100644
--- a/hypervisor/arch/arm64/setup.c
+++ b/hypervisor/arch/arm64/setup.c
@@ -13,23 +13,43 @@
#include <jailhouse/entry.h>
#include <jailhouse/printk.h>
#include <asm/control.h>
+#include <asm/irqchip.h>
#include <asm/setup.h>

int arch_init_early(void)
{
- return arch_mmu_cell_init(&root_cell);

+ int err = 0;
+

+ err = arch_mmu_cell_init(&root_cell);

+ if (err)
+ return err;
+

+ return irqchip_init();
}

int arch_cpu_init(struct per_cpu *cpu_data)

{
+ int err = 0;
+

/* switch to the permanent page tables */
enable_mmu_el2(hv_paging_structs.root_table);

- return arch_mmu_cpu_cell_init(cpu_data);

+ err = arch_mmu_cpu_cell_init(cpu_data);
+ if (err)

+ return err;
+
+ return irqchip_cpu_init(cpu_data);
}

int arch_init_late(void)
{
+ int err;
+
+ /* Setup the SPI bitmap */
+ err = irqchip_cell_init(&root_cell);

+ if (err)
+ return err;
+

return map_root_memory_regions();
}

diff --git a/hypervisor/arch/arm64/traps.c b/hypervisor/arch/arm64/traps.c
index cc6fe6c..fec057d 100644
--- a/hypervisor/arch/arm64/traps.c
+++ b/hypervisor/arch/arm64/traps.c
@@ -21,6 +21,7 @@
#include <asm/sysregs.h>
#include <asm/traps.h>
#include <asm/processor.h>
+#include <asm/irqchip.h>

void arch_skip_instruction(struct trap_context *ctx)
{
@@ -143,6 +144,10 @@ struct registers *arch_handle_exit(struct per_cpu *cpu_data,
cpu_data->stats[JAILHOUSE_CPU_STAT_VMEXITS_TOTAL]++;

switch (regs->exit_reason) {
+ case EXIT_REASON_EL1_IRQ:
+ irqchip_handle_irq(cpu_data);
+ break;
+
case EXIT_REASON_EL1_ABORT:
arch_handle_trap(cpu_data, regs);
break;
--
2.7.0

[antonios...@huawei.com]

From: Antonios Motakis <antonios...@huawei.com>

Implement spinlocks for the hypervisor firmware, on AArch64.

Signed-off-by: Antonios Motakis <antonios...@huawei.com>
---

hypervisor/arch/arm64/include/asm/spinlock.h | 71 ++++++++++++++++++++++++++++
1 file changed, 71 insertions(+)
create mode 100644 hypervisor/arch/arm64/include/asm/spinlock.h

diff --git a/hypervisor/arch/arm64/include/asm/spinlock.h b/hypervisor/arch/arm64/include/asm/spinlock.h
new file mode 100644
index 0000000..2be5fe8
--- /dev/null
+++ b/hypervisor/arch/arm64/include/asm/spinlock.h
@@ -0,0 +1,71 @@

+/*
+ * Jailhouse AArch64 support
+ *
+ * Copyright (C) 2015 Huawei Technologies Duesseldorf GmbH
+ *
+ * Authors:
+ * Antonios Motakis <antonios...@huawei.com>
+ *

+ * Spinlock implementation copied from
+ * arch/arm64/include/asm/spinlock.h in Linux
+ *
+ * Copyright (C) 2012 ARM Ltd.

+ *
+ * This work is licensed under the terms of the GNU GPL, version 2. See
+ * the COPYING file in the top-level directory.
+ *

+ */
+
+#ifndef _JAILHOUSE_ASM_SPINLOCK_H
+#define _JAILHOUSE_ASM_SPINLOCK_H
+
+#define DEFINE_SPINLOCK(name) spinlock_t (name)
+#define TICKET_SHIFT 16
+
+/* TODO: fix this if we add support for BE */
+typedef struct {
+ u16 owner;
+ u16 next;
+} spinlock_t __attribute__((aligned(4)));
+
+static inline void spin_lock(spinlock_t *lock)
+{
+ unsigned int tmp;
+ spinlock_t lockval, newval;
+
+ asm volatile(
+ /* Atomically increment the next ticket. */
+" prfm pstl1strm, %3\n"
+"1: ldaxr %w0, %3\n"
+" add %w1, %w0, %w5\n"
+" stxr %w2, %w1, %3\n"
+" cbnz %w2, 1b\n"
+ /* Did we get the lock? */
+" eor %w1, %w0, %w0, ror #16\n"
+" cbz %w1, 3f\n"
+ /*
+ * No: spin on the owner. Send a local event to avoid missing an
+ * unlock before the exclusive load.
+ */
+" sevl\n"
+"2: wfe\n"
+" ldaxrh %w2, %4\n"
+" eor %w1, %w2, %w0, lsr #16\n"
+" cbnz %w1, 2b\n"
+ /* We got the lock. Critical section starts here. */
+"3:"
+ : "=&r" (lockval), "=&r" (newval), "=&r" (tmp), "+Q" (*lock)
+ : "Q" (lock->owner), "I" (1 << TICKET_SHIFT)
+ : "memory");
+}
+
+static inline void spin_unlock(spinlock_t *lock)
+{
+ asm volatile(
+" stlrh %w1, %0\n"
+ : "=Q" (lock->owner)
+ : "r" (lock->owner + 1)
+ : "memory");
+}
+
+#endif /* !_JAILHOUSE_ASM_SPINLOCK_H */
--
2.7.0

[antonios...@huawei.com]

From: Jan Kiszka <jan.k...@siemens.com>

This adds arm64 to CI via the AMD Seattle configuration.

Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
---

.travis.yml | 2 +-
ci/build-all-configs.sh | 6 +-
ci/gen-kernel-build.sh | 1 +

ci/kernel-config-amd-seattle | 3091 ++++++++++++++++++++++++++++++++++++++++++
4 files changed, 3098 insertions(+), 2 deletions(-)
create mode 100644 ci/kernel-config-amd-seattle

diff --git a/.travis.yml b/.travis.yml
index c0c6931..868e40e 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -27,7 +27,7 @@ install:
# Install additional packages
- sudo apt-add-repository 'deb http://archive.ubuntu.com/ubuntu wily main'
- sudo apt-get update -qq
- - sudo apt-get install -qq python-mako gcc-arm-linux-gnueabihf make
+ - sudo apt-get install -qq python-mako gcc-arm-linux-gnueabihf gcc-aarch64-linux-gnu make
# Install kernel build environment
- pushd ci
- wget -q http://www.kiszka.org/downloads/travis-ci/kernel-build.tar.xz
diff --git a/ci/build-all-configs.sh b/ci/build-all-configs.sh
index cbcbb86..0ea8bc2 100755
--- a/ci/build-all-configs.sh
+++ b/ci/build-all-configs.sh
@@ -13,7 +13,7 @@

set -e

-CONFIGS="x86 banana-pi vexpress"
+CONFIGS="x86 banana-pi vexpress amd-seattle"

# only build a specific config if the branch selects it
if [ ${TRAVIS_BRANCH#coverity_scan-} != ${TRAVIS_BRANCH} ]; then
@@ -37,6 +37,10 @@ for CONFIG in $CONFIGS; do
ARCH=x86_64
CROSS_COMPILE=
;;
+ amd-seattle)
+ ARCH=arm64
+ CROSS_COMPILE=aarch64-linux-gnu-
+ ;;
*)
ARCH=arm
CROSS_COMPILE=arm-linux-gnueabihf-
diff --git a/ci/gen-kernel-build.sh b/ci/gen-kernel-build.sh
index 48d3c9f..bfc3320 100755
--- a/ci/gen-kernel-build.sh
+++ b/ci/gen-kernel-build.sh
@@ -61,4 +61,5 @@ prepare_kernel
build_kernel x86 x86_64
build_kernel banana-pi arm arm-linux-gnueabihf-
build_kernel vexpress arm arm-linux-gnueabihf-
+build_kernel amd-seattle arm64 aarch64-linux-gnu-
package_out
diff --git a/ci/kernel-config-amd-seattle b/ci/kernel-config-amd-seattle
new file mode 100644
index 0000000..bac625e
--- /dev/null
+++ b/ci/kernel-config-amd-seattle
@@ -0,0 +1,3091 @@
+#
+# Automatically generated file; DO NOT EDIT.
+# Linux/arm64 4.4.0 Kernel Configuration
+#
+CONFIG_ARM64=y
+CONFIG_64BIT=y
+CONFIG_ARCH_PHYS_ADDR_T_64BIT=y
+CONFIG_MMU=y
+CONFIG_STACKTRACE_SUPPORT=y
+CONFIG_ILLEGAL_POINTER_VALUE=0xdead000000000000
+CONFIG_LOCKDEP_SUPPORT=y
+CONFIG_TRACE_IRQFLAGS_SUPPORT=y
+CONFIG_RWSEM_XCHGADD_ALGORITHM=y
+CONFIG_GENERIC_BUG=y
+CONFIG_GENERIC_BUG_RELATIVE_POINTERS=y
+CONFIG_GENERIC_HWEIGHT=y
+CONFIG_GENERIC_CSUM=y
+CONFIG_GENERIC_CALIBRATE_DELAY=y
+CONFIG_ZONE_DMA=y
+CONFIG_HAVE_GENERIC_RCU_GUP=y
+CONFIG_ARCH_DMA_ADDR_T_64BIT=y
+CONFIG_NEED_DMA_MAP_STATE=y
+CONFIG_NEED_SG_DMA_LENGTH=y
+CONFIG_SMP=y
+CONFIG_SWIOTLB=y
+CONFIG_IOMMU_HELPER=y
+CONFIG_KERNEL_MODE_NEON=y
+CONFIG_FIX_EARLYCON_MEM=y
+CONFIG_PGTABLE_LEVELS=3
+CONFIG_DEFCONFIG_LIST="/lib/modules/$UNAME_RELEASE/.config"
+CONFIG_IRQ_WORK=y
+CONFIG_BUILDTIME_EXTABLE_SORT=y
+
+#
+# General setup
+#
+CONFIG_INIT_ENV_ARG_LIMIT=32
+CONFIG_CROSS_COMPILE=""
+# CONFIG_COMPILE_TEST is not set
+CONFIG_LOCALVERSION=""
+# CONFIG_LOCALVERSION_AUTO is not set
+CONFIG_DEFAULT_HOSTNAME="(none)"
+CONFIG_SWAP=y
+CONFIG_SYSVIPC=y
+CONFIG_SYSVIPC_SYSCTL=y
+CONFIG_POSIX_MQUEUE=y
+CONFIG_POSIX_MQUEUE_SYSCTL=y
+CONFIG_CROSS_MEMORY_ATTACH=y
+CONFIG_FHANDLE=y
+# CONFIG_USELIB is not set
+CONFIG_AUDIT=y
+CONFIG_HAVE_ARCH_AUDITSYSCALL=y
+CONFIG_AUDITSYSCALL=y
+CONFIG_AUDIT_WATCH=y
+CONFIG_AUDIT_TREE=y
+
+#
+# IRQ subsystem
+#
+CONFIG_GENERIC_IRQ_PROBE=y
+CONFIG_GENERIC_IRQ_SHOW=y
+CONFIG_GENERIC_IRQ_SHOW_LEVEL=y
+CONFIG_GENERIC_IRQ_MIGRATION=y
+CONFIG_HARDIRQS_SW_RESEND=y
+CONFIG_IRQ_DOMAIN=y
+CONFIG_IRQ_DOMAIN_HIERARCHY=y
+CONFIG_GENERIC_MSI_IRQ=y
+CONFIG_GENERIC_MSI_IRQ_DOMAIN=y
+CONFIG_HANDLE_DOMAIN_IRQ=y
+# CONFIG_IRQ_DOMAIN_DEBUG is not set
+CONFIG_IRQ_FORCED_THREADING=y
+CONFIG_SPARSE_IRQ=y
+CONFIG_GENERIC_TIME_VSYSCALL=y
+CONFIG_GENERIC_CLOCKEVENTS=y
+CONFIG_ARCH_HAS_TICK_BROADCAST=y
+CONFIG_GENERIC_CLOCKEVENTS_BROADCAST=y
+
+#
+# Timers subsystem
+#
+CONFIG_TICK_ONESHOT=y
+CONFIG_NO_HZ_COMMON=y
+# CONFIG_HZ_PERIODIC is not set
+# CONFIG_NO_HZ_IDLE is not set
+CONFIG_NO_HZ_FULL=y
+# CONFIG_NO_HZ_FULL_ALL is not set
+# CONFIG_NO_HZ_FULL_SYSIDLE is not set
+CONFIG_NO_HZ=y
+CONFIG_HIGH_RES_TIMERS=y
+
+#
+# CPU/Task time and stats accounting
+#
+CONFIG_VIRT_CPU_ACCOUNTING=y
+CONFIG_VIRT_CPU_ACCOUNTING_GEN=y
+CONFIG_BSD_PROCESS_ACCT=y
+CONFIG_BSD_PROCESS_ACCT_V3=y
+CONFIG_TASKSTATS=y
+CONFIG_TASK_DELAY_ACCT=y
+CONFIG_TASK_XACCT=y
+CONFIG_TASK_IO_ACCOUNTING=y
+
+#
+# RCU Subsystem
+#
+CONFIG_PREEMPT_RCU=y
+# CONFIG_RCU_EXPERT is not set
+CONFIG_SRCU=y
+# CONFIG_TASKS_RCU is not set
+CONFIG_RCU_STALL_COMMON=y
+CONFIG_CONTEXT_TRACKING=y
+# CONFIG_CONTEXT_TRACKING_FORCE is not set
+# CONFIG_TREE_RCU_TRACE is not set
+CONFIG_RCU_NOCB_CPU=y
+CONFIG_RCU_NOCB_CPU_NONE=y
+# CONFIG_RCU_NOCB_CPU_ZERO is not set
+# CONFIG_RCU_NOCB_CPU_ALL is not set
+# CONFIG_RCU_EXPEDITE_BOOT is not set
+CONFIG_BUILD_BIN2C=y
+CONFIG_IKCONFIG=y
+# CONFIG_IKCONFIG_PROC is not set
+CONFIG_LOG_BUF_SHIFT=14
+CONFIG_LOG_CPU_MAX_BUF_SHIFT=12
+CONFIG_GENERIC_SCHED_CLOCK=y
+CONFIG_CGROUPS=y
+# CONFIG_CGROUP_DEBUG is not set
+CONFIG_CGROUP_FREEZER=y
+# CONFIG_CGROUP_PIDS is not set
+CONFIG_CGROUP_DEVICE=y
+CONFIG_CPUSETS=y
+CONFIG_PROC_PID_CPUSET=y
+CONFIG_CGROUP_CPUACCT=y
+# CONFIG_MEMCG is not set
+# CONFIG_CGROUP_HUGETLB is not set
+CONFIG_CGROUP_PERF=y
+CONFIG_CGROUP_SCHED=y
+CONFIG_FAIR_GROUP_SCHED=y
+CONFIG_CFS_BANDWIDTH=y
+# CONFIG_RT_GROUP_SCHED is not set
+CONFIG_BLK_CGROUP=y
+# CONFIG_DEBUG_BLK_CGROUP is not set
+# CONFIG_CHECKPOINT_RESTORE is not set
+CONFIG_NAMESPACES=y
+CONFIG_UTS_NS=y
+CONFIG_IPC_NS=y
+CONFIG_USER_NS=y
+CONFIG_PID_NS=y
+CONFIG_NET_NS=y
+CONFIG_SCHED_AUTOGROUP=y
+# CONFIG_SYSFS_DEPRECATED is not set
+CONFIG_RELAY=y
+CONFIG_BLK_DEV_INITRD=y
+CONFIG_INITRAMFS_SOURCE=""
+CONFIG_RD_GZIP=y
+CONFIG_RD_BZIP2=y
+CONFIG_RD_LZMA=y
+CONFIG_RD_XZ=y
+CONFIG_RD_LZO=y
+CONFIG_RD_LZ4=y
+# CONFIG_CC_OPTIMIZE_FOR_SIZE is not set
+CONFIG_SYSCTL=y
+CONFIG_ANON_INODES=y
+CONFIG_SYSCTL_EXCEPTION_TRACE=y
+CONFIG_BPF=y
+# CONFIG_EXPERT is not set
+CONFIG_MULTIUSER=y
+# CONFIG_SGETMASK_SYSCALL is not set
+CONFIG_SYSFS_SYSCALL=y
+# CONFIG_SYSCTL_SYSCALL is not set
+CONFIG_KALLSYMS=y
+CONFIG_KALLSYMS_ALL=y
+CONFIG_PRINTK=y
+CONFIG_BUG=y
+CONFIG_ELF_CORE=y
+CONFIG_BASE_FULL=y
+CONFIG_FUTEX=y
+CONFIG_EPOLL=y
+CONFIG_SIGNALFD=y
+CONFIG_TIMERFD=y
+CONFIG_EVENTFD=y
+# CONFIG_BPF_SYSCALL is not set
+CONFIG_SHMEM=y
+CONFIG_AIO=y
+CONFIG_ADVISE_SYSCALLS=y
+# CONFIG_USERFAULTFD is not set
+CONFIG_PCI_QUIRKS=y
+CONFIG_MEMBARRIER=y
+# CONFIG_EMBEDDED is not set
+CONFIG_HAVE_PERF_EVENTS=y
+CONFIG_PERF_USE_VMALLOC=y
+
+#
+# Kernel Performance Events And Counters
+#
+CONFIG_PERF_EVENTS=y
+# CONFIG_DEBUG_PERF_USE_VMALLOC is not set
+CONFIG_VM_EVENT_COUNTERS=y
+CONFIG_SLUB_DEBUG=y
+# CONFIG_COMPAT_BRK is not set
+# CONFIG_SLAB is not set
+CONFIG_SLUB=y
+CONFIG_SLUB_CPU_PARTIAL=y
+# CONFIG_SYSTEM_DATA_VERIFICATION is not set
+CONFIG_PROFILING=y
+CONFIG_TRACEPOINTS=y
+CONFIG_JUMP_LABEL=y
+# CONFIG_STATIC_KEYS_SELFTEST is not set
+# CONFIG_UPROBES is not set
+# CONFIG_HAVE_64BIT_ALIGNED_ACCESS is not set
+CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS=y
+CONFIG_HAVE_ARCH_TRACEHOOK=y
+CONFIG_HAVE_DMA_ATTRS=y
+CONFIG_HAVE_DMA_CONTIGUOUS=y
+CONFIG_GENERIC_SMP_IDLE_THREAD=y
+CONFIG_GENERIC_IDLE_POLL_SETUP=y
+CONFIG_HAVE_CLK=y
+CONFIG_HAVE_DMA_API_DEBUG=y
+CONFIG_HAVE_HW_BREAKPOINT=y
+CONFIG_HAVE_PERF_REGS=y
+CONFIG_HAVE_PERF_USER_STACK_DUMP=y
+CONFIG_HAVE_ARCH_JUMP_LABEL=y
+CONFIG_HAVE_RCU_TABLE_FREE=y
+CONFIG_HAVE_ALIGNED_STRUCT_PAGE=y
+CONFIG_HAVE_CMPXCHG_LOCAL=y
+CONFIG_HAVE_CMPXCHG_DOUBLE=y
+CONFIG_ARCH_WANT_COMPAT_IPC_PARSE_VERSION=y
+CONFIG_HAVE_ARCH_SECCOMP_FILTER=y
+CONFIG_HAVE_CC_STACKPROTECTOR=y
+# CONFIG_CC_STACKPROTECTOR is not set
+CONFIG_CC_STACKPROTECTOR_NONE=y
+# CONFIG_CC_STACKPROTECTOR_REGULAR is not set
+# CONFIG_CC_STACKPROTECTOR_STRONG is not set
+CONFIG_HAVE_CONTEXT_TRACKING=y
+CONFIG_HAVE_VIRT_CPU_ACCOUNTING_GEN=y
+CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE=y
+CONFIG_MODULES_USE_ELF_RELA=y
+CONFIG_ARCH_HAS_ELF_RANDOMIZE=y
+CONFIG_CLONE_BACKWARDS=y
+
+#
+# GCOV-based kernel profiling
+#
+# CONFIG_GCOV_KERNEL is not set
+CONFIG_ARCH_HAS_GCOV_PROFILE_ALL=y
+CONFIG_HAVE_GENERIC_DMA_COHERENT=y
+CONFIG_SLABINFO=y
+CONFIG_RT_MUTEXES=y
+CONFIG_BASE_SMALL=0
+CONFIG_MODULES=y
+# CONFIG_MODULE_FORCE_LOAD is not set
+CONFIG_MODULE_UNLOAD=y
+# CONFIG_MODULE_FORCE_UNLOAD is not set
+# CONFIG_MODVERSIONS is not set
+# CONFIG_MODULE_SRCVERSION_ALL is not set
+# CONFIG_MODULE_SIG is not set
+# CONFIG_MODULE_COMPRESS is not set
+CONFIG_MODULES_TREE_LOOKUP=y
+CONFIG_BLOCK=y
+CONFIG_BLK_DEV_BSG=y
+CONFIG_BLK_DEV_BSGLIB=y
+CONFIG_BLK_DEV_INTEGRITY=y
+CONFIG_BLK_DEV_THROTTLING=y
+# CONFIG_BLK_CMDLINE_PARSER is not set
+
+#
+# Partition Types
+#
+CONFIG_PARTITION_ADVANCED=y
+# CONFIG_ACORN_PARTITION is not set
+CONFIG_AIX_PARTITION=y
+CONFIG_OSF_PARTITION=y
+CONFIG_AMIGA_PARTITION=y
+# CONFIG_ATARI_PARTITION is not set
+CONFIG_MAC_PARTITION=y
+CONFIG_MSDOS_PARTITION=y
+CONFIG_BSD_DISKLABEL=y
+CONFIG_MINIX_SUBPARTITION=y
+CONFIG_SOLARIS_X86_PARTITION=y
+CONFIG_UNIXWARE_DISKLABEL=y
+CONFIG_LDM_PARTITION=y
+# CONFIG_LDM_DEBUG is not set
+CONFIG_SGI_PARTITION=y
+# CONFIG_ULTRIX_PARTITION is not set
+CONFIG_SUN_PARTITION=y
+CONFIG_KARMA_PARTITION=y
+CONFIG_EFI_PARTITION=y
+# CONFIG_SYSV68_PARTITION is not set
+# CONFIG_CMDLINE_PARTITION is not set
+
+#
+# IO Schedulers
+#
+CONFIG_IOSCHED_NOOP=y
+CONFIG_IOSCHED_DEADLINE=y
+CONFIG_IOSCHED_CFQ=y
+CONFIG_CFQ_GROUP_IOSCHED=y
+# CONFIG_DEFAULT_DEADLINE is not set
+CONFIG_DEFAULT_CFQ=y
+# CONFIG_DEFAULT_NOOP is not set
+CONFIG_DEFAULT_IOSCHED="cfq"
+CONFIG_UNINLINE_SPIN_UNLOCK=y
+CONFIG_ARCH_SUPPORTS_ATOMIC_RMW=y
+CONFIG_MUTEX_SPIN_ON_OWNER=y
+CONFIG_RWSEM_SPIN_ON_OWNER=y
+CONFIG_LOCK_SPIN_ON_OWNER=y
+CONFIG_FREEZER=y
+
+#
+# Platform selection
+#
+# CONFIG_ARCH_BCM_IPROC is not set
+# CONFIG_ARCH_BERLIN is not set
+# CONFIG_ARCH_EXYNOS7 is not set
+# CONFIG_ARCH_LAYERSCAPE is not set
+# CONFIG_ARCH_HISI is not set
+# CONFIG_ARCH_MEDIATEK is not set
+# CONFIG_ARCH_QCOM is not set
+# CONFIG_ARCH_ROCKCHIP is not set
+CONFIG_ARCH_SEATTLE=y
+# CONFIG_ARCH_STRATIX10 is not set
+# CONFIG_ARCH_TEGRA is not set
+# CONFIG_ARCH_SPRD is not set
+# CONFIG_ARCH_THUNDER is not set
+# CONFIG_ARCH_VEXPRESS is not set
+# CONFIG_ARCH_XGENE is not set
+# CONFIG_ARCH_ZYNQMP is not set
+
+#
+# Bus support
+#
+CONFIG_PCI=y
+CONFIG_PCI_DOMAINS=y
+CONFIG_PCI_DOMAINS_GENERIC=y
+CONFIG_PCI_SYSCALL=y
+CONFIG_PCI_BUS_ADDR_T_64BIT=y
+CONFIG_PCI_MSI=y
+CONFIG_PCI_MSI_IRQ_DOMAIN=y
+# CONFIG_PCI_DEBUG is not set
+# CONFIG_PCI_REALLOC_ENABLE_AUTO is not set
+CONFIG_PCI_STUB=y
+CONFIG_PCI_ATS=y
+CONFIG_PCI_IOV=y
+CONFIG_PCI_PRI=y
+CONFIG_PCI_PASID=y
+CONFIG_PCI_LABEL=y
+
+#
+# PCI host controller drivers
+#
+# CONFIG_PCI_HOST_GENERIC is not set
+# CONFIG_PCIE_IPROC is not set
+# CONFIG_PCI_HISI is not set
+CONFIG_PCIEPORTBUS=y
+CONFIG_HOTPLUG_PCI_PCIE=y
+CONFIG_PCIEAER=y
+CONFIG_PCIE_ECRC=y
+# CONFIG_PCIEAER_INJECT is not set
+CONFIG_PCIEASPM=y
+# CONFIG_PCIEASPM_DEBUG is not set
+CONFIG_PCIEASPM_DEFAULT=y
+# CONFIG_PCIEASPM_POWERSAVE is not set
+# CONFIG_PCIEASPM_PERFORMANCE is not set
+CONFIG_HOTPLUG_PCI=y
+# CONFIG_HOTPLUG_PCI_CPCI is not set
+# CONFIG_HOTPLUG_PCI_SHPC is not set
+
+#
+# Kernel Features
+#
+
+#
+# ARM errata workarounds via the alternatives framework
+#
+CONFIG_ARM64_ERRATUM_826319=y
+CONFIG_ARM64_ERRATUM_827319=y
+CONFIG_ARM64_ERRATUM_824069=y
+CONFIG_ARM64_ERRATUM_819472=y
+CONFIG_ARM64_ERRATUM_832075=y
+CONFIG_ARM64_ERRATUM_843419=y
+CONFIG_CAVIUM_ERRATUM_22375=y
+CONFIG_CAVIUM_ERRATUM_23154=y
+CONFIG_ARM64_4K_PAGES=y
+# CONFIG_ARM64_16K_PAGES is not set
+# CONFIG_ARM64_64K_PAGES is not set
+CONFIG_ARM64_VA_BITS_39=y
+# CONFIG_ARM64_VA_BITS_48 is not set
+CONFIG_ARM64_VA_BITS=39
+# CONFIG_CPU_BIG_ENDIAN is not set
+CONFIG_SCHED_MC=y
+CONFIG_SCHED_SMT=y
+CONFIG_NR_CPUS=8
+CONFIG_HOTPLUG_CPU=y
+# CONFIG_PREEMPT_NONE is not set
+# CONFIG_PREEMPT_VOLUNTARY is not set
+CONFIG_PREEMPT=y
+CONFIG_PREEMPT_COUNT=y
+# CONFIG_HZ_100 is not set
+CONFIG_HZ_250=y
+# CONFIG_HZ_300 is not set
+# CONFIG_HZ_1000 is not set
+CONFIG_HZ=250
+CONFIG_SCHED_HRTICK=y
+CONFIG_ARCH_HAS_HOLES_MEMORYMODEL=y
+CONFIG_ARCH_SPARSEMEM_ENABLE=y
+CONFIG_ARCH_SPARSEMEM_DEFAULT=y
+CONFIG_ARCH_SELECT_MEMORY_MODEL=y
+CONFIG_HAVE_ARCH_PFN_VALID=y
+CONFIG_HW_PERF_EVENTS=y
+CONFIG_SYS_SUPPORTS_HUGETLBFS=y
+CONFIG_ARCH_WANT_GENERAL_HUGETLB=y
+CONFIG_ARCH_WANT_HUGE_PMD_SHARE=y
+CONFIG_ARCH_HAS_CACHE_LINE_SIZE=y
+CONFIG_SELECT_MEMORY_MODEL=y
+CONFIG_SPARSEMEM_MANUAL=y
+CONFIG_SPARSEMEM=y
+CONFIG_HAVE_MEMORY_PRESENT=y
+CONFIG_SPARSEMEM_EXTREME=y
+CONFIG_SPARSEMEM_VMEMMAP_ENABLE=y
+CONFIG_SPARSEMEM_VMEMMAP=y
+CONFIG_HAVE_MEMBLOCK=y
+CONFIG_NO_BOOTMEM=y
+# CONFIG_HAVE_BOOTMEM_INFO_NODE is not set
+CONFIG_SPLIT_PTLOCK_CPUS=4
+CONFIG_COMPACTION=y
+CONFIG_MIGRATION=y
+CONFIG_PHYS_ADDR_T_64BIT=y
+CONFIG_ZONE_DMA_FLAG=1
+CONFIG_BOUNCE=y
+CONFIG_KSM=y
+CONFIG_DEFAULT_MMAP_MIN_ADDR=4096
+# CONFIG_TRANSPARENT_HUGEPAGE is not set
+CONFIG_CLEANCACHE=y
+CONFIG_FRONTSWAP=y
+# CONFIG_CMA is not set
+CONFIG_ZSWAP=y
+CONFIG_ZPOOL=y
+# CONFIG_ZBUD is not set
+CONFIG_ZSMALLOC=y
+# CONFIG_PGTABLE_MAPPING is not set
+# CONFIG_ZSMALLOC_STAT is not set
+CONFIG_GENERIC_EARLY_IOREMAP=y
+# CONFIG_IDLE_PAGE_TRACKING is not set
+# CONFIG_SECCOMP is not set
+# CONFIG_XEN is not set
+CONFIG_FORCE_MAX_ZONEORDER=11
+
+#
+# ARMv8.1 architectural features
+#
+CONFIG_ARM64_HW_AFDBM=y
+CONFIG_ARM64_PAN=y
+# CONFIG_ARM64_LSE_ATOMICS is not set
+
+#
+# Boot options
+#
+CONFIG_CMDLINE="mem=16G"
+# CONFIG_CMDLINE_FORCE is not set
+CONFIG_EFI_STUB=y
+CONFIG_EFI=y
+CONFIG_DMI=y
+
+#
+# Userspace binary formats
+#
+CONFIG_BINFMT_ELF=y
+CONFIG_CORE_DUMP_DEFAULT_ELF_HEADERS=y
+CONFIG_BINFMT_SCRIPT=y
+# CONFIG_HAVE_AOUT is not set
+# CONFIG_BINFMT_MISC is not set
+CONFIG_COREDUMP=y
+# CONFIG_COMPAT is not set
+
+#
+# Power management options
+#
+# CONFIG_SUSPEND is not set
+# CONFIG_PM is not set
+CONFIG_ARCH_SUSPEND_POSSIBLE=y
+
+#
+# CPU Power Management
+#
+
+#
+# CPU Idle
+#
+# CONFIG_CPU_IDLE is not set
+# CONFIG_ARCH_NEEDS_CPU_IDLE_COUPLED is not set
+
+#
+# CPU Frequency scaling
+#
+# CONFIG_CPU_FREQ is not set
+CONFIG_NET=y
+CONFIG_NET_INGRESS=y
+
+#
+# Networking options
+#
+CONFIG_PACKET=y
+# CONFIG_PACKET_DIAG is not set
+CONFIG_UNIX=y
+# CONFIG_UNIX_DIAG is not set
+CONFIG_XFRM=y
+CONFIG_XFRM_ALGO=y
+CONFIG_XFRM_USER=y
+CONFIG_XFRM_SUB_POLICY=y
+CONFIG_XFRM_MIGRATE=y
+CONFIG_XFRM_STATISTICS=y
+# CONFIG_NET_KEY is not set
+CONFIG_INET=y
+CONFIG_IP_MULTICAST=y
+CONFIG_IP_ADVANCED_ROUTER=y
+CONFIG_IP_FIB_TRIE_STATS=y
+CONFIG_IP_MULTIPLE_TABLES=y
+CONFIG_IP_ROUTE_MULTIPATH=y
+CONFIG_IP_ROUTE_VERBOSE=y
+# CONFIG_IP_PNP is not set
+# CONFIG_NET_IPIP is not set
+# CONFIG_NET_IPGRE_DEMUX is not set
+# CONFIG_NET_IP_TUNNEL is not set
+CONFIG_IP_MROUTE=y
+CONFIG_IP_MROUTE_MULTIPLE_TABLES=y
+CONFIG_IP_PIMSM_V1=y
+CONFIG_IP_PIMSM_V2=y
+CONFIG_SYN_COOKIES=y
+# CONFIG_NET_UDP_TUNNEL is not set
+# CONFIG_NET_FOU is not set
+# CONFIG_INET_AH is not set
+# CONFIG_INET_ESP is not set
+# CONFIG_INET_IPCOMP is not set
+# CONFIG_INET_XFRM_TUNNEL is not set
+# CONFIG_INET_TUNNEL is not set
+# CONFIG_INET_XFRM_MODE_TRANSPORT is not set
+# CONFIG_INET_XFRM_MODE_TUNNEL is not set
+# CONFIG_INET_XFRM_MODE_BEET is not set
+CONFIG_INET_LRO=y
+# CONFIG_INET_DIAG is not set
+CONFIG_TCP_CONG_ADVANCED=y
+# CONFIG_TCP_CONG_BIC is not set
+CONFIG_TCP_CONG_CUBIC=y
+# CONFIG_TCP_CONG_WESTWOOD is not set
+# CONFIG_TCP_CONG_HTCP is not set
+# CONFIG_TCP_CONG_HSTCP is not set
+# CONFIG_TCP_CONG_HYBLA is not set
+# CONFIG_TCP_CONG_VEGAS is not set
+# CONFIG_TCP_CONG_SCALABLE is not set
+# CONFIG_TCP_CONG_LP is not set
+# CONFIG_TCP_CONG_VENO is not set
+# CONFIG_TCP_CONG_YEAH is not set
+# CONFIG_TCP_CONG_ILLINOIS is not set
+# CONFIG_TCP_CONG_DCTCP is not set
+# CONFIG_TCP_CONG_CDG is not set
+CONFIG_DEFAULT_CUBIC=y
+# CONFIG_DEFAULT_RENO is not set
+CONFIG_DEFAULT_TCP_CONG="cubic"
+CONFIG_TCP_MD5SIG=y
+CONFIG_IPV6=y
+CONFIG_IPV6_ROUTER_PREF=y
+CONFIG_IPV6_ROUTE_INFO=y
+CONFIG_IPV6_OPTIMISTIC_DAD=y
+# CONFIG_INET6_AH is not set
+# CONFIG_INET6_ESP is not set
+# CONFIG_INET6_IPCOMP is not set
+CONFIG_IPV6_MIP6=y
+# CONFIG_IPV6_ILA is not set
+# CONFIG_INET6_XFRM_TUNNEL is not set
+# CONFIG_INET6_TUNNEL is not set
+# CONFIG_INET6_XFRM_MODE_TRANSPORT is not set
+# CONFIG_INET6_XFRM_MODE_TUNNEL is not set
+# CONFIG_INET6_XFRM_MODE_BEET is not set
+# CONFIG_INET6_XFRM_MODE_ROUTEOPTIMIZATION is not set
+# CONFIG_IPV6_SIT is not set
+# CONFIG_IPV6_TUNNEL is not set
+# CONFIG_IPV6_GRE is not set
+CONFIG_IPV6_MULTIPLE_TABLES=y
+CONFIG_IPV6_SUBTREES=y
+CONFIG_IPV6_MROUTE=y
+CONFIG_IPV6_MROUTE_MULTIPLE_TABLES=y
+CONFIG_IPV6_PIMSM_V2=y
+CONFIG_NETLABEL=y
+CONFIG_NETWORK_SECMARK=y
+CONFIG_NET_PTP_CLASSIFY=y
+CONFIG_NETWORK_PHY_TIMESTAMPING=y
+CONFIG_NETFILTER=y
+# CONFIG_NETFILTER_DEBUG is not set
+CONFIG_NETFILTER_ADVANCED=y
+
+#
+# Core Netfilter Configuration
+#
+CONFIG_NETFILTER_INGRESS=y
+# CONFIG_NETFILTER_NETLINK_ACCT is not set
+# CONFIG_NETFILTER_NETLINK_QUEUE is not set
+# CONFIG_NETFILTER_NETLINK_LOG is not set
+# CONFIG_NF_CONNTRACK is not set
+# CONFIG_NF_TABLES is not set
+CONFIG_NETFILTER_XTABLES=y
+
+#
+# Xtables combined modules
+#
+# CONFIG_NETFILTER_XT_MARK is not set
+
+#
+# Xtables targets
+#
+# CONFIG_NETFILTER_XT_TARGET_AUDIT is not set
+# CONFIG_NETFILTER_XT_TARGET_CLASSIFY is not set
+# CONFIG_NETFILTER_XT_TARGET_HMARK is not set
+# CONFIG_NETFILTER_XT_TARGET_IDLETIMER is not set
+# CONFIG_NETFILTER_XT_TARGET_LED is not set
+# CONFIG_NETFILTER_XT_TARGET_LOG is not set
+# CONFIG_NETFILTER_XT_TARGET_MARK is not set
+# CONFIG_NETFILTER_XT_TARGET_NFLOG is not set
+# CONFIG_NETFILTER_XT_TARGET_NFQUEUE is not set
+# CONFIG_NETFILTER_XT_TARGET_RATEEST is not set
+# CONFIG_NETFILTER_XT_TARGET_TEE is not set
+# CONFIG_NETFILTER_XT_TARGET_SECMARK is not set
+# CONFIG_NETFILTER_XT_TARGET_TCPMSS is not set
+
+#
+# Xtables matches
+#
+# CONFIG_NETFILTER_XT_MATCH_ADDRTYPE is not set
+# CONFIG_NETFILTER_XT_MATCH_BPF is not set
+# CONFIG_NETFILTER_XT_MATCH_CGROUP is not set
+# CONFIG_NETFILTER_XT_MATCH_COMMENT is not set
+# CONFIG_NETFILTER_XT_MATCH_CPU is not set
+# CONFIG_NETFILTER_XT_MATCH_DCCP is not set
+# CONFIG_NETFILTER_XT_MATCH_DEVGROUP is not set
+# CONFIG_NETFILTER_XT_MATCH_DSCP is not set
+# CONFIG_NETFILTER_XT_MATCH_ECN is not set
+# CONFIG_NETFILTER_XT_MATCH_ESP is not set
+# CONFIG_NETFILTER_XT_MATCH_HASHLIMIT is not set
+# CONFIG_NETFILTER_XT_MATCH_HL is not set
+# CONFIG_NETFILTER_XT_MATCH_IPCOMP is not set
+# CONFIG_NETFILTER_XT_MATCH_IPRANGE is not set
+# CONFIG_NETFILTER_XT_MATCH_L2TP is not set
+# CONFIG_NETFILTER_XT_MATCH_LENGTH is not set
+# CONFIG_NETFILTER_XT_MATCH_LIMIT is not set
+# CONFIG_NETFILTER_XT_MATCH_MAC is not set
+# CONFIG_NETFILTER_XT_MATCH_MARK is not set
+# CONFIG_NETFILTER_XT_MATCH_MULTIPORT is not set
+# CONFIG_NETFILTER_XT_MATCH_NFACCT is not set
+# CONFIG_NETFILTER_XT_MATCH_OWNER is not set
+# CONFIG_NETFILTER_XT_MATCH_POLICY is not set
+# CONFIG_NETFILTER_XT_MATCH_PKTTYPE is not set
+# CONFIG_NETFILTER_XT_MATCH_QUOTA is not set
+# CONFIG_NETFILTER_XT_MATCH_RATEEST is not set
+# CONFIG_NETFILTER_XT_MATCH_REALM is not set
+# CONFIG_NETFILTER_XT_MATCH_RECENT is not set
+# CONFIG_NETFILTER_XT_MATCH_SCTP is not set
+# CONFIG_NETFILTER_XT_MATCH_SOCKET is not set
+# CONFIG_NETFILTER_XT_MATCH_STATISTIC is not set
+# CONFIG_NETFILTER_XT_MATCH_STRING is not set
+# CONFIG_NETFILTER_XT_MATCH_TCPMSS is not set
+# CONFIG_NETFILTER_XT_MATCH_TIME is not set
+# CONFIG_NETFILTER_XT_MATCH_U32 is not set
+# CONFIG_IP_SET is not set
+# CONFIG_IP_VS is not set
+
+#
+# IP: Netfilter Configuration
+#
+# CONFIG_NF_DEFRAG_IPV4 is not set
+# CONFIG_NF_DUP_IPV4 is not set
+# CONFIG_NF_LOG_ARP is not set
+# CONFIG_NF_LOG_IPV4 is not set
+CONFIG_NF_REJECT_IPV4=y
+CONFIG_IP_NF_IPTABLES=y
+# CONFIG_IP_NF_MATCH_AH is not set
+# CONFIG_IP_NF_MATCH_ECN is not set
+# CONFIG_IP_NF_MATCH_TTL is not set
+CONFIG_IP_NF_FILTER=y
+CONFIG_IP_NF_TARGET_REJECT=y
+# CONFIG_IP_NF_MANGLE is not set
+# CONFIG_IP_NF_RAW is not set
+# CONFIG_IP_NF_SECURITY is not set
+# CONFIG_IP_NF_ARPTABLES is not set
+
+#
+# IPv6: Netfilter Configuration
+#
+# CONFIG_NF_DEFRAG_IPV6 is not set
+# CONFIG_NF_DUP_IPV6 is not set
+# CONFIG_NF_REJECT_IPV6 is not set
+# CONFIG_NF_LOG_IPV6 is not set
+# CONFIG_IP6_NF_IPTABLES is not set
+# CONFIG_IP_DCCP is not set
+# CONFIG_IP_SCTP is not set
+# CONFIG_RDS is not set
+# CONFIG_TIPC is not set
+# CONFIG_ATM is not set
+# CONFIG_L2TP is not set
+# CONFIG_BRIDGE is not set
+CONFIG_HAVE_NET_DSA=y
+# CONFIG_VLAN_8021Q is not set
+# CONFIG_DECNET is not set
+# CONFIG_LLC2 is not set
+# CONFIG_IPX is not set
+# CONFIG_ATALK is not set
+# CONFIG_X25 is not set
+# CONFIG_LAPB is not set
+# CONFIG_PHONET is not set
+# CONFIG_6LOWPAN is not set
+# CONFIG_IEEE802154 is not set
+CONFIG_NET_SCHED=y
+
+#
+# Queueing/Scheduling
+#
+# CONFIG_NET_SCH_CBQ is not set
+# CONFIG_NET_SCH_HTB is not set
+# CONFIG_NET_SCH_HFSC is not set
+# CONFIG_NET_SCH_PRIO is not set
+# CONFIG_NET_SCH_MULTIQ is not set
+# CONFIG_NET_SCH_RED is not set
+# CONFIG_NET_SCH_SFB is not set
+# CONFIG_NET_SCH_SFQ is not set
+# CONFIG_NET_SCH_TEQL is not set
+# CONFIG_NET_SCH_TBF is not set
+# CONFIG_NET_SCH_GRED is not set
+# CONFIG_NET_SCH_DSMARK is not set
+# CONFIG_NET_SCH_NETEM is not set
+# CONFIG_NET_SCH_DRR is not set
+# CONFIG_NET_SCH_MQPRIO is not set
+# CONFIG_NET_SCH_CHOKE is not set
+# CONFIG_NET_SCH_QFQ is not set
+# CONFIG_NET_SCH_CODEL is not set
+CONFIG_NET_SCH_FQ_CODEL=y
+# CONFIG_NET_SCH_FQ is not set
+# CONFIG_NET_SCH_HHF is not set
+# CONFIG_NET_SCH_PIE is not set
+# CONFIG_NET_SCH_INGRESS is not set
+# CONFIG_NET_SCH_PLUG is not set
+
+#
+# Classification
+#
+CONFIG_NET_CLS=y
+# CONFIG_NET_CLS_BASIC is not set
+# CONFIG_NET_CLS_TCINDEX is not set
+# CONFIG_NET_CLS_ROUTE4 is not set
+# CONFIG_NET_CLS_FW is not set
+# CONFIG_NET_CLS_U32 is not set
+# CONFIG_NET_CLS_RSVP is not set
+# CONFIG_NET_CLS_RSVP6 is not set
+# CONFIG_NET_CLS_FLOW is not set
+CONFIG_NET_CLS_CGROUP=y
+# CONFIG_NET_CLS_BPF is not set
+# CONFIG_NET_CLS_FLOWER is not set
+CONFIG_NET_EMATCH=y
+CONFIG_NET_EMATCH_STACK=32
+# CONFIG_NET_EMATCH_CMP is not set
+# CONFIG_NET_EMATCH_NBYTE is not set
+# CONFIG_NET_EMATCH_U32 is not set
+# CONFIG_NET_EMATCH_META is not set
+# CONFIG_NET_EMATCH_TEXT is not set
+CONFIG_NET_CLS_ACT=y
+# CONFIG_NET_ACT_POLICE is not set
+# CONFIG_NET_ACT_GACT is not set
+# CONFIG_NET_ACT_MIRRED is not set
+# CONFIG_NET_ACT_IPT is not set
+# CONFIG_NET_ACT_NAT is not set
+# CONFIG_NET_ACT_PEDIT is not set
+# CONFIG_NET_ACT_SIMP is not set
+# CONFIG_NET_ACT_SKBEDIT is not set
+# CONFIG_NET_ACT_CSUM is not set
+# CONFIG_NET_ACT_VLAN is not set
+# CONFIG_NET_ACT_BPF is not set
+CONFIG_NET_SCH_FIFO=y
+CONFIG_DCB=y
+CONFIG_DNS_RESOLVER=y
+# CONFIG_BATMAN_ADV is not set
+# CONFIG_OPENVSWITCH is not set
+# CONFIG_VSOCKETS is not set
+CONFIG_NETLINK_MMAP=y
+# CONFIG_NETLINK_DIAG is not set
+# CONFIG_MPLS is not set
+# CONFIG_HSR is not set
+# CONFIG_NET_SWITCHDEV is not set
+# CONFIG_NET_L3_MASTER_DEV is not set
+CONFIG_RPS=y
+CONFIG_RFS_ACCEL=y
+CONFIG_XPS=y
+CONFIG_CGROUP_NET_PRIO=y
+CONFIG_CGROUP_NET_CLASSID=y
+CONFIG_NET_RX_BUSY_POLL=y
+CONFIG_BQL=y
+# CONFIG_BPF_JIT is not set
+CONFIG_NET_FLOW_LIMIT=y
+
+#
+# Network testing
+#
+# CONFIG_NET_PKTGEN is not set
+CONFIG_NET_DROP_MONITOR=y
+# CONFIG_HAMRADIO is not set
+# CONFIG_CAN is not set
+# CONFIG_IRDA is not set
+# CONFIG_BT is not set
+# CONFIG_AF_RXRPC is not set
+CONFIG_FIB_RULES=y
+CONFIG_WIRELESS=y
+# CONFIG_CFG80211 is not set
+# CONFIG_LIB80211 is not set
+
+#
+# CFG80211 needs to be enabled for MAC80211
+#
+CONFIG_MAC80211_STA_HASH_MAX_SIZE=0
+# CONFIG_WIMAX is not set
+# CONFIG_RFKILL is not set
+# CONFIG_NET_9P is not set
+# CONFIG_CAIF is not set
+# CONFIG_CEPH_LIB is not set
+# CONFIG_NFC is not set
+# CONFIG_LWTUNNEL is not set
+CONFIG_HAVE_BPF_JIT=y
+
+#
+# Device Drivers
+#
+CONFIG_ARM_AMBA=y
+# CONFIG_TEGRA_AHB is not set
+
+#
+# Generic Driver Options
+#
+# CONFIG_UEVENT_HELPER is not set
+CONFIG_DEVTMPFS=y
+CONFIG_DEVTMPFS_MOUNT=y
+CONFIG_STANDALONE=y
+CONFIG_PREVENT_FIRMWARE_BUILD=y
+CONFIG_FW_LOADER=y
+# CONFIG_FIRMWARE_IN_KERNEL is not set
+CONFIG_EXTRA_FIRMWARE=""
+# CONFIG_FW_LOADER_USER_HELPER_FALLBACK is not set
+CONFIG_ALLOW_DEV_COREDUMP=y
+# CONFIG_DEBUG_DRIVER is not set
+CONFIG_DEBUG_DEVRES=y
+# CONFIG_SYS_HYPERVISOR is not set
+# CONFIG_GENERIC_CPU_DEVICES is not set
+CONFIG_GENERIC_CPU_AUTOPROBE=y
+CONFIG_REGMAP=y
+CONFIG_REGMAP_MMIO=y
+# CONFIG_DMA_SHARED_BUFFER is not set
+
+#
+# Bus devices
+#
+# CONFIG_ARM_CCI400_PMU is not set
+# CONFIG_ARM_CCI500_PMU is not set
+CONFIG_ARM_CCN=y
+CONFIG_VEXPRESS_CONFIG=y
+CONFIG_CONNECTOR=y
+CONFIG_PROC_EVENTS=y
+# CONFIG_MTD is not set
+CONFIG_DTC=y
+CONFIG_OF=y
+# CONFIG_OF_UNITTEST is not set
+CONFIG_OF_FLATTREE=y
+CONFIG_OF_EARLY_FLATTREE=y
+CONFIG_OF_ADDRESS=y
+CONFIG_OF_ADDRESS_PCI=y
+CONFIG_OF_IRQ=y
+CONFIG_OF_NET=y
+CONFIG_OF_MDIO=y
+CONFIG_OF_PCI=y
+CONFIG_OF_PCI_IRQ=y
+CONFIG_OF_RESERVED_MEM=y
+# CONFIG_OF_OVERLAY is not set
+# CONFIG_PARPORT is not set
+CONFIG_BLK_DEV=y
+# CONFIG_BLK_DEV_NULL_BLK is not set
+# CONFIG_BLK_DEV_PCIESSD_MTIP32XX is not set
+# CONFIG_ZRAM is not set
+# CONFIG_BLK_CPQ_CISS_DA is not set
+# CONFIG_BLK_DEV_DAC960 is not set
+# CONFIG_BLK_DEV_UMEM is not set
+# CONFIG_BLK_DEV_COW_COMMON is not set
+# CONFIG_BLK_DEV_LOOP is not set
+# CONFIG_BLK_DEV_DRBD is not set
+# CONFIG_BLK_DEV_NBD is not set
+# CONFIG_BLK_DEV_SKD is not set
+# CONFIG_BLK_DEV_SX8 is not set
+# CONFIG_BLK_DEV_RAM is not set
+# CONFIG_CDROM_PKTCDVD is not set
+# CONFIG_ATA_OVER_ETH is not set
+# CONFIG_BLK_DEV_RBD is not set
+# CONFIG_BLK_DEV_RSXX is not set
+# CONFIG_BLK_DEV_NVME is not set
+
+#
+# Misc devices
+#
+# CONFIG_SENSORS_LIS3LV02D is not set
+# CONFIG_DUMMY_IRQ is not set
+# CONFIG_PHANTOM is not set
+# CONFIG_SGI_IOC4 is not set
+# CONFIG_TIFM_CORE is not set
+# CONFIG_ENCLOSURE_SERVICES is not set
+# CONFIG_HP_ILO is not set
+# CONFIG_SRAM is not set
+CONFIG_VEXPRESS_SYSCFG=y
+# CONFIG_C2PORT is not set
+
+#
+# EEPROM support
+#
+# CONFIG_EEPROM_93CX6 is not set
+# CONFIG_CB710_CORE is not set
+
+#
+# Texas Instruments shared transport line discipline
+#
+# CONFIG_TI_ST is not set
+
+#
+# Altera FPGA firmware download module
+#
+
+#
+# Intel MIC Bus Driver
+#
+
+#
+# SCIF Bus Driver
+#
+
+#
+# Intel MIC Host Driver
+#
+
+#
+# Intel MIC Card Driver
+#
+
+#
+# SCIF Driver
+#
+
+#
+# Intel MIC Coprocessor State Management (COSM) Drivers
+#
+# CONFIG_GENWQE is not set
+# CONFIG_ECHO is not set
+# CONFIG_CXL_BASE is not set
+# CONFIG_CXL_KERNEL_API is not set
+# CONFIG_CXL_EEH is not set
+
+#
+# SCSI device support
+#
+CONFIG_SCSI_MOD=y
+# CONFIG_RAID_ATTRS is not set
+CONFIG_SCSI=y
+CONFIG_SCSI_DMA=y
+# CONFIG_SCSI_NETLINK is not set
+# CONFIG_SCSI_MQ_DEFAULT is not set
+CONFIG_SCSI_PROC_FS=y
+
+#
+# SCSI support type (disk, tape, CD-ROM)
+#
+CONFIG_BLK_DEV_SD=y
+# CONFIG_CHR_DEV_ST is not set
+# CONFIG_CHR_DEV_OSST is not set
+CONFIG_BLK_DEV_SR=y
+CONFIG_BLK_DEV_SR_VENDOR=y
+CONFIG_CHR_DEV_SG=y
+# CONFIG_CHR_DEV_SCH is not set
+CONFIG_SCSI_CONSTANTS=y
+CONFIG_SCSI_LOGGING=y
+CONFIG_SCSI_SCAN_ASYNC=y
+
+#
+# SCSI Transports
+#
+# CONFIG_SCSI_SPI_ATTRS is not set
+# CONFIG_SCSI_FC_ATTRS is not set
+# CONFIG_SCSI_ISCSI_ATTRS is not set
+# CONFIG_SCSI_SAS_ATTRS is not set
+# CONFIG_SCSI_SAS_LIBSAS is not set
+# CONFIG_SCSI_SRP_ATTRS is not set
+CONFIG_SCSI_LOWLEVEL=y
+# CONFIG_ISCSI_TCP is not set
+# CONFIG_ISCSI_BOOT_SYSFS is not set
+# CONFIG_SCSI_CXGB3_ISCSI is not set
+# CONFIG_SCSI_CXGB4_ISCSI is not set
+# CONFIG_SCSI_BNX2_ISCSI is not set
+# CONFIG_BE2ISCSI is not set
+# CONFIG_BLK_DEV_3W_XXXX_RAID is not set
+# CONFIG_SCSI_HPSA is not set
+# CONFIG_SCSI_3W_9XXX is not set
+# CONFIG_SCSI_3W_SAS is not set
+# CONFIG_SCSI_ACARD is not set
+# CONFIG_SCSI_AACRAID is not set
+# CONFIG_SCSI_AIC7XXX is not set
+# CONFIG_SCSI_AIC79XX is not set
+# CONFIG_SCSI_AIC94XX is not set
+# CONFIG_SCSI_MVSAS is not set
+# CONFIG_SCSI_MVUMI is not set
+# CONFIG_SCSI_ADVANSYS is not set
+# CONFIG_SCSI_ARCMSR is not set
+# CONFIG_SCSI_ESAS2R is not set
+CONFIG_MEGARAID_NEWGEN=y
+# CONFIG_MEGARAID_MM is not set
+# CONFIG_MEGARAID_LEGACY is not set
+# CONFIG_MEGARAID_SAS is not set
+# CONFIG_SCSI_MPT3SAS is not set
+# CONFIG_SCSI_MPT2SAS is not set
+# CONFIG_SCSI_UFSHCD is not set
+# CONFIG_SCSI_HPTIOP is not set
+# CONFIG_SCSI_SNIC is not set
+# CONFIG_SCSI_DMX3191D is not set
+# CONFIG_SCSI_FUTURE_DOMAIN is not set
+# CONFIG_SCSI_IPS is not set
+# CONFIG_SCSI_INITIO is not set
+# CONFIG_SCSI_INIA100 is not set
+# CONFIG_SCSI_STEX is not set
+# CONFIG_SCSI_SYM53C8XX_2 is not set
+# CONFIG_SCSI_IPR is not set
+# CONFIG_SCSI_QLOGIC_1280 is not set
+# CONFIG_SCSI_QLA_ISCSI is not set
+# CONFIG_SCSI_DC395x is not set
+# CONFIG_SCSI_AM53C974 is not set
+# CONFIG_SCSI_WD719X is not set
+# CONFIG_SCSI_DEBUG is not set
+# CONFIG_SCSI_PMCRAID is not set
+# CONFIG_SCSI_PM8001 is not set
+# CONFIG_SCSI_LOWLEVEL_PCMCIA is not set
+CONFIG_SCSI_DH=y
+# CONFIG_SCSI_DH_RDAC is not set
+# CONFIG_SCSI_DH_HP_SW is not set
+# CONFIG_SCSI_DH_EMC is not set
+# CONFIG_SCSI_DH_ALUA is not set
+# CONFIG_SCSI_OSD_INITIATOR is not set
+CONFIG_HAVE_PATA_PLATFORM=y
+CONFIG_ATA=y
+# CONFIG_ATA_NONSTANDARD is not set
+CONFIG_ATA_VERBOSE_ERROR=y
+CONFIG_SATA_PMP=y
+
+#
+# Controllers with non-SFF native interface
+#
+CONFIG_SATA_AHCI=y
+CONFIG_SATA_AHCI_PLATFORM=y
+# CONFIG_AHCI_CEVA is not set
+CONFIG_AHCI_XGENE=y
+# CONFIG_AHCI_QORIQ is not set
+# CONFIG_SATA_INIC162X is not set
+# CONFIG_SATA_ACARD_AHCI is not set
+# CONFIG_SATA_SIL24 is not set
+CONFIG_ATA_SFF=y
+
+#
+# SFF controllers with custom DMA interface
+#
+# CONFIG_PDC_ADMA is not set
+# CONFIG_SATA_QSTOR is not set
+# CONFIG_SATA_SX4 is not set
+CONFIG_ATA_BMDMA=y
+
+#
+# SATA SFF controllers with BMDMA
+#
+CONFIG_ATA_PIIX=y
+# CONFIG_SATA_MV is not set
+# CONFIG_SATA_NV is not set
+# CONFIG_SATA_PROMISE is not set
+# CONFIG_SATA_SIL is not set
+# CONFIG_SATA_SIS is not set
+# CONFIG_SATA_SVW is not set
+# CONFIG_SATA_ULI is not set
+# CONFIG_SATA_VIA is not set
+# CONFIG_SATA_VITESSE is not set
+
+#
+# PATA SFF controllers with BMDMA
+#
+# CONFIG_PATA_ALI is not set
+# CONFIG_PATA_AMD is not set
+# CONFIG_PATA_ARTOP is not set
+# CONFIG_PATA_ATIIXP is not set
+# CONFIG_PATA_ATP867X is not set
+# CONFIG_PATA_CMD64X is not set
+# CONFIG_PATA_CYPRESS is not set
+# CONFIG_PATA_EFAR is not set
+# CONFIG_PATA_HPT366 is not set
+# CONFIG_PATA_HPT37X is not set
+# CONFIG_PATA_HPT3X2N is not set
+# CONFIG_PATA_HPT3X3 is not set
+# CONFIG_PATA_IT8213 is not set
+# CONFIG_PATA_IT821X is not set
+# CONFIG_PATA_JMICRON is not set
+# CONFIG_PATA_MARVELL is not set
+# CONFIG_PATA_NETCELL is not set
+# CONFIG_PATA_NINJA32 is not set
+# CONFIG_PATA_NS87415 is not set
+# CONFIG_PATA_OLDPIIX is not set
+# CONFIG_PATA_OPTIDMA is not set
+# CONFIG_PATA_PDC2027X is not set
+# CONFIG_PATA_PDC_OLD is not set
+# CONFIG_PATA_RADISYS is not set
+# CONFIG_PATA_RDC is not set
+# CONFIG_PATA_SCH is not set
+# CONFIG_PATA_SERVERWORKS is not set
+# CONFIG_PATA_SIL680 is not set
+# CONFIG_PATA_SIS is not set
+# CONFIG_PATA_TOSHIBA is not set
+# CONFIG_PATA_TRIFLEX is not set
+# CONFIG_PATA_VIA is not set
+# CONFIG_PATA_WINBOND is not set
+
+#
+# PIO-only SFF controllers
+#
+# CONFIG_PATA_CMD640_PCI is not set
+# CONFIG_PATA_MPIIX is not set
+# CONFIG_PATA_NS87410 is not set
+# CONFIG_PATA_OPTI is not set
+# CONFIG_PATA_PLATFORM is not set
+# CONFIG_PATA_RZ1000 is not set
+
+#
+# Generic fallback / legacy drivers
+#
+# CONFIG_ATA_GENERIC is not set
+# CONFIG_PATA_LEGACY is not set
+CONFIG_MD=y
+CONFIG_BLK_DEV_MD=y
+CONFIG_MD_AUTODETECT=y
+# CONFIG_MD_LINEAR is not set
+# CONFIG_MD_RAID0 is not set
+# CONFIG_MD_RAID1 is not set
+# CONFIG_MD_RAID10 is not set
+# CONFIG_MD_RAID456 is not set
+# CONFIG_MD_MULTIPATH is not set
+# CONFIG_MD_FAULTY is not set
+# CONFIG_BCACHE is not set
+CONFIG_BLK_DEV_DM_BUILTIN=y
+CONFIG_BLK_DEV_DM=y
+# CONFIG_DM_MQ_DEFAULT is not set
+CONFIG_DM_DEBUG=y
+CONFIG_DM_BUFIO=y
+# CONFIG_DM_CRYPT is not set
+CONFIG_DM_SNAPSHOT=y
+# CONFIG_DM_THIN_PROVISIONING is not set
+# CONFIG_DM_CACHE is not set
+# CONFIG_DM_ERA is not set
+CONFIG_DM_MIRROR=y
+# CONFIG_DM_LOG_USERSPACE is not set
+# CONFIG_DM_RAID is not set
+CONFIG_DM_ZERO=y
+# CONFIG_DM_MULTIPATH is not set
+# CONFIG_DM_DELAY is not set
+CONFIG_DM_UEVENT=y
+# CONFIG_DM_FLAKEY is not set
+# CONFIG_DM_VERITY is not set
+# CONFIG_DM_SWITCH is not set
+# CONFIG_DM_LOG_WRITES is not set
+# CONFIG_TARGET_CORE is not set
+# CONFIG_FUSION is not set
+
+#
+# IEEE 1394 (FireWire) support
+#
+# CONFIG_FIREWIRE is not set
+# CONFIG_FIREWIRE_NOSY is not set
+CONFIG_NETDEVICES=y
+CONFIG_NET_CORE=y
+# CONFIG_BONDING is not set
+# CONFIG_DUMMY is not set
+# CONFIG_EQUALIZER is not set
+CONFIG_NET_FC=y
+# CONFIG_IFB is not set
+# CONFIG_NET_TEAM is not set
+# CONFIG_MACVLAN is not set
+# CONFIG_IPVLAN is not set
+# CONFIG_VXLAN is not set
+# CONFIG_NETCONSOLE is not set
+# CONFIG_NETPOLL is not set
+# CONFIG_NET_POLL_CONTROLLER is not set
+# CONFIG_TUN is not set
+# CONFIG_TUN_VNET_CROSS_LE is not set
+# CONFIG_VETH is not set
+# CONFIG_NLMON is not set
+# CONFIG_ARCNET is not set
+
+#
+# CAIF transport drivers
+#
+
+#
+# Distributed Switch Architecture drivers
+#
+# CONFIG_NET_DSA_MV88E6XXX is not set
+# CONFIG_NET_DSA_MV88E6XXX_NEED_PPU is not set
+CONFIG_ETHERNET=y
+# CONFIG_NET_VENDOR_3COM is not set
+# CONFIG_NET_VENDOR_ADAPTEC is not set
+CONFIG_NET_VENDOR_AGERE=y
+# CONFIG_ET131X is not set
+CONFIG_NET_VENDOR_ALTEON=y
+# CONFIG_ACENIC is not set
+# CONFIG_ALTERA_TSE is not set
+CONFIG_NET_VENDOR_AMD=y
+# CONFIG_AMD8111_ETH is not set
+# CONFIG_PCNET32 is not set
+CONFIG_AMD_XGBE=y
+# CONFIG_AMD_XGBE_DCB is not set
+CONFIG_NET_VENDOR_ARC=y
+# CONFIG_ARC_EMAC is not set
+CONFIG_NET_VENDOR_ATHEROS=y
+# CONFIG_ATL2 is not set
+# CONFIG_ATL1 is not set
+# CONFIG_ATL1E is not set
+# CONFIG_ATL1C is not set
+# CONFIG_ALX is not set
+# CONFIG_NET_VENDOR_AURORA is not set
+CONFIG_NET_CADENCE=y
+# CONFIG_MACB is not set
+CONFIG_NET_VENDOR_BROADCOM=y
+# CONFIG_B44 is not set
+# CONFIG_BCMGENET is not set
+# CONFIG_BNX2 is not set
+# CONFIG_CNIC is not set
+# CONFIG_TIGON3 is not set
+# CONFIG_BNX2X is not set
+# CONFIG_SYSTEMPORT is not set
+# CONFIG_BNXT is not set
+# CONFIG_NET_VENDOR_BROCADE is not set
+CONFIG_NET_VENDOR_CAVIUM=y
+# CONFIG_THUNDER_NIC_PF is not set
+# CONFIG_THUNDER_NIC_VF is not set
+# CONFIG_THUNDER_NIC_BGX is not set
+# CONFIG_LIQUIDIO is not set
+# CONFIG_NET_VENDOR_CHELSIO is not set
+# CONFIG_NET_VENDOR_CISCO is not set
+# CONFIG_DNET is not set
+# CONFIG_NET_VENDOR_DEC is not set
+CONFIG_NET_VENDOR_DLINK=y
+# CONFIG_DL2K is not set
+# CONFIG_SUNDANCE is not set
+# CONFIG_NET_VENDOR_EMULEX is not set
+CONFIG_NET_VENDOR_EZCHIP=y
+# CONFIG_EZCHIP_NPS_MANAGEMENT_ENET is not set
+# CONFIG_NET_VENDOR_EXAR is not set
+CONFIG_NET_VENDOR_HISILICON=y
+# CONFIG_HIX5HD2_GMAC is not set
+# CONFIG_HIP04_ETH is not set
+# CONFIG_HNS is not set
+# CONFIG_HNS_DSAF is not set
+# CONFIG_HNS_ENET is not set
+# CONFIG_NET_VENDOR_HP is not set
+CONFIG_NET_VENDOR_INTEL=y
+# CONFIG_E100 is not set
+# CONFIG_E1000 is not set
+# CONFIG_E1000E is not set
+# CONFIG_IGB is not set
+# CONFIG_IGBVF is not set
+# CONFIG_IXGB is not set
+# CONFIG_IXGBE is not set
+# CONFIG_IXGBEVF is not set
+# CONFIG_I40E is not set
+# CONFIG_I40EVF is not set
+# CONFIG_FM10K is not set
+# CONFIG_NET_VENDOR_I825XX is not set
+# CONFIG_JME is not set
+CONFIG_NET_VENDOR_MARVELL=y
+# CONFIG_MVMDIO is not set
+# CONFIG_SKGE is not set
+# CONFIG_SKY2 is not set
+# CONFIG_NET_VENDOR_MELLANOX is not set
+CONFIG_NET_VENDOR_MICREL=y
+# CONFIG_KS8842 is not set
+# CONFIG_KS8851_MLL is not set
+# CONFIG_KSZ884X_PCI is not set
+CONFIG_NET_VENDOR_MYRI=y
+# CONFIG_MYRI10GE is not set
+# CONFIG_FEALNX is not set
+CONFIG_NET_VENDOR_NATSEMI=y
+# CONFIG_NATSEMI is not set
+# CONFIG_NS83820 is not set
+CONFIG_NET_VENDOR_8390=y
+# CONFIG_NE2K_PCI is not set
+CONFIG_NET_VENDOR_NVIDIA=y
+# CONFIG_FORCEDETH is not set
+CONFIG_NET_VENDOR_OKI=y
+# CONFIG_ETHOC is not set
+CONFIG_NET_PACKET_ENGINE=y
+# CONFIG_HAMACHI is not set
+# CONFIG_YELLOWFIN is not set
+# CONFIG_NET_VENDOR_QLOGIC is not set
+CONFIG_NET_VENDOR_QUALCOMM=y
+CONFIG_NET_VENDOR_REALTEK=y
+# CONFIG_8139CP is not set
+# CONFIG_8139TOO is not set
+# CONFIG_R8169 is not set
+CONFIG_NET_VENDOR_RENESAS=y
+CONFIG_NET_VENDOR_RDC=y
+# CONFIG_R6040 is not set
+CONFIG_NET_VENDOR_ROCKER=y
+# CONFIG_NET_VENDOR_SAMSUNG is not set
+# CONFIG_NET_VENDOR_SEEQ is not set
+CONFIG_NET_VENDOR_SILAN=y
+# CONFIG_SC92031 is not set
+CONFIG_NET_VENDOR_SIS=y
+# CONFIG_SIS900 is not set
+# CONFIG_SIS190 is not set
+# CONFIG_SFC is not set
+CONFIG_NET_VENDOR_SMSC=y
+# CONFIG_SMC91X is not set
+# CONFIG_EPIC100 is not set
+# CONFIG_SMSC911X is not set
+# CONFIG_SMSC9420 is not set
+CONFIG_NET_VENDOR_STMICRO=y
+# CONFIG_STMMAC_ETH is not set
+# CONFIG_NET_VENDOR_SUN is not set
+CONFIG_NET_VENDOR_SYNOPSYS=y
+# CONFIG_SYNOPSYS_DWC_ETH_QOS is not set
+CONFIG_NET_VENDOR_TEHUTI=y
+# CONFIG_TEHUTI is not set
+CONFIG_NET_VENDOR_TI=y
+# CONFIG_TI_CPSW_ALE is not set
+# CONFIG_TLAN is not set
+CONFIG_NET_VENDOR_VIA=y
+# CONFIG_VIA_RHINE is not set
+# CONFIG_VIA_VELOCITY is not set
+# CONFIG_NET_VENDOR_WIZNET is not set
+# CONFIG_FDDI is not set
+# CONFIG_HIPPI is not set
+CONFIG_PHYLIB=y
+
+#
+# MII PHY device drivers
+#
+# CONFIG_AQUANTIA_PHY is not set
+# CONFIG_AT803X_PHY is not set
+# CONFIG_AMD_PHY is not set
+# CONFIG_MARVELL_PHY is not set
+# CONFIG_DAVICOM_PHY is not set
+# CONFIG_QSEMI_PHY is not set
+# CONFIG_LXT_PHY is not set
+# CONFIG_CICADA_PHY is not set
+# CONFIG_VITESSE_PHY is not set
+# CONFIG_TERANETICS_PHY is not set
+# CONFIG_SMSC_PHY is not set
+# CONFIG_BROADCOM_PHY is not set
+# CONFIG_BCM7XXX_PHY is not set
+# CONFIG_BCM87XX_PHY is not set
+# CONFIG_ICPLUS_PHY is not set
+# CONFIG_REALTEK_PHY is not set
+# CONFIG_NATIONAL_PHY is not set
+# CONFIG_STE10XP is not set
+# CONFIG_LSI_ET1011C_PHY is not set
+# CONFIG_MICREL_PHY is not set
+# CONFIG_DP83848_PHY is not set
+# CONFIG_DP83867_PHY is not set
+# CONFIG_MICROCHIP_PHY is not set
+CONFIG_FIXED_PHY=y
+# CONFIG_MDIO_BITBANG is not set
+# CONFIG_MDIO_OCTEON is not set
+# CONFIG_MDIO_BUS_MUX_GPIO is not set
+# CONFIG_MDIO_BUS_MUX_MMIOREG is not set
+# CONFIG_MDIO_BCM_UNIMAC is not set
+# CONFIG_PPP is not set
+# CONFIG_SLIP is not set
+CONFIG_USB_NET_DRIVERS=y
+# CONFIG_USB_CATC is not set
+# CONFIG_USB_KAWETH is not set
+# CONFIG_USB_PEGASUS is not set
+# CONFIG_USB_RTL8150 is not set
+# CONFIG_USB_RTL8152 is not set
+# CONFIG_USB_LAN78XX is not set
+# CONFIG_USB_USBNET is not set
+# CONFIG_USB_IPHETH is not set
+CONFIG_WLAN=y
+# CONFIG_PRISM54 is not set
+# CONFIG_HOSTAP is not set
+# CONFIG_WL_MEDIATEK is not set
+# CONFIG_WL_TI is not set
+
+#
+# Enable WiMAX (Networking options) to see the WiMAX drivers
+#
+# CONFIG_WAN is not set
+# CONFIG_VMXNET3 is not set
+# CONFIG_ISDN is not set
+# CONFIG_NVM is not set
+
+#
+# Input device support
+#
+CONFIG_INPUT=y
+CONFIG_INPUT_LEDS=y
+CONFIG_INPUT_FF_MEMLESS=y
+# CONFIG_INPUT_POLLDEV is not set
+# CONFIG_INPUT_SPARSEKMAP is not set
+# CONFIG_INPUT_MATRIXKMAP is not set
+
+#
+# Userland interfaces
+#
+CONFIG_INPUT_MOUSEDEV=y
+# CONFIG_INPUT_MOUSEDEV_PSAUX is not set
+CONFIG_INPUT_MOUSEDEV_SCREEN_X=1024
+CONFIG_INPUT_MOUSEDEV_SCREEN_Y=768
+# CONFIG_INPUT_JOYDEV is not set
+CONFIG_INPUT_EVDEV=y
+# CONFIG_INPUT_EVBUG is not set
+
+#
+# Input Device Drivers
+#
+CONFIG_INPUT_KEYBOARD=y
+CONFIG_KEYBOARD_ATKBD=y
+# CONFIG_KEYBOARD_LKKBD is not set
+# CONFIG_KEYBOARD_GPIO is not set
+# CONFIG_KEYBOARD_GPIO_POLLED is not set
+# CONFIG_KEYBOARD_MATRIX is not set
+# CONFIG_KEYBOARD_NEWTON is not set
+# CONFIG_KEYBOARD_OPENCORES is not set
+# CONFIG_KEYBOARD_SAMSUNG is not set
+# CONFIG_KEYBOARD_STOWAWAY is not set
+# CONFIG_KEYBOARD_SUNKBD is not set
+# CONFIG_KEYBOARD_OMAP4 is not set
+# CONFIG_KEYBOARD_XTKBD is not set
+# CONFIG_KEYBOARD_BCM is not set
+CONFIG_INPUT_MOUSE=y
+CONFIG_MOUSE_PS2=y
+CONFIG_MOUSE_PS2_ALPS=y
+CONFIG_MOUSE_PS2_LOGIPS2PP=y
+CONFIG_MOUSE_PS2_SYNAPTICS=y
+CONFIG_MOUSE_PS2_CYPRESS=y
+CONFIG_MOUSE_PS2_TRACKPOINT=y
+CONFIG_MOUSE_PS2_ELANTECH=y
+CONFIG_MOUSE_PS2_SENTELIC=y
+# CONFIG_MOUSE_PS2_TOUCHKIT is not set
+CONFIG_MOUSE_PS2_FOCALTECH=y
+# CONFIG_MOUSE_SERIAL is not set
+# CONFIG_MOUSE_APPLETOUCH is not set
+# CONFIG_MOUSE_BCM5974 is not set
+# CONFIG_MOUSE_VSXXXAA is not set
+# CONFIG_MOUSE_GPIO is not set
+# CONFIG_MOUSE_SYNAPTICS_USB is not set
+CONFIG_INPUT_JOYSTICK=y
+# CONFIG_JOYSTICK_ANALOG is not set
+# CONFIG_JOYSTICK_A3D is not set
+# CONFIG_JOYSTICK_ADI is not set
+# CONFIG_JOYSTICK_COBRA is not set
+# CONFIG_JOYSTICK_GF2K is not set
+# CONFIG_JOYSTICK_GRIP is not set
+# CONFIG_JOYSTICK_GRIP_MP is not set
+# CONFIG_JOYSTICK_GUILLEMOT is not set
+# CONFIG_JOYSTICK_INTERACT is not set
+# CONFIG_JOYSTICK_SIDEWINDER is not set
+# CONFIG_JOYSTICK_TMDC is not set
+# CONFIG_JOYSTICK_IFORCE is not set
+# CONFIG_JOYSTICK_WARRIOR is not set
+# CONFIG_JOYSTICK_MAGELLAN is not set
+# CONFIG_JOYSTICK_SPACEORB is not set
+# CONFIG_JOYSTICK_SPACEBALL is not set
+# CONFIG_JOYSTICK_STINGER is not set
+# CONFIG_JOYSTICK_TWIDJOY is not set
+# CONFIG_JOYSTICK_ZHENHUA is not set
+# CONFIG_JOYSTICK_JOYDUMP is not set
+# CONFIG_JOYSTICK_XPAD is not set
+CONFIG_INPUT_TABLET=y
+# CONFIG_TABLET_USB_ACECAD is not set
+# CONFIG_TABLET_USB_AIPTEK is not set
+# CONFIG_TABLET_USB_GTCO is not set
+# CONFIG_TABLET_USB_HANWANG is not set
+# CONFIG_TABLET_USB_KBTAB is not set
+# CONFIG_TABLET_SERIAL_WACOM4 is not set
+CONFIG_INPUT_TOUCHSCREEN=y
+CONFIG_TOUCHSCREEN_PROPERTIES=y
+# CONFIG_TOUCHSCREEN_AD7879 is not set
+# CONFIG_TOUCHSCREEN_CYTTSP_CORE is not set
+# CONFIG_TOUCHSCREEN_CYTTSP4_CORE is not set
+# CONFIG_TOUCHSCREEN_DYNAPRO is not set
+# CONFIG_TOUCHSCREEN_HAMPSHIRE is not set
+# CONFIG_TOUCHSCREEN_FUJITSU is not set
+# CONFIG_TOUCHSCREEN_GUNZE is not set
+# CONFIG_TOUCHSCREEN_ELO is not set
+# CONFIG_TOUCHSCREEN_WACOM_W8001 is not set
+# CONFIG_TOUCHSCREEN_MTOUCH is not set
+# CONFIG_TOUCHSCREEN_IMX6UL_TSC is not set
+# CONFIG_TOUCHSCREEN_INEXIO is not set
+# CONFIG_TOUCHSCREEN_MK712 is not set
+# CONFIG_TOUCHSCREEN_PENMOUNT is not set
+# CONFIG_TOUCHSCREEN_TOUCHRIGHT is not set
+# CONFIG_TOUCHSCREEN_TOUCHWIN is not set
+# CONFIG_TOUCHSCREEN_USB_COMPOSITE is not set
+# CONFIG_TOUCHSCREEN_TOUCHIT213 is not set
+# CONFIG_TOUCHSCREEN_TSC_SERIO is not set
+CONFIG_INPUT_MISC=y
+# CONFIG_INPUT_AD714X is not set
+# CONFIG_INPUT_E3X0_BUTTON is not set
+# CONFIG_INPUT_GPIO_BEEPER is not set
+# CONFIG_INPUT_GPIO_TILT_POLLED is not set
+# CONFIG_INPUT_ATI_REMOTE2 is not set
+# CONFIG_INPUT_KEYSPAN_REMOTE is not set
+# CONFIG_INPUT_POWERMATE is not set
+# CONFIG_INPUT_YEALINK is not set
+# CONFIG_INPUT_CM109 is not set
+# CONFIG_INPUT_UINPUT is not set
+# CONFIG_INPUT_PWM_BEEPER is not set
+# CONFIG_INPUT_GPIO_ROTARY_ENCODER is not set
+# CONFIG_INPUT_ADXL34X is not set
+# CONFIG_INPUT_IMS_PCU is not set
+# CONFIG_INPUT_CMA3000 is not set
+
+#
+# Hardware I/O ports
+#
+CONFIG_SERIO=y
+CONFIG_SERIO_SERPORT=y
+CONFIG_SERIO_AMBAKMI=y
+# CONFIG_SERIO_PCIPS2 is not set
+CONFIG_SERIO_LIBPS2=y
+# CONFIG_SERIO_RAW is not set
+# CONFIG_SERIO_ALTERA_PS2 is not set
+# CONFIG_SERIO_PS2MULT is not set
+# CONFIG_SERIO_ARC_PS2 is not set
+# CONFIG_SERIO_APBPS2 is not set
+# CONFIG_USERIO is not set
+# CONFIG_GAMEPORT is not set
+
+#
+# Character devices
+#
+CONFIG_TTY=y
+CONFIG_VT=y
+CONFIG_CONSOLE_TRANSLATIONS=y
+CONFIG_VT_CONSOLE=y
+CONFIG_HW_CONSOLE=y
+CONFIG_VT_HW_CONSOLE_BINDING=y
+CONFIG_UNIX98_PTYS=y
+CONFIG_DEVPTS_MULTIPLE_INSTANCES=y
+# CONFIG_LEGACY_PTYS is not set
+CONFIG_SERIAL_NONSTANDARD=y
+# CONFIG_ROCKETPORT is not set
+# CONFIG_CYCLADES is not set
+# CONFIG_MOXA_INTELLIO is not set
+# CONFIG_MOXA_SMARTIO is not set
+# CONFIG_SYNCLINKMP is not set
+# CONFIG_SYNCLINK_GT is not set
+# CONFIG_NOZOMI is not set
+# CONFIG_ISI is not set
+# CONFIG_N_HDLC is not set
+# CONFIG_N_GSM is not set
+# CONFIG_TRACE_SINK is not set
+CONFIG_DEVMEM=y
+# CONFIG_DEVKMEM is not set
+
+#
+# Serial drivers
+#
+CONFIG_SERIAL_EARLYCON=y
+CONFIG_SERIAL_8250=y
+# CONFIG_SERIAL_8250_DEPRECATED_OPTIONS is not set
+CONFIG_SERIAL_8250_CONSOLE=y
+CONFIG_SERIAL_8250_DMA=y
+CONFIG_SERIAL_8250_PCI=y
+CONFIG_SERIAL_8250_NR_UARTS=32
+CONFIG_SERIAL_8250_RUNTIME_UARTS=4
+CONFIG_SERIAL_8250_EXTENDED=y
+CONFIG_SERIAL_8250_MANY_PORTS=y
+CONFIG_SERIAL_8250_SHARE_IRQ=y
+# CONFIG_SERIAL_8250_DETECT_IRQ is not set
+CONFIG_SERIAL_8250_RSA=y
+CONFIG_SERIAL_8250_FSL=y
+# CONFIG_SERIAL_8250_DW is not set
+# CONFIG_SERIAL_8250_RT288X is not set
+# CONFIG_SERIAL_8250_INGENIC is not set
+# CONFIG_SERIAL_8250_MID is not set
+
+#
+# Non-8250 serial port support
+#
+# CONFIG_SERIAL_AMBA_PL010 is not set
+CONFIG_SERIAL_AMBA_PL011=y
+CONFIG_SERIAL_AMBA_PL011_CONSOLE=y
+CONFIG_SERIAL_EARLYCON_ARM_SEMIHOST=y
+# CONFIG_SERIAL_UARTLITE is not set
+CONFIG_SERIAL_CORE=y
+CONFIG_SERIAL_CORE_CONSOLE=y
+# CONFIG_SERIAL_JSM is not set
+CONFIG_SERIAL_OF_PLATFORM=y
+# CONFIG_SERIAL_SCCNXP is not set
+# CONFIG_SERIAL_ALTERA_JTAGUART is not set
+# CONFIG_SERIAL_ALTERA_UART is not set
+# CONFIG_SERIAL_XILINX_PS_UART is not set
+# CONFIG_SERIAL_ARC is not set
+# CONFIG_SERIAL_RP2 is not set
+# CONFIG_SERIAL_FSL_LPUART is not set
+# CONFIG_SERIAL_CONEXANT_DIGICOLOR is not set
+# CONFIG_HVC_DCC is not set
+# CONFIG_IPMI_HANDLER is not set
+CONFIG_HW_RANDOM=y
+# CONFIG_HW_RANDOM_TIMERIOMEM is not set
+# CONFIG_R3964 is not set
+# CONFIG_APPLICOM is not set
+
+#
+# PCMCIA character devices
+#
+CONFIG_RAW_DRIVER=y
+CONFIG_MAX_RAW_DEVS=8192
+# CONFIG_TCG_TPM is not set
+CONFIG_DEVPORT=y
+# CONFIG_XILLYBUS is not set
+
+#
+# I2C support
+#
+# CONFIG_I2C is not set
+# CONFIG_SPI is not set
+# CONFIG_SPMI is not set
+# CONFIG_HSI is not set
+
+#
+# PPS support
+#
+CONFIG_PPS=y
+# CONFIG_PPS_DEBUG is not set
+
+#
+# PPS clients support
+#
+# CONFIG_PPS_CLIENT_KTIMER is not set
+# CONFIG_PPS_CLIENT_LDISC is not set
+# CONFIG_PPS_CLIENT_GPIO is not set
+
+#
+# PPS generators support
+#
+
+#
+# PTP clock support
+#
+CONFIG_PTP_1588_CLOCK=y
+# CONFIG_DP83640_PHY is not set
+CONFIG_ARCH_WANT_OPTIONAL_GPIOLIB=y
+CONFIG_GPIOLIB=y
+CONFIG_GPIO_DEVRES=y
+CONFIG_OF_GPIO=y
+# CONFIG_DEBUG_GPIO is not set
+CONFIG_GPIO_SYSFS=y
+CONFIG_GPIO_GENERIC=y
+
+#
+# Memory mapped GPIO drivers
+#
+# CONFIG_GPIO_74XX_MMIO is not set
+# CONFIG_GPIO_ALTERA is not set
+# CONFIG_GPIO_DWAPB is not set
+CONFIG_GPIO_GENERIC_PLATFORM=y
+# CONFIG_GPIO_GRGPIO is not set
+# CONFIG_GPIO_PL061 is not set
+# CONFIG_GPIO_SYSCON is not set
+# CONFIG_GPIO_VX855 is not set
+# CONFIG_GPIO_XGENE is not set
+# CONFIG_GPIO_XILINX is not set
+# CONFIG_GPIO_ZX is not set
+
+#
+# MFD GPIO expanders
+#
+
+#
+# PCI GPIO expanders
+#
+# CONFIG_GPIO_AMD8111 is not set
+# CONFIG_GPIO_BT8XX is not set
+# CONFIG_GPIO_ML_IOH is not set
+# CONFIG_GPIO_RDC321X is not set
+
+#
+# USB GPIO expanders
+#
+# CONFIG_W1 is not set
+CONFIG_POWER_SUPPLY=y
+# CONFIG_POWER_SUPPLY_DEBUG is not set
+# CONFIG_PDA_POWER is not set
+# CONFIG_TEST_POWER is not set
+# CONFIG_BATTERY_DS2780 is not set
+# CONFIG_BATTERY_DS2781 is not set
+# CONFIG_BATTERY_BQ27XXX is not set
+# CONFIG_CHARGER_MAX8903 is not set
+# CONFIG_CHARGER_GPIO is not set
+CONFIG_POWER_RESET=y
+CONFIG_POWER_RESET_GPIO=y
+# CONFIG_POWER_RESET_GPIO_RESTART is not set
+# CONFIG_POWER_RESET_LTC2952 is not set
+# CONFIG_POWER_RESET_RESTART is not set
+CONFIG_POWER_RESET_VEXPRESS=y
+CONFIG_POWER_RESET_XGENE=y
+# CONFIG_POWER_RESET_SYSCON is not set
+# CONFIG_POWER_RESET_SYSCON_POWEROFF is not set
+# CONFIG_POWER_AVS is not set
+CONFIG_HWMON=y
+# CONFIG_HWMON_VID is not set
+# CONFIG_HWMON_DEBUG_CHIP is not set
+
+#
+# Native drivers
+#
+# CONFIG_SENSORS_I5K_AMB is not set
+# CONFIG_SENSORS_F71805F is not set
+# CONFIG_SENSORS_F71882FG is not set
+# CONFIG_SENSORS_GPIO_FAN is not set
+# CONFIG_SENSORS_IT87 is not set
+# CONFIG_SENSORS_MAX197 is not set
+# CONFIG_SENSORS_PC87360 is not set
+# CONFIG_SENSORS_PC87427 is not set
+# CONFIG_SENSORS_NTC_THERMISTOR is not set
+# CONFIG_SENSORS_NCT6683 is not set
+# CONFIG_SENSORS_NCT6775 is not set
+# CONFIG_SENSORS_PWM_FAN is not set
+# CONFIG_SENSORS_SHT15 is not set
+# CONFIG_SENSORS_SIS5595 is not set
+# CONFIG_SENSORS_SMSC47M1 is not set
+# CONFIG_SENSORS_SMSC47B397 is not set
+# CONFIG_SENSORS_SCH56XX_COMMON is not set
+# CONFIG_SENSORS_SCH5627 is not set
+# CONFIG_SENSORS_SCH5636 is not set
+# CONFIG_SENSORS_VEXPRESS is not set
+# CONFIG_SENSORS_VIA686A is not set
+# CONFIG_SENSORS_VT1211 is not set
+# CONFIG_SENSORS_VT8231 is not set
+# CONFIG_SENSORS_W83627HF is not set
+# CONFIG_SENSORS_W83627EHF is not set
+# CONFIG_THERMAL is not set
+CONFIG_WATCHDOG=y
+CONFIG_WATCHDOG_CORE=y
+# CONFIG_WATCHDOG_NOWAYOUT is not set
+
+#
+# Watchdog Device Drivers
+#
+# CONFIG_SOFT_WATCHDOG is not set
+# CONFIG_GPIO_WATCHDOG is not set
+# CONFIG_XILINX_WATCHDOG is not set
+# CONFIG_ARM_SP805_WATCHDOG is not set
+# CONFIG_CADENCE_WATCHDOG is not set
+# CONFIG_DW_WATCHDOG is not set
+# CONFIG_MAX63XX_WATCHDOG is not set
+# CONFIG_ALIM7101_WDT is not set
+# CONFIG_I6300ESB_WDT is not set
+# CONFIG_BCM7038_WDT is not set
+# CONFIG_MEN_A21_WDT is not set
+
+#
+# PCI-based Watchdog Cards
+#
+# CONFIG_PCIPCWATCHDOG is not set
+# CONFIG_WDTPCI is not set
+
+#
+# USB-based Watchdog Cards
+#
+# CONFIG_USBPCWATCHDOG is not set
+CONFIG_SSB_POSSIBLE=y
+
+#
+# Sonics Silicon Backplane
+#
+# CONFIG_SSB is not set
+CONFIG_BCMA_POSSIBLE=y
+
+#
+# Broadcom specific AMBA
+#
+# CONFIG_BCMA is not set
+
+#
+# Multifunction device drivers
+#
+CONFIG_MFD_CORE=y
+# CONFIG_MFD_ATMEL_FLEXCOM is not set
+# CONFIG_MFD_ATMEL_HLCDC is not set
+# CONFIG_MFD_DLN2 is not set
+# CONFIG_MFD_HI6421_PMIC is not set
+# CONFIG_HTC_PASIC3 is not set
+# CONFIG_LPC_ICH is not set
+# CONFIG_LPC_SCH is not set
+# CONFIG_MFD_JANZ_CMODIO is not set
+# CONFIG_MFD_KEMPLD is not set
+# CONFIG_MFD_MT6397 is not set
+# CONFIG_MFD_VIPERBOARD is not set
+# CONFIG_MFD_RDC321X is not set
+# CONFIG_MFD_RTSX_PCI is not set
+# CONFIG_MFD_RTSX_USB is not set
+# CONFIG_MFD_SM501 is not set
+# CONFIG_ABX500_CORE is not set
+CONFIG_MFD_SYSCON=y
+# CONFIG_MFD_TI_AM335X_TSCADC is not set
+# CONFIG_MFD_TPS65912 is not set
+# CONFIG_MFD_TMIO is not set
+# CONFIG_MFD_VX855 is not set
+CONFIG_MFD_VEXPRESS_SYSREG=y
+# CONFIG_REGULATOR is not set
+# CONFIG_MEDIA_SUPPORT is not set
+
+#
+# Graphics support
+#
+CONFIG_VGA_ARB=y
+CONFIG_VGA_ARB_MAX_GPUS=16
+# CONFIG_DRM is not set
+
+#
+# Frame buffer Devices
+#
+CONFIG_FB=y
+# CONFIG_FIRMWARE_EDID is not set
+CONFIG_FB_CMDLINE=y
+# CONFIG_FB_DDC is not set
+# CONFIG_FB_BOOT_VESA_SUPPORT is not set
+CONFIG_FB_CFB_FILLRECT=y
+CONFIG_FB_CFB_COPYAREA=y
+CONFIG_FB_CFB_IMAGEBLIT=y
+# CONFIG_FB_CFB_REV_PIXELS_IN_BYTE is not set
+# CONFIG_FB_SYS_FILLRECT is not set
+# CONFIG_FB_SYS_COPYAREA is not set
+# CONFIG_FB_SYS_IMAGEBLIT is not set
+# CONFIG_FB_FOREIGN_ENDIAN is not set
+# CONFIG_FB_SYS_FOPS is not set
+# CONFIG_FB_SVGALIB is not set
+# CONFIG_FB_MACMODES is not set
+# CONFIG_FB_BACKLIGHT is not set
+CONFIG_FB_MODE_HELPERS=y
+CONFIG_FB_TILEBLITTING=y
+
+#
+# Frame buffer hardware drivers
+#
+# CONFIG_FB_CIRRUS is not set
+# CONFIG_FB_PM2 is not set
+CONFIG_FB_ARMCLCD=y
+# CONFIG_FB_CYBER2000 is not set
+# CONFIG_FB_ASILIANT is not set
+# CONFIG_FB_IMSTT is not set
+# CONFIG_FB_UVESA is not set
+# CONFIG_FB_OPENCORES is not set
+# CONFIG_FB_S1D13XXX is not set
+# CONFIG_FB_NVIDIA is not set
+# CONFIG_FB_RIVA is not set
+# CONFIG_FB_I740 is not set
+# CONFIG_FB_MATROX is not set
+# CONFIG_FB_RADEON is not set
+# CONFIG_FB_ATY128 is not set
+# CONFIG_FB_ATY is not set
+# CONFIG_FB_S3 is not set
+# CONFIG_FB_SAVAGE is not set
+# CONFIG_FB_SIS is not set
+# CONFIG_FB_NEOMAGIC is not set
+# CONFIG_FB_KYRO is not set
+# CONFIG_FB_3DFX is not set
+# CONFIG_FB_VOODOO1 is not set
+# CONFIG_FB_VT8623 is not set
+# CONFIG_FB_TRIDENT is not set
+# CONFIG_FB_ARK is not set
+# CONFIG_FB_PM3 is not set
+# CONFIG_FB_CARMINE is not set
+# CONFIG_FB_SMSCUFX is not set
+# CONFIG_FB_UDL is not set
+# CONFIG_FB_IBM_GXT4500 is not set
+# CONFIG_FB_VIRTUAL is not set
+# CONFIG_FB_METRONOME is not set
+# CONFIG_FB_MB862XX is not set
+# CONFIG_FB_BROADSHEET is not set
+# CONFIG_FB_AUO_K190X is not set
+CONFIG_FB_SIMPLE=y
+# CONFIG_FB_SM712 is not set
+CONFIG_BACKLIGHT_LCD_SUPPORT=y
+# CONFIG_LCD_CLASS_DEVICE is not set
+CONFIG_BACKLIGHT_CLASS_DEVICE=y
+# CONFIG_BACKLIGHT_GENERIC is not set
+# CONFIG_BACKLIGHT_PWM is not set
+# CONFIG_BACKLIGHT_PM8941_WLED is not set
+# CONFIG_BACKLIGHT_GPIO is not set
+# CONFIG_VGASTATE is not set
+CONFIG_VIDEOMODE_HELPERS=y
+
+#
+# Console display driver support
+#
+CONFIG_DUMMY_CONSOLE=y
+CONFIG_DUMMY_CONSOLE_COLUMNS=80
+CONFIG_DUMMY_CONSOLE_ROWS=25
+CONFIG_FRAMEBUFFER_CONSOLE=y
+CONFIG_FRAMEBUFFER_CONSOLE_DETECT_PRIMARY=y
+CONFIG_FRAMEBUFFER_CONSOLE_ROTATION=y
+CONFIG_LOGO=y
+# CONFIG_LOGO_LINUX_MONO is not set
+# CONFIG_LOGO_LINUX_VGA16 is not set
+CONFIG_LOGO_LINUX_CLUT224=y
+# CONFIG_SOUND is not set
+
+#
+# HID support
+#
+CONFIG_HID=y
+CONFIG_HID_BATTERY_STRENGTH=y
+CONFIG_HIDRAW=y
+# CONFIG_UHID is not set
+CONFIG_HID_GENERIC=y
+
+#
+# Special HID drivers
+#
+CONFIG_HID_A4TECH=y
+# CONFIG_HID_ACRUX is not set
+CONFIG_HID_APPLE=y
+# CONFIG_HID_APPLEIR is not set
+# CONFIG_HID_AUREAL is not set
+CONFIG_HID_BELKIN=y
+# CONFIG_HID_BETOP_FF is not set
+CONFIG_HID_CHERRY=y
+CONFIG_HID_CHICONY=y
+# CONFIG_HID_CORSAIR is not set
+CONFIG_HID_CYPRESS=y
+# CONFIG_HID_DRAGONRISE is not set
+# CONFIG_HID_EMS_FF is not set
+# CONFIG_HID_ELECOM is not set
+# CONFIG_HID_ELO is not set
+CONFIG_HID_EZKEY=y
+# CONFIG_HID_GEMBIRD is not set
+# CONFIG_HID_GFRM is not set
+# CONFIG_HID_HOLTEK is not set
+# CONFIG_HID_GT683R is not set
+# CONFIG_HID_KEYTOUCH is not set
+# CONFIG_HID_KYE is not set
+# CONFIG_HID_UCLOGIC is not set
+# CONFIG_HID_WALTOP is not set
+# CONFIG_HID_GYRATION is not set
+# CONFIG_HID_ICADE is not set
+# CONFIG_HID_TWINHAN is not set
+CONFIG_HID_KENSINGTON=y
+# CONFIG_HID_LCPOWER is not set
+# CONFIG_HID_LENOVO is not set
+CONFIG_HID_LOGITECH=y
+# CONFIG_HID_LOGITECH_DJ is not set
+# CONFIG_HID_LOGITECH_HIDPP is not set
+CONFIG_LOGITECH_FF=y
+CONFIG_LOGIRUMBLEPAD2_FF=y
+CONFIG_LOGIG940_FF=y
+CONFIG_LOGIWHEELS_FF=y
+CONFIG_HID_MAGICMOUSE=y
+CONFIG_HID_MICROSOFT=y
+CONFIG_HID_MONTEREY=y
+# CONFIG_HID_MULTITOUCH is not set
+CONFIG_HID_NTRIG=y
+# CONFIG_HID_ORTEK is not set
+# CONFIG_HID_PANTHERLORD is not set
+# CONFIG_HID_PENMOUNT is not set
+# CONFIG_HID_PETALYNX is not set
+# CONFIG_HID_PICOLCD is not set
+# CONFIG_HID_PLANTRONICS is not set
+# CONFIG_HID_PRIMAX is not set
+# CONFIG_HID_ROCCAT is not set
+# CONFIG_HID_SAITEK is not set
+# CONFIG_HID_SAMSUNG is not set
+# CONFIG_HID_SONY is not set
+# CONFIG_HID_SPEEDLINK is not set
+# CONFIG_HID_STEELSERIES is not set
+# CONFIG_HID_SUNPLUS is not set
+# CONFIG_HID_RMI is not set
+# CONFIG_HID_GREENASIA is not set
+# CONFIG_HID_SMARTJOYPLUS is not set
+# CONFIG_HID_TIVO is not set
+# CONFIG_HID_TOPSEED is not set
+# CONFIG_HID_THINGM is not set
+# CONFIG_HID_THRUSTMASTER is not set
+# CONFIG_HID_WACOM is not set
+# CONFIG_HID_WIIMOTE is not set
+# CONFIG_HID_XINMO is not set
+# CONFIG_HID_ZEROPLUS is not set
+# CONFIG_HID_ZYDACRON is not set
+# CONFIG_HID_SENSOR_HUB is not set
+
+#
+# USB HID support
+#
+CONFIG_USB_HID=y
+CONFIG_HID_PID=y
+CONFIG_USB_HIDDEV=y
+CONFIG_USB_OHCI_LITTLE_ENDIAN=y
+CONFIG_USB_SUPPORT=y
+CONFIG_USB_COMMON=y
+CONFIG_USB_ARCH_HAS_HCD=y
+CONFIG_USB=y
+CONFIG_USB_ANNOUNCE_NEW_DEVICES=y
+
+#
+# Miscellaneous USB options
+#
+CONFIG_USB_DEFAULT_PERSIST=y
+# CONFIG_USB_DYNAMIC_MINORS is not set
+# CONFIG_USB_OTG_WHITELIST is not set
+# CONFIG_USB_ULPI_BUS is not set
+CONFIG_USB_MON=y
+# CONFIG_USB_WUSB_CBAF is not set
+
+#
+# USB Host Controller Drivers
+#
+# CONFIG_USB_C67X00_HCD is not set
+CONFIG_USB_XHCI_HCD=y
+CONFIG_USB_XHCI_PCI=y
+# CONFIG_USB_XHCI_PLATFORM is not set
+CONFIG_USB_EHCI_HCD=y
+CONFIG_USB_EHCI_ROOT_HUB_TT=y
+CONFIG_USB_EHCI_TT_NEWSCHED=y
+CONFIG_USB_EHCI_PCI=y
+# CONFIG_USB_EHCI_HCD_PLATFORM is not set
+# CONFIG_USB_OXU210HP_HCD is not set
+# CONFIG_USB_ISP116X_HCD is not set
+# CONFIG_USB_ISP1362_HCD is not set
+# CONFIG_USB_FOTG210_HCD is not set
+CONFIG_USB_OHCI_HCD=y
+CONFIG_USB_OHCI_HCD_PCI=y
+# CONFIG_USB_OHCI_HCD_PLATFORM is not set
+CONFIG_USB_UHCI_HCD=y
+# CONFIG_USB_SL811_HCD is not set
+# CONFIG_USB_R8A66597_HCD is not set
+# CONFIG_USB_HCD_TEST_MODE is not set
+
+#
+# USB Device Class drivers
+#
+# CONFIG_USB_ACM is not set
+# CONFIG_USB_PRINTER is not set
+# CONFIG_USB_WDM is not set
+# CONFIG_USB_TMC is not set
+
+#
+# NOTE: USB_STORAGE depends on SCSI but BLK_DEV_SD may
+#
+
+#
+# also be needed; see USB_STORAGE Help for more info
+#
+# CONFIG_USB_STORAGE is not set
+
+#
+# USB Imaging devices
+#
+# CONFIG_USB_MDC800 is not set
+# CONFIG_USB_MICROTEK is not set
+# CONFIG_USBIP_CORE is not set
+# CONFIG_USB_MUSB_HDRC is not set
+# CONFIG_USB_DWC3 is not set
+# CONFIG_USB_DWC2 is not set
+# CONFIG_USB_CHIPIDEA is not set
+# CONFIG_USB_ISP1760 is not set
+
+#
+# USB port drivers
+#
+CONFIG_USB_SERIAL=y
+CONFIG_USB_SERIAL_CONSOLE=y
+CONFIG_USB_SERIAL_GENERIC=y
+# CONFIG_USB_SERIAL_SIMPLE is not set
+# CONFIG_USB_SERIAL_AIRCABLE is not set
+# CONFIG_USB_SERIAL_ARK3116 is not set
+# CONFIG_USB_SERIAL_BELKIN is not set
+# CONFIG_USB_SERIAL_CH341 is not set
+# CONFIG_USB_SERIAL_WHITEHEAT is not set
+# CONFIG_USB_SERIAL_DIGI_ACCELEPORT is not set
+# CONFIG_USB_SERIAL_CP210X is not set
+# CONFIG_USB_SERIAL_CYPRESS_M8 is not set
+# CONFIG_USB_SERIAL_EMPEG is not set
+# CONFIG_USB_SERIAL_FTDI_SIO is not set
+# CONFIG_USB_SERIAL_VISOR is not set
+# CONFIG_USB_SERIAL_IPAQ is not set
+# CONFIG_USB_SERIAL_IR is not set
+# CONFIG_USB_SERIAL_EDGEPORT is not set
+# CONFIG_USB_SERIAL_EDGEPORT_TI is not set
+# CONFIG_USB_SERIAL_F81232 is not set
+# CONFIG_USB_SERIAL_GARMIN is not set
+# CONFIG_USB_SERIAL_IPW is not set
+# CONFIG_USB_SERIAL_IUU is not set
+# CONFIG_USB_SERIAL_KEYSPAN_PDA is not set
+# CONFIG_USB_SERIAL_KEYSPAN is not set
+# CONFIG_USB_SERIAL_KLSI is not set
+# CONFIG_USB_SERIAL_KOBIL_SCT is not set
+# CONFIG_USB_SERIAL_MCT_U232 is not set
+# CONFIG_USB_SERIAL_METRO is not set
+# CONFIG_USB_SERIAL_MOS7720 is not set
+# CONFIG_USB_SERIAL_MOS7840 is not set
+# CONFIG_USB_SERIAL_MXUPORT is not set
+# CONFIG_USB_SERIAL_NAVMAN is not set
+# CONFIG_USB_SERIAL_PL2303 is not set
+# CONFIG_USB_SERIAL_OTI6858 is not set
+# CONFIG_USB_SERIAL_QCAUX is not set
+# CONFIG_USB_SERIAL_QUALCOMM is not set
+# CONFIG_USB_SERIAL_SPCP8X5 is not set
+# CONFIG_USB_SERIAL_SAFE is not set
+# CONFIG_USB_SERIAL_SIERRAWIRELESS is not set
+# CONFIG_USB_SERIAL_SYMBOL is not set
+# CONFIG_USB_SERIAL_TI is not set
+# CONFIG_USB_SERIAL_CYBERJACK is not set
+# CONFIG_USB_SERIAL_XIRCOM is not set
+# CONFIG_USB_SERIAL_OPTION is not set
+# CONFIG_USB_SERIAL_OMNINET is not set
+# CONFIG_USB_SERIAL_OPTICON is not set
+# CONFIG_USB_SERIAL_XSENS_MT is not set
+# CONFIG_USB_SERIAL_WISHBONE is not set
+# CONFIG_USB_SERIAL_SSU100 is not set
+# CONFIG_USB_SERIAL_QT2 is not set
+# CONFIG_USB_SERIAL_DEBUG is not set
+
+#
+# USB Miscellaneous drivers
+#
+# CONFIG_USB_EMI62 is not set
+# CONFIG_USB_EMI26 is not set
+# CONFIG_USB_ADUTUX is not set
+# CONFIG_USB_SEVSEG is not set
+# CONFIG_USB_RIO500 is not set
+# CONFIG_USB_LEGOTOWER is not set
+# CONFIG_USB_LCD is not set
+# CONFIG_USB_LED is not set
+# CONFIG_USB_CYPRESS_CY7C63 is not set
+# CONFIG_USB_CYTHERM is not set
+# CONFIG_USB_IDMOUSE is not set
+# CONFIG_USB_FTDI_ELAN is not set
+# CONFIG_USB_APPLEDISPLAY is not set
+# CONFIG_USB_SISUSBVGA is not set
+# CONFIG_USB_LD is not set
+# CONFIG_USB_TRANCEVIBRATOR is not set
+# CONFIG_USB_IOWARRIOR is not set
+# CONFIG_USB_TEST is not set
+# CONFIG_USB_EHSET_TEST_FIXTURE is not set
+# CONFIG_USB_ISIGHTFW is not set
+# CONFIG_USB_YUREX is not set
+# CONFIG_USB_EZUSB_FX2 is not set
+# CONFIG_USB_LINK_LAYER_TEST is not set
+# CONFIG_USB_CHAOSKEY is not set
+
+#
+# USB Physical Layer drivers
+#
+# CONFIG_USB_PHY is not set
+# CONFIG_NOP_USB_XCEIV is not set
+# CONFIG_USB_GPIO_VBUS is not set
+# CONFIG_USB_ULPI is not set
+# CONFIG_USB_GADGET is not set
+# CONFIG_USB_LED_TRIG is not set
+# CONFIG_UWB is not set
+# CONFIG_MMC is not set
+# CONFIG_MEMSTICK is not set
+CONFIG_NEW_LEDS=y
+CONFIG_LEDS_CLASS=y
+# CONFIG_LEDS_CLASS_FLASH is not set
+
+#
+# LED drivers
+#
+# CONFIG_LEDS_BCM6328 is not set
+# CONFIG_LEDS_BCM6358 is not set
+# CONFIG_LEDS_GPIO is not set
+# CONFIG_LEDS_PWM is not set
+# CONFIG_LEDS_INTEL_SS4200 is not set
+# CONFIG_LEDS_LT3593 is not set
+
+#
+# LED driver for blink(1) USB RGB LED is under Special HID drivers (HID_THINGM)
+#
+# CONFIG_LEDS_SYSCON is not set
+
+#
+# LED Triggers
+#
+CONFIG_LEDS_TRIGGERS=y
+# CONFIG_LEDS_TRIGGER_TIMER is not set
+# CONFIG_LEDS_TRIGGER_ONESHOT is not set
+# CONFIG_LEDS_TRIGGER_HEARTBEAT is not set
+# CONFIG_LEDS_TRIGGER_BACKLIGHT is not set
+# CONFIG_LEDS_TRIGGER_CPU is not set
+# CONFIG_LEDS_TRIGGER_GPIO is not set
+# CONFIG_LEDS_TRIGGER_DEFAULT_ON is not set
+
+#
+# iptables trigger is under Netfilter config (LED target)
+#
+# CONFIG_LEDS_TRIGGER_TRANSIENT is not set
+# CONFIG_LEDS_TRIGGER_CAMERA is not set
+CONFIG_ACCESSIBILITY=y
+CONFIG_A11Y_BRAILLE_CONSOLE=y
+# CONFIG_INFINIBAND is not set
+CONFIG_EDAC_SUPPORT=y
+# CONFIG_EDAC is not set
+CONFIG_RTC_LIB=y
+CONFIG_RTC_CLASS=y
+CONFIG_RTC_HCTOSYS=y
+CONFIG_RTC_HCTOSYS_DEVICE="rtc0"
+# CONFIG_RTC_SYSTOHC is not set
+# CONFIG_RTC_DEBUG is not set
+
+#
+# RTC interfaces
+#
+CONFIG_RTC_INTF_SYSFS=y
+CONFIG_RTC_INTF_PROC=y
+CONFIG_RTC_INTF_DEV=y
+# CONFIG_RTC_INTF_DEV_UIE_EMUL is not set
+# CONFIG_RTC_DRV_TEST is not set
+
+#
+# SPI RTC drivers
+#
+
+#
+# Platform RTC drivers
+#
+# CONFIG_RTC_DRV_DS1286 is not set
+# CONFIG_RTC_DRV_DS1511 is not set
+# CONFIG_RTC_DRV_DS1553 is not set
+# CONFIG_RTC_DRV_DS1685_FAMILY is not set
+# CONFIG_RTC_DRV_DS1742 is not set
+# CONFIG_RTC_DRV_DS2404 is not set
+# CONFIG_RTC_DRV_EFI is not set
+# CONFIG_RTC_DRV_STK17TA8 is not set
+# CONFIG_RTC_DRV_M48T86 is not set
+# CONFIG_RTC_DRV_M48T35 is not set
+# CONFIG_RTC_DRV_M48T59 is not set
+# CONFIG_RTC_DRV_MSM6242 is not set
+# CONFIG_RTC_DRV_BQ4802 is not set
+# CONFIG_RTC_DRV_RP5C01 is not set
+# CONFIG_RTC_DRV_V3020 is not set
+# CONFIG_RTC_DRV_ZYNQMP is not set
+
+#
+# on-CPU RTC drivers
+#
+# CONFIG_RTC_DRV_PL030 is not set
+# CONFIG_RTC_DRV_PL031 is not set
+# CONFIG_RTC_DRV_SNVS is not set
+
+#
+# HID Sensor RTC drivers
+#
+# CONFIG_RTC_DRV_HID_SENSOR_TIME is not set
+CONFIG_DMADEVICES=y
+# CONFIG_DMADEVICES_DEBUG is not set
+
+#
+# DMA Devices
+#
+CONFIG_DMA_ENGINE=y
+CONFIG_DMA_OF=y
+# CONFIG_AMBA_PL08X is not set
+# CONFIG_FSL_EDMA is not set
+# CONFIG_INTEL_IDMA64 is not set
+# CONFIG_PL330_DMA is not set
+# CONFIG_DW_DMAC is not set
+# CONFIG_DW_DMAC_PCI is not set
+
+#
+# DMA Clients
+#
+CONFIG_ASYNC_TX_DMA=y
+# CONFIG_DMATEST is not set
+CONFIG_AUXDISPLAY=y
+# CONFIG_UIO is not set
+CONFIG_VFIO_IOMMU_TYPE1=y
+CONFIG_VFIO_VIRQFD=y
+CONFIG_VFIO=y
+CONFIG_VFIO_PCI=y
+CONFIG_VFIO_PCI_MMAP=y
+CONFIG_VFIO_PCI_INTX=y
+# CONFIG_VFIO_PLATFORM is not set
+CONFIG_IRQ_BYPASS_MANAGER=y
+# CONFIG_VIRT_DRIVERS is not set
+
+#
+# Virtio drivers
+#
+# CONFIG_VIRTIO_PCI is not set
+# CONFIG_VIRTIO_MMIO is not set
+
+#
+# Microsoft Hyper-V guest support
+#
+CONFIG_STAGING=y
+# CONFIG_COMEDI is not set
+# CONFIG_RTL8192U is not set
+# CONFIG_RTLLIB is not set
+# CONFIG_R8712U is not set
+# CONFIG_RTS5208 is not set
+# CONFIG_FB_SM750 is not set
+# CONFIG_FB_XGI is not set
+
+#
+# Speakup console speech
+#
+# CONFIG_SPEAKUP is not set
+CONFIG_STAGING_MEDIA=y
+
+#
+# Android
+#
+# CONFIG_STAGING_BOARD is not set
+# CONFIG_WIMAX_GDM72XX is not set
+# CONFIG_LTE_GDM724X is not set
+# CONFIG_LUSTRE_FS is not set
+# CONFIG_DGNC is not set
+# CONFIG_DGAP is not set
+# CONFIG_GS_FPGABOOT is not set
+# CONFIG_COMMON_CLK_XLNX_CLKWZRD is not set
+# CONFIG_FSL_MC_BUS is not set
+# CONFIG_MOST is not set
+# CONFIG_CHROME_PLATFORMS is not set
+CONFIG_CLKDEV_LOOKUP=y
+CONFIG_HAVE_CLK_PREPARE=y
+CONFIG_COMMON_CLK=y
+
+#
+# Common Clock Framework
+#
+CONFIG_COMMON_CLK_VERSATILE=y
+CONFIG_CLK_SP810=y
+CONFIG_CLK_VEXPRESS_OSC=y
+# CONFIG_CLK_QORIQ is not set
+CONFIG_COMMON_CLK_XGENE=y
+# CONFIG_COMMON_CLK_PWM is not set
+# CONFIG_COMMON_CLK_PXA is not set
+
+#
+# Hardware Spinlock drivers
+#
+
+#
+# Clock Source drivers
+#
+CONFIG_CLKSRC_OF=y
+CONFIG_CLKSRC_PROBE=y
+CONFIG_CLKSRC_MMIO=y
+CONFIG_ARM_ARCH_TIMER=y
+CONFIG_ARM_ARCH_TIMER_EVTSTREAM=y
+# CONFIG_ARM_TIMER_SP804 is not set
+# CONFIG_ATMEL_PIT is not set
+# CONFIG_SH_TIMER_CMT is not set
+# CONFIG_SH_TIMER_MTU2 is not set
+# CONFIG_SH_TIMER_TMU is not set
+# CONFIG_EM_TIMER_STI is not set
+# CONFIG_MAILBOX is not set
+CONFIG_IOMMU_API=y
+CONFIG_IOMMU_SUPPORT=y
+
+#
+# Generic IOMMU Pagetable Support
+#
+CONFIG_IOMMU_IO_PGTABLE=y
+CONFIG_IOMMU_IO_PGTABLE_LPAE=y
+# CONFIG_IOMMU_IO_PGTABLE_LPAE_SELFTEST is not set
+CONFIG_IOMMU_IOVA=y
+CONFIG_OF_IOMMU=y
+CONFIG_IOMMU_DMA=y
+CONFIG_ARM_SMMU=y
+# CONFIG_ARM_SMMU_V3 is not set
+
+#
+# Remoteproc drivers
+#
+# CONFIG_STE_MODEM_RPROC is not set
+
+#
+# Rpmsg drivers
+#
+
+#
+# SOC (System On Chip) specific Drivers
+#
+# CONFIG_SUNXI_SRAM is not set
+# CONFIG_SOC_TI is not set
+# CONFIG_PM_DEVFREQ is not set
+# CONFIG_EXTCON is not set
+# CONFIG_MEMORY is not set
+# CONFIG_IIO is not set
+# CONFIG_NTB is not set
+# CONFIG_VME_BUS is not set
+CONFIG_PWM=y
+CONFIG_PWM_SYSFS=y
+# CONFIG_PWM_FSL_FTM is not set
+CONFIG_IRQCHIP=y
+CONFIG_ARM_GIC=y
+CONFIG_ARM_GIC_V2M=y
+CONFIG_ARM_GIC_V3=y
+CONFIG_ARM_GIC_V3_ITS=y
+# CONFIG_IPACK_BUS is not set
+CONFIG_RESET_CONTROLLER=y
+# CONFIG_FMC is not set
+
+#
+# PHY Subsystem
+#
+CONFIG_GENERIC_PHY=y
+# CONFIG_PHY_PXA_28NM_HSIC is not set
+# CONFIG_PHY_PXA_28NM_USB2 is not set
+# CONFIG_BCM_KONA_USB2_PHY is not set
+CONFIG_PHY_XGENE=y
+# CONFIG_POWERCAP is not set
+# CONFIG_MCB is not set
+
+#
+# Performance monitor support
+#
+CONFIG_ARM_PMU=y
+CONFIG_RAS=y
+# CONFIG_THUNDERBOLT is not set
+
+#
+# Android
+#
+# CONFIG_ANDROID is not set
+# CONFIG_LIBNVDIMM is not set
+# CONFIG_NVMEM is not set
+# CONFIG_STM is not set
+# CONFIG_STM_DUMMY is not set
+# CONFIG_STM_SOURCE_CONSOLE is not set
+# CONFIG_INTEL_TH is not set
+
+#
+# FPGA Configuration Support
+#
+# CONFIG_FPGA is not set
+
+#
+# Firmware Drivers
+#
+CONFIG_ARM_PSCI_FW=y
+# CONFIG_FIRMWARE_MEMMAP is not set
+CONFIG_DMIID=y
+# CONFIG_DMI_SYSFS is not set
+
+#
+# EFI (Extensible Firmware Interface) Support
+#
+# CONFIG_EFI_VARS is not set
+CONFIG_EFI_ESRT=y
+CONFIG_EFI_PARAMS_FROM_FDT=y
+CONFIG_EFI_RUNTIME_WRAPPERS=y
+CONFIG_EFI_ARMSTUB=y
+
+#
+# File systems
+#
+CONFIG_DCACHE_WORD_ACCESS=y
+# CONFIG_EXT2_FS is not set
+# CONFIG_EXT3_FS is not set
+CONFIG_EXT4_FS=y
+CONFIG_EXT4_USE_FOR_EXT2=y
+CONFIG_EXT4_FS_POSIX_ACL=y
+CONFIG_EXT4_FS_SECURITY=y
+# CONFIG_EXT4_ENCRYPTION is not set
+# CONFIG_EXT4_DEBUG is not set
+CONFIG_JBD2=y
+# CONFIG_JBD2_DEBUG is not set
+CONFIG_FS_MBCACHE=y
+# CONFIG_REISERFS_FS is not set
+# CONFIG_JFS_FS is not set
+# CONFIG_XFS_FS is not set
+# CONFIG_GFS2_FS is not set
+# CONFIG_OCFS2_FS is not set
+# CONFIG_BTRFS_FS is not set
+# CONFIG_NILFS2_FS is not set
+# CONFIG_F2FS_FS is not set
+# CONFIG_FS_DAX is not set
+CONFIG_FS_POSIX_ACL=y
+CONFIG_EXPORTFS=y
+CONFIG_FILE_LOCKING=y
+CONFIG_FSNOTIFY=y
+CONFIG_DNOTIFY=y
+CONFIG_INOTIFY_USER=y
+CONFIG_FANOTIFY=y
+CONFIG_FANOTIFY_ACCESS_PERMISSIONS=y
+CONFIG_QUOTA=y
+CONFIG_QUOTA_NETLINK_INTERFACE=y
+# CONFIG_PRINT_QUOTA_WARNING is not set
+# CONFIG_QUOTA_DEBUG is not set
+CONFIG_QUOTA_TREE=y
+# CONFIG_QFMT_V1 is not set
+CONFIG_QFMT_V2=y
+CONFIG_QUOTACTL=y
+CONFIG_AUTOFS4_FS=y
+# CONFIG_FUSE_FS is not set
+# CONFIG_OVERLAY_FS is not set
+
+#
+# Caches
+#
+# CONFIG_FSCACHE is not set
+
+#
+# CD-ROM/DVD Filesystems
+#
+# CONFIG_ISO9660_FS is not set
+# CONFIG_UDF_FS is not set
+
+#
+# DOS/FAT/NT Filesystems
+#
+CONFIG_FAT_FS=y
+# CONFIG_MSDOS_FS is not set
+CONFIG_VFAT_FS=y
+CONFIG_FAT_DEFAULT_CODEPAGE=437
+CONFIG_FAT_DEFAULT_IOCHARSET="ascii"
+# CONFIG_NTFS_FS is not set
+
+#
+# Pseudo filesystems
+#
+CONFIG_PROC_FS=y
+CONFIG_PROC_KCORE=y
+CONFIG_PROC_SYSCTL=y
+CONFIG_PROC_PAGE_MONITOR=y
+# CONFIG_PROC_CHILDREN is not set
+CONFIG_KERNFS=y
+CONFIG_SYSFS=y
+CONFIG_TMPFS=y
+CONFIG_TMPFS_POSIX_ACL=y
+CONFIG_TMPFS_XATTR=y
+CONFIG_HUGETLBFS=y
+CONFIG_HUGETLB_PAGE=y
+CONFIG_CONFIGFS_FS=y
+CONFIG_EFIVAR_FS=y
+CONFIG_MISC_FILESYSTEMS=y
+# CONFIG_ADFS_FS is not set
+# CONFIG_AFFS_FS is not set
+# CONFIG_ECRYPT_FS is not set
+# CONFIG_HFS_FS is not set
+# CONFIG_HFSPLUS_FS is not set
+# CONFIG_BEFS_FS is not set
+# CONFIG_BFS_FS is not set
+# CONFIG_EFS_FS is not set
+# CONFIG_LOGFS is not set
+# CONFIG_CRAMFS is not set
+# CONFIG_SQUASHFS is not set
+# CONFIG_VXFS_FS is not set
+# CONFIG_MINIX_FS is not set
+# CONFIG_OMFS_FS is not set
+# CONFIG_HPFS_FS is not set
+# CONFIG_QNX4FS_FS is not set
+# CONFIG_QNX6FS_FS is not set
+# CONFIG_ROMFS_FS is not set
+CONFIG_PSTORE=y
+# CONFIG_PSTORE_CONSOLE is not set
+# CONFIG_PSTORE_PMSG is not set
+# CONFIG_PSTORE_FTRACE is not set
+# CONFIG_PSTORE_RAM is not set
+# CONFIG_SYSV_FS is not set
+# CONFIG_UFS_FS is not set
+CONFIG_NETWORK_FILESYSTEMS=y
+CONFIG_NFS_FS=y
+# CONFIG_NFS_V2 is not set
+CONFIG_NFS_V3=y
+CONFIG_NFS_V3_ACL=y
+CONFIG_NFS_V4=y
+CONFIG_NFS_SWAP=y
+CONFIG_NFS_V4_1=y
+CONFIG_NFS_V4_2=y
+CONFIG_PNFS_FILE_LAYOUT=y
+CONFIG_PNFS_BLOCK=y
+CONFIG_PNFS_FLEXFILE_LAYOUT=m
+CONFIG_NFS_V4_1_IMPLEMENTATION_ID_DOMAIN="kernel.org"
+# CONFIG_NFS_V4_1_MIGRATION is not set
+CONFIG_NFS_V4_SECURITY_LABEL=y
+# CONFIG_NFS_USE_LEGACY_DNS is not set
+CONFIG_NFS_USE_KERNEL_DNS=y
+CONFIG_NFS_DEBUG=y
+CONFIG_NFSD=y
+CONFIG_NFSD_V2_ACL=y
+CONFIG_NFSD_V3=y
+CONFIG_NFSD_V3_ACL=y
+CONFIG_NFSD_V4=y
+# CONFIG_NFSD_PNFS is not set
+CONFIG_NFSD_V4_SECURITY_LABEL=y
+# CONFIG_NFSD_FAULT_INJECTION is not set
+CONFIG_GRACE_PERIOD=y
+CONFIG_LOCKD=y
+CONFIG_LOCKD_V4=y
+CONFIG_NFS_ACL_SUPPORT=y
+CONFIG_NFS_COMMON=y
+CONFIG_SUNRPC=y
+CONFIG_SUNRPC_GSS=y
+CONFIG_SUNRPC_BACKCHANNEL=y
+CONFIG_SUNRPC_SWAP=y
+CONFIG_RPCSEC_GSS_KRB5=y
+CONFIG_SUNRPC_DEBUG=y
+# CONFIG_CEPH_FS is not set
+# CONFIG_CIFS is not set
+# CONFIG_NCP_FS is not set
+# CONFIG_CODA_FS is not set
+# CONFIG_AFS_FS is not set
+CONFIG_NLS=y
+CONFIG_NLS_DEFAULT="utf8"
+CONFIG_NLS_CODEPAGE_437=y
+# CONFIG_NLS_CODEPAGE_737 is not set
+# CONFIG_NLS_CODEPAGE_775 is not set
+# CONFIG_NLS_CODEPAGE_850 is not set
+# CONFIG_NLS_CODEPAGE_852 is not set
+# CONFIG_NLS_CODEPAGE_855 is not set
+# CONFIG_NLS_CODEPAGE_857 is not set
+# CONFIG_NLS_CODEPAGE_860 is not set
+# CONFIG_NLS_CODEPAGE_861 is not set
+# CONFIG_NLS_CODEPAGE_862 is not set
+# CONFIG_NLS_CODEPAGE_863 is not set
+# CONFIG_NLS_CODEPAGE_864 is not set
+# CONFIG_NLS_CODEPAGE_865 is not set
+# CONFIG_NLS_CODEPAGE_866 is not set
+# CONFIG_NLS_CODEPAGE_869 is not set
+# CONFIG_NLS_CODEPAGE_936 is not set
+# CONFIG_NLS_CODEPAGE_950 is not set
+# CONFIG_NLS_CODEPAGE_932 is not set
+# CONFIG_NLS_CODEPAGE_949 is not set
+# CONFIG_NLS_CODEPAGE_874 is not set
+# CONFIG_NLS_ISO8859_8 is not set
+# CONFIG_NLS_CODEPAGE_1250 is not set
+# CONFIG_NLS_CODEPAGE_1251 is not set
+CONFIG_NLS_ASCII=y
+# CONFIG_NLS_ISO8859_1 is not set
+# CONFIG_NLS_ISO8859_2 is not set
+# CONFIG_NLS_ISO8859_3 is not set
+# CONFIG_NLS_ISO8859_4 is not set
+# CONFIG_NLS_ISO8859_5 is not set
+# CONFIG_NLS_ISO8859_6 is not set
+# CONFIG_NLS_ISO8859_7 is not set
+# CONFIG_NLS_ISO8859_9 is not set
+# CONFIG_NLS_ISO8859_13 is not set
+# CONFIG_NLS_ISO8859_14 is not set
+# CONFIG_NLS_ISO8859_15 is not set
+# CONFIG_NLS_KOI8_R is not set
+# CONFIG_NLS_KOI8_U is not set
+# CONFIG_NLS_MAC_ROMAN is not set
+# CONFIG_NLS_MAC_CELTIC is not set
+# CONFIG_NLS_MAC_CENTEURO is not set
+# CONFIG_NLS_MAC_CROATIAN is not set
+# CONFIG_NLS_MAC_CYRILLIC is not set
+# CONFIG_NLS_MAC_GAELIC is not set
+# CONFIG_NLS_MAC_GREEK is not set
+# CONFIG_NLS_MAC_ICELAND is not set
+# CONFIG_NLS_MAC_INUIT is not set
+# CONFIG_NLS_MAC_ROMANIAN is not set
+# CONFIG_NLS_MAC_TURKISH is not set
+# CONFIG_NLS_UTF8 is not set
+# CONFIG_DLM is not set
+# CONFIG_VIRTUALIZATION is not set
+
+#
+# Kernel hacking
+#
+
+#
+# printk and dmesg options
+#
+CONFIG_PRINTK_TIME=y
+CONFIG_MESSAGE_LOGLEVEL_DEFAULT=4
+CONFIG_BOOT_PRINTK_DELAY=y
+CONFIG_DYNAMIC_DEBUG=y
+
+#
+# Compile-time checks and compiler options
+#
+# CONFIG_DEBUG_INFO is not set
+# CONFIG_ENABLE_WARN_DEPRECATED is not set
+CONFIG_ENABLE_MUST_CHECK=y
+CONFIG_FRAME_WARN=1024
+CONFIG_STRIP_ASM_SYMS=y
+# CONFIG_READABLE_ASM is not set
+CONFIG_UNUSED_SYMBOLS=y
+# CONFIG_PAGE_OWNER is not set
+CONFIG_DEBUG_FS=y
+# CONFIG_HEADERS_CHECK is not set
+# CONFIG_DEBUG_SECTION_MISMATCH is not set
+CONFIG_SECTION_MISMATCH_WARN_ONLY=y
+CONFIG_ARCH_WANT_FRAME_POINTERS=y
+CONFIG_FRAME_POINTER=y
+# CONFIG_DEBUG_FORCE_WEAK_PER_CPU is not set
+CONFIG_MAGIC_SYSRQ=y
+CONFIG_MAGIC_SYSRQ_DEFAULT_ENABLE=0x0
+CONFIG_DEBUG_KERNEL=y
+
+#
+# Memory Debugging
+#
+# CONFIG_PAGE_EXTENSION is not set
+# CONFIG_DEBUG_PAGEALLOC is not set
+# CONFIG_DEBUG_OBJECTS is not set
+# CONFIG_SLUB_DEBUG_ON is not set
+# CONFIG_SLUB_STATS is not set
+CONFIG_HAVE_DEBUG_KMEMLEAK=y
+# CONFIG_DEBUG_KMEMLEAK is not set
+# CONFIG_DEBUG_STACK_USAGE is not set
+# CONFIG_DEBUG_VM is not set
+CONFIG_DEBUG_MEMORY_INIT=y
+# CONFIG_DEBUG_PER_CPU_MAPS is not set
+CONFIG_HAVE_ARCH_KASAN=y
+# CONFIG_KASAN is not set
+CONFIG_DEBUG_SHIRQ=y
+
+#
+# Debug Lockups and Hangs
+#
+CONFIG_LOCKUP_DETECTOR=y
+# CONFIG_BOOTPARAM_SOFTLOCKUP_PANIC is not set
+CONFIG_BOOTPARAM_SOFTLOCKUP_PANIC_VALUE=0
+# CONFIG_DETECT_HUNG_TASK is not set
+# CONFIG_PANIC_ON_OOPS is not set
+CONFIG_PANIC_ON_OOPS_VALUE=0
+CONFIG_PANIC_TIMEOUT=0
+CONFIG_SCHED_DEBUG=y
+CONFIG_SCHED_INFO=y
+# CONFIG_SCHEDSTATS is not set
+# CONFIG_SCHED_STACK_END_CHECK is not set
+# CONFIG_DEBUG_TIMEKEEPING is not set
+CONFIG_TIMER_STATS=y
+# CONFIG_DEBUG_PREEMPT is not set
+
+#
+# Lock Debugging (spinlocks, mutexes, etc...)
+#
+# CONFIG_DEBUG_RT_MUTEXES is not set
+# CONFIG_DEBUG_SPINLOCK is not set
+# CONFIG_DEBUG_MUTEXES is not set
+# CONFIG_DEBUG_WW_MUTEX_SLOWPATH is not set
+# CONFIG_DEBUG_LOCK_ALLOC is not set
+# CONFIG_PROVE_LOCKING is not set
+# CONFIG_LOCK_STAT is not set
+# CONFIG_DEBUG_ATOMIC_SLEEP is not set
+# CONFIG_DEBUG_LOCKING_API_SELFTESTS is not set
+# CONFIG_LOCK_TORTURE_TEST is not set
+CONFIG_STACKTRACE=y
+# CONFIG_DEBUG_KOBJECT is not set
+CONFIG_HAVE_DEBUG_BUGVERBOSE=y
+CONFIG_DEBUG_BUGVERBOSE=y
+# CONFIG_DEBUG_LIST is not set
+# CONFIG_DEBUG_PI_LIST is not set
+# CONFIG_DEBUG_SG is not set
+# CONFIG_DEBUG_NOTIFIERS is not set
+# CONFIG_DEBUG_CREDENTIALS is not set
+
+#
+# RCU Debugging
+#
+# CONFIG_PROVE_RCU is not set
+CONFIG_SPARSE_RCU_POINTER=y
+# CONFIG_TORTURE_TEST is not set
+# CONFIG_RCU_TORTURE_TEST is not set
+CONFIG_RCU_CPU_STALL_TIMEOUT=60
+# CONFIG_RCU_TRACE is not set
+# CONFIG_RCU_EQS_DEBUG is not set
+# CONFIG_DEBUG_BLOCK_EXT_DEVT is not set
+# CONFIG_NOTIFIER_ERROR_INJECTION is not set
+# CONFIG_FAULT_INJECTION is not set
+CONFIG_NOP_TRACER=y
+CONFIG_HAVE_FUNCTION_TRACER=y
+CONFIG_HAVE_FUNCTION_GRAPH_TRACER=y
+CONFIG_HAVE_DYNAMIC_FTRACE=y
+CONFIG_HAVE_FTRACE_MCOUNT_RECORD=y
+CONFIG_HAVE_SYSCALL_TRACEPOINTS=y
+CONFIG_HAVE_C_RECORDMCOUNT=y
+CONFIG_TRACER_MAX_TRACE=y
+CONFIG_TRACE_CLOCK=y
+CONFIG_RING_BUFFER=y
+CONFIG_EVENT_TRACING=y
+CONFIG_CONTEXT_SWITCH_TRACER=y
+CONFIG_TRACING=y
+CONFIG_GENERIC_TRACER=y
+CONFIG_TRACING_SUPPORT=y
+CONFIG_FTRACE=y
+CONFIG_FUNCTION_TRACER=y
+# CONFIG_FUNCTION_GRAPH_TRACER is not set
+# CONFIG_IRQSOFF_TRACER is not set
+# CONFIG_PREEMPT_TRACER is not set
+CONFIG_SCHED_TRACER=y
+CONFIG_FTRACE_SYSCALLS=y
+CONFIG_TRACER_SNAPSHOT=y
+# CONFIG_TRACER_SNAPSHOT_PER_CPU_SWAP is not set
+CONFIG_BRANCH_PROFILE_NONE=y
+# CONFIG_PROFILE_ANNOTATED_BRANCHES is not set
+# CONFIG_PROFILE_ALL_BRANCHES is not set
+CONFIG_STACK_TRACER=y
+CONFIG_BLK_DEV_IO_TRACE=y
+# CONFIG_PROBE_EVENTS is not set
+CONFIG_DYNAMIC_FTRACE=y
+CONFIG_FUNCTION_PROFILER=y
+CONFIG_FTRACE_MCOUNT_RECORD=y
+# CONFIG_FTRACE_STARTUP_TEST is not set
+# CONFIG_TRACEPOINT_BENCHMARK is not set
+# CONFIG_RING_BUFFER_BENCHMARK is not set
+# CONFIG_RING_BUFFER_STARTUP_TEST is not set
+# CONFIG_TRACE_ENUM_MAP_FILE is not set
+CONFIG_TRACING_EVENTS_GPIO=y
+
+#
+# Runtime Testing
+#
+# CONFIG_LKDTM is not set
+# CONFIG_TEST_LIST_SORT is not set
+# CONFIG_BACKTRACE_SELF_TEST is not set
+# CONFIG_RBTREE_TEST is not set
+# CONFIG_INTERVAL_TREE_TEST is not set
+# CONFIG_PERCPU_TEST is not set
+CONFIG_ATOMIC64_SELFTEST=y
+# CONFIG_TEST_HEXDUMP is not set
+# CONFIG_TEST_STRING_HELPERS is not set
+CONFIG_TEST_KSTRTOX=y
+# CONFIG_TEST_PRINTF is not set
+# CONFIG_TEST_RHASHTABLE is not set
+# CONFIG_DMA_API_DEBUG is not set
+# CONFIG_TEST_LKM is not set
+# CONFIG_TEST_USER_COPY is not set
+# CONFIG_TEST_BPF is not set
+# CONFIG_TEST_FIRMWARE is not set
+# CONFIG_TEST_UDELAY is not set
+# CONFIG_MEMTEST is not set
+# CONFIG_TEST_STATIC_KEYS is not set
+# CONFIG_SAMPLES is not set
+CONFIG_HAVE_ARCH_KGDB=y
+# CONFIG_KGDB is not set
+# CONFIG_ARM64_PTDUMP is not set
+CONFIG_STRICT_DEVMEM=y
+# CONFIG_PID_IN_CONTEXTIDR is not set
+# CONFIG_ARM64_RANDOMIZE_TEXT_OFFSET is not set
+# CONFIG_DEBUG_SET_MODULE_RONX is not set
+# CONFIG_DEBUG_RODATA is not set
+# CONFIG_CORESIGHT is not set
+
+#
+# Security options
+#
+CONFIG_KEYS=y
+CONFIG_PERSISTENT_KEYRINGS=y
+CONFIG_BIG_KEYS=y
+# CONFIG_ENCRYPTED_KEYS is not set
+# CONFIG_SECURITY_DMESG_RESTRICT is not set
+CONFIG_SECURITY=y
+CONFIG_SECURITYFS=y
+CONFIG_SECURITY_NETWORK=y
+CONFIG_SECURITY_NETWORK_XFRM=y
+# CONFIG_SECURITY_PATH is not set
+CONFIG_LSM_MMAP_MIN_ADDR=65536
+CONFIG_SECURITY_SELINUX=y
+CONFIG_SECURITY_SELINUX_BOOTPARAM=y
+CONFIG_SECURITY_SELINUX_BOOTPARAM_VALUE=1
+CONFIG_SECURITY_SELINUX_DISABLE=y
+CONFIG_SECURITY_SELINUX_DEVELOP=y
+CONFIG_SECURITY_SELINUX_AVC_STATS=y
+CONFIG_SECURITY_SELINUX_CHECKREQPROT_VALUE=1
+# CONFIG_SECURITY_SELINUX_POLICYDB_VERSION_MAX is not set
+# CONFIG_SECURITY_SMACK is not set
+# CONFIG_SECURITY_TOMOYO is not set
+# CONFIG_SECURITY_APPARMOR is not set
+# CONFIG_SECURITY_YAMA is not set
+CONFIG_INTEGRITY=y
+# CONFIG_INTEGRITY_SIGNATURE is not set
+CONFIG_INTEGRITY_AUDIT=y
+# CONFIG_IMA is not set
+# CONFIG_EVM is not set
+CONFIG_DEFAULT_SECURITY_SELINUX=y
+# CONFIG_DEFAULT_SECURITY_DAC is not set
+CONFIG_DEFAULT_SECURITY="selinux"
+CONFIG_CRYPTO=y
+
+#
+# Crypto core or helper
+#
+CONFIG_CRYPTO_ALGAPI=y
+CONFIG_CRYPTO_ALGAPI2=y
+CONFIG_CRYPTO_AEAD=y
+CONFIG_CRYPTO_AEAD2=y
+CONFIG_CRYPTO_BLKCIPHER=y
+CONFIG_CRYPTO_BLKCIPHER2=y
+CONFIG_CRYPTO_HASH=y
+CONFIG_CRYPTO_HASH2=y
+CONFIG_CRYPTO_RNG=y
+CONFIG_CRYPTO_RNG2=y
+CONFIG_CRYPTO_RNG_DEFAULT=y
+CONFIG_CRYPTO_PCOMP2=y
+CONFIG_CRYPTO_AKCIPHER2=y
+# CONFIG_CRYPTO_RSA is not set
+CONFIG_CRYPTO_MANAGER=y
+CONFIG_CRYPTO_MANAGER2=y
+# CONFIG_CRYPTO_USER is not set
+# CONFIG_CRYPTO_MANAGER_DISABLE_TESTS is not set
+# CONFIG_CRYPTO_GF128MUL is not set
+CONFIG_CRYPTO_NULL=y
+CONFIG_CRYPTO_NULL2=y
+# CONFIG_CRYPTO_PCRYPT is not set
+CONFIG_CRYPTO_WORKQUEUE=y
+CONFIG_CRYPTO_CRYPTD=y
+# CONFIG_CRYPTO_MCRYPTD is not set
+# CONFIG_CRYPTO_AUTHENC is not set
+# CONFIG_CRYPTO_TEST is not set
+CONFIG_CRYPTO_ABLK_HELPER=y
+
+#
+# Authenticated Encryption with Associated Data
+#
+# CONFIG_CRYPTO_CCM is not set
+# CONFIG_CRYPTO_GCM is not set
+# CONFIG_CRYPTO_CHACHA20POLY1305 is not set
+CONFIG_CRYPTO_SEQIV=y
+CONFIG_CRYPTO_ECHAINIV=m
+
+#
+# Block modes
+#
+CONFIG_CRYPTO_CBC=y
+CONFIG_CRYPTO_CTR=y
+CONFIG_CRYPTO_CTS=y
+CONFIG_CRYPTO_ECB=y
+# CONFIG_CRYPTO_LRW is not set
+# CONFIG_CRYPTO_PCBC is not set
+# CONFIG_CRYPTO_XTS is not set
+# CONFIG_CRYPTO_KEYWRAP is not set
+
+#
+# Hash modes
+#
+# CONFIG_CRYPTO_CMAC is not set
+CONFIG_CRYPTO_HMAC=y
+# CONFIG_CRYPTO_XCBC is not set
+# CONFIG_CRYPTO_VMAC is not set
+
+#
+# Digest
+#
+CONFIG_CRYPTO_CRC32C=y
+# CONFIG_CRYPTO_CRC32 is not set
+CONFIG_CRYPTO_CRCT10DIF=y
+# CONFIG_CRYPTO_GHASH is not set
+# CONFIG_CRYPTO_POLY1305 is not set
+# CONFIG_CRYPTO_MD4 is not set
+CONFIG_CRYPTO_MD5=y
+# CONFIG_CRYPTO_MICHAEL_MIC is not set
+# CONFIG_CRYPTO_RMD128 is not set
+# CONFIG_CRYPTO_RMD160 is not set
+# CONFIG_CRYPTO_RMD256 is not set
+# CONFIG_CRYPTO_RMD320 is not set
+CONFIG_CRYPTO_SHA1=y
+CONFIG_CRYPTO_SHA256=y
+# CONFIG_CRYPTO_SHA512 is not set
+# CONFIG_CRYPTO_TGR192 is not set
+# CONFIG_CRYPTO_WP512 is not set
+
+#
+# Ciphers
+#
+CONFIG_CRYPTO_AES=y
+# CONFIG_CRYPTO_ANUBIS is not set
+CONFIG_CRYPTO_ARC4=y
+# CONFIG_CRYPTO_BLOWFISH is not set
+# CONFIG_CRYPTO_CAMELLIA is not set
+# CONFIG_CRYPTO_CAST5 is not set
+# CONFIG_CRYPTO_CAST6 is not set
+CONFIG_CRYPTO_DES=y
+# CONFIG_CRYPTO_FCRYPT is not set
+# CONFIG_CRYPTO_KHAZAD is not set
+# CONFIG_CRYPTO_SALSA20 is not set
+# CONFIG_CRYPTO_CHACHA20 is not set
+# CONFIG_CRYPTO_SEED is not set
+# CONFIG_CRYPTO_SERPENT is not set
+# CONFIG_CRYPTO_TEA is not set
+# CONFIG_CRYPTO_TWOFISH is not set
+
+#
+# Compression
+#
+# CONFIG_CRYPTO_DEFLATE is not set
+# CONFIG_CRYPTO_ZLIB is not set
+CONFIG_CRYPTO_LZO=y
+# CONFIG_CRYPTO_842 is not set
+# CONFIG_CRYPTO_LZ4 is not set
+# CONFIG_CRYPTO_LZ4HC is not set
+
+#
+# Random Number Generation
+#
+# CONFIG_CRYPTO_ANSI_CPRNG is not set
+CONFIG_CRYPTO_DRBG_MENU=y
+CONFIG_CRYPTO_DRBG_HMAC=y
+# CONFIG_CRYPTO_DRBG_HASH is not set
+# CONFIG_CRYPTO_DRBG_CTR is not set
+CONFIG_CRYPTO_DRBG=y
+CONFIG_CRYPTO_JITTERENTROPY=y
+CONFIG_CRYPTO_USER_API=y
+CONFIG_CRYPTO_USER_API_HASH=y
+CONFIG_CRYPTO_USER_API_SKCIPHER=y
+# CONFIG_CRYPTO_USER_API_RNG is not set
+# CONFIG_CRYPTO_USER_API_AEAD is not set
+CONFIG_CRYPTO_HW=y
+CONFIG_CRYPTO_DEV_CCP=y
+# CONFIG_CRYPTO_DEV_CCP_DD is not set
+# CONFIG_ASYMMETRIC_KEY_TYPE is not set
+
+#
+# Certificates for signature checking
+#
+# CONFIG_SYSTEM_TRUSTED_KEYRING is not set
+CONFIG_ARM64_CRYPTO=y
+CONFIG_CRYPTO_SHA1_ARM64_CE=y
+CONFIG_CRYPTO_SHA2_ARM64_CE=y
+CONFIG_CRYPTO_GHASH_ARM64_CE=y
+CONFIG_CRYPTO_AES_ARM64_CE=y
+# CONFIG_CRYPTO_AES_ARM64_CE_CCM is not set
+CONFIG_CRYPTO_AES_ARM64_CE_BLK=y
+# CONFIG_CRYPTO_AES_ARM64_NEON_BLK is not set
+# CONFIG_CRYPTO_CRC32_ARM64 is not set
+CONFIG_BINARY_PRINTF=y
+
+#
+# Library routines
+#
+CONFIG_BITREVERSE=y
+CONFIG_HAVE_ARCH_BITREVERSE=y
+CONFIG_RATIONAL=y
+CONFIG_GENERIC_STRNCPY_FROM_USER=y
+CONFIG_GENERIC_STRNLEN_USER=y
+CONFIG_GENERIC_NET_UTILS=y
+CONFIG_GENERIC_PCI_IOMAP=y
+CONFIG_GENERIC_IO=y
+CONFIG_ARCH_USE_CMPXCHG_LOCKREF=y
+# CONFIG_CRC_CCITT is not set
+CONFIG_CRC16=y
+CONFIG_CRC_T10DIF=y
+# CONFIG_CRC_ITU_T is not set
+CONFIG_CRC32=y
+# CONFIG_CRC32_SELFTEST is not set
+CONFIG_CRC32_SLICEBY8=y
+# CONFIG_CRC32_SLICEBY4 is not set
+# CONFIG_CRC32_SARWATE is not set
+# CONFIG_CRC32_BIT is not set
+# CONFIG_CRC7 is not set
+# CONFIG_LIBCRC32C is not set
+# CONFIG_CRC8 is not set
+CONFIG_AUDIT_GENERIC=y
+CONFIG_AUDIT_ARCH_COMPAT_GENERIC=y
+# CONFIG_RANDOM32_SELFTEST is not set
+CONFIG_ZLIB_INFLATE=y
+CONFIG_ZLIB_DEFLATE=y
+CONFIG_LZO_COMPRESS=y
+CONFIG_LZO_DECOMPRESS=y
+CONFIG_LZ4_DECOMPRESS=y
+CONFIG_XZ_DEC=y
+CONFIG_XZ_DEC_X86=y
+CONFIG_XZ_DEC_POWERPC=y
+CONFIG_XZ_DEC_IA64=y
+CONFIG_XZ_DEC_ARM=y
+CONFIG_XZ_DEC_ARMTHUMB=y
+CONFIG_XZ_DEC_SPARC=y
+CONFIG_XZ_DEC_BCJ=y
+# CONFIG_XZ_DEC_TEST is not set
+CONFIG_DECOMPRESS_GZIP=y
+CONFIG_DECOMPRESS_BZIP2=y
+CONFIG_DECOMPRESS_LZMA=y
+CONFIG_DECOMPRESS_XZ=y
+CONFIG_DECOMPRESS_LZO=y
+CONFIG_DECOMPRESS_LZ4=y
+CONFIG_GENERIC_ALLOCATOR=y
+CONFIG_ASSOCIATIVE_ARRAY=y
+CONFIG_HAS_IOMEM=y
+CONFIG_HAS_IOPORT_MAP=y
+CONFIG_HAS_DMA=y
+CONFIG_CPU_RMAP=y
+CONFIG_DQL=y
+CONFIG_GLOB=y
+# CONFIG_GLOB_SELFTEST is not set
+CONFIG_NLATTR=y
+CONFIG_ARCH_HAS_ATOMIC64_DEC_IF_POSITIVE=y
+# CONFIG_CORDIC is not set
+# CONFIG_DDR is not set
+CONFIG_LIBFDT=y
+CONFIG_OID_REGISTRY=y
+CONFIG_UCS2_STRING=y
+CONFIG_FONT_SUPPORT=y
+# CONFIG_FONTS is not set
+CONFIG_FONT_8x8=y
+CONFIG_FONT_8x16=y
+# CONFIG_SG_SPLIT is not set
+CONFIG_ARCH_HAS_SG_CHAIN=y
--
2.7.0

[antonios...@huawei.com]

From: Dmitry Voytik <dmitry...@huawei.com>

Add the asm/types.h header file, which defines the size of the data
types.

Signed-off-by: Dmitry Voytik <dmitry...@huawei.com>
---

hypervisor/arch/arm64/include/asm/types.h | 46 +++++++++++++++++++++++++++++++
1 file changed, 46 insertions(+)
create mode 100644 hypervisor/arch/arm64/include/asm/types.h

diff --git a/hypervisor/arch/arm64/include/asm/types.h b/hypervisor/arch/arm64/include/asm/types.h
new file mode 100644
index 0000000..10760a0
--- /dev/null
+++ b/hypervisor/arch/arm64/include/asm/types.h
@@ -0,0 +1,46 @@

+/*
+ * Jailhouse AArch64 support
+ *
+ * Copyright (C) 2015 Huawei Technologies Duesseldorf GmbH
+ *
+ * Authors:

+ * Dmitry Voytik <dmitry...@huawei.com>

+ *
+ * This work is licensed under the terms of the GNU GPL, version 2. See
+ * the COPYING file in the top-level directory.

+ */
+
+#ifndef _JAILHOUSE_ASM_TYPES_H
+#define _JAILHOUSE_ASM_TYPES_H
+
+#define BITS_PER_LONG 64
+
+#ifndef __ASSEMBLY__
+
+typedef signed char s8;
+typedef unsigned char u8;
+
+typedef signed short s16;
+typedef unsigned short u16;
+
+typedef signed int s32;
+typedef unsigned int u32;
+
+typedef signed long long s64;
+typedef unsigned long long u64;
+
+typedef s8 __s8;
+typedef u8 __u8;
+
+typedef s16 __s16;
+typedef u16 __u16;
+
+typedef s32 __s32;
+typedef u32 __u32;
+
+typedef s64 __s64;
+typedef u64 __u64;

+
+#endif /* !__ASSEMBLY__ */
+

+#endif /* !_JAILHOUSE_ASM_TYPES_H */
--
2.7.0

[antonios...@huawei.com]

From: Dmitry Voytik <dmitry...@huawei.com>

Add ./scripts/arm64-parsedump.py tool. This tool decodes an ARM64
exception dump and prints human-readable stack trace like this:

[0x00000000fc008688] arch_handle_dabt mmio.c:97
[0x00000000fc009acc] arch_handle_trap traps.c:143

The tool can read dumps from files (passed via -f parameter)
or from stdin stream (which can be also piped-in).

Signed-off-by: Dmitry Voytik <dmitry...@huawei.com>
---

scripts/arm64-parsedump.py | 167 +++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 167 insertions(+)
create mode 100755 scripts/arm64-parsedump.py

diff --git a/scripts/arm64-parsedump.py b/scripts/arm64-parsedump.py
new file mode 100755
index 0000000..9519eb5
--- /dev/null
+++ b/scripts/arm64-parsedump.py
@@ -0,0 +1,167 @@
+#!/usr/bin/env python
+
+# Jailhouse, a Linux-based partitioning hypervisor
+#
+# Copyright (C) 2015-2016 Huawei Technologies Duesseldorf GmbH
+#
+# Authors:
+# Dmitry Voytik <dmitry...@huawei.com>
+#
+# ARM64 dump parser.
+# Usage ./scripts/arm64-parsedump.py [dump.txt]
+#
+# This work is licensed under the terms of the GNU GPL, version 2. See
+# the COPYING file in the top-level directory.
+
+
+from __future__ import print_function
+import subprocess
+import sys
+import fileinput
+import os
+import argparse
+
+split1 = "Cell's stack before exception "
+split2 = "Hypervisor stack before exception "
+
+
+# yep, this is most important feature
+class Col:
+ ENDC = '\033[0m'
+ BOLD = '\033[1m'
+ FAIL = '\033[91m'
+
+ @staticmethod
+ def init():
+ t = os.environ['TERM']
+ if t == '' or t == 'dumb' or t == 'vt220' or t == 'vt100':
+ # The terminal doesn't support colors
+ Col.ENDC = ''
+ Col.BOLD = ''
+ Col.FAIL = ''
+
+ @staticmethod
+ def bold(string):
+ return Col.BOLD + str(string) + Col.ENDC
+
+ @staticmethod
+ def pr_err(string):
+ print(Col.FAIL + "ERROR: " + Col.ENDC + str(string))
+
+ @staticmethod
+ def pr_note(string):
+ print(Col.BOLD + "NOTE: " + Col.ENDC + str(string))
+
+
+def addr2line(addr):
+ return subprocess.check_output(["addr2line", "-a", "-f", "-p", "-e",
+ objpath, hex(addr)])
+
+
+def print_faddr(addr):
+ s = str(addr2line(addr))
+ if s.find("?") != -1:
+ print("[{:#016x}] {}".format(addr, Col.bold("uknown")))
+ return
+ s = s.strip().split(" ")
+ print("[{}] {} {}".format(s[0][:-1], Col.bold(s[1]),
+ s[3].split('/')[-1]))
+
+
+class Dump:
+ def __init__(self, dump_str):
+ if len(dump_str) < 50:
+ raise ValueError('Dump is too small')
+ # parse CPU state
+ pc_i = dump_str.find("pc:") + 4
+ self.pc = int(dump_str[pc_i: pc_i+16], 16)
+ pc_i = dump_str.find("sp:") + 4
+ self.sp = int(dump_str[pc_i: pc_i+16], 16)
+ el_i = dump_str.rfind("EL") + 2
+ self.el = int(dump_str[el_i:el_i+1])
+ if (self.el != 2):
+ Col.pr_err("This version supports only EL2 exception dump")
+
+ # TODO: parse other registers: ESR, etc
+
+ # parse stack dump
+ stack_start = str.rfind(dump_str, split1)
+ if (stack_start == -1):
+ stack_start = str.rfind(dump_str, split2)
+ if (stack_start == -1):
+ raise ValueError('Dump is damaged')
+
+ stack_str = dump_str[stack_start:].strip().split('\n')
+ stack_addr_start = int(stack_str[0][35:53], 16)
+ stack_addr_end = int(stack_str[0][56:74], 16)
+
+ # parse stack memory dump
+ stack = []
+ for line in stack_str[1:]:
+ if (len(line) < 5):
+ continue
+ if (line[4] != ':'):
+ continue
+ line = line[5:].strip().split(" ")
+ for value in line:
+ stack.append(int(value, 16))
+
+ self.stack_mem = stack
+ self.stack_start = stack_addr_start
+ self.stack_end = stack_addr_end
+
+ def stack_get64(self, addr):
+ assert addr >= self.sp
+ i = int((addr - self.sp) / 4)
+ hi32 = self.stack_mem[i]
+ lo32 = self.stack_mem[i + 1]
+ return lo32 + (hi32 << 32)
+
+ def print_unwinded_stack(self):
+ print_faddr(self.pc)
+ addr = self.sp
+ while True:
+ prev_sp = self.stack_get64(addr)
+ print_faddr(self.stack_get64(addr+4))
+ addr = prev_sp
+ if (addr > self.stack_end - 256):
+ break
+
+
+def main():
+ Col.init()
+
+ parser = argparse.ArgumentParser(description='ARM64 exception dump parser')
+ parser.add_argument('--objpath', '-o', default="./hypervisor/hypervisor.o",
+ type=str, help="Path to hypervisor.o file")
+ parser.add_argument('-f', '--filedump', default="", type=str,
+ help="Exception dump text file")
+ args = parser.parse_args()
+
+ global objpath
+ objpath = args.objpath
+
+ stdin_used = False
+ infile = [args.filedump]
+ if args.filedump == "":
+ infile = []
+ Col.pr_note("Input dumped text then press Enter, Control+D, Control+D")
+ stdin_used = True
+
+ ilines = []
+ for line in fileinput.input(infile):
+ ilines.append(line)
+ dump_str = "".join(ilines)
+ if (not stdin_used):
+ print(dump_str)
+ else:
+ print("\n")
+ try:
+ dump = Dump(dump_str)
+ except ValueError as err:
+ Col.pr_err(err)
+ return
+ dump.print_unwinded_stack()
+
+if __name__ == "__main__":
+ main()
--
2.7.0

[antonios...@huawei.com]

From: Dmitry Voytik <dmitry...@huawei.com>

Add the header file control.h to the AArch64 port of Jailhouse.

Signed-off-by: Dmitry Voytik <dmitry...@huawei.com>
Signed-off-by: Antonios Motakis <antonios...@huawei.com>
[antonios...@huawei.com: split off as a separate patch]
---
hypervisor/arch/arm64/include/asm/control.h | 42 +++++++++++++++++++++++++++++
1 file changed, 42 insertions(+)
create mode 100644 hypervisor/arch/arm64/include/asm/control.h

diff --git a/hypervisor/arch/arm64/include/asm/control.h b/hypervisor/arch/arm64/include/asm/control.h
new file mode 100644
index 0000000..1957d55
--- /dev/null
+++ b/hypervisor/arch/arm64/include/asm/control.h
@@ -0,0 +1,42 @@

+/*
+ * Jailhouse AArch64 support
+ *
+ * Copyright (C) 2015 Huawei Technologies Duesseldorf GmbH
+ *
+ * Authors:
+ * Dmitry Voytik <dmitry...@huawei.com>
+ *

+ * This work is licensed under the terms of the GNU GPL, version 2. See
+ * the COPYING file in the top-level directory.
+ */
+
+#ifndef _JAILHOUSE_ASM_CONTROL_H
+#define _JAILHOUSE_ASM_CONTROL_H
+
+#define SGI_INJECT 0
+#define SGI_CPU_OFF 1
+
+#define CACHES_CLEAN 0
+#define CACHES_CLEAN_INVALIDATE 1
+
+#include <asm/percpu.h>
+
+static inline void arch_cpu_dcaches_flush(unsigned int action) { }
+static inline void arch_cpu_icache_flush(void) { }
+
+void arch_cpu_tlb_flush(struct per_cpu *cpu_data);
+void arch_cell_caches_flush(struct cell *cell);
+int arch_mmu_cell_init(struct cell *cell);
+void arch_mmu_cell_destroy(struct cell *cell);
+int arch_mmu_cpu_cell_init(struct per_cpu *cpu_data);
+void arch_handle_sgi(struct per_cpu *cpu_data, u32 irqn);

+struct registers* arch_handle_exit(struct per_cpu *cpu_data,

+ struct registers *regs);
+bool arch_handle_phys_irq(struct per_cpu *cpu_data, u32 irqn);
+void arch_reset_self(struct per_cpu *cpu_data);

+void arch_shutdown_self(struct per_cpu *cpu_data);
+

+void __attribute__((noreturn)) vmreturn(struct registers *guest_regs);
+void __attribute__((noreturn)) arch_shutdown_mmu(struct per_cpu *cpu_data);
+
+#endif /* !_JAILHOUSE_ASM_CONTROL_H */
--
2.7.0

[antonios...@huawei.com]

From: Claudio Fontana <claudio...@huawei.com>

implement set_bit, clear_bit, test_and_set_bit.

test_and_set_bit is used only in the panic_printk apparently,
while set_bit and clear_bit are used in page table handling code.

Signed-off-by: Claudio Fontana <claudio...@huawei.com>
---
hypervisor/arch/arm64/include/asm/bitops.h | 141 +++++++++++++++++++++++++++++
1 file changed, 141 insertions(+)
create mode 100644 hypervisor/arch/arm64/include/asm/bitops.h

diff --git a/hypervisor/arch/arm64/include/asm/bitops.h b/hypervisor/arch/arm64/include/asm/bitops.h
new file mode 100644
index 0000000..c4b1ea4
--- /dev/null
+++ b/hypervisor/arch/arm64/include/asm/bitops.h
@@ -0,0 +1,141 @@

+/*
+ * Jailhouse AArch64 support
+ *
+ * Copyright (C) 2015 Huawei Technologies Duesseldorf GmbH
+ *
+ * Authors:

+ * Claudio Fontana <claudio...@huawei.com>

+ * Antonios Motakis <antonios...@huawei.com>

+ *
+ * This work is licensed under the terms of the GNU GPL, version 2. See
+ * the COPYING file in the top-level directory.
+ */
+

+#ifndef _JAILHOUSE_ASM_BITOPS_H
+#define _JAILHOUSE_ASM_BITOPS_H
+
+#include <jailhouse/types.h>
+
+#ifndef __ASSEMBLY__
+
+#define BITOPT_ALIGN(bits, addr) \
+ do { \
+ (addr) = (unsigned long *)((u64)(addr) & ~0x7) \
+ + (bits) / BITS_PER_LONG; \
+ (bits) %= BITS_PER_LONG; \
+ } while (0)
+
+static inline __attribute__((always_inline)) void
+clear_bit(int nr, volatile unsigned long *addr)
+{
+ u32 ret;
+ u64 tmp;
+
+ BITOPT_ALIGN(nr, addr);
+
+ /* AARCH64_TODO: do we need to preload? */
+ do {
+ asm volatile (
+ "ldxr %2, %1\n\t"
+ "bic %2, %2, %3\n\t"
+ "stxr %w0, %2, %1\n\t"
+ : "=r" (ret),
+ "+Q" (*(volatile unsigned long *)addr),
+ "=r" (tmp)
+ : "r" (1ul << nr));
+ } while (ret);
+}
+
+static inline __attribute__((always_inline)) void
+set_bit(unsigned int nr, volatile unsigned long *addr)
+{
+ u32 ret;
+ u64 tmp;
+
+ BITOPT_ALIGN(nr, addr);
+
+ /* AARCH64_TODO: do we need to preload? */
+ do {
+ asm volatile (
+ "ldxr %2, %1\n\t"
+ "orr %2, %2, %3\n\t"
+ "stxr %w0, %2, %1\n\t"
+ : "=r" (ret),
+ "+Q" (*(volatile unsigned long *)addr),
+ "=r" (tmp)
+ : "r" (1ul << nr));
+ } while (ret);
+}
+
+static inline __attribute__((always_inline)) int
+test_bit(unsigned int nr, const volatile unsigned long *addr)
+{
+ return ((1UL << (nr % BITS_PER_LONG)) &
+ (addr[nr / BITS_PER_LONG])) != 0;
+}
+
+static inline int test_and_set_bit(int nr, volatile unsigned long *addr)
+{
+ u32 ret;
+ u64 test, tmp;
+
+ BITOPT_ALIGN(nr, addr);
+
+ /* AARCH64_TODO: using Inner Shareable DMB at the moment,
+ * revisit when we will deal with shareability domains */
+
+ do {
+ asm volatile (
+ "ldxr %3, %2\n\t"
+ "ands %1, %3, %4\n\t"
+ "b.ne 1f\n\t"
+ "orr %3, %3, %4\n\t"
+ "1:\n\t"
+ "stxr %w0, %3, %2\n\t"
+ "dmb ish\n\t"
+ : "=r" (ret), "=&r" (test),
+ "+Q" (*(volatile unsigned long *)addr),
+ "=r" (tmp)
+ : "r" (1ul << nr));
+ } while (ret);
+ return !!(test);
+}
+
+/* Count leading zeroes */
+static inline unsigned long clz(unsigned long word)
+{
+ unsigned long val;
+
+ asm volatile ("clz %0, %1" : "=r" (val) : "r" (word));
+ return val;
+}
+
+/* Returns the position of the least significant 1, MSB=63, LSB=0*/
+static inline unsigned long ffsl(unsigned long word)
+{
+ if (!word)
+ return 0;
+ asm volatile ("rbit %0, %0" : "+r" (word));
+ return clz(word);
+}
+
+static inline unsigned long ffzl(unsigned long word)
+{
+ return ffsl(~word);
+}
+
+/* AARCH64_TODO: we can use SXTB, SXTH, SXTW */
+/* Extend the value of 'size' bits to a signed long */
+static inline unsigned long sign_extend(unsigned long val, unsigned int size)
+{
+ unsigned long mask;
+
+ if (size >= sizeof(unsigned long) * 8)
+ return val;
+
+ mask = 1ul << (size - 1);
+ return (val ^ mask) - mask;
+}

+
+#endif /* !__ASSEMBLY__ */

+#endif /* !_JAILHOUSE_ASM_BITOPS_H */
--
2.7.0

[antonios...@huawei.com]

From: Dmitry Voytik <dmitry...@huawei.com>

Dump stack in the following cases:
* exception in EL2. We can determine the stack size
* unhandled exceptions in EL1/0. We can't determine the stack
size thus we just print 512 bytes.

For EL2 exceptions the debug output will be like this:

FATAL: Unhandled HYP exception: synchronous abort from EL2
pc: 00000000fc00469c lr: 00000000fc004688 spsr: 200003c9 EL2
sp: 00000000fc015e30 esr: 25 1 0000044
x0: ffffffff00000000 x1: 0000000000000001 x2: 00000000fc00bd14
x3: ffffff80ffffffc8 x4: 00000000fc010000 x5: 0000000000000004
x6: ffffffc000afe000 x7: 00000000ffffe188 x8: 0000000000005d25
x9: 0000000000000001 x10: ffffffc035766a40 x11: ffffffbdc2d23f80
x12: 0000000000000862 x13: 0000007f92bd7cb0 x14: 0000007f92a67bc8
x15: 0000000000005798 x16: ffffffc0000a2794 x17: 0000000000412288
x18: 0000000000000000 x19: 0000000001930047 x20: 0000000000000004
x21: 0000000000000001 x22: 0000000000000001 x23: 00000000fc015eb8
x24: 00000000000001c0 x25: 0000000000000000 x26: ffffffc000afe6d8
x27: ffffffc035470000 x28: ffffffc034e08000 x29: 00000000fc015e30

Hypervisor stack before exception (0x00000000fc015e30 - 0x00000000fc016000):
5e20: fc015e90 00000000 fc00a298 00000000
5e40: fc015f00 00000000 fc015000 00000000 00559cb8 ffffffc0 00b69000 ffffffc0
5e60: 00b00000 ffffffc0 000001c0 00000000 0000001e 00000000 fc015cf5 00000000
5e80: fc015e90 00000000 0000001e 001e0100 fc015ee0 00000000 fc00a3b0 00000000
5ea0: fc015f00 00000000 fc00b478 00000000 fc015ee0 00000000 fc015f08 00000000
5ec0: 93930047 00000000 200001c5 00000000 00310820 ffffffc0 34e0bbc0 ffffffc0
5ee0: 00000000 00000000 fc009c54 00000000 00040000 00000000 00b00ea0 ffffffc0
5f00: 00000001 00000000 00000f00 ffffff80 00000004 00000000 00000040 00000000
5f20: 00b00ee0 ffffffc0 00559cc0 ffffffc0 00000004 00000000 00afe000 ffffffc0
5f40: ffffe188 00000000 00005d25 00000000 00000001 00000000 35766a40 ffffffc0
5f60: c2d23f80 ffffffbd 00000862 00000000 92bd7cb0 0000007f 92a67bc8 0000007f
5f80: 00005798 00000000 000a2794 ffffffc0 00412288 00000000 00000000 00000000
5fa0: 00040000 00000000 00b00ea0 ffffffc0 00559cb8 ffffffc0 00b69000 ffffffc0
5fc0: 00b00000 ffffffc0 000001c0 00000000 00000000 00000000 00afe6d8 ffffffc0
5fe0: 35470000 ffffffc0 34e08000 ffffffc0 34e0bbc0 ffffffc0 003107fc ffffffc0
6000: 8008a800

Signed-off-by: Dmitry Voytik <dmitry...@huawei.com>
---

hypervisor/arch/arm64/traps.c | 38 +++++++++++++++++++++++++++++++++++++-
1 file changed, 37 insertions(+), 1 deletion(-)

diff --git a/hypervisor/arch/arm64/traps.c b/hypervisor/arch/arm64/traps.c
index b27bb2e..199b497 100644
--- a/hypervisor/arch/arm64/traps.c
+++ b/hypervisor/arch/arm64/traps.c
@@ -42,6 +42,38 @@ static void dump_regs(struct trap_context *ctx)
panic_printk("\n");
}

+/* TODO: move this function to an arch-independent code if other architectures
+ * will need it.
+ */
+static void dump_mem(unsigned long start, unsigned long stop)
+{
+ unsigned long caddr = start & ~0x1f;
+
+ if (stop <= start)
+ return;
+ printk("(0x%016lx - 0x%016lx):", start, stop);
+ for (;;) {
+ printk("\n%04lx: ", caddr & 0xffe0);
+ do {
+ if (caddr >= start)
+ printk("%08x ", *(unsigned int *)caddr);
+ else
+ printk(" ", *(unsigned int *)caddr);
+ caddr += 4;
+ } while ((caddr & 0x1f) && caddr < stop);
+ if (caddr >= stop)
+ break;
+ }
+ printk("\n");
+}
+
+static void dump_hyp_stack(const struct trap_context *ctx)
+{
+ panic_printk("Hypervisor stack before exception ");
+ dump_mem(ctx->sp, (unsigned long)this_cpu_data()->stack +
+ PERCPU_STACK_END);
+}
+

static void fill_trap_context(struct trap_context *ctx, struct registers *regs)

{
arm_read_sysreg(ELR_EL2, ctx->pc);
@@ -52,7 +84,10 @@ static void fill_trap_context(struct trap_context *ctx, struct registers *regs)
case 1:
arm_read_sysreg(SP_EL1, ctx->sp); break;
case 2:
- arm_read_sysreg(SP_EL2, ctx->sp); break;
+ /* SP_EL2 is not accessible in EL2. To obtain SP value before
+ * the excepton we can use the addres of *regs parameter. *regs
+ * is located in the stack (see handle_vmexit in exception.S) */
+ ctx->sp = (u64)(regs) + 16 * 16; break;
default:

ctx->sp = 0; break; /* should never happen */
}

@@ -93,6 +128,7 @@ static void arch_dump_exit(struct registers *regs, const char *reason)
fill_trap_context(&ctx, regs);

panic_printk("\nFATAL: Unhandled HYP exception: %s\n", reason);

dump_regs(&ctx);
+ dump_hyp_stack(&ctx);

}

struct registers *arch_handle_exit(struct per_cpu *cpu_data,

--
2.7.0

[Jan Kiszka]

On 2016-02-24 17:25, antonios...@huawei.com wrote:
> From: Antonios Motakis <antonios...@huawei.com>
>

> This patch series includes the port of Jailhouse on ARM AArch64.
> This is the first version of the series, where we drop the RFC
> tag, as the port has matured a lot since the first release.
>
> In this version of the series, we have splited some of the
> functionality and included in two separate series; one with
> any preparatory changes, and a separate series with the inmate
> demos for AArch64.
>
> These apply in succession (preparatory -> main -> inmates series),
> on the latest next branch from upstream, and they can also be
> pulled from https://github.com/tvelocity/jailhouse.git (branch arm64_v8)

Thanks for the update, looking forward to give them a try (but my test
board is in Barcelona ;) )! Please don't expect a full review of the
architecture part from me, I will mostly focus on generic bits and
formal aspects.

Some formal nit already regarding patch tagging: We use "core:" for
anything that touches the generic hypervisor parts and "<arch>:" for
architecture-specific hypervisor changes, but there is no "hypervisor:".
Furthermore we have "config:", "tools:", "drivers:" so far. If you touch
multiple subsystems, use "tag1, tag2, ...:".

Jan

--
Siemens AG, Corporate Technology, CT RDA ITP SES-DE
Corporate Competence Center Embedded Linux

[Antonios Motakis]

Ok, thanks for the clarification! (I swear I saw the hypervisor tag somewhere...)

It seems not all patches managed to get through our email server, at least not yet... :(
Let's see if I will need to resend those.

Cheers
Tony

--
Antonios Motakis
Virtualization Engineer
Huawei Technologies Duesseldorf GmbH
European Research Center
Riesstrasse 25, 80992 München

[Jan Kiszka]

On 2016-02-25 13:50, Antonios Motakis wrote:
>
>
> On 25-Feb-16 08:58, Jan Kiszka wrote:
>> On 2016-02-24 17:25, antonios...@huawei.com wrote:
>>> From: Antonios Motakis <antonios...@huawei.com>
>>>
>>> This patch series includes the port of Jailhouse on ARM AArch64.
>>> This is the first version of the series, where we drop the RFC
>>> tag, as the port has matured a lot since the first release.
>>>
>>> In this version of the series, we have splited some of the
>>> functionality and included in two separate series; one with
>>> any preparatory changes, and a separate series with the inmate
>>> demos for AArch64.
>>>
>>> These apply in succession (preparatory -> main -> inmates series),
>>> on the latest next branch from upstream, and they can also be
>>> pulled from https://github.com/tvelocity/jailhouse.git (branch arm64_v8)
>>
>> Thanks for the update, looking forward to give them a try (but my test
>> board is in Barcelona ;) )! Please don't expect a full review of the
>> architecture part from me, I will mostly focus on generic bits and
>> formal aspects.
>>
>> Some formal nit already regarding patch tagging: We use "core:" for
>> anything that touches the generic hypervisor parts and "<arch>:" for
>> architecture-specific hypervisor changes, but there is no "hypervisor:".
>> Furthermore we have "config:", "tools:", "drivers:" so far. If you touch
>> multiple subsystems, use "tag1, tag2, ...:".
>
> Ok, thanks for the clarification! (I swear I saw the hypervisor tag somewhere...)

Yeah, there are some, and 2 of them by me...

>
> It seems not all patches managed to get through our email server, at least not yet... :(
> Let's see if I will need to resend those.

No, I'm seeing complete queues, both in my inbox as well as in the list
archive.

[Antonios Motakis]

Ok, so the problem is on my client's side!

Good to know they went through then!

[Henning Schild]

"User-Agent: ... Windows NT 6.1... " i am not surprised ;)

I also had to wait for some of the mails for some time when they where
already in the archives.

Henning

[Jan Kiszka]

On 2016-02-25 08:58, Jan Kiszka wrote:
> On 2016-02-24 17:25, antonios...@huawei.com wrote:
>> From: Antonios Motakis <antonios...@huawei.com>
>>
>> This patch series includes the port of Jailhouse on ARM AArch64.
>> This is the first version of the series, where we drop the RFC
>> tag, as the port has matured a lot since the first release.
>>
>> In this version of the series, we have splited some of the
>> functionality and included in two separate series; one with
>> any preparatory changes, and a separate series with the inmate
>> demos for AArch64.
>>
>> These apply in succession (preparatory -> main -> inmates series),
>> on the latest next branch from upstream, and they can also be
>> pulled from https://github.com/tvelocity/jailhouse.git (branch arm64_v8)
>
> Thanks for the update, looking forward to give them a try (but my test
> board is in Barcelona ;) )! Please don't expect a full review of the
> architecture part from me, I will mostly focus on generic bits and
> formal aspects.

Just tried again on the Seattle, but now with some newer firmware, not
sure if that matters:

Version 2.17.1249. Copyright (C) 2016 American Megatrends, Inc.
BIOS Date: 02/09/2016 15:15:23 Ver: ROD1001A00

First finding: jailhouse disable is still broken (massive panic after a
simple enable/disable cycle). It starts with

Unable to handle kernel paging request at virtual address fffffc0016000000

repeated for each core.

Cell destruction and recreation works fine.

I've once seen some instability of the root cell. Still trying to
reproduce, will report in that case.

[Jan Kiszka]

Just as I pressed "send":

FATAL: exception unhandled trap
Cell state before exception:
pc: 0000000042c86764 lr: 0000000042c80270 spsr: 200001c5 EL1
sp: fffffe001bfb3b80 esr: 17 1 0000000
x0: 00000000c200ff00 x1: 00000083ff322018 x2: 000000000000000c
x3: 0000000000000000 x4: 0000000000000000 x5: 0000000000000000
x6: 0000000000000000 x7: 0000000000000000 x8: ffff7f7f7f7f7f7f
x9: fffffe0000762000 x10: 0101010101010101 x11: 0000000000000020
x12: fffffe0000dd05a8 x13: 000000000000006d x14: fffffdff00074d60
x15: fffffe001b350080 x16: 0000000000000008 x17: 0000000000000T0i0m1e
xf1i8r:e d0,0 0j0i0t0t0e0rd:d c5679c x1 9 :2 7f1f9f fnfse,0 0m0i0nd:6
75d0 x20: f2f7f1f1f en0s0,0 0mda6x7:0 00
x21: f f f3f4f7e90 0n1sb
b3cb0 x22: 0000000000000140 x23: fffffe00bff0f900
x24: 0000000000000000 x25: 0000000000000000 x26: 0000000000000000
x27: fffffe0000b30000 x28: 0000000000000000 x29: fffffe001bfb3b90

Parking CPU 2 (Cell: "AMD Seattle")
/root/jailhouse/hypervisor/arch/arm64/control.c:271: returning error -EINVAL


Output garbled by the gic-demo. There is no fault address reported,
unfortunately. What could it mean?

[Jan Kiszka]

On 2016-02-24 17:25, antonios...@huawei.com wrote:
> From: Antonios Motakis <antonios...@huawei.com>
>

> Add the root cell configuration and necessary headers to build
> and run Jailhouse on the AMD Seattle development board.

>
> Signed-off-by: Antonios Motakis <antonios...@huawei.com>
> ---

> ci/jailhouse-config-amd-seattle.h | 5 +
> configs/amd-seattle.c | 162 +++++++++++++++++++++++++++
> hypervisor/arch/arm/include/asm/uart_pl011.h | 2 +
> hypervisor/arch/arm64/include/asm/platform.h | 19 ++++
> 4 files changed, 188 insertions(+)
> create mode 100644 ci/jailhouse-config-amd-seattle.h
> create mode 100644 configs/amd-seattle.c
>
> diff --git a/ci/jailhouse-config-amd-seattle.h b/ci/jailhouse-config-amd-seattle.h
> new file mode 100644
> index 0000000..c721a46
> --- /dev/null
> +++ b/ci/jailhouse-config-amd-seattle.h
> @@ -0,0 +1,5 @@
> +#define CONFIG_TRACE_ERROR 1
> +#define CONFIG_ARM_GIC 1
> +#define CONFIG_MACH_AMD_SEATTLE 1
> +#define CONFIG_SERIAL_AMBA_PL011 1
> +#define JAILHOUSE_BASE 0x82fc000000
> diff --git a/configs/amd-seattle.c b/configs/amd-seattle.c
> new file mode 100644
> index 0000000..2bc27eb
> --- /dev/null
> +++ b/configs/amd-seattle.c
> @@ -0,0 +1,162 @@

> +/*
> + * Jailhouse AArch64 support
> + *
> + * Copyright (C) 2015 Huawei Technologies Duesseldorf GmbH
> + *
> + * Authors:

> + * Antonios Motakis <antonios...@huawei.com>
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2. See
> + * the COPYING file in the top-level directory.
> + */
> +

> +#include <linux/types.h>
> +#include <jailhouse/cell-config.h>
> +
> +#define ARRAY_SIZE(a) (sizeof(a) / sizeof(a[0]))
> +
> +struct {
> + struct jailhouse_system header;
> + __u64 cpus[1];
> + struct jailhouse_memory mem_regions[15];
> + struct jailhouse_irqchip irqchips[1];
> +} __attribute__((packed)) config = {
> + .header = {
> + .signature = JAILHOUSE_SYSTEM_SIGNATURE,
> + .hypervisor_memory = {
> + .phys_start = 0x82fc000000,
> + .size = 0x4000000,
> + },
> + .debug_console = {
> + .phys_start = 0xe1010000,
> + .size = 0x1000,
> + .flags = JAILHOUSE_MEM_IO,
> + },
> + .root_cell = {
> + .name = "AMD Seattle",

Spaces in the name are unhandy for jailhouse bash completion:

seattle:~/jailhouse # tools/jailhouse cell destroy
0 1 2 AMD gic-demo linux-inmate-demo --name Seattle

Use "-" or "_" instead. Applies to the Foundation Model as well.

[Jan Kiszka]

Apparently not a regression, just got basically the same after switching
back to v7:

FATAL: exception unhandled trap
Cell state before exception:

pc: 0000000042c86764 lr: 0000000042c80050 spsr: 200001c5 EL1
sp: fffffe00875cbae0 esr: 17 1 0000000

x0: 00000000c200ff00 x1: 00000083ff322018 x2: 000000000000000c
x3: 0000000000000000 x4: 0000000000000000 x5: 0000000000000000

x6: 0000000000000000 x7: 0000000000000000 x8: 0000000000000000
x9: 7f7f7f7f7f7f7f7f x10: 0101010101010101 x11: 7f7f7f7f7f7f7f7f
x12: 0101010101010101 x13: 0000000000000008 x14: 000003ff9e2afc70
x15: 000003ff9dff2800 x16: Tfifmfefrf ef0i0r0e0d2,1 ajbi5t0t e rx:1 7:
0000000 0 03034297c 4n2s8,
mxi1n8:: 000000000 0 020701090 1n s ,x 1m9a:x :f ffffe0000d 6 7359d803
nxs2
: fffffe0000d67000
0
x21: fffffe00875cbc30 x22: 0000000000000140 x23: fffffe00875cbc20
x24: fffffe00036c1200 x25: 0000000000000400 x26: fffffe0003628240
x27: 000000000bb13b30 x28: fffffe00875cbd70 x29: fffffe00875cbaf0

Parking CPU 3 (Cell: "AMD Seattle")
/root/jailhouse/hypervisor/arch/arm64/control.c:270: returning error -EINVAL


That used to work - firmware problem?

@Mazda: Could we try to roll back the firmware version of my board?

Thanks,

[Antonios Motakis]

Ack!

[Antonios Motakis]

I haven't run into these unfortunately.
I should try to reproduce using same kernel config. Our BIOS version is much older as well.

[Jan Kiszka]

I'm carefully optimistic: Mazda just pointed me to a firmware tunable
that was mistuned after the last update (ACLOCK -> 2 GHz), and now
things run smoothly for a couple of minutes already. Even latencies look
good again. Will keep an eye on this.

[Jan Kiszka]

And because the board is mystically connected with my email client, it
generated first the root cell crash just after I sent this, and then,
some seconds later, it produced a 60 µs latency peak. Sigh.

Tony, could you dig out your firmware version so that Mazda can provide
me the very same one for testing on my board? TIA!

[Antonios Motakis]

I get the following output when booting the board:

NOTICE: BL3-1:
NOTICE: BL3-1: Built : 15:50:01, May 7 2015
INFO: BL3-1: Initializing runtime services
INFO: BL3-1: Preparing for EL3 exit to normal world
INFO: BL3-1: Next image address = 0x8000000000
INFO: BL3-1: Next image spsr = 0x3c9
Boot firmware (version built at 15:52:31 on May 7 2015)

I don't know which exact version of the firmware that is.

For now I have the GIC cell demo running on the board, just in case I didn't get the crash out of luck... so far so good, I think I will leave it for a few hours.

Could you provide me also with your current kernel version and config?

[Jan Kiszka]

The actual version is printed on the screen after this one, but maybe
Mazda can already derive the version from the build date.

>
> I don't know which exact version of the firmware that is.
>
> For now I have the GIC cell demo running on the board, just in case I didn't get the crash out of luck... so far so good, I think I will leave it for a few hours.

For me it's enough to just enable Jailhouse without creating any cells.
After some minutes, sometimes more, I get the lock-up.

FATAL: exception unhandled trap
Cell state before exception:

pc: 0000000022c86764 lr: 0000000022c80270 spsr: 200001c5 EL1
sp: fffffe001bc5fb80 esr: 17 1 0000000

x0: 00000000c200ff00 x1: 00000083ff322018 x2: 000000000000000c
x3: 0000000000000000 x4: 0000000000000000 x5: 0000000000000000
x6: 0000000000000000 x7: 0000000000000000 x8: ffff7f7f7f7f7f7f

x9: fffffe0000796000 x10: 0101010101010101 x11: 0000000000000020
x12: fffffe001de294a8 x13: 0000000000000107 x14: fffffdfee006fae0
x15: fffffe0019eb0000 x16: 0000000000000006 x17: 0000000000000001
x18: 0000000000000001 x19: fffffe0000db91d8 x20: fffffe0000db9000
x21: fffffe001bc5fcb0 x22: 0000000000000140 x23: 0000000000000000
x24: 0000000000000000 x25: 0000000000000000 x26: fffffe0000c55fb1
x27: 0000000000000000 x28: fffffe001a83d300 x29: fffffe001bc5fb90

Parking CPU 6 (Cell: "AMD Seattle")

/root/jailhouse/hypervisor/arch/arm64/control.c:271: returning error -EINVAL

>

> Could you provide me also with your current kernel version and config?

I'm now on v4.5 (exactly). Config is attached.

[Sabony, Mazda]

Hi Antonios,
You can find the firmware version in the Bios scree. Just press SEC ode DEL when you boot and you will be in the Bios menu.

Thanks
Best Regards
Mazda

-----Ursprüngliche Nachricht-----
Von: Jan Kiszka [mailto:jan.k...@siemens.com]
Gesendet: Mittwoch, 23. März 2016 22:02
An: Antonios Motakis <antonios...@huawei.com>; jailho...@googlegroups.com; Sabony, Mazda <Mazda....@amd.com>
Cc: claudio...@huawei.com; jani.k...@huawei.com; dmitry...@huawei.com; veacesla...@huawei.com; jean-phili...@arm.com; marc.z...@arm.com; edgar.i...@xilinx.com; wuqi...@huawei.com
Betreff: Re: [PATCH v8 00/28] Jailhouse port to ARM AArch64 processors

[Antonios Motakis]

Now I have been running Jailhouse here for a day, however with the GIC demo cell running.
When you say without creating any cells, do you mean that creating a cell is not needed to trigger the lockup, or that it triggers only when there is no cell running? Does it make sense for me to try again with no cells created?

I wonder if patch 06 of the inmate series (which is a temporary hack, a work around the current cell configuration format limitations) is causing any issues. Maybe you could try to undo this patch and try again to load Jailhouse?

> FATAL: exception unhandled trap
> Cell state before exception:
> pc: 0000000022c86764 lr: 0000000022c80270 spsr: 200001c5 EL1
> sp: fffffe001bc5fb80 esr: 17 1 0000000
> x0: 00000000c200ff00 x1: 00000083ff322018 x2: 000000000000000c
> x3: 0000000000000000 x4: 0000000000000000 x5: 0000000000000000
> x6: 0000000000000000 x7: 0000000000000000 x8: ffff7f7f7f7f7f7f
> x9: fffffe0000796000 x10: 0101010101010101 x11: 0000000000000020
> x12: fffffe001de294a8 x13: 0000000000000107 x14: fffffdfee006fae0
> x15: fffffe0019eb0000 x16: 0000000000000006 x17: 0000000000000001
> x18: 0000000000000001 x19: fffffe0000db91d8 x20: fffffe0000db9000
> x21: fffffe001bc5fcb0 x22: 0000000000000140 x23: 0000000000000000
> x24: 0000000000000000 x25: 0000000000000000 x26: fffffe0000c55fb1
> x27: 0000000000000000 x28: fffffe001a83d300 x29: fffffe001bc5fb90
>
> Parking CPU 6 (Cell: "AMD Seattle")
> /root/jailhouse/hypervisor/arch/arm64/control.c:271: returning error -EINVAL
>
>>
>> Could you provide me also with your current kernel version and config?
>
> I'm now on v4.5 (exactly). Config is attached.
>
> Jan
>

--

[Antonios Motakis]

Hello,

I have this BIOS information in the BIOS screen:

BIOS Information
BIOS Vendor American Megatrends
Core Version 5.011
Compliancy UEFI 2.4; PI 1.3
Project Version ROD0081B 0.00
Build Date and Time 05/07/2015 15:50:15

Best regards,
Antonios

[Sabony, Mazda]

Thanks Antonios,
Hi Jan,
Attached you will find the ROD0081B firmware version and Here is the instructions to upgrade BIOS:

/* Overdrive BIOS upgrade */

1) unzip firmware the to your Windows PC and it should have a BIOS file (xxx.ROM) and an EFI file (SsiFlashARM64.EFI)

2) Boot up Linux on the Overdrive

# cd /boot/efi
# mkdir BIOS
# ifconfig /* find out the IP address */

3) On your Windows PC, use WinSCP with the IP address (download and install free software WinSCP if needed) to copy the xxx.ROM and SsiFlashARM64.EFI from Windows to Linux (/boot/efi/BIOS/)

4) Reboot the Linux and stop it at EFI Shell

Shell> FS0:
Shell> cd BIOS
Shell> SsiFlashARM64.EFI xxx.ROM /U

After upgrading the BIOS, you also need to configure CPU frequency to 2000MHz.

Thanks
Best Regards
Mazda

-----Ursprüngliche Nachricht-----

Von: Antonios Motakis [mailto:antonios...@huawei.com]
Gesendet: Donnerstag, 24. März 2016 13:01
An: Sabony, Mazda <Mazda....@amd.com>; Jan Kiszka <jan.k...@siemens.com>; jailho...@googlegroups.com

Cc: claudio...@huawei.com; jani.k...@huawei.com; dmitry...@huawei.com; veacesla...@huawei.com; jean-phili...@arm.com; marc.z...@arm.com; edgar.i...@xilinx.com; wuqi...@huawei.com

Betreff: Re: AW: [PATCH v8 00/28] Jailhouse port to ARM AArch64 processors

[Jan Kiszka]

On 2016-03-24 13:29, Sabony, Mazda wrote:
> Thanks Antonios,
> Hi Jan,
> Attached you will find the ROD0081B firmware version and Here is the instructions to upgrade BIOS:
>
> /* Overdrive BIOS upgrade */
>
> 1) unzip firmware the to your Windows PC and it should have a BIOS file (xxx.ROM) and an EFI file (SsiFlashARM64.EFI)
>
> 2) Boot up Linux on the Overdrive
>
> # cd /boot/efi
> # mkdir BIOS
> # ifconfig /* find out the IP address */
>
> 3) On your Windows PC, use WinSCP with the IP address (download and install free software WinSCP if needed) to copy the xxx.ROM and SsiFlashARM64.EFI from Windows to Linux (/boot/efi/BIOS/)
>
> 4) Reboot the Linux and stop it at EFI Shell
>
> Shell> FS0:
> Shell> cd BIOS
> Shell> SsiFlashARM64.EFI xxx.ROM /U
>
> After upgrading the BIOS, you also need to configure CPU frequency to 2000MHz.

Seems like I bricked the device this way: no more output on the uart
even after minutes, apparently no boot-up of system. Maybe it's not
powering up (although the power switch is bridged), can't check that
before Tuesday (remote access only).

Jan

[Jan Kiszka]

I'm getting the crash both with only the root cell and with an
additional cell being in use. So that should not help to trigger.

>
> I wonder if patch 06 of the inmate series (which is a temporary hack, a work around the current cell configuration format limitations) is causing any issues. Maybe you could try to undo this patch and try again to load Jailhouse?

I've tried with the previously working code base as well (IIRC, current
wip/arm64 on github, but I can no longer confirm the exact hash now),
and I saw the same crashes. That makes me believe it's at least firmware
induced, if not caused.

Jan

[Jan Kiszka]

To follow-up on this: We have a working setup again with firmware
version WOD6120X (01/18/2016). However, crashes are back as well,
latencies tend to rise above 60 µs, too - with the old firmware that
should have been on the board previously, with the old kernel (v4.4) and
the old Jailhouse revision (x-mas edition).

Maybe debugging that issue is now required (if I had more time...).
However, as a start, I captured the exception class value: 0x17. Is that
ESR_EC_SMC64? A call into monitor mode?

FATAL: exception unhandled trap (exception class 0x17)

Cell state before exception:
pc: 0000000042c86764 lr: 0000000042c80270 spsr: 200001c5 EL1

sp: fffffe000519fb80 esr: 17 1 0000000

x0: 00000000c200ff00 x1: 00000083ff322018 x2: 000000000000000c
x3: 0000000000000000 x4: 0000000000000000 x5: 0000000000000000
x6: 0000000000000000 x7: 0000000000000000 x8: ffff7f7f7f7f7f7f
x9: fffffe0000762000 x10: 0101010101010101 x11: 0000000000000020

x12: fffffe0000dd04a8 x13: 0000000000000052 x14: fffffdff0026d620
x15: fffffe0099580000 x16: 0000000000000006 x17: 0000000000000001
x18: fffffffffffffe09 x19: fffffe0000d675d0 x20: fffffe0000d67000
x21: fffffe000519fcb0 x22: 0000000000000140 x23: fffffe00bfecf900

x24: 0000000000000000 x25: 0000000000000000 x26: 0000000000000000

x27: fffffe0000b30000 x28: 0000000000000000 x29: fffffe000519fb90

The caller is in userspace? Does that make any sense? Any magic daemon
to kill that could trigger this?

[Jan Kiszka]

EL1 = kernel space. OK, but pc looks userspace'ish. If this was supposed
to be a PSCI service call, x0 should have something like c4xxxxxx - but
it comes as c2xxxxxx. So the SMC is rejected. Are we just executing junk
in the root cell? Some spontaneous problem with the memory mappings?

If I look at 32-bit ARM, it bluntly forwards unrecognised SMCs - tried
that, but it just confirmed that the root cell is already corrupted and
can't recover from this state.

Good news: I've a working non-root Linux cell now (thanks Tony!).

Let's cross fingers that the crash doesn't happen too often - or only on
specific boards...?

[Antonios Motakis]

Yes indeed... Could the newer Seattle firmware include some extra monitor calls for some functionality?
Quite interesting.

Any chance you have ended up with a firmware using PSCI 0.1? In that case the function IDs can differ, and are set in the device tree. (we got PSCI 0.2 over here though, even if only partially).

>
> If I look at 32-bit ARM, it bluntly forwards unrecognised SMCs - tried
> that, but it just confirmed that the root cell is already corrupted and
> can't recover from this state.

Yeah, and I'm not a fan of this approach. I would rather abort on unrecognized SMCs.

>
> Good news: I've a working non-root Linux cell now (thanks Tony!).
>
> Let's cross fingers that the crash doesn't happen too often - or only on
> specific boards...?

I'm glad that you got an inmate working - however when I return to China I definitely need to upgrade my firmware to match yours, along with kernel versions etc. I will be happier if I manage to reproduce the problem so I can look into it.

>
> Jan

[Jan Kiszka]

I suspect that the kernel is eventually branching into junk or gets junk
mapped in. And then it hits that SMC call. On the other hand, it's
always the same junk.

>
> Any chance you have ended up with a firmware using PSCI 0.1? In that case the function IDs can differ, and are set in the device tree. (we got PSCI 0.2 over here though, even if only partially).

# cat /sys/firmware/devicetree/base/psci/compatible
arm,psci-0.2

Nope.

>
>>
>> If I look at 32-bit ARM, it bluntly forwards unrecognised SMCs - tried
>> that, but it just confirmed that the root cell is already corrupted and
>> can't recover from this state.
>
> Yeah, and I'm not a fan of this approach. I would rather abort on unrecognized SMCs.

I will try to make the guest kernel crash "properly" (set pc to 0). That
used to help on x86 to get back traces and possibly pointers to the
origin of some trap.

>
>>
>> Good news: I've a working non-root Linux cell now (thanks Tony!).
>>
>> Let's cross fingers that the crash doesn't happen too often - or only on
>> specific boards...?
>
> I'm glad that you got an inmate working - however when I return to China I definitely need to upgrade my firmware to match yours, along with kernel versions etc. I will be happier if I manage to reproduce the problem so I can look into it.

Yeah, that will be great. I'm not overly optimistic to resolve the issue
quickly (unless it vanishes magically with the different hardware I'll
have in San Diego).

[Jan Kiszka]

Yeah, worked:

[ 696.293174] [< (null)>] (null)
[ 696.297866] [<0000000042c3fe44>] 0x42c3fe44
[ 696.302036] [<0000000042c3d564>] 0x42c3d564
[ 696.306210] [<fffffe000058ab8c>] virt_efi_set_time+0x5c/0xa0
[ 696.311858] [<fffffe0000564994>] efi_set_time+0x74/0x90
[ 696.317071] [<fffffe00005624dc>] rtc_set_time+0x6c/0x118
[ 696.322371] [<fffffe0000561524>] rtc_set_ntp_time+0x6c/0xb8
[ 696.327931] [<fffffe000011db40>] sync_cmos_clock+0x90/0x178
[ 696.333492] [<fffffe00000c75d4>] process_one_work+0x16c/0x400
[ 696.339225] [<fffffe00000c7994>] worker_thread+0x12c/0x4b8
[ 696.344698] [<fffffe00000ce1cc>] kthread+0xe4/0xf8
[ 696.349478] [<fffffe0000093c10>] ret_from_fork+0x10/0x40

We are deep in some EFI callbacks to set the system time, aren't we?

For now, I'll try to kill ntpd and see if pigs can fly then.

[Jan Kiszka]

Yep, they fly high and don't crash on the ground any more. EFI is a bit
frightening.

Latency peaks remain, though. Just saw 69 µs. Maybe power management -
different topic.

[Antonios Motakis]

Hello,

I found that our Fedora installation here by default uses Chrony instead of NTPD. I switched to NTPD, now I'm running a Linux inmate Cell and I'm waiting to see if something moves...

However if there's a firmware related issue triggered by NTPD in your side, it would probably(-ish) trigger with Chrony as well from our side... Let's wait and see. Maybe trying a firmware upgrade here would be next.

Cheers,
Tony

>
> Latency peaks remain, though. Just saw 69 µs. Maybe power management -
> different topic.
>
> Jan
>

--

[Jan Kiszka]

Also check if you have CONFIG_EFI_RTC enabled. Disabling it also
resolves the issue because the kernel no longer calls into that firmware
path.

[Antonios Motakis]

Hah, indeed! It is disabled on the kernel I have currently booted. This is probably it. I'll try again with it enabled, and I'll report back.

[Antonios Motakis]

On 18-Apr-16 17:54, Jan Kiszka wrote:

Actually CONFIG_EFI_RTC depends on IA64 over here... do you mean CONFIG_RTC_DRV_EFI?

Cheers,
Tony

[Jan Kiszka]

Yeah, I do. Don't have the actual target config in reach, so I only
guessed ;)

[Peng Fan]

Hi Antonios, Jan

Sorry for breaking this thread to a question.

I am interested in jailhouse on AArch64, and would like to try
it on FVP.

Do you have simple guide how to run jailhouse on FVP?
Do you use the kernel source from linaro aarch64 or else?

Thanks,
Peng.

>>>>>>>>>> latencies tend to rise above 60 盜, too - with the old firmware that

>--
>You received this message because you are subscribed to the Google Groups "Jailhouse" group.
>To unsubscribe from this group and stop receiving emails from it, send an email to jailhouse-de...@googlegroups.com.
>For more options, visit https://groups.google.com/d/optout.

[Antonios Motakis]

On 20-Apr-16 16:06, Peng Fan wrote:
> Hi Antonios, Jan
>
> Sorry for breaking this thread to a question.
>
> I am interested in jailhouse on AArch64, and would like to try
> it on FVP.
>
> Do you have simple guide how to run jailhouse on FVP?
> Do you use the kernel source from linaro aarch64 or else?
>

Hello Peng,

The series includes a root cell configuration for FVP models (configs/foundation-v8.c). There is no guide for the models specifically, but running it is not that different than the other targets.

Generally I use upstream kernels. However, you are going to need PSCI 0.2 support. I used the upstream boot-wrapper with these PSCI patches in order to boot Linux on the model. http://lists.infradead.org/pipermail/linux-arm-kernel/2015-January/315545.html

The I built Jailhouse against my kernel, and install against the rootfs I use with the Fast Model. So after getting a working Linux kernel and user land on FastModels, using Jailhouse on that is quite straightforward.

Cheers,
Antonios

>>>>>>>>>>> latencies tend to rise above 60 µs, too - with the old firmware that

[Antonios Motakis]

So now I finally have a working system again (Debian based this time), and a build of the exact same configuration/version of the kernel that you used. I am running Jailhouse on it now, (with ntpd in the background), and I'm waiting to see if something moves this time. Fingers crossed...

Cheers

[Antonios Motakis]

Well, the thing has been running overnight, and is still stable... latency between min 2495 ns and max 3535 ns.

ntpd running in the background... I wonder if enabling some obscure configuration of ntpd does the trick, but I doubt it.

[Jan Kiszka]

Very strange, not only the missing calls into the monitor (did you check
that the EFI RTC driver is stressed at all?), but also that low
latencies. As soon as I start a kernel build, e.g., they jump to 60 or
more µs here as well as on the demo system I can in San Diego.

[Antonios Motakis]

Not so far, but yeah I still need to check what happens with the RTC driver.

However, I have now crashes when disabling Jailhouse, so there's at least one thing I can debug now!

[Antonios Motakis]

Hello,

I think Jailhouse disable crashes when the root cell kernel is using 64K pages only. That is a very good clue, so I should be able to find the root cause quickly now, at least for this issue.

Cheers,