[PATCH 0/2] aarch64: boot memory fixups

[Stewart Hildebrand]

I'd appreciate some review and testing. Do I have the correct understanding
about the OSv mmu subsystem? I may have inadvertently made some assumptions, so
please check my work. I marked the 2nd patch RFC since the fix wastes quite a
lot of RAM, and I'm wondering if there's a better way. I've tested on
LynxSecure hypervisor, and qemu-system-aarch64 -machine virt. I have not tested
this on firecracker or Xen.

Stewart Hildebrand (2):
aarch64: fixup system RAM range
aarch64: map the DTB range

arch/aarch64/arch-setup.cc | 13 +++++++++++--
1 file changed, 11 insertions(+), 2 deletions(-)

--
2.30.0

[Stewart Hildebrand]

The arguments to the free_initial_memory_range() call specify a memory range
beyond the end of system RAM. Subtract the reserved boot memory size from the
physical memory size so we don't exceed the end of physical RAM.

Signed-off-by: Stewart Hildebrand <stewart.h...@dornerworks.com>
---
arch/aarch64/arch-setup.cc | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/arch/aarch64/arch-setup.cc b/arch/aarch64/arch-setup.cc
index 30fb0db6..6aff9580 100644
--- a/arch/aarch64/arch-setup.cc
+++ b/arch/aarch64/arch-setup.cc
@@ -81,7 +81,8 @@ void arch_setup_free_memory()
extern void *elf_start;

mmu::phys addr = (mmu::phys)elf_start + elf_size;
- mmu::free_initial_memory_range(addr, memory::phys_mem_size);
+ size_t reserved_size = addr - mmu::mem_addr;
+ mmu::free_initial_memory_range(addr, memory::phys_mem_size - reserved_size);

/* linear_map [TTBR1] */
for (auto&& area : mmu::identity_mapped_areas) {
--
2.30.0

[Stewart Hildebrand]

I'm trying to parse some additional values from the device tree, and I'm running
into some issues. If I understand the logic in arch_setup_free_memory()
correctly [1], it reserves a region of RAM for the ELF, and makes the rest
available for use by the memory allocator / mmu subsystem.

If the bootloader happens to locate the DTB after the elf, it would not get
mapped in TTBR0 and we would no longer be able to address it via the global dtb
pointer. Further, it would risk getting overwritten by new mapped memory.

Add a check for this scenario, and reserve all memory up until the end of either
the dtb or elf, whichever address is higher.

On the qemu virt model, start of RAM is 0x40000000, and qemu places the dtb at
0x48000000, so unfortunately this wastes ~128MB of precious lowmem. I'm not very
familiar with the OSv mmu subsystem - does OSv have a way to reserve regions of
memory and guarantee that those pages won't be given away by a potential future
malloc?

As I was writing this, I just now realized that OSv parses the relevant DTB
values and stores them in known locations for later use. Is this the preferred
way to parse the DTB? If so, I can develop a patch assigning NULL to the global
DTB pointer when we switch to the runtime tables, along with a comment to
explain this.

[1] https://github.com/cloudius-systems/osv/blob/master/arch/aarch64/arch-setup.cc#L75

Signed-off-by: Stewart Hildebrand <stewart.h...@dornerworks.com>
---

arch/aarch64/arch-setup.cc | 10 +++++++++-
1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/arch/aarch64/arch-setup.cc b/arch/aarch64/arch-setup.cc
index 6aff9580..29b704f0 100644
--- a/arch/aarch64/arch-setup.cc
+++ b/arch/aarch64/arch-setup.cc
@@ -29,6 +29,12 @@

#include <alloca.h>

+extern "C" {
+#include "libfdt.h"
+}
+
+#define MAX(a,b) ((a)>(b) ? (a) : (b))
+
void setup_temporary_phys_map()
{
// duplicate 1:1 mapping into phys_mem
@@ -80,7 +86,9 @@ void arch_setup_free_memory()
extern size_t elf_size;
extern void *elf_start;

- mmu::phys addr = (mmu::phys)elf_start + elf_size;
+ mmu::phys elf_end_addr = (mmu::phys)elf_start + elf_size;
+ mmu::phys dtb_end_addr = (mmu::phys)dtb + fdt_totalsize(dtb);
+ mmu::phys addr = MAX(elf_end_addr, dtb_end_addr);

size_t reserved_size = addr - mmu::mem_addr;

mmu::free_initial_memory_range(addr, memory::phys_mem_size - reserved_size);

--
2.30.0

[Stewart Hildebrand]

>-----Original Message-----
>From: Stewart Hildebrand <Stewart.H...@dornerworks.com>
>Sent: Wednesday, February 17, 2021 11:19 AM
>To: OSv Development <osv...@googlegroups.com>
>Cc: Stewart Hildebrand <Stewart.H...@dornerworks.com>; Waldek Kozaczuk <jwkoz...@gmail.com>
>Subject: [PATCH 1/2] aarch64: fixup system RAM range
>
>The arguments to the free_initial_memory_range() call specify a memory range
>beyond the end of system RAM. Subtract the reserved boot memory size from the
>physical memory size so we don't exceed the end of physical RAM.

I just realized that this is actually already taken care of in dtb_setup() [1]. I only ran into the issue once I started mapping the DTB in patch 2. So this patch is clearly not right, please disregard.

[1] https://github.com/cloudius-systems/osv/blob/master/arch/aarch64/arch-dtb.cc#L715

[Waldek Kozaczuk]

I wonder if this wiki might help a bit - https://github.com/cloudius-systems/osv/wiki/Memory-Management.

Also regarding DTB, you might find this commit relevant to what you are trying to do - https://github.com/cloudius-systems/osv/commit/69a0ce39eb7c5af76bd2c714939c02ceb7ea398f