Pigweed Live Notes: 2023-09-11

[chadn...@google.com]

2023-09-11

Landed recently highlights (full changelog)

• SEED 0107: Pigweed Communications has been approved! Pigweed will adopt a new sockets API as its primary networking abstraction. The sockets API will be backed by a new, lightweight embedded-focused network protocol stack inspired by TCP/IP.
• SEED 0108: Emulators Frontend has also been approved!
• Bazel: You can now implement your own pw_assert backends without producing cyclic dependencies (CL).
• pw_log: All upstream modules are now compatible with Abseil logging (b/299520256).
• pw_web: log viewer has a “download logs as .txt” option alongside various bug fixes.

Coming soon

• SEED for Bazel toolchain API
• SEED for build system strategy
• Complete rework of Pigweed’s sample project to follow updated best practices
• New pw_emu (emulator frontend) module, along with Renode and QEMU integrations
• Bazel: customizable in-repo configuration for Arm GCC

Open SEEDs

• 0103: pw_protobuf: Past, present, and future - CL - Rendered Doc [LAST CALL]
• 0104: Display Support - CL - Rendered Doc [LAST CALL]
• 0105: Nested Tokens and Tokenized Args - CL - Rendered Doc
• 0106: Project Template - CL - Rendered Doc
• 0109: Communication Buffers - CL - Rendered Doc
• 0110: Memory Allocation Interfaces - CL - Rendered Doc

Agenda

SEED-0110: Memory Allocation Interfaces - CL - Rendered Doc -

Alexei Frolov

• Projects tend to pre-allocate more memory than they need when dynamic allocation can’t be used
• We want to providing a generic interface for memory allocation that can be implemented by both upstream and downstream libraries
• This proposal is based on our experience with using Pigweed in commercial products
• Proposes a C malloc-style allocator and a C++17-style allocator interface
• They include the concept of equality, where you can generically allocate/deallocate from equal allocators
• Why not polymorphic resources (PMR), which this is very similar to?
• We would strongly prefer to use PMR and/or be very similar/compatible with PMR, but it just doesn’t work in our applications right now, e.g., PMR needs exceptions to signal failures
• This proposal also adds desirable features like efficient reallocations
• The generic allocator provides no formal guarantees, but proposed subclasses (semantic interfaces) can signal support for certain guarantees (ThreadSafeAllocator, InterruptSafeAllocator, etc.). In practice, these are just aliases for the generic allocator which does support those features as an implementation detail.
• Properties like “thread safe” and “interrupt safe” are not mutually exclusive, but the subclasses are. What if you want both properties?
• STL does this via allocator traits; probably need to think of doing something similar
• These interfaces are intended to be used at the lowest level; “application” code should use the generic interfaces
• Layered allocators: they sit on top of other allocators. Example: PoolAllocator manages a memory pool allocated from some other allocator. ObjectAllocator builds on this: allocates and deallocates on construction/destruction.
• We’ve had requests from product teams to have a way to manage multiple memory pools with a single handle. The ChainedAllocatorPool proposes a solution. Includes fallback, example: if you have a kernel pool and a user pool, and you fail to allocate from the kernel pool, you can fall back to the user pool.
• Tagged allocator: allocator includes a 32-bit integer as metadata, allowing tracking allocations. Intriguingly, pw_tokenizer could be used to granularly and precisely track the source of allocations via a macro that tags allocators with tokenized strings (function name, file and line number, etc.)
• Should the allocators return spans instead of raw pointers? Having a definitive size avoids many common errors, and allocators may return a different amount of memory than requested (e.g., rounding up).
• Question comes down to: should we create a familiar interface or a safe interface?
• Similarly, could return something like a result type instead of a null pointer
• The current proposal puts STL/new/delete integration out of scope, so it’s proposing more of a conventional C-style API
• Need a section in the SEED that describes what to do if you just want to use new

Sent from document [PUBLIC] Pigweed Live Notes