Use with RTOS-based Embedded Systems

[Kenny Koller]

Hi,

My team and I were hoping to use Cap'n P on our system to pass messages around via UDP. There is a powerful Linux system running a control system but the nodes are STM32F7 devices (1 MB flash, 320 kB of RAM, 216 MHz Cortex-M7 processor) running FreeRTOS with the LWIP networking stack.

When I tried to build for the embedded system I ran into some errors related to a dependency on pthreads (in the KJ library I believe).

I did some searches and saw discussions in this group from 2015-16 that suggested things like inheriting from MessageBuilder to get around using malloc or using nanopb instead.

I don't fully understand what CAPNP_LITE gets me so maybe this would be helpful.

What is the thinking today on this topic?

Thanks,

Kenny

[Kenton Varda]

Hi Kenny,

The code is fairly modular, so I think it should be possible to pull out a subset that has a reasonably small code footprint. However, this is admittedly not a well-traveled path, so you'll be a bit on your own. The maintainers of the C++ implementation (such as myself) mainly use it on big beefy servers, so we haven't had a whole lot of reason to work on optimizing code footprint.

CAPNP_LITE might be a good starting point. Note that that option was not really designed to minimize code footprint, but rather was designed to identify a subset of code that could be compiled on MSVC circa 2013. MSVC has since improved, and is now able to compile the whole codebase. Hence, CAPNP_LITE is a bit vestigial now and may have bitrotted somewhat. But, it definitely does cut out a lot of bulkier parts of the library, which you'd also want to remove in an embedded system. So it might be a good guide.

Another option that might be less work would be to use c-capnproto, which I think is more explicitly designed for constrained use cases.

-Kenton

--
You received this message because you are subscribed to the Google Groups "Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/81988491-bcde-4025-b9b8-74f9a06c6eebn%40googlegroups.com.

[Kenny Koller]

Thank you Kenton.

I certainly understand that the "big server" use-case is far more common (embedded software engineers everywhere are weeping in their Cap'n Crunch**) and I appreciate the clarification regarding CAPNP_LITE.

Personally, I find that code space is much less of a concern these days within reason (though it shouldn't be completely ignored). Many of these microcontrollers have quite a bit of flash. This is obviously not true for very inexpensive devices but there are plenty of applications where the cost of the processor is negligible relative to the device as a whole.

The more pressing issue is avoiding (possible) memory fragmentation when using new or malloc. C++ has been my language for choice for my embedded development for more than 20 years. I suspect, if I had time, I could have looked into overriding some classes to provide my own chunks of memory. Just having say 16 256 byte chunks that I could recycle would probably do the trick.

Thanks again,

Kenny

**With crunch berries because we're adults.

[Kenton Varda]

Hi Kenny,

Avoiding malloc should be easy. With custom subclasses of MessageReader and MessageBuilder, you can do your memory management any way you want. Cap'n Proto's serialization layer is very good about avoiding memory allocation unless you ask for it.

(The RPC layer, on the other hand, does lots of dynamic allocation, and there's no way around that. But, it sounds like you're interested in the serialization only.)

-Kenton

[Kenny Koller]

That sounds promising!

[Mikhail Natalenko]

Hi Kenny,

Sorry to bother you. I'm looking for a similar solution, could you share your experience?

Kind regards,

Mikhail

четверг, 10 июня 2021 г. в 02:47:30 UTC+4, Kenny Koller:

[Keir Mierle]

I realize this is a Cap'n Proto list, but I thought I'd send this along in case it's useful. My team does microcontroller projects and ended up building a custom protocol buffer library just for embedded projects due to the code size issue. It is challenging balancing the tradeoffs on these systems, especially when both RAM and code size are tight.

You can see our protobuf system here:

https://pigweed.dev/pw_protobuf/ <-- note the size report; try clicking the down arrow by "FLASH"

And our RPC system that that is often paired with it (though you can use whatever serialization you want):

https://pigweed.dev/pw_rpc/

Fun fact: protocol buffer wire format, when directly manipulated, is both RAM and code size efficient (though not time efficient). pw_protobuf enables directly interacting with the wire format, which several of our customers leverage for minimal RAM and code impact.

To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/3f25dd54-9452-4949-92c4-94661519d350n%40googlegroups.com.