Cap'N Proto for hardware (FPGA/ASIC)

[John Demme]

Hi Guys-

I'm thinking about implementing Cap'n Proto for hardware. I believe the lower encoding/decoding makes this viable for acceleration communication. There are, however, certain aspects of capnproto which are non-optimal for hardware. It would be nearly perfect with some additions to the type system:

• A non-nullable, inline struct-like type. Basically, a C struct.
• While CPUs are excellent at pointer dereferencing and manipulation, that hardware is relatively expensive and very often unnecessary. Ideally, for any fixed-size message, the individual field offsets would be statically computable -- which they are except for structs. Groups don't cut it since users want to reuse types. Elevating groups to a first-class citizen would also work.
• Also, there are applications where data is and used before the message is entirely read to prevent requiring large buffers. Structs sitting after that message's inline (non-pointer) data would prevent those applications from using them or require potentially large re-ordering buffers.
• C-style arrays
• Fixed-lenth arrays are quite common in hardware and it is, again, very useful to be able to statically compute the offsets from the beginning of the containing struct. The array length will be a capnp constant.
• I know this is likely a non-starter, but generic types support for non-pointer types and constants
• To support the two above features

I realize that I could implement this via annotations and then require a custom transform or encoding of the messages, but that would have a performance and/or hardware area impact. I really want to just DMA messages out of host memory. I could also realize this with a custom c++ backend, but I'd like all the languages to eventually get this support. Imagine talking Cap'n Proto RPC directly to hardware from any language!

Is there any interest in adding these features to the spec? Are there complications of which I'm not aware? Am I misunderstanding something fundamental about Cap'n Proto?

~John

[Kenton Varda]

Hi John,

Inline structs and arrays have come up a few times. Here's the most recent discussion: https://github.com/capnproto/capnproto/issues/907

And an older, much longer discussion: https://github.com/capnproto/capnproto/issues/205

In short, it seems like we're nearing consensus on how it should be done, but there are still worries about code complexity introduced (an earlier, different approach failed due to over-complexity).

Do you envision designing protocols that don't use pointers at all? How would you handle data that really is variable-width?

-Kenton

--
You received this message because you are subscribed to the Google Groups "Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/e9bf07d3-bfe3-42ac-8171-3b11b1e23073%40googlegroups.com.

[John Demme]

Hi Kenton-

That's excellent... I still haven't had the time to wrap my head around those two discussions at more than a superficial level, but I may have some input when I do.

RE pointerless protocols: Yes, at first this is the only type I plan on supporting. My plan is to use capnp structs, but use an annotation indicating they should be inline. It should just be a simple matter of inserting a schema transform between capnp and the codegen plugin to convert that struct to a group. That would destroy version compatibility, so it'll also change the message IDs deterministically. I think it's reasonable to require the same version on both the PCIe attached device and the host software... at least for the time being.  Eventually, I'll want to support the capnp wire format especially over the network for RPC purposes. This is where inline structs being standard would be nice so people have the option to avoid the re-ordering buffer overhead with protocol optimizations.

RE variably-sized data: Any large fixed or variably-sized data can be treated as a stream. In general, in hardware one has to consider both the temporal and spatial dimensions, which leads to off-by-one bugs in both dimensions! Lotsa fun to debug.

~John

To unsubscribe from this group and stop receiving emails from it, send an email to capn...@googlegroups.com.