Message has too many segments

[Irwin Billing]

Hi everyone,

Enjoying this library.  Today I ran into this issue: "requirement not met: expected segmentCount < 512; Message has too many segments".

The source says the check is in place for security reasons.

https://github.com/kentonv/capnproto/blob/f536c8cfbc749f98e43b679242b24afbab8e74b2/c%2B%2B/src/capnp/serialize.c%2B%2B#L149

What are these security reasons and what causes high segment counts?  It would help to know to figure out the best way to work around (or with) it.

Thanks!

[David Renshaw]

Hi Irwin,

One security concern is that we allocate `moreSizes` on the stack, with size is proportional to `segmentCount`. If we were to accept any `uint32_t` value for `segmentCount`, then a malicious sender could trivially blow through the receiver's stack.

Segments get allocated as needed when you build a message. If you are using `MallocMessageBuilder` and setting `allocationStrategy` to `SUGGESTED_ALLOCATION_STRATEGY`, i.e. `GROW_HEURISTICALLY`, then the segment size grows exponentially, and you ought to run out of memory long before you use 512 segments.

Perhaps you are accidentally using the `FIXED_SIZE` allocation strategy?

Or perhaps you are trying to read a packed message and you forgot to use `PackedInputStream`?

- David

--
You received this message because you are subscribed to the Google Groups "Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+...@googlegroups.com.
Visit this group at http://groups.google.com/group/capnproto.

[Kenton Varda]

David is correct. It is almost certainly the case that some sort of corruption has happened here, as it's pretty hard to actually produce a message with over 512 segments.

That said, we should probably replace the stack allocation with KJ_STACK_ARRAY so that it reverts to heap for larger arrays, and then up the limit to something like 65536. It occurs to me that the 512 limit could actually affect Sandstorm's packaging format, which embeds each file as its own segment after mmaping them. We are probably breaking this limit already, but escaping trouble because we read by mmap() rather than streaming...

-Kenton

[Irwin Billing]

Hey guys,

Thanks for the assistance!  Gave me some guidance for where to look for the issue.

Post mortem:  Turns out in my case, it was the caused by corruption.  I was reading from a FD in a bad state.

Thanks again!

Irwin

—
Sent from Mailbox