wireshark dissector for capnp RPC

[Andreas Stenius]

Hi all,

I know I'm not alone, trying to wrap my head around the capnp RPC logic.

The calculator example is excellent and cool in every way, but fails to show
the lower level workings of the protocol. And frankly, looking at hexdumps
doesn't help much (and saving it in a format to use `capnp decode` on it not
so much either).

So, I've put a few days aside and implemented a little lua dissector for
wireshark to give a live, browsable view of any RPC message captured on your
nic's.

I think it's pretty awesome, any who :p

Without further ado, here's a link to the source code at github:
https://github.com/kaos/wireshark-plugins.

Have fun!

//Andreas

Ps.
Performance has not been a prio at this point. That said, I haven't
experienced any issues with the limited testing I've done based on the
calculator example.
Also note, that there are a few missing features (and probably bugs too) in
the message dissection..

[Kenton Varda]

Cool! Should I add a link to the web site?

In cases where you know the schema, maybe it makes sense to write the content in Cap'n Proto text format?

With a little bookkeeping, you should be able to determine the types of request and response messages, in order to print them better. When you see a "Call" message, you can look up the request type using the interface ID and method number. If you also make note of those and associate them with the question ID, then on "Return" you will also no the response type. You can use the C++ SchemaParser API to parse a bunch of user-provided schema files upfront in order to have a database of interface definitions for this purpose. (And you can use the C++ pretty-print API to print things while you're at it.)

FWIW, the RPC unit test has a really crappy "RpcDumper" class that I used to analyze traffic while testing. Might be useful to look at:

  https://github.com/kentonv/capnproto/blob/master/c++/src/capnp/rpc-test.c++

-Kenton

[Kenton Varda]

Oh I see, the output is designed to produce a tree view in wireshark, hence why it's not Cap'n Proto text format. Duh.

[Andreas Stenius]

(cc'ing the list again, I missed the reply all button..)

Regarding the RpcDumper, I think it could be neat to provide one entry in the
tree view with that version too, as it's much more compact, and gives you
quick summary of what it's all about, while the more detailed tree can be
handy if there's anything odd you want to take a closer look at.

On Thursday 08 May 2014 08.53.54 you wrote:
> Bah, forgot the link:
>
> [1] https://github.com/calio/lua-capnproto/issues/1
>
> On Thursday 08 May 2014 08.52.26 you wrote:
> > On Wednesday 07 May 2014 23.47.47 you wrote:
> > > Oh, are you not using the Lua implementation of Cap'n Proto?
> >
> > I tried to, but got stuck on issues with it [1].
> >
> > But I've also come to realize that pretty much the same goes here as with
> > the C++ api's. My needs in this case are a bit special..
> >
> > Also, I'm not sure how I would make wireshark use luajit rather than (the
> > built in) lua it was compiled with, to make it work in runtime. Perhaps it
> > would work to use it to compile the schemas to lua.. but I'm not sure the
> > schemas even work without also using luajit, so it may not even work, in
> > this case.
> >
> > > On Wed, May 7, 2014 at 10:56 PM, Andreas Stenius
> > >
> > > <andreas...@astekk.se>wrote:
> > > > Yeah, it was easy to convert a text dump of a CodeGeneratorRequest
> > > > into
> > > > lua code.
> > > > Den 7 maj 2014 23:10 skrev "Kenton Varda" <temp...@gmail.com>:
> > > >
> > > > On Wed, May 7, 2014 at 1:52 PM, Andreas Stenius
> > > > <andreas...@astekk.se
> > > >
> > > >> > wrote:
> > > >>> Also, for the long term, it really should be implemented in C++ (I
> > > >>> wanted
> > > >>> something working without the need to mess with rebuilding wireshark
> > > >>> from
> > > >>> source, so went with a Lua version for now). However, I don't know
> > > >>> the
> > > >>> API's
> > > >>> that well. Is it possible to determine the byte offsets that the
> > > >>> various
> > > >>> data
> > > >>> comes from when you get a parsed result out?
> > > >>> That's needed to get a nice connection between the tree view data
> > > >>> and
> > > >>> the hex
> > > >>> dump in wireshark.
> > > >>
> > > >> In theory, it would not be hard to extend the API to provide that,
> > > >> but
> > > >> it
> > > >> might also not add much over what you're doing currently. I think the
> > > >> main
> > > >> useful thing you could get out of the C++ API is parsing the schemas.
> > > >> You
> > > >> could also have people parse schemas into a schema database
> > > >> separately
> > > >> (that is, a file containing a collection of compiled schema nodes),
> > > >> and
> > > >> then load that in, which would avoid the need to call C++ code at all
> > > >> in
> > > >> the actual plugin. Perhaps it makes sense for the capnp tool to
> > > >> provide
> > > >> a
> > > >> standard "schema database" format for this purpose. (Right now you
> > > >> could
> > > >> hack something based on the -o/bin/cat trick.)
> > > >>
> > > >> -Kenton

[Andreas Stenius]

OK, I got inspired..

Kenton, how about this?

``
Cap'n Proto RPC Protocol: restore(0) objectId="calculator\0"
Cap'n Proto RPC Protocol: call(1) (promisedAnswer=(0,
[]))::10923537602090224694->method(0) (capTable=[], content=()) return to:
caller, tail call: false
Cap'n Proto RPC Protocol: call(2) (promisedAnswer=(1,
[getPointerField=0]))::14116142932258867410->method(0) (capTable=[],
content=()) return to: caller, tail call: false
Cap'n Proto RPC Protocol: return(0) releaseParamCaps=true
results(capTable=[(senderHosted=0)], content=cap(0))
Cap'n Proto RPC Protocol: return(1) releaseParamCaps=false
results(capTable=[(senderHosted=1)], content=())
Cap'n Proto RPC Protocol: finish(0) releaseResultCaps=false
Cap'n Proto RPC Protocol: return(2) releaseParamCaps=false
results(capTable=[], content=())
Cap'n Proto RPC Protocol: finish(2) releaseResultCaps=false
Cap'n Proto RPC Protocol: finish(1) releaseResultCaps=false
Cap'n Proto RPC Protocol: release(1) referenceCount=1
``

The "only" thing missing now, is the ability to hook in the interface schemas
so we can parse the methods, params and results too..

//Andreas

[Kenton Varda]

Neat!

It looks like some of your formatting is based on my RpcDumper code for rpc-test. I bet the formatting could be improved to make things more readable. E.g. the question/answer IDs could be moved to the front of the line, and there could be some indicator that distinguishes the direction of the question (client -> server calls vs. server -> client calls) so that it's easy to match up a call with its return and not get it confused with completely unrelated IDs going the other way. The metadata could be less verbose, too, e.g. not writing out common identifiers like "releaseResultCaps" over and over again.

That's all just tweaks, though. Good job on the hard part. :)

-Kenton

[Andreas Stenius]

On Thursday 08 May 2014 19.24.42 Kenton Varda wrote:
> Neat!
>
> It looks like some of your formatting is based on my RpcDumper code for
> rpc-test.
> I bet the formatting could be improved to make things more
> readable.

Indeed it is. I tried to mimick it as far as possible, to ease the need for me
to think about it :p.

> E.g. the question/answer IDs could be moved to the front of the
> line, and there could be some indicator that distinguishes the direction of
> the question (client -> server calls vs. server -> client calls) so that
> it's easy to match up a call with its return and not get it confused with
> completely unrelated IDs going the other way. The metadata could be less
> verbose, too, e.g. not writing out common identifiers like
> "releaseResultCaps" over and over again.

I added the directional markers yesterday (figured the same):
https://github.com/kaos/wireshark-plugins/commit/4f09ada2a2c68925854d6b14249f8597b1dea214

I can drop the verbose release caps info.. perhaps leave such detailed
summaries on a secondary line (so you still don't need to dig through the
entire tree structure to find it).

>
> That's all just tweaks, though. Good job on the hard part. :)

Yeah, and it's real easy to tweak now too, as I have entry points on a type-
by-type basis for the summary text.
https://github.com/kaos/wireshark-plugins/blob/efdd4b4dca8fa4e17a2771d7c7b8657816421301/plugins/capnp.lua#L486

Thanks for the feedback!

I hope people will find good use of this little plugin :)

//Andreas