Cap'n Proto v0.3: Python support, better tools, new features

[Kenton Varda]

Hi capnproto-announce,

(permalink)

The first release of Cap'n Proto came three months after the project was announced. The second release came six weeks after that. And the third release is three weeks later. If the pattern holds, there will be an infinite number of releases before the end of this month.

Version 0.3 is not a paradigm-shifting release, but rather a slew of new features largely made possible by building on the rewritten compiler from the last release. Let's go through the list…

Python Support!

Thanks to the tireless efforts of contributor Jason Paryani, I can now comfortably claim that Cap'n Proto supports multiple languages. His Python implementation wraps the C++ library and exposes most of its features in a nice, easy-to-use way.

And I have to say, it's way better than the old Python Protobuf implementation that I helped put together at Google. Here's why:

• Jason‘s implementation parses Cap’n Proto schema files at runtime. There is no need to run a compiler to generate code every time you update your schema, as with protobufs. So, you get to use Python the way Python was intended to be used. In fact, he's hooked into the Python import mechanism, so you can basically import a .capnp schema file as if it were a .py module. It‘s even convenient to load schema files and play with Cap’n Proto messages from the interactive interpreter prompt.
• It's fast. Whereas the Python Protobuf implementation — which we made the mistake of implementing in pure-Python — is slow. And while technically there is an experimental C-extension-based Python Protobuf implementation (which isn‘t enabled by default due to various obscure problems), Jason’s Cap'n Proto implementation is faster than that, too.

Go check it out!

By the way, there is also a budding Erlang implementation (by Andreas Stenius), and work continues on Rust (David Renshaw) and Ruby (Charles Strahan) implementations.

Tools: Cap'n Proto on the Command Line

The capnp command-line tool previously served mostly to generate code, via the capnp compile command. It now additionally supports converting encoded Cap'n Proto messages to a human-readable text format via capnp decode, and converting that format back to binary with capnp encode. These tools are, of course, critical for debugging.

You can also use the new capnp eval command to do something interesting: given a schema file and the name of a constant defined therein, it will print out the value of that constant, or optionally encode it to binary. This is more interesting than it sounds because the schema language supports variable substitution in the definitions of these constants. This means you can build a large structure by importing smaller bits from many different files. This may make it convenient to use Cap'n Proto schemas as a config format: define your service configuration as a constant in a schema file, importing bits specific to each client from other files that those clients submit to you. Use capnp eval to “compile” the whole thing to binary for deployment. (This has always been a common use case for Protobuf text format, which doesn't even support variable substitution or imports.)

Anyway, check out the full documentation for more.

New Features

The core product has been updated as well:

• Support for unnamed unions reduces the need for noise-words, improving code readability. Additionally, the syntax for unions has been simplified by removing the unnecessary ordinal number.
• Groups pair nicely with unions.
• Constants are now implemented in C++. Additionally, they can now be defined in terms of other constants (variable substitution), as described earlier.
• The schema API and schema.capnp have been radically refactored, in particular to take advantage of the new union and group features, making the code more readable.
• More and better tests, bug fixes, etc.

Users!

Some news originating outside of the project itself:

• Debian Unstable (sid) now features a Cap'n Proto package, thanks to Tom Lee. Of course, since package updates take some time, this package is still v0.2.1 as of this writing, but it will be updated to v0.3 soon enough.
• Popular OSX-based text editor TextMate now uses Cap'n Proto internally, and the developer's feedback lead directly to several usability improvements included in this release.
• Many people using Cap'n Proto haven't bothered to tell us about it! Please, if you use it, let us know about your experience, both what you like and especially what you don‘t like. This is the critical time where the system is usable but can still be changed if it’s not right, so your feedback is critical to our long-term success.
• I have revenue! A whopping $1.25 per week! >_> It's totally worth it; I love this project. (But thanks for the tips!)

-Kenton