go-capnproto RPC support available

[Ross Light]

Hi all,

I'm excited to announce a Cap'n Proto package for Go that supports the RPC protocol.  This comes along with a thorough overhaul of the runtime library's innards to make it leaner, more Go idiomatic, and (hopefully) easier to understand.  My fork has been available for a while, but I have waited to make a more general announcement until after I was comfortable enough to make an API compatibility guarantee.

For those familiar with the existing Go package, this is a fork from Jason's package (glycerine/go-capnproto), but it is not API-compatible.  Consult the repo news and the docs for more details on how to migrate.

Please take a look at github.com/zombiezen/go-capnproto.  If you encounter any issues, please report them on the bug tracker.

Thank you!

-Ross

[Asheesh Laroia]

Thanks for this, Ross!

I was talking with a friend who has been using go's built-in rpc system, and I was wondering if you could write a page somewhere indicating some similarities and differences between https://golang.org/pkg/net/rpc/ and go-capnproto, or perhaps write a code sample similar to what's on https://golang.org/pkg/net/rpc/ but written for go-capnproto instead.

I can think of some differences, but since I'm not personally a "Golang person", I don't know that I have the context to explain the advantages of go-capnproto to a "Golang person". You probably do, though!

If you can't, that's OK too.

I say this because the first thing that jumped to my mind when I clicked on https://github.com/zombiezen/go-capnproto was that I was hoping to see some concise sample code.

I love that https://godoc.org/zombiezen.com/go/capnproto dives straight into what go-capnproto does under the hood ("depth first"). However, my role in the Cap'n Proto ecosystem is as much cheerleader as implementation reader/writer, and so my reaction here is, "Give me something I can send to my friend Paul so I can convince him to drop go-rpc and use this instead," and so I am hoping for some crisp readable sample server+client code in the first page of scroll, somewhere, that shows how nice and usable this is.

Cheers, and many thanks for writing and sharing this!

-- Asheesh.

--
You received this message because you are subscribed to the Google Groups "Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+...@googlegroups.com.
Visit this group at http://groups.google.com/group/capnproto.

[Ross Light]

Excellent point!  I opened up a new issue for improving the docs.  In the meantime, here's what this means for Gophers:

The rpc subpackage in go-capnproto is the analogue to the net/rpc package and includes an example in the docs.  Unfortunately, godoc.org doesn't give me sufficient control to make the example expanded by default.  I'll try to either write some GitHub pages or a wiki page that's more beginner-friendly.

The advantages I see over net/rpc:

• go-capnproto's wire protocol (Cap'n Proto) is language-agnostic.
• encoding/gob (the package that net/rpc is based on) is limited to in-memory data structures and doesn't support random access.  Hypothetically, you could write an implementation of Arena that dynamically loads in segments from a huge message.
• "Infinitely less" decoding time.
• Promise pipelining and the advantages that brings.
• No reflection overhead; all code generation.
• Support for context/deadline propagation.

Disadvantages over net/rpc:

• Code generation makes your build process somewhat more complicated.  The `go generate` command alleviates this.
• Doesn't support being served over an HTTP endpoint out-of-the-box.  Pull requests for such a Transport implementation are welcome. :)

This is great feedback Asheesh, thank you!

-Ross

[Kenton Varda]

Hi Ross,

Sorry for the delay in replying; email backlog.

Would you like me to add this to the capnproto.org "other languages" list now?

I think "object capabilities" should be on your list of advantages. :)

-Kenton

[Ross Light]

Yes!  The API is now stable.

I have written up a more well-formatted version of the advantages, which does include capabilities: https://github.com/zombiezen/go-capnproto/wiki/Comparison-with-stdlib-rpc

-Ross

[Kenton Varda]

OK, link added!

We now have two Go links. Do you (Ross and Jason Aten) have any thoughts on how this should eventually be resolved? On one hand it seems like new projects should use Ross's version so that they can use RPC, on the other hand a lot of existing code uses Jason's version and I guess there are some breaking API changes. Maybe Ross's version should be considered "2.0" in semver style?

-Kenton

[J]

//edited: reposting without my email signature.

Nice work Ross, congratulations!

I think the go-capnproto-2.0 label is very appropriate for your work, given the significance of rpm and capabilities to users.

Moreover it would help me clarify the dependencies to users when describing tools such as my go-based capnproto schema generator bambam (at github.com/glycerine/bambam) that depend on features in go-capnproto-1.0 that were added after your branch.

Again, great work!

Jason

[Ross Light]

Marking it as 2.0 seems reasonable, but I'm unsure of how to go about it.  Unlike other languages like Node or Python, there isn't a central registry of the "canonical" form of a package in Go.  The import path is the identifier.  The two packages are are already distinct for dependency calculations because their import paths are different.  Should the distinction be made more explicit?

Option 0: Keep the import paths as-is:

Jason's: github.com/glycerine/go-capnproto

Mine: zombiezen.com/go/capnproto

Option 1a: Suffix a version number

Jason's: github.com/glycerine/go-capnproto1

Mine: zombiezen.com/go/capnproto2

Option 1b: Use gopkg.in to do the same (requires a repo merge):

Jason's: gopkg.in/capnproto.v1 ...which corresponds to branch v1 of github.com/go-capnproto/capnproto

Mine: gopkg.in/capnproto.v2 ...which corresponds to branch v2 of github.com/go-capnproto/capnproto

Option 1a/b does seem to disambiguate it in the short term, but is that distinction worth it long-term?  A newcomer to Cap'n Proto may be confused as to which to use or why there's a v2 (especially if a new maintainer comes along in the 1a case).  If go-capnproto was to be submitted into the main repo, then the import path could be:

Option 2: Use the main Cap'n Proto repo:

github.com/sandstorm-io/capnproto/go/

  capnp

  rpc

  ...

I tend to favor options 0 or 2.  Since this issue is likely to come up for other language bindings, it would be good to develop some general guidelines.  Thoughts?

-Ross

[Jason E. Aten]

My suggestion would be to have Ross' new version 2 protocol clearly distinguished in the final import path segment, with a final 2, like:

 zombiezen.com/go/go-capnproto2

I'll put a note on my readme that points to zombiezen (I think there's one already there actually). The current import path for github.com/glycerine/go-capnproto needs to remain the same for backwards compatibility purposes.

[Kamal Marhubi]

Regarding naming and repos, would it make sense at some point to migrate Cap'n Proto and associated projects to a capnproto org on GitHub?

[Kenton Varda]

On Thu, Sep 17, 2015 at 2:08 PM, Kamal Marhubi <ka...@marhubi.com> wrote:

Regarding naming and repos, would it make sense at some point to migrate Cap'n Proto and associated projects to a capnproto org on GitHub?

I think the current arrangement does a good job of communicating that each implementation is owned by the implementer. If we were to create an organization, I would want there to be more oversight and review involved, since I think users would assume that all projects in the org are "endorsed" by me or by Sandstorm.io. I think it would be a very good thing to have such a setup, but at present we don't have the resources for it. Maybe in a year or two when Sandstorm is larger.

-Kenton

[Ross Light]

SGTM. Based on this discussion, I created a new repository at github.com/zombiezen/go-capnproto2 and updated the README of the old repository to reference it.  All existing users should update their import path to "zombiezen.com/go/capnproto2".

-Ross