Stubby RPC

[tsuna]

Hi,
at OSCON Rob Pike said that the Go implementation of Stubby RPC was
probably the easiest to open-source. That would be really awesome :)

I think pretty much all the Google Go team attended OSCON, that was
pretty cool. Thanks for coming.

--
Benoit "tsuna" Sigoure
Software Engineer @ www.StumbleUpon.com

[Rob 'Commander' Pike]

On Aug 5, 2011, at 4:55 PM, tsuna wrote:

> Hi,
> at OSCON Rob Pike said that the Go implementation of Stubby RPC was
> probably the easiest to open-source. That would be really awesome :)

Easiest but not easy. That was not a promise.

-rob

[mattn]

> Hi,
> at OSCON Rob Pike said that the Go implementation of Stubby RPC was
> probably the easiest to open-source.  That would be really awesome :)

Easiest but not easy.  That was not a promise.

Just finished to write this code.

https://gist.github.com/1127348

[Russ Cox]

> at OSCON Rob Pike said that the Go implementation of Stubby RPC was
> probably the easiest to open-source.  That would be really awesome :)
>
> I think pretty much all the Google Go team attended OSCON, that was
> pretty cool.  Thanks for coming.

If you are interested in RPC in the context of Go alone,
an alternative that is available today is Go's own
rpc package: http://golang.org/pkg/rpc/. It uses a
Go-specific encoding called gob instead of protocol
buffers, but it provides the same functionality.
(Again the drawback is that it is Go-only, not cross
language like protobufs, but that's not too much
different from having only the Go Stubby available.)

Russ

[mattn]

Ah, probably, I have misunderstanding about "Stubby RPC".

Sorry, turn off talking this thread, What is "Stubby RPC"?

[André Moraes]

MessagePack has a RPC protocol,
with a considerable number of bindings for others languages.

http://msgpack.org/

Are any know drawbacks when using MessagePack RPC?

--
André Moraes
http://andredevchannel.blogspot.com/

[David Symonds]

On Fri, Aug 5, 2011 at 10:13 PM, André Moraes <and...@gmail.com> wrote:

> Are any know drawbacks when using MessagePack RPC?

MessagePack is dynamically typed, schemaless, has patchy documentation
and highly uneven implementation completeness across languages. If it
works for you, great, but you might want to do some thorough research
first.

Dave.

[mattn]

Golang implements in MessagePack does not work. The code uses old manipulating of reflect package.

[Brian Ketelsen]

My fork on GitHub works as of R58. Bketelsen on github

Sent from my iPhone

[mattn]

My fork on GitHub works as of R58.  Bketelsen on github

 

Awesome

 

[Brad Fitzpatrick]

I was going to try to wrestle the microphone away from DiBona to point out that making an RPC system for protocol buffers isn't that hard.

Stubby is probably 90% legacy stuff, and that's what makes it hard to open source.  There's some good stuff in it, but it's nothing that can't be replicated.

You can just speak protocol buffer requests/responses over HTTP or Gearman and get most of the way there.  Just don't think Google is sitting on too much secret sauce and allow that to intimidate you from just writing it.

[Russ Cox]

On Fri, Aug 5, 2011 at 10:46, Brad Fitzpatrick <brad...@golang.org> wrote:
> I was going to try to wrestle the microphone away from DiBona to point out
> that making an RPC system for protocol buffers isn't that hard.

And it's nearly trivial in Go: you could just implement
rpc.ClientCodec and rpc.ServerCodec using protobufs.

Russ

[tsuna]

On Fri, Aug 5, 2011 at 7:46 AM, Brad Fitzpatrick <brad...@golang.org> wrote:
> I was going to try to wrestle the microphone away from DiBona to point out
> that making an RPC system for protocol buffers isn't that hard.
> Stubby is probably 90% legacy stuff, and that's what makes it hard to open
> source.  There's some good stuff in it, but it's nothing that can't be
> replicated.
> You can just speak protocol buffer requests/responses over HTTP or Gearman

HTTP isn't appropriate because of the inefficiency of the protocol.
One of its biggest shortcomings is the lack of concurrent requests.
HTTP pipelining still doesn't allow out of order responses. Also the
encoding used is very inefficient. I think it's rather use SPDY.

> and get most of the way there.  Just don't think Google is sitting on too
> much secret sauce and allow that to intimidate you from just writing it.

Disclaimer: I worked at Google 2 years (08-09) on some of the largest
serving systems there, so I'm pretty familiar with Stubby RPC.

There's more to Stubby RPC than just wrapping a ProtoBuf into another
one and looking up the corresponding method to invoke on the other
side. There's how you manage connections, timeouts, load balancing
strategies, stats, etc. I consider this to be part of the core of
Stubby (unlike Dapper, the authentication mechanism, etc).

I know there was no promise to open-source anything, but there was
definitely interest on the Google side, and interest in the community.
A lot of people have picked up ProtoBufs – thanks for releasing that
– but they're missing a good RPC system. There are no good
alternatives right now in the open-source world. Someone already
mentioned some of MessagePack's shortcomings. Thrift's code quality
is also ... questionable, let's say, and for many years they had no
proper async RPC support (dunno what it looks like these days, I sort
of gave up on Thrift).

Stubby RPC has become a de facto standard within Google, and may as
well end up being adopted as a de facto standard outside of Google
too.

[Rob 'Commander' Pike]

That's a bit of a leap. I agree it would be nice to have an RPC system everybody can share, but Stubby wouldn't be my starting point. Its long history makes it complicated and not the design I'd start with if I were building from scratch. Ditto protobufs, in fact, as I've said before (http://blog.golang.org/2011/03/gobs-of-data.html); they're designed for and work best in C++, for one thing.

I plan to write the rpc interface code so people can use Go's rpc package with protobuf transport. Maybe that'll scratch your itch.

-rob