Binary Protocol and Loadbalancing

47 views
Skip to first unread message

TomTom

unread,
Nov 30, 2009, 9:21:14 AM11/30/09
to Scribe Server
Hey Fellows,
One of my back of the napkin designs for Scribe was to stick a modern
F5 load balancer in front the aggregation servers, and then use the
F5's irules to switch the contexts and send them to the right server.

I was looking at the protocol the other day in Wireshark. It looks to
be all binary data. I couldn't find a text context in the stream.

My questions are:
1) Was I looking at the thrift TTransport data?
2) And if so, how much different from "normal" TCP is it. The
thrift incubator website, doesn't have much to grok on it. Do you guys
know any other sources? I unfortunately can't read C++. Perhaps I
should hire a CS student to read through the code and explain to me
how it works.
3) I also read in the thrift documentation that a single non-blocking
TCP socket/thread can eat 200GB. How in the world is this possible? I
have never heard of a single app server that can do this. Do you guys
know of any in the wild?
Is it the boost code. Are those libraries so pimp, that you can do
what I used to think was impossible? Or is the non-blocking binary-
data eating 200GB magic somehow implemented in the Thrift libraries.

If anyone knows I'd greatly appreciate the explanation. Either way,
the boost, thrift, and scribe guys are doing some great CS work.
Thanks again for OS them and and giving it out to the community.

I hope everyone had a great Thanksgiving.
-Tom
Reply all
Reply to author
Forward
0 new messages