finagle (and util and ostrich) on scala 2.9.1

Showing 1-25 of 25 messages
finagle (and util and ostrich) on scala 2.9.1 marius a. eriksen 12/5/11 2:31 PM
we have a first port of the twitter core scala libraries to scala 2.9.1. they're published with a "_2.9.1" suffix. we'd love testers. these all unit- and regression-test but we have yet to run them in production.

the relevant versions are:

        util-*_2.9.1                1.12.8
        ostrich_2.9.1                4.10.1
        finagle-*_2.9.1        1.9.11

cheers,
        marius.

Re: finagle (and util and ostrich) on scala 2.9.1 Durgesh Mankekar 12/5/11 2:48 PM
This is excellent news, Marius and team. We have tried our own patched versions in QA with no noticeable issues. Exciting to see first official release for 2.9.1. Thanks again.

-Durgesh
Re: finagle (and util and ostrich) on scala 2.9.1 tsuna 12/5/11 3:08 PM

+1, thank you guys.  I was also eager to get on the 2.9 train.

--
Benoit "tsuna" Sigoure
Software Engineer @ www.StumbleUpon.com

Re: finagle (and util and ostrich) on scala 2.9.1 tsuna 12/6/11 12:28 AM
On Mon, Dec 5, 2011 at 3:08 PM, tsuna <tsun...@gmail.com> wrote:
> On Mon, Dec 5, 2011 at 2:48 PM, Durgesh Mankekar <dur...@gmail.com> wrote:
>> This is excellent news, Marius and team. We have tried our own patched
>> versions in QA with no noticeable issues. Exciting to see first official
>> release for 2.9.1. Thanks again.
>
> +1, thank you guys.  I was also eager to get on the 2.9 train.

With Berk's help, I'm now running Finagle (core, http, memcached)
1.9.11 with Scala 2.9.1 in production.  I'm testing it with about 800
QPS for now, and so far everything is fine.  My latency profile is
identical to my other instances (1.9.7 / Scala 2.8.1) with a 99th
percentile around 10ms.

I'll report back if I find any issue.

--
Benoit "tsuna" Sigoure
Software Engineer @ www.StumbleUpon.com

Re: finagle (and util and ostrich) on scala 2.9.1 tsuna 12/6/11 9:11 AM
On Tue, Dec 6, 2011 at 12:28 AM, tsuna <tsun...@gmail.com> wrote:
> With Berk's help, I'm now running Finagle (core, http, memcached)
> 1.9.11 with Scala 2.9.1 in production.  I'm testing it with about 800
> QPS for now, and so far everything is fine.  My latency profile is
> identical to my other instances (1.9.7 / Scala 2.8.1) with a 99th
> percentile around 10ms.
>
> I'll report back if I find any issue.

Now in production with 1400 QPS.  The server has been up since my
email was sent.  We collect various statistics every few seconds
(including the average / median / stddev / 95th and 99th percentile of
latency).  I analyzed a few thousand data points from OpenTSDB and
found no statistically significant difference at this time.  The new
server has a worst 99th percentile somewhat more frequently than
others, I am not sure yet that the differences are significant (we're
talking about 9ms more on the 99th percentile), especially because the
median and the 95th are almost identical.

So I'd say we're happy with it :)

--
Benoit "tsuna" Sigoure
Software Engineer @ www.StumbleUpon.com

Re: finagle (and util and ostrich) on scala 2.9.1 marius a. eriksen 12/6/11 9:56 AM
out of curiosity - what do you guys use finagle for (presumably at StumbleUpon)? do you use it with java or scala, external facing service or internal?

        marius.

Re: finagle (and util and ostrich) on scala 2.9.1 tsuna 12/6/11 10:22 AM
On Tue, Dec 6, 2011 at 9:56 AM, marius a. eriksen <mar...@twitter.com> wrote:
> out of curiosity - what do you guys use finagle for (presumably at StumbleUpon)? do you use it with java or scala, external facing service or internal?

We use it in 2 of our user-facing services at StumbleUpon.  We use it
with Scala, of course :)

The service I'm talking about uses a memcache cluster, several MySQL
databases and an HBase cluster.  For HBase we use asynchbase as it's
fully asynchronous / non-blocking, and very easy to integrate with
Finagle.  For MySQL I wrote my own connection pooling with an
asynchronous query interface, after giving up on Querulous because I
couldn't stop it from leaking connections [1].  It's a very simple
single-class thing where I have a FuturePool and each thread owns one
MySQL JDBC connection, and it works like a charm.

We serve thousands of QPS with a 99th percentile generally below 20ms
and >99.95% availability.

I've used Netty extensively in OpenTSDB, asynchbase and some other
internal projects at StumbleUpon, and I've been rather happy with it.
Finagle is a really nice Scala layer on top of it.  I recommended it
to other groups at StumbleUpon who might be interested in jumping on
the Scala boat.  So thank you guys for writing it and sharing it with
the rest of the world.


  [1] https://github.com/nkallen/querulous/issues/24

--
Benoit "tsuna" Sigoure
Software Engineer @ www.StumbleUpon.com

Re: finagle (and util and ostrich) on scala 2.9.1 Steve Jenson 12/6/11 10:27 AM
On Tue, Dec 6, 2011 at 10:22 AM, tsuna <tsun...@gmail.com> wrote:
> On Tue, Dec 6, 2011 at 9:56 AM, marius a. eriksen <mar...@twitter.com> wrote:
>> out of curiosity - what do you guys use finagle for (presumably at StumbleUpon)? do you use it with java or scala, external facing service or internal?
>
> We use it in 2 of our user-facing services at StumbleUpon.  We use it
> with Scala, of course :)
>
> The service I'm talking about uses a memcache cluster, several MySQL
> databases and an HBase cluster.  For HBase we use asynchbase as it's
> fully asynchronous / non-blocking, and very easy to integrate with
> Finagle.  For MySQL I wrote my own connection pooling with an
> asynchronous query interface, after giving up on Querulous because I
> couldn't stop it from leaking connections [1].

I believe I fixed this querulous issue a few weeks ago and we're using
it in production without seeing connection leaks or Timer issues.

Thanks,
Steve

Re: finagle (and util and ostrich) on scala 2.9.1 tsuna 12/6/11 10:35 AM
On Tue, Dec 6, 2011 at 10:27 AM, Steve Jenson <ste...@twitter.com> wrote:
> I believe I fixed this querulous issue a few weeks ago and we're using
> it in production without seeing connection leaks or Timer issues.

OK, I'd be curious to see the fix.  I spent several hours (!)
debugging this issue and navigating all the layers of Querulous, and
from I could tell it was properly calling "close()" on each DBCP
connection after each query.  Yet for some reason DBCP kept creating
new connections all the time.  I probably did something wrong/stupid
somewhere (as linked from the issue above, my code at the time is
available on https://gist.github.com/1344664).

I liked being able to use Querulous, but I generally found it hard to
use, undocumented, and having too many layers (in addition of DBCP /
JDBC).  My little pooling / asynchronous layer is only 288 SLOC and
doesn't bring in DBCP or other dependencies than just the dreaded JDBC
stuff.

Anyways, overall we're pretty happy with Finagle and its ecosystem.
You guys did a good job and made our life much easier to adopt Scala
in some of our projects.  So thank you.

--
Benoit "tsuna" Sigoure
Software Engineer @ www.StumbleUpon.com

Re: finagle (and util and ostrich) on scala 2.9.1 Steve Jenson 12/6/11 10:43 AM
On Tue, Dec 6, 2011 at 10:35 AM, tsuna <tsun...@gmail.com> wrote:
> On Tue, Dec 6, 2011 at 10:27 AM, Steve Jenson <ste...@twitter.com> wrote:
>> I believe I fixed this querulous issue a few weeks ago and we're using
>> it in production without seeing connection leaks or Timer issues.
>
> OK, I'd be curious to see the fix.  I spent several hours (!)
> debugging this issue and navigating all the layers of Querulous, and
> from I could tell it was properly calling "close()" on each DBCP
> connection after each query.  Yet for some reason DBCP kept creating
> new connections all the time.  I probably did something wrong/stupid
> somewhere (as linked from the issue above, my code at the time is
> available on https://gist.github.com/1344664).

I updated the ticket with the specific fixes.

> I liked being able to use Querulous, but I generally found it hard to
> use, undocumented, and having too many layers (in addition of DBCP /
> JDBC).  My little pooling / asynchronous layer is only 288 SLOC and
> doesn't bring in DBCP or other dependencies than just the dreaded JDBC
> stuff.

Is it on github?

Thanks,
Steve

Re: finagle (and util and ostrich) on scala 2.9.1 tsuna 12/6/11 10:47 AM
On Tue, Dec 6, 2011 at 10:43 AM, Steve Jenson <ste...@twitter.com> wrote:
> I updated the ticket with the specific fixes.

Thanks.

> Is it on github?

No but I could put it up there.  Maybe you guys would be interested it
in having it as part of Finagle?

--
Benoit "tsuna" Sigoure
Software Engineer @ www.StumbleUpon.com

Re: finagle (and util and ostrich) on scala 2.9.1 Berk D. Demir 12/6/11 11:49 AM


On Tue, Dec 6, 2011 at 9:56 AM, marius a. eriksen <mar...@twitter.com> wrote:
out of curiosity - what do you guys use finagle for (presumably at StumbleUpon)? do you use it with java or scala, external facing service or internal?


The other project is a content addressable storage built on top of HBase, where the RPC is ReSTful HTTP.
We use it to store our BLOBs such as site thumbnails, user pictures, assets (Javascript, CSS, Flash files, etc).
Juggling a modest multi terra bytes of data and enjoying natural deduplication thanks to cryptographic hash function as the content key.

With a Varnish frontend, it is our CDN origin server. With heavy caching on Varnish and by the CDN Edge servers, traffic is not much of challenge unlike the project Benoît described but when we were aggressively populating the store at initial roll out, gigabit links were saturated with both upstream and downstream traffic without sweating the JVM that much.

It has only two dependencies. Finagle (core+http+ostrich4) and Asynchronous HBase Client (http://github.com/stumbleupon/asynchbase)

Before Finagle, we had different versions of it but never made it to production because it always suffered from multiple issues; either code bloat or unpleasant performance.

In previous trials/versions of this project, non-JVM implementations used Thrift to speak to HBase but it consistently performed unsatisfactory due to the nature of Thrift gateway implementation of HBase. Java + Netty ones were too verbose and needed to implement many things like monitoring, service composition and the resulting code was large, yet still not complete. Intimidating to any new comer because of the paradigm shift introduced by Netty (pipelines, channels, codecs, oh my!). Did I mention it was criminally verbose? Oh yeah! I did.

There were also other mini trials with some actors frameworks. Took me a bit to understand why these overhyped thingies were quite the wrong approach to my rather small problem.

Thanks to Finagle (and twitter-util) many of the heavy lifting is done behind the scenes through a beautiful API. It is predictable and source is actually readable. Showing the source code of Finagle projects to complete Scala newbies is always good idea™ to pitch Scala. It's just a very well designed framework to build async, high performance RPC systems on JVM. Reading the source just feels right and best of all there's no magic involved (hello RoR!).

So; Marius, Steve and other Finagle/twitter-util contributors, you made our lives easier with your creations and gave a great reason to masses to learn and adopt Scala. (...and they are learning it from your Scala School)

Thank you so much!
Re: finagle (and util and ostrich) on scala 2.9.1 tsuna 12/11/11 1:49 PM
Just thought I'd follow up here.  The service I was referring to has
been serving 100% of its traffic using Scala 2.9.1 with no issues.
The median latency is a tad bit better (600µs vs 700µs on 2.8) and I
believe this is due to the upgrade, because the median latency is
generally representative of how much time I need to serve a request
out of my in-memory LRU caches (so no other backend calls are required
– which generally add a lot of variance).

So from now on we'll run Scala 2.9 everywhere in production.  Thanks guys.

--
Benoit "tsuna" Sigoure
Software Engineer @ www.StumbleUpon.com

Re: finagle (and util and ostrich) on scala 2.9.1 Vaughan 12/22/11 8:11 PM
I am receiving the following unresolved dependency message:

[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] ::          UNRESOLVED DEPENDENCIES         ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: org.apache.thrift#libthrift;0.5.0: not found
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] ::              FAILED DOWNLOADS            ::
[warn] :: ^ see resolution messages for details  ^ ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: javax.jms#jms;1.1!jms.jar
[warn] :: com.sun.jdmk#jmxtools;1.2.1!jmxtools.jar
[warn] :: com.sun.jmx#jmxri;1.2.1!jmxri.jar
[warn] ::::::::::::::::::::::::::::::::::::::::::::::

http://maven.twttr.com/thrift/libthrift/0.5.0/ does not contain a POM for libthrift which is causing the failure.

The javax dep is search for here: http://repo1.maven.org/maven2/javax/jms/jms/1.1/ but this directory does not contain the jar.

Has anyone else experienced these problems?
Re: finagle (and util and ostrich) on scala 2.9.1 John Sirois 12/22/11 8:24 PM


On Thu, Dec 22, 2011 at 9:11 PM, Vaughan <vrou...@gmail.com> wrote:
I am receiving the following unresolved dependency message:

[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] ::          UNRESOLVED DEPENDENCIES         ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: org.apache.thrift#libthrift;0.5.0: not found
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] ::              FAILED DOWNLOADS            ::
[warn] :: ^ see resolution messages for details  ^ ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: javax.jms#jms;1.1!jms.jar
[warn] :: com.sun.jdmk#jmxtools;1.2.1!jmxtools.jar
[warn] :: com.sun.jmx#jmxri;1.2.1!jmxri.jar
[warn] ::::::::::::::::::::::::::::::::::::::::::::::

http://maven.twttr.com/thrift/libthrift/0.5.0/ does not contain a POM for libthrift which is causing the failure.

Added a pom - you should be good to go there.
 

The javax dep is search for here: http://repo1.maven.org/maven2/javax/jms/jms/1.1/ but this directory does not contain the jar.

We have hit the 3 failed downloads - these are deps pulled in by log4j's pom.  You'll need to add excludes via your project's managed dependency mechanism.


Has anyone else experienced these problems?



--
John Sirois
303-512-3301





Re: finagle (and util and ostrich) on scala 2.9.1 Vaughan 12/22/11 8:33 PM
Thanks for adding the pom.

I have resolved it by locating the missing deps manually and adding this to my sbt libraryDependencies:

Re: finagle (and util and ostrich) on scala 2.9.1 tsuna 3/29/12 5:22 PM

I just Gisted the code as some other people requested it:
https://gist.github.com/2245176

I don't know if this is something Finagle would be interested in
integrating to help people use clusters of MySQL servers in their
Finagle apps.  Ideally it would be better to have a MySQL client
written with Finagle, to be asynchronous from the ground up and avoid
dealing with all the JDBC bullshit.

--
Benoit "tsuna" Sigoure
Software Engineer @ www.StumbleUpon.com

Re: finagle (and util and ostrich) on scala 2.9.1 marius a. eriksen 3/30/12 8:51 AM
https://github.com/twitter/twitter.github.com/wiki/Google-Summer-of-Code-2012

MySQL codec

        * Brief explanation: Implement the MySQL protocol as a codec in finagle.
        * Knowledge Prerequisite: Scala, familiarity with asynchronous IO, network protocols.
        * Mentor: Marius Eriksen (@marius) and Blake Matheny @bmatheny)

@stevegury also has a prototype that I believe he's going to be putting on github.

        marius.

Re: finagle (and util and ostrich) on scala 2.9.1 tsuna 3/30/12 9:00 AM
On Fri, Mar 30, 2012 at 8:51 AM, marius a. eriksen <mar...@twitter.com> wrote:
> https://github.com/twitter/twitter.github.com/wiki/Google-Summer-of-Code-2012
>
> MySQL codec
>
>        * Brief explanation: Implement the MySQL protocol as a codec in finagle.
>        * Knowledge Prerequisite: Scala, familiarity with asynchronous IO, network protocols.
>        * Mentor: Marius Eriksen (@marius) and Blake Matheny @bmatheny)
>
> @stevegury also has a prototype that I believe he's going to be putting on github.

Looking forward to this awesome addition to Finagle!

--
Benoit "tsuna" Sigoure
Software Engineer @ www.StumbleUpon.com

Re: finagle (and util and ostrich) on scala 2.9.1 Steve Gury 3/30/12 10:30 AM
I just pushed my MySQL codec on my github account https://github.com/stevegury/finagle
Warning: That's just a proof of concept, there's no documentations nor tests.

Here's a log of the codec in action:

Feel free to fork/fix/improve.

Enjoy
Steve
Re: finagle (and util and ostrich) on scala 2.9.1 wil lu 9/11/12 1:33 AM
Hi tsuna,

I am having the same connection leaking issue even with latest Querulous 3+, could you let me know if you have found the solution to the issue and how? Thanks a lot.

Wil
Re: finagle (and util and ostrich) on scala 2.9.1 tsuna 9/11/12 10:30 AM
On Tue, Sep 11, 2012 at 1:33 AM, wil lu <fumi...@gmail.com> wrote:
> I am having the same connection leaking issue even with latest Querulous 3+,
> could you let me know if you have found the solution to the issue and how?
> Thanks a lot.

No, as I said in a later post, I gave up on Querulous, and posted a
simple bit of code i put together to replace it at
https://gist.github.com/2245176

The future now lies in the MySQL codec in Finagle itself.

--
Benoit "tsuna" Sigoure
Re: finagle (and util and ostrich) on scala 2.9.1 wil lu 9/11/12 7:05 PM
Hi tsuna,

Thank you very much for the quick response (bit surprised coz last post was half year ago...)

I've also gone through Querulous source (also for hours) and seems everything is correctly handled / closed.. Anyway, as all I want is just fast / reliable simple SQL wrapper layer, I then found https://github.com/jpersson/prequel, it's more lightweight and even provides a little better usage (selected row implicit type mapping), but when went through the source code found that it's using Statement rather than PreparedStatement, and did very basic sql escape, so I may decide to modify prequel to use PreparedStatement instead.

I've just checked your code on github, it's more basic and compact. And I have 2 questions that I'd like to hear your thoughts:
1. the select returns "Future[Seq[T]]" which is the same idea as AsyncQueryEvaluator I guess, could you tell me in what case async way is handy than sync way? I am not sure if async way is better than sync way in all cases or just for some special condition.

2. why didn't you use some 3rd party pool jar like BoneCP or c3p0, actually I am also in the process of seeking a connection pool solution, so would like to hear what you think.

Thanks again.
Wil
Re: finagle (and util and ostrich) on scala 2.9.1 tsuna 9/11/12 7:52 PM
On Tue, Sep 11, 2012 at 7:05 PM, wil lu <fumi...@gmail.com> wrote:
> 1. the select returns "Future[Seq[T]]" which is the same idea as
> AsyncQueryEvaluator I guess, could you tell me in what case async way is
> handy than sync way? I am not sure if async way is better than sync way in
> all cases or just for some special condition.

If you use Finagle, you want everything to be async.  That was my
case, hence the need for an async interface.  You don't wanna block
Finagle I/O threads.

> 2. why didn't you use some 3rd party pool jar like BoneCP or c3p0, actually
> I am also in the process of seeking a connection pool solution, so would
> like to hear what you think.

Because they're big dependencies that I don't need.  In my code I
start a fixed number of threads, and pre-open the same number of
connections, and put a queue in between.  I don't need anything else.

--
Benoit "tsuna" Sigoure
Re: finagle (and util and ostrich) on scala 2.9.1 wil lu 9/12/12 6:39 PM

For pool, I'd like make it pluggable; for async I'll think about it, i am not using finagle, but i will keep in mind in case the case of needing async db op popups. Thanks tsuna.

More topics »