Tornado and Cassandra

[Elias Torres]

Anyone here using Cassandra with Tornado? To block or not too block?

-Elias

[Stuart King]

I am using both on a startup. We make every call to that is to an
external service, whether it be db, webservice, search index, keystore
(cassandra) as a non-blocking call.

Stu

[Elias Torres]

Very cool. Which Cassandra client are you using?

-Elias

Sent from my iPhone

On Jun 29, 2010, at 10:13 AM, Stuart King <stuart...@gmail.com>
wrote:

[Stuart King]

Using Pycassa. The entire stack is Nginx / Tornado / Cassandra, with
Solr and Lucene for indexing and searching.

I wasnt sure about Pycass initially, but after testing against other
options it was easiest and most flexible to use. It has also picked up
some traction with the Cassandra committers contributing to it.

Stu

On Jun 30, 6:47 am, Elias Torres <el...@torrez.us> wrote:
> Very cool. Which Cassandra client are you using?
>
> -Elias
>
> Sent from my iPhone
>

> On Jun 29, 2010, at 10:13 AM, Stuart King <stuartrexk...@gmail.com>  

[Elias Torres]

So did you have to make pycassa asynchronous?

-Elias

Sent from my iPhone

On Jun 29, 2010, at 9:29 PM, Stuart King <stuart...@gmail.com>
wrote:

[Japhy Bartlett]

not to drag this too off-topic, but did you try Mongo at all? The
python bindings are quite good.

I'm using an nginx / tornado / mongo stack, and it is a joy to develop
with. Not much load / traffic yet though! ;)

- Japhy

[Juan J.]

El mié, 30-06-2010 a las 10:39 -0400, Japhy Bartlett escribió:
> not to drag this too off-topic, but did you try Mongo at all? The
> python bindings are quite good.
>
> I'm using an nginx / tornado / mongo stack, and it is a joy to develop
> with. Not much load / traffic yet though! ;)

I'm using Varnish / Cherokee / Tornado / Redis stack, and I'm very happy
too with the results.

Cheers,

Juanjo

--
jjm's home: http://www.usebox.net/jjm/
blackshell: http://blackshell.usebox.net/
ramble on: http://rambleon.usebox.net/

[Stephen Leung]

I use that stack too, but our site still in development, I would happy to share this result when our site done, also like to listen any issues/problems from you. :)

Mongodb is good. did you tried the mongo-async-python with tornado? it based on Twisted, so I haven't given a try. 

--
The Best Regard
Stephen Leung
Phone 13761842559
MSN   one...@gmail.com

[Stuart King]

I think there might be some misunderstanding caused by the term
"asynchronous". I think non-blocking is a better term to specify
tornados behaviour. From non-blocking, you can get asynchronous
features (via long lived connections, etc).

I didn't change pycassa at all. All that happens is that rather than
have the request thread interact with the services, it's passed off to
epoll to execute the tasks. When they return, the callback is called
and the response committed. I am sure the tornado committers can give
a more detailed and technical explanation, but looking at the source,
and the AsyncHttpClient, this is how I understand it.

Could anyone else comment on this?

Stu

On Jun 30, 9:45 pm, Elias Torres <el...@torrez.us> wrote:
> So did you have to make pycassa asynchronous?
>
> -Elias
>
> Sent from my iPhone
>

> On Jun 29, 2010, at 9:29 PM, Stuart King <stuartrexk...@gmail.com>  

[Stanislav]

Stu,

That is not how it works,

epoll is simply used to manage many file descripters

If you are using pycassa then it is in fact blocking your main thread
while fetching from Cassandra (however this should be fast and not
noticeable). If you wish to not block then you must tweak pycassa to
use use non-blocking sockets, which I don't believe would be easy
because you would have to go tweak Thrift, rather then pycassa since
you talk to Cassandra via Thrift.

I may be wrong here about Pycassa and it might have a special Tornado
specific connection class (which I don't believe it does).

Tornado isn't magic.