txRedis - Redis Client for Twisted

[Dorian Raymer]

Hello,
I have created a Twisted version of redis.py.
I needed an asynchronous version of the python client for use in Twisted based projects, and I have not seen any other efforts on this front. Please let me know if I completely overlooked an existing Twisted+Redis Client project.

I have translated all of the doc tests in redis.py to Trial unit tests, and they are all passing.

Version 0.1.1 is available on pypi, and here is the source code git repository.

To the Twisted gurus, feedback on the implementation is very welcome. I would like to make this is a solid client library.

Thank you,
Dorian Raymer

[Salvatore Sanfilippo]

On Thu, Oct 22, 2009 at 12:14 AM, Dorian Raymer <deld...@gmail.com> wrote:
> Hello,
> I have created a Twisted version of redis.py.

Hello Dorian,

thank you! As far as I can tell this is the first attempt to create a
Twisted redis client.
I'm adding a link to the project home page in the Redis home in order
to start to get visibility to the client.

Thanks!
Salvatore

--
Salvatore 'antirez' Sanfilippo
http://invece.org

"Once you have something that grows faster than education grows,
you’re always going to get a pop culture.", Alan Kay

[Brian Hammond]

Interesting. Have you benchmarked the non-twisted version against your
twisted version? I'm curious why you need an async client for redis.
It's already so damn fast that I'd think it possible that the overhead
of dealing with callbacks/deferreds would take longer than just
waiting for redis to return. But, I know nothing for certain.

On Oct 21, 6:14 pm, Dorian Raymer <deldo...@gmail.com> wrote:
> Hello,
> I have created a Twisted version of redis.py.
> I needed an asynchronous version of the python client for use in Twisted
> based projects, and I have not seen any other efforts on this front. Please
> let me know if I completely overlooked an existing Twisted+Redis Client
> project.
>
> I have translated all of the doc tests in redis.py to Trial unit tests, and
> they are all passing.
>

> Version 0.1.1 is available on pypi<http://pypi.python.org/pypi/txredis/0.1.1>,

> and here is the source code

> <http://ooici.net:82/?p=txredis.git;a=summary>git repository.

[Brian Hammond]

Oh I see 'yield'. Perhaps I'm out of the loop on the latest with
Twisted. This must be the generator stuff - ok. Nevermind.

I think 'scard' is missing BTW.

On Oct 27, 2:39 pm, Brian Hammond <br...@fictorial.com> wrote:
> Interesting. Have you benchmarked the non-twisted version against your
> twisted version? I'm curious why you need an async client for redis.
> It's already so damn fast that I'd think it possible that the overhead
> of dealing with callbacks/deferreds would take longer than just
> waiting for redis to return. But, I know nothing for certain.
>
> On Oct 21, 6:14 pm, Dorian Raymer <deldo...@gmail.com> wrote:
>
>
>
> > Hello,
> > I have created a Twisted version of redis.py.

> > I needed an asynchronous version of thepythonclient for use in Twisted

[Dorian Raymer]

Hi Brian,

I have not benchmarked, but I would predict the two would have similar performance for serial requests. For multiple/parallel Redis requests/connections, the async client is more flexible (powerful).

The purpose of writing txRedis was to have a proper Twisted protocol implementation that can utilize the framework. 

Writing applications with Twisted is about letting the reactor drive, and mixing in blocking I/O code sort of defeats the purpose. There is some conceptual/design overhead when dealing with deferreds, but there are also great benefits.

 

It is good point, though: txRedis is just a Twisted compatible version of redis.py, which you only would want if you are writing higher concurrency apps with Twisted already. 

On Tue, Oct 27, 2009 at 11:42 AM, Brian Hammond <br...@fictorial.com> wrote:

Oh I see 'yield'.  Perhaps I'm out of the loop on the latest with
Twisted.  This must be the generator stuff - ok.  Nevermind.

In this code, yield is used in conjunction with the @defer.inlineCallbacks decorator. 

This allows you to deal with functions that return deferreds in a regular script fashion. Instead of adding a callback function to a deferred to handle the async result, the result is returned right there, inline! Inline callbacks is a Twisted concept; you can find the official documentation in the twisted.internet.defer module.

 

I think 'scard' is missing BTW.

 Yea, looks like a few set operators are missing. The command set code mostly comes from redis.py, which is also missing 'scard'. I'll look into this more carefully and double check that all commands get implemented. 

Thanks!

-Dorian

[Brian Hammond]

> txRedis is just a Twisted compatible version of
> redis.py, which you only would want if you are writing
> higher concurrency apps with Twisted already.

This is what I figured. Thanks!

[Brian Hammond]

On Oct 27, 3:29 pm, Dorian Raymer <deldo...@gmail.com> wrote:
> > I think 'scard' is missing BTW.
>
>  Yea, looks like a few set operators are missing. The command set code
> mostly comes from redis.py, which is also missing 'scard'. I'll look into
> this more carefully and double check that all commands get implemented.
> Thanks!
>
> -Dorian

FYI http://github.com/sanxiyn/redis/commit/a759ed28d81bea840a4e7850f26a422e73b35562