Using Cassandra and Twisted (Python)

[Jonathan Ballet]

[as posted on the Cassandra-user mailing list previously]

Hi,

I'd like to write some Python applications using Twisted to talk to a Cassandra cluster.

It seems like the Datastax Python library from https://github.com/datastax/python-driver does support Twisted, but it's not exactly clear how I would use this library along with Twisted. The documentation for the async API is very sparse and there's no mention on how to plug this into Twisted event-loop.

Does anyone have a small working example on how to use both of these?

Thanks!

Jonathan

[Adam Holmberg]

The driver presently has a reactor implementation that uses the Twisted event loop. That is used simply by specifying TwistedConnection for connection_class in the Cluster init:

from cassandra.io.twistedreactor import TwistedConnection
cluster = Cluster(connection_class==TwistedConnection)

Asynchronous request execution is documented here.

Not implemented is a complete API returning deferreds for all blocking operations. There is some discussion in PYTHON-9.

Regards,

Adam Holmberg

To unsubscribe from this group and stop receiving emails from it, send an email to python-driver-u...@lists.datastax.com.

[Jason Williams]

I keep running into issue when trying to use the native Twisted CQL client Silverberg...so looking at the Datastax driver.  Is anyone successfully using the Datastax driver with Twisted code in production (wrapped in a deferToThread or a hybrid using the DSE callbacks)? 

Any related experience is greatly appreciated.

-J

To unsubscribe from this group and stop receiving emails from it, send an email to python-driver-user+unsub...@lists.datastax.com.

[Dana Powers]

We run the python cassandra driver in production within a twisted app. We have not experimented with any of the non-default event loops (cassandra.io), including the twisted reactor. Instead, we simply let the default cassandra driver run separately via threading. So basically:

```

def _session(d):

  reactor.callFromThread(d.callback, Cluster(hosts).connect())

def get_session():

  d = defer.Deferred()

  reactor.callInThread(_session, d)

  return d

def _future_to_deferred(future):

  d = defer.Deferred()

  future.add_callback(lambda result: reactor.callFromThread(d.callback, result))

  future.add_errback(lambda error: reactor.callFromThread(d.errback, error))

  return d

def query(session, cql):

  return _future_to_deferred(session.execute_async(cql))

```

So that is functional -- whether it scales for your needs is another story. That default setup for us is generally bottlenecked on CPU, which issues are discussed in other threads (pun!) on the mailing list.

-Dana

[Jason J. W. Williams]

Thanks Dana! That gives me some confidence. I wrote a thin wrapper using deferToThread on Friday. 

I couldn't get Silverberg to play nicely with Cassandra 2.1.x at all. 

-J

Sent via iPhone

To unsubscribe from this group and stop receiving emails from it, send an email to python-driver-u...@lists.datastax.com.