Tornado, concurrency and py2neo 1.3

[Nigel Small]

It's fair to say that some people's experience with py2neo 1.2 has been a little shaky due to the introduction of Tornado for HTTP messaging. This library was chiefly brought in so that asynchronous Cypher queries could be executed but has dragged along with it a considerable overhead in resource usage, poor support for concurrency and questionable operation under Windows.

For these reasons, I have decided to move away from Tornado for py2neo 1.3 and am currently building a simpler, but just as functional HTTP layer based on the standard httplib module. This will of course also remove the need for a third party dependency when using py2neo in future. In addition, I aim to provide full support for concurrent applications since this is clearly an area of functionality that many people require.

If you are willing and able to help with testing for this new version, I would be very grateful for the feedback, good or bad! The code currently within GitHub contains a mostly operational version of the new HTTP layer so please let me know if you are able to help and raise any issues you find within GitHub.

Thanks and kind regards
Nige

[Nigel Small]

Cheers, Nico... I'll keep an eye on the requests project :-)

Nige

On 2 August 2012 16:27, Nicolas Wack <wac...@gmail.com> wrote:

Hi,

haven't been using py2neo yet, although I'll definitely have a look at it in the future.

I just wanted to mention that if you want to do HTTP requests in python, there is this library, from Kenneth Reitz, which is quickly becoming a de-facto standard and is a joy to use:

http://docs.python-requests.org/

Hope this helps,

Cheers,

Nico.

[Javier de la Rosa]

On Thursday, August 2, 2012, Nicolas Wack wrote:

http://docs.python-requests.org/

+1. A must for me too.

 

Hope this helps,

Cheers,

Nico.

On Thursday, August 2, 2012 12:10:36 PM UTC+2, Nigel Small wrote:

--
Javier de la Rosa
http://versae.es

[Lasse Westh-Nielsen]

Nigel,

Good call. We migrated off py2neo for a tiny project we were doing for exactly this reason (Tornado dependency). I'd been keen to use it again when we revisit that particular project.

Lasse

[Julian Simpson]

On Fri, Aug 3, 2012 at 9:18 AM, Lasse Westh-Nielsen <lasse.wes...@neopersistence.com> wrote:

Nigel,

Good call. We migrated off py2neo for a tiny project we were doing for exactly this reason (Tornado dependency). I'd been keen to use it again when we revisit that particular project.

+1

Removing py2neo from our little project immediately proved py2neo's value: it really hurt to remove it :)

J.

 

Lasse

On Thu, Aug 2, 2012 at 11:10 AM, Nigel Small <ni...@nigelsmall.net> wrote:

It's fair to say that some people's experience with py2neo 1.2 has been a little shaky due to the introduction of Tornado for HTTP messaging. This library was chiefly brought in so that asynchronous Cypher queries could be executed but has dragged along with it a considerable overhead in resource usage, poor support for concurrency and questionable operation under Windows.

For these reasons, I have decided to move away from Tornado for py2neo 1.3 and am currently building a simpler, but just as functional HTTP layer based on the standard httplib module. This will of course also remove the need for a third party dependency when using py2neo in future. In addition, I aim to provide full support for concurrent applications since this is clearly an area of functionality that many people require.

If you are willing and able to help with testing for this new version, I would be very grateful for the feedback, good or bad! The code currently within GitHub contains a mostly operational version of the new HTTP layer so please let me know if you are able to help and raise any issues you find within GitHub.

Thanks and kind regards
Nige

--
Julian Simpson 
Software Engineer | Neo Technology
+447967189579 | @builddoctor
As a friend of Neo, register for GraphConnect and get a 20% discount. Code: GCFON

[Nigel Small]

What problems was Tornado causing you? Similar to the ones I listed?

[Julian Simpson]

On Fri, Aug 3, 2012 at 9:31 AM, Nigel Small <ni...@nigelsmall.net> wrote:

What problems was Tornado causing you? Similar to the ones I listed?

We were using py2neo under Jython, and there was a transitive dependency (via Tornado) on a native library.

While there's an open PR for making Tornado work fully under Jython, I didn't want to wait.

cheers

J.

[Nigel Small]

All of which says that removing the dependency is a good idea - I hadn't actually considered the Jython case.

If you get a few mins to give the new code a quick try instead, it would be great to hear how it performs.

Cheers
Nige

[Nigel Small]

Hi Joao

Thanks for the link - I wasn't aware of the fork. It looks like an interesting project, are you currently using it?

One of the (many) reasons for moving away from Tornado for py2neo however was to remove the dependency on a third party library. The code now relies solely on httplib which is obviously part of the Python standard library and therefore has a much greater degree of portability.

Cheers

Nige

On 5 August 2012 23:29, João Martins <jcma...@gmail.com> wrote:

Why not change to http://cyclone.io/ and  https://github.com/fiorix/cyclone

regards
joao martins

Em sexta-feira, 3 de agosto de 2012 07h01min46s UTC-3, Nigel Small escreveu:

All of which says that removing the dependency is a good idea - I hadn't actually considered the Jython case.

If you get a few mins to give the new code a quick try instead, it would be great to hear how it performs.

Cheers
Nige

On 3 August 2012 09:57, Julian Simpson <julian.simpson@neotechnology.com> wrote:

On Fri, Aug 3, 2012 at 9:31 AM, Nigel Small <ni...@nigelsmall.net> wrote:

What problems was Tornado causing you? Similar to the ones I listed?

We were using py2neo under Jython, and there was a transitive dependency (via Tornado) on a native library.

While there's an open PR for making Tornado work fully under Jython, I didn't want to wait.

cheers

J.

On 3 August 2012 09:22, Julian Simpson <julian.simpson@neotechnology.com> wrote:

On Fri, Aug 3, 2012 at 9:18 AM, Lasse Westh-Nielsen <lasse.westh-nielsen@neopersistence.com> wrote:

Nigel,

Good call. We migrated off py2neo for a tiny project we were doing for exactly this reason (Tornado dependency). I'd been keen to use it again when we revisit that particular project.

+1

Removing py2neo from our little project immediately proved py2neo's value: it really hurt to remove it :)

J.

 

Lasse

On Thu, Aug 2, 2012 at 11:10 AM, Nigel Small <ni...@nigelsmall.net> wrote:

It's fair to say that some people's experience with py2neo 1.2 has been a little shaky due to the introduction of Tornado for HTTP messaging. This library was chiefly brought in so that asynchronous Cypher queries could be executed but has dragged along with it a considerable overhead in resource usage, poor support for concurrency and questionable operation under Windows.

For these reasons, I have decided to move away from Tornado for py2neo 1.3 and am currently building a simpler, but just as functional HTTP layer based on the standard httplib module. This will of course also remove the need for a third party dependency when using py2neo in future. In addition, I aim to provide full support for concurrent applications since this is clearly an area of functionality that many people require.

If you are willing and able to help with testing for this new version, I would be very grateful for the feedback, good or bad! The code currently within GitHub contains a mostly operational version of the new HTTP layer so please let me know if you are able to help and raise any issues you find within GitHub.

Thanks and kind regards
Nige

--
Julian Simpson 
Software Engineer | Neo Technology
+447967189579 | @builddoctor
As a friend of Neo, register for GraphConnect and get a 20% discount. Code: GCFON

[João Martins]

Hi,

the guys(Alex and Gleicon) who made the cyclone.io are my friends. You can talk with @gleicon in twitter. Gleicon is a specialist in Nosql.

regards,

Joao

[Leonardo Martins]

cyclone.io rocks.

On 3 August 2012 09:22, Julian Simpson <julian....@neotechnology.com> wrote:

On Fri, Aug 3, 2012 at 9:18 AM, Lasse Westh-Nielsen <lasse.wes...@neopersistence.com> wrote:

Nigel,

Good call. We migrated off py2neo for a tiny project we were doing for exactly this reason (Tornado dependency). I'd been keen to use it again when we revisit that particular project.

+1

Removing py2neo from our little project immediately proved py2neo's value: it really hurt to remove it :)

J.

 

Lasse

On Thu, Aug 2, 2012 at 11:10 AM, Nigel Small <ni...@nigelsmall.net> wrote:

It's fair to say that some people's experience with py2neo 1.2 has been a little shaky due to the introduction of Tornado for HTTP messaging. This library was chiefly brought in so that asynchronous Cypher queries could be executed but has dragged along with it a considerable overhead in resource usage, poor support for concurrency and questionable operation under Windows.

For these reasons, I have decided to move away from Tornado for py2neo 1.3 and am currently building a simpler, but just as functional HTTP layer based on the standard httplib module. This will of course also remove the need for a third party dependency when using py2neo in future. In addition, I aim to provide full support for concurrent applications since this is clearly an area of functionality that many people require.

If you are willing and able to help with testing for this new version, I would be very grateful for the feedback, good or bad! The code currently within GitHub contains a mostly operational version of the new HTTP layer so please let me know if you are able to help and raise any issues you find within GitHub.

Thanks and kind regards
Nige

--
Julian Simpson 
Software Engineer | Neo Technology
+447967189579 | @builddoctor
As a friend of Neo, register for GraphConnect and get a 20% discount. Code: GCFON