Streaming REST API in master now

[Peter Neubauer]

Hi everyone,
you might be interested to hear that the REST API is going to be a LOT
faster while being exactly compatible, using a streaming approach. 50K
node REST representations in 8 seconds with almost no memory overhead.
Read more here:

http://blog.neo4j.org/2012/04/streaming-rest-api-interview-with.html

And feel free to feedback on how we can make things even faster. Thank
you Michael and Jake for working on these areas!

Cheers,

/peter neubauer

G:  neubauer.peter
S:  peter.neubauer
P:  +46 704 106975
L:   http://www.linkedin.com/in/neubauer
T:   @peterneubauer

If you can write, you can code - @coderdojomalmo
If you can sketch, you can use a graph database - @neo4j

[Pablo Pareja]

Cool !

Will the Data Browser interface from the Web Admin also benefit from this improvement in terms of performance?

Cheers,

Pablo

--
Pablo Pareja Tobes 

My site     http://about.me/pablopareja

LinkedIn    http://www.linkedin.com/in/pabloparejatobes

Twitter       http://www.twitter.com/pablopareja

Creator of Bio4j --> http://www.bio4j.com

http://www.ohnosequences.com

[Javier de la Rosa]

That's great! I hope REST clients will benefit from this soon.

--
Javier de la Rosa
http://versae.es

[Jim Webber]

Are the extension points (managed and unmanaged plugins) going to remain the same for server-side developers?

Will the streaming features be available automatically for managed extensions?

Also: it's be nice if clients could choose whether they want streamed responses using the TE request header paired with the corresponding Transport-Encoding response header (as Michael and I have discussed offlist).

Jim

[Michael Hunger]

Am 23.04.2012 um 23:47 schrieb Jim Webber:

> Are the extension points (managed and unmanaged plugins) going to remain the same for server-side developers?

Yes they stay the same, people should take care to return Iterables and also use ListRepresentation(type, iterable) to benefit from streaming.

>
> Will the streaming features be available automatically for managed extensions?

Yes, see above.

>
> Also: it's be nice if clients could choose whether they want streamed responses using the TE request header paired with the corresponding Transport-Encoding response header (as Michael and I have discussed offlist).

Right that will be changed after we figured out how exactly to do it.

[Tatham Oddie]

The .NET Neo4jClient library will now automatically upgrade to streaming when connecting to 1.8+:

http://hg.readify.net/neo4jclient/changeset/1806f2373fc1

http://nuget.org/packages/Neo4jClient/1.0.0.330

(Note: version comparisons don't work against snapshots because the root endpoint doesn't give us sufficient information, thus this will only kick in once the milestone drops.)

-- Tatham

[Peter Neubauer]

Thanks a lot Tatham!
Can you see any performance improvements yet?

[Michael Hunger]

Did you test the performance impact?

I pushed another change last night (json-utf8-generator-buffer tuning) which added another 20%.

Michael

[Michael Klishin]

Tatham Oddie:

> The .NET Neo4jClient library will now automatically upgrade to streaming when connecting to 1.8+:

This raises a question: should Neo4J Server clients use version numbers to detect
features like this? I'd be happy to add this to Neocons [1] soon but not entirely sure if it is a good idea
to use version-based capabilities detection.

Can capabilities like this be exposed with extensions information during discovery? What do Neo4J engineers think?

1. https://github.com/michaelklishin/neocons

MK

[Peter Neubauer]

I think that is a good thing. Versions are already exposed, see e.g. 1.7,

http://docs.neo4j.org/chunked/1.7/rest-api-service-root.html#rest-api-get-service-root
versus SNAPSHOT
http://docs.neo4j.org/chunked/snapshot/rest-api-service-root.html#rest-api-get-service-root

So I think this is possible ...

Cheers,

/peter neubauer

G:  neubauer.peter
S:  peter.neubauer
P:  +46 704 106975
L:   http://www.linkedin.com/in/neubauer
T:   @peterneubauer

If you can write, you can code - @coderdojomalmo
If you can sketch, you can use a graph database - @neo4j

[Michael Hunger]

But adding capabilities would probably make more sense.

Cypher itself is able to handle versions by prefixing them.

The other features of the server have (so far) been tied to a version but might be interesting to add an explicit feature-descriptor to the root resource.

Michael

[Nigel Small]

I'm inclined to agree with Michael - feature detection would be more flexible. Py2neo has a few instances where I've had to sniff the version and from that, ascertain feature availability. That can look a bit unintuative when reading the code back at a later date and can become very cumbersome as more versions are released. On top of that, it's quite hard to account for intermediary releases and the situations where optional components are installed, e.g. Geoff / streaming Cypher lib.

Along the same lines, did we ever establish the "proper" way to ascertain the root URI for a service, given only the URI of an entity (I vaguely recall talking about this some time ago). That is, if I have Node("http://localhost:7474/db/data/node/123") how do I best establish that the service root is GraphDatabaseService("http://localhost:7474/db/data/")? At the moment, I'm forced to sniff out "/node" and chopd off everything from there onwards. IMHO not particularly discoverable or future-proof.

Nige

[Tatham Oddie]

Michael,

I just made sure that the library could still talk to the server ok; I haven't done any performance tests as yet.