Pagination, Sort, Skip, Limit

[Amirouche B.]

Héllo,

I would like to paginate my results. Right now I can filter for the good vertices but I don't want all the results. The operators I need are the following:

- Sort : I saw something like that in the ML but I could not get it working, what is the syntax ?
- Skip + Limit or Range I assume I can use RangeFilterPipe to do the pagination.

Thanks


[Peter Neubauer]

There is an example at http://docs.neo4j.org/chunked/snapshot/gremlin-plugin.html#rest-api-chunking-and-offsetting-in-gremlin

Does that help?

Cheers,

/peter neubauer

GTalk:      neubauer.peter
Skype       peter.neubauer
Phone       +46 704 106975
LinkedIn   http://www.linkedin.com/in/neubauer
Twitter      http://twitter.com/peterneubauer

http://www.neo4j.org              - NOSQL for the Enterprise.
http://startupbootcamp.org/    - Öresund - Innovation happens HERE.

[Pierre De Wilde]

Hey,

As Peter indicates, range [0..10] may be used to limit results in Gremlin.

Rexster parameters may also be used for pagination: rexster.offset.start and rexster.offset.end

More info at https://github.com/tinkerpop/rexster/wiki/Basic-REST-API

HTH,

Pierre

[Amirouche B.]

I should have said that earlier, the application I try to do is a twitter-like application.

It's usefull thanks for pointing out this documentation. That said it's still hard to do pagination.

- The sort operator returns a ListIter
- I still need to (somehow) slice it
- It doesn't seem to support DESC sorting
- It can be optimised... since it's a common webdev operation

I would rather write a gremlin filter to do the pagination something that would replace the SQL : « LIMIT A[, B] ORDER_BY [ASC| DESC] ». I don't know how hard it is to write filters, does the code belong in gremlin or blueprint maybe both ?

[Amirouche B.]

Slicing the result in Rexster is not a solution if you have several millions of results, performance wise.

I don't use Rexster REST API anymore. I don't see how it can be useful if it doesn't support transactions. I use Gremlin/Groovy extension through Rexster.

[stephen mallette]

I'm not sure if it's useful to you, but have you looked at that Batch Kibble?

https://github.com/tinkerpop/rexster-kibbles/tree/master/batch-kibble

It might be helpful if you want to do a series of create/update/delete operations in a series wrapped in a transaction.

Best regards,

Stephen

[Amirouche B.]

I don't have problemes doing multiple insert or delete with rexster since I send gremlin script. The current problem is about pagination which I have to do client side because blueprint/gremlin/whatever don't expose such operator

[James Thornton]

Hey Amirouche -

Here's how to do sorting with Gremlin:

https://groups.google.com/d/topic/gremlin-users/A8bZHSOxgyA/discussion

And here's how to do paging the Gremlin way (example in Bulbs):

>>> from bulbs.graph import Graph

>>> g = Graph('http://localhost:8182/neo4jenron')

>>> g.gremlin.query('g.V[0..9]')

- James

[Amirouche B.]

You are right I can use rexster.offset.start and rexster.offset.end to limit the result. I did not want to use this method, because I assume that all the nodes will be in memory which might hurt performance wise. I shouldn't care about optimisation right now.

[James Thornton]

Hey Amirouche -

The Neo4j docs say to "assume the graph is always in memory" (http://wiki.neo4j.org/content/Guidelines_for_Building_a_Neo_App#Assume_everything.27s_automatically_persistent_.28Neo4j_will_make_it_so.29) -- what's OrientDB's philosophy on this?

Regardless, Stephen recently added support for mutliget (https://github.com/tinkerpop/rexster-kibbles/issues/7) so a performance-optimization strategy might be to have your Gremlin queries return only the element IDs, cache them in Redis, and then use Rexster's multiget to lazy-load the elements as needed.

Here's the original issue in the issue tracker...

* https://github.com/tinkerpop/rexster/issues/116

There's more discussion in these threads...

* https://groups.google.com/d/msg/gremlin-users/fr3nr69O-eM/ueWxVj6AqzwJ

* https://groups.google.com/d/msg/gremlin-users/kPH68Wh4sT8/GSbxwfiPr_UJ

- James

[Luca Garulli]

On 23 November 2011 19:16, James Thornton <james.t...@gmail.com> wrote:

Hey Amirouche -

The Neo4j docs say to "assume the graph is always in memory" (http://wiki.neo4j.org/content/Guidelines_for_Building_a_Neo_App#Assume_everything.27s_automatically_persistent_.28Neo4j_will_make_it_so.29) -- what's OrientDB's philosophy on this?

Hi,

if you use Blueprints (and therefore Rexster) it's exactly the same. But if you use raw API you've more control on this:

http://code.google.com/p/orient/wiki/GraphDatabase

Lvc@