Spatial Cypher in REST

[Javier de la Rosa]

Hi all,

I was starting to do some experiments with Spatial Neo4j when realized that there is no easy way to make queries using Cypher. All i found is a couple of threads [1, 2]. So, I was just wondering if there is any way to make real spatial queries using Cypher, and therefore using REST API.

I was thinking in something like the standard defined by OGC [3], that is used in spatial extensions like PostGIS or SpatiaLite. With this, we could have complex geometry classes like Geometry, Point, Curve, LineString, Surface, Polygon, PolyhedralSurface, GeomCollection, MultiCurve, MultiLineString, MultiSurface, MultiPolygon, and MultiPoint. And the corresponding representation in binary, text, GeoJSON or WKT. I like WKT and it is a very versatile and powerful way to express geometrical objects.

Let's say that we can define those objects using WKT and a prefix, 'ST_', to denote spatial function

CREATE n = {name : 'Andres', title : 'Developer', place : ST_WKT('POLYGON ((15 56, 15 57, 16 57, 16 56, 15 56))')}

And then, guess we have an index with nodes that represent world boundaries. In this case, what I would like to have is something like

CREATE n = {label : 'Germany', area : ST_WKT('MULTYPOLYGON((...))')}

START n=node(*), m=node(*), b=node:WORLD_BOUNDARIES("label:Germany") MATCH n-[r:FRIEND]->m WHERE ST_Contains(b.area, n.place) AND ST_Contains(b.area, m.place) RETURN ST_Distance(ST_Centroid(n.place), ST_Centroid(m.place))

This query above should return the distance between two friends that live in Germany.

All functions are already defined in the standard [4] that includes ST_Equals, ST_Disjoint, ST_Intersects, ST_Touches, ST_Crosses, ST_Within, ST_Contains, ST_Overlaps, ST_Intersection, ST_Difference, ST_Union, ST_StartPoint, ST_EndPoint, ST_IsRing, ST_Length, ST_NumPoints, ST_PointN, ST_Centroid, ST_PointOnSurface, ST_Area and more.

So, is there any plan to support complex spatial queries in Cypher like this ones? I wish I knew enough Java to give a hand.

Best regards.

[1] http://neo4j.org/nabble/#nabble-td3500531

[2] https://groups.google.com/forum/#!msg/neo4j/7fBtZBLAwSY/zlKNBqteFEwJ

[3] http://www.opengeospatial.org/standards/sfs

[4] http://portal.opengeospatial.org/files/?artifact_id=25354

--

Javier de la Rosa

http://versae.es

[Peter Neubauer]

Javier,
I put in a SpatialIndexProvider into Neo4j Spatial, so you can ask
these providers via Cypher, see
https://github.com/neo4j/spatial/blob/master/src/test/java/org/neo4j/gis/spatial/IndexProviderTest.java#L138
.

For more complex things, we could possibly implement the IndexProvider
to accept e.g. CQL
http://udig.refractions.net:8080/confluence/pages/viewpage.action?pageId=11469
since there are already DynamicLayers being defined in CQL that would
fit the bill. Feel free to think of more way to define a good index
provider DSL without re-inventing the wheel.

WDYT?

Cheers,

/peter neubauer

G: neubauer.peter
S: peter.neubauer
P: +46 704 106975
L: http://www.linkedin.com/in/neubauer
T: @peterneubauer

Wanna learn something new? Come to http://graphconnect.com

> --
>
>

[Javier de la Rosa]

On Tue, Oct 2, 2012 at 4:52 PM, Peter Neubauer
<peter.n...@neotechnology.com> wrote:
> I put in a SpatialIndexProvider into Neo4j Spatial, so you can ask
> these providers via Cypher, see
> https://github.com/neo4j/spatial/blob/master/src/test/java/org/neo4j/gis/spatial/IndexProviderTest.java#L138

I see, but will I be able to also create the index if I need it using
Cypher? AFAIK that is not an option now, isnt' it?

> For more complex things, we could possibly implement the IndexProvider
> to accept e.g. CQL
> http://udig.refractions.net:8080/confluence/pages/viewpage.action?pageId=11469
> since there are already DynamicLayers being defined in CQL that would
> fit the bill. Feel free to think of more way to define a good index
> provider DSL without re-inventing the wheel.

CQL looks like an interesting approach. Although it hasn't the power
of the standard that many other databases have already adopted. On the
other hand, using standards would reduce the entry level when somebody
wants to change from spatial and relational databases to Spatial
Neo4j.

[Michael Hunger]

But you can create it using the REST API, or define your auto-index to be a spatial index.

Michael

> --
>
>

[Javier de la Rosa]

I didn't know that. Then, how can I create a spatial index using REST API?

--
Sent using a cell-phone, so sorry for the typos and wrong auto-corrections.

--

[Peter Neubauer]

Here you go: https://github.com/neo4j/spatial/blob/master/src/test/java/org/neo4j/gis/spatial/SpatialPluginFunctionalTest.java#L52

you can run that test to see the exact REST calls if they are not
obvious from the code. Would love to put the generated manual
somewhere, actually ...

Cheers,

/peter neubauer

G: neubauer.peter
S: peter.neubauer
P: +46 704 106975
L: http://www.linkedin.com/in/neubauer
T: @peterneubauer

Neo4j 1.8 GA - http://www.dzone.com/links/neo4j_18_release_fluent_graph_literacy.html

> --
>
>

[Javier de la Rosa]

I see, maybe I was misunderstanding the concept behind. Then, if a
node isn't on a SpatialIndex, can't be spatially queried, isn't? I
think this is a bit different than other systems work. So, if I
already have nodes with properties that represent coordinates, there
is no way to get the distance between.

What about SRID and projection systems?

Can I put all nodes with spatial information in the same spatial index
and do any kind of queries with them, like distance, or which ones of
them are inside this area defined in this another node?

Sorry for all the questions, but I am thinking about applications in
archaeology.

Best regards.

On Wed, Oct 3, 2012 at 5:32 PM, Peter Neubauer

> --

[Craig Taverner]

Personally I very much like your ideas, and internally neo4j-spatial supports everything you've said, but the external API's are not up to it (yet). The concepts of dataset and layer as most interesting when considering accessing spatial data without needing to use the index. You can iterate through spatial objects and convert between the geometry and graph versions of these objects. The geometry versions are all using JTS, so you can do anything with them that JTS supports (which is, of course, pretty much anything spatial). However, all of this is only possible in the Java API. To get this exposed in the REST API or in Cypher would require some more work. I have an idea how to do this in the REST API, but not really in Cypher.

For Cypher, I would assume that it needs the ability to have plugable extensions, that add to the function list. Much like using 'require' in Ruby. I have no idea if Cypher supports extensions like this, and I think we would need the Cypher guys to chip in on that question.

I see, maybe I was misunderstanding the concept behind. Then, if a
node isn't on a SpatialIndex, can't be spatially queried, isn't? I
think this is a bit different than other systems work. So, if I
already have nodes with properties that represent coordinates, there
is no way to get the distance between.

You do not need to use the spatial index. You can be working directly with the graph using whatever domain-specific code you wrote to deal with your own graph model (independent of whether or not the graph is spatially enabled). The GeometryEncoder for your spatial layer can be used to convert between nodes and JTS Geometries. If you use the DefaultLayer, then this means it reads the WKB contents of the_geom property of the node and makes a Geometry. If using the OSMLayer, it traverses the complete sub-graph representing the geometry you are interested in, and builds a complete JTS Geometry from the graph. So you can perform any geometry operations supported by JTS. However, as mentioned before this is only supported in the Java API.

What about SRID and projection systems?

Each customer layer allows the setting of projections. However, this is for storage only. So, for example, when importing an OSM layer, we store the WGS84 projection and report that to any consumers of the data (so it is viewed correctly in GeoServer or uDig, etc.). When importing a Shapefile, the projection in the shapefile is stored in the layer and reported to consumers also.

However, we have not written any support for operations on projection. No support for re-projection. It is assumed that it would be done by client code.

Can I put all nodes with spatial information in the same spatial index
and do any kind of queries with them, like distance, or which ones of
them are inside this area defined in this another node?

The point of the spatial index is to accelerate spatial searches, and it is used for any kind of search (search in polygon, etc.). For distance, I would just get the two geometries of interest and get their distance (no need for the index, unless you used it to get the geometries in the first place). You would not need the index in the case you had domain specific code that operated directly on the graph, and so were working directly with neo4j nodes. Then if you had two nodes representing two geometries, you could convert them to geometries with your layers geometry-encoder, and start making JTS calls directly on the resulting objects. For example, look at http://tsusiatsoftware.net/jts/javadoc/com/vividsolutions/jts/operation/distance/DistanceOp.html.

Sorry for all the questions, but I am thinking about applications in
archaeology.

Are you using the REST API or Cypher? In that case I think this gets trick. If you are using the Java API, the world is your oyster :)

[Javier de la Rosa]

Thanks for all the explanation.

On Fri, Oct 5, 2012 at 11:35 AM, Craig Taverner <cr...@amanzi.com> wrote:
> I have an idea how to do this in the REST API, but not really in
> Cypher.

I'd be gald to hear that, I am using a Python client to connect through REST.

>
> For Cypher, I would assume that it needs the ability to have plugable
> extensions, that add to the function list.

+1. And be usable from REST. Or even to have a repository of
extensions like packages or something.

> So you can perform any geometry operations
> supported by JTS. However, as mentioned before this is only supported in the
> Java API.

It would be awesome if there was also available in REST and/or Cypher.

> we store the
> WGS84 projection and report that to any consumers of the data

Glad to read you use WSG84 :)

> However, we have not written any support for operations on projection. No
> support for re-projection. It is assumed that it would be done by client
> code.

Or just set in the neo4.properties file?

> Are you using the REST API or Cypher? In that case I think this gets trick.

It is.

> If you are using the Java API, the world is your oyster :)

I can imagine.

Thank again for this useful information.

Best regards.

[Craig Taverner]

Hi Bob,

It is interesting that you are using the cypher query within Java code. If you are already in Java, it is much more powerful to use the Java API for Neo4j-Spatial, which supports a much wider range of features. However, let's first try answer your question directly. The ExecutionResult is a table of columns and rows (one column, since you only return one result per row using your 'return n' statement). So we could, for example, get the first node with some ugly code like:

Node n = (Node)result.iterator().next().values().iterator().next();

It would be better, of course, to iterate over the rows and columns like this:

for(Map<String,Object> row:result) {

  for(String column: row.keySet()){

     Node value = (Node)row.get(column);

     System.out.println("\t"+column+":\t"+value);

  }

}

And then you have the Node object you can use to investigate the local graph with direct queries on that nodes neighborhood. And if you want to return to cypher, you could pass that nodes id back into subsequent  cypher queries.

If you would rather use the Java API, which I honestly think makes sense if you are writing in Java. Then you could look at the many examples in the test cases in github. For example, your withinDistance query could be written as described in my aging, but still relevant blog post at http://blog.neo4j.org/2011/03/neo4j-spatial-part1-finding-things.html.

The key part of this would be the query itself, which in this case looks like:

// Search for nearby locations

Coordinate myPosition = new Coordinate(13.76, 55.56);

List<SpatialDatabaseRecord> results =

layer.findClosestPointsTo(myPosition, 10.0);

And the results object can be iterated over, and will also return Nodes for the geometries it contains. You can operate on both the Node and Geometry objects as you wish. The Geometry objects are true JTS Geometries, allowing all JTS functions too, which is rather convenient. You will not get those directly from the cypher results as you do in the Java API.

On Mon, Jan 28, 2013 at 9:58 PM, Bob Wilson <bwil...@hotmail.com> wrote:

Craig,

We're new to Neo4j-spatial and have been trying to determine how to chain a distance query on the spatial layer with a secondary filter on the base nodes in our graph. The goal is to run one query to get the number of office locations within a provided distance, and within that result set from the spatial layer, additionally filter on both office properties(type of office) and relationships(provides a service) on the base node graph. I've seen this discussed in a few places but we have not been able to figure out the syntax. Starting with this Java query that was posted, could you advise on how to expand this and chain the result set of the spatial nodes query down to the underlying nodes that have the relationships we need to filter on?

        ExecutionResult result = engine.execute("start n=node:layer3('withinDistance:[44.44, 33.32, 5.0]') return n");

Thank you.
Bob

--
You received this message because you are subscribed to the Google Groups "Neo4j" group.
To unsubscribe from this group, send email to neo4j+un...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Bob Wilson]

Hi Craig, we were able to work through our Neo4j-spatial issues by decoupling the spatial and the secondary graph call. It performs fine in two parts.

Now we have a problem with the second query. We initially wrote it in Cypher embedded in Java code and although the query was quick, looping through and parsing the ExecutionResult was prohibitively slow.

Next we are trying the traversal API which we heard was faster than embedded Cypher. We are having this issue with the traversal api - any ideas please pass them on. Thanks

W

e are trying to use the Traversal API to retrieve, for a given set of company nodes, only the company nodes that provide all the products that are contained in the product Nodes list. In this example: Company 1 provides product A and B  -> Company 2 provides product A and B and C  -> Company 3 provides product A and C If all 3 companies are included in company list query, and product A and C are in product list in query, we would want returned only Company 2 and 3 since they provide product A and C. Here is our query: for ( Path position : Traversal.description() .depthFirst() .uniqueness(Uniqueness.NODE_GLOBAL) .relationships(Rels.PROVIDES_PRODUCT, Direction.OUTGOING) .evaluator((Evaluator) Evaluators.includeWhereEndNodeIs(productNodes)) .traverse(companyNodes)) If we use the Evaluator.includeWhereEndNodeIs(productNodes) we get back all the companies that provide ANY of the products in the productNodes list(all 3 companies in above example). If we use the Evaluators.includeIfContainsAll(productNodes) evaluator, we don't get any company nodes back if there is more than one product in the product nodes list. We were looking at the BiDirectionalTraversal approach but have not been able to get that working either. Any suggestions appreciated. neo4j