neo4j-spatial: Creating a WKT-based index for polygon geometries

[Jonathan Winterflood]

Hi all,

I'm tying to create a spatial index for polygon data (stored as WKT in a property of my nodes)
I'm using Java with an embedded DB (with a view to use a plugin later, possibly)

From the spatial unit tests and neo4j docs, this is what I've gathered:

First, I need to create an index in code, with a specific configuration
    IndexManager indexMan = graphDb.index();
    Index<Node> index = indexMan.forNodes("myindex", config);

The config needs to specify {provider=spatial} to use SpatialIndexProvider

creating the index forst time works, but recovering it on the second run causes an error:

    Exception in thread "main" java.lang.IllegalArgumentException: Supplied index configuration:
    {provider=spatial}
    doesn't match stored config in a valid way:
    {provider=spatial}
    for 'abcdefg'
        at org.neo4j.kernel.IndexManagerImpl.assertConfigMatches(IndexManagerImpl.java:156)
        at org.neo4j.kernel.IndexManagerImpl.findIndexConfig(IndexManagerImpl.java:137)
        at org.neo4j.kernel.IndexManagerImpl.getOrCreateIndexConfig(IndexManagerImpl.java:198)
        at org.neo4j.kernel.IndexManagerImpl.getOrCreateNodeIndex(IndexManagerImpl.java:301)
        at org.neo4j.kernel.IndexManagerImpl.forNodes(IndexManagerImpl.java:289)
        at naiad.Indexer.run(Indexer.java:83)
        at naiad.Indexer.main(Indexer.java:30)

From my debugging, it seems that the configuration checking isn't implemented:
    SpatialIndexProvider$SpatialIndexImplementation.configMatches(Map<String,String>, Map<String,String>) line: 87
just returns false without checking anything.


Also, as I was trying to debug this, I noticed that LayerNodeIndex seems to not be deletable:
    LayerNodeIndex.delete() line: 168
immediately returns with no effect.



Am I trying to do something currently unsupported, or have I missed some critical bit of documentation?

My neo4j revision is 1.8
My neo4j-spatial revision is 0.10-SNAPSHOT, built from git on 2012-11-07 (nothing changed since, AFAICT)


Best regards,
Jonathan.

[Jonathan Winterflood]

Anybody got some advice on this?

I've "fixed" the creation problem by correcting

SpatialIndexProvider$SpatialIndexImplementation.configMatches(Map<String,String>, Map<String,String>) line: 87

to actually compare the configuration (a plain .equals() )

I'll do a pull request if anyone's interested.


But then I found that LayerNodeIndex doesn't accept WKT for non-point data, so I fixed that too.


I've now found that attempting to create an invalid index will add the index configuration to the database, but instantiating the LayerNodeIndex throws an error, leaving the invalid configuration in the database.
AFAICT there's no API to delete said index from the database (the only way is to call the (unimplemented) delete() method on the un-instanciable LayerNodeIndex)


I have no fix for this yet, as I just resorted to using a clean database for my perfomance tests (see my upcoming post on that subject.)

Regards,
Jonathan

[Peter Neubauer]

Jonathan,

yes,  apull request on these (together with a test) would be much appreciated!

/peter

Cheers,

/peter neubauer

G:  neubauer.peter
S:  peter.neubauer
P:  +46 704 106975
L:   http://www.linkedin.com/in/neubauer
T:   @peterneubauer

Neo4j 1.8 GA - http://www.dzone.com/links/neo4j_18_release_fluent_graph_literacy.html

--
 
 

[Jonathan Winterflood]

I can do a pull req for the configMatches, as-is (but monday, I'm not at work anymore)

For non-point data, I suggest that the "wkt" key be checked for even when "geometry_type" is absent.
An alternative would be to make geometry_type mandatory. Thoughts?

For invalid indexes, I have no idea how it _should_ be handled. Anyone know what Lucene indexes do?
I do think, however, that an index object needs to be createable whether it is valid or not, because an index object seems to be required for all operations in neo4j IndexManager.


As for index deletion, I dont see any methods for this in the Index Manager, I'm unsure how it should be done (Both removing the configuration, and cleanly purging the whole RTree would be needed)


I figure these issues should be raised in the trackers on github once they're clarified a bit.


Is there an official contract that index implementations should follow, documented in neo4j (or is someone responsible for that aspect of the API following this?)


Regards,
Jonathan

--
Jonathan

--
 
 

[Jonathan Winterflood]

Cross-reference:

Spatial index (LayerNodeIndex) cannot be retrieved with IndexManager.forNodes(indexName, customConfiguration)
https://github.com/neo4j/spatial/issues/68
https://github.com/neo4j/spatial/pull/77 (Testcase & fix)

LayerNodeIndex only supports Point[lat/long, WKT]
https://github.com/neo4j/spatial/issues/69
https://github.com/neo4j/spatial/pull/76 (Testcase & proposed fix)

Spatial indexes cannot be deleted
https://github.com/neo4j/spatial/issues/70
https://github.com/neo4j/spatial/pull/75 (Testcase only)

Creating a spatial index with invalid configuration leaves it in the database, rendering that index name unuseable.
https://github.com/neo4j/spatial/issues/71
https://github.com/neo4j/spatial/pull/74 (Testcase only)

Cheers,
Jonathan

[Craig Taverner]

Thanks Jonathan. I will try take a look at these on the weekend.

--