Create/Update complex index

[Matthew Young]

Ashish Dalela took up the question of pushing/pulling complex beans in Neo4j (his "Nested properties" question)....

I wrote a bit in that question but decided to create a new forum question with on slightly different angle.  I have the same problem.  I want to save beans.  The Spring Neo4j is great.  But beans may not always be unique only off a single or multiple simple properties.  When I write "simple" I mean simple types.  Beans can have complex properties (other beans).

For example, let's say I want to represent Software as a Bean and save it down in Neo4j with Spring Neo4j.  The Software Bean has stuff like a name and a version and bunch of other simple properties but also an Organization property that is complex (another bean).  Organizations are themselves valid candidates a complex beans I want to save in Neo4j.  Software is unique in my world based on the following properties:

    Software.name

    Software.version

    Software.Organization.name

where the jump between Software and Organization with be a relation.  So uniqueness needs to be a conglomeration of different fields.  Not problem in Lucene.  That is the whole idea of a Document/Field (multiple) indexing.  And the putIfAbsent methods on the Index implementation allow me to construct an index which a Value Object array (made up of the values in my properties regardless where they come from).  And the key can be whatever I want (like "Software unique index 1").  This won't be a searchable index only an index used for locking when creating/updating.

Haven't done an example but in theory this should work.  The integration with Spring Neo4j is the sticky part.  The @Index annotation is for a single property.  I want to override/extend the Index annotation or integrate something else so that save/create methods of the Neo4jTemplate work off my Index annotation that populates the Value with my fields.  Ideas?

[Matthew Young]

Thinking about extending the DelegatingGraphDayabasr and overriding

the getOrAndCreate methods with my own UniqueFactory. Thing is that not sure if this will effect all save and create calls from the template? The getOrCreate seem to lead back to the EntityPersistor implementations.

[Michael Hunger]

Matthew,

if you use graphDatabase.getOrCreateNode you can use your own key and value for the value used for the unique lookup.

Another (more hacky) version would be to create an artificial field that would contain the aggregated information and is @Index(unique=true) and you update it whenever one of the relevant fields changes?

Then you would even get constraint violation exceptions if one of your software instances would clash with another one, e.g. due to a incorrect version or organisation change.

It should even work if you make this field transient, so that it is only used for the index-lookup and not stored in the graph. It would be great if you could check that out.

Thanks

Michael

[Matthew Young]

Interesting exit with the hacky alternative. ;-). Will look into the override idea of the getOrCreate methods. This would be more straight forward if these methods are used later on by the entity persisters. Will step w the debugger on monday and get back.

[Matthew Young]

Looked into different alternatives today. Ran into a stumbling block
right from the get-go in the annotations can't be extended in Java.
Wow, forget little stuff like that fast these days. So I can't just
extend the @Indexed annotations to allow for more than one field.
Plus there is the fact that if I looked at the Spring Neo4j code with
a patient eye I would have noticed that getOrCreateNode operates only
on a particular field in relation to the id property and nothing else
like the actual bean is available once inside the UniqueFactory
getOrCreate method. So I would have needed to consolidate my
composite keys into a single field like Michael suggested in his
"hacky" alternative anyways. Also I learned that copying over the
remaining non-unique properties is done in a separate transaction
inside the SourceStateTransmitter.copyPropertiesTo method. Little
confused here since the LuceneIndex code is the one that takes the
lock so not sure if this will have side-effect for what I am planing
on doing which is....

Exactly what Michael suggested. Inside a TransactionEventHandler I
will consolidate my composite keys into a single field that the
@Indexed is placed on inside the beforeCommit method (gives me access
to all of the nodes). This gives me uniqueness, hopefully.

What I am missing now is a way to integrate "version" optimistic
locking to cancel out overwrites. To that I need access to the
existing node if I am doing an update. Haven't looked carefully but
gut feelings is the old state isn't going to be readily accessible
without a degree of overriding.

[Matthew Young]

After a cup of coffee the whole version thing shouldn't be a huge deal
either. The beforeCommit gives me the current value and this should
be enough for now (not sure if another trans can work it's way in
since no lock is held except on the index).

[Michael Hunger]

The indexing property change listener should take care of overrides, it blows up (your tx) when you get back a different node for the current value (with putIfAbsent)

also you should create a tx spanning the whole operation, so that both, modifications as well as final persisting is handled in a single tx.

Michael

P.S. nested tx participate in the main tx in neo4j, they only affect the main tx with rollback.

[Matthew Young]

@Michael, the hacky idea ran into a speed bump. Thing is that I need
to apply an index name by the type of Node being saved. If I apply
@Indexed on an abstract class the default name on extended classes is
the class name of the base class. Otherwise to get a unique name per
concrete class I have to duplicate the @Indexed annotation everywhere
and hard code the index name. This would work but I would like a
better (more flexible) solution.

Thought about tricking the annotated configuration and overriding
certain beans like the EntityStateHandler (among other to handle a new
unique cass) so I could redo the createUniqueNode class. Was going to
create a new annotation called @CompositeIndexed at the TYPE target
and allow for multiple field names (using the BeanWrapper to handle
nested properties). Problem is that the post construction of the
MappingInfrastructure used a final EntityStateHandler when building
the Persister and hands over the EntityStateHandler to the converter
eventually calling the useOrCreateState method. The last method is
where I want to hook in stuff. Looking for my composite annotation
and so forth (to later adjust the UniqueFactory to handle multiple
fields blah blah blah).

Either I fork off the Neo4j project and try to integrate my stuff that
way or continue on down the "hacky" line. Have you'll thought about
complex/composite indexes? Guess what I am asking is if I fork and
bone out such an index if it would fit in to future plans with unique
indexes? Will post back what I do with the "hacky" alternative.

[Michael Hunger]

What about indexed(type=INSTANCE) that should do what you want

Sent from mobile device