WriteTransaction.getPropertyIndexRecord NullPointerException

[Balazs E. Pataki]

Hi,


I recently upgraded from 1.4.2 to 1.7. It seems that something is broken
between 1.5.2 and 1.6.2, becuase I have a batch insert program, which
worked fine until 1.5.2 (broken with 1.6.2 and 1.7.0). More precisely:
the inserting is OK, but when I try to traverese the whole database I
get the exceeption.

I have an NPE at this line of my code (with neo4j 1.6.2):

Iterator<String> it = nodeOrRel.getPropertyKeys().iterator();


And the stack trace goes like this:

java.lang.NullPointerException
at
org.neo4j.kernel.impl.nioneo.xa.WriteTransaction.getPropertyIndexRecord(WriteTransaction.java:1589)
at
org.neo4j.kernel.impl.nioneo.xa.WriteTransaction.loadIndex(WriteTransaction.java:1433)
at
org.neo4j.kernel.impl.persistence.PersistenceManager.loadIndex(PersistenceManager.java:92)
at
org.neo4j.kernel.impl.core.PropertyIndexManager.getIndexFor(PropertyIndexManager.java:141)
at org.neo4j.kernel.impl.core.NodeManager.getIndexFor(NodeManager.java:951)
at org.neo4j.kernel.impl.core.Primitive.getPropertyKeys(Primitive.java:130)
at org.neo4j.kernel.impl.core.NodeImpl.getPropertyKeys(NodeImpl.java:54)
at org.neo4j.kernel.impl.core.NodeProxy.getPropertyKeys(NodeProxy.java:141)


Unfortunately I cannot provide a simple test case to reveal the bug,
because if I just insert 2 or 3 graphs then the next traversal will be
OK, but when I inserts 5000 graphs I get this exception.

Do you have any ideas what could have gone wrong? Could it be related to
the Lucene 3.1->3.5 upgrade?

Thanks in advance,
---
balazs

[Mattias Persson]

Thanks for reporting this bug. I found, reproduced and fixed it. See https://github.com/neo4j/community/commit/3e75a76bb7ad1fc0caaa36f16ab59c1c50c86de8

2012/5/3 Balazs E. Pataki <pat...@dsd.sztaki.hu>

--
Mattias Persson, [mat...@neotechnology.com]
Hacker, Neo Technology
www.neotechnology.com

[Balazs E. Pataki]

Any idea regarding this? It really bugs me. I thought it might be
related to inappropriate comitting of session, or some mixup of
transactions in multiple threads. I tried with a single thread, and the
NPE still happens and the transactions seem to be committed alright. It
happens really deep in the neo4j kernel so I cannot see what I could do
wrong in my code that could cause this property index related bug.

Thanks for any hints,
---
balazs

[Balazs E. Pataki]

In the meantime, I figured out the following:

- In 1.5.2 in WriteTransaction's propIndexRecords member has been
declared and initialized like this:

private final Map<Integer,PropertyIndexRecord> propIndexRecords =
new HashMap<Integer,PropertyIndexRecord>();

- In 1.6.2, however, it is declared but is not initialized at
construction time:

private Map<Integer,PropertyIndexRecord> propIndexRecords;

Although I don't know what 'propIndexRecords' is for but if it not
initialized somewhere in the 1.6.2 WriteTransaction implementation then
it will be definitely null in

PropertyIndexRecord getPropertyIndexRecord( int id )
{
return propIndexRecords.get( id );
}

So, the hack I tried is this in 1.6.2:

PropertyIndexRecord getPropertyIndexRecord( int id )
{
if (propIndexRecords == null) {
return null;
}
return propIndexRecords.get( id );
}

And now it seems to work for me as is did in 1.5.2 and earlier. It maybe
OK to return null here, since getPropertyIndexRecord() is invoked from
WriteTransaction#loadIndex() like this:


@Override
public String loadIndex( int id )
{
PropertyIndexStore indexStore = getPropertyStore().getIndexStore();
PropertyIndexRecord index = getPropertyIndexRecord( id );
if ( index == null )
{
index = indexStore.getRecord( id );
}
if ( index.isLight() )
{
indexStore.makeHeavy( index );
}
return indexStore.getStringFor( index );
}

If getPropertyIndexRecord() returns null, then indexStore.getRecord()
will create the index, as far as I understand.

But still, these are only my guesses and would be glad to have some
neo4j kernel expert to help me out. :-)

Best regards,
---
balazs

[Peter Neubauer]

Balazs,
I think Mattias found the bug and fixed it earlier in this thread?

Cheers,

/peter neubauer

G:  neubauer.peter
S:  peter.neubauer
P:  +46 704 106975
L:   http://www.linkedin.com/in/neubauer
T:   @peterneubauer

If you can write, you can code - @coderdojomalmo
If you can sketch, you can use a graph database - @neo4j

[Balazs E. Pataki]

Oh, yes, sorry, I missed that. And he seems to have the same fix I came
up with. Great! :-)

Thanks again!

Regards,
---
balazs

[Mattias Persson]

In the meantime (if you can't wait for the next milestone) I'm sure this workaround does the trick:

* Start up your graph db
* Loop through all your property keys that your db has while not inside a write transaction
* Start using your db as usual

2012/5/4 Balazs E. Pataki <pat...@dsd.sztaki.hu>