Re: [Neo4j] unable to close store using batchinserter

[Peter Neubauer]

What neo4j version are you doing this on, Chen?

Cheers,

/peter neubauer

Neo4j 1.8.RC1 "Vindeln Vy" -
http://blog.neo4j.org/2012/09/neo4j-18rc1-really-careful-ftw.html

G: neubauer.peter
S: peter.neubauer
P: +46 704 106975
L: http://www.linkedin.com/in/neubauer
T: @peterneubauer

Wanna learn something new? Come to @graphconnect.


On Sun, Sep 23, 2012 at 7:38 PM, bin chen <binchen...@gmail.com> wrote:
> Hi, I got the following error while using batchinserter. It works well for a
> small set but does not work for a larger test set consisting of millions of
> relationships. I suspected it has something to do with the memory, but
> havenot fixed it. Any hint would be appreciated.
>
> Exception in thread "main"
> org.neo4j.kernel.impl.nioneo.store.UnderlyingStorageException: Unable to
> close store D:\....\data\test\neo4j-store1\neostore.relationshipstore.db
> at
> org.neo4j.kernel.impl.nioneo.store.CommonAbstractStore.close(CommonAbstractStore.java:699)
> at
> org.neo4j.kernel.impl.nioneo.store.RelationshipStore.close(RelationshipStore.java:79)
> at
> org.neo4j.kernel.impl.nioneo.store.NeoStore.closeStorage(NeoStore.java:231)
> at
> org.neo4j.kernel.impl.nioneo.store.CommonAbstractStore.close(CommonAbstractStore.java:634)
> at
> org.neo4j.unsafe.batchinsert.BatchInserterImpl.shutdown(BatchInserterImpl.java:700)
> at com.linkingpilot.app.BatchImporterImpl.finish(BatchImporterImpl.java:134)
> at com.linkingpilot.app.ImporterImpl.buildNewDb(ImporterImpl.java:47)
> at com.linkingpilot.app.ImporterTest.main(ImporterTest.java:35)
> Caused by: java.io.IOException: The requested operation cannot be performed
> on a file with a user-mapped section open
> at sun.nio.ch.FileChannelImpl.truncate0(Native Method)
> at sun.nio.ch.FileChannelImpl.truncate(FileChannelImpl.java:340)
> at
> org.neo4j.kernel.impl.nioneo.store.CommonAbstractStore.close(CommonAbstractStore.java:669)
> ... 7 more
>
> Thanks.
>
> --
>
>

[bin chen]

I am using 1.8.RC1. Thanks.

Best, Bin

[bin chen]

Hi, Peter,  here is an observation that sounds quite strange. If I am using the debug model and set a breaking point at the db.shutdown(), it works surprisingly...Any clue?

[Jacob Hansson]

Hey Bin,

does this happen every time you try it? Would you be able to provide a test case that exhibits this behavior?

/jake

[Mattias Persson]

There is a known problem on Windows regarding releasing memory mapped areas of a file relying on garbage collection as there's no public* way of forcing such a release. This effectively means that neo4j races against the GC and sometimes such things happen. It's interesting that you can produce it so often though.

* There are ways, but probably not supported on all jvms and platforms.

2012/9/27 Jacob Hansson <jacob....@neotechnology.com>

--
 
 

--
Mattias Persson, [mat...@neotechnology.com]
Hacker, Neo Technology
www.neotechnology.com

[bin chen]

Thanks.

I modified the batchinserter class and added some other functions (e.g., indexing) there. It happened very often when I was using a larger dataset (see 4m edges), but it may sometimes work if the dataset has hundreds of thousands edges.... my pc has 4G ram, running Windows XP...

Bin

--
 
 

--
Bin Chen
PHD student, Informatics,
Indiana University at Bloomington
http://cheminfo.informatics.indiana.edu/~binchen

[matthias...@googlemail.com]

Hi Bin,

I am currently facing the same issue on my Win7 32bit, 4G Ram system with Neo4j-community 1.9. It sounds that you found a solution by adjusting the batchinserter class. Can you please provide the changes that make it work, thanks.