Re: [Neo4j] Batch Importer Issue -> exception: bad value 'none' or 'soft' or 'weak' for setting cache_type

[Peter Neubauer]

Matthias,
can you try with cache_type=none

? The batchinserter is not using caches.

/peter

Cheers,

/peter neubauer

G: neubauer.peter
S: peter.neubauer
P: +46 704 106975
L: http://www.linkedin.com/in/neubauer
T: @peterneubauer

Neo4j 1.9 GA is out -
http://blog.neo4j.org/2013/05/neo4j-19-general-availability.html
Neo4j questions? Please use SO - http://stackoverflow.com/search?q=neo4j


On Thu, May 23, 2013 at 7:13 AM, <matthias...@googlemail.com> wrote:
>
> Hello,
>
> i am pretty new to neo4j and want to use the batch importer. So far it
> worked all fine, but the execution of the batch-importer failed due to the
> cache_type setting.
> It recommends gcr settings, but these are only available within the
> enterprise edition.
>
> Is there a way to overcome this issue. Help is appreciated, thanks.
>
> Below are further information that might help to solve the problem.
>
> System-Info:
> Win 7 32bit, 4G (3G usable), JRE 7
>
> Batch.properties:
> dump_configuration=false
> cache_type=weak
> use_memory_mapped_buffers=true
> neostore.propertystore.db.index.keys.mapped_memory=5M
> neostore.propertystore.db.index.mapped_memory=5M
> neostore.nodestore.db.mapped_memory=200M
> neostore.relationshipstore.db.mapped_memory=500M
> neostore.propertystore.db.mapped_memory=200M
> neostore.propertystore.db.strings.mapped_memory=200M
>
> Graph Data:
> very small sample graph
>
> Nodes
> id label
> 1 test1
> 2 test2
> 3 test3
> 4 test4
> 5 test5
> 6 test6
> 7 test7
> 8 test8
> 9 test9
> 13 test10
> 23 test11
> 34 test12
> 56 test13
>
> Rels:
> start end type status
> 1 2 directed ok
> 1 3 directed ok
> 1 4 directed n.a
> 2 3 directed ok
> 2 5 directed ok
> 2 7 directed ok
> 3 5 directed ok
> 4 6 directed ok
> 4 7 directed ok
> 5 7 directed n.a
> 5 8 directed n.a
> 5 9 directed n.a
> 5 13 directed n.a
> 5 23 directed n.a
> 6 34 directed n.a
> 6 56 directed n.a
>
> Error:
>
> C:\Daten\Studium\LV HU Berlin\SS 2013\Datamanagement and BI\Neuer
> Ordner>java -server -Xmx1G -jar target\batch
> -import-jar-with-dependencies.jar target\db nodes.csv rels.csv
> Using Existing Configuration File
> Exception in thread "main" java.lang.IllegalArgumentException: Bad value
> 'weak' for setting 'cache_type': must
> be one of [gcr]
> at
> org.neo4j.helpers.Settings$DefaultSetting.apply(Settings.java:788)
> at
> org.neo4j.helpers.Settings$DefaultSetting.apply(Settings.java:708)
> at
> org.neo4j.graphdb.factory.GraphDatabaseSetting$SettingWrapper.apply(GraphDatabaseSetting.java:215)
> at
> org.neo4j.graphdb.factory.GraphDatabaseSetting$SettingWrapper.apply(GraphDatabaseSetting.java:189)
> at
> org.neo4j.kernel.configuration.ConfigurationValidator.validate(ConfigurationValidator.java:50)
> at
> org.neo4j.kernel.configuration.Config.applyChanges(Config.java:121)
> at org.neo4j.kernel.configuration.Config.<init>(Config.java:89)
> at org.neo4j.kernel.configuration.Config.<init>(Config.java:79)
> at
> org.neo4j.unsafe.batchinsert.BatchInserterImpl.<init>(BatchInserterImpl.java:83)
> at
> org.neo4j.unsafe.batchinsert.BatchInserterImpl.<init>(BatchInserterImpl.java:67)
> at
> org.neo4j.unsafe.batchinsert.BatchInserters.inserter(BatchInserters.java:60)
> at
> org.neo4j.batchimport.Importer.createBatchInserter(Importer.java:40)
> at org.neo4j.batchimport.Importer.<init>(Importer.java:26)
> at org.neo4j.batchimport.Importer.main(Importer.java:54)
>
> --
> You received this message because you are subscribed to the Google Groups
> "Neo4j" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to neo4j+un...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[matthias...@googlemail.com]

Hi Peter,

I tried it with all cache_type settings including "none". The problem is the same. It still requires a type of gcr.

[Michael Hunger]

Never seen that.

Can you describe the steps you took to get there?

Sent from mobile device

[matthias...@googlemail.com]

I just followed the steps, described here:

http://maxdemarzi.com/2012/02/28/batch-importer-part-1/
https://github.com/jexp/batch-import

is there a difference that I might have overseen?

I got a successful build with maven:

[INFO] ------------------------------------------------------------------------
[INFO] Building Neo4j Batch Importer
[INFO]    task-segment: [clean, compile, assembly:single]
[INFO] ------------------------------------------------------------------------
[INFO] [clean:clean {execution: default-clean}]
[INFO] Deleting directory C:\Daten\Studium\LV HU Berlin\SS 2013\Datamanagement and BI\Neuer Ordner\target
[INFO] [resources:resources {execution: default-resources}]
[WARNING] Using platform encoding (Cp1252 actually) to copy filtered resources, i.e. build is platform depende
nt!
[INFO] Copying 1 resource
[INFO] [compiler:compile {execution: default-compile}]
[WARNING] File encoding has not been set, using platform encoding Cp1252, i.e. build is platform dependent!
[INFO] Compiling 43 source files to C:\Daten\Studium\LV HU Berlin\SS 2013\Datamanagement and BI\Neuer Ordner\t
arget\classes
[INFO] [assembly:single {execution: default-cli}]
[INFO] Processing DependencySet (output=)
[INFO] Building jar: C:\Daten\Studium\LV HU Berlin\SS 2013\Datamanagement and BI\Neuer Ordner\target\batch-imp
ort-jar-with-dependencies.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 52 seconds
[INFO] Finished at: Thu May 23 14:57:46 CEST 2013
[INFO] Final Memory: 19M/70M
[INFO] ------------------------------------------------------------------------

[Peter Neubauer]

Mattias,
could you please execute the batchimporter with mvn exec:java from the
batchimport maven project instead of from an uber-jar? I am suspecting
your compression might overwrite and destroy some of the manifests in
the ingoing jars, which lead to these subte errors.

http://mojo.codehaus.org/exec-maven-plugin/usage.html

/peter

Cheers,

/peter neubauer

G: neubauer.peter
S: peter.neubauer
P: +46 704 106975
L: http://www.linkedin.com/in/neubauer
T: @peterneubauer

Neo4j 1.9 GA is out -
http://blog.neo4j.org/2013/05/neo4j-19-general-availability.html
Neo4j questions? Please use SO - http://stackoverflow.com/search?q=neo4j

[matthias...@googlemail.com]

Hi Peter,

can you please provide the exact name of the main-class that i have to put in?
mvn exec:java -Dexec.mainClass"???"

Do I need any arguments for to execute it?

[Peter Neubauer]

Try something like

mvn clean compile exec:java
-Dexec.mainClass="org.neo4j.batchimport.Importer"
-Dexec.args="target/db nodes.csv rels.csv"

/peter

Cheers,

/peter neubauer

G: neubauer.peter
S: peter.neubauer
P: +46 704 106975
L: http://www.linkedin.com/in/neubauer
T: @peterneubauer

Neo4j 1.9 GA is out -
http://blog.neo4j.org/2013/05/neo4j-19-general-availability.html
Neo4j questions? Please use SO - http://stackoverflow.com/search?q=neo4j

[matthias...@googlemail.com]

Hi,

thanks, almost there. Now it creates all nodes but failed when it should create the relationship. The following error occurs after creating the nodes:

Importing 601 Nodes took 0 seconds

Total import time: 2 seconds
[WARNING]
java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:601)
        at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:297)
        at java.lang.Thread.run(Thread.java:722)
Caused by: org.neo4j.graphdb.NotFoundException: id=1416
        at org.neo4j.unsafe.batchinsert.BatchInserterImpl.getNodeRecord(BatchInserterImpl.java:882)
        at org.neo4j.unsafe.batchinsert.BatchInserterImpl.createRelationship(BatchInserterImpl.java:436)
        at org.neo4j.batchimport.Importer.importRelationships(Importer.java:108)
        at org.neo4j.batchimport.Importer.main(Importer.java:63)
        ... 6 more
[INFO] ------------------------------------------------------------------------
[ERROR] BUILD ERROR
[INFO] ------------------------------------------------------------------------
[INFO] An exception occured while executing the Java class. null

id=1416
[INFO] ------------------------------------------------------------------------
[INFO] For more information, run Maven with the -e switch
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 11 seconds
[INFO] Finished at: Mon May 27 13:35:30 CEST 2013
[INFO] Final Memory: 24M/62M
[INFO] ------------------------------------------------------------------------

[Michael Hunger]

The id's in the start and end columns of the relationships-file refer to the line numbers in the node's files, starting with 1 (as the 0th node is already in the db).

Looks like a node at the end. You only created 601 nodes so it should point at most to id 601.

Michael

Cheers,

Michael

(you)-[:GET]->(graphconnect-ticket)<-[:DISCOUNT {value:25%}]-(NEO4GC)