Titan 0.5 does not work with storage.batch-loading=true ?

[Yuriy]

I've built Titan 0.5 and tried bulk loading data into it.

Unfortunately, it keeps throwing on commit.

 

Here is how I built it:

on Windows 8.1 x64:

git clone -b titan05 https://github.com/thinkaurelius/titan.git

mvn clean install -DskipTests=true

 

Then I used gremlin.bat from titan\bin.

 

 

The simplest repro I came up with is (backend does not matter):

 

conf = new BaseConfiguration();

conf.setProperty("storage.directory", "/tmp/graph1");

conf.setProperty("storage.backend", "berkeleyje");

conf.setProperty('storage.batch-loading','true');

conf.setProperty('autotype', 'none');

g = TitanFactory.open(conf);

  

g.makeEdgeLabel('linked').make();

 

 

 v = g.addVertex();

 u = g.addVertex();

 g.addEdge(v,u,'linked');

g.commit();

 

Then I get:

 

3    [main] ERROR com.thinkaurelius.titan.graphdb.database.StandardTitanGraph  - Could not commit transaction [1] due to exception

java.lang.IllegalArgumentException: Attempting to create schema elements in inconsistent state

        at com.google.common.base.Preconditions.checkArgument(Preconditions.java:93)

        at com.thinkaurelius.titan.graphdb.database.StandardTitanGraph.commit(StandardTitanGraph.java:493)

        at com.thinkaurelius.titan.graphdb.transaction.StandardTitanTx.commit(StandardTitanTx.java:1213)

        at com.thinkaurelius.titan.graphdb.blueprints.TitanBlueprintsGraph.commit(TitanBlueprintsGraph.java:60)

        at com.tinkerpop.blueprints.TransactionalGraph$commit.call(Unknown Source)

        at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)

        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)

        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:112)

        at groovysh_evaluate.run(groovysh_evaluate:57)

        at groovysh_evaluate$run.call(Unknown Source)

        at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)

        at groovysh_evaluate$run.call(Unknown Source)

        at org.codehaus.groovy.tools.shell.Interpreter.evaluate(Interpreter.groovy:67)

        at org.codehaus.groovy.tools.shell.Interpreter$evaluate.call(Unknown Source)

        at org.codehaus.groovy.tools.shell.Groovysh.execute(Groovysh.groovy:152)

        at org.codehaus.groovy.tools.shell.Shell.leftShift(Shell.groovy:114)

        at org.codehaus.groovy.tools.shell.Shell$leftShift$0.call(Unknown Source)

        at org.codehaus.groovy.tools.shell.ShellRunner.work(ShellRunner.groovy:88)

        at org.codehaus.groovy.tools.shell.InteractiveShellRunner.super$2$work(InteractiveShellRunner.groovy)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:606)

        at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)

        at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:233)

        at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1079)

        at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:128)

        at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuper0(ScriptBytecodeAdapter.java:148)

        at org.codehaus.groovy.tools.shell.InteractiveShellRunner.work(InteractiveShellRunner.groovy:100)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:606)

        at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:272)

        at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:52)

        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:137)

        at org.codehaus.groovy.tools.shell.ShellRunner.run(ShellRunner.groovy:57)

        at org.codehaus.groovy.tools.shell.InteractiveShellRunner.super$2$run(InteractiveShellRunner.groovy)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:606)

        at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)

        at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:233)

        at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1079)

        at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:128)

        at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuper0(ScriptBytecodeAdapter.java:148)

        at org.codehaus.groovy.tools.shell.InteractiveShellRunner.run(InteractiveShellRunner.groovy:66)

        at com.thinkaurelius.titan.tinkerpop.gremlin.Console.<init>(Console.java:58)

        at com.thinkaurelius.titan.tinkerpop.gremlin.Console.<init>(Console.java:71)

        at com.thinkaurelius.titan.tinkerpop.gremlin.Console.main(Console.java:97)

Could not commit transaction due to exception during persistence

Display stack trace? [yN]

 

Note, that there is a new API to make edges in 0.5 - g.makeEdgeLabel('linked') and I might be using it incorrectly, but in EdgeSerializerTest.java this API is used in similar way...

I tried committing after creating an edge () because your blog says "Always commit at the end of type creation and before loading data to the graph instance" but then the code throws with exactly the same error after the commit there.

Removing conf.setProperty('storage.batch-loading','true') fixes this problem, but I need bulk loading...

 

Please let me know if this is just a bug or am I doing something wrong.

 

Thanks,

Yuriy

 

[Matthias Broecheler]

We won't allow schema changes (which includes creating edge labels, etc) during batch-loading transactions because those disable locking which can cause schema inconsistencies. That's been the case in Titan 0.4.x as well but is more strictly enforced now.

Hence, either

1) create your schema up front

2) use separate transaction that don't have batch-loading enabled to create the schema on the fly (use the transaction builder with buildTransaction() on graph)

In general, give us one more week to wrap up the M1 release of 0.5 - there are still some loose ends we are patching up.

Thanks,

Matthias

--
You received this message because you are subscribed to the Google Groups "Aurelius" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aureliusgraph...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Matthias Broecheler
http://www.matthiasb.com

[Yuriy]

Thanks Matthias

I got it to work.

If someone else is curious about more details:

1) I connected to Titan in non-bulk insert mode (conf.setProperty('storage.batch-loading','false') ), created the edge label, committed the change and shutdown the graph.

2) Then I connected again with conf.setProperty('storage.batch-loading','true') and inserted the data - the insertion of 10 millions of edges succeeded.