Titan to JanusGraph migration Elasticsearc| "janusgraph" index is created and used , not titan

[akshay...@gmail.com]

Refer https://github.com/JanusGraph/janusgraph/issues/228 for other related issue with migration. 

It seems the default name of elastic search index is also changed from 'titan' to  'janusgraph'. 

"index.search.index-name" property name option should help. 

However I am afraid its not working. I am using BerkeleyDB+Embedded ES option.  

1.  I created the DB with Titan. 

`storage.backend=berkeleyje

storage.directory=data/titan-janus/berkeley

index.search.backend=elasticsearch

index.search.directory=data/titan-janus/es

index.search.elasticsearch.client-only=false

index.search.elasticsearch.local-mode=true

`

2. At this point titan index is created. http://localhost:9200/titan returns index definition.  http://localhost:9200/janusgraph/ returns IndexMissingException. 

3. Data is created in titan index. 

4. Now I migrate the code to Janus. I thought index-name should work and added - 

'index.search.index-name=titan'

5. Now when I start , storage backend works seamlessly. I can query nodes added with Titan. And new nodes are added. 

6. The index backend does not work well. "janusgraph" index is created ( even when index-name is titan). The new nodes never go to "titan" , they are created in "janusgraph" index. 

Akshya

[Jason Plurad]

Hi Akshya,

Thanks for the info, and unfortunately it looks like some scenarios were missed. Note that if you created the Titan graph originally with the `index.search.index-name` property set, it would work without additional configuration in JanusGraph.

In your scenario, when opening the graph in JanusGraph with  `index.search.index-name=titan`, this warning is shown:

WARN  org.janusgraph.graphdb.configuration.GraphDatabaseConfiguration  - Local setting index.search.index-name=titan (Type: GLOBAL_OFFLINE) is overridden by globally managed value (janusgraph).  Use the ManagementSystem interface instead of the local configuration to control this setting.

The additional work you need to do is:

mgmt = graph.openManagement()
mgmt.set('index.search.index-name', 'titan')
mgmt.commit()
// at this point the graph is closed, so reopen it with the same properties

[Akshaya Rawat]

Hi Jason,
Thanks for quick response.
Will try this and it should work.

Thanks
Akshaya

[Akshaya Rawat]

Hi Jason, 

I tried to do that. 

JanusGraphManagement janusGraphManagement=janusGraph.openManagement();

janusGraphManagement.set("index.search.index-name", "titan");

janusGraphManagement.commit();

But I got error - 

Caused by: java.lang.IllegalArgumentException: Cannot change offline config option [root.index.index-name] since multiple instances are currently open: [c0a838679548-NOIARAWA3569221, c0a8386711772-NOIARAWA3569221, c0a8386712068-NOIARAWA3569221, c0a8386712596-NOIARAWA3569221]

at com.google.common.base.Preconditions.checkArgument(Preconditions.java:145)

at org.janusgraph.graphdb.database.management.ManagementSystem$1.verifyModification(ManagementSystem.java:180)

at org.janusgraph.diskstorage.configuration.UserModifiableConfiguration.set(UserModifiableConfiguration.java:99)

at org.janusgraph.graphdb.database.management.ManagementSystem.set(ManagementSystem.java:1316)

If I close the graph, then set , I get another error - 

janusGraph.close();

JanusGraphManagement janusGraphManagement=janusGraph.openManagement();

janusGraphManagement.set("index.search.index-name", "titan");

janusGraphManagement.commit();

Caused by: java.lang.IllegalStateException: Environment is closed.

at com.sleepycat.je.Environment.getNonNullEnvImpl(Environment.java:2496)

at com.sleepycat.je.Environment.checkOpen(Environment.java:2472)

at com.sleepycat.je.Environment.openDatabase(Environment.java:643)

at org.janusgraph.diskstorage.berkeleyje.BerkeleyJEStoreManager.openDatabase(BerkeleyJEStoreManager.java:185)

at org.janusgraph.diskstorage.berkeleyje.BerkeleyJEStoreManager.openDatabase(BerkeleyJEStoreManager.java:48)

at org.janusgraph.diskstorage.keycolumnvalue.keyvalue.OrderedKeyValueStoreManagerAdapter.openDatabase(OrderedKeyValueStoreManagerAdapter.java:89)

at org.janusgraph.diskstorage.keycolumnvalue.keyvalue.OrderedKeyValueStoreManagerAdapter.openDatabase(OrderedKeyValueStoreManagerAdapter.java:38)

at org.janusgraph.diskstorage.log.kcvs.KCVSLogManager.openLog(KCVSLogManager.java:225)

at org.janusgraph.diskstorage.Backend.getSystemMgmtLog(Backend.java:337)

at org.janusgraph.graphdb.database.StandardJanusGraph.openManagement(StandardJanusGraph.java:303)

I am not sure how to move ahead. 

Thanks

Akshaya

[Jason Plurad]

The first error message said that there were multiple open instances. In order to close them, again you must work through the management system (do not close the graph you are already connect to).

Try something like this:

mgmt = graph.openManagement()
open = mgmt.openInstances() // returns a set of unique instance ids
for (String inst : open) { if (!inst.endsWith("(current)")) mgmt.forceCloseInstance(inst) }

mgmt.set('index.search.index-name', 'titan')
mgmt.commit()

https://github.com/JanusGraph/janusgraph/blob/v0.1.0/janusgraph-core/src/main/java/org/janusgraph/core/schema/JanusGraphManagement.java#L364

[Akshaya Rawat]

This solutions worked for me. Commit had to be done after the force closure. 

JanusGraphManagement jMgmt = jGraph.openManagement();

Set<String> open = jMgmt.getOpenInstances();

for (String string : open) {

if (!string.endsWith("(current)")) {

jMgmt.forceCloseInstance(string);

} 

}

jMgmt.commit();

jMgmt = jGraph.openManagement();

jMgmt.set("index.search.index-name", "titan");

jMgmt.commit();

The graph has to be initialized all over again after this since its closed after this. 

This looks to be disruptive solution though the graph is closed and has to be initialized all over again. However this only has to be done once. Once this configuration is forced, it persists. So this could be done as one time migration may be using a script and then it works then on. 

Thanks for the solution. 

Akshaya

[Akshaya Rawat]

I tried this with Cassandra + Embedded Elastic configuration. It works there as well, however one has to add configuration to use "titan" keyspace name. 

storage.cassandra.keyspace=titan

In this case there is not need to force configuration like it needs to be done for Elastic search index name. 

Just checking if there is a good reason for different approaches. 

Akshaya