how to specify Es instance in securegraph

125 views
Skip to first unread message

mohit kaushik

unread,
Oct 6, 2014, 2:18:52 AM10/6/14
to lum...@googlegroups.com
Hi,

First, I want to ask, Is this the right place to ask queries about Securegraph. Is there any seperate group for securegraph?

I am using securegraph and I want use my Es instance. I have tried setting properties in ElasticSearchSearchIndexBase.java like Es_locations and index_name but no effect. What else i need to do? and how can i specify it in program?


thanks

David Singley

unread,
Oct 8, 2014, 9:03:41 AM10/8/14
to lum...@googlegroups.com
The Elasticsearch server(s) and index used by Securegraph is configured via the .properties file with the following options:

graph.search
=org.securegraph.elasticsearch.ElasticSearchParentChildSearchIndex
graph.search.locations=192.168.33.10
graph.search.indexName=securegraph

see https://github.com/lumifyio/lumify/blob/master/config/lumify.properties#L96

mohit kaushik

unread,
Oct 10, 2014, 12:26:37 AM10/10/14
to lum...@googlegroups.com
Thanks David, but i don not find any .properties file other than es.plugin.properies in latest securegraph.
Where can i find or create it?

Joe Ferner

unread,
Oct 10, 2014, 10:08:30 AM10/10/14
to lum...@googlegroups.com

mohit kaushik

unread,
Oct 13, 2014, 8:16:44 AM10/13/14
to lum...@googlegroups.com
Thanks Joe,

But I am not tring to build a ES node. I just want it to connect with my existing setup.

let me elaborate.

I have attached a screen shot which shows that even after successfull build of securegraph. It gives error on importing "org.securegraph.elasticsearch.ElasticSearchSearchIndexBase;" so i have to add the project securegraph-elasticsearch-base to securegraph-accumulo. which removes the error but still do not connects it with my ES instance even after setting "cluster-name" and "es_locations".

This might  also be the root of the second problem where i am tring to integrate gremlin with securegraph.
which gives the exception "no node available" (i also posted that as a topic). I am quite sure about the procedure and dependecies version compatibility.
but its not going well.

plz suggest something.

-Mohit
Screenshot.png

Joe Ferner

unread,
Oct 14, 2014, 9:28:07 AM10/14/14
to lum...@googlegroups.com
Can you try running (changing localhost to your ES).


Then try running

curl -XGET 'http://localhost:9200/_search?q=*'

This should give us an indication if ES is working.

mohit kaushik

unread,
Oct 16, 2014, 8:25:02 AM10/16/14
to lum...@googlegroups.com
Joe,This is the output and ES is definitely running.


[root@mohit ~]# curl -XGET 'http://localhost:9200/_cluster/health?pretty=true'
{
  "cluster_name" : "lumify",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 1,
  "number_of_data_nodes" : 1,
  "active_primary_shards" : 5,
  "active_shards" : 5,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0
}
[root@mohit ~]# curl -XGET 'http://localhost:9200/_search?q=*'
{"took":86,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":0,"max_score":null,"hits":[]}}[root@mohit ~]#

mohit kaushik

unread,
Oct 16, 2014, 8:33:01 AM10/16/14
to lum...@googlegroups.com
I am using Accumulo-1.6.1 over cdh4.7.0 and ElasticSearch-1.1.2.........uploaded corrosoponding jar files from lib folders
and getting new ecxeption.......

[root@mohit bin]# ./gremlin.sh gremlin-sg.script

         \,,,/
         (o o)
-----oOOo-(_)-oOOo-----
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/root/Downloads/gremlin-groovy-2.6.0/lib/logback-classic-0.9.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/root/Downloads/gremlin-groovy-2.6.0/lib/slf4j-log4j12-1.7.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
17:47:54.825 [main] INFO  o.s.a.AccumuloGraphConfiguration - Connecting to accumulo instance [lumify] zookeeper servers [192.168.0.121]
org.securegraph.SecureGraphException: Could not create accumulo graph
    at org.securegraph.accumulo.blueprints.AccumuloSecureGraphBlueprintsGraphFactory.createAccumuloGraph(AccumuloSecureGraphBlueprintsGraphFactory.java:25)
    at org.securegraph.accumulo.blueprints.AccumuloSecureGraphBlueprintsGraphFactory.createGraph(AccumuloSecureGraphBlueprintsGraphFactory.java:14)
    at org.securegraph.blueprints.SecureGraphBlueprintsFactory.open(SecureGraphBlueprintsFactory.java:23)
    at org.securegraph.blueprints.SecureGraphBlueprintsFactory$open.call(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
    at groovysh_evaluate.run(groovysh_evaluate:51)
    at groovysh_evaluate$run.call(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:112)
    at org.codehaus.groovy.tools.shell.Interpreter.evaluate(Interpreter.groovy:67)
    at org.codehaus.groovy.tools.shell.Interpreter$evaluate.call(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
    at org.codehaus.groovy.tools.shell.Groovysh.execute(Groovysh.groovy:152)
    at com.tinkerpop.gremlin.groovy.console.Console.initializeShellWithScript(Console.java:87)
    at com.tinkerpop.gremlin.groovy.console.Console.<init>(Console.java:57)
    at com.tinkerpop.gremlin.groovy.console.Console.main(Console.java:102)
Caused by: org.securegraph.SecureGraphException: java.lang.reflect.InvocationTargetException
    at org.securegraph.util.ConfigurationUtils.createProvider(ConfigurationUtils.java:38)
    at org.securegraph.util.ConfigurationUtils.createProvider(ConfigurationUtils.java:14)
    at org.securegraph.GraphConfiguration.createSearchIndex(GraphConfiguration.java:72)
    at org.securegraph.accumulo.AccumuloGraph.create(AccumuloGraph.java:94)
    at org.securegraph.accumulo.AccumuloGraph.create(AccumuloGraph.java:129)
    at org.securegraph.accumulo.blueprints.AccumuloSecureGraphBlueprintsGraphFactory.createAccumuloGraph(AccumuloSecureGraphBlueprintsGraphFactory.java:23)
    ... 20 more
Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
    at org.securegraph.util.ConfigurationUtils.createProvider(ConfigurationUtils.java:24)
    ... 25 more
Caused by: org.securegraph.SecureGraphException: search.192.168.0.121 is a required configuration parameter
    at org.securegraph.elasticsearch.ElasticSearchSearchIndexBase.readConfig(ElasticSearchSearchIndexBase.java:120)
    at org.securegraph.elasticsearch.ElasticSearchSearchIndexBase.<init>(ElasticSearchSearchIndexBase.java:76)
    at org.securegraph.elasticsearch.ElasticSearchSearchIndex.<init>(ElasticSearchSearchIndex.java:27)
    ... 30 more
Exception in thread "main" org.securegraph.SecureGraphException: Could not create accumulo graph
    at org.securegraph.accumulo.blueprints.AccumuloSecureGraphBlueprintsGraphFactory.createAccumuloGraph(AccumuloSecureGraphBlueprintsGraphFactory.java:25)
    at org.securegraph.accumulo.blueprints.AccumuloSecureGraphBlueprintsGraphFactory.createGraph(AccumuloSecureGraphBlueprintsGraphFactory.java:14)
    at org.securegraph.blueprints.SecureGraphBlueprintsFactory.open(SecureGraphBlueprintsFactory.java:23)
    at org.securegraph.blueprints.SecureGraphBlueprintsFactory$open.call(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
    at groovysh_evaluate.run(groovysh_evaluate:51)
    at groovysh_evaluate$run.call(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:112)
    at org.codehaus.groovy.tools.shell.Interpreter.evaluate(Interpreter.groovy:67)
    at org.codehaus.groovy.tools.shell.Interpreter$evaluate.call(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
    at org.codehaus.groovy.tools.shell.Groovysh.execute(Groovysh.groovy:152)
    at com.tinkerpop.gremlin.groovy.console.Console.initializeShellWithScript(Console.java:87)
    at com.tinkerpop.gremlin.groovy.console.Console.<init>(Console.java:57)
    at com.tinkerpop.gremlin.groovy.console.Console.main(Console.java:102)
Caused by: org.securegraph.SecureGraphException: java.lang.reflect.InvocationTargetException
    at org.securegraph.util.ConfigurationUtils.createProvider(ConfigurationUtils.java:38)
    at org.securegraph.util.ConfigurationUtils.createProvider(ConfigurationUtils.java:14)
    at org.securegraph.GraphConfiguration.createSearchIndex(GraphConfiguration.java:72)
    at org.securegraph.accumulo.AccumuloGraph.create(AccumuloGraph.java:94)
    at org.securegraph.accumulo.AccumuloGraph.create(AccumuloGraph.java:129)
    at org.securegraph.accumulo.blueprints.AccumuloSecureGraphBlueprintsGraphFactory.createAccumuloGraph(AccumuloSecureGraphBlueprintsGraphFactory.java:23)
    ... 20 more
Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
    at org.securegraph.util.ConfigurationUtils.createProvider(ConfigurationUtils.java:24)
    ... 25 more
Caused by: org.securegraph.SecureGraphException: search.192.168.0.121 is a required configuration parameter
    at org.securegraph.elasticsearch.ElasticSearchSearchIndexBase.readConfig(ElasticSearchSearchIndexBase.java:120)
    at org.securegraph.elasticsearch.ElasticSearchSearchIndexBase.<init>(ElasticSearchSearchIndexBase.java:76)
    at org.securegraph.elasticsearch.ElasticSearchSearchIndex.<init>(ElasticSearchSearchIndex.java:27)
    ... 30 more
[root@mohit bin]#

I also uploaded the config and script....
Any help...



gremlin-sg.script
gremlin-sg-accumulo.config

Joe Ferner

unread,
Oct 16, 2014, 10:01:58 AM10/16/14
to lum...@googlegroups.com
Did you modify the code? it looks like what used to be the word "locations" is now "192.168.0.121" based on your stacktrace?

mohit kaushik

unread,
Oct 17, 2014, 12:46:01 AM10/17/14
to lum...@googlegroups.com
No, I jsut changed the accumulo version in pom files to 1.6.1. and the ES_LOCATIONS and CLUSTER_NAME in class ElasticSearchSearchIndexBase because I did not find any other way to set cluster name.
Can i set CLUSTER_NAME in config file.

public abstract class ElasticSearchSearchIndexBase implements SearchIndex, DisableEdgeIndexSupport {
    private static final Logger LOGGER = LoggerFactory.getLogger(ElasticSearchSearchIndexBase.class);
    public static final String CONFIG_STORE_SOURCE_DATA = "storeSourceData";
    public static final String CONFIG_ES_LOCATIONS = "192.168.0.121";
    public static final String CONFIG_INDEX_NAME = "indexName";
    private static final String DEFAULT_INDEX_NAME = "securegraph";
    public static final String CONFIG_INDICES_TO_QUERY = "indicesToQuery";
    public static final String CONFIG_IN_EDGE_BOOST = "inEdgeBoost";
    private static final double DEFAULT_IN_EDGE_BOOST = 1.2;
    public static final String CONFIG_OUT_EDGE_BOOST = "outEdgeBoost";
    private static final double DEFAULT_OUT_EDGE_BOOST = 1.1;
    public static final String CONFIG_USE_EDGE_BOOST = "useEdgeBoost";
    private static final boolean DEFAULT_USE_EDGE_BOOST = true;
    public static final String CONFIG_INDEX_EDGES = "indexEdges";
    private static final boolean DEFAULT_INDEX_EDGES = true;
    public static final String ELEMENT_TYPE = "element";
    public static final String ELEMENT_TYPE_FIELD_NAME = "__elementType";
    public static final String VISIBILITY_FIELD_NAME = "__visibility";
    public static final String IN_EDGE_COUNT_FIELD_NAME = "__inEdgeCount";
    public static final String OUT_EDGE_COUNT_FIELD_NAME = "__outEdgeCount";
    public static final String ELEMENT_TYPE_VERTEX = "vertex";
    public static final String ELEMENT_TYPE_EDGE = "edge";
    public static final String SETTING_CLUSTER_NAME = "lumify";
    public static final int DEFAULT_ES_PORT = 9300;
    public static final String EXACT_MATCH_PROPERTY_NAME_SUFFIX = "_exactMatch";
    private final TransportClient client;
    private final boolean autoflush;
    private final boolean storeSourceData;
    private String defaultIndexName;
    private String[] indicesToQuery;
    private final Map<String, IndexInfo> indexInfos = new HashMap<String, IndexInfo>();
    private int indexInfosLastSize = 0; // Used to prevent creating a index name array each time
    private String[] indexNamesAsArray;
    private String[] esLocations;
    private double inEdgeBoost;
    private double outEdgeBoost;
    private boolean useEdgeBoost;
    private boolean indexEdges;

Joe Ferner

unread,
Oct 20, 2014, 9:54:14 AM10/20/14
to lum...@googlegroups.com
Looks like this line is incorrect "public static final String CONFIG_ES_LOCATIONS = "192.168.0.121";" It should be https://github.com/lumifyio/securegraph/blob/e775366b9f7de6b01a0f7b2c911898ef0e3f04ef/securegraph-elasticsearch-base/src/main/java/org/securegraph/elasticsearch/ElasticSearchSearchIndexBase.java#L40

Then in your properties file you should specify "search.locations=192.168.0.121". Alternatively if you are using a map to configure like in the readme you would add: "mapConfig.put("search.locations", "192.168.0.121");"
Reply all
Reply to author
Forward
0 new messages