How to really use Elasticsearch with (and without) Titan?

3,039 views
Skip to first unread message

Daniel Kuppitz

unread,
Apr 17, 2013, 8:46:19 AM4/17/13
to aureliu...@googlegroups.com
Hi,

I've played around with Elasticsearch as (remote) index backend. It wasn't hard to setup and vertices are indexed as expected. But how do I efficently use the index for queries? Example:

g.makeType().name("description").dataType(String.class).indexed("search", Vertex.class).unique(OUT).makePropertyKey()
g.commit()

g.addVertex(['description':'Lorem ipsum dolor sit amet, consectetur adipiscing elit. Maecenas vitae.'])
g.addVertex(['description':'Proin in varius augue. Sed aliquet fermentum scelerisque. In odio.'])
g.addVertex(['description':'The quick brown fox jumps over the lazy dog.'])

g.query().has("description",Text.CONTAINS,"fox").vertices().out().groupCount()

What can I do with the result of .vertices()? It seems to be impossible to further navigate through the graph. Also, how can I take advantage of facets (to mention only one example)? When I directly query the index from my application, there's nothing like a vertex id that I could use to query the graph.


{
 <snip> 
    "hits" : [ {
      "_index" : "titan",
      "_type" : "vertex",
      "_id" : "10r6",
      "_score" : 1.0, "_source" : {"2F0LaTPQOC":"Proin in varius augue. Sed aliquet fermentum scelerisque. In odio."}
    }, {
      "_index" : "titan",
      "_type" : "vertex",
      "_id" : "10r2",
      "_score" : 1.0, "_source" : {"2F0LaTPQOC":"Lorem ipsum dolor sit amet, consectetur adipiscing elit. Maecenas vitae."}
    }, {
      "_index" : "titan",
      "_type" : "vertex",
      "_id" : "10ra",
      "_score" : 1.0, "_source" : {"2F0LaTPQOC":"The quick brown fox jumps over the lazy dog."}
    } ]
  }
}

Cheers,
Daniel 

Marko Rodriguez

unread,
Apr 17, 2013, 10:28:48 AM4/17/13
to aureliu...@googlegroups.com
Hi Daniel,

Currently Gremlin does not have any nice syntax for dealing with Graph.query(). This will come in TinkerPop 2.4.0

For now, to do what you want, use _(). E.g.,

g.query().has("description",Text.CONTAINS,"fox").vertices()._().out().groupCount() // assuming Gremlin Groovy.

HTH,
Marko.
--
You received this message because you are subscribed to the Google Groups "Aurelius" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aureliusgraph...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

Daniel Kuppitz

unread,
Apr 17, 2013, 11:27:34 AM4/17/13
to aureliu...@googlegroups.com
Great, that works. But what if I want to send some queries from my application directly to ES and than use the result to query the graph. Is there a way to get ti vertex ids out of ES?

Cheers,
Daniel

Marko Rodriguez

unread,
Apr 17, 2013, 11:30:58 AM4/17/13
to aureliu...@googlegroups.com
Hi Daniel,

I don't quite understand your problem, so my solution may be a "yea, I know that!"

g.query().has('name',CONTAINS,'Smith').vertices()._().id

Marko.

Daniel Kuppitz

unread,
Apr 17, 2013, 12:32:55 PM4/17/13
to aureliu...@googlegroups.com
No, I mean I want to use ES directly from my application, for example to send some faceted search queries (as far as I know that's not possible through Titan). Then I would need the vertex ids of all matches, to query the rest of the data from Titan.
So the process would be: Application sends query to ES -> ES returns result -> Application uses result to send query to Titan and gather more data or filter even more.

Pseudo code example for the application:

documents = elasticsearchClient.search(searchTerm, facetFilter);
ids = documents.select(doc -> doc.vertexId);
finalResult = titanClient.query("g.v(" + ids + ").filter{it.isActive}");

Hope it's a bit clearer now.

Cheers,
Daniel

Daniel Kuppitz

unread,
Apr 17, 2013, 1:27:56 PM4/17/13
to aureliu...@googlegroups.com
Uhm, Marko, I believe you can forget my question. A document in ES contains only one property from any Titan vertex/edge, right? Consequently it's not really possible to make faceted search queries and other fancy stuff. What I need is a document in ES that contains all (at least more than 1) properties of a vertex and that's not what the ES indexing backend in Titan is made for, right? So I need to index and query all the relavant data from my application.

Matthias Broecheler

unread,
Apr 17, 2013, 2:45:54 PM4/17/13
to aureliu...@googlegroups.com
So, Titan indexes vertices and edges in ES as documents, meaning, all indexed properties are grouped by vertex/edge. I believe that would facilitate faceted search (which, as you correctly pointed out, is not supported in Titan atm).
However, Titan encode property keys when its stores them for efficiency and to maintain the typing system of Titan. So, you won't find the property "age" in es, it will be a base64 encoded integer.
Hence, we are thinking on how we can provide access to "raw" ES through Titan since going around Titan would be a big hack job right now.

Best,
Matthias
Matthias Broecheler
http://www.matthiasb.com

Juan Batiz-Benet

unread,
Jul 15, 2013, 7:35:04 PM7/15/13
to aureliu...@googlegroups.com
Any update on this, and whether it's possible to access the elastic search index without going through titan?

I think this is a really important feature, in that many times applications need very fast responses on certain search queries (e.g. autocomplete), where going through multiple services would introduce way too much latency. Many web apps use read-only elasticsearch directly from web browser javascript.

My use case is like this:

At this point, its unclear to me whether the browser <---> ES arrows are possible, and if they are, how to formulate the queries. Thoughts?

Cheers,
Juan

Matthias Broecheler

unread,
Jul 16, 2013, 5:31:25 PM7/16/13
to aureliu...@googlegroups.com
Going to ElasticSearch immediately without Titan in the picture is an interesting use case that I haven't considered. However, elastic search stores the ids of vertices. What would you do with those in JS on the client side?

Juan Batiz-Benet

unread,
Jul 16, 2013, 5:39:40 PM7/16/13
to aureliu...@googlegroups.com, aureliu...@googlegroups.com
I'm using vertex ids as the object ids themselves, so I can lookup objects directly.

​Others could either expose an api mapping { vid : other_id }, use other_id as the vid itself, or store other_id directly into the vertex as a property (though, would it be base64-mangled as you described above?)

Sent from Mailbox for iPhone


You received this message because you are subscribed to a topic in the Google Groups "Aurelius" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/aureliusgraphs/OS4Lsf-QsFg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to aureliusgraph...@googlegroups.com.

Guido García

unread,
Aug 9, 2013, 10:22:42 AM8/9/13
to aureliu...@googlegroups.com
Hi, this is an example obtained from ElasticSearch:

"hits": {
      "total": 4,
      "max_score": 1,
      "hits": [
         {
            "_index": "titan",
            "_type": "vertex",
            "_id": "4HmI",
            "_score": 1,
            "_source": {
               "2F0LaTPReW": "Proin in varius augue. Sed aliquet fermentum scelerisque. In odio."
            }
         },
...


Matthias, I guess that "_id" is the vertex id you are talking about. How can you get  the vertex with id "4HmI" using gremlin?
Thank you.

Viktoras Veitas

unread,
Aug 12, 2013, 6:23:53 AM8/12/13
to aureliu...@googlegroups.com
Hello Juan,

On Tue, Jul 16, 2013 at 11:39 PM, Juan Batiz-Benet <ju...@benet.ai> wrote:
I'm using vertex ids as the object ids themselves, so I can lookup objects directly.

Others could either expose an api mapping { vid : other_id }, use other_id as the vid itself, or store other_id directly into the vertex as a property (though, would it be base64-mangled as you described above?)

Could you please describe how do you do that?

I cannot make any sense of "_id" in this:


{
      "_index" : "titan",
      "_type" : "vertex",
      "_id" : "d5HW",
      "_score" : 1.0, "_source" : {"2F0LaTPQBQ":".I f  both i nputs a re  1,  for  example t,h en n odes 0 0  and 01  become a ctive, ,in   other w ords, t heyh h avei i ntensity >  >  0,  which is  equivalen t o  saying t hat the message0 0s0   and 0 1  arep p laced o n  the m essagl leis t. "}
    }

"_id" is definitely not titan's vertex id and does not seem to be any base64 encoding...
I am thinking of querying ElasticSearch directly for n-gram, wildcard, phrase search or similar.

Thanks,
Viktoras.

 

Matthias Broecheler

unread,
Aug 12, 2013, 11:31:17 PM8/12/13
to aureliu...@googlegroups.com
Awesome, I love how you guys deconstruct Titan to get more out of ElasticSearch. That's a good incentive for us to improve our interface.

In the meantime, its not Base64 but Base62 encoding to avoid special characters. Check out the class:
com.thinkaurelius.titan.util.encoding.LongEncoding
you can use that to decode.

HTH,
Matthias

PS: Use at your own risk ;-)

jca...@cpbibliography.com

unread,
Aug 13, 2013, 9:38:36 PM8/13/13
to aureliu...@googlegroups.com
Matthias,

Where do the indexes in _source come from? A long index id? Is there a way to extract that from Titan?

Regards,
Jared

Guido García

unread,
Aug 14, 2013, 3:22:29 PM8/14/13
to aureliu...@googlegroups.com
Hi Jared,

Doing some research, you can extract them from Titan:

Graph graph = ...
long typeId = graph.getType("your_property_name_here").getId(); // i.e. 36028797018964050
String encodedId = LongEncode.encode(typeId); // i.e. 2F0LaTPQBQ

(I've not found how the long id is generated in the first place yet)

With that info you can go and query ElasticSearch, for example:

   "query" : {
       "wildcard": {
          "2F0LaTPQBQ": { 
             "value": "*"
          }
       }
   }
}

HTH,
Guido
PD. Matthias, I agree with you, it would be better to include this query capabilities as part of titan. I was thinking about offering it as:
   g.query().has("your_property_name_here", Text.CONTAINS_WILDCARD, "*")


But I'm pretty new to graph databases, titan, blueprints, gremlin... so I don't know if this is something that you are already thinking about, how to get started with titan's codebase (classes to start looking at, etc), or if my approach is wrong or too limited.

Jared Camins-Esakov

unread,
Aug 14, 2013, 4:06:28 PM8/14/13
to aureliu...@googlegroups.com
Guido,

Thanks! I think adding Text.CONTAINS_WILDCARD, Text.PREFIX, and Text.CONTAINS_QUERY (the latter for Lucene's boolean search syntax) would be a great idea, and and maybe even fairly straightforward, based on my reading of the code (but this is my first time working with Titan, ElasticSearch, and Lucene, so I could easily be wrong). I tried to get a development environment set up so I could test that theory, but I ran into some problems with that and put it on the backburner for after I have gotten my catalog interface working with the features Titan already has.

Regards,
Jared
Jared Camins-Esakov
Bibliographer, C & P Bibliography Services, LLC

Songtao Zheng

unread,
Apr 8, 2014, 6:52:59 PM4/8/14
to aureliu...@googlegroups.com
Hello guys,

I am quite new to Titan and ES though have been playing with it for a while now. 

I suppose after loading the data into Titan through Gremlin, there should be index created in my ES index backend. 

gremlin> g = TitanFactory.open('../conf/titan-cassandra-es.properties')
==>titangraph[cassandrathrift:127.0.0.1]
gremlin> GraphOfTheGodsFactory.load(g)
==>null

However I am not seeing any indices created by calling "http://localhost:9200/_status" through my browser. 

Basically what I expect to see is the elasticsaerch output Daniel posted. 

Any one could teach me how to find the ES indices Titan created internally through elasticsearch call? 

Thank you in advance.
Songtao

Daniel Kuppitz

unread,
Apr 8, 2014, 7:39:32 PM4/8/14
to aureliu...@googlegroups.com
Hi Songtao,

the default titan-cassandra-es.properties file configures ES in embedded mode, hence you can't access it via http://localhost:9200/. If you start your own ES server locally and replace the existing ES configuration with the following configuration:

storage.index.search.backend=elasticsearch
storage.index.search.hostname=localhost
storage.index.search.client-only=true
storage.index.search.local-mode=false

...then you should see what you expect.

Cheers,
Daniel



--
You received this message because you are subscribed to the Google Groups "Aurelius" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aureliusgraph...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Songtao Zheng

unread,
Apr 8, 2014, 8:23:17 PM4/8/14
to aureliu...@googlegroups.com
Hi DanieL, 

Thanks for the prompt response. 

I agree with you and updated my current configuration to: 

storage.backend=cassandrathrift
storage.hostname=127.0.0.1
cache.db-cache = true
cache.db-cache-clean-wait = 20
cache.db-cache-time = 180000
cache.db-cache-size = 0.25

storage.index.search.backend=elasticsearch
storage.index.search.hostname=localhost
storage.index.search.directory=../db/es
storage.index.search.client-only=true
storage.index.search.local-mode=false

This time I received an error saying: any ideas again? 

gremlin> g = TitanFactory.open('../conf/titan-cassandra-es.properties')
Could not instantiate implementation: com.thinkaurelius.titan.diskstorage.es.ElasticSearchIndex
Display stack trace? [yN] y
java.lang.IllegalArgumentException: Could not instantiate implementation: com.thinkaurelius.titan.diskstorage.es.ElasticSearchIndex
        at com.thinkaurelius.titan.diskstorage.Backend.instantiate(Backend.java:355)

I double checked "Elasticsearch Remote Configuration" in session "Using Elastic Search" but not finding anything. What am I missing here? 

PS: my bundles are: titan-server-0.4.2 + apache-cassandra-2.0.6 + elasticsearch-0.90.13 on Windows 7 (32 bit). 

Thank you,
Songtao

Daniel Kuppitz

unread,
Apr 8, 2014, 8:53:35 PM4/8/14
to aureliu...@googlegroups.com
Hm, my first intention was that you cannot use hostname and directory, but a test on my local machine has shown that this misconfiguration doesn't lead to an error. Anyway, specify only hostname (remote configuration) or directory (local / embedded). However, that doesn't solve your problem. All the versions you've listed should be compatible, so I'm not quite sure what's wrong.

Which bundle of Titan did you download? Use the Titan Server bundle to make sure that nothing is missing. If that doesn't help, please post the full stack trace, maybe it unveils some more useful information.

Cheers,
Daniel

Songtao Zheng

unread,
Apr 8, 2014, 10:28:29 PM4/8/14
to aureliu...@googlegroups.com
Hi Daniel, 

I am using the lastest Titan-Server bundle - Titan Server 0.4.2

I removed storage.index.search.directory setting and keeping storage.index.search.hostname

The error is not going away as you expected. Here is the full error stack: 

gremlin> g = TitanFactory.open('../conf/titan-cassandra-es.properties')
Could not instantiate implementation: com.thinkaurelius.titan.diskstorage.es.ElasticSearchIndex
Display stack trace? [yN] y
java.lang.IllegalArgumentException: Could not instantiate implementation: com.thinkaurelius.titan.diskstorage.es.ElasticSearchIndex
        at com.thinkaurelius.titan.diskstorage.Backend.instantiate(Backend.java:355)
        at com.thinkaurelius.titan.diskstorage.Backend.getImplementationClass(Backend.java:367)
        at com.thinkaurelius.titan.diskstorage.Backend.getIndexes(Backend.java:331)
        at com.thinkaurelius.titan.diskstorage.Backend.<init>(Backend.java:122)
        at com.thinkaurelius.titan.graphdb.configuration.GraphDatabaseConfiguration.getBackend(GraphDatabaseConfiguration.java:1163)
        at com.thinkaurelius.titan.graphdb.database.StandardTitanGraph.<init>(StandardTitanGraph.java:75)
        at com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:40)
        at com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:29)
        at com.thinkaurelius.titan.core.TitanFactory$open.call(Unknown Source)
        at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
        at groovysh_evaluate.run(groovysh_evaluate:56)
        at groovysh_evaluate$run.call(Unknown Source)
        at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:112)
        at org.codehaus.groovy.tools.shell.Interpreter.evaluate(Interpreter.groovy:67)
        at org.codehaus.groovy.tools.shell.Interpreter$evaluate.call(Unknown Source)
        at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
        at org.codehaus.groovy.tools.shell.Groovysh.execute(Groovysh.groovy:152)
        at org.codehaus.groovy.tools.shell.Shell.leftShift(Shell.groovy:114)
        at org.codehaus.groovy.tools.shell.Shell$leftShift$0.call(Unknown Source)
        at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
        at org.codehaus.groovy.tools.shell.ShellRunner.work(ShellRunner.groovy:88)
        at org.codehaus.groovy.tools.shell.InteractiveShellRunner.super$2$work(InteractiveShellRunner.groovy)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:483)
        at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)
        at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:233)
        at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1079)
        at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:128)
        at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuper0(ScriptBytecodeAdapter.java:148)
        at org.codehaus.groovy.tools.shell.InteractiveShellRunner.work(InteractiveShellRunner.groovy:100)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:483)
        at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:272)
        at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:52)
        at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:46)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:133)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:137)
        at org.codehaus.groovy.tools.shell.ShellRunner.run(ShellRunner.groovy:57)
        at org.codehaus.groovy.tools.shell.InteractiveShellRunner.super$2$run(InteractiveShellRunner.groovy)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:483)
        at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)
        at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:233)
        at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1079)
        at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:128)
        at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuper0(ScriptBytecodeAdapter.java:148)
        at org.codehaus.groovy.tools.shell.InteractiveShellRunner.run(InteractiveShellRunner.groovy:66)
        at com.thinkaurelius.titan.tinkerpop.gremlin.Console.<init>(Console.java:57)
        at com.thinkaurelius.titan.tinkerpop.gremlin.Console.<init>(Console.java:70)
        at com.thinkaurelius.titan.tinkerpop.gremlin.Console.main(Console.java:96)
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
        at com.thinkaurelius.titan.diskstorage.Backend.instantiate(Backend.java:344)
        ... 63 more
Caused by: org.elasticsearch.client.transport.NoNodeAvailableException: No node available
        at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:205)
        at org.elasticsearch.client.transport.support.InternalTransportClusterAdminClient.execute(InternalTransportClusterAdminClient.java:8
5)
        at org.elasticsearch.client.support.AbstractClusterAdminClient.health(AbstractClusterAdminClient.java:87)
        at org.elasticsearch.action.admin.cluster.health.ClusterHealthRequestBuilder.doExecute(ClusterHealthRequestBuilder.java:98)
        at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:85)
        at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:59)
        at com.thinkaurelius.titan.diskstorage.es.ElasticSearchIndex.<init>(ElasticSearchIndex.java:161)
        ... 68 more

Daniel Kuppitz

unread,
Apr 8, 2014, 10:33:28 PM4/8/14
to aureliu...@googlegroups.com
"No node available" - that means, your ES server is not running. Can you access http://localhost:9200/ in your browser?
If you're not sure how to start the ES server, see: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/setup.html

Cheers,
Daniel

Songtao Zheng

unread,
Apr 8, 2014, 10:38:45 PM4/8/14
to aureliu...@googlegroups.com
Hi Daniel, 

Yes, I can access http://localhost:9200/ in my browser. I started ElasticSearch as a Windows service running in background. 

Content from localhost:9200: 
{
  • oktrue,
  • status200,
  • name"Lightmaster",
  • version
    {
    • number"0.90.13",
    • build_hash"249c9c5e06765c9e929e92b1d235e1ba4dc679fa",
    • build_timestamp"2014-03-25T15:27:12Z",
    • build_snapshotfalse,
    • lucene_version"4.6"
    },
  • tagline"You Know, for Search"
}

The first error was throwing out by java.lang.IllegalArgumentException: Could not instantiate implementation: com.thinkaurelius.titan.diskstorage.es.ElasticSearchIndex

Not sure the what is going wrong.

Thanks,
Songtao

Matthias Broecheler

unread,
Apr 8, 2014, 11:00:41 PM4/8/14
to aureliu...@googlegroups.com
Hi Songtao,

we currently only test our distributions on different flavors of Linux, so there might be some Windows specific aspect interfering here. For instance, might the elastic search client server be listening on a different port to the clients? The default is 9300. You can configure this via the "port" option.

Good luck,
Matthias 
Matthias Broecheler
http://www.matthiasb.com

Songtao Zheng

unread,
Apr 9, 2014, 3:04:56 AM4/9/14
to aureliu...@googlegroups.com
Thanks Matthias & Daniel, yeah it works perfectly with my Ubuntu-12.04.3. 

ElasticSearch is listening on port 9200 for HTTP testing. This issue on Windows 7 (32 bit) persists though. 

Best,
Songtao
<font fac
...

Songtao Zheng

unread,
Apr 9, 2014, 5:11:14 PM4/9/14
to aureliu...@googlegroups.com
Hey Daniel, 

I can see that Titan ES document only has one property, so we are not really able to use advanced features (stemming, top-notch pagination, highlighting, auto-complete, advanced search queries (like quotes, +, - etc. spelling suggestions, "more like this" etc..)) out-of-the-box with ES search engine. Basically what we want is the features like a real search engine provides.

Daniel, can I ask what did you end up with - indexing the data yourself through your application instead of Titan/Gremlin API? 

Also, is there any example information where someone is using ES on top of Titan? Articles, blogs, etc? 

Again, I am newbie to Titan server, forgive me if I miss some searching features Titan already supported. 

Thank you,
Songtao

Daniel Kuppitz

unread,
Apr 9, 2014, 6:34:33 PM4/9/14
to aureliu...@googlegroups.com
We have a client who excessively uses ES. Titan is used to index and retrieve data, but besides that the ES API is used directly to make use of more advanced features (more like this, etc.). Unfortunately I can't provide more details or code samples, but I can assure you it works. However, since Titan began to support direct index queries, things became already a lot easier and few things from your list are covered by this feature.

Cheers,
Daniel



For more options, visit https://groups.google.com/d/optout.

Songtao Zheng

unread,
Apr 10, 2014, 3:21:59 PM4/10/14
to aureliu...@googlegroups.com
Hi Daniel, 

I really appreciate you giving a thought to the structure.

I am not quite sure "Titan is used to index and retrieve data"...AFAIK, Titan currently does not support arbitrary indexing capabilities, so you can't actually fully rely on Titan for indexing management and need additional indexing enhancements from ES on top of Titan, correct? Do you maintain a separate ElasticSearch index for your data (and then you are able to use the ES advanced features)? A high level explanation would work for me ;)

Thanks,
Songtao

Daniel Kuppitz

unread,
Apr 10, 2014, 4:26:33 PM4/10/14
to aureliu...@googlegroups.com
Hi Songtao,

indeed you cannot change the way how Titan creates the index, but I've also never had a use case where it was necessary. So far we've done

stemming, top-notch pagination, highlighting, auto-complete, advanced search queries (like quotes, +, - etc. spelling suggestions, "more like this"

I think your question specifically targets stemming. As far as I know you can modify an ES index after it's created, that way you can modify the default analyzers etc. as you like.

Cheers,
Daniel

Songtao Zheng

unread,
Apr 11, 2014, 12:39:31 AM4/11/14
to aureliu...@googlegroups.com
Thank you Daniel. 

Actually we want a full-fledged search engine in the long term, so you are right we need to rely on ES index and API to reach that at this time. 

Since TItan ES index has only one property each document and properties keys are encoded, I agree with you that modifying the ES index is necessary. Unfortunately I am also new to ES, so which ES API you guys using to modify the index? The ES udpate api? http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-update.html 

Thank you,
Songtao

Daniel Kuppitz

unread,
Apr 11, 2014, 9:01:54 AM4/11/14
to aureliu...@googlegroups.com
Hi Songtao,

as I've said, so far I've never had a use case where it was necessary to modify an ES index and besides that I think the ES mailing list or SO are better places to ask those in-depth ES questions.

Cheers,
Daniel



Songtao Zheng

unread,
Apr 11, 2014, 11:38:35 AM4/11/14
to aureliu...@googlegroups.com
Appreciate your clarification - Basically your use cases can all be done through Titan/Gremlin and no touch with ElasticSearch API. 

Thanks,
Songtao

manish kumar

unread,
May 16, 2014, 11:14:09 PM5/16/14
to aureliu...@googlegroups.com
So any solution of using ES without Titan??? I am also facing problem with Titan-ES combination. Specially with pagination of ES result coming out of Titan. Latest titan version doesn't support pagination of ES result and that is the basic requirement of any search application.

Say I want to get all vertices and their property having BookTitle=XYZ. What will be the steps/query to get a vertex property from ES directly ??
Reply all
Reply to author
Forward
0 new messages