document vs graph database

[Allan Johns]

Hi all,

I have a system implemented on Titan (Cassandra backend), but the nature of our data means that sometimes we need to do document-style searches over a very large number of objects, whilst other times we do more traversal-style searches that graph databases are good at. The problem is that Titan (and I'm assuming that, by extension, all graph databases) does not seem particularly fast when it comes to document-style lookups.

We're now considering augmenting the system by keeping a NoSQL database updated in sync with Titan, linked by vertex eids. A document-style lookup would begin in this db, the eid result set is collected, then this would be used to start a search in Titan. Since we're already using Cassandra as the backend, it would make sense to simply store this secondary db in another keyspace on the same Cassandra instance.

So my question is a general one - does this approach sound reasonable? Does this use case match what other people are doing, and how did you go about solving the problem?

thx
A

[Daniel Kuppitz]

Hi Allan,

there's a nice blog post about this scenario. See: Polyglot Persistence and Query with Gremlin

Cheers,

Daniel

[Marko Rodriguez]

Hello,

> We're now considering augmenting the system by keeping a NoSQL database updated in sync with Titan, linked by vertex eids. A document-style lookup would begin in this db, the eid result set is collected, then this would be used to start a search in Titan. Since we're already using Cassandra as the backend, it would make sense to simply store this secondary db in another keyspace on the same Cassandra instance.
> So my question is a general one - does this approach sound reasonable? Does this use case match what other people are doing, and how did you go about solving the problem?

There is a lot of talk about "polyglot persistence" on the web. In my experience, this tends to lead to a rats nest of id resolution and various data consistency issues. If your application logic is responsible for such tasks, then you will be spending a lot of your time coding an a database --- and not your application. You would be creating a meta-database trying to unify to other databases.

HTH,
Marko.

http://thinkaurelius.com

[Matthias Broecheler]

Hey Allan,

what types of searches would this setup support that Elasticsearch does not?

Thanks,

Matthias

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
Matthias Broecheler
http://www.matthiasb.com

[Jonathan Haddad]

I'm going to agree w/ Marko on this.  We started out on shift.com using MongoDB and Neo4J.  (We've since switched to Titan).  We're in the process of removing Mongo from our stack completely.  It's caused us an unbelievable amount of aggravation. 

I've always found it much more manageable to have a canonical source of my data that's highly reliable.  In our case, Titan + Cassandra.  We store some additional data in cassandra (custom indexes), and we use redis as a high performance cache.  We use elastic search for full text search.   But ultimately, we only need cassandra to rebuild everything else.

[Allan Johns]

Hi guys,

Here's my scenario - we have millions of objects in the database with various fields. Some aren't consistent (ie they are custom fields).

Users often need to do a direct object lookup based on several exact field matches. It's here where the system is not coping - since you can only use an index at the start of a query (ala g.V("key","value")).

How do I solve this problem in a graph database? I need the graph (because these objects have input/output relationships that graph dbs are great at traversing), but I need some serious speed when doing direct lookup also.

I was about to start investigating the polyglot approach but now after you're input I'm not sure what to do.

Help!
A

--

[Matthias Broecheler]

Hey Allan,

can you give an example of a query where you would use Mongo for object retrieval?

Thanks,

Matthias

[Allan Johns]

Sure. We have 'product' vertices with various fields. A common field is "job", and an example custom field is "aov". There are many products belonging to the same job, and many with the same aov.

So one query example is, "get me all the products with aov X that belong to job Y, created by user U".

These fields (job, aov, user) are all indexed. I maintain some metrics in the system to try and predict which index is the least hot, this is then used as the starting point for the query (and the order of lookup on the following fields). So let's assume the coolest index here is user, the query becomes something like:

g.V('user','joe.bloggs').has('aov','shadow').has('job','test').has('element_type','product')

But even with product counts in the sub million, queries like this are getting very slow.

Also, adding these objects to the database is causing more load than I expected, I'm wondering if that has anything to do with the indexing - some of these indexes will be very hot (for example, 'job' will easily match millions of objects in the database).

thx
A

[Matthias Broecheler]

Yes, I can see that executing one index call and post-filtering can be slow. Have you considered creating a composite index? I.e. create another field that is aov+job+user and indexing based on that? That would significantly reduce the load on the index and make index calls much faster due to lower selectivity.

Cheers,

Matthias

[Allan Johns]

The nature of user queries is too free-form, it is not possible to predict the combination of fields that might be searched on.

I'll tell you where I started going with an approach to solving this and why I abandoned it (although it sounds like I might go back there again now), would be good to get your input:

So 'products' in the system are grouped under 'version' nodes, where typically you have hundreds or maybe thousands of products connected to a single version node (no more than this). What I have been thinking about doing is converting every key,value pair on a product into a node in its own right, with an indexed field like so:

_string_kv: "aov|shadow"

These "field" nodes would then be connected to the version node containing the matching product. A counter would also be stored in the field node, incremented each time a matching product is added. What I was then going to do is use these field nodes as a kind of broad-phase filter - that is, find all the field nodes matching a product lookup, choose the one with the smallest counter, and start the search there, bouncing back and forth between version node and field node to narrow down the search:

g.V("_string_kv","aov|shadow").out("has_field").as('x')
.in("has_field").has("_string_kv","user|joe.bloggs").back('x')

.(and so on and so forth).

Then, the tail end of the query would jump down into the products and filter once again. Since products that all belong to the same version will tend to have a lot of the same fields, this approach will touch a lot less data during the bulk of the query, and should perform well.

I then moved away from this idea, simply because I started to think I was trying to replicate the kind of behaviour that NoSQL databases are implementing internally anyway! Which became relevant when I started thinking about taking the polyglot approach. Based on what people have said though, I'm a lot less enthusiastic about going down this road.

Thoughts?

[Matthias Broecheler]

Hey Allan,

here is my (pretty high level) thinking on this: 

Ignoring Titan and NoSQL for second, no matter what database you use, you have to make an initial index call to find the products that match a combination of attributes. If you have individual indexes on those attributes, then that is possible but if all of them are of low selectivity (many results) and only the combination is highly selective (few results) then the only way to make this efficient is a composite index.

Now, you correctly pointed out that composite indexes require that you know the combination of attributes and in your case there may simply be too many (due to free form search). However, you only have to define those composite indexes for combination of attributes iff

a) They are individually of low selectivity

b) They are queried for

That should reduce the space.

HTH,

Matthias

[David]

Allan,

Sounds like Matthias and Marko are well on their way to helping you with the specifics of your problem,
but since I've had similar thoughts/problems as you, I thought I'd chime in.

If you don't have the luxury of a greenfield environment then polyglot may be a viable option for adding graph capabilities
to an existing environment.  I think this is a common approach for companies with existing systems that want to
add the power of graph technology and not re-write everything.

There are also "shades of polyglot" in my opinion. 

We use HBase directly along with Titan backed by HBase.
We  run numerous Pig Map/Reduce jobs on the HBase directly and then sweep
results into the graph where that data is used in traversals.

We use a single, unique ID to connect the HBase-only data with the graph data.

This is not as intense a polyglot situation as trying to keep data in two different stores in synch all the time.
We don't do that.

I am not a Ninja with Gremlin in Titan (and especially when I use Faunus).
But there are certain types of queries needed to support my UI that are awkward for me to code in Gremlin
and "apache sqoop-ing" parts of my graph into a PostgreSQL db to run these queries with good old
fashion SQL seems alluring.   (the article referenced by Daniel doesn't apply here).  My jury
is still out on whether I can support an OLAP sort of UI directly on top of Titan, which is what I am
trying right now.

[Jonathan Haddad]

Hey Allan,

It really sounds like you've got a search problem, as opposed to a document database or graph.  If you're looking to match a bunch of ad-hoc criteria, my recommendation would be elastic search, solr, or sphinx.  All will give you great performance and are optimized to do these exact type of operations.  In the past, I've kept all my data in 2 spots - my main database, and my search engine.  It's served me very well throughout the years.

Jon

[Allan Johns]

So what does that mean in the context of ElasticSearch support that's now in Titan. Should I be going down this route, or a separate ElasticSearch/solr server do you think?

[Matthias Broecheler]

Jon makes a good point. You could get started with elastic search through titans integration. That should get you up and running very quickly.

[Jonathan Haddad]

Hard for me to say - given that I don't know your domain.  It's going to be a tradeoff between your immediate needs, how much work you want to do, and how flexible you want to be.

Here's some things to consider:

• Do you want facets?  I find it convenient to do a broad search sometimes then drill down with filters via facets.
• Are you running complex queries with custom weighting?  ES is really flexible and you can do a lot of crazy stuff, if you need to.
• Do you have the time to build out the additional support for another type of database?  There's definitely going to be more than a few days spent working on this.  If you support the search as an external data source, you'll need to manually reindex your search doc whenever you make changes to your vertices.  
• Do you want to build a custom search document - that is, does your search criteria contain a lot of extra meta data that you don't currently store on the vertex?  I typically end up storing a lot of extra data such as category, tags, or other meta data from the adjacent vertices. 
  

We're currently running Titan 0.2.0, so I don't know how flexible the query model for ES is yet, so I can't offer any practical advice there.  It looks like Titan supports numeric queries (I assume this doc applies: http://www.elasticsearch.org/guide/reference/query-dsl/range-query/), Full Text and Geo queries, so if you don't need facets or an extremely custom search schema you could probably make it work.  

Jon