OLTP & OLAP concurrently?

[Jamie Briant]

I have need for a graph which can support both OLTP (real-time) interactions on known nodes and also OLAP (or replies in minutes) queries across the whole dataset. Does Titan support both of the models at the same time? I can find docs on setting up OLTP cluster and others on OLAP cluster, but they seem to be mutually exclusive configurations. Thanks for any pointers.

[Marko Rodriguez]

Hello,

> I have need for a graph which can support both OLTP (real-time) interactions on known nodes and also OLAP (or replies in minutes) queries across the whole dataset. Does Titan support both of the models at the same time? I can find docs on setting up OLTP cluster and others on OLAP cluster, but they seem to be mutually exclusive configurations. Thanks for any pointers.

Titan does support both models. However, I don't know what you mean by "at the same time."

1. You can run OLTP queries against a Titan cluster while, at the same time, an OLAP query is executing against it. The problem with this is that typically OLAP queries touch a lot of data and thus, it can slow down your OLTP response time. For applications that have a required OLTP response time, you will have a "cloned cluster" where you run your analytic/OLAP jobs.

2. An OLAP query can be composed of OLAP/OLTP components and thus, depending on the compilation, the first part may be served by an OLAP job, the second by an OLTP job, the third by an OLAP, etc…. While not "at the same time," the same traversal can move between OLAP and OLTP (depending on compilation).

HTH,
Marko.

http://markorodriguez.com

[Jamie Briant]

I am confused by the options for backend. It seemed to me that OLTP would be titan-cassandra and OLAP would be titan-hbase, but I am gathering that this is incorrect. I'm also confused by how to do OLAP queries. If I have 20 ec2 instances, do I run titan on 10, cassandra on 10, and then spin up an EMR Spark cluster for doing OLAP? Or do the titan nodes handle the OLAP queries directly? 

Thanks,

Jamie

[Marko Rodriguez]

Hi,

I am confused by the options for backend. It seemed to me that OLTP would be titan-cassandra and OLAP would be titan-hbase, but I am gathering that this is incorrect.

Cassandra and HBase are both OLTP databases. You would use Hadoop (or Spark) to do the OLAP operations over those databases.

I'm also confused by how to do OLAP queries. If I have 20 ec2 instances, do I run titan on 10, cassandra on 10, and then spin up an EMR Spark cluster for doing OLAP? Or do the titan nodes handle the OLAP queries directly? 

If you have your Cassandra cluster and a collocated Spark cluster, then you simply just change your "g" from:

g = graph.traversal()

to

g = graph.traversal(computer(SparkGraphComputer))

g.V().count() is now an OLAP Spark job.

HTH,

Marko.

http://markorodriguez.com

[Jamie Briant]

Does the Spark cluster talk to cassandra directly then? Or does it talk to the titan servers in front of cassandra?

Thanks,

Jamie

[Marko Rodriguez]

Hi,

Titan has a TitanCassandraInputFormat that wraps CassandraInputFormat. CassandraInputFormat does a serial stream of the rows in Cassandra. TitanCassandraInputFormat then converts those ByteBuffers into vertices and edges.

HTH,

Marko.

http://markorodriguez.com

--
You received this message because you are subscribed to the Google Groups "Aurelius" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aureliusgraph...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/aureliusgraphs/668c336f-c1c8-4757-be88-26f31f019f87%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.