[Blog] Educating the Planet with Pearson

[Marko Rodriguez]

Hi,

Aurelius has been working with Pearson Education for the last 2 years to apply graphs to various problems in online education. Over the last ~5 months, with generous support from Pearson, Aurelius has developed and executed a large-scale benchmark to demonstrate Titan's suitability to represent all universities, students, teachers, their artifacts, courses, discussions, etc. within a single global graph capable of supporting numerous concurrent users. In other words, this benchmark demonstrates that Titan (along with Pearson's technology above) is capable of educating the planet. In general, we believe this post speaks to the suitability of using Titan to support large-scale models in other domains.

The culmination of this effort can be found here:

http://thinkaurelius.com/2013/05/13/educating-the-planet-with-pearson/

https://twitter.com/aureliusgraphs/status/333982991279874048

Please enjoy the information therein,

Marko.

http://markorodriguez.com

[Daniel Quest]

This is just great Marko!  I wish it was around when I first learned gremlin.  

Any idea how Titan performs relative to Urika?  Or are we comparing Apples and Oranges?  e.g. is Titan specialized at building large scale backends for web properties where Urika is better for data mining?

http://www.reuters.com/article/2012/02/29/idUS121560+29-Feb-2012+MW20120229

http://www.cray.com/Products/BigData/uRiKA.aspx

Thanks as always for the interesting post!

Dan

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Marko Rodriguez]

Hi,

Any idea how Titan performs relative to Urika?

I don't know.

 Or are we comparing Apples and Oranges?  e.g. is Titan specialized at building large scale backends for web properties where Urika is better for data mining?

At the end of the day, you have graph data and you need to process it. How you represent that data (and the support data structures that make it efficient to traverse) is what makes a system good for its task. Even if you say: "everything is in-memory" (e.g Urika), you still have to make sure your in-memory representation is efficient for your use case. One of the cool things about Titan is the fact that even though it is a disk-based system, its disk and in-memory data structures around supernodes makes it extremely performant (log(n) vs n). No amount of CPU power will make a linear scan efficient. I don't know what Urika is doing in terms of their representation -- just a general comment.

Thanks as always for the interesting post!

Thanks for reading.

Dan

Marko.

[James Thornton]

This is truly an epic post. 

I hadn't heard of the AWS cluster group limitation before -- is 16 units the limit? -- I just asked about it on the Hacker News thread if you want to respond there.

Amazing work guys. 

- James

[Matthias Broecheler]

Not sure if its a hard limit. Whenever we tried to launch more than 20 instances, 20 instances would end up in one PL and the remaining in another. Hence, we figured there was probably a limit there. AWS is not really good about communicating their limits.

--
You received this message because you are subscribed to the Google Groups "Aurelius" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aureliusgraph...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
Matthias Broecheler
http://www.matthiasb.com

[James Thornton]

I don't think Google Compute Engine has these type of limits -- you can spin up 600K servers in a blink -- but I'm not sure what its "high memory" offering is at this point. Still, the IO is supposed to be several time faster than AWS so it would be interesting to see how they compare.

- James

[Matthias Broecheler]

You can do the same with EC2 i think. The limitation here is the size of the placement group. A placement group is a physical co-location (I think) of machines which guarantees better connectivity between instances. Quite naturally, there is a limitation there and it seems to be 20.

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.