Blazegraph, path to GPU-accelerated Gremlin?

[James Thornton]

Hi All -

MapGraph -- the GPU-accelerated graph engine -- has been rolled into Blazegraph, and it looks like this means OLTP and OLAP can be combined into a blazing-fast, single OLXP system.

From: https://www.blazegraph.com/product/gpu-accelerated/

The original work was funded by DARPA and presented at the 2014 SIGMOD conference in a paper entitled, MapGraph: A High Level API for Graphs [1]. This work is available in open source. Later work, in collaboration with the University of Utah SCI Institute [2] and funded by DARPA, applied multi-core techniques running on over 750 M cores on the Titan Supercomputer to extend this to Multi-GPU traversal with Breadth First Search (BFS).  On a cluster of 64 NVIDIA K40 GPUs, it demonstrated a throughput of 32 Billion Traversed Edges Per Second (32 GTEPS), traversing a scale-free graph of 4.3 billion directed edges in .15 seconds, which was featured in a presentation IEEE Bigdata Conference.

[1] https://www.blazegraph.com/whitepapers/MapGraph-SIGMOD-2014.pdf

[2] http://www.sci.utah.edu/publications/Fu2014a/UUSCI-2014-002.pdf

Blazegraph's engine has support for GPU-accelerated SPARQL. 

Daniel Kuppitz wrote a SPARQL-to-Gremlin compiler for the Gremlin Traversal Machine (http://www.datastax.com/dev/blog/the-benefits-of-the-gremlin-graph-traversal-machine)

Would a Gremlin-to-SPARQL compiler provide us with a path to GPU-accelerated Gremlin?

- James

--

James Thornton, http://electricspeed.com

[Michael Grove]

I can't say anything about that, but when we rolled our Gremlin-to-SPARQL compiler last year it opened up a lot of new features not previously available in property graph databases, such as rules and integrity constraints over the contents of the graph.

Cheers,

Mike

 

- James

--

James Thornton, http://electricspeed.com

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/CANva_A0C%2By03mU-5bO0PCGGVautQAzpgbuEAQ8vwCW68QxaWRQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Dylan Bethune-Waddell]

I came across a framework called "Gunrock" for running multi-GPU code at a high level of abstraction - I posted an issue to their git repo about implementing a GraphComputer and have been comparing Marko's Gremlin paper to theirs to see if the abstractions match and so far it looks pretty good:

https://github.com/gunrock/gunrock/issues/111

Besides, "gunrock-gremlin" just sounds pretty cool doesn't it? Here's a link to the Gunrock paper:

http://arxiv.org/abs/1501.05387

I would think others with an interest in looking into are more likely to know if it'll work and how to do it, so feel free. For me it's a learning exercise more than anything right now.

[Mike Personick]

James,

Thanks for your post.  We are actively working this from a few different angles.  As you point out, OLTP should be pretty straightforward - we can use our current Blazegraph/TP3 implementation (which translates Gremlin to Sparql) on top of the Blazegraph GPU Sparql Engine to get GPU-accelerated OLTP Gremlin.

For OLAP, we were just discussing earlier this week doing a second TP3 implementation on top of our new DASL ("dazzle") API.  Blazegraph DASL is a new domain-specific language that enables algorithms for large-scale machine learning and graph analytics that efficiently run on GPUs, without the need for parallel programming expertise or low-level device optimization.  DASL replaces the old MapGraph Gather-Apply-Scatter (GAS) API.

https://www.blazegraph.com/product/gpu-accelerated/

Thanks,

Mike

[James Thornton]

FYI...

WikiData selected BlazeGraph to back its new Query Service:

• https://lists.wikimedia.org/pipermail/wikidata-tech/2015-March/000740.html
  

Here's a spreadsheet showing WikiData's evaluation of each candidate graph database:

• https://docs.google.com/spreadsheets/d/1MXikljoSUVP77w7JKf9EXN40OB-ZkMqT8Y5b2NYVKbU/edit#gid=0
  

- James

[Venkata Phani Kumar Mangipudi]

Thank you for the insights James, But this evaluation took place one year back!! 24 Mar 2015!

Can you please share if there any latest evaluation on this one?

Thanks and Regards,

Phani 

[Bryan Thompson]

This was a rare opportunity.  Normally the selection criteria are not as transparent as they were with the WDS (wiki data query service), and it is pretty rare that people are willing to share the entire selection process.  So, there is no update on that specific worksheet.  However, we are happy to announce that WDS recently rolled out Blazegraph 2.0.  Among other things, they have reported a significant improvement in load performance using the new "data loader" servlet.

It's been a pleasure working with the WDS people.  We've helped them to create a variety of extensions, such as support for their concept of date time, which is significantly more flexible than xsd:dateTime and which has to be able to express things such as the age of the universe.  Right now we are working to integrate the new spatial index in Blazegraph into WDS.

Thanks,

Bryan

[Venkata Phani Kumar Mangipudi]

Thank you for the clarification and the insights Bryan.