Cayley with Freebase data?

[LonnieTC]

Greetings All,

I hope that you are doing well.

Recently, I have embarked upon learning "Go" while coming from a C/C++ (and other languages) background.

In particular, I am interested in the Cayley graph database (https://github.com/google/cayley) which is also a new area to me as well although I have had a fair amount of experience with SQL/NoSQL databases in the past on various Linux and Windows platforms.

I am using an Ubuntu Linux platform now and am going to install "Go" after reviewing a number of tutorials and watching some online videos about the language which I must say seem to be very exciting and has huge benefits for what I am interested in doing.

Long story short, I am wondering if the data which can be downloaded from the Freebase site (https://developers.google.com/freebase/data) can be loaded into and queried from the Cayley graph database?

If, so then, I would like to follow-on to ask, although I would assume that it would be true however could be wrong, but would I be correct in assuming that the graph data (Freebase data) can be extended by adding more entries to the graph database via Cayley?

Sorry for the long post, but I am very excited about this area and would like to set up a test server with Cayley + Freebase data as I am not clear on Gremlin + TinkerPop and how these come together.

Any information that someone might be willing to share with me on this whole topic would be greatly appreciated.

Cheers,
Lonnie

[Denys Smirnov]

Greetings Lonnie,

1) Yes, theoretically this data can be loaded into Cayley. But there is a few points. The size of the data is pretty large, so you'll need to setup MongoDB or PostgreSQL backend at least. I haven't tried to load that dump myself, but quick benchmark on quad loader shows that it will require at least 3 hours on my machine to parse the data without loading it to the datastore. You can try it, but you may also consider to filter data first.

2) Sure, you can alter or extend the data whatever you want. You can easily import or remove quads using HTTP API or CLI. More complex manipulations will require you to write some Go code. Feel free to ask any technical questions regarding using Cayley from Golang - I've been hacking on the codebase for a while, so I think I might help you with that :)

пятница, 12 февраля 2016 г., 21:21:15 UTC+2 пользователь LonnieTC написал:

[Robert Melton]

On Sat, Feb 13, 2016, at 12:35, Denys Smirnov wrote:

Greetings Lonnie,

 

1) Yes, theoretically this data can be loaded into Cayley. But there is a few points. The size of the data is pretty large, so you'll need to setup MongoDB or PostgreSQL backend at least. I haven't tried to load that dump myself, but quick benchmark on quad loader shows that it will require at least 3 hours on my machine to parse the data without loading it to the datastore. You can try it, but you may also consider to filter data first.

 

Is there a reason BoltDB couldn't work?  BoltDB claims to have DBs as large as 1TB in use in production (https://github.com/boltdb/bolt).  I realize it is single writer, but for loading is that really a huge issue?

 

--

Robert Melton | rme...@gmail.com

 

[Denys Smirnov]

сб, 13 февр. 2016 г. в 22:34, Robert Melton <rme...@gmail.com>:

 
As I told before, I haven't tried to load that amount of data yet. But unpacked database dump seems to be near 250 GB and bolt backend implementation will generate some additional indexes at least in 4 quad directions. Thus, data may expand to some degree. And I can't say for sure would it have a good performance or not. 

--
You received this message because you are subscribed to a topic in the Google Groups "cayley-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/cayley-users/uRrYiXpf_Zs/unsubscribe.
To unsubscribe from this group and all its topics, send an email to cayley-users...@googlegroups.com.
To post to this group, send email to cayley...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cayley-users/1455395692.1537717.520405602.23403373%40webmail.messagingengine.com.
For more options, visit https://groups.google.com/d/optout.