best practice for graph-like entities

[Thomas Alisi]

hi everyone

I'm designing a g+ application for a big international brand. the entities I need to create are pretty much in form of a graph, hence a lot of one-to-many relations (arcs) connecting nodes that can be traversed in both directions. I'm reading all the readable docs online, but I haven't found anything so far specific to ndb design best practices and guidelines. unfortunately I am under nda, and cannot reveal details of the app, but it can match almost one to one the context of scientific conferences with proceedings, authors, papers and topics.

below the list of entities envisioned so far (with context shifted to match the topics mentioned):

- organization (e.g. acm)

- conference (e.g. acm multimedia)

- conference issue (e.g. acm multimedia 13)

- conference track (e.g. nosql, machine learning, computer vision, etc.)

- author (e.g. myself)

- paper (e.g. "designing graph like db for ndb")

as you can see, I can visit and traverse the graph through any direction (or facet, from a frontend point of view): 

- author with co-authors

- author to conference tracks

- conference tracks to papers

- ...

and so on, you fill the list.

I want to make it straight and solid because it will launch with a lot of p.r. and will need to scale consistently overtime, both in content and number of users. I would like to code it from scratch hence designing my own models, restful api to read/write this data, avoiding non-rel django and keeping the presentation layer to a minimum template mechanism. I need to check with the company where I work, but we might be able to release part of the code with a decent open source license (ideally, a restful service for ndb models).

if anyone could point me towards the right direction, that would be awesome.

I posted this same topic on stackoverlow, here: http://stackoverflow.com/questions/18017150/best-practice-for-graph-like-entities-on-appengine-ndb and will do my best to keep both sides up to date

thanks!

thomas

[Gregory Nicholas]

don't over think it, and don't over design it. sounds complex, maybe an approach is stepping through the actual application, and having that guide the data you need, when, and that should give you the starting point you need.

check this post for options for storing trees: http://stackoverflow.com/questions/1011814/storing-hierarchical-data-in-google-app-engine-datastore

[Thomas Alisi]

hey

thanks for the link gregory,

I'll probably follow your suggestion and start designing entities, adjusting the structure on the go. I reckon at the moment the main doubt I have is if it's worth storing the arcs in my graph as separate entities, and it seems I'll need to do it at least for the heaviest entities (something that could potentially hit the limit of query size if stored as repeated keyProperties)

I'll keep searching similar implementations and report while getting along