Making changes to a compressed graph

[rsm...@gmail.com]

Hello there,

Here is my situation:

1. I define my graph schema by building out a NFGraphSpec.
2. I build the graph using the NFBuildGraph object and I populate it with the data I currently have. Let's say I have 10 movies and 15 actors and I populated my NFBuildGraph with that data.
3. I compress my NFBuildGraph to a NFCompressedGraph, which I serialize and pass on to some Graph consumers. I also discard my NFBuildGraph object and data.
4. Now lets say a 11th movie comes along. In order to provide an updated graph to my consumers, do I need to start from scratch? Basically, build a spec again, build the graph again, and serialize again?

Wouldn't it be useful to have a way of generating an NFBuildGraph object out of a compressed and serialized graph data?

Please let me know your suggestions,
Thanks,
RS

[Drew Koszewnik]

Hi RS,

Basically, build a spec again, build the graph again, and serialize again? 

Yes, this is the way each of our use cases works.  Our data only flows in one direction -- from the sources of truth to the caches on the services using the data.  We periodically rebuild the graph data, then send the updated data to the graph consumers.  We don't have a mechanism for recreating an NFBuildGraph from an NFCompressedGraph.

I can see how this would be a useful addition for some use cases, and shouldn't be too difficult to implement: I imagine we'll instantiate a new NFBuildGraph with the same NFGraphSpec from the NFCompressedGraph, then iterate over each type, each node, and each property, filling the NFBuildGraph with the connections in the NFCompressedGraph.

In order to get this to work, I think we'll need the ability to get the NFGraphSpec out of an existing NFGraph, and the max ordinal for a given type.  We also need a way to determine if a node actually contains any data or not (whether the node pointer != -1).

I think it would be a valuable addition -- I can add it to my wish list, but can't make any promises about the timing.  Alternatively, if you need it sooner than I can get to it, I'll be more than happy to review a pull request if you add it yourself!

Thanks for the feedback!

Drew.