Re: Documentation for building a datastore.

[Madan Jampani]

Hi Abhishek,

I'm assuming the application you intend to build has some state you want to maintain in a highly available fashion. To this end ONOS provides some core state management primitives. A brief description of these primitives can be found here: https://wiki.onosproject.org/display/ONOS/Distributed+Primitives. If you had a chance to do the distributed tutorial, it has a lab exercise specifically dealing with distributed state management. (Link)

Hope that helps.

Madan.

On Wed, Jul 22, 2015 at 4:38 PM, Thomas Vachuska <t...@onlab.us> wrote:

Thomas

Begin forwarded message:

From: Abhishek Dwaraki <abhishek...@gmail.com>
Date: July 22, 2015 at 16:31:49 PDT
To: Thomas Vachuska <t...@onlab.us>
Subject: Documentation for building a datastore.

Hey Thomas,
                         Apologies for the direct mail. But you reply really fast. I just wanted to find out if there is any documentation to build a datastore on ONOS? I am building a service for which I need to build my own datastore. Was wondering if you could point me in the right direction.

Regards,
Abhishek.

[Abhishek Dwaraki]

Hi Madan,

                     I was taking a look at the implementation of the GossipIntentStore as a starting point. I will also take a look at the primitives and the tutorial that you pointed out. I will let you know if I have any questions. Appreciate the help and the really quick response.

Regards,

Abhishek.

[Abhishek Dwaraki]

Hi Madan,

                     I was taking a look at the primitives that have been provided and had a couple of questions. Most of the primitives are map/hash based (please correct me if my understanding is wrong). If for my use case, a different data structure is desirable, how would I take that DS and convert it into a backing datastore for my service? How is all the information serialized to disk? What datastore is used in the back end? Any information will be helpful. Thanks a ton. :)

Regards,

Abhishek.

On Jul 22, 2015, at 4:47 PM, Madan Jampani <ma...@onlab.us> wrote:

[Madan Jampani]

Hello Abhishek,

There are 2 map variants (Consistent and EventuallyConsistent) that get used the most.
There is a Set primitive (DistributedSet) that as you can imagine simply builds on top of the ConsistentMap primitive.

AtomicCounter is equivalent to AtomicLong in a distributed setting and similarly AtomicValue is the distributed equivalent of AtomicReference.
There is also a Queue primitive (DistributedQueue) that supports a producer-consumer use case with exactly-once delivery semantics and non-blocking consumers via long polling support.

It is relatively straightforward to add a newer primitive if one desires. The underlying implementation that makes these data structures possible is a Replicated State Machine backed by Raft consensus. The implementation in ONOS partitions (or shards) the key space and runs multiple such state machines, one for each partition. So in a given ConsistentMap, keys a distributed (randomly) onto different partitions and each partition maintains its state in a state machine backed by Raft.

If you can describe the data structure you want to implement, I will be able to provide some specific pointers.

Madan.

--
You received this message because you are subscribed to the Google Groups "ONOS Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to onos-dev+u...@onosproject.org.
To post to this group, send email to onos...@onosproject.org.
Visit this group at http://groups.google.com/a/onosproject.org/group/onos-dev/.
To view this discussion on the web visit https://groups.google.com/a/onosproject.org/d/msgid/onos-dev/C0DDF798-98D6-41DF-A7B9-F20E0F36EA72%40gmail.com.

[Abhishek Dwaraki]

Hi Madan,

                    Thank you for your detailed response and apologies for the late reply. I am mostly looking at a combination of data structures for my purposes. Right now, I am evaluating how a combination of a tree-based DS and a hash-map based solution fares against a multidimensional trie for my purposes. Like you pointed out, the only DSes available that suit my requirements right now are the map variants. Even if I did decide to go along with the trie implementation and pulled in some libraries available in Java for the same, could you please give me some pointers how to back this with Raft? (This is mostly in my next phase, but if it isn’t too elaborate, I will implement by datastore in this phase itself).

Regards,

Abhishek.