ShareDB Architecture Layers?

[Curran Kelleher]

Greetings,

Has anyone tried to separate ShareDB-specific logic from application logic?

We're working on a ShareDB application and trying to implement the Clean Architecture. We're having trouble identifying ways to refactor such that the client side code interfaces only indirectly with ShareDB, via some sort of "service layer" API that speaks the language of the application domain and encapsulates the ShareDB interaction.

We're considering only using the ShareDB client on the server, and implementing a WebSockets-based service layer to the browser and frontend code. One fear I have with this approach is that we'd be reinventing the wheel, and might as well just use the ShareDB client in the browser, but wrap it in a service layer within the browser JS.

Just curious in general, has anyone tried or experienced something similar? Thank you.

Best regards,

Curran

[John Hewson]

On Oct 24, 2017, at 00:55, Curran Kelleher <curran....@gmail.com> wrote:

Greetings,

Has anyone tried to separate ShareDB-specific logic from application logic?

We're working on a ShareDB application and trying to implement the Clean Architecture. We're having trouble identifying ways to refactor such that the client side code interfaces only indirectly with ShareDB, via some sort of "service layer" API that speaks the language of the application domain and encapsulates the ShareDB interaction.

We're considering only using the ShareDB client on the server, and implementing a WebSockets-based service layer to the browser and frontend code. One fear I have with this approach is that we'd be reinventing the wheel, and might as well just use the ShareDB client in the browser, but wrap it in a service layer within the browser JS.

OT as used by ShareDB needs to run on the local client, 1) because it maintains all client document state, 2) because OT’s main role is to minimise client-server latency by allowing optimistic updates on the client.

Because of OT the ShareDB client library does a lot of heavy lifting - so yes, if you want abstraction then you should wrap it in a service later within the browser. You can use the underlying ottypes to create a fairly simple mockable persistence layer then build further business logic on top of that.

— John

Just curious in general, has anyone tried or experienced something similar? Thank you.

Best regards,

Curran

--
You received this message because you are subscribed to the Google Groups "ShareJS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sharejs+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Curran Kelleher]

Hi John,

Thank you for your thoughts here.

OT as used by ShareDB needs to run on the local client, 1) because it maintains all client document state, 2) because OT’s main role is to minimise client-server latency by allowing optimistic updates on the client.

 

This confirms what I thought as well - that the ShareDB client really does do some nice things, and it would be rather a shame to not use it in the browser.

 

Because of OT the ShareDB client library does a lot of heavy lifting - so yes, if you want abstraction then you should wrap it in a service later within the browser. You can use the underlying ottypes to create a fairly simple mockable persistence layer then build further business logic on top of that.

This makes sense. Here's a first draft of how I think we'll go about this (we're using React):

• Only do explicit document subscription in top-level components, and do it only via a browser-side service layer.
  
• Pass the ShareDB document reference down the component hierarchy, but treat it as opaque (don't use its API) in non-leaf components.
• Allow leaf components to speak directly to ShareDB via libraries like sharedb-string-binding and codemirror-binding.
• These libraries do subscription internally, and their "destroy" methods must be reliably invoked in component lifecycle.

Best regards,

Curran

[John Hewson]

Seems reasonable. There’s a few different approaches which could be taken with React, depending on if you have stateful components and/or use Flux. Most of the time I’d expect a ShareDB document can be represented by an object with just a submitOp(..) function and an on(‘op’, …) event emitter; I’ve had success using that for mocking. When handling remote ops, you have the choice of using the ShareDB-managed snapshot available on doc.data, or using the ‘op’ event handler and applying the OT operation yourself using ottypes such as json0, which is not actually that hard - this is what the bindings do.

For Flux/Redux you could use doc.data as your top-level state, but be warned that ShareDB directly mutates it. Alternatively you could turn each ‘op’ event into an Action and use ottypes to update your central state. This is what I do but I don’t use deeply nested models so I can’t offer any specific pointers there.

— John

[Curran Kelleher]

Hi John,

Seems reasonable. There’s a few different approaches which could be taken with React, depending on if you have stateful components and/or use Flux.

Redux is tempting. We are not using Redux currently, and we do have stateful components that interact with ShareDB at intermediate levels.

Most of the time I’d expect a ShareDB document can be represented by an object with just a submitOp(..) function and an on(‘op’, …) event emitter; I’ve had success using that for mocking.

That sounds generally good, however I'd like to have the components be "ignorant" of the OT types. Currently some of our components generate JSON0 ops in the same file as JSX, which smells bad to me.

One solution would be to expose submitOp() in components, but also strictly use "op generator functions", so instead of what we have now in one of our React components:

    doc.submitOp([{

      p: ['collaborators', doc.data.collaborators.length],                                                                     

      li: { id }               

    }])

we could have

    doc.submitOp(addCollaborator(doc, id))

or better yet

    addCollaborator(doc, id)

where "addCollaborator" comes from a sort of "service layer" collection of modules.

 

When handling remote ops, you have the choice of using the ShareDB-managed snapshot available on doc.data, or using the ‘op’ event handler and applying the OT operation yourself using ottypes such as json0, which is not actually that hard - this is what the bindings do.

For Flux/Redux you could use doc.data as your top-level state, but be warned that ShareDB directly mutates it. Alternatively you could turn each ‘op’ event into an Action and use ottypes to update your central state. This is what I do but I don’t use deeply nested models so I can’t offer any specific pointers there.

 

This is interesting. It almost seems that the ShareDB doc API overlaps in functionality with Redux. Both accept "actions" (OPs), and both let you know when things change. Perhaps we can try treating the ShareDB doc similarly to a Redux store and see if that helps clean up our code.

— John

Thanks again John for your thoughts. It's an interesting discussion.

— Curran

P. S. If anyone here is available for remote freelancing and knows ShareDB well, please let me know - cur...@datavis.tech .

[Stian Håklev]

Hi Curran,

we use ShareDB heavily in our app, and really want the underlying components (actually plugins) to not have to worry to much about the mechanics. I wrote a long post very recently [here](https://groups.google.com/d/msg/sharejs/N7QBY-qI2O4/EhFPxF2GAgAJ). You can look at our generateReactiveFunctions file (https://github.com/chili-epfl/FROG/blob/develop/frog-utils/src/generateReactiveFn.js). Basically we give to the plugins data and dataFn as a prop. Data is the current state of doc.data, and dataFn is the result of the file I mentioned above, which has a reference to the doc, and wraps the op-types. This is working pretty well for us, although I'm sure there are things that could be improved. 

We also have an example of a collaborative text field, which we can "mount" onto any of the paths of the document. Here: https://github.com/chili-epfl/FROG/blob/develop/frog-utils/src/ReactiveText.js

Happy to continue discussing ShareDB, React integration, best practices etc. Feel free to contact me directly as well. sha...@gmail.com