ShareDB with Redux?

[Curran Kelleher]

Greetings,

Has anyone worked with ShareDB and Redux together?

Redux seems like a great pattern that would be compatible with ShareDB with certain glue. I came across immutable-js-diff and immutable-js-patch, which may be directly mappable to/from the json0 OT Types. For strings, maybe sharedb-string-binding could also be hooked into the Redux fold somehow. Anyone working in this direction?

Awesome work Avital on the Leaderboard Demo with React. Great stuff!

Best regards,

Curran

[Richard]

I don't know if anyone has done this, but I've been wondering about it, myself.

-Richard

[Jonatan Lundin]

Hey guys! I have also been looking in to this lately. 

I'm currently playing with the idea of writing a set of RxJS bindings for the ShareDB client (which would also be useful 

outside of this context) and then use redux-observable to fit those in to our Redux app. Together with something like 

the diff/patch libraries you linked we would have most of the necessary pieces for a first implementation.

The question then is what actions (and action creators) would be needed and what the state should look like. 

I've drafted a rough outline of what I think would be needed bellow:

## State shape ##

Collections shape

{ [name]: <collection reducer> }

Collection shape

{

  subscriptions: {

    [query]: {

      isFetching: <bool>,

      subscribers: count,

      result: [list of doc ids]

     }

    // ...

  },

  requests: {

    [query]: {

      isFetching: <bool>,

      lastUpdated: <timestamp>,

      result: [list of doc ids]

    }

  },

  documents: {

    [id]: snapshot,

    // ...

  }

}

(The collection reduces should probably also be broken in to smaller pieces...)

## Actions ##

SHARE_FETCH (collection, query)

SHARE_SUBSCRIBE (collection, query)

SHARE_UNSUBSCRIBE (collection, query)

SHARE_FETCH_DOC (collection, id)

SHARE_SUBSCRIBE_DOC (collection, id)

SHARE_UNSUBSCRIBE_DOC (collection, id)

SHARE_OP_ * (collection, id, payload) (eg, insert, update, increment etc)

SHARE_UPDATE (collection, { id: snapshot, ... })

SHARE_FETCH_SUCCESS

We could then write a set of action creators that diffs against the state as needed in order to generate the required OPs. 

All actions (except SHARE_UPDATE and SHARE_FETCH_SUCCESS) would then be handled by the RxJS layer, 

which would emit SHARE_UPDATE actions whenever a query result or document changes and 

SHARE_FETCH_SUCCESS once a fetch completes. A set of generic selectors could then be used to get the correct 

documents/requests states/etc for each query or fetch.

(The state shape and action list above is obviously not complete. Some additional props and actions are needed in order

to properly handle errors, etc.)

The main drawback of this approach is that we cant use plain reducers to describe the state changes for any of our 

ShareDB documents/collections. Some kind of solution that observers the redux state and then dispatches OPs 

accordingly would fix that, but then we have a whole new set of problems of matching state to ShareDB docs and 

probably a bit too much magic going on behind the scenes. One of the main benefits of redux IMO is the whole 

no-magic thing.

Doing text based OP would probably require a custom input component as well, in order to not mess with the 

cursor/caret too much. DraftJS (or maybe Quill) could be a good fit if you want to do more than just plain text. 

What are your thoughts on this?

Cheers,

Jonatan

[Curran Kelleher]

It's awesome to see there is some interest in this!

It's a cool idea to involve RxJS, but my gut feeling is that it should be possible to do with Redux only (and perhaps redux-thunk).

Some references that may be relevant:

• Full-Stack Redux Tutorial - This tutorial does real-time synchronization over WebSockets, but with a simple implementation that transfers the full state all the time. The technique we maybe can use from here is the section Sending Actions To The Server Using Redux Middleware. Maybe a similar middleware could be used to detect when the state changes on the client and trigger computation of the OPs to send to the server. This matches your idea of "Some kind of solution that observes the redux state and then dispatches OPs accordingly".
  
• Async Actions - Many of the actions you describe (which I think are a great start!) including SHARE_FETCH, SHARE_SUBSCRIBE, SHARE_FETCH_DOC, and SHARE_SUBSCRIBE_DOC could be implemented as thunk action creators (via redux-thunk) that subscribe to ShareDB and dispatch the OP actions as they arrive.

I think you're correct that the text OT would probably require a custom input component that deals with the cursor correctly. The logic for this is implemented by Nate in a fairly recent module - sharedb-string-binding. I wonder if/how this module could be reconciled with Redux - it dispatches and receives the OPs itself in the current implementation. It does accept a ShareDB doc as a constructor argument "StringBinding(element, doc, path)" - maybe that could be replaced with an implementation of the ShareDB Doc API that actually goes through Redux rather than ShareDB directly.

Seeing the responses here makes me feel inspired to try building a prototype of this..

Best regards,

Curran

Datavis.tech

--
You received this message because you are subscribed to a topic in the Google Groups "ShareJS" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/sharejs/MVLSc39NuPg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to sharejs+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jonatan Lundin]

Yeah, it's definitely cool that there seems to be some interest for this!

I started in the end of using thunks, but that doesn't offer enough control. It's hard to keep track of subscriptions, etc, while 

still maintaining a serializable state. It's also hard to interrupt running subscriptions, etc (unsubscribing/change filtering) with 

thunks/promises.

We're currently using Horizon, which is build on top of RxJS. The current version Horizon requires manual re-subscriptions if the 

connection goes down. This is also something that's manageable to do with RxJS, but would quickly turn in to mess otherwise. 

With that said, ShareDB seem to handle reconnections better than Horizon, which would eliminate one of the strongest 

arguments for sticking with RxJS. How to handle unsubscriptions and dynamic filtering would need some though, but you 

could definitely solve those things as well.

Would love to see what you can come up with!

Cheers,

Jonatan

Datavis.tech

To unsubscribe from this group and all its topics, send an email to sharejs+u...@googlegroups.com.

[ry...@retrium.com]

+1

[Curran Kelleher]

I came across an extremely similar project - redux-swarmlog

It uses swarmlog instead of ShareDB.

[Jonatan Lundin]

Yeah, I saw that too yesterday. It’s a lot simpler on a conceptual level. There are a few problems with that approach though if you actually wan too use it in a centralised manner, mostly around how to do access control and auth. 

Replacing swarmlog with (the underlying) hyperlog lib together with websockets and a centralised server could be a way to handle that. Swarmlog could also probably still be used in addition to that for sideways replication if you really need that.

On 21 Sep 2016, at 10:21, Curran Kelleher <curran....@gmail.com> wrote:

I came across an extremely similar project - redux-swarmlog

It uses swarmlog instead of ShareDB.

--
You received this message because you are subscribed to the Google Groups "ShareJS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sharejs+u...@googlegroups.com.

[Curran Kelleher]

I'd like to share what I've got so far on the Redux + ShareDB front - here's a WIP pull request on ShareDB that adds a new example. It contains a collaborative textarea implemented as a React component.

It occurred to me that maybe ShareDB does not fit that well with Redux "all the way down". For example, consider the subtlety of cursor transformation implemented in sharedb-string-binding. Maybe a better approach would be to encapsulate ShareDB-bound documents at the level of React components, and allow the ShareDB documents to be the "source of truth" state containers that they really are, rather than duplicating their state in a Redux store.

Any feedback on the PR would be welcome :)

Best regards,

Curran

[Jonatan Lundin]

I skimmed trough the PR and it looks like a good start! I totally agree that forcing things in to redux doesn't make sense for the text editing case. Great work :)

How would you handle other use cases besides pure text editing, like setting/getting properties, sorting docs, etc, with a component based approach? 

One way could be to borrow a few ideas from recompose on how to wrap components in state with a HoC. We could then use a ShareDB doc as backing instance for that.. Might be worth exploring.  

As a side note, I also spent some time last week implementing a collaborative editing interface with Firebase + redux-observable. Also turned out quite nice. The main benefit of that approach is the decoupling actions provide - all you ever do is describe your intents. Keeps the individual components (react, store, rx epics, etc) small and testable. 

Cheers,

Jonatan

[Curran Kelleher]

Thanks for reviewing the code! I appreciate it.

Thanks also for the pointer to recompose. Implementing full ShareDB integration (with arbitrary JSON Doc trees) seems possible with a Higher-order Component as you suggest. I like the way react-redux works, where the <Provider> gives its descendants access to getState() and dispatch(). Maybe one promising approach would be to mimic this API, but have something like <ShareDBDocProvider> that gives its descendants access to a single ShareDB document - some analog to getState() could return doc.data, and some analog to dispatch() could invoke doc.submitOp() or doc.del().

The PR with the new example is working well and ready for review, with routing implemented with react-router. I'd welcome any feedback. Thank you.

Best regards,

Curran

[Curran Kelleher]

I found a few more "in the wild" projects that are similar to the goal of ShareDB bindings for React:

• pouch-websocket-sync-example (and accompanying article) - This uses Redux middleware to sync Redux store state with remote state (via pouch-redux-middleware).
  
• reactfire - "ReactJS mixin for easy Firebase integration" - Interestingly, they use the approach of creating a Mixin for the integration.
  
• firedux - "Firebase + Redux for ReactJS" - This one introduces a reducer that appears to duplicate the Firebase state inside the Redux store state.

[Wout Mertens]

Some random thoughts from reading this thread, probably already occurred to the participants but adding to make sure:

• Ops should be dispatched as actions, that send them to the server. They can simultaneously be optimistically applied.
• All ops should be kept, at least from the last op the server confirmed, and whenever the server sends new ops they have to be merged with OT
• OT can be done by a reducer since it is a pure operation. It would maintain a log of ops, a snapshot, and pointers to the last ack-ed op and the last in-flight one.
• It is up to the actions to send the ops when the reducer changes those pointers, Redux philosophy.
• You can handle the cursor position in a text field with OT as well, make a text-with-cursors type that gets the same OT applied on cursors as on the text. Cursors are (start,stop) tuples, you can send those positions around as well and see others' cursors.