A few questions re: OpenCoweb

[Nick Fitzgerald]

Hello everyone,

I am very psyched on what I have seen of OpenCoweb so far (mainly just the shopping cart tutorial, and a glance through the JSOE source). I have a few questions:

What is the recommended way to save documents on the server? My understanding is that this is not necessarily built in at this point. Correct me if I am wrong, but it is also my understanding that the Operational Transformation engine is only implemented in JavaScript, and not for the servers, and this is why the server side doesn't know the whole state of the document. Would it make sense to implement a bot which has its own representation of the document and applies operations to it and saves them to some database on the server? Is this the kind of thing that bots are used for?

Aside: if my understandings are correct, this seems to be a pretty good argument for a nodejs/some other commonjs server, so that it could reuse the JSOE on the server side and save documents on the server in a much easier fashion rather than have to reimplement it all. This is why I decided to use nodejs for my little OT implementation[0] rather than something like Erlang.

If I were to use coweb to implement simple collaborative editing of a plain text document, would I just make the textbox a single item with a single ID, or would I try and break it down line by line and perform updates on each line and inserts for new lines, etc? I guess I am having a hard time translating between the unordered, each item in a shopping cart stands by itself, paradigm of the tutorial in to the paradigm I am more familiar with of inserts/deletes/retains at the character level which is used by Wave and described here[1]. For example, in the shopping cart tutorial, if two users simultaneously change the name of an item, will both of their changes propagate to each other and a combination of them results, or will one of them win? So would something like this happen:

    Apples
   /      \
Oranges  Bananas
   \      /
  OrangesBananas

or something like this:

    Apples
   /      \
Oranges  Bananas
   \      /
   Bananas

?

Thanks for your patience,

_Nick_

[0]: See https://github.com/fitzgen/operational-transformation and https://github.com/fitzgen/operational-transformation-example

[1]: http://www.codecommit.com/blog/java/understanding-and-applying-operational-transformation

[Brian]

Nick,

You are correct in that the op engine is only implemented on the
client in JS.

I think your line of thought is correct regarding a bot. The bot
would be a good place to put a server side op engine. You wouldn't
have to implement the entire server in nodejs you would only have to
write a nodejs bot wrapper. This could be modeled closely to the
python bot wrapper. At that point you could write a javascript bot
running in node and in bring in the op engine.

We would have change a couple of things in the current server
implementation so that your bot has the correct initial state info.
Right now bots do not receive late join state. Your bot would need
that. Also bots can receive application sync messages but I don't
believe they can receive op engine sync messages. Your bot would need
those as well. These are relatively minor changes to the server.

I think this is a good discussion to have. Let's keep this thread
going.

Regards,
Brian

> [1]:http://www.codecommit.com/blog/java/understanding-and-applying-operat...

[Nick Fitzgerald]

I think your line of thought is correct regarding a bot.  The bot
would be a good place to put a server side op engine.  You wouldn't
have to implement the entire server in nodejs you would only have to
write a nodejs bot wrapper.  This could be modeled closely to the
python bot wrapper.  At that point you could write a javascript bot
running in node and in bring in the op engine.

Ok, that sounds much nicer to implement and maintain. I haven't dug in to the JSOE too much, but am I correct in assuming that it doesn't rely on any browser-specific APIs? (Aside: I really like how AMD gently pushes you to separate browser-specific code from the rest of your app's logic :) ).
 
When I get some time to start working on a nodejs bot wrapper, would it make sense for it to (eventually) live in the coweb repository, or should it stay separate?

We would have change a couple of things in the current server
implementation so that your bot has the correct initial state info.
Right now bots do not receive late join state.  Your bot would need
that.  Also bots can receive application sync messages but I don't
believe they can receive op engine sync messages.  Your bot would need
those as well.  These are relatively minor changes to the server.

What is the difference between an application sync message and an operation engine sync message? I couldn't find anything in the docs; I apologize if I just missed it.

If there is interest, I would be willing to dive in to the python server and work out a pull request to implement these changes, but I might need a little bit of hand holding at the beginning as far as just figuring out where the changes should be in the source, etc as I acquaint myself with the code base ;)

_Nick_

[Nick Fitzgerald]

Quick question: what is "pbs"? It is referred to a few times in here: http://opencoweb.org/ocwdocs/protocol/bot_bayeux.html

_Nick_

[Nick Fitzgerald]

Excuse me, it is actually referred to here: http://opencoweb.org/ocwdocs/python/bots.html

_Nick_

[Peter Parente]

That's a leftover reference in the doc to the IBM project where opencoweb started. The code is correct, but the doc needs an update. Thanks for reporting it.

[Brian Burns]

pbs is short for Project Blue Spruce.  This is the internal IBM project which opencoweb came from.  Those references need to be changed in the docs.

As far as the open engine syncs vs application syncs, check out this issue discussion. https://github.com/opencoweb/coweb/issues/59 .  We just started separating these for another feature called moderated sessions.  This is where you may want to have only a few moderators driving the session and the rest of the participants are just listening / readonly.  

The nodejs bot wrapper could live in either coweb or cowebx.  We should setup a directory structure for other bot wrapper implementations.  The python server has the ability to launch bots in a separate process which is ideal for multi language bots.  The Java server does not have this ability but we should try and remain neutral between the 2 servers impls.  

If you would like you could join us on our team call on Tuesday 11pm EST.  

Brian

[Peter Parente]

On Friday, April 29, 2011 4:09:48 PM UTC-4, fitzgen wrote:

I think your line of thought is correct regarding a bot.  The bot
would be a good place to put a server side op engine.  You wouldn't
have to implement the entire server in nodejs you would only have to
write a nodejs bot wrapper.  This could be modeled closely to the
python bot wrapper.  At that point you could write a javascript bot
running in node and in bring in the op engine.

Ok, that sounds much nicer to implement and maintain. I haven't dug in to the JSOE too much, but am I correct in assuming that it doesn't rely on any browser-specific APIs? (Aside: I really like how AMD gently pushes you to separate browser-specific code from the rest of your app's logic :) ).

I don't think there's anything in the op engine that's browser specific, but we didn't really take that into mind when writing it. If there is, it can surely be removed.

 

What is the difference between an application sync message and an operation engine sync message? I couldn't find anything in the docs; I apologize if I just missed it.

Op engine sync message is used for operation garbage collection. When sites are "inactive", they need to occasionally send a context vector representing the highest op received from each site so other sites can throw away ops that are no longer needed.

[Peter Parente]

If you don't care about the session history at the level of individual operations, an alternative to doing any OT on the server would be to take full-state snapshots at some interval. The server could be extended or a bot written to request full state from a user in the session for persistence. The last state request would become the initial state provided by the server / bot at the start of the next session instance.

This approach is far simpler if you're willing to sacrifice a fine-grain history. The one difficulty I foresee is guaranteeing that the final app state is persisted as the last user is leaving the session. Some browsers (e.g., Mobile Safari) are very quick to squash network activity on page unload, possibly preventing the final app state from reaching the server / bot.

[Nick Fitzgerald]

I think that requesting full-state snapshots at an interval might be enough for the things which I am thinking of. Would I just have the bot use the bot wrapper's "publish" method to request the full state and go from there?

How would I join the call? Is it on Skype?

Take it easy,

_Nick_

[Vinomaster]

Call in number for our weekly mtg (11am EST Tue) will be posted in IRC channel prior to the call.

[Vinomaster]

Nick -

Hopefully you can join us in an upcoming project team call. If you can not make it this week, let us know when so we can add this topic to a forthcoming agenda. I will add it for this week just in case you can join us.

I would like to ask you to consider outlining your requirements in a issue for consideration. Issue #56 https://github.com/opencoweb/coweb/issues/56

is actually an open ended issue at this time that touches on the need for a Rich Text Editor Co-web App.

Once we better understand the requirements for the app we can then collectively discuss the appropriate design and areas of impact. As you can see from issue #56 we are also interested in such a coweb editor. 

[Nick Fitzgerald]

Vinomaster,

I don't have any specific requirements because I don't have anything too specific in mind. I just know that whatever I do, I don't want it to be ephemeral; I'll need to persist the application's state and be able to bring it back again. I'm still in the experimentation phase ;)

I don't think I have too much to contribute to the meeting, but I would love to listen in, if that is alright. I'll be in IRC tomorrow morning.

Take it easy,

_Nick_