Status messages

[Neil Fraser]

On Aug 12, 2:49 am, Joe Walker <jwal...@mozilla.com> wrote:
> The Mozilla version of mobwrite adds the following:
> h/H: Handle. A username:ip-addr combination, 'client' header but added
> by the server, so it can be trusted. Needed to handle non world-write
> access.
> C: Collaborators. Server added header informing the client of handles
> of other clients collaborating on the file.
> x/X: Close view. The client may be able to tell the server to close a
> view, reducing memory consumption, and allowing more accurate display
> of collaborators.
> E: Error: Server added header informing the client of a failure
> condition.
> O: Read Only: Server added to inform the client that access to the
> given file is read-only. Attempts to alter it will be ignored.
> S: Save state: The mobwrite version of the file is not considered
> 'final' until saved. This server added header informs clients if the
> file has changed since editing. (Planned, not implemented)
> q: Debug. This is a temporary fix to allow additional debug to be
> displayed on the server console.

I'm *finally* able to start work on the status functionality
(conflicting internal priorities here at Google got in the way). The
idea is for each client to optionally submit some status information
along with their regular sync. The server would rebroadcast this
information to other clients. Thus a client could add a coloured
marker in the text to represent a collaborator's cursor or selection.
Next to your scrollbar could be a display showing where collaborators
are looking within the document. A nickname for each collaborator
could be displayed so that you know "is anybody there?". Likewise
their current idle time could be displayed so you know if they are out
to lunch.

The format I had envisioned:
S: {"cursor":27,"top":7,"bottom":50,"user":'"Neil Fraser","idle":
120,"msg":"Out to lunch"}
None of these fields would be trusted, none would be required, no
delivery is guaranteed. From the server's point of view the whole
{...} is an opaque block to to be received, stored temporarily and
replayed to other clients. From the clients' point of view it's a
JSON object with arbitrary parameters. Conventions would be needed to
support these parameters, here are some suggestions:
* 'cursor' would indicate the number of characters from the start of
the document to the cursor.
* 'top', 'bottom', 'left' and 'right' would be the offset in
characters (left/right) or lines (top/bottom) from the upper-left
which describes what is currently visible to the user.
* 'select' would indicate the offset location from cursor for where
the current selection ends. Thus for the string "my cat", {cursor:
6,select:-3} would indicate the cursor is at the end of the string and
'cat' is selected. if 'select' is absent or 0, there is no selection.
* 'user' is a human-readable name of the human user.
* 'idle' is the number of seconds since the user interacted with the
document. (Does this mean mouse movement, cursoring, or diff finding
a local change?)
* 'msg' is a user-provided status message, like in Google Chat.
* 'foo-bar' is a namespace convention where rich client 'foo' has a
message of type 'bar' it wishes to send, without worrying that some
other client has defined an incompatible extension using the same
name. E.g. 'moz-temerature'.
If the client sends "S: {...}", it is submitting a status message and
it requests in response all the status messages from other clients.
If the client sends "s: {...}" it is submitting a status message, but
does not require receipt of messages from other clients. If the
client sends nothing, it will get nothing.
The server would have an internal timeout (default: one minute) after
which it will discard and stop rebroadcasting a client's last status
message if it has not heard from the client in that time. A courteous
client would send a null "s:" to the server on disconnect to indicate
its departure.

Just to be totally clear, this status line is COMPLETELY OPTIONAL.
Even if supported, which of the above fields are read or written is
also completely up to the author. It enables richer clients to
exchange information between each other. The non-existence of this
functionality is fine, there are no reverse-compatibility issues with
having a client which supports this and a server which does not, or
vice versa, or a mixture of supporting and non-supporting clients.

Joe, you mentioned that Bespin has already implemented this, so since
you are the first-mover I'd like to maintain compatibility with your
implementation. But I don't see this field in the protocol extensions
listed above. Could you give me a spec for such a field if you have
one?

[Brian Slesinsky]

Although the software may not trust it, users will, and they could
easily be spoofed. I don't think it's a problem with mobwrite URL's
shared with a few friends over chat, so long as those URL's are very
hard to guess.

- Brian

[Joe Walker]

On Wed, Dec 2, 2009 at 2:06 AM, Neil Fraser <neil....@gmail.com> wrote:

...

Joe, you mentioned that Bespin has already implemented this, so since
you are the first-mover I'd like to maintain compatibility with your
implementation.  But I don't see this field in the protocol extensions
listed above.  Could you give me a spec for such a field if you have
one?

https://wiki.mozilla.org/Labs/Bespin/DesignDocs/Collaboration/Mobwrite

I'll try to update the doc to be more complete by the time you get into work ;)

Joe.

[Joe Walker]

 
My notes are probably not well formed enough for the wiki yet:

- Our format is currently { c: { s: cursor.startOffset, e: cursor.endOffset } }
  - it's more compact. There are clear pros and cons to this
  - we support selection in addition to simple cursor (although the UI doesn't display this)
- I'm not keen on having untrusted usernames. There are too many dangers for my liking. I can see it's value in semi-trusted environments like inside a firewall, and for low value sharing, but the potential for abuse is too high for my liking
- We use m for Metadata in place of S, but that's a small thing
- I punted on the whole status thing for now, but this seems like a good way of doing it.
- We mandate JSON by parsing on the server. Some form of server side validation is required, and this seemed like a simple solution.

We've implemented this already. I've not pushed the changes from the bespin repo to my HG mobwrite clone, but I can so if you're interested.

Joe.

[Neil Fraser]

On Dec 2, 2:25 am, Joe Walker <j...@getahead.org> wrote:
> - Our format is currently { c: { s: cursor.startOffset, e: cursor.endOffset} }

Hmm, this is very web-DOM specific. I'd prefer a more platform-
neutral method of specifying the cursor/selection location.

> - I'm not keen on having untrusted usernames. There are too many dangers for
> my liking. I can see it's value in semi-trusted environments like inside a
> firewall, and for low value sharing, but the potential for abuse is too high
> for my liking

Your and Brian's objections are noted. The MobWrite service doesn't
do any user authentication, that's not its job and this suggested
field muddies the issue. Let me think about this some more.

> - We use m for Metadata in place of S, but that's a small thing

'm' sounds good. Can we use 'M' vs 'm' where the former means the
client expects a response whereas the latter means no response is
expected?

> - We mandate JSON by parsing on the server. Some form of server side
> validation is required, and this seemed like a simple solution.

This makes sense. That way the client can trust that the responses
are safe, so JavaScript can merely call eval() rather than invoke an
expensive parser.

I'll start the server-side upgrades right now.

[Joe Walker]

On Wed, Dec 2, 2009 at 7:43 PM, Neil Fraser <neil....@gmail.com> wrote:

On Dec 2, 2:25 am, Joe Walker <j...@getahead.org> wrote:
> - Our format is currently { c: { s: cursor.startOffset, e: cursor.endOffset} }

Hmm, this is very web-DOM specific.  I'd prefer a more platform-
neutral method of specifying the cursor/selection location.

> - I'm not keen on having untrusted usernames. There are too many dangers for
> my liking. I can see it's value in semi-trusted environments like inside a
> firewall, and for low value sharing, but the potential for abuse is too high
> for my liking

Your and Brian's objections are noted.  The MobWrite service doesn't
do any user authentication, that's not its job and this suggested
field muddies the issue.  Let me think about this some more.

To put my issue into words, how do you explain to your Aunt Flo the difference between a website like Facebook where "Joe Walker" been authenticated and a Mobwrite one where "Joe Walker" who hasn't, when Aunt Flo suffers from the same TL;DR as everyone else on the net.

I find Etherpad interesting. I've frequently trusted the username field there even though I know it to be untrustworthy. This is because almost always I've been on Skype/IM when using Etherpad, so I auth through another channel.

Perhaps allowing developers to use something they dream up to store usernames is just as easy, and allows for use-cases like Etherpad, without encouraging more widespread use of something more dangerous. i.e. Just don't specify a username field???

> - We use m for Metadata in place of S, but that's a small thing

'm' sounds good.  Can we use 'M' vs 'm' where the former means the
client expects a response whereas the latter means no response is
expected?

Sounds good to me.
 

> - We mandate JSON by parsing on the server. Some form of server side
> validation is required, and this seemed like a simple solution.

This makes sense.  That way the client can trust that the responses
are safe, so JavaScript can merely call eval() rather than invoke an
expensive parser.

It also means that the server could alter the JSON if required, e.g. by adding a trusted username.

Joe.

[Brian Slesinsky]

On Dec 2, 3:06 pm, Joe Walker <j...@getahead.org> wrote:

> To put my issue into words, how do you explain to your Aunt Flo the
> difference between a website like Facebook where "Joe Walker" been
> authenticated and a Mobwrite one where "Joe Walker" who hasn't, when Aunt
> Flo suffers from the same TL;DR as everyone else on the net.

One thing that would help a little is calling it a "nickname" in the
UI.

[Paul Stone]

> Your and Brian's objections are noted.  The MobWrite service doesn't
> do any user authentication, that's not its job and this suggested
> field muddies the issue.  Let me think about this some more.

Could the q.(py|php|jsp) files be changed to allow a username field to
be easily (and optionally) set by the server.

It might also be an idea to refactor the 'q' scripts into a function
that can be included and called (that does all the work of
communicating with the daemon) plus an example file. This would make
integrating mobwrite into existing apps easier e.g:

q.php:
require_once 'mobwrite-gateway.php';
mobwrite_gateway();

This could then be easily adapted by the user to something like:

require_once 'mobwrite-gateway.php';
session_start();
if (empty($_SESSION['username'])) {
header("HTTP/1.0 401 Authorization Required");
} else {
mobwrite_gateway($_SESSION['username']);
}

[Neil Fraser]

On Dec 2, 2:25 am, Joe Walker <j...@getahead.org> wrote:

> - Our format is currently { c: { s: cursor.startOffset, e: cursor.endOffset} }

Oh, I take back what I said about this being web-centric; I was
thinking of the full interface of Range locations (which includes
nodes) not just the character-level offsets within a textarea.

However there is one small hole. In a selection the cursor is either
at the start or the end of a selection. The DOM API does not provide
information other than the boundries of the selection. Nor does it
provide an API for restoring the cursor -- which always appears at the
end of the selection. Even if this information cannot be used by web
clients, it would be smart to encode the cursor reporting format such
that we do not handicap more capable clients do have access to this
information.

So I think that having a cursor offset variable, and an optional
relative selection length variable (which could be posative or
negative) gives us the ability to define left-hanging or right-hanging
selections. Converting this to DOM cursor specs (with the loss of one
bit of information) is:
selection = selection || 0;
startOffset = Math.min(cursor, cursor + selection);
endOffset = Math.max(cursor, cursor + selection);


Brian and I have been discussing the issues of trust and
authentication. Like Bespin, we've concluded that the server must
become an active participant in message by injecting a trusted ID.
Our reasoning is that this is required for persistence. Iif a client
gets a message from each of two clients, it can pick arbitrary colours
then display a red cursor at one location and a blue cursor at the
other location. But a few seconds later when the client gets two more
messages, it needs a way of knowing which is the blue client and which
is the red client. One solution is for clients to pass a randomly
created identifier along with their messages. The issue here is that
armed with the identifiers of all the other clients, any client could
send spoof messages. Another solution is for clients to use public/
private keys and sign their messages. The issue here is that I have
no desire to reimplement PGP in JavaScript just to push a cursor
around. The only good solution is to do what Bespin has already done,
and add an ID to every message it receives. On MobWrite's case this
ID doesn't mean anything with regard to who the person is, it's just a
persistent label for that browser window to allow for consistency.

There are two ways of doing this. One is to inject an ID parameter
into the data as Bespin has done. Thus a client would send the
following:
{"c":2}
Then receive the following array:
[{"U":"abc","c":7},{"U":"xyz","c":9}]
Alternatively the server could leave the data untouched, and form a
named hash instead:
{"abc":{"c":7},"xyz":{"c":9}}
Advantages of the latter include a) guarantee of no duplicate messages
from the same client, b) faster lookups by client, c) shorter, d)
clear separation of server and client data. The main disadvantage we
see is that this is not what Bespin has implemented.

Joe, other than "c":{"s":n,"e":m} what other fields have you defined?

[Neil Fraser]

On Dec 3, 3:07 am, Paul Stone <stoned...@gmail.com> wrote:
> This could then be easily adapted by the user to something like:
>
> require_once 'mobwrite-gateway.php';
> session_start();
> if (empty($_SESSION['username']))  {
>    header("HTTP/1.0 401 Authorization Required");} else {
>
>    mobwrite_gateway($_SESSION['username']);
>
> }

Alternatively, add this to the top of q.php:

if (empty($_SESSION['username'])) {
header("HTTP/1.0 401 Authorization Required");

exit();
}

And later on:
text = "A:" + $_SESSION['username'] + "\n" + text;

Where A is a new field.
(forgive the syntax errors, PHP is not one of my native languages)

[Paul Stone]

> Alternatively, add this to the top of q.php:
>
> if (empty($_SESSION['username']))  {
>   header("HTTP/1.0 401 Authorization Required");
>  exit();
> }
>
> And later on:
> text = "A:" + $_SESSION['username'] + "\n" + text;
>
> Where A is a new field.
> (forgive the syntax errors, PHP is not one of my native languages)

My reason for suggesting putting all this into a function is that
developers who are integrating mobwrite into their application
probably don't want to be manipulating / parsing mobwrite protocol
messages. They'll just want a simple way to handle authentication
without worrying about breaking the protocol (e.g does it matter if
the username goes at the start or end of the message, should there be
a space after the colon, do I have to escape any characters in the
username, should it be \n or \r\n etc..)

Yes, adding a username like above is very simple, but I expect there
will be other things happening at the gateway level in future like
authorisation, i.e. can a user read / write this field? To do that, it
will be necessary to parse out each field name from the message and
then modify the message after the auth checks have been made.

It'd be much cleaner to have simple hooks to do these things, and hide
the implementation details from the developer. Should the protocol
change between versions, they won't have to change their code much, if
at all.