What is cometd?
Martin Tyler
interested seeing the buzz around ajax/comet in recent months.
My question 'What is cometd?' is not meant as a simple newbie question,
it is not clear to me exactly what it is from the information
available. Is it a server implementation? is it the protocol spec? Does
the protocol have a seperate name and other servers can implement it?
Is there a client side library implementation, and is that part of
cometd or just a reference/example implementation?
Where do the transport types fit in? are the implementations of
different transports optional in a server/client?
Is there meant to be any API/spec for the backend side of a cometd
application? is the backend application meant to receive the JSON
messages directly? What does cometd give the backend - authentication,
session management, subscription management? Is there any support for
permissioning of channels?
Regards
Martin
Alex Russell
> Hi, I have worked commercially in this area for 9 years so I have
> been interested seeing the buzz around ajax/comet in recent months.
>
> My question 'What is cometd?' is not meant as a simple newbie
> question, it is not clear to me exactly what it is from the
> information available. Is it a server implementation?
Yes.
> is it the protocol spec?
Unfortunantly, yes. We were just discussing on IRC this evening how the
protocol needs a different name than the reference implementations.
Suggestions welcome = )
> Does the protocol have a seperate name and other
> servers can implement it? Is there a client side library
> implementation, and is that part of cometd or just a
> reference/example implementation?
I'm implementing the client side at:
http://trac.dojotoolkit.org/browser/trunk/src/io/cometd.js
The server I'm developing against is:
http://svn.xantus.org/shortbus/trunk/cometd-twisted/cometd.py
A description of the protocol thus far is at the top of the python
implementation.
> Where do the transport types fit in? are the implementations of
> different transports optional in a server/client?
We haven't gotten far enough to define what's mandatory and what's not,
but my current feeling is that the "long-polling" and "http-polling"
transport types will be mandatory while others like "iframe",
"callback-polling", and "mime-message-block" will be optional. However,
once the basic connection types are supported, the others are pretty
straightforward (if not downright trivial).
> Is there meant to be any API/spec for the backend side of a cometd
> application?
TBD, I think, although we clearly need good interfaces.
> is the backend application meant to receive the JSON
> messages directly? What does cometd give the backend -
> authentication, session management, subscription management?
Most of that has been punted in the spec and the implementations thus
far. Your input on these topics would be most welcome.
Right now I'm imagining the authentication and authorization as
hook-able, negotiated properties on both sides of the connection, with
a default open-passthrough policy. Since we're plaintext on the wire,
we do have some constraints.
> Is there any support for permissioning of channels?
Nope. We still have to flesh out globbing = )
What do you suggest that this should look like?
Regards
--
Alex Russell
al...@sitepen.com A99F 8785 F491 D5FD 04D7 ACD9 4158 FFDF 2894 6876
al...@dojotoolkit.org BE03 E88D EABB 2116 CC49 8259 CF78 E242 59C3 9723
Martin Tyler
I would have thought the seperation of components would be something
like this, which may be what you are doing already or have planned...
An implementation of a client side library, this uses a defined spec to
talk to a server. The cometd server would just be 'an implementation'
again. This then leaves the backend. I think this would be up to the
cometd server, for example you might like to come up with another
spec/API for the backend - but it feels to me that this would be more
implementation specific rather than a standard spec/API that all
implementations of a cometd server has to implement. This decouples the
client and backend API's, which allows people to implement a simple
cometd server without having to implement a backend API.
Subscription management is something that is very useful to implement
in the server rather than letting a backend app have to deal with
knowing where to send everything. I guess it depends on what style of
apps people might use this for, but for apps that send messages that go
to multiple subscribers you really want that handled for you in an
efficient way.
Hookable auth is good (another API), permissioning of channels can also
fall into this area - probably read/write permissions.
One other thing to consider is message size. When/if you get to the
point of an application that can support high message rates, especially
to high numbers of users, then every byte counts and can have a
dramatic effect on performance. Your messages dont look overly verbose
(unlike some XML based protocols), but it might be something to
consider at this stage before it's too late to change things.
Regards
Martin
Alex Russell
> Hi Alex,
>
> I would have thought the seperation of components would be something
> like this, which may be what you are doing already or have planned...
> An implementation of a client side library, this uses a defined spec
> to talk to a server.
Yep, that's what we're working against right now. The complicating
factor is that we're still working out what the "right"
envelope/preamble/tunnel-init/signoff messages are per pluggable
transport type. I'm evolving those in the python implementation right
now, but I think they can (and should) be standardized such that the
protocol for what gets sent over the wire is fully-constrained.
> The cometd server would just be 'an
> implementation' again.
Exactly. We're pursuing 2 server implementations right now to proove
interop (and I'll probably write a second client that doesn't have a
Dojo dependency).
> This then leaves the backend. I think this
> would be up to the cometd server, for example you might like to come
> up with another spec/API for the backend - but it feels to me that
> this would be more implementation specific rather than a standard
> spec/API that all implementations of a cometd server has to
> implement. This decouples the client and backend API's, which allows
> people to implement a simple cometd server without having to
> implement a backend API.
I'm not sure I grok "backend" in this context. I can think of a couple
of different layers which could be specified here. What are you
thinking of in particular?
> Subscription management is something that is very useful to implement
> in the server rather than letting a backend app have to deal with
> knowing where to send everything.
Agreed. The python version right now is just a dumb topic router...even
apps would just "throw" messages at it rather than knowing anything
about the channel dispatch mechanism specifically.
> I guess it depends on what style of
> apps people might use this for, but for apps that send messages that
> go to multiple subscribers you really want that handled for you in an
> efficient way.
Yep. I don't know that my current implementation is very efficient, but
I suspect that it's not a horrible first pass.
> Hookable auth is good (another API), permissioning of channels can
> also fall into this area - probably read/write permissions.
Would you view that as part of the spec? Or server-dependant?
> One other thing to consider is message size. When/if you get to the
> point of an application that can support high message rates,
> especially to high numbers of users, then every byte counts and can
> have a dramatic effect on performance. Your messages dont look overly
> verbose (unlike some XML based protocols), but it might be something
> to consider at this stage before it's too late to change things.
My initial latency concerns revolve around message batching. In a lot of
transport types it's very possible for there to be many messages
delivered to a client between client connects, and I want to be able to
specify a way to multiplex multiple event delivery along a single
message such that the envelope overhead is minimized.
Beyond that, I'd love to know what your thoughts on the current
implementation and proposed spec are.
Martin Tyler
> I'm not sure I grok "backend" in this context. I can think of a couple
> of different layers which could be specified here. What are you
> thinking of in particular?
The backend in my view is the application on the server, not cometd,
but whatever is behind it with the application specific logic. In my
world this is generally some kind of data feed handler which uses an
API/Library into our system, but it could be any kind of application
wanting to send and receive realtime data with clients. This can work
in lots of ways, broadcasting data into the server (cometd) which
caches them for subscriptions, or the server could subscribe to
channels from the application only when clients require them. The
second way requires a bit more work for the application but is a better
model.
In some systems the backend application doesnt need to know anything
about the clients, ie how many or who is subscribed.. but other apps
will need private channels and the backend application would be
providing data specific to a client.
> > Hookable auth is good (another API), permissioning of channels can
> > also fall into this area - probably read/write permissions.
>
> Would you view that as part of the spec? Or server-dependant?
I think server dependant. Auth/security requirements can complicate
things, and like with the backend API, you dont want barriers
preventing people implementing applications using your cometd client
side protocol. The client side part of the auth, ie the fields you have
specified in your messages etc can easily be part of the spec, but how
the server handles this with a hookable API is probably best as a
server dependant feature.
> My initial latency concerns revolve around message batching. In a lot of
> transport types it's very possible for there to be many messages
> delivered to a client between client connects, and I want to be able to
> specify a way to multiplex multiple event delivery along a single
> message such that the envelope overhead is minimized.
Yes, batching together of messages on fast updating data can be a big
win, regardless of whether you are reducing the overall message size
due to header/envelope, it helps lots of things deal with the load in a
better way. Obviously it makes the server implementation more
difficult, but it's worth it.
> Beyond that, I'd love to know what your thoughts on the current
> implementation and proposed spec are.
I'll have a better read of the spec soon and post my thoughts.
Regards
Martin