Real-Time Communication in WEB-browsers (rtcweb)

38 views
Skip to first unread message

Tony Garnock-Jones

unread,
May 13, 2011, 12:43:08 PM5/13/11
to sp-discu...@googlegroups.com
Hi all,

Are we collectively aware of the "Real-Time Communication in WEB-browsers (rtcweb)" working group, https://datatracker.ietf.org/wg/rtcweb/ ? Their usecases look like interesting things to think about in our context, too: https://datatracker.ietf.org/doc/draft-holmberg-rtcweb-ucreqs/?include_text=1

Regards,
  Tony
--
Tony Garnock-Jones
tonygarn...@gmail.com
http://homepages.kcbbs.gen.nz/tonyg/

Brian Granger

unread,
May 13, 2011, 12:44:51 PM5/13/11
to sp-discu...@googlegroups.com
I should note that we are already building web apps that consist of a
very thin adapter between websockets and zeromq sockets. The
impedance mismatch is almost 0 and it is working extremely well.

Cheers,

Brian

> --
> Note Well: This discussion group is meant to become an IETF working group in
> the future. Thus, the posts to this discussion should comply with IETF
> contribution policy as explained here:
> http://www.ietf.org/about/note-well.html
>

--
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgra...@calpoly.edu and elli...@gmail.com

Martin Sustrik

unread,
May 14, 2011, 5:13:40 AM5/14/11
to sp-discu...@googlegroups.com, Brian Granger
What about adding WebSockets to the list of transports to check:

TCP, UDP, SCTP, PGM, WebSockets?

Martin

Pieter Hintjens

unread,
May 14, 2011, 5:18:02 AM5/14/11
to sp-discu...@googlegroups.com, Brian Granger
On Sat, May 14, 2011 at 11:13 AM, Martin Sustrik <sus...@250bpm.com> wrote:

> What about adding WebSockets to the list of transports to check:
> TCP, UDP, SCTP, PGM, WebSockets?

Yes. SP over WebSockets is a killer mix, IMO. WebSockets will be the
key to accessing the cloud...

-Pieter

Dirkjan Ochtman

unread,
May 14, 2011, 8:14:33 AM5/14/11
to sp-discu...@googlegroups.com
On Fri, May 13, 2011 at 18:44, Brian Granger <elli...@gmail.com> wrote:
> I should note that we are already building web apps that consist of a
> very thin adapter between websockets and zeromq sockets.  The
> impedance mismatch is almost 0 and it is working extremely well.

This might be slightly offtopic, but we're doing this as well. Do you
happen to have your adapter code published somewhere? It might be
interesting to compare notes (using pyzmq, no less).

Cheers,

Dirkjan

Brian Granger

unread,
May 16, 2011, 1:13:48 AM5/16/11
to sp-discu...@googlegroups.com
> This might be slightly offtopic, but we're doing this as well. Do you
> happen to have your adapter code published somewhere? It might be
> interesting to compare notes (using pyzmq, no less).

We are using tornado for this. The code is pretty new and hidden in a
development branch for now. But here is the main part of it:

https://github.com/ipython/ipython/blob/htmlnotebook/IPython/frontend/html/notebook/notebook.py

The main logic is in the ZMQStreamRouter:

https://github.com/ipython/ipython/blob/htmlnotebook/IPython/frontend/html/notebook/notebook.py#L55

that connects multiple WebSocket clients to a single zeromq socket on
the backend. We tried using a single zeromq socket per WebSocket, but
that simply created too many sockets on the server (but the design was
simpler). For our particular usage case, having one zeromq socket for
many WebSocket clients works well and also allows us to start/stop the
zeromq socket while leaving the WebSockets connected.

Cheers,

Brian

> Cheers,
>
> Dirkjan

Martin Sustrik

unread,
May 16, 2011, 2:40:58 AM5/16/11
to sp-discu...@googlegroups.com, Brian Granger
Hi Brian,

> that connects multiple WebSocket clients to a single zeromq socket on
> the backend. We tried using a single zeromq socket per WebSocket, but
> that simply created too many sockets on the server (but the design was
> simpler). For our particular usage case, having one zeromq socket for
> many WebSocket clients works well and also allows us to start/stop the
> zeromq socket while leaving the WebSockets connected.

Any experience with the protocol? Did you have to pass any additional
data on top of WebSockets or did the WebSocket protocol do as is?

Martin

Brian Granger

unread,
May 16, 2011, 12:31:45 PM5/16/11
to Martin Sustrik, sp-discu...@googlegroups.com

No, I don't have any experience with the underlying protocol. Our
usage consists of simply forwarding JSON messages back and forth
betwee the WebSocket and the zeromq socket. The only think we have
really had to add was a way to track different WebSocket clients, but
for that we can add a field to our JSON data. These client IDs are
really application level logic though.

The main issue we run into with WebSockets is that a WebSocket allows
*any* send/recv pattern, but zeromq sockets don't. This means that a
WebSocket can cause a zeromq socket to raise an exception in Python be
illegal send/recv pattern (for example send on a SUB socket).

Cheers,

Brian

> Martin

Paul Colomiets

unread,
May 16, 2011, 5:50:36 PM5/16/11
to sp-discu...@googlegroups.com


On Mon, May 16, 2011 at 7:31 PM, Brian Granger <elli...@gmail.com> wrote:

The main issue we run into with WebSockets is that a WebSocket allows
*any* send/recv pattern, but zeromq sockets don't.  This means that a
WebSocket can cause a zeromq socket to raise an exception in Python be
illegal send/recv pattern (for example send on a SUB socket)

We use zerogw (http://github.com/tailhook/zerogw) which is a HTTP and WebSocket server written in C, which talks zeromq with backend. We use push socket to send data to the backend and sub to send data to the client. Push works well because we want to process any incoming message only once. And pub/sub on other side sends message to several zerogw instances, and basically every zerogw instance knows what users it has and forwards data apropriately by websockets (or using http comet for old clients).

Can you describe your aproach in details?


On Sun, May 15, 2011 at 11:40 PM, Martin Sustrik <sus...@250bpm.com> wrote:
Any experience with the protocol?
Yes :)
 
Did you have to pass any additional data on top of WebSockets or did the WebSocket protocol do as is?
We map each websocket message into single zeromq message and add connection_id (generated in zerogw) as message first message part. This way it works perfectly.

But my aim was not fully implement zeromq on top of websockets, but to have a backend protocol using zeromq. This is probably true for all web applications, connections are very expensive in browser, so we try to multiplex several streams with different patterns on top of websocket connection instead of creating multiple sockets.

If you want to discuss usage of websocket protocol as one of the protocols of zeromq without respect to browsers, there are only few problems. Websocket protocol has http handshake, which adds complexity for implementation, but adds extensibility. E.g. you can transmit identity as a header value like "X-Zeromq-Identity: 0xABCD", and use protocol "Sec-Websocket-Protocol: zeromq-pub". The downside is there are no message parts in the protocol. There is message fragmenting in the protocol, but intermediaries do not required to preserve them. Ok, seems the protocols we are speaking about in this ml don't have message parts either. Protocol also has mandatory masking of frames to literally confuse bad written intermediaries (http proxies) to solve their vulnerability problems. Masking has big impact on performance. Protocol also has control messages, among them: ping/pong and close. Each message has extension flags which allows to have some (yet not specified) extension fields on each message. Extesions must be negotiated at the connection time. Otherwise it's just protocol which allows passing messages in both ways without limits.

Does that answer you question?

--
Paul

Martin Sustrik

unread,
May 17, 2011, 6:34:19 AM5/17/11
to sp-discu...@googlegroups.com, Paul Colomiets
Hi Paul,

Thanks for valuable input!

> We use zerogw (http://github.com/tailhook/zerogw) which is a HTTP and
> WebSocket server written in C, which talks zeromq with backend.

Nice project. I haven't noticed it till now :(

> This is probably true for all web
> applications, connections are very expensive in browser, so we try to
> multiplex several streams with different patterns on top of websocket
> connection instead of creating multiple sockets.

Yes. Overhead associated with connection negotiation is often a problem.
And not only in web environment: think of SCTP streams.

From our point of view, I would say, the problem can be restated as
"tunnelling multiple SP feeds over a single underlying connection". I
have some ideas about that, but I would like to postpone the discussion
until at least the framing and labeling is more or less agreed on.

> If you want to discuss usage of websocket protocol as one of the
> protocols of zeromq without respect to browsers, there are only few
> problems.

Please, distinguish between 0MQ as an particular messaging
implementation and SP as (attempt for) a protocol. If, say, your way of
sending messages over WebSocket is standardised, zerogw project would be
SP-compliant, but the code interacting with the websocket would still be
your code, not 0MQ.

> Websocket protocol has http handshake, which adds complexity
> for implementation,

Yes. But AFAIU that's the complexity we just have to accept if we want
to use WebSockets as an underlying transport layer.

> but adds extensibility. E.g. you can transmit
> identity as a header value like "X-Zeromq-Identity: 0xABCD", and use
> protocol "Sec-Websocket-Protocol: zeromq-pub".

I would like to postpone the connection initiation discussion a bit and
focus on framing for now.

> The downside is there are
> no message parts in the protocol. There is message fragmenting in the
> protocol, but intermediaries do not required to preserve them. Ok, seems
> the protocols we are speaking about in this ml don't have message parts
> either.

Multi-part messages are a fancy feature in 0MQ, but are unlikely to
appear even in Linux kernel implementation of SP. The standard way of
addressing the related use cases are gather/scatter arrays.

> Protocol also has mandatory masking of frames to literally
> confuse bad written intermediaries (http proxies) to solve their
> vulnerability problems. Masking has big impact on performance.

Ok. Good to know, but once again, it's something we can't change and
have to simply accept.

> Protocol
> also has control messages, among them: ping/pong and close. Each message
> has extension flags which allows to have some (yet not specified)
> extension fields on each message. Extesions must be negotiated at the
> connection time. Otherwise it's just protocol which allows passing
> messages in both ways without limits.

We'll need "labels" to be attached to SP messages later on (pub/sub
topics, req/rep reply-to's etc.) -- maybe WebSocket extensions could be
used for that.

Martin

Brian Granger

unread,
May 17, 2011, 12:46:17 PM5/17/11
to sp-discu...@googlegroups.com
On Mon, May 16, 2011 at 2:50 PM, Paul Colomiets <pa...@colomiets.name> wrote:
>
>
> On Mon, May 16, 2011 at 7:31 PM, Brian Granger <elli...@gmail.com> wrote:
>>
>> The main issue we run into with WebSockets is that a WebSocket allows
>> *any* send/recv pattern, but zeromq sockets don't.  This means that a
>> WebSocket can cause a zeromq socket to raise an exception in Python be
>> illegal send/recv pattern (for example send on a SUB socket)
>
> We use zerogw (http://github.com/tailhook/zerogw) which is a HTTP and
> WebSocket server written in C, which talks zeromq with backend. We use push
> socket to send data to the backend and sub to send data to the client. Push
> works well because we want to process any incoming message only once. And
> pub/sub on other side sends message to several zerogw instances, and
> basically every zerogw instance knows what users it has and forwards data
> apropriately by websockets (or using http comet for old clients).
> Can you describe your aproach in details?

Good to know about this project. Our backend uses a more application
specific set of zeromq sockets, so we have different socket types
(XREP/XREQ, PUB/SUB). But as we scale it up, we will probably end up
with something similar to what you have, which is in turn similar to
Mongrel2.

> --
> Note Well: This discussion group is meant to become an IETF working group in
> the future. Thus, the posts to this discussion should comply with IETF
> contribution policy as explained here:
> http://www.ietf.org/about/note-well.html
>

--

Reply all
Reply to author
Forward
0 new messages