Cheers,
Brian
> --
> Note Well: This discussion group is meant to become an IETF working group in
> the future. Thus, the posts to this discussion should comply with IETF
> contribution policy as explained here:
> http://www.ietf.org/about/note-well.html
>
--
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgra...@calpoly.edu and elli...@gmail.com
TCP, UDP, SCTP, PGM, WebSockets?
Martin
> What about adding WebSockets to the list of transports to check:
> TCP, UDP, SCTP, PGM, WebSockets?
Yes. SP over WebSockets is a killer mix, IMO. WebSockets will be the
key to accessing the cloud...
-Pieter
This might be slightly offtopic, but we're doing this as well. Do you
happen to have your adapter code published somewhere? It might be
interesting to compare notes (using pyzmq, no less).
Cheers,
Dirkjan
We are using tornado for this. The code is pretty new and hidden in a
development branch for now. But here is the main part of it:
https://github.com/ipython/ipython/blob/htmlnotebook/IPython/frontend/html/notebook/notebook.py
The main logic is in the ZMQStreamRouter:
https://github.com/ipython/ipython/blob/htmlnotebook/IPython/frontend/html/notebook/notebook.py#L55
that connects multiple WebSocket clients to a single zeromq socket on
the backend. We tried using a single zeromq socket per WebSocket, but
that simply created too many sockets on the server (but the design was
simpler). For our particular usage case, having one zeromq socket for
many WebSocket clients works well and also allows us to start/stop the
zeromq socket while leaving the WebSockets connected.
Cheers,
Brian
> Cheers,
>
> Dirkjan
> that connects multiple WebSocket clients to a single zeromq socket on
> the backend. We tried using a single zeromq socket per WebSocket, but
> that simply created too many sockets on the server (but the design was
> simpler). For our particular usage case, having one zeromq socket for
> many WebSocket clients works well and also allows us to start/stop the
> zeromq socket while leaving the WebSockets connected.
Any experience with the protocol? Did you have to pass any additional
data on top of WebSockets or did the WebSocket protocol do as is?
Martin
No, I don't have any experience with the underlying protocol. Our
usage consists of simply forwarding JSON messages back and forth
betwee the WebSocket and the zeromq socket. The only think we have
really had to add was a way to track different WebSocket clients, but
for that we can add a field to our JSON data. These client IDs are
really application level logic though.
The main issue we run into with WebSockets is that a WebSocket allows
*any* send/recv pattern, but zeromq sockets don't. This means that a
WebSocket can cause a zeromq socket to raise an exception in Python be
illegal send/recv pattern (for example send on a SUB socket).
Cheers,
Brian
> Martin
The main issue we run into with WebSockets is that a WebSocket allows
*any* send/recv pattern, but zeromq sockets don't. This means that a
WebSocket can cause a zeromq socket to raise an exception in Python be
illegal send/recv pattern (for example send on a SUB socket)
Any experience with the protocol?
Did you have to pass any additional data on top of WebSockets or did the WebSocket protocol do as is?
Thanks for valuable input!
> We use zerogw (http://github.com/tailhook/zerogw) which is a HTTP and
> WebSocket server written in C, which talks zeromq with backend.
Nice project. I haven't noticed it till now :(
> This is probably true for all web
> applications, connections are very expensive in browser, so we try to
> multiplex several streams with different patterns on top of websocket
> connection instead of creating multiple sockets.
Yes. Overhead associated with connection negotiation is often a problem.
And not only in web environment: think of SCTP streams.
From our point of view, I would say, the problem can be restated as
"tunnelling multiple SP feeds over a single underlying connection". I
have some ideas about that, but I would like to postpone the discussion
until at least the framing and labeling is more or less agreed on.
> If you want to discuss usage of websocket protocol as one of the
> protocols of zeromq without respect to browsers, there are only few
> problems.
Please, distinguish between 0MQ as an particular messaging
implementation and SP as (attempt for) a protocol. If, say, your way of
sending messages over WebSocket is standardised, zerogw project would be
SP-compliant, but the code interacting with the websocket would still be
your code, not 0MQ.
> Websocket protocol has http handshake, which adds complexity
> for implementation,
Yes. But AFAIU that's the complexity we just have to accept if we want
to use WebSockets as an underlying transport layer.
> but adds extensibility. E.g. you can transmit
> identity as a header value like "X-Zeromq-Identity: 0xABCD", and use
> protocol "Sec-Websocket-Protocol: zeromq-pub".
I would like to postpone the connection initiation discussion a bit and
focus on framing for now.
> The downside is there are
> no message parts in the protocol. There is message fragmenting in the
> protocol, but intermediaries do not required to preserve them. Ok, seems
> the protocols we are speaking about in this ml don't have message parts
> either.
Multi-part messages are a fancy feature in 0MQ, but are unlikely to
appear even in Linux kernel implementation of SP. The standard way of
addressing the related use cases are gather/scatter arrays.
> Protocol also has mandatory masking of frames to literally
> confuse bad written intermediaries (http proxies) to solve their
> vulnerability problems. Masking has big impact on performance.
Ok. Good to know, but once again, it's something we can't change and
have to simply accept.
> Protocol
> also has control messages, among them: ping/pong and close. Each message
> has extension flags which allows to have some (yet not specified)
> extension fields on each message. Extesions must be negotiated at the
> connection time. Otherwise it's just protocol which allows passing
> messages in both ways without limits.
We'll need "labels" to be attached to SP messages later on (pub/sub
topics, req/rep reply-to's etc.) -- maybe WebSocket extensions could be
used for that.
Martin
Good to know about this project. Our backend uses a more application
specific set of zeromq sockets, so we have different socket types
(XREP/XREQ, PUB/SUB). But as we scale it up, we will probably end up
with something similar to what you have, which is in turn similar to
Mongrel2.
> --
> Note Well: This discussion group is meant to become an IETF working group in
> the future. Thus, the posts to this discussion should comply with IETF
> contribution policy as explained here:
> http://www.ietf.org/about/note-well.html
>
--