In my project I need not only read from environ['wsgi.input'] but also
write in the opposite direction too. After some hours of goooogling I
supposed there is no any way to obtain output file object. So, I've
added just one line at httpserver.py:210
,'wsgi.output': self.wfile
and this did the trick.
Is there a better solution? Why there is 'wsgi.input' and no
'wsgi.output' and do not consider adding 'wsgi.output' to environ as
'wsgi.input' is?
On Thu, Jul 29, 2010 at 6:06 PM, eigenein <eigen...@gmail.com> wrote: > In my project I need not only read from environ['wsgi.input'] but also > write in the opposite direction too. After some hours of goooogling I > supposed there is no any way to obtain output file object. So, I've > added just one line at httpserver.py:210
> ,'wsgi.output': self.wfile
> and this did the trick.
> Is there a better solution? Why there is 'wsgi.input' and no > 'wsgi.output' and do not consider adding 'wsgi.output' to environ as > 'wsgi.input' is?
Well, that's just not how it works. Generally if you really need to push stuff out you should use the start_response writer, like:
The problem is that I need to pass some output file-like object to a
third-party code without writing any stuff myself (I'm embedding
mod_pywebsockets extension into project, so my code should provide
input and output file-like objects to the extension and give a control
to the extension code). In this case wfile fully satisfies me...
On Jul 30, 5:41 am, Ian Bicking <i...@colorstudy.com> wrote:
> On Thu, Jul 29, 2010 at 6:06 PM, eigenein <eigen...@gmail.com> wrote:
> > In my project I need not only read from environ['wsgi.input'] but also
> > write in the opposite direction too. After some hours of goooogling I
> > supposed there is no any way to obtain output file object. So, I've
> > added just one line at httpserver.py:210
> > ,'wsgi.output': self.wfile
> > and this did the trick.
> > Is there a better solution? Why there is 'wsgi.input' and no
> > 'wsgi.output' and do not consider adding 'wsgi.output' to environ as
> > 'wsgi.input' is?
> Well, that's just not how it works. Generally if you really need to push
> stuff out you should use the start_response writer, like:
On Fri, Jul 30, 2010 at 6:50 AM, eigenein <eigen...@gmail.com> wrote: > The problem is that I need to pass some output file-like object to a > third-party code without writing any stuff myself (I'm embedding > mod_pywebsockets extension into project, so my code should provide > input and output file-like objects to the extension and give a control > to the extension code). In this case wfile fully satisfies me...
You'll be fully out of the WSGI domain then, e.g., middleware won't work, or testing frameworks.
Does the code want to write the entire HTTP response (headers and all)?
Yes, the code want to write the entire response, and also the code
will not close connection immediately but will left connection opened
for a long time to be able to send data periodically (accordingly to
WebSockets protocol over an opened connection).
On Jul 30, 6:23 pm, Ian Bicking <i...@colorstudy.com> wrote:
> On Fri, Jul 30, 2010 at 6:50 AM, eigenein <eigen...@gmail.com> wrote:
> > The problem is that I need to pass some output file-like object to a
> > third-party code without writing any stuff myself (I'm embedding
> > mod_pywebsockets extension into project, so my code should provide
> > input and output file-like objects to the extension and give a control
> > to the extension code). In this case wfile fully satisfies me...
> You'll be fully out of the WSGI domain then, e.g., middleware won't work, or
> testing frameworks.
> Does the code want to write the entire HTTP response (headers and all)?
On Fri, Jul 30, 2010 at 10:35 AM, eigenein <eigen...@gmail.com> wrote: > Yes, the code want to write the entire response, and also the code > will not close connection immediately but will left connection opened > for a long time to be able to send data periodically (accordingly to > WebSockets protocol over an opened connection).
> On Jul 30, 6:23 pm, Ian Bicking <i...@colorstudy.com> wrote: > > On Fri, Jul 30, 2010 at 6:50 AM, eigenein <eigen...@gmail.com> wrote: > > > The problem is that I need to pass some output file-like object to a > > > third-party code without writing any stuff myself (I'm embedding > > > mod_pywebsockets extension into project, so my code should provide > > > input and output file-like objects to the extension and give a control > > > to the extension code). In this case wfile fully satisfies me...
> > You'll be fully out of the WSGI domain then, e.g., middleware won't work, > or > > testing frameworks.
> > Does the code want to write the entire HTTP response (headers and all)?
> -- > You received this message because you are subscribed to the Google Groups > "Paste Users" group. > To post to this group, send email to paste-users@googlegroups.com. > To unsubscribe from this group, send email to > paste-users+unsubscribe@googlegroups.com<paste-users%2Bunsubscribe@googlegr oups.com> > . > For more options, visit this group at > http://groups.google.com/group/paste-users?hl=en.
I recollect when I had a quick look at WebSockets it wasn't able to be
implemented on top of a standards conforming WSGI server. Even when
Google tried to do an implementation for Apache for Python using
mod_python they created an abomination because in the handler they
reached down into the connection layer to bypass the standard Apache
body processing.
To do WebSockets properly seems to require direct access to the actual
web server connection layer and cannot be done using higher level
abstractions that provide a more restrictive interface such as WSGI or
even mod_python.
Graham
On Jul 31, 1:41 am, Ian Bicking <i...@colorstudy.com> wrote:
> That's going to interact rather poorly with WSGI, I'd suggest running that
> server on an entirely separate port.
> On Fri, Jul 30, 2010 at 10:35 AM, eigenein <eigen...@gmail.com> wrote:
> > Yes, the code want to write the entire response, and also the code
> > will not close connection immediately but will left connection opened
> > for a long time to be able to send data periodically (accordingly to
> > WebSockets protocol over an opened connection).
> > On Jul 30, 6:23 pm, Ian Bicking <i...@colorstudy.com> wrote:
> > > On Fri, Jul 30, 2010 at 6:50 AM, eigenein <eigen...@gmail.com> wrote:
> > > > The problem is that I need to pass some output file-like object to a
> > > > third-party code without writing any stuff myself (I'm embedding
> > > > mod_pywebsockets extension into project, so my code should provide
> > > > input and output file-like objects to the extension and give a control
> > > > to the extension code). In this case wfile fully satisfies me...
> > > You'll be fully out of the WSGI domain then, e.g., middleware won't work,
> > or
> > > testing frameworks.
> > > Does the code want to write the entire HTTP response (headers and all)?
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Paste Users" group.
> > To post to this group, send email to paste-users@googlegroups.com.
> > To unsubscribe from this group, send email to
> > paste-users+unsubscribe@googlegroups.com<paste-users%2Bunsubscribe@googlegr oups.com>
> > .
> > For more options, visit this group at
> >http://groups.google.com/group/paste-users?hl=en.
<graham.dumple...@gmail.com> wrote: > I recollect when I had a quick look at WebSockets it wasn't able to be > implemented on top of a standards conforming WSGI server. Even when > Google tried to do an implementation for Apache for Python using > mod_python they created an abomination because in the handler they > reached down into the connection layer to bypass the standard Apache > body processing.
> To do WebSockets properly seems to require direct access to the actual > web server connection layer and cannot be done using higher level > abstractions that provide a more restrictive interface such as WSGI or > even mod_python.
> Graham
WebSockets definitely hates on the WSGI spec. For what its worth, I've not seen anything in the Rack/Plack/Jack specs that deal with this either.
In Gunicorn we added two undocumented features to allow for WebSocket support. First, we place an object into the environ that represents the underlying socket connection so applications can take control of the actual I/O. Then we have an undocumented singleton variable that applications can return so that the usual WSGI semantics are disabled. This is obviously broke as all get out and no one should expect such things to work in real life.
A secondary aspect of this was that I chatted with a Gunicorn user about trying to get django-websockets working. Looking through the code briefly I found that django-websockets was reaching deep and pulling out a copy of the raw socket form the request which obviously breaks lots of things. But I think this is a sign that the next WSGI is going to have to support connection upgrades.
In the recent work I've been stewing over for a WSGI that works on both 3.x and 2.x I wrote in a small bit based on my Gunicorn experience. I'm was aiming to support the old write callable for legacy support while removing the start_response callable that all other WSGI-ish specs did away with. That coupled with the WebSocket use case gave me the idea to have environ["wsgi.upgrade"] which is a callable that returns some object that acts like a socket. There's a wart on this, in that the only way I could think to let middleware and server authors know if a connection was still HTTP or not was to have an environ["wsgi.upgraded"] callable that returned a boolean. Not optimal. There's also the question on whether the spec should allow a connection to be downgraded after an upgrade (ie, resume HTTP traffic). I've done zero research on whether this is expected behavior, so it may be a non-issue.
Anyway, I've got some code and docs for this WSGI stuff I was working on. I've spent the last few evenings staring at the WebOb sources trying to figure out how in the world to crack into patching it to work with what I've drafted out. I've given myself until Sunday before I just push this stuff out to see what people think. I was hoping to have something like webob support first to try and minimize bike shedding, but webob turned out to be much bigger than I thought it was.
> On Jul 31, 1:41 am, Ian Bicking <i...@colorstudy.com> wrote: >> That's going to interact rather poorly with WSGI, I'd suggest running that >> server on an entirely separate port.
>> On Fri, Jul 30, 2010 at 10:35 AM, eigenein <eigen...@gmail.com> wrote: >> > Yes, the code want to write the entire response, and also the code >> > will not close connection immediately but will left connection opened >> > for a long time to be able to send data periodically (accordingly to >> > WebSockets protocol over an opened connection).
>> > On Jul 30, 6:23 pm, Ian Bicking <i...@colorstudy.com> wrote: >> > > On Fri, Jul 30, 2010 at 6:50 AM, eigenein <eigen...@gmail.com> wrote: >> > > > The problem is that I need to pass some output file-like object to a >> > > > third-party code without writing any stuff myself (I'm embedding >> > > > mod_pywebsockets extension into project, so my code should provide >> > > > input and output file-like objects to the extension and give a control >> > > > to the extension code). In this case wfile fully satisfies me...
>> > > You'll be fully out of the WSGI domain then, e.g., middleware won't work, >> > or >> > > testing frameworks.
>> > > Does the code want to write the entire HTTP response (headers and all)?
>> > -- >> > You received this message because you are subscribed to the Google Groups >> > "Paste Users" group. >> > To post to this group, send email to paste-users@googlegroups.com. >> > To unsubscribe from this group, send email to >> > paste-users+unsubscribe@googlegroups.com<paste-users%2Bunsubscribe@googlegr oups.com> >> > . >> > For more options, visit this group at >> >http://groups.google.com/group/paste-users?hl=en.
> -- > You received this message because you are subscribed to the Google Groups "Paste Users" group. > To post to this group, send email to paste-users@googlegroups.com. > To unsubscribe from this group, send email to paste-users+unsubscribe@googlegroups.com. > For more options, visit this group at http://groups.google.com/group/paste-users?hl=en.
> On Fri, Jul 30, 2010 at 9:47 PM, Graham Dumpleton
> <graham.dumple...@gmail.com> wrote:
> > I recollect when I had a quick look at WebSockets it wasn't able to be
> > implemented on top of a standards conforming WSGI server. Even when
> > Google tried to do an implementation for Apache for Python using
> > mod_python they created an abomination because in the handler they
> > reached down into the connection layer to bypass the standard Apache
> > body processing.
> > To do WebSockets properly seems to require direct access to the actual
> > web server connection layer and cannot be done using higher level
> > abstractions that provide a more restrictive interface such as WSGI or
> > even mod_python.
> > Graham
> WebSockets definitely hates on the WSGI spec. For what its worth, I've
> not seen anything in the Rack/Plack/Jack specs that deal with this
> either.
> In Gunicorn we added two undocumented features to allow for WebSocket
> support. First, we place an object into the environ that represents
> the underlying socket connection so applications can take control of
> the actual I/O. Then we have an undocumented singleton variable that
> applications can return so that the usual WSGI semantics are disabled.
> This is obviously broke as all get out and no one should expect such
> things to work in real life.
> A secondary aspect of this was that I chatted with a Gunicorn user
> about trying to get django-websockets working. Looking through the
> code briefly I found that django-websockets was reaching deep and
> pulling out a copy of the raw socket form the request which obviously
> breaks lots of things. But I think this is a sign that the next WSGI
> is going to have to support connection upgrades.
Not at all. When something can't work on the majority of web hosting
mechanisms that support a WSGI interface, why would you shove it into
the WSGI specification itself.
You have to realise that hosting mechanisms such as mod_python,
mod_wsgi, mod_uwsgi, mod_fastcgi, mod_fcgid, mod_scgi, mod_ajp,
mod_cgi etc, aren't going to be able to support this concept as they
are based on an assumption that standard HTTP is used. Some of them
are even more constrained than that and don't even permit things that
technically should be possible with HTTP.
So, some things should simply never be a part of the WSGI
specification as they destroy the promise of portability that WSGI has
brought.
There has been a similar argument about adding asynchronous features
to WSGI. They want special hooks which they felt would help them do
what they needed. Problem was that outside of that one little feature,
they still had heaps of code which was dependent upon the specific
asynchronous framework being used, eg., Twisted, Tornado etc. If that
is going to be the case and you aren't going to have a system which
allows one to write the code to the interface once and have it work on
different frameworks, what is the point.
That all said, I am interested to learn more about WebSockets. When I
tried to look at it last time I couldn't find a good introductory
description of it and was stuck trying to work it out from the
specification and what Google had done with mod_python.
My reasons for wanting to know is that although mod_wsgi and
mod_python may not be adequate vehicles for implementing it for
Apache, I have been working on/thinking about some new stuff in
relation to Apache that might make writing custom Apache modules in
Python viable, whereby one is working with raw Apache APIs and not a
restricted high level API.
So, know of any resources that explain WebSockets in simple terms with
simple examples?
> In the recent work I've been stewing over for a WSGI that works on
> both 3.x and 2.x I wrote in a small bit based on my Gunicorn
> experience. I'm was aiming to support the old write callable for
> legacy support while removing the start_response callable that all
> other WSGI-ish specs did away with. That coupled with the WebSocket
> use case gave me the idea to have environ["wsgi.upgrade"] which is a
> callable that returns some object that acts like a socket. There's a
> wart on this, in that the only way I could think to let middleware and
> server authors know if a connection was still HTTP or not was to have
> an environ["wsgi.upgraded"] callable that returned a boolean. Not
> optimal. There's also the question on whether the spec should allow a
> connection to be downgraded after an upgrade (ie, resume HTTP
> traffic). I've done zero research on whether this is expected
> behavior, so it may be a non-issue.
> Anyway, I've got some code and docs for this WSGI stuff I was working
> on. I've spent the last few evenings staring at the WebOb sources
> trying to figure out how in the world to crack into patching it to
> work with what I've drafted out. I've given myself until Sunday before
> I just push this stuff out to see what people think. I was hoping to
> have something like webob support first to try and minimize bike
> shedding, but webob turned out to be much bigger than I thought it
> was.
> Paul Davis
> > On Jul 31, 1:41 am, Ian Bicking <i...@colorstudy.com> wrote:
> >> That's going to interact rather poorly with WSGI, I'd suggest running that
> >> server on an entirely separate port.
> >> On Fri, Jul 30, 2010 at 10:35 AM, eigenein <eigen...@gmail.com> wrote:
> >> > Yes, the code want to write the entire response, and also the code
> >> > will not close connection immediately but will left connection opened
> >> > for a long time to be able to send data periodically (accordingly to
> >> > WebSockets protocol over an opened connection).
> >> > On Jul 30, 6:23 pm, Ian Bicking <i...@colorstudy.com> wrote:
> >> > > On Fri, Jul 30, 2010 at 6:50 AM, eigenein <eigen...@gmail.com> wrote:
> >> > > > The problem is that I need to pass some output file-like object to a
> >> > > > third-party code without writing any stuff myself (I'm embedding
> >> > > > mod_pywebsockets extension into project, so my code should provide
> >> > > > input and output file-like objects to the extension and give a control
> >> > > > to the extension code). In this case wfile fully satisfies me...
> >> > > You'll be fully out of the WSGI domain then, e.g., middleware won't work,
> >> > or
> >> > > testing frameworks.
> >> > > Does the code want to write the entire HTTP response (headers and all)?
> >> > --
> >> > You received this message because you are subscribed to the Google Groups
> >> > "Paste Users" group.
> >> > To post to this group, send email to paste-users@googlegroups.com.
> >> > To unsubscribe from this group, send email to
> >> > paste-users+unsubscribe@googlegroups.com<paste-users%2Bunsubscribe@googlegr oups.com>
> >> > .
> >> > For more options, visit this group at
> >> >http://groups.google.com/group/paste-users?hl=en.
> > --
> > You received this message because you are subscribed to the Google Groups "Paste Users" group.
> > To post to this group, send email to paste-users@googlegroups.com.
> > To unsubscribe from this group, send email to paste-users+unsubscribe@googlegroups.com.
> > For more options, visit this group athttp://groups.google.com/group/paste-users?hl=en.
It seems I've got WebSockets to work properly with the described
wsgi.output hack, and mod_pywebsocket module and a couple of adapter
classes. The point I'm started at was two brief articles in Russian...
I've failed in attempts to google any better descriptions.
On Jul 31, 8:02 am, Graham Dumpleton <graham.dumple...@gmail.com>
wrote:
> On Jul 31, 2:01 pm, Paul Davis <paul.joseph.da...@gmail.com> wrote:
> > On Fri, Jul 30, 2010 at 9:47 PM, Graham Dumpleton
> > <graham.dumple...@gmail.com> wrote:
> > > I recollect when I had a quick look at WebSockets it wasn't able to be
> > > implemented on top of a standards conforming WSGI server. Even when
> > > Google tried to do an implementation for Apache for Python using
> > > mod_python they created an abomination because in the handler they
> > > reached down into the connection layer to bypass the standard Apache
> > > body processing.
> > > To do WebSockets properly seems to require direct access to the actual
> > > web server connection layer and cannot be done using higher level
> > > abstractions that provide a more restrictive interface such as WSGI or
> > > even mod_python.
> > > Graham
> > WebSockets definitely hates on the WSGI spec. For what its worth, I've
> > not seen anything in the Rack/Plack/Jack specs that deal with this
> > either.
> > In Gunicorn we added two undocumented features to allow for WebSocket
> > support. First, we place an object into the environ that represents
> > the underlying socket connection so applications can take control of
> > the actual I/O. Then we have an undocumented singleton variable that
> > applications can return so that the usual WSGI semantics are disabled.
> > This is obviously broke as all get out and no one should expect such
> > things to work in real life.
> > A secondary aspect of this was that I chatted with a Gunicorn user
> > about trying to get django-websockets working. Looking through the
> > code briefly I found that django-websockets was reaching deep and
> > pulling out a copy of the raw socket form the request which obviously
> > breaks lots of things. But I think this is a sign that the next WSGI
> > is going to have to support connection upgrades.
> Not at all. When something can't work on the majority of web hosting
> mechanisms that support a WSGI interface, why would you shove it into
> the WSGI specification itself.
> You have to realise that hosting mechanisms such as mod_python,
> mod_wsgi, mod_uwsgi, mod_fastcgi, mod_fcgid, mod_scgi, mod_ajp,
> mod_cgi etc, aren't going to be able to support this concept as they
> are based on an assumption that standard HTTP is used. Some of them
> are even more constrained than that and don't even permit things that
> technically should be possible with HTTP.
> So, some things should simply never be a part of the WSGI
> specification as they destroy the promise of portability that WSGI has
> brought.
> There has been a similar argument about adding asynchronous features
> to WSGI. They want special hooks which they felt would help them do
> what they needed. Problem was that outside of that one little feature,
> they still had heaps of code which was dependent upon the specific
> asynchronous framework being used, eg., Twisted, Tornado etc. If that
> is going to be the case and you aren't going to have a system which
> allows one to write the code to the interface once and have it work on
> different frameworks, what is the point.
> That all said, I am interested to learn more about WebSockets. When I
> tried to look at it last time I couldn't find a good introductory
> description of it and was stuck trying to work it out from the
> specification and what Google had done with mod_python.
> My reasons for wanting to know is that although mod_wsgi and
> mod_python may not be adequate vehicles for implementing it for
> Apache, I have been working on/thinking about some new stuff in
> relation to Apache that might make writing custom Apache modules in
> Python viable, whereby one is working with raw Apache APIs and not a
> restricted high level API.
> So, know of any resources that explain WebSockets in simple terms with
> simple examples?
> Graham
> > In the recent work I've been stewing over for a WSGI that works on
> > both 3.x and 2.x I wrote in a small bit based on my Gunicorn
> > experience. I'm was aiming to support the old write callable for
> > legacy support while removing the start_response callable that all
> > other WSGI-ish specs did away with. That coupled with the WebSocket
> > use case gave me the idea to have environ["wsgi.upgrade"] which is a
> > callable that returns some object that acts like a socket. There's a
> > wart on this, in that the only way I could think to let middleware and
> > server authors know if a connection was still HTTP or not was to have
> > an environ["wsgi.upgraded"] callable that returned a boolean. Not
> > optimal. There's also the question on whether the spec should allow a
> > connection to be downgraded after an upgrade (ie, resume HTTP
> > traffic). I've done zero research on whether this is expected
> > behavior, so it may be a non-issue.
> > Anyway, I've got some code and docs for this WSGI stuff I was working
> > on. I've spent the last few evenings staring at the WebOb sources
> > trying to figure out how in the world to crack into patching it to
> > work with what I've drafted out. I've given myself until Sunday before
> > I just push this stuff out to see what people think. I was hoping to
> > have something like webob support first to try and minimize bike
> > shedding, but webob turned out to be much bigger than I thought it
> > was.
> > Paul Davis
> > > On Jul 31, 1:41 am, Ian Bicking <i...@colorstudy.com> wrote:
> > >> That's going to interact rather poorly with WSGI, I'd suggest running that
> > >> server on an entirely separate port.
> > >> On Fri, Jul 30, 2010 at 10:35 AM, eigenein <eigen...@gmail.com> wrote:
> > >> > Yes, the code want to write the entire response, and also the code
> > >> > will not close connection immediately but will left connection opened
> > >> > for a long time to be able to send data periodically (accordingly to
> > >> > WebSockets protocol over an opened connection).
> > >> > On Jul 30, 6:23 pm, Ian Bicking <i...@colorstudy.com> wrote:
> > >> > > On Fri, Jul 30, 2010 at 6:50 AM, eigenein <eigen...@gmail.com> wrote:
> > >> > > > The problem is that I need to pass some output file-like object to a
> > >> > > > third-party code without writing any stuff myself (I'm embedding
> > >> > > > mod_pywebsockets extension into project, so my code should provide
> > >> > > > input and output file-like objects to the extension and give a control
> > >> > > > to the extension code). In this case wfile fully satisfies me...
> > >> > > You'll be fully out of the WSGI domain then, e.g., middleware won't work,
> > >> > or
> > >> > > testing frameworks.
> > >> > > Does the code want to write the entire HTTP response (headers and all)?
> > >> > --
> > >> > You received this message because you are subscribed to the Google Groups
> > >> > "Paste Users" group.
> > >> > To post to this group, send email to paste-users@googlegroups.com.
> > >> > To unsubscribe from this group, send email to
> > >> > paste-users+unsubscribe@googlegroups.com<paste-users%2Bunsubscribe@googlegr oups.com>
> > >> > .
> > >> > For more options, visit this group at
> > >> >http://groups.google.com/group/paste-users?hl=en.
> > > --
> > > You received this message because you are subscribed to the Google Groups "Paste Users" group.
> > > To post to this group, send email to paste-users@googlegroups.com.
> > > To unsubscribe from this group, send email to paste-users+unsubscribe@googlegroups.com.
> > > For more options, visit this group athttp://groups.google.com/group/paste-users?hl=en.
---------- Forwarded message ---------- From: Paul Davis <paul.joseph.da...@gmail.com> Date: Sat, Jul 31, 2010 at 12:05 PM Subject: Re: [Paste] Re: 'wsgi.output' in environ? To: Graham Dumpleton <graham.dumple...@gmail.com>
On Sat, Jul 31, 2010 at 1:02 AM, Graham Dumpleton <graham.dumple...@gmail.com> wrote:
> On Jul 31, 2:01 pm, Paul Davis <paul.joseph.da...@gmail.com> wrote: >> On Fri, Jul 30, 2010 at 9:47 PM, Graham Dumpleton
>> <graham.dumple...@gmail.com> wrote: >> > I recollect when I had a quick look at WebSockets it wasn't able to be >> > implemented on top of a standards conforming WSGI server. Even when >> > Google tried to do an implementation for Apache for Python using >> > mod_python they created an abomination because in the handler they >> > reached down into the connection layer to bypass the standard Apache >> > body processing.
>> > To do WebSockets properly seems to require direct access to the actual >> > web server connection layer and cannot be done using higher level >> > abstractions that provide a more restrictive interface such as WSGI or >> > even mod_python.
>> > Graham
>> WebSockets definitely hates on the WSGI spec. For what its worth, I've >> not seen anything in the Rack/Plack/Jack specs that deal with this >> either.
>> In Gunicorn we added two undocumented features to allow for WebSocket >> support. First, we place an object into the environ that represents >> the underlying socket connection so applications can take control of >> the actual I/O. Then we have an undocumented singleton variable that >> applications can return so that the usual WSGI semantics are disabled. >> This is obviously broke as all get out and no one should expect such >> things to work in real life.
>> A secondary aspect of this was that I chatted with a Gunicorn user >> about trying to get django-websockets working. Looking through the >> code briefly I found that django-websockets was reaching deep and >> pulling out a copy of the raw socket form the request which obviously >> breaks lots of things. But I think this is a sign that the next WSGI >> is going to have to support connection upgrades.
> Not at all. When something can't work on the majority of web hosting > mechanisms that support a WSGI interface, why would you shove it into > the WSGI specification itself.
> You have to realise that hosting mechanisms such as mod_python, > mod_wsgi, mod_uwsgi, mod_fastcgi, mod_fcgid, mod_scgi, mod_ajp, > mod_cgi etc, aren't going to be able to support this concept as they > are based on an assumption that standard HTTP is used. Some of them > are even more constrained than that and don't even permit things that > technically should be possible with HTTP.
One of the first sentences of my notes was something along the lines of "Its 2010 and its about time that WSGI focus on being HTTP compliant and leave the constraints of CGI behind."
I hadn't at all researched the various embedded implementations to figure out if it was viable yet. Though I'm a tad confused on how these embeddings support the write callable without being able to access the read side of the socket.
> So, some things should simply never be a part of the WSGI > specification as they destroy the promise of portability that WSGI has > brought.
I agree that we should minimize the surface area to make it as easy to implement as possible. I'm not tied to any of this at all. It was just a fun little by product of how I rearranged the write callable action.
> There has been a similar argument about adding asynchronous features > to WSGI. They want special hooks which they felt would help them do > what they needed. Problem was that outside of that one little feature, > they still had heaps of code which was dependent upon the specific > asynchronous framework being used, eg., Twisted, Tornado etc. If that > is going to be the case and you aren't going to have a system which > allows one to write the code to the interface once and have it work on > different frameworks, what is the point.
One of the ideas I've had simmering for a bit was to figure out how to write a series of wsgi apps that exercise the entire WSGI specification. That way you could just run it behind your favorite server to check the validity of its implementation. Basically, wsgiref.validator, but in reverse.
> That all said, I am interested to learn more about WebSockets. When I > tried to look at it last time I couldn't find a good introductory > description of it and was stuck trying to work it out from the > specification and what Google had done with mod_python.
> My reasons for wanting to know is that although mod_wsgi and > mod_python may not be adequate vehicles for implementing it for > Apache, I have been working on/thinking about some new stuff in > relation to Apache that might make writing custom Apache modules in > Python viable, whereby one is working with raw Apache APIs and not a > restricted high level API.
> So, know of any resources that explain WebSockets in simple terms with > simple examples?
This was a fun little experiment at one point, but now that the traffic has died down its lost its effectiveness. Right when it hit Hacker News there were probably 40-50 cursors floating around showing everyone's mouse movements.
Benoit Chesneau (the other Gunicorn developer) made a quick example and short movie of websockets using the WSGI hacks.
Its not as amusing as the communal mouse cursors, but it should do the trick in showing the client code for such things. The spec is a bit dense, but the basic premise is to have a bidirectional channel open to the browser to avoid the current state of the art that is long poll et al.
Paul Davis
>> In the recent work I've been stewing over for a WSGI that works on >> both 3.x and 2.x I wrote in a small bit based on my Gunicorn >> experience. I'm was aiming to support the old write callable for >> legacy support while removing the start_response callable that all >> other WSGI-ish specs did away with. That coupled with the WebSocket >> use case gave me the idea to have environ["wsgi.upgrade"] which is a >> callable that returns some object that acts like a socket. There's a >> wart on this, in that the only way I could think to let middleware and >> server authors know if a connection was still HTTP or not was to have >> an environ["wsgi.upgraded"] callable that returned a boolean. Not >> optimal. There's also the question on whether the spec should allow a >> connection to be downgraded after an upgrade (ie, resume HTTP >> traffic). I've done zero research on whether this is expected >> behavior, so it may be a non-issue.
>> Anyway, I've got some code and docs for this WSGI stuff I was working >> on. I've spent the last few evenings staring at the WebOb sources >> trying to figure out how in the world to crack into patching it to >> work with what I've drafted out. I've given myself until Sunday before >> I just push this stuff out to see what people think. I was hoping to >> have something like webob support first to try and minimize bike >> shedding, but webob turned out to be much bigger than I thought it >> was.
>> Paul Davis
>> > On Jul 31, 1:41 am, Ian Bicking <i...@colorstudy.com> wrote: >> >> That's going to interact rather poorly with WSGI, I'd suggest running that >> >> server on an entirely separate port.
>> >> On Fri, Jul 30, 2010 at 10:35 AM, eigenein <eigen...@gmail.com> wrote: >> >> > Yes, the code want to write the entire response, and also the code >> >> > will not close connection immediately but will left connection opened >> >> > for a long time to be able to send data periodically (accordingly to >> >> > WebSockets protocol over an opened connection).
>> >> > On Jul 30, 6:23 pm, Ian Bicking <i...@colorstudy.com> wrote: >> >> > > On Fri, Jul 30, 2010 at 6:50 AM, eigenein <eigen...@gmail.com> wrote: >> >> > > > The problem is that I need to pass some output file-like object to a >> >> > > > third-party code without writing any stuff myself (I'm embedding >> >> > > > mod_pywebsockets extension into project, so my code should provide >> >> > > > input and output file-like objects to the extension and give a control >> >> > > > to the extension code). In this case wfile fully satisfies me...
>> >> > > You'll be fully out of the WSGI domain then, e.g., middleware won't work, >> >> > or >> >> > > testing frameworks.
>> >> > > Does the code want to write the entire HTTP response (headers and all)?
>> >> > -- >> >> > You received this message because you are subscribed to the Google Groups >> >> > "Paste Users" group. >> >> > To post to this group, send email to paste-users@googlegroups.com. >> >> > To unsubscribe from this group, send email to >> >> > paste-users+unsubscribe@googlegroups.com<paste-users%2Bunsubscribe@googlegr oups.com> >> >> > . >> >> > For more options, visit this group at >> >> >http://groups.google.com/group/paste-users?hl=en.
>> > -- >> > You received this message because you are subscribed to the Google Groups "Paste Users" group. >> > To post to this group, send email to paste-users@googlegroups.com. >> > To unsubscribe from this group, send email to paste-users+unsubscribe@googlegroups.com. >> > For more options, visit this group athttp://groups.google.com/group/paste-users?hl=en.
> -- > You received this message because you are subscribed to the Google Groups "Paste Users" group. > To post to this group, send email to paste-users@googlegroups.com. > To unsubscribe from this group, send email to paste-users+unsubscribe@googlegroups.com. > For more options, visit
> WebSockets definitely hates on the WSGI spec. For what its worth, I've > not seen anything in the Rack/Plack/Jack specs that deal with this > either.
> In Gunicorn we added two undocumented features to allow for WebSocket > support. First, we place an object into the environ that represents > the underlying socket connection so applications can take control of > the actual I/O. Then we have an undocumented singleton variable that > applications can return so that the usual WSGI semantics are disabled. > This is obviously broke as all get out and no one should expect such > things to work in real life.
> A secondary aspect of this was that I chatted with a Gunicorn user > about trying to get django-websockets working. Looking through the > code briefly I found that django-websockets was reaching deep and > pulling out a copy of the raw socket form the request which obviously > breaks lots of things. But I think this is a sign that the next WSGI > is going to have to support connection upgrades.
Huh, I'm surprised. My basic understanding of WebSockets is that it works kind of like:
You get a request with normal HTTP headers Plus a bunch of special WebSocket negotiation stuff that everyone is complaining about (special headers, and special response headers) Then the socket has delimited messages (length+bytes I guess)? They go in both directions.
It doesn't seem like you should need (or even have) direct access to the sockets, because it's not streaming and there's no parsing of the socket itself (except for parsing the WebSocket messages, which the server can do and is spec'd out).
I can imagine an incoming WebSocket message (first message or later messages) could be represented as a WSGI request, just adding a connection ID and cloning the HTTP headers each time. Responses could be... well, that's tricky, as there's no request/response pairing. You could put in something like environ['websockets.send_message']('a message to send'). But then the actual WSGI response becomes meaningless. So it's not a great fit. But at least it is always initiated by the client.
Implementing this without a WebSockets-aware server seems like a bad idea. Anything not at the server level is going to be a nasty hack.
On Sat, Jul 31, 2010 at 1:08 PM, Ian Bicking <i...@colorstudy.com> wrote: > On Fri, Jul 30, 2010 at 11:01 PM, Paul Davis <paul.joseph.da...@gmail.com> > wrote:
>> WebSockets definitely hates on the WSGI spec. For what its worth, I've >> not seen anything in the Rack/Plack/Jack specs that deal with this >> either.
>> In Gunicorn we added two undocumented features to allow for WebSocket >> support. First, we place an object into the environ that represents >> the underlying socket connection so applications can take control of >> the actual I/O. Then we have an undocumented singleton variable that >> applications can return so that the usual WSGI semantics are disabled. >> This is obviously broke as all get out and no one should expect such >> things to work in real life.
>> A secondary aspect of this was that I chatted with a Gunicorn user >> about trying to get django-websockets working. Looking through the >> code briefly I found that django-websockets was reaching deep and >> pulling out a copy of the raw socket form the request which obviously >> breaks lots of things. But I think this is a sign that the next WSGI >> is going to have to support connection upgrades.
> Huh, I'm surprised. My basic understanding of WebSockets is that it works > kind of like:
> You get a request with normal HTTP headers > Plus a bunch of special WebSocket negotiation stuff that everyone is > complaining about (special headers, and special response headers) > Then the socket has delimited messages (length+bytes I guess)? They go in > both directions.
> It doesn't seem like you should need (or even have) direct access to the > sockets, because it's not streaming and there's no parsing of the socket > itself (except for parsing the WebSocket messages, which the server can do > and is spec'd out).
I don't think streaming is the right word, but the effect is the same. The websocket connection is a full duplex asynchronous message passing channel. Ie, the client and server can both send messages whenever they want. All the websocket spec is really doing is specifying the HTTP upgrade (the special headers and response), the line protocol after upgrade (the length delimited messages) and the JavaScript API in the browser.
After that its basically a full tcp connection which means that anything between the client code and webapp code needs to make sure that the channel is unbuffered if applications are going to work correctly.
> I can imagine an incoming WebSocket message (first message or later > messages) could be represented as a WSGI request, just adding a connection > ID and cloning the HTTP headers each time. Responses could be... well, > that's tricky, as there's no request/response pairing. You could put in > something like environ['websockets.send_message']('a message to send'). But > then the actual WSGI response becomes meaningless. So it's not a great > fit. But at least it is always initiated by the client.
That's a pretty interesting idea. Though it breaks the server initiated message part of websockets. My orginal wsgi.upgrade was basically to say "this connection is no longer HTTP and the app code has taken responsibility for handling the connection". As Graham pointed out, that could be shot down by the number of implementations that wouldn't be able to handle it.
> Implementing this without a WebSockets-aware server seems like a bad idea. > Anything not at the server level is going to be a nasty hack.
I definitely agree. For awhile I toyed around with the idea of making the upgrade thing an optional but conventional way to do websockets. But then I had similar fears to what Graham voiced about that splitting the wsgi server implementations up fairly drastically.
In the end I'm more worried about properly supporting HTTP. WebSockets would be nice, but they're not exactly HTTP.
> On Sat, Jul 31, 2010 at 1:08 PM, Ian Bicking <i...@colorstudy.com> wrote:
> > On Fri, Jul 30, 2010 at 11:01 PM, Paul Davis <paul.joseph.da...@gmail.com>
> > wrote:
> >> WebSockets definitely hates on the WSGI spec. For what its worth, I've
> >> not seen anything in the Rack/Plack/Jack specs that deal with this
> >> either.
> >> In Gunicorn we added two undocumented features to allow for WebSocket
> >> support. First, we place an object into the environ that represents
> >> the underlying socket connection so applications can take control of
> >> the actual I/O. Then we have an undocumented singleton variable that
> >> applications can return so that the usual WSGI semantics are disabled.
> >> This is obviously broke as all get out and no one should expect such
> >> things to work in real life.
> >> A secondary aspect of this was that I chatted with a Gunicorn user
> >> about trying to get django-websockets working. Looking through the
> >> code briefly I found that django-websockets was reaching deep and
> >> pulling out a copy of the raw socket form the request which obviously
> >> breaks lots of things. But I think this is a sign that the next WSGI
> >> is going to have to support connection upgrades.
> > Huh, I'm surprised. My basic understanding of WebSockets is that it works
> > kind of like:
> > You get a request with normal HTTP headers
> > Plus a bunch of special WebSocket negotiation stuff that everyone is
> > complaining about (special headers, and special response headers)
> > Then the socket has delimited messages (length+bytes I guess)? They go in
> > both directions.
> > It doesn't seem like you should need (or even have) direct access to the
> > sockets, because it's not streaming and there's no parsing of the socket
> > itself (except for parsing the WebSocket messages, which the server can do
> > and is spec'd out).
> I don't think streaming is the right word, but the effect is the same.
> The websocket connection is a full duplex asynchronous message passing
> channel. Ie, the client and server can both send messages whenever
> they want.
In the past I have called it both ways streaming and technically the
HTTP protocol can support it without special tricks of upgrading the
protocol. All it means is that you need to use chunked request content
as well as chunked response content. Plus, implement your own message
boundary mechanism on top, which is effectively what WebSockets is
doing anyway.
The problem is that chunked request content is not supported by CGI,
FASTCGI, SCGI, AJP and certain other custom protocols, nor does WSGI
as a higher level API that can ride on these and other hosting
mechanisms.
So, even though I have described in the past how WSGI could support
chunked requests, and thus both way streaming, it doesn't help that
most WSGI implementations wouldn't be able to support it.
The main problem there is that implementations of protocols like CGI,
FASTCGI etc, assume that they can read all request content and send it
onto the application before they need to start reading the response
content. In other words, they cant be sending request content through
to the application and reading and returning response content at the
same time.
> All the websocket spec is really doing is specifying the
> HTTP upgrade (the special headers and response), the line protocol
> after upgrade (the length delimited messages) and the JavaScript API
> in the browser.
> After that its basically a full tcp connection which means that
> anything between the client code and webapp code needs to make sure
> that the channel is unbuffered if applications are going to work
> correctly.
> > I can imagine an incoming WebSocket message (first message or later
> > messages) could be represented as a WSGI request, just adding a connection
> > ID and cloning the HTTP headers each time. Responses could be... well,
> > that's tricky, as there's no request/response pairing. You could put in
> > something like environ['websockets.send_message']('a message to send'). But
> > then the actual WSGI response becomes meaningless. So it's not a great
> > fit. But at least it is always initiated by the client.
> That's a pretty interesting idea. Though it breaks the server
> initiated message part of websockets. My orginal wsgi.upgrade was
> basically to say "this connection is no longer HTTP and the app code
> has taken responsibility for handling the connection". As Graham
> pointed out, that could be shot down by the number of implementations
> that wouldn't be able to handle it.
> > Implementing this without a WebSockets-aware server seems like a bad idea.
> > Anything not at the server level is going to be a nasty hack.
> I definitely agree. For awhile I toyed around with the idea of making
> the upgrade thing an optional but conventional way to do websockets.
> But then I had similar fears to what Graham voiced about that
> splitting the wsgi server implementations up fairly drastically.
> In the end I'm more worried about properly supporting HTTP. WebSockets
> would be nice, but they're not exactly HTTP.
On Sat, Jul 31, 2010 at 3:17 PM, Paul Davis <paul.joseph.da...@gmail.com>wrote:
> > I can imagine an incoming WebSocket message (first message or later > > messages) could be represented as a WSGI request, just adding a > connection > > ID and cloning the HTTP headers each time. Responses could be... well, > > that's tricky, as there's no request/response pairing. You could put in > > something like environ['websockets.send_message']('a message to send'). > But > > then the actual WSGI response becomes meaningless. So it's not a great > > fit. But at least it is always initiated by the client.
> That's a pretty interesting idea. Though it breaks the server > initiated message part of websockets. My orginal wsgi.upgrade was > basically to say "this connection is no longer HTTP and the app code > has taken responsibility for handling the connection".
The server can only initiate a message, not a connection, so the ability to save websockets.send_message (and use whatever queuing or task process you had to send messages) would help. In most applications you probably won't even need another set of processes, as messages would be sent because of something else, e.g., in a chat application a message will be sent out when a message comes in (but across different sockets). It does seem like an asynchronous API would be helpful in this case, as there's no reason to block the one client in order to send out a message to other clients... though I suppose all websocket requests will in effect be asynchronous because the WSGI response wouldn't mean anything. I guess you also need server-initiated close, since the request wouldn't close on its own (though maybe require a 204 No Content response to indicate the application understands web sockets, and otherwise simply abort).
