Request for Input: WSGI 2.0

[Cory Benfield]

All,

An effort has begun over in the PSF Web-SIG special interest group to come up with a new version of the WSGI specification. The goal is to develop a successor to WSGI that will be more useful in the development of modern, asynchronous web applications, with the goal of making available all of the improvements in both HTTP and Python that have come through in the years since PEP 333.

As Django is one of the major users of WSGI, any attempt to specify a new version without getting input from the Django community and development team would be extremely foolish! This is therefore a request for input: I'd like it very much if the Django development team could come up with a response to the email linked earlier, indicating what the Django team would like from a new version of WSGI and what you believe the priorities should be. For obvious reasons there may not be consensus around this issue amongst the Django developers, and that's fine: multiple submissions from Django would be entirely acceptable.

Trying to improve something as well-established as WSGI is going to be extremely difficult, and it'll only work if communities like Django's step up to help build something that we can all be proud of. I hope that you're as excited by that idea as I am!

A side note: I'm very aware of Andrew Godwin's work with channels, and it may be that the Django community believes that that model is the future of web programming in Python. If that's the case, then please let us know. I personally don't believe that it's the right direction for WSGI, but I may be wrong (I've been wrong before!).

If you have any questions, please don't hesitate to ask. I'm looking forward to working with you all!

Cory

[Marc Tamlyn]

Hi Cory,

[Disclaimer - I don't suppose to speak for the whole community here, this is a personal opinion of how I see things]

Django's wsgi handling code has remained largely unchanged for many years, so to answer the question of "what does Django itself need from new wsgi, ignoring channels", I'd suggest the answer is nothing - wsgi as is works well for me. As an application developer, I'd also mention that I've never had a need to dip into the wsgi objects at all, so I can't say I need a wsgi2 either. In short, the idea of wsgi2 doesn't excite me at all. Perhaps you could reiterate what the benefits would be to a common garden web developer?

However, when you bring channels in it changes the topic quite significantly. I would definitely like channels' communication with the outside world to be as based on a stable, external python platform as possible, if only for the volume of code we need to maintain in Django.

I hope that helps a bit..

Marc

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/aea5f21c-20c7-49ab-a1ca-2376ca0b92f2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Andrew Godwin]

Hi Cory,

I'll reply on the Web-SIG mailing list too, but as I've outlined elsewhere, I don't think ASGI would be a suitable replacement for WSGI in the current form; in particular, I suspect it will have a performance disadvantage, though I've not quantified yet.

That said, if Django Channels does become the primary method for communication with web clients, the only thing we would want out of WSGI 2 would be something that could easily plug into ASGI - that is, something that supports WebSockets, can handle simultaneous connections with many clients in the same thread via cooperative multitasking or similar, and allows raw access to sockets. Given those things, an ASGI HTTP/WebSocket protocol server could simply be a WSGI 2 application, allowing much better code reuse.

I still believe in the ASGI model for protocols other than HTTP/WS, and for background task handling, so I don't think I'd want to throw it away and adopt WSGI 2 directly if it came along, but Django would definitely keep a direct-WSGI mode for that as it always has.

Andrew

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAMwjO1HHJ35a3cXpUMmRGxKxUOXLrxutdLWuGOJJ72uAbW2ERg%40mail.gmail.com.

[Aymeric Augustin]

Hi Cory,

I’m not subscribed to web-sig but I read the discussion there. Feel free to forward my answer to the group if you think it’s useful.

I have roughly the same convictions as Graham Dumpleton. If you want to support HTTP/2 and WebSockets, don’t start with design decisions anchored in CGI. Figure out what a simple and flexible API for these new protocols would be, specify it, implement it, and make sure it degrades gracefully to HTTP/1. You may be able to channel most of the communication through a single generator, but it’s unclear to me that this will be the most convenient design.

If you want to improve WSGI, here’s a list of mistakes or shortcomings in PEP 3333 that you can take a stab at. There’s a general theme: for a specification that looks at the future, I believe that making modern PaaS-based deployments secure by default matters more than not implementing anything beyond what’s available in legacy CGI-based deployments.

1. WSGI is prone to header injection vulnerabilities issues by design due to the conversion of HTTP headers to CGI-style environment variables: if the server doesn’t specifically prevent it, X-Foo and X_Foo both become HTTP_X_Foo. I don’t believe it’s a good choice to destructively encode headers, expect applications to undo the damage somehow, and introduce security vulnerabilities in the process. If mimicking CGI is still considered a must-have — 1% of current Python web programmers may have heard about it, most of them from PEP 3333 — then that burden should be pushed onto the server, not the application.

2. More generally, I fail to see how mixing HTTP headers, server-related inputs, and environment variables in a dict adds values. It prevents iterating on each collection separately. It only makes sense if not offering more features than CGI is a design goal; in that case, this discussion doesn’t serve a purpose anyway. It would be nicer and possibly more secure if the application received separately:

a. Configuration information, which servers could read from environment variables by default for backwards compatibility, but could also get through more secure channels and restrict to what the application needs in order to better isolate it from the entire OS.

b. Server APIs mandated by the spec, per request.

c. HTTP headers, per request.

3. Stop pretending that HTTP is a unicode protocol, or at least stop ignoring reality when doing so. WSGI enforces ISO-8859-1-decoded str objects in the environ, which is just wrong. It’s all the more a surprising choice since this change was driven by Python 3, that UTF-8 is the correct choice, and that Python 3 defaults to UTF-8. Django has to re-encode and re-decode before doing anything with HTTP headers: https://github.com/django/django/blob/d5b90c8e120687863c1d41cf92a4cdb11413ad7f/django/core/handlers/wsgi.py#L231-L253

4. Normalize the way to tell the application about the original protocol, IP address and port. When dev and ops responsibilities are separate, this is clearly an ops responsibility, but due to the lack of standardization devs end up dealing with this problem in custom middleware, when they do it at all. Everyone keeps getting it wrong, which introduces security vulnerabilities. Also it always breaks silently on infrastructure changes.

5. Improve request / response length handling and connection closure. Armin and Graham have talked about in the past and know the topic better than I do. There’s also a rejected PEP by Armin which made sense to me.

As you can see from these comments, I don’t quite share the design choices that led to WSGI as it currently stands. I think it will be easier to build a new standard than evolve the current one.

I hope this helps!

-- 

Aymeric.

[Cory Benfield]

On Tuesday, January 5, 2016 at 10:37:32 AM UTC, Aymeric Augustin wrote:

Hi Cory,

I’m not subscribed to web-sig but I read the discussion there. Feel free to forward my answer to the group if you think it’s useful.

Thanks Aymeric, this feedback was extremely valuable. I've forwarded it to the web-sig list so that I can keep track of it all, which will let me get a feel for how the wider community thinks about WSGI.

Cory 

[Aymeric Augustin]

One more thing — while this isn’t a use case I have myself, some people rely on the ability to run an application at a sub-part e.g. under https://example.com/appname/.

WSGI should normalize the headers involved in that case so that applications don’t have custom code for various servers. Django currently contains such custom code.

-- 

Aymeric.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/fb4ad1a4-f2fb-469e-a4f7-56f407f52311%40googlegroups.com.