Re: Support for WSGI applications within Django (Ticket #12091)

[Carl Meyer]

Hello Gustavo,

On 04/11/2013 08:31 AM, Gustavo Narea wrote:
> In the interest of finding a resolution to Ticket #12091
> <https://code.djangoproject.com/ticket/12091>, a 3-year-old ticket, I
> wanted to bring this issue to your attention.
>
> I think that Django should have built-in support for running WSGI
> applications from inside views. (NB: I really mean running WSGI apps
> from inside Django views -- I'm not talking about using a WSGI
> middleware that routes requests to your Django project or another WSGI
> application.)
>
> If you're able to embed WSGI applications in your Django project, you'd
> be able to do many things. For example:
>
> * Control access to static files, while still letting your Web server
> do the file transfer: http://pythonhosted.org/xsendfile/
> * Re-use your Django-based authentication in other applications, even
> applications not written in Python. For example, if your Django site
> runs at "/" and your Wordpress blog and Trac instance live at
> "/blog/" and "/tickets/", respectively, your users would only have
> to log in once if your Django site proxies requests to Wordpress and
> Trac. Here's how I've integrate them in the
> past: https://bitbucket.org/Gustavo/weesgo/
> and http://gustavonarea.net/files/talks/europython2010/all-materials.zip
> * Alter the response returned by another web application or web site,
> using HttpResponse objects.
>
> It basically comes down to being able to integrate other applications
> and websites, and then filter the requests they get and/or the responses
> they return -- Using Django request and response objects.

This does seem like a useful feature for some situations, though it's
not a feature that I've seen much demand for.

> I decided to reimplement the patch I created 3 years ago in a separate
> project (twod.wsgi
> <http://pythonhosted.org/twod.wsgi/manual/embedded-apps.html>), which
> means that it's possible to do this without changing Django. However, it
> doesn't support embedding Django projects because of Django's use of
> global data (DJANGO_SETTINGS_MODULE and django.conf.settings), which I
> think can only be solved by introducing thread locals in django.conf. At
> the moment, the only way to accomplish this would be by proxying the
> second Django site with inter-process communication (e.g., CGI, FastCGI
> or HTTP).

If I understand correctly, this latter limitation is not something that
would be solved by integrating twod.wsgi into Django core. Are there any
issues or problems with twod.wsgi that would be solved by integrating it
into core?

> So, do you agree that Django should have built-in support for running
> WSGI applications from inside views?

I'm inclined to agree with Aymeric's position on the ticket: this is a
useful feature for some scenarios, but it works just fine as a
standalone project, and I just don't see enough demand for it to justify
adding the maintenance burden for it to core.

Carl

[Alex Ogier]

On Thu, Apr 11, 2013 at 1:02 PM, Carl Meyer <ca...@oddbird.net> wrote:
> On 04/11/2013 08:31 AM, Gustavo Narea wrote:
>> I decided to reimplement the patch I created 3 years ago in a separate
>> project (twod.wsgi
>> <http://pythonhosted.org/twod.wsgi/manual/embedded-apps.html>), which
>> means that it's possible to do this without changing Django. However, it
>> doesn't support embedding Django projects because of Django's use of
>> global data (DJANGO_SETTINGS_MODULE and django.conf.settings), which I
>> think can only be solved by introducing thread locals in django.conf. At
>> the moment, the only way to accomplish this would be by proxying the
>> second Django site with inter-process communication (e.g., CGI, FastCGI
>> or HTTP).
>
> If I understand correctly, this latter limitation is not something that
> would be solved by integrating twod.wsgi into Django core. Are there any
> issues or problems with twod.wsgi that would be solved by integrating it
> into core?

There's a wiki article on the global state issue in Django at
https://code.djangoproject.com/wiki/GlobalState but this is unlikely
to be solved soon. It's something of a painful transition, because
Django has built up a large ecosystem that depends on this global
state. The reason is basically that it simplified implementation
greatly, especially considering that Python's module/namespace system
is deeply global. Frameworks like Flask have worked hard to avoid
global state from the very beginning, for example Flask requires all
"official extensions" to support multi-tenant applications explicitly,
and recommends a pattern where WSGI applications are created by a
factory function. Changing the Django ecosystem to allow for reliable
multitenancy would be prohibitively disruptive.

>> So, do you agree that Django should have built-in support for running
>> WSGI applications from inside views?
>
> I'm inclined to agree with Aymeric's position on the ticket: this is a
> useful feature for some scenarios, but it works just fine as a
> standalone project, and I just don't see enough demand for it to justify
> adding the maintenance burden for it to core.
>
> Carl

I think this is a common (and I think correct) response to many
feature requests: If there is anything in core that is blocking the
external implementation of a feature like this, it's worth looking at,
else just implement it outside of core. Particularly in this instance
since it will still be impossible to embed Django applications
anyways.

Best,
Alex Ogier

[Gustavo Narea]

Hello, Carl and Alex,

You're right: Apart from not being able to embed Django sites, there's nothing preventing twod.wsgi from working with Django as it is now.

So I guess that means that it won't be considered for inclusion in Django? If so, I'll close that ticket.

Cheers,

 - Gustavo.

On Thursday, April 11, 2013 3:31:51 PM UTC+1, Gustavo Narea wrote:

Hello, everybody.

In the interest of finding a resolution to Ticket #12091, a 3-year-old ticket, I wanted to bring this issue to your attention.

I think that Django should have built-in support for running WSGI applications from inside views. (NB: I really mean running WSGI apps from inside Django views -- I'm not talking about using a WSGI middleware that routes requests to your Django project or another WSGI application.)

If you're able to embed WSGI applications in your Django project, you'd be able to do many things. For example:

• Control access to static files, while still letting your Web server do the file transfer: http://pythonhosted.org/xsendfile/

• Re-use your Django-based authentication in other applications, even applications not written in Python. For example, if your Django site runs at "/" and your Wordpress blog and Trac instance live at "/blog/" and "/tickets/", respectively, your users would only have to log in once if your Django site proxies requests to Wordpress and Trac. Here's how I've integrate them in the past: https://bitbucket.org/Gustavo/weesgo/ and http://gustavonarea.net/files/talks/europython2010/all-materials.zip

• Alter the response returned by another web application or web site, using HttpResponse objects.

It basically comes down to being able to integrate other applications and websites, and then filter the requests they get and/or the responses they return -- Using Django request and response objects.

I decided to reimplement the patch I created 3 years ago in a separate project (twod.wsgi), which means that it's possible to do this without changing Django. However, it doesn't support embedding Django projects because of Django's use of global data (DJANGO_SETTINGS_MODULE and django.conf.settings), which I think can only be solved by introducing thread locals in django.conf. At the moment, the only way to accomplish this would be by proxying the second Django site with inter-process communication (e.g., CGI, FastCGI or HTTP).

I'm not going to insist on adding support for embedding other Django projects, because I personally haven't had the need to do that and introducing thread locals in django.conf may be controversial. But I guess it might be useful to other people.

So, do you agree that Django should have built-in support for running WSGI applications from inside views?

Thanks in advance!

 - Gustavo Narea.

[Jacob Kaplan-Moss]

On Fri, Apr 12, 2013 at 8:54 AM, Gustavo Narea
<gustav...@2degreesnetwork.com> wrote:
> So I guess that means that it won't be considered for inclusion in Django?
> If so, I'll close that ticket.

Hm, I wouldn't be so hasty -- it's something I, at least, would like to include.

Of course, the concerns about the implementation are still there, and
they're non-trivial. So my actual yes/no would be based more on the
concrete proposal. But I think it'd be a good idea to include in core
*if* it can be done well.

I know it's a lot to ask given the mixed messages so far, but would
you be OK preparing a patch/pull request for us to discuss?

Jacob

[Gustavo Narea]

Hi Jacob,

I'm glad to hear you're interested.

I'm absolutely confident that it can be done well. (I'm pretty familiar with WSGI and Django's handlers implementation, and I've been using my implementation in a high-traffic site for 3 years with not a single bug found so far.)

I'd be happy to update the patch I attached to the ticket and make a pull request. But before I go ahead and spend time doing that, would you mind if I address any concerns that you may still have now? I'd prefer to save the time if it's not going to be accepted anyway.

Cheers,

 - Gustavo.

[Gustavo Narea]

Hello,

Any update on this?

Cheers.

 - Gustavo.

[Gustavo Narea]

Hello, Jacob et al.

I was wondering whether there's an update on this.

I'm still interested to learn about the concerns about the implementation, so that I can know what to focus on. And also whether it's even worth me doing a new implementation.

Cheers,

 - Gustavo.

[Gustavo Narea]

Hello, everybody.

Sorry for re-opening this old conversation, but I still haven't heard from you.

I'm not sure whether that means that there's no interest in this, or it's just that you've been busy.

Cheers,

 - Gustavo.