Deprecate FCGI support in Django 1.7

1,793 views
Skip to first unread message

Florian Apolloner

unread,
Jul 14, 2013, 3:17:00 PM7/14/13
to django-d...@googlegroups.com
Hi,

I'd like to get rid of everything FCGI-specific in Django sooner or later (rather sooner). Flup isn't maintained since a long time and there is no ticket tracker to report stuff. Graham pointed out that if someone wants to use FCGI they can use http://uwsgi-docs.readthedocs.org/en/latest/Options.html#fastcgi-socket which doesn't even require flup, which sounds like a good compromise to me. I'd need some help for the docs from some uWSGI users, since I have no idea about it ;)

Thoughts, objections?

Cheers,
Florian

gilberto dos santos alves

unread,
Jul 14, 2013, 9:35:56 PM7/14/13
to django-d...@googlegroups.com
i start 2 months ago using fcgi inside an shared host (hostgator.com)
and after lots of tries with wsgi only using fcgi was worked with
apache2. but i will read and learn about uwsgi and try this. my app
use version 1.6a of django is 1.6b worked using python 2.6. because
parts of my app is with status development i will test this with
python 2.7 too. but i agree that docs is not clear, because they mixed
concepts of apache2 and django (directories static, admin etc). i am
reviewing these docs soon for clarify concepts about wsgi, fcgi and if
necessary uwsgi. If someone have advices or additional ref. is
welcome! ;>)

2013/7/14 Florian Apolloner <f.apo...@gmail.com>:
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-develop...@googlegroups.com.
> To post to this group, send email to django-d...@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>



--
gilberto dos santos alves
+55.11.98646-5049
sao paulo - sp - brasil

Curtis Maloney

unread,
Jul 14, 2013, 10:06:15 PM7/14/13
to django-d...@googlegroups.com
As much as I recognise FastCGI is pretty much a dead technology in the Python world, for people stuck with cPanel sites like HostGator, it still appears to be, pretty much, the only option.

And installing uWSGI there is simply not an option there.

So unless there's a pure python FastCGI -> WSGI library built that supports Django, we're just closing the door on this avenue...

[I say this despite my personal loathing of HostGator... :)]

--
Curtis Maloney

Russell Keith-Magee

unread,
Jul 15, 2013, 12:44:18 AM7/15/13
to django-d...@googlegroups.com
On Mon, Jul 15, 2013 at 10:06 AM, Curtis Maloney <cur...@acommoncreative.com> wrote:
As much as I recognise FastCGI is pretty much a dead technology in the Python world, for people stuck with cPanel sites like HostGator, it still appears to be, pretty much, the only option.

And installing uWSGI there is simply not an option there.

So unless there's a pure python FastCGI -> WSGI library built that supports Django, we're just closing the door on this avenue...

[I say this despite my personal loathing of HostGator... :)]
 
This would be my concern as well. 

I have no love for FastCGI as a platform, but there's a wide range of hosting providers out there, especially at the cheap end, that don't provide a WSGI option. 

The right option here may well be to say "Tough Luck" to these people and and encourage them to move to different hosting providers. However, I don't want to rush into this decision without considering the effect on the low end of our user base.

Yours,
Russ Magee %-)

Michael Manfre

unread,
Jul 15, 2013, 1:04:05 AM7/15/13
to django-d...@googlegroups.com
Has anyone thought to contact HostGator and see how they would react to Django dropping FastCGI and/or whether they would be willing to support a WSGI option.

Regards,
Michael Manfre


Russell Keith-Magee

unread,
Jul 15, 2013, 1:07:08 AM7/15/13
to django-d...@googlegroups.com

If it were just Hostgator that was the problem, that might be a worthwhile approach. But that's not the true scope of the problem. We're talking about every hosting provider in the "$5 per month" hosting category that currently supports PHP, and provides FCGI as the fastest way they can support any other framework without having to think too hard. 

I'm not arguing that FastCGI is a good option, just that it's prevalent. And if we're going to stop supporting it, we need to be aware of who we're cutting off.

Yours,
Russ Magee %-)

Florian Apolloner

unread,
Jul 15, 2013, 7:24:56 AM7/15/13
to django-d...@googlegroups.com
On Monday, July 15, 2013 7:07:08 AM UTC+2, Russell Keith-Magee wrote:
I'm not arguing that FastCGI is a good option, just that it's prevalent. And if we're going to stop supporting it, we need to be aware of who we're cutting off.

We won't cut anything off (maybe aside from the runfcgi command). Supporting FCGI should be as easy as shown in the flask docs: http://flask.pocoo.org/docs/deploying/fastcgi/ -- This means that even if we remove everything FCGI specific from Django this doesn't prevent people from using FCGI+Django, it just means they have a bit more work to do (although if you look at the flask docs, it's really just a .fcgi file).

Cheers,
Florian

Some Developer

unread,
Jul 15, 2013, 3:38:33 AM7/15/13
to django-d...@googlegroups.com
This would be a mistake in my opinion.

I run websites based on lots of different programming languages /
frameworks and the one constant between them all is that they all
support FastCGI which eases administrative overhead significantly.

If the only real concern is that flup is unmaintained and there have
been attempts to contact the original author with no response or no
interest then I am sure all interested parties could work something out
between them even if it is just a fork purely for bug fixes and nothing
more.

I haven't really spent any time looking at uWSGI and perhaps that should
be my next port of call but I wouldn't be so hasty as to deprecate
FastCGI support.

mjl Martin J. Laubach

unread,
Jul 15, 2013, 8:13:47 AM7/15/13
to django-d...@googlegroups.com
  Is there actually a problem with flup? Not being maintained doesn't mean it's totally broken and it obviously works just fine for a lot of folks?

        mjl

Some Developer

unread,
Jul 15, 2013, 8:24:32 AM7/15/13
to django-d...@googlegroups.com
I use it myself and haven't had any issues with it either I was just
responding to one of the previous posters who stated that because flup
was unmaintained it was a good reason to deprecate FastCGI in Django. He
did make a good point about there being no bug tracker but as I said if
it became a critical issue then a fork is always on the cards.

I'd be interested if any contact has been made with the author of flup
to see whether he has dropped the project or whether he just doesn't get
any bug reports warranting a new release.

Personally I haven't had any issues with flup at all. In fact for me
Django with flup + nginx + daemontools works wonderfully for production
sites. Perhaps not as fast as uWSGI but it is solid enough.

Message has been deleted

Jannis Leidel

unread,
Jul 15, 2013, 10:14:43 AM7/15/13
to django-d...@googlegroups.com
If you're suggesting to move the FastCGI code into a separate app: +1

In other words, I don't think we can remove it from Django and not provide a migration path for FastCGI users. It's simple enough to move the runfcgi management command and the flup based handler into a separate app without having to risk changes in behavior.

Jannis

PS: I've just registered django-fcgi on PyPI in case you want to release it under that name.

Florian Apolloner

unread,
Jul 15, 2013, 11:10:53 AM7/15/13
to django-d...@googlegroups.com
On Monday, July 15, 2013 4:14:43 PM UTC+2, Jannis Leidel wrote:
If you're suggesting to move the FastCGI code into a separate app: +1

I'd have just dropped it, but yes we can move it out; although someone else will have to step up to continue maintaining it (if there is a need to maintain it).

Florian

Some Developer

unread,
Jul 15, 2013, 12:20:45 PM7/15/13
to django-d...@googlegroups.com
What about SCGI and AJP support? Is that going?

Seems silly to drop FastCGI which is probably the most popular of three
and leave the other two intact. If you're going to drop something drop
all three.

At least then it is clear to users that WSGI is the only supported
option in Django.

Third parties can then maintain FastCGI, SCGI and AJP support (which I
believe all come from flup anayway).

gilberto dos santos alves

unread,
Jul 15, 2013, 12:45:22 PM7/15/13
to django-d...@googlegroups.com
remember that on "shared hosts" we do not have option to use, explicity http ports, like examples from [1]. we do not have root access, nor http.conf access for apache2/root configs.

when we hare dedicated server it is easy run app with django.

on shared host, all config for running django fcgi is made using with user ssh terminal, and with sftp/ftp (example filezilla), .htaccess files and cpanel.

for install uwsgi we need c compiler, that we do not have on shared hosts.

It is more easy and less expensive for "first-time companies using web apps" use shared hosts and only solution that i find is using fcgi.

if necessary additional work todo (installs, .htaccess, etc) it is not the problem


2013/7/15 Some Developer <someukd...@gmail.com>
--
You received this message because you are subscribed to the Google Groups "Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscribe@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.



--

Some Developer

unread,
Jul 15, 2013, 1:32:49 PM7/15/13
to django-d...@googlegroups.com
On 15/07/2013 13:37, Florian Apolloner wrote:
> On Monday, July 15, 2013 9:38:33 AM UTC+2, Some Developer wrote:
>
> This would be a mistake in my opinion.
>
>
> Based on what? Django would still support FCGI via flup like it does
> now, we'd just get rid of the management commands you usually can't use
> on shared hosting either way�

OK. Your initial post was somewhat confusing in that case.

> I'd like to get rid of everything FCGI-specific in Django sooner or later (rather sooner).

The question is why? There is very little FastCGI boilerplate in Django
as it is and what is there hasn't been touched for a number of years
(the runfcgi management command file hasn't been touched for 6 years).

Why not just remove the documentation for FastCGI, SCGI and AJP and make
it clear that WSGI is the only supported option for deploying Django
applications but keep the legacy code available for those who already
use it.

Developers then don't need to worry about breaking it since it is no
longer supported and those that do use FastCGI, SCGI and AJP can submit
patches to fix it if it ever does get broken. This also includes
submitting patches to flup if it is still maintained or taking it over
if it is not.

This way everyone wins.

Aymeric Augustin

unread,
Jul 15, 2013, 3:55:51 PM7/15/13
to django-d...@googlegroups.com
/Disclaimer: I don't know much about FCGI. I've never used it and I don't plan to. Please forgive me if I say something stupid./

For 4,5 years (August 2008 to March 2013), we've had a silly bug in Django that was a workaround for a bug in flup, and to this day no one can say if the original bug in flup is fixed or not. By fixing the bug in Django I've probably re-introduced the bug for users of flup. See https://code.djangoproject.com/ticket/17550. I can't tell if that was the right choice. I hope no one will complain.

That wasn't a good experience and I don't plan to spend (read: waste) any more time on bugs related to FCGI.

On 15 juil. 2013, at 09:38, Some Developer <someukd...@gmail.com> wrote:
> If the only real concern is that flup is unmaintained and there have been attempts to contact the original author with no response or no interest then I am sure all interested parties could work something out between them even if it is just a fork purely for bug fixes and nothing more.

In practice who are the interested parties?
- flup's developers have disappeared;
- Django's core developers don't use FCGI — at least, I don't know any active core dev who does.

That makes FCGI a dead end. At some point we'll have to pull the plug. Right now seems early.

With the rise of hosting platforms dedicated to "modern" frameworks and applications — where "modern" means less than 10 years old, which is a long time in today's tech world — I hope cheap WSGI hosting will become a reality, and then we'll have decent arguments for dropping official support for FCGI.

An alternative is to leave it up to the community to provide and maintain a FCGI to WSGI adapter. However, writing a robust adapter sounds hard. I'm not sure this plan would work.

--
Aymeric.




Aymeric Augustin

unread,
Jul 15, 2013, 4:07:53 PM7/15/13
to django-d...@googlegroups.com
On 15 juil. 2013, at 14:13, mjl Martin J. Laubach <goo...@emsi.priv.at> wrote:

> Is there actually a problem with flup?

Yes, there are.

About a hundred tickets related to fastcgi or flup were reported in the history of Django: https://code.djangoproject.com/search?q=flup&ticket=on
https://code.djangoproject.com/search?q=fastcgi&ticket=on
https://code.djangoproject.com/search?q=runfcgi&ticket=on

Even though the flow of new tickets has died off since 2010, several tickets are still open:
https://code.djangoproject.com/ticket/9191
https://code.djangoproject.com/ticket/10464
https://code.djangoproject.com/ticket/11694
https://code.djangoproject.com/ticket/11754
https://code.djangoproject.com/ticket/12322
https://code.djangoproject.com/ticket/12464
https://code.djangoproject.com/ticket/13603
https://code.djangoproject.com/ticket/14958
https://code.djangoproject.com/ticket/17081
https://code.djangoproject.com/ticket/20751

(Some look easy but haven't received much love.)

Besides bugs, support for deployment techniques other than WSGI accounts for significant complexity in the core request handling. The situation has improved since we dropped support for mod_python, but the code isn't in very good shape overall, and I find it hard to work in this area.

--
Aymeric.




Florian Apolloner

unread,
Jul 15, 2013, 5:19:07 PM7/15/13
to django-d...@googlegroups.com
On Monday, July 15, 2013 6:20:45 PM UTC+2, Some Developer wrote:
What about SCGI and AJP support? Is that going?
Yes, it's either all or nothing.

Florian Apolloner

unread,
Jul 15, 2013, 5:31:25 PM7/15/13
to django-d...@googlegroups.com
On Monday, July 15, 2013 9:55:51 PM UTC+2, Aymeric Augustin wrote:
- Django's core developers don't use FCGI — at least, I don't know any active core dev who does.

I do, but with a patched flup and a patched suexec and what else, so I am hardly a typical usecase; and I am in the process of moving the systems to a sane setup.
 
That makes FCGI a dead end. At some point we'll have to pull the plug. Right now seems early.

The plug is pulled when it's removed from Django, deprecating doesn't mean right now. Also, if we move it outside of django-core we can send a good signal that FCGI in Django is basically "Use at your own risk" (which it is already if you ask me).

Jacob Kaplan-Moss

unread,
Jul 15, 2013, 5:45:54 PM7/15/13
to django-developers
On Mon, Jul 15, 2013 at 4:31 PM, Florian Apolloner <f.apo...@gmail.com> wrote:
Also, if we move it outside of django-core we can send a good signal that FCGI in Django is basically "Use at your own risk" (which it is already if you ask me).

This, for me, is the key: anything that's not a WSGI container is basically the wrong way to serve Django. Full stop. By continuing to include these other weird protocols we're implying that they're fully supported when they're actually totally not. Back in the day, we needed FCGI support because the options for hosting Python stuff was super limited. Today, there are a ton of options, even in the ultra-low-cost category.

Let's start FCGI (and the others) down the path to deprecation. If there's truly a community that finds value in these things -- and frankly I think that once they try modern options they'll quickly switch and never look back -- then let them maintain the code.

Jacob

Ramiro Morales

unread,
Jul 15, 2013, 10:19:23 PM7/15/13
to django-d...@googlegroups.com
On Mon, Jul 15, 2013 at 6:45 PM, Jacob Kaplan-Moss <ja...@jacobian.org> wrote:
>
> Let's start FCGI (and the others) down the path to deprecation. If there's
> truly a community that finds value in these things -- and frankly I think
> that once they try modern options they'll quickly switch and never look back
> -- then let them maintain the code.

Coincidentally, Juan Luis Boya has just attached a [1]patch to ticket
#20751.

It includes an initial FastCGI testing infrastructure that could be of
help to the maintainers of any project that result of this discussion.

--
Ramiro Morales
@ramiromorales

1. https://code.djangoproject.com/attachment/ticket/20751/0001-socketumask-argument-for-runfcgi-tests.patch

Aymeric Augustin

unread,
Jul 17, 2013, 11:00:18 AM7/17/13
to django-d...@googlegroups.com
On 15 juil. 2013, at 23:45, Jacob Kaplan-Moss <ja...@jacobian.org> wrote:

> Let's start FCGI (and the others) down the path to deprecation. If there's truly a community that finds value in these things -- and frankly I think that once they try modern options they'll quickly switch and never look back -- then let them maintain the code.

OK, I'm convinced, let's do that!

--
Aymeric.


Juan Luis Boya

unread,
Jul 18, 2013, 6:30:22 PM7/18/13
to django-d...@googlegroups.com
I'm against deprecating FastCGI at all because it's the only cross platform, language independent, minimally scalable standard for deploying web applications in web servers (apart from HTTP proxy).

WSGI may be nice to Python developers, but the world doesn't end after Python. Not all web servers support WSGI nor many people will choose to deploy a Django application if he or she needs another web server for that task. (Yes, big applications may require their own physical or virtual server, but that's not the case for everyone.) Fulfilling a standard like FastCGI is a must.

Also, FastCGI is **not that terrible**: You choose a prefix and tell the web server to redirect requests to the FastCGI server. Those are the basics. There are issues in some servers which make you have to configure rewrites if you want to serve static files from the same prefix but those are not real issues of FastCGI.

On the other hand, flup is dead in many ways, but I think uWSGI with FastCGI may be a better alternative. We wouldn't need to worry about FastCGI ourselves, have more features and have no change in our infrastructure (uWSGI uses WSGI to communicate with the web application independently of the technology it uses to communicate with the web server).

tl;dr:
FastCGI: We have to support it.
runfcgi + flup: Deprecating it may be a good decision.
uWSGI + FastCGI: We should have nice docs about this.

Javier Guerra Giraldez

unread,
Jul 18, 2013, 6:53:48 PM7/18/13
to django-d...@googlegroups.com
On Thu, Jul 18, 2013 at 5:30 PM, Juan Luis Boya <ntr...@gmail.com> wrote:
> uWSGI + FastCGI: We should have nice docs about this.


as others have previously said, uWSGI isn't viable for everybody. is
there any other pure-python fcgi-wsgi server with reasonable
performance?

i think several people like to use gunicorn for http-wsgi, if it (or
something similar: flask? wep.py?, werkzeug?) supports fcgi, it could
be the recommended fcgi solution.


--
Javier

Curtis Maloney

unread,
Jul 18, 2013, 8:31:15 PM7/18/13
to django-d...@googlegroups.com
For those who are keen to keep support for FastCGI, would you be interested in helping me develop/maintain a Pure Python FastCGI->WSGI(Django-specific) publisher package?

I agree it's valuable to have, as many have said [and I believe all have agreed] but to keep it in core is unmaintainable.

And by "help", it could just be as much as feature requests, documentation, or testing [always testing! :)]

--
Curtis Maloney



--
You received this message because you are subscribed to the Google Groups "Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.

Some Developer

unread,
Jul 19, 2013, 12:12:30 AM7/19/13
to django-d...@googlegroups.com
On 19/07/13 01:31, Curtis Maloney wrote:
> For those who are keen to keep support for FastCGI, would you be
> interested in helping me develop/maintain a Pure Python
> FastCGI->WSGI(Django-specific) publisher package?
>
> I agree it's valuable to have, as many have said [and I believe all have
> agreed] but to keep it in core is unmaintainable.
>
> And by "help", it could just be as much as feature requests,
> documentation, or testing [always testing! :)]
>
> --
> Curtis Maloney

Post a link to a Github repo for the project and I'll watch it. I'm not
sure how much time I'd be able to contribute to it but I'll certainly
help where I can.

Juan Luis Boya

unread,
Jul 19, 2013, 8:03:42 AM7/19/13
to django-d...@googlegroups.com
On Friday, July 19, 2013 12:53:48 AM UTC+2, Javier Guerra wrote:
i think several people like to use gunicorn for http-wsgi, if it (or
something similar: flask? wep.py?, werkzeug?) supports fcgi, it could
be the recommended fcgi solution.

flask, webpy, werkzeug: all of those use flup. gunicorn is an HTTP server, unrelated AFAIK.

Juan Luis Boya

unread,
Jul 19, 2013, 8:14:15 AM7/19/13
to django-d...@googlegroups.com
For those who are keen to keep support for FastCGI, would you be interested in helping me develop/maintain a Pure Python FastCGI->WSGI(Django-specific) publisher package?

That exists and it's called flup. The code base is relatively small. What about simply forking it to something like flup2 and fixing the issues?

Christian Schmitt

unread,
Jul 20, 2013, 7:32:44 AM7/20/13
to django-d...@googlegroups.com
I would recommand to remove FastCGI. 
That's the thing i love with the Python Community. Remove depracted or obsolete things.

The thing is, there are way more important things to do, than supporting a 'depracted' way to deploy. And it's really not that hard to get mod_wsgi with a httpd running.
You could strip that down so easily and compile it, even on CentOS.

Maybe it's better to improve the Docs about Apache2 / nginx / HAProxy deployment with mod_wsgi than supporting FCGI.
I mean some things like that:


I think that these things are really important. Howto get from development to Production, with security.
(btw. the talk is from 2011 and some things aren't up 2 date, like the pgpool single point of failure thing, etc. today you could really well deploy a good enviroment or 2 or more servers where you don't have any single point of failure where you don't need to add any extra line of code. okai maybe you need to find a good way to hold cookies in a good memory based database, that you could share them between machines, but that is the only thing you should care about, since that is the only you shouldn't hold in your legacy database.)

Am Sonntag, 14. Juli 2013 21:17:00 UTC+2 schrieb Florian Apolloner:
Hi,


I'd like to get rid of everything FCGI-specific in Django sooner or later (rather sooner). Flup isn't maintained since a long time and there is no ticket tracker to report stuff. Graham pointed out that if someone wants to use FCGI they can use http://uwsgi-docs.readthedocs.org/en/latest/Options.html#fastcgi-socket which doesn't even require flup, which sounds like a good compromise to me. I'd need some help for the docs from some uWSGI users, since I have no idea about it ;)

Thoughts, objections?

Cheers,
Florian

Some Developer

unread,
Jul 20, 2013, 7:56:07 AM7/20/13
to django-d...@googlegroups.com
On 20/07/13 12:32, Christian Schmitt wrote:
> I would recommand to remove FastCGI.
> That's the thing i love with the Python Community. Remove depracted or
> obsolete things.
>
> The thing is, there are way more important things to do, than supporting
> a 'depracted' way to deploy. And it's really not that hard to get
> mod_wsgi with a httpd running.
> You could strip that down so easily and compile it, even on CentOS.

The thing that people seem to be missing here is that FastCGI is not
obsolete. There is currently no other deployment technology that can
reasonably replace it for the people who make use of it (with the
possible exception of SCGI).

FastCGI is essential in environments where websites are hosted that use
different languages and frameworks. WSGI is Python only and therefore
would require two deployment methods based on technology used increasing
maintenance overhead and ultimately cost.

uWSGI is starting to get a few more plugins written for it for other
languages but it is still lacking somewhat (and when I have a couple of
days spare I may well work on adding some of what I need).

Nor is FastCGI obsolete. People in the technology field suffer from a
problem were old = bad when in fact old means stability which is crucial
for important production sites. Flup may well have issues and I would be
more than happy to work on fixing those issues but lets not mistake
problems with Flup as problems with FastCGI itself. Flup is an
implementation. FastCGI is a protocol and the protocol is just fine.


Christian Schmitt

unread,
Jul 20, 2013, 8:24:49 AM7/20/13
to django-d...@googlegroups.com
No FastCGI isn't a bad protocol. But it's not recommand to deploy Python Application's with FastCGI. FastCGI is good for deploying PHP or even sometimes Golang (but here, the plain Webserver is better).
But! FastCGI is a low level protocol and WSGI tries to get rid of that.
Flup is just a server/middleware that makes a WSGI based Application to a SCGI/FCGI one. It's never recommand to use a low level api, when you don't have to.
Also people who deployed on FastCGI should change that behavior to mod_wsgi, and there are a shitload of hosters that already supporting mod_wsgi, so it's really not that easy to change your hoster.

Btw. the djangoproject also have a good list which lists some hosters:

Some Developer

unread,
Jul 20, 2013, 8:52:00 AM7/20/13
to django-d...@googlegroups.com
On 20/07/13 13:24, Christian Schmitt wrote:
> No FastCGI isn't a bad protocol. But it's not recommand to deploy Python
> Application's with FastCGI. FastCGI is good for deploying PHP or even
> sometimes Golang (but here, the plain Webserver is better).
> But! FastCGI is a low level protocol and WSGI tries to get rid of that.
> Flup is just a server/middleware that makes a WSGI based Application to
> a SCGI/FCGI one. It's never recommand to use a low level api, when you
> don't have to.
> Also people who deployed on FastCGI should change that behavior to
> mod_wsgi, and there are a shitload of hosters that already supporting
> mod_wsgi, so it's really not that easy to change your hoster.
>
> Btw. the djangoproject also have a good list which lists some hosters:
> https://code.djangoproject.com/wiki/DjangoFriendlyWebHosts

I think you have completely missed my point. The people I am talking
about are the ones who maintain their own servers. Plus there are lots
of languages out there than can be deployed using FastCGI (off the top
of my head I know people use FastCGI for Haskell, Scheme, PHP, Go, Rust,
Python, Perl, C, and C++).

Your argument about FastCGI being a low level protocol is a bit weak to
be honest. Those using it know about the protocol and know how to work
with it. mod_wsgi does not solve the problem I laid out in my previous
post which is that it is better to have one deployment technology in use
regardless of the language used. This is what makes FastCGI unique at
the moment.

I'm longing for the day when uwsgi reaches the same level of adoption
but it won't be for sometime yet.

Curtis Maloney

unread,
Jul 20, 2013, 9:02:58 AM7/20/13
to django-d...@googlegroups.com
I'm more or less building atop flup as it is, however I plan to shed anything not related to FastCGI.

For me it's a chance to get down and dirty with raw protocols again... I do agree there is a shorter path to just applying Django's "fixes" to a fork of flup.

--
Curtis



On 19 July 2013 22:14, Juan Luis Boya <ntr...@gmail.com> wrote:
For those who are keen to keep support for FastCGI, would you be interested in helping me develop/maintain a Pure Python FastCGI->WSGI(Django-specific) publisher package?

That exists and it's called flup. The code base is relatively small. What about simply forking it to something like flup2 and fixing the issues?

--

Some Developer

unread,
Jul 20, 2013, 12:30:50 PM7/20/13
to django-d...@googlegroups.com
On 20/07/2013 14:02, Curtis Maloney wrote:
> I'm more or less building atop flup as it is, however I plan to shed
> anything not related to FastCGI.
>
> For me it's a chance to get down and dirty with raw protocols again... I
> do agree there is a shorter path to just applying Django's "fixes" to a
> fork of flup.
>
> --
> Curtis

If you really want to get down and dirty with raw protocols have you
considered an implementation that used epoll or kqueue for the
networking? Combined with nginx as the front end HTTP server it should
(theoretically) result in a huge increase in performance.

Basing the work on one of the Python async networking libraries
(Twisted, Eventlet, gevent etc) should result in better performance and
might well make FastCGI competitive with uWSGI and WSGI again. I really
think that flup is the bottleneck when it comes to using FastCGI with
Django. Having said that I haven't had a chance to look at the Django
side of the code yet so there might be some optimisations that are
possible there as well.

Juan Luis Boya

unread,
Jul 20, 2013, 12:54:30 PM7/20/13
to django-d...@googlegroups.com

If you really want to get down and dirty with raw protocols have you
considered an implementation that used epoll or kqueue for the
networking? Combined with nginx as the front end HTTP server it should
(theoretically) result in a huge increase in performance.

Basing the work on one of the Python async networking libraries
(Twisted, Eventlet, gevent etc) should result in better performance and
might well make FastCGI competitive with uWSGI and WSGI again. I really
think that flup is the bottleneck when it comes to using FastCGI with
Django. Having said that I haven't had a chance to look at the Django
side of the code yet so there might be some optimisations that are
possible there as well.

 
AFAIK Django is not an asynchronous server, so each thread or process will handle only one connection at a time. That renders the asynchronous approach near useless.

Correct me if I'm wrong.

Curtis Maloney

unread,
Jul 21, 2013, 12:06:22 AM7/21/13
to django-d...@googlegroups.com
Juan, technically Django isn't a server at all... whether it's sync, async, multi-threaded, multi-process, or a mix of any of them is up to the wsgi publisher used.

Take, for example, gunicorn, which can be any of the above, with options to plug in others when they're thought up.

--
Curtis



--

Curtis Maloney

unread,
Jul 21, 2013, 12:08:54 AM7/21/13
to django-d...@googlegroups.com
On 21 July 2013 02:30, Some Developer <someukd...@gmail.com> wrote:
On 20/07/2013 14:02, Curtis Maloney wrote:
I'm more or less building atop flup as it is, however I plan to shed
anything not related to FastCGI.

For me it's a chance to get down and dirty with raw protocols again... I
do agree there is a shorter path to just applying Django's "fixes" to a
fork of flup.
 
If you really want to get down and dirty with raw protocols have you considered an implementation that used epoll or kqueue for the networking? Combined with nginx as the front end HTTP server it should (theoretically) result in a huge increase in performance.

Well, that remains to be seen... it's early days yet, but I hope to be able to support various work models. 

 
Basing the work on one of the Python async networking libraries (Twisted, Eventlet, gevent etc) should result in better performance and might well make FastCGI competitive with uWSGI and WSGI again. I really think that flup is the bottleneck when it comes to using FastCGI with Django. Having said that I haven't had a chance to look at the Django side of the code yet so there might be some optimisations that are possible there as well.

I wasn't aware there was a particular performance issue, but I'll certainly keep it in mind.

Mostly, I was hoping to raise a new project to bear the burden of maintenance, instead of Django core.  From what I've seen of late there's various parts of the code which could be simplified if core assumes it's only ever a WSGI app to be published.

--
Curtis

Some Developer

unread,
Jul 21, 2013, 3:28:13 AM7/21/13
to django-d...@googlegroups.com
On 21/07/2013 05:08, Curtis Maloney wrote:
> I wasn't aware there was a particular performance issue, but I'll
> certainly keep it in mind.

Take a look at this:

http://www.peterbe.com/plog/fcgi-vs-gunicorn-vs-uwsgi

I've probably already said it but if you want some help with this
project I'd be interested.

Juan Luis Boya

unread,
Jul 21, 2013, 12:39:20 PM7/21/13
to django-d...@googlegroups.com

Juan, technically Django isn't a server at all...

Django is a web application framework whose operation consists on waiting for HTTP requests from the network (encapsulated with WSGI) and replying each one of them with a HTTP response. Call it what you want.
 
whether it's sync, async, multi-threaded, multi-process, or a mix of any of them is up to the wsgi publisher used.

This is what I understand with 'asynchronous' and 'synchronous' servers:

* A synchronous server attends only one client connection at a time for each worker thread. Concurrency is achieved with operating system mechanisms like threads and processes.
* An asynchronous server may be attending many client connections at a time for each worker thread. Concurrency is achieved within the thread using an application specific dispatcher. In an asynchronous server, there is only one blocking call accepted, which selects events from all connections. Typically this call is select(), epoll() or kqueue() depending on the system. **Other blocking calls must not be used because they would block the entire server until they return, preventing all available processing for other connections and thus killing server performance**. Therefore, asynchronous programming impose additional restrictions to the programmer, like:
   * Do not read or write files yourself, cause read() and write() are blocking by default. You must pass those operations through the dispatcher.
   * All database operations must pass through the dispatcher too. You can't block the thread until a SQL statement ends its execution.
   * All additional HTTP requests (like those to external APIs) must pass through the dispatcher too in order to not block the server until they are replied.
and so on...

There are also mixed approaches, i.e. an asynchronous load balancer which delivers new connections to synchronous worker threads or processes using IPC mechanisms.

Since most (if not all) Django applications rely on blocking operations like those stated before, trying to make Django an asynchronous web platform expecting a huge performance improvement is a wrong idea. Putting that apart, the asynchronous load balancer (which can create and delete threads/processes depending on the server load) is not a bad idea.

Javier Guerra Giraldez

unread,
Jul 21, 2013, 11:03:30 PM7/21/13
to django-d...@googlegroups.com
On Sun, Jul 21, 2013 at 11:39 AM, Juan Luis Boya <ntr...@gmail.com> wrote:
> * An asynchronous server may be attending many client connections at a time
> for each worker thread. Concurrency is achieved within the thread using an
> application specific dispatcher. In an asynchronous server, there is only
> one blocking call accepted, which selects events from all connections.
> Typically this call is select(), epoll() or kqueue() depending on the
> system. **Other blocking calls must not be used because they would block the
> entire server until they return, preventing all available processing for
> other connections and thus killing server performance**. Therefore,
> asynchronous programming impose additional restrictions to the programmer,
> like:


this is mostly right; but python, being a high level language, leaves
a lot of space to change things.

greenlets, gevents and eventles take advantage of that, monkepatching
several crucial parts of the standard library, making it
asynchronous-friendly. in the end, reasonably good code becomes
mostly good for an asynchronous sever.

That's what happens when you run use gunicorn to run Django, which is
a very popular option (and isn't bad at all). An asynchronous
FCGI-WSGI tool could have a similar performance without any changes in
Django.

in fact, i still think that maybe (just maybe) the easiest would be to
replace the HTTP parser in gunicorn with an FCGI one, keeping all the
asynchronous structure and WSGI handling code.

--
Javier

Curtis Maloney

unread,
Jul 21, 2013, 11:05:18 PM7/21/13
to django-d...@googlegroups.com
My understanding was Django is a WSGI application, and as such its role is to process requests handed to it.  It isn't the server as such.

It's the role of a WSGI publisher to accept requests and pass them on to an appropriate handler.  So, uWSGI, gunicorn, flup and mod_wsgi as the Publishers we most often hear of, and it's generally their place to implement the fork/thread/greenlet model used.



--

Some Developer

unread,
Jul 22, 2013, 12:21:08 AM7/22/13
to django-d...@googlegroups.com
On 14/07/13 20:17, Florian Apolloner wrote:
> Hi,
>
> I'd like to get rid of everything FCGI-specific in Django sooner or
> later (rather sooner). Flup isn't maintained since a long time and there
> is no ticket tracker to report stuff. Graham pointed out that if someone
> wants to use FCGI they can use
> http://uwsgi-docs.readthedocs.org/en/latest/Options.html#fastcgi-socket
> which doesn't even require flup, which sounds like a good compromise to
> me. I'd need some help for the docs from some uWSGI users, since I have
> no idea about it ;)
>
> Thoughts, objections?
>
> Cheers,
> Florian

So has this decision been made yet? If it has myself and others can
start coming up with solutions for those who want FastCGI. If it hasen't
then we can avoid wasting time on something that is going to be a non-issue.

The sooner that work is started before 1.7 though means that it might
not actually be a problem for users since FastCGI will still work.

Russell Keith-Magee

unread,
Jul 22, 2013, 12:47:16 AM7/22/13
to django-d...@googlegroups.com
Based on the feedback from this thread (and the fact that multiple core devs, including a BDFL have weighed in), I think it's safe to say that the decision has been made. The only question at this point is how long it takes before someone commits the changeset that implements the start of the deprecation process.

Yours,
Russ Magee %-)

Some Developer

unread,
Jul 22, 2013, 4:48:31 AM7/22/13
to django-d...@googlegroups.com
On 22/07/13 05:47, Russell Keith-Magee wrote:
> Based on the feedback from this thread (and the fact that multiple core
> devs, including a BDFL have weighed in), I think it's safe to say that
> the decision has been made. The only question at this point is how long
> it takes before someone commits the changeset that implements the start
> of the deprecation process.
>
> Yours,
> Russ Magee %-)

Thanks. Its always better to know that a definitive decision has been made.

I guess now we can start looking into third party alternatives.

To anyone else reading this thread who is still wanting FastCGI I am
willing to help with a third party solution if something is going to
come together. Based on the normal Django release cycle I'd expect 1.7
to be out in either Q4 this year or Q1 next year so something needs to
be done by then (preferably including testing in production - something
which I can help with).

P.S What's a BDFL? :)

Russell Keith-Magee

unread,
Jul 22, 2013, 5:00:01 AM7/22/13
to django-d...@googlegroups.com
The hope is to start accelerating the release cycle slightly; however, this also depends on someone actually doing the work :-) 
 
P.S What's a BDFL? :)

Aymeric Augustin

unread,
Jul 22, 2013, 5:00:29 AM7/22/13
to django-d...@googlegroups.com
> To anyone else reading this thread who is still wanting FastCGI I am willing to help with a third party solution if something is going to come together. Based on the normal Django release cycle I'd expect 1.7 to be out in either Q4 this year or Q1 next year so something needs to be done by then (preferably including testing in production - something which I can help with).

FastCGI will be pending deprecation in Django 1.7 and deprecated in Django 1.8. It wont' be removed until Django 1.9. You actually need a solution by 2015!

> P.S What's a BDFL? :)

It's an acronym for "Benevolent Dictator for Life". See https://en.wikipedia.org/wiki/Benevolent_Dictator_for_Life for details.

--
Aymeric.

Some Developer

unread,
Jul 22, 2013, 1:25:29 PM7/22/13
to django-d...@googlegroups.com
On 22/07/13 10:00, Russell Keith-Magee wrote:
> The hope is to start accelerating the release cycle slightly; however,
> this also depends on someone actually doing the work :-)

Sounds good. I'd love to start helping with Django myself but I feel a
bit lost when it comes to useful things to do. Maybe I'll start checking
out the bug tracker and seeing what I can do there.

Curtis Maloney

unread,
Jul 22, 2013, 9:07:11 PM7/22/13
to django-d...@googlegroups.com
Well, having started looking over the flup source, I can see a lot of useful code to crib, as well as a few critical path bits that can be optimised [without even reducing readability!]

So, if you feel like helping out with django-fastcgi ... I'm happy for the help :)

--
Curtis



--
You received this message because you are subscribed to the Google Groups "Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscribe@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.

Some Developer

unread,
Jul 23, 2013, 1:19:28 AM7/23/13
to django-d...@googlegroups.com
On 23/07/13 02:07, Curtis Maloney wrote:
> Well, having started looking over the flup source, I can see a lot of
> useful code to crib, as well as a few critical path bits that can be
> optimised [without even reducing readability!]
>
> So, if you feel like helping out with django-fastcgi ... I'm happy for
> the help :)
>
> --
> Curtis

Fork it and upload it to Github and then we can see where we stand.
Github probably has the best chance of getting people involved with it.

But yeah I'm available to help. Although I'm back at work next week so
my time will be somewhat more limited than it is at the moment.

Tim Graham

unread,
Jul 23, 2013, 7:53:57 AM7/23/13
to django-d...@googlegroups.com
Here's a ticket and pull request for the deprecation.

Joe Tennies

unread,
Jul 24, 2013, 1:04:28 AM7/24/13
to django-d...@googlegroups.com

Is there a specific reason to call this "django-fastcgi"? I think it should most likely be a generic wsgi to fastcgi binding, right? Or is this project mainly the django management commands?

To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.

Curtis Maloney

unread,
Jul 24, 2013, 1:30:36 AM7/24/13
to django-d...@googlegroups.com
Well, mostly to indicate it's primary focus is on providing a FastCGI server for Django.

I'm quite happy to keep it app agnostic, and be a general FastCGI->WSGI publisher...

Tom Evans

unread,
Jul 24, 2013, 5:56:45 AM7/24/13
to django-d...@googlegroups.com
Did you notice that the first thing that anyone does when benchmarking
HTTP servers is that they reduce the work being done by the server so
that it is trivial. That link explains it best, if you throw away
everything that Django provides and simply use it to render a static
string, then uWSGI is faster than gunicorn, which is faster than FCGI.
If you leave any of it enabled however, they all benchmark at
precisely the same speed.

So, if your django website does not use sessions nor databases nor
templates, then sure, you get a huge performance benefit from fiddling
around with serving mechanisms.

Cheers

Tom

Tim Graham

unread,
Aug 5, 2013, 12:38:33 PM8/5/13
to django-d...@googlegroups.com, teva...@googlemail.com
Is there a consensus on the project that'll be created to handle this outside of Django? Would be nice to have something to point people to when closing Trac tickets for FastCGI.

VernonCole

unread,
Aug 6, 2013, 7:28:49 AM8/6/13
to django-d...@googlegroups.com
Let me second the need for a good "How To" document on deployment.  As a neophyte django user I suddenly felt very lost last week. After several months of development, the ADO interface to the Microsoft SQL server from Linux is working (it still needs some tuning, but the data does go in and out using runserver) and it is now time to deploy.  The boss requested an Nginx server, so I started searching the web for how to do it.  I had come the the conclusion that FastCGI was the only way that I could deploy and was planning on starting that this morning, until I read this thread.  Now I know that I was about to start down an obsolescent path. A "How To" in the official documentation would have prevented this false start.

Does anyone know of an existing cookbook that I can refer to as a starting point? Or should I be making accurate records of my work with a view to contributing a new document?
--
Vernon Cole
 

On Saturday, July 20, 2013 5:32:44 AM UTC-6, Christian Schmitt wrote:
I would recommand to remove FastCGI. 
That's the thing i love with the Python Community. Remove depracted or obsolete things.

The thing is, there are way more important things to do, than supporting a 'depracted' way to deploy. And it's really not that hard to get mod_wsgi with a httpd running.
You could strip that down so easily and compile it, even on CentOS.

Maybe it's better to improve the Docs about Apache2 / nginx / HAProxy deployment with mod_wsgi than supporting FCGI.
I mean some things like that:


I think that these things are really important. Howto get from development to Production, with security.
(btw. the talk is from 2011 and some things aren't up 2 date, like the pgpool single point of failure thing, etc. today you could really well deploy a good enviroment or 2 or more servers where you don't have any single point of failure where you don't need to add any extra line of code. okai maybe you need to find a good way to hold cookies in a good memory based database, that you could share them between machines, but that is the only thing you should care about, since that is the only you shouldn't hold in your legacy database.)

Javier Guerra Giraldez

unread,
Aug 6, 2013, 9:18:52 AM8/6/13
to django-d...@googlegroups.com
On Tue, Aug 6, 2013 at 6:28 AM, VernonCole <verno...@gmail.com> wrote:
> I had come the the conclusion that FastCGI was the only way


behind NginX, the best two options seem to be uWSGI and gunicorn.
personally, I like uWSGI, but both are quite performant and
low-resource.

--
Javier

Andre Terra

unread,
Aug 6, 2013, 10:40:31 AM8/6/13
to django-d...@googlegroups.com
Assuming a UNIX environment. In Windowsland, nginx is still one of the best alternatives.


Cheers,
AT


Javier Guerra Giraldez

unread,
Aug 6, 2013, 11:13:03 AM8/6/13
to django-d...@googlegroups.com
On Tue, Aug 6, 2013 at 9:40 AM, Andre Terra <andre...@gmail.com> wrote:
> Assuming a UNIX environment. In Windowsland, nginx is still one of the best
> alternatives.


I meant use those _behind_ NginX. neither is really good as an
external-facing HTTP process.

--
Javier
Reply all
Reply to author
Forward
0 new messages