Does anyone know any blogs that write about how to quickly install and deploy a django application on Amazon EC2?
Chen Xu
⚡ Chen Xu ⚡
Alec Taylor
Stuart Laughlin
the following reasons:
# author admits he is a non-sysadmin noob
# he installs everything globally rather than using virtualenv
# he uses apache instead of... well... anything else
# there does not appear to be anything in there that is specific to
ec2
Bonus reason:
# he uses mysql instead of postgres
In fact there really isn't much of a difference between installing/
deploying to ec2 compared to installing anywhere else, so any good
article you find on django deployment will work on ec2.
Where this does not hold true is if you are looking to create the node
and deploy your app all in one step. In this case you will want to use
something like chef (or puppet I assume, though I have only used
chef). Be forewarned that it gets quite deep in terms of mastering
all the concepts.
http://wiki.opscode.com/display/chef/Build+a+Django+Stack
I reckon you could also do something like that, perhaps a bit more
crudely, with shell scripts and fabric. I haven't seen any good
examples of that though.
--Stuart
Javier Guerra Giraldez
> # author admits he is a non-sysadmin noob
ad-hominem
> # he installs everything globally rather than using virtualenv
on single-purpose servers it's not so important. still good practice,
but not mandatory.
> # he uses apache instead of... well... anything else
well-tuned apache and mod_wsgi is on the same league as the cool boys.
(i still prefer nginx, and i don't think this is a good example of
apache tuning, but nothing bad about it)
> # there does not appear to be anything in there that is specific to
> ec2
point conceded, but lots of people will look for EC2 instructions,
even if it's just "just like any old server"
> Bonus reason:
> # he uses mysql instead of postgres
again, might not be the bestest choice but nowhere near a bad one.
--
Javier
Stuart Laughlin
On Thursday, January 12, 2012 10:09:13 AM UTC-6, Javier Guerra wrote:
On Thu, Jan 12, 2012 at 9:54 AM, Stuart Laughlin <stu...@bistrotech.net> wrote:
> # author admits he is a non-sysadmin noobad-hominem
"I’m a sys admin NOOB. I am also teaching myself to code. This is my first time setting up this stack so there are probably going to be some bugs/security issues I conveniently side stepped just to get it to work."
"This is obviously not secure enough for a ‘production’ environment."
If that's what you want to model your deployment after, be my guest. I hardly think it's fallacious of me to suggest someone do otherwise.
> # he uses apache instead of... well... anything else
well-tuned apache and mod_wsgi is on the same league as the cool boys.
(i still prefer nginx, and i don't think this is a good example of
apache tuning, but nothing bad about it)
> Bonus reason:> # he uses mysql instead of postgres
again, might not be the bestest choice but nowhere near a bad one.
--Stuart
Jeff Heard
--To view this discussion on the web visit https://groups.google.com/d/msg/django-users/-/ByBrEdUj8HUJ.
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to django...@googlegroups.com.
To unsubscribe from this group, send email to django-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Stuart Laughlin
default AMIs (see http://alestic.com/ ). Here are a few links that I
have found particularly useful in the past, for various aspects of the
deployment process...
* http://www.saltycrane.com/blog/2009/05/notes-using-pip-and-virtualenv-django/
* http://www.saltycrane.com/blog/2010/10/how-install-pil-ubuntu/
* http://brandonkonkle.com/blog/2010/jun/25/provisioning-new-ubuntu-server-django/
Hope that helps,
--Stuart
Mario Gudelj
Andres Reyes
https://docs.djangoproject.com/en/dev/howto/deployment/ is not a good
choice?
2012/1/12 Stuart Laughlin <stu...@bistrotech.net>:
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/django-users/-/ByBrEdUj8HUJ.
>
> To post to this group, send email to django...@googlegroups.com.
> To unsubscribe from this group, send email to
> django-users...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-users?hl=en.
--
Andrés Reyes Monge
arm...@gmail.com
+(505)-8873-7217
Tom Evans
> I don't care a whit about what "the cool boys" are doing. What I care about
> is a production deployment that works efficiently and reliably and that is
> diagnosable when something doesn't work. Apache fails those criteria; nginx
> and lighttpd pass (in my opinion and the opinion of many other developers
> who have used these technologies for non-trivial web applications in
> production environments).
Despite the relentless attacks from people who have some sort of
grudge against Apache, I'd remind any readers that Apache is the most
used and trusted web server on the internet. It forms an integral part
of the technology that the company I work for is built around, and
performs extremely well for our usage, and I would thoroughly
recommend it.
Besides which, arguing about which web server is more efficient is a
clear indicator that you are concentrating on the wrong areas. nginx
or lighthttpd vs apache is meaningless when your the bottleneck will
always be the python application and/or DB server.
Cheers
Tom
Stuart Laughlin
> On Thu, Jan 12, 2012 at 11:32 PM, Stuart Laughlin <stu...@bistrotech.net> wrote:
> > I don't care a whit about what "the cool boys" are doing. What I care about
> > is a production deployment that works efficiently and reliably and that is
> > diagnosable when something doesn't work. Apache fails those criteria; nginx
> > and lighttpd pass (in my opinion and the opinion of many other developers
> > who have used these technologies for non-trivial web applications in
> > production environments).
>
> Despite the relentless attacks from people who have some sort of
> grudge against Apache, I'd remind any readers that Apache is the most
> used and trusted web server on the internet. It forms an integral part
> of the technology that the company I work for is built around, and
> performs extremely well for our usage, and I would thoroughly
> recommend it.
>
(57% and declining according to the most recent numbers I've seen).
"Most trusted" is questionable, being such a subjective quality, but
that's just a quibble. If apache forms an integral part of the
technology that your company uses, then obviously it makes sense for
you to use apache. Many people in this forum have no such investments
in apache and have no idea how to tune/configure it optimally. In that
case, I believe using apache is a mistake.
I hope you are not lumping me in with the relentless grudge-bearing
apache attackers. (After all, I've already been accused of ad hominem
attacks and some sort of fanboy me-too'ism on this thread [seriously
guys, is this a tough room or what?]). I have nothing against apache
per se -- it's a great server with a rich feature-set not found in the
lighter servers. The only thing is, I don't want or need any of those
extra features for my django projects. So the only things those extra
features would contribute to my projects are additional complexity and
inefficiency. Servers like nginx, lighttpd, and cherokee were designed
specifically for people in this situation.
> Besides which, arguing about which web server is more efficient is a
> clear indicator that you are concentrating on the wrong areas. nginx
> or lighthttpd vs apache is meaningless when your the bottleneck will
> always be the python application and/or DB server.
>
clearly more efficient and performant than apache, but my main concern
is being able to diagnose problems when they arise. I used apache/
mod_wsgi for my first few production deployments, and it was a very
unpleasant experience -- weird memory usage statistics, users getting
intermittent error messages, etc. -- and tracking down the problem was
basically impossible. I found apache to be an opaque black box. The
only thing that saved me was Graham Dumpleton sweeping in to rescue me
with the perfect mod_wsgi config. He's the killer feature of django on
apache as far as I'm concerned.
I hope this helps to clarify my comments regarding apache and why
someone who is not already an apache expert might wisely choose
something other than apache to serve their django projects.
--Stuart
Stuart Laughlin
> You mean that the webserver that Django itself recommends in https://docs.djangoproject.com/en/dev/howto/deployment/is not a good
> choice?
>
best, easiest, and most stable option. That combination still works
today and many people use it successfully. To that extent, it is a
good choice.
However, I am saying it is not the best choice anymore -- at least not
for people who are just getting started and/or people who lack prior
apache experience. In this case I would recommend nginx or lighttpd or
cherokee for the web server and gunicorn or uwsgi for the application
server. By this time they are stable, and they are faster and easier
than apache/mod_wsgi.
I wrote more about this in a reply to this thread where I changed the
subject to "Web Servers for Django Projects".
If I recall correctly, I have heard some of the core devs mention
updating the docs to include some of these other options. I wonder if
anyone has submitted a patch for those doc updates. If not, maybe I've
found my weekend project.
A final thought that I did not consider until just now -- cross-
platform compatibility might be a factor. I use linux for everything,
so that's all I care about. To be frank I have no idea whether nginx /
lighttpd / cherokee / gunicorn / uwsgi run on Windows or Mac. If they
do not then that can tend to complicate the docs somewhat.
--Stuart
Andre Terra
I've even managed to compile a version on cygwin with an additionally patched module that allows for file upload progress tracking.
If anyone wants it, i can put it on github with a sample .conf file, but no strings attached at all.
Cheers,
AT
--Stuart
--
You received this message because you are subscribed to the Google Groups "Django users" group.
Tom Evans
> I hope you are not lumping me in with the relentless grudge-bearing
> apache attackers. (After all, I've already been accused of ad hominem
> attacks and some sort of fanboy me-too'ism on this thread [seriously
> guys, is this a tough room or what?]). I have nothing against apache
> per se -- it's a great server with a rich feature-set not found in the
> lighter servers. The only thing is, I don't want or need any of those
> extra features for my django projects. So the only things those extra
> features would contribute to my projects are additional complexity and
> inefficiency. Servers like nginx, lighttpd, and cherokee were designed
> specifically for people in this situation.
I don't know what causes it, however this view - that lighty, nginx or
cherokee are higher performing than Apache - is just nonsense. It gets
rolled out whenever someone decides not to use Apache, and is without
basis in fact. Apache is also designed for extremely light weight
processes, if you configure it thus.
[Side note - none of this is personal to you! I'm not trying to say
that you are the fore-front of a horde of rabid anti-Apache-ism, it's
just a common view that is propagated that really winds me up!]
Apache covers almost every scenario related to HTTP, but commonly the
particular version that a distro will package up is not necessarily
the version that is correct for your particular scenario. Commonly, it
will come with every bell and whistle enabled, and using a
safe-but-slow request processing model, to accommodate mod_php.
What happens is that someone who does not know how to use Apache uses
the stock version supplied by their distro, leaves all features
enabled, and then benchmarks it against lighty with a minimal
configuration, and draws incorrect conclusions.
It's similar to benchmarking a Mercedes with a manual transmission
against a Ford with an automatic transmission, driving the Mercedes
only in first gear and concluding that the Ford is faster.
> I'm not concerned with micro-performance enhancements. Nginx is
> clearly more efficient and performant than apache, but my main concern
> is being able to diagnose problems when they arise. I used apache/
> mod_wsgi for my first few production deployments, and it was a very
> unpleasant experience -- weird memory usage statistics, users getting
> intermittent error messages, etc. -- and tracking down the problem was
> basically impossible. I found apache to be an opaque black box. The
> only thing that saved me was Graham Dumpleton sweeping in to rescue me
> with the perfect mod_wsgi config. He's the killer feature of django on
> apache as far as I'm concerned.
>
Memory usage is one of the more common non-complaints aimed at Apache.
Apache does not usually leak memory, and process sizes normally stay
consistent for long long periods. The only time this does not happen
is when you embed a language interpreter and application inside
Apache, and that leaks. This is typical when running with mod_wsgi
embedded mode, and I would guess that Graham had you change your setup
to daemon mode.
I would also say that Apache is a complex beast - there are lots of
options and modules. It also has a very responsive and helpful support
community. If anyone using Django needs help with any aspect of
Apache, try us...@httpd.org .
Cheers
Tom
PS: I would agree that Graham is one of the top features of Apache :)
His mod_wsgi is excellent, and his support for Django is very
admirable, given that (IIRC) he doesn't actually use it himself much!
Stuart Laughlin
But my other point, which you haven't addressed as much, is that apache is more difficult to troubleshoot when problems do arise. Because apache is a complex beast with lots of options and modules, it's very difficult for a non-expert to figure out where and why things have broken down. In my experience, the more simplistic servers are easier to troubleshoot (stands to reason, no?). I guess you are saying that this is no big deal because apache has a responsive and helpful support community, and that's fair enough, but I do think my point remains. After all, in spite of Graham coming through with The Perfect Config, I personally decided to not put myself through that experience again and moved away from apache on subsequent projects. I've been pleased with that decision, and I know many others share my experience.
So anyway, there are two aspects -- the performance/stability aspect where I reckon we are in essential agreement, and then there is the troubleshooting/diagnosing aspect where I am advocating that apache is more complicated than the lighter servers.
Hope that helps,
--Stuart
Stuart Laughlin
Just wanted to clarify that there are in fact some instructions out there for options other than apache/mod_wsgi, even though mod_wsgi remains the official preferred option ("If you’re new to deploying Django and/or Python, we’d recommend you try mod_wsgi first. In most cases it’ll be the easiest, fastest, and most stable deployment choice." [sic])
--Stuart
Javier Guerra Giraldez
> Just wanted to clarify that there are in fact some instructions out there
> for options other than apache/mod_wsgi, even though mod_wsgi remains the
> official preferred option ("If you’re new to deploying Django and/or Python,
> we’d recommend you try mod_wsgi first. In most cases it’ll be the easiest,
> fastest, and most stable deployment choice." [sic])
and Disqus agrees with that. check slide 14 of their 'Scaling
Disqus'[1] presentation at PyCon2011
[1] http://www.slideshare.net/zeeg/pycon-2011-scaling-disqus-7251315
--
Javier
Stuart Laughlin
The slide you reference shows two graphs comparing apache + mod_wsgi
with nginx + uwsgi. The graphs demonstrate that according to their
tests the two are roughly equal in terms of requests-per-second and
memory size. They conclude that the application (not the web server)
is the bottleneck and recommend that people use what they're
comfortable with.
I don't construe that as Disqus officially preferring apache, but you
are certainly entitled to your own interpretation.
Regards,
--Stuart
Javier Guerra Giraldez
> I don't construe that as Disqus officially preferring apache, but you
> are certainly entitled to your own interpretation.
right. their commentary at the PyCon (from memory, i haven't checked
the video) was something like "why apache and not nginx or anything
else? well... it's the same thing. get over it"
--
Javier
Sells, Fred
You referenced Graham’s “The Perfect Config”. I’ve been googling without success to find that. Do you have a link?
From: django...@googlegroups.com [mailto:django...@googlegroups.com] On Behalf Of Stuart Laughlin
Sent: Friday, January 13, 2012 12:52 PM
To: django...@googlegroups.com
Subject: Re: Web Servers for Django Projects [WAS: Does anyone know any blogs...]
Snip
kenneth gonsalves
> You referenced Graham’s “The Perfect Config”. I’ve been googling
> without success to find that. Do you have a link?
>
>
he means Graham's config is perfect. Google for django+wsgi
--
regards
Kenneth Gonsalves
Stuart Laughlin
Graham turned up on a thread in another forum where I was asking about how to fix a strange error I had been seeing intermittently for months. He instructed me on how to put mod-wsgi into daemon mode, among other things, and finally solved my problem.
Here's a link to the thread, if you're interested.
http://groups.google.com/group/satchmo-users/browse_thread/thread/6a65b4ab28df38dc/a9b22c236d63c323
--Stuart
Graham Dumpleton
On Jan 15, 9:40 am, Stuart Laughlin <stu...@bistrotech.net> wrote:
> Graham turned up on a thread in another forum where I was asking about how
> to fix a strange error I had been seeing intermittently for months. He
> instructed me on how to put mod-wsgi into daemon mode, among other things,
> and finally solved my problem.
>
> Here's a link to the thread, if you're interested.
>
And the problem was actually due to a likely bug in psycopg2 and not
mod_wsgi.
You would have had the same problems with knowing what the cause was
if you had hit this bug when using uWSGI in one of the ways that it
uses sub interpreters.
So, that the issue could arise isn't unique to mod_wsgi, but an issue
for any system using sub interpreters, which for WSGI hosting means
both mod_wsgi and uWSGI.
Ultimately, if an issue like this had occurred with gunicorn you would
have been equally stuffed. This is because the problem was in a C
extension module and problems in C extension modules like that are
hard to debug and invariably are fixed through thinking about the
problem and looking at source code and not through trying to run pdb
or even gdb on a running process. You are lucky when someone else
knows what the issue is already.
That all said, that mod_wsgi and uWSGI in some modes use sub
interpreters does cause more than it share of problems. This though
pretty well always derives from the fact that people writing C
extension modules take short cuts and don't even contemplate that the
extension module may be used in sub interpreters or even in multiple
sub interpreters at the same time.
There have even been examples where in the Python interpreter itself
they have broken things for sub interpreters. For example, fork() is
broken in Python 2.7.2 in sub interpreters. So, even the core
developers don't always get it quite right and they would generally
know more than most people who write C extension modules.
One of the benefits at least is that mod_wsgi is configurable enough
that one can configure around broken third party software when these
sorts of issues arise, other systems may not provide you as much
flexibility. The config you used was also not a 'perfect config' in
the general sense, but was one that just solved your specific problem.
What configuration is best is going to be specific to a particular
application and the requirements.
Now being part of the Apache eco system, mod_wsgi is a bit ham strung
by the default configs forced on people by PHP and Linux distros. So
the default isn't necessarily a good config for sites. Even so, it
still serves adequately for a lot people because the server generally
isn't the issue anyway. The only big problem is usually memory usage
because Apache/mod_wsgi for a Linux distro is embedded and prefork
MPM, which is bad for fat Python web applications.
As to ease of debugging, if you are debugging your code and need to
run pdb or other IDE debugger, although it can be done under mod_wsgi
with the right knowledge, you are much better off doing it in a proper
development system. Debugging is not the missing thing here. What most
people really need is decent monitoring of their production systems.
What use is the ability to debug code if you have no idea what the
problems are that are occurring in your production system to start
with.
So monitoring is going to be more important going forward. The WSGI
hosting issue is reasonably solved and debugging is something one
could always do.
Think about it. Do you really have any idea what your production
system is doing? How much time is spent in your application vs
database, vs time spent calling out to external web services, memcache
etc etc. Do you know how long the application is even taking to do a
connect to your database for every Django request and how much time
that is as a percentage of overall response time. If you don't know,
how do you expect to be able to improve the performance of your
application. This is why monitoring is going to be key and where
Python web applications have been really lacking in the past.
Graham
> --Stuart
>
>
>
>
>
>
>
> > You referenced Graham’s “The Perfect Config”. I’ve been googling without
>
> > ** **
>
> > *From:* django...@googlegroups.com [mailto:
> > django...@googlegroups.com] *On Behalf Of *Stuart Laughlin
> > *Sent:* Friday, January 13, 2012 12:52 PM
> > *To:* django...@googlegroups.com
> > *Subject:* Re: Web Servers for Django Projects [WAS: Does anyone know any
> > blogs...]****
>
> > ** **
>
> > Snip****
> > After all, in spite of Graham coming through with The Perfect Config, I
> > personally decided to not put myself through that experience again and
> > moved away from apache on subsequent projects. I've been pleased with that
> > decision, and I know many others share my experience.
>
Stuart Laughlin
Graham, thanks for your input, which I found enlightening, especially the information regarding extension modules and sub interpreters. I can see how some of the things I said previously may have been unfairly biased against apache, when the real issues were inherent to how WSGI works and so might very well have manifested no matter the web server involved (is that a fair way to state it?).
Regarding the topic of this thread, I am still inclined to think that the lighter servers are a better choice than apache for django devs in general (i.e. people who are not sysadmins or apache experts) simply because fewer options and fewer unneeded features makes for an easier/better installation, easier learning curve, and the like. I also note that everyone who advocates for apache concedes that the default installation/configuration is really not appropriate for django apps. I intend to look more deeply into the suitability of the default configurations of nginx / lighttpd / cherokee as delivered by various linux distros. Finally, your point about mod_wsgi's superior configurability is a good one, and something I had not considered.
I especially appreciate and echo your points about the importance of knowing what your production system is doing and the key role of monitoring. I'm hoping the work you are doing with New Relic will go a long way towards solving those problems.
--Stuart
Tom Evans
> I also note that everyone who advocates
> for apache concedes that the default installation/configuration is really
> not appropriate for django apps. I intend to look more deeply into the
> suitability of the default configurations of nginx / lighttpd / cherokee as
> delivered by various linux distros. Finally, your point about mod_wsgi's
> superior configurability is a good one, and something I had not considered.
You're inventing words that you think I said. The 'default' config is
entirely appropriate for django apps, it is however not high
performance.
I say 'default', because it is only the default on certain OS. On the
platform we serve on, FreeBSD, it is a simple choice which MPM to
choose, and the default configuration for Apache configured with
worker or event MPM is entirely on par with nginx/lighty.
On top of that, you get all the advantages of being part of the Apache
ecosystem. Apache 2.4.0 will be released this week (probably), with a
mass of enhancements:
http://httpd.apache.org/docs/trunk/new_features_2_4.html
I'd thoroughly recommend not ignoring Apache in search of the new cool.
Cheers
Tom
Stuart Laughlin
> On Mon, Jan 16, 2012 at 11:12 PM, Stuart Laughlin <stu...@bistrotech.net> wrote:
>> I also note that everyone who advocates
>> for apache concedes that the default installation/configuration is really
>> not appropriate for django apps. I intend to look more deeply into the
>> suitability of the default configurations of nginx / lighttpd / cherokee as
>> delivered by various linux distros. Finally, your point about mod_wsgi's
>> superior configurability is a good one, and something I had not considered.
>
> You're inventing words that you think I said. The 'default' config is
> entirely appropriate for django apps, it is however not high
> performance.
>
> I say 'default', because it is only the default on certain OS. On the
> platform we serve on, FreeBSD, it is a simple choice which MPM to
> choose, and the default configuration for Apache configured with
> worker or event MPM is entirely on par with nginx/lighty.
>
Fair enough, Tom.
"The default config of apache on certain operating systems (e.g. most
linux distros) will not yield high performance for your django apps."
I suspect it was my unqualified usage of the nebulous phrase "not
appropriate" that prompted you to accuse me of "inventing words that
[I] think [you] said," but in fact the more clearly stated proposition
above is what I had in mind in the first place.
Regards,
--Stuart
Naveen Arjunan
It runs on EC2 and you get all the advantges of EC2.
http://olddocs.dotcloud.com/tutorials/django/
On Thu, Jan 12, 2012 at 12:31 AM, Chen Xu <xuch...@gmail.com> wrote:
> Hi, everyone:
> Does anyone know any blogs that write about how to quickly install and
> deploy a django application on Amazon EC2?
>
>
> Thanks very much
> Best regards
>
> --
> ⚡ Chen Xu ⚡
Dale
What's the consensus (wink) on choice of web server if one is forced
to use the same server for dynamic Django/Python _and_ PHP content,
and also static content? I guess no one's really forcing me, but it
feels like if I have to setup multiple boxes/VMs with a different web
server on each, and learn how to configure the web servers, I'll never
have time to code my idea for next great web app that started me on
this journey (my background is BSD/Apache/PHP, and while anything new
will use Django/Python, there are a couple PHP apps I want to keep
using).
Dale
On Jan 14, 8:46 pm, Graham Dumpleton <graham.dumple...@gmail.com>
wrote:
>
> - Show quoted text -
Stuart Laughlin
I see you asked this question nearly two weeks ago, but for some reason your message was delivered to my inbox just today.
Since your background is apache, I would stay there if I were you (and use mod_wsgi for your django apps). The only caveat is that I have no experience running php and django side-by-side, but presumably it is possible and perhaps even fairly common, e.g. http://stackoverflow.com/questions/1020390/how-do-i-run-django-and-php-together-on-one-apache-server
Hope that helps,
--Stuart