Are my Python/Pylons concerns justified?

[walterbyrd]

I have been looking at different PHP and Python frameworks. The only
Python framework that I have actually toyed with at all, is Django.

I like Python better than PHP, but I think there may be compelling
advantages to PHP when it comes to frameworks. Issues that concerns me
with Python, and Python frameworks, include: hosting, deployment,
configuration and administrative overhead.

I get the impression that Python framework developers just assume the
user has complete control over the server. Those who develop Python
frameworks seem to have little regard for the realities of shared
hosting. System requirements tend to be sky-high: Apache 2.x,
mod_python (latest version), fastcgi (at least), command line access,
PostgreSQL (recommended). Assuming you can meet the system
requirements, you still have server configuration issues such as
setting up the Path, and maybe the .htaccess file.

Then there is the ongoing configuration. I wonder if Pylons is any
better than Django in this regard, or maybe it's not as big an issue
it seems.

For example, as I understand it, in Django:
* The database model has to be manually synced to database from the
command line, anytime any change is made to the database.
* A configuration file has to be created for every application.
* Urls have to be configured in the urls.py file to match the view.
* The webserver has to be restarted every time modify your code - or
touch every file if using fastcgi.

>From what I am gathering, Pylons requires a similar amount of ongoing
configuration. I don't think this amount of configuration is required
by Rails, or the PHP frameworks. Am I wrong about that? And is it
really an issue?

[Echo]

On 8/8/07, walterbyrd <walte...@iname.com> wrote:
>
> I have been looking at different PHP and Python frameworks. The only
> Python framework that I have actually toyed with at all, is Django.
>
> I like Python better than PHP, but I think there may be compelling
> advantages to PHP when it comes to frameworks. Issues that concerns me
> with Python, and Python frameworks, include: hosting, deployment,
> configuration and administrative overhead.
>
> I get the impression that Python framework developers just assume the
> user has complete control over the server. Those who develop Python
> frameworks seem to have little regard for the realities of shared
> hosting. System requirements tend to be sky-high: Apache 2.x,
> mod_python (latest version), fastcgi (at least), command line access,
> PostgreSQL (recommended). Assuming you can meet the system
> requirements, you still have server configuration issues such as
> setting up the Path, and maybe the .htaccess file.

PostgreSQL is definatly not required. You should be able to use any
database that SQLAlchemy can work with(That include MySQL which I am
currently using).
For the hosting, I agree that it doesn't look the easiest. However
there is a hosting company that supports Pylons,
http://www.webfaction.com/pylons-hosting. I plan on going with them
once I finish the site that I am working on.

> * The webserver has to be restarted every time modify your code - or
> touch every file if using fastcgi.

If you use the paste server, it can automatically reload on any change.

>
> >From what I am gathering, Pylons requires a similar amount of ongoing
> configuration. I don't think this amount of configuration is required
> by Rails, or the PHP frameworks. Am I wrong about that? And is it
> really an issue?

I rarely make any changes to my configuration. Once I got it
connecting the the database(easy) and got AuthKit working(took a bit)
I havn't touched the configuration since.

--
-Echo

[ToddG]

On Aug 8, 1:57 pm, walterbyrd <walterb...@iname.com> wrote:
> I like Python better than PHP, but I think there may be compelling
> advantages to PHP when it comes to frameworks. Issues that concerns me
> with Python, and Python frameworks, include: hosting, deployment,
> configuration and administrative overhead.
>
> I get the impression that Python framework developers just assume the
> user has complete control over the server. Those who develop Python
> frameworks seem to have little regard for the realities of shared
> hosting.

At this point, pretty much all languages have a bit more of a hurdle
than PHP on the server for various technical reasons. You can
eliminate some of the rough spots by going with hosting that is
familiar with what you're using, who often have systems to help smooth
things -- webfaction.com for python for example, and there are others
who specialize in Rails as well.

> From what I am gathering, Pylons requires a similar amount of ongoing
> configuration. I don't think this amount of configuration is required
> by Rails, or the PHP frameworks. Am I wrong about that? And is it
> really an issue?

Rails does, as does pretty much other language, including many PHP
frameworks. If you research hosting you should be able to find a place
with a decent setup, but if for financial or whatever reasons you need
to deploy on supercheapcrappyhosting.com you might want to stick with
ol' PHP.

[Garland, Ken R]

Just wanted to echo Echo's comments. I actually do use WebFaction and
they are great, highly recommend them.

I've only touched my config in the intial setup process, and with my
current WebFaction project I've actually never touched it since they
do that for you (of course you can change it later on if you wish).

[Ben Bangert]

On Aug 8, 2007, at 10:57 AM, walterbyrd wrote:

I like Python better than PHP, but I think there may be compelling

advantages to PHP when it comes to frameworks. Issues that concerns me

with Python, and Python frameworks, include: hosting, deployment,

configuration and administrative overhead.

Of course, no tool is perfect for every job.

I get the impression that Python framework developers just assume the

user has complete control over the server. Those who develop Python

frameworks seem to have little regard for the realities of shared

hosting. System requirements tend to be sky-high: Apache 2.x,

mod_python (latest version), fastcgi (at least), command line access,

PostgreSQL (recommended). Assuming you can meet the system

requirements, you still have server configuration issues such as

setting up the Path, and maybe the .htaccess file.

You could use Apache 1.x as well, or nginx, or lighttpd, or any webserver capable of proxying requests (assuming you want to run a standalone process for the Pylons app).

Then there is the ongoing configuration. I wonder if Pylons is any

better than Django in this regard, or maybe it's not as big an issue

it seems.

I think they both have similar options, Pylons has a few more since it uses less globals. Ie, you can run it stand-alone and its quite fast, then you merely proxy back to it. This uses less resources since a single threaded app can handle multiple requests at once, rather than loading up Apache+mod_python and having your app in every server process.

For example, as I understand it, in Django:

* The database model has to be manually synced to database from the

command line, anytime any change is made to the database.

Using SQLalchemy's table introspection, no changes would be needed to your models if the database is changed.

* A configuration file has to be created for every application.

Of course, since every application might have different requirements, different databases it talks to, etc. This is common in most every framework I've seen, including PHP frameworks (you do configure the db uri somewhere, no?).

* Urls have to be configured in the urls.py file to match the view.

Yep, in Pylons there's Routes which is a little easier than writing regular expressions. You also don't need to write a new one for every controller, as the default routing can handle all your controllers and views with no additional tweaking to a routing file.

* The webserver has to be restarted every time modify your code - or

touch every file if using fastcgi.

This applies to Pylons as well, unless you're updating templates. Template updates (in Mako, Myghty and Genshi at least) do not require server restarts.

From what I am gathering, Pylons requires a similar amount of ongoing

configuration. I don't think this amount of configuration is required

by Rails, or the PHP frameworks. Am I wrong about that? And is it

really an issue?

Rails involves all the same things, if not more since the recommended way to deploy Rails is to run a Mongrel cluster of processes that are then proxied back to (preferably with the proxy load balancer in Apache 2.2). You also have config files, and the same routing stuff as in Pylons.

There are quite a few cheap hosting solutions that do allow command access, and let you run your own process and/or use Fast CGI. So I don't think the hosting situation was as bleak for those not running their own servers as it was a year or two ago.

WebFaction comes with a control panel to manage your Pylons app:

http://www.webfaction.com/pylons-hosting

And ppl are running Pylons, Django, and Rails apps on Dreamhost and other hosting providers as well.

It is generally assumed that part of the appeal of these frameworks, is a more rigorous development cycle with testing. It's usually rather easy to run these frameworks stand-alone on your own computer to test and develop them, then when you're sure it works, you push it out to the server.

HTH,

Ben

[walterbyrd]

On Aug 8, 12:16 pm, Echo <oshe...@gmail.com> wrote:

> On 8/8/07, walterbyrd <walterb...@iname.com> wrote:
>
> If you use the paste server, it can automatically reload on any change.
>

Let's suppose I have a Pylons based site up and running, on shared
hosting, using Apache. If I want to make a change to the code, what do
I do?

Will Paste work on shared hosting? If I make changes using Paste, with
the Apache server pick up on those changes, or will the Apache server
have to be restarted? Or will I have to touch all my files, or some of
my files?

It seems to me that the PHP frameworks only require me to edit the
file and save the changes. Which seems much easier to me.

[Jose Galvez]

Apache will not have to re restarted, but you pylons application will have to be restarted.  Also if you're in a shared environment you might want to take a look at workingenv which simplifies using python in that environment
Jose

On 8/8/07, walterbyrd <walte...@iname.com> wrote:

[programmer.py]

On Aug 8, 2:35 pm, walterbyrd <walterb...@iname.com> wrote:
> On Aug 8, 12:16 pm, Echo <oshe...@gmail.com> wrote:
>
> > On 8/8/07, walterbyrd <walterb...@iname.com> wrote:
>
> > If you use the paste server, it can automatically reload on any change.
>
> Let's suppose I have a Pylons based site up and running, on shared
> hosting, using Apache. If I want to make a change to the code, what do
> I do?
>

If you have paster set to --reload, then it should just pick up your
changes. If you're proxying requests back-and-forth then apache has
no reason to know that your back-end server has changed. It's nice
and clean if you stop and think about it...

Request comes in
request --> apache --> paster

Response goes out
response-from-your-pyons-app --> apache --> client

I think --reload is nice for development. Heck, you could probably
get away with it on a light loaded server...

If you use mod_wsgi, like me, I'm not sure if you can get your changes
noticed without reloading apache. Well, it is WSGI, maybe you could
plug something into the WSGI stack, to look for changed modules and
reload them on the fly... But, this may really bork the innards if
you do it like this, I'm not sure.

> Will Paste work on shared hosting? If I make changes using Paste, with
> the Apache server pick up on those changes, or will the Apache server
> have to be restarted? Or will I have to touch all my files, or some of
> my files?
>

If you make changes to your files, and are using apache to proxy with
paster then it should automagically happen.

> It seems to me that the PHP frameworks only require me to edit the
> file and save the changes. Which seems much easier to me.

You have to realize that when your pylons app is up and running, it is
an actual running program. There is a python interpreter working
magic, mapping requests to controllers. When the interpreter cranks
up, it caches modules for performance reasons -- In a typical python
program there is no need to reload a module. But webapps aren't
really typical apps.

I don't know enough about PHP to comment on it. But I used to monkey
with it a long time ago and I did stay at a Holiday Inn, so I'll
recall my experiences. From what I recall, apache pretty much mapped
requests to the .php files. Once it noticed it was a .php file, it
sent it off to the php handler, which would *run* the php script, and
send the response back to the requesting client.

The difference in handling requests is obvious. PHP files are sent to
the `php handler` each time they are requested, automagically by
apache. This makes `seeing changes` very easy, because PHP knows
exactly which script is being executed.

For pylons, the request is simply forwarded from apache back to the
pylons back end. Pylons has to decide what to do with it and send the
response back. Usually `what to do` means routes finds the
appropriate controller based on the requested path.

Hope that helps some!
jw

[Graham Dumpleton]

On Aug 9, 12:24 pm, "programmer.py" <programmer...@gmail.com> wrote:
> If you use mod_wsgi, like me, I'm not sure if you can get your changes
> noticed without reloading apache.

If you are using 'daemon' mode of mod_wsgi you don't necessarily have
to restart the whole of Apache for your changes to be picked up. What
you need to do though depends a bit on your configuration.

If you are using a single daemon process with many threads for the
application, then you can embed a page in your application (password
protected hopefully), that sends a signal to itself. Ie.,

import signal
import os

os.kill(os.getpid(), signal.SIGINT)

This will cause the daemon process your application is in to shutdown.
The Apache process supervisor will then automatically restart your
process ready for subsequent requests. On the restart it will pick up
your new code. This way you can control a reload from your application
itself. You could feasibly even have a thread which goes around
looking to see if file timestamps have changed occasionally and
trigger a restart if they have.

You can also send a signal from an external application, but problem
there is identifying which process to send the signal. If you are
running the daemon process(es) as a distinct user/group to Apache and
each application is running as different user then you could just look
for the Apache (httpd) processes owned by you, as opposed to Apache
user, and send them all signals.

This latter approach is probably what shared web hosting sites would
be likely to provide if they pick up on mod_wsgi for Python hosting as
each users applications would run as the user and so easy to apply.
Thus they can trigger the restart from their control panel.

If you are using 'embedded' mode of mod_wsgi so as to squeeze the most
performance out of Apache, then you can't use the above approach. The
most reliable way of having code reloaded in this case is to restart
Apache.

That said, mod_wsgi does support an optional 'Interpreter' reload
mechanism that can be used in either 'daemon' mode or 'embedded' mode.
That is, when the main WSGI script file is changed/touched, then the
Python sub interpreter corresponding to the application will be
destroyed and a new Python sub interpreter created within the same
running process. The application will then be reloaded in the new
Python sub interpreter.

Although 'Interpreter' reloading is a good idea in theory, in practice
it will not always work. This isn't because of mod_wsgi or Python, but
because there are various third party Python C extension modules out
there that haven't been written properly so as to be able to cope with
Python sub interpreters being destroyed and recreated within the one
process. More details on this can be found at:

http://code.google.com/p/modwsgi/issues/detail?id=12

Another option being looked at for a future version of mod_wsgi is
'Process' reloading. This would only work with 'daemon' mode, but
would cause an automatic restart of a daemon process when the WSGI
script file has been changed. Getting this functionality working
correctly is non trivial though and would result in some additional
overheads.

FWIW. The OP has started up numerous discussions on various groups
complaining about poor Python web hosting, yet based on what they are
saying they don't want to pay anything but the absolute minimum they
can get away with. So, a specialised Python web hosting company
charges more than what they want to pay. Since true commodity web
hosting in their price range is tailored to PHP and wants to achieve
site densities that would preclude true long running processes, I
don't believe they will find an acceptable solution at this point. I
recently commented on some of these problems with using mod_wsgi for
commodity web hosting on my blog and some of the comments I make there
are probably relevant to the discussion.

http://blog.dscpl.com.au/2007/07/commodity-shared-hosting-and-modwsgi.html

Graham

[Marcin Kasperski]

A lot of answers already, so just two points:

a) There are plenty of cheap VPS hosting options (you get
your own virtual machine where you have root and can do whatever
you like). See for example http://linode.com/ (but maybe
some Xen-based provider can be better as it offers better
resource separation).

b) If you want to have someone administering your webserver and
just put the pages, then, well, you must use tools which
your provider handle. The question is then whether to choose
provider and then use his tools, or maybe choose the tools
and find the correct provider ;-)

IMO a) is the way to go unless someone is afraid of administering
the machine.

[walterbyrd]

On Aug 9, 4:35 am, Marcin Kasperski <Marcin.Kasper...@softax.com.pl>
wrote:

> A lot of answers already, so just two points:
>
> a) There are plenty of cheap VPS hosting options (you get

I guess it depends on what you call cheap. I can get php5 hosting for
$10/year (http://dollar-hosting.net/).

[Cliff Wells]

And I expect you get what you pay for. To be frank someone who's only
willing to pay $10/year for hosting probably isn't too picky and I'd
expect this lack of...um... "discrimination" to extend to the
development stack as well (if you pay this little, expect to use PHP).

It's a bit like pointing out that you can buy a new Hyundai for $6000
and then complaining *to Dodge* that their V10 Hemi engine doesn't fit
in it.

Regards,
Cliff

[iain duncan]

But how much do you charge for development? I have found the best way to
allay client concerns over hosting fees is to explain to them that the
additional hosting cost, including cost of migrating their domain from
their current budget host along with email, pales in comparison to the
cost saved in developer hours.

HTH
Iain