Python web development, really
Afanasiy
I have so far been confused by the options, or perhaps lack thereof,
given my strict requirements. I would appreciate suggestions.
I've looked at some Python options and have experience in quite a
few platforms previously. I have always disliked PHP as a language,
but at the same time have found it very accessible and useful.
Possible solutions should do 99% of the following :
* Run on the same hardware I use currently
* Be as fast or faster than Apache+PHP
(I'm sorry to say Zope/Roxen are not)
(This might mean it must use Apache)
* Use a similar amount of memory
(ie. not Java/.NET related)
* Provide a fairly equivalent framework w/ sessions, cookies, get, post
(Described in http://www.boddie.org.uk/python/web_frameworks.html)
(eg. I should not have to write code to parse query strings, etc.)
(PHP is somewhat unique in the ways it can bring in post variables)
* Allow all errors to be caught and handled so users never see them
(I currently send them to an email address)
(Users are notified an error occurred and a developer contacted)
* Allow sending of email
* Allow sending of raw binary data, for restricted file downloading
(eg. http://example.com/sendfile.php?id=A6DE12FAB3...etc)
(This requires header manipulation, specifically the mime type)
(That sort of thing should be part of the framework 3 bullets up)
* Allow SSL secured authorize.net credit card processing
(I currently use PHP's libcurl+ssl module)
* Allow similarly powerful regex capabilities
(I currently use PHP's pcre module)
* Big plus, but optional, auto prepend/append files
(eg. Apache+PHP has .htaccess directives like this )
( php_value auto_prepend_file "_header.php" )
( php_value auto_append_file "_footer.php" )
(granular down to the directory
* Finally, very optional plus, global application scoped variables
(PHP does NOT allow this, but I know why)
(I do not want to hear why you think this is bad)
(I know why it is good, I know why it can be bad)
Some of these things likely already exist in Python itself and thus
in anything which uses Python in a web development solution, but I am
mentioning them explicitly because they are indeed still requirements.
Plus explicit is better than implicit, as I'm sure everyone knows. ;-)
Some links for future and others' reference :
http://www.boddie.org.uk/python/web_frameworks.html
http://www.boddie.org.uk/python/web_modules.html
http://webware.sf.net/
http://modsnake.sf.net/ (dead? that's too bad)
http://www.apache.org/dist/httpd/modpython/
http://pmz.sourceforge.net/
http://www.mems-exchange.org/software/quixote/
http://skunkweb.sourceforge.net/
I hope I have not forgotten anything which will lead this thread to hell.
Jon Ribbens
> * be as fast or faster than apache+php
You want to use FastCGI (www.fastcgi.com) or mod_python
(www.modpython.org) then I should imagine.
> * provide a fairly equivalent framework w/ sessions, cookies, get, post
Personally I would recommend http://jonpy.sf.net/ , but then I would ;-)
This allows you to write your code easily using an object-oriented
system, and means that it will work unchanged if your application is
being called via CGI or FastCGI or mod_python. There is also a nice
templating system if you want to use it.
jonpy together with the facilities already provided by a normal Python
install provides everything on your list, I think. You already found
Paul Boddie's list of other frameworks, so you know the alternatives
too ;-)
Max M
> * Run on the same hardware I use currently
>
> * Be as fast or faster than Apache+PHP
> (I'm sorry to say Zope/Roxen are not)
> (This might mean it must use Apache)
>
> * Use a similar amount of memory
> (ie. not Java/.NET related)
So you want the advantages of Python, but none of the trade-off's. Well
wouldn't we all?
You would have to write test for your app to test this. But they should
be roughly in the same ballpark.
>
> * Provide a fairly equivalent framework w/ sessions, cookies, get, post
> (Described in http://www.boddie.org.uk/python/web_frameworks.html)
> (eg. I should not have to write code to parse query strings, etc.)
> (PHP is somewhat unique in the ways it can bring in post variables)
I don't know about sessions, but my guess is that there is something out
there. cookies, get and post's can be handled nicely by the cgi module.
> * Allow all errors to be caught and handled so users never see them
> (I currently send them to an email address)
> (Users are notified an error occurred and a developer contacted)
try/except works like a charm for this. Use it myself.
> * Allow sending of email
smtplib is your friend. Not quite as friendly as some email packages.
But together with the email module it is very powerfull
> * Allow sending of raw binary data, for restricted file downloading
> (eg. http://example.com/sendfile.php?id=A6DE12FAB3...etc)
> (This requires header manipulation, specifically the mime type)
> (That sort of thing should be part of the framework 3 bullets up)
It is not quite clear what you mean here. But if you want to send a file
from your server to another webserver/form, you just use urllib/urllib2.
If you want to receive files on your server it is built into the cgi module.
> * Allow SSL secured authorize.net credit card processing
> (I currently use PHP's libcurl+ssl module)
Doesn't Apache do this?
> * Allow similarly powerful regex capabilities
> (I currently use PHP's pcre module)
Pythons regexp's are probably as powerfull as any. Se the re module.
> * Big plus, but optional, auto prepend/append files
> (eg. Apache+PHP has .htaccess directives like this )
> ( php_value auto_prepend_file "_header.php" )
> ( php_value auto_append_file "_footer.php" )
> (granular down to the directory
hmmm! Don't know about this one. Python isn't embedded like that out of
the box. Should be pretty easy to write a function wich does something
like it though.
> * Finally, very optional plus, global application scoped variables
> (PHP does NOT allow this, but I know why)
> (I do not want to hear why you think this is bad)
> (I know why it is good, I know why it can be bad)
Use the Borg pattern:
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/66531
regards Max M
--
hilsen/regards Max M
http://www.futureport.dk/
Fremtiden, videnskab, skeptiscisme og transhumanisme
Peter Hansen
>
> * Finally, very optional plus, global application scoped variables
> (PHP does NOT allow this, but I know why)
> (I do not want to hear why you think this is bad)
> (I know why it is good, I know why it can be bad)
If you really understand the pluses and minuses of this, then
you'll be quite happy with what Python can provide. A simple
module which collects those "global" variables will provide
all the benefits and fewer of the disadvantages.
Of course, there are often better ways to.... oh, wait. Sorry. ;-)
-Peter
Cameron Laird
Afanasiy <abeli...@hotmail.com> wrote:
> .
> .
> .
>Possible solutions should do 99% of the following :
>
>* Run on the same hardware I use currently
>
>* Be as fast or faster than Apache+PHP
> (I'm sorry to say Zope/Roxen are not)
> (This might mean it must use Apache)
> .
> .
? I'm having parse difficulties. Are you saying you used
Zope through Roxen CGI, and found the performance of *that*
combination inadequate? I guess that's not much of a surprise
... Roxen's a speed demon if you use its native scripting,
and there are also faster Zope architectures than CGI.
--
Cameron Laird <Cam...@Lairds.com>
Business: http://www.Phaseit.net
Personal: http://phaseit.net/claird/home.html
ko...@aesaeion.com
> In article <j9bo2v0lo0qfngcfp...@4ax.com>,
> Afanasiy <abeli...@hotmail.com> wrote:
> > .
> > .
> ? I'm having parse difficulties. Are you saying you used
> Zope through Roxen CGI, and found the performance of *that*
> combination inadequate? I guess that's not much of a surprise
> ... Roxen's a speed demon if you use its native scripting,
> and there are also faster Zope architectures than CGI.
> --
Agree I find that CGI with zope is a HUGE performance hit. 50% or more in
most cases since it doesn't handle multiple connections at the same time
very well anymore. Need to use something like mod_proxy with apache or
mod_rewrite in proxy mode if you want it to run fast that way.
Afanasiy
wrote:
>In article <j9bo2v0lo0qfngcfp...@4ax.com>,
>Afanasiy <abeli...@hotmail.com> wrote:
>> .
>> .
>> .
>>Possible solutions should do 99% of the following :
>>
>>* Run on the same hardware I use currently
>>
>>* Be as fast or faster than Apache+PHP
>> (I'm sorry to say Zope/Roxen are not)
>> (This might mean it must use Apache)
>> .
>> .
>> .
>? I'm having parse difficulties. Are you saying you used
>Zope through Roxen CGI, and found the performance of *that*
>combination inadequate? I guess that's not much of a surprise
>... Roxen's a speed demon if you use its native scripting,
>and there are also faster Zope architectures than CGI.
No that's not what I am saying. I have grouped Roxen and Zope together
as similar products that also happen to be similarly too slow for me.
Afanasiy
>Afanasiy wrote:
>
>> * Run on the same hardware I use currently
>>
>> * Be as fast or faster than Apache+PHP
>> (I'm sorry to say Zope/Roxen are not)
>> (This might mean it must use Apache)
>>
>> * Use a similar amount of memory
>> (ie. not Java/.NET related)
>
>
>So you want the advantages of Python, but none of the trade-off's. Well
>wouldn't we all?
I was not aware Python required trade-offs involving memory or cpu use in
order to compete with PHP. In my experience Python is much much faster for
basic scripting. I hoped that speed could be put to use on a web platform.
>> * Allow all errors to be caught and handled so users never see them
>> (I currently send them to an email address)
>> (Users are notified an error occurred and a developer contacted)
>
>
>try/except works like a charm for this. Use it myself.
That works, yes. A wider scope error handler is preferred, but this
this can be irrelevant if an auto prepend/append file is allowed.
>> * Allow sending of raw binary data, for restricted file downloading
>> (eg. http://example.com/sendfile.php?id=A6DE12FAB3...etc)
>> (This requires header manipulation, specifically the mime type)
>> (That sort of thing should be part of the framework 3 bullets up)
>
>
>It is not quite clear what you mean here. But if you want to send a file
>from your server to another webserver/form, you just use urllib/urllib2.
>If you want to receive files on your server it is built into the cgi module.
To clarify, this requirement means I need to be able to completely replace
the request result such that it is in effect returning a binary file for
download. This is often used for secure downloading of protected files.
This requirement implies some method for customizing request result
headers, specifically Content-type/disposition/transfer-encoding/etc.
A good platform will simply allow customization of all headers.
(Header modification, as mentioned, is part of the framework requirement)
Additionally, some platforms might have a problem with including the
binary file as the actual result of the request (after the headers).
>> * Allow SSL secured authorize.net credit card processing
>> (I currently use PHP's libcurl+ssl module)
>
>
>Doesn't Apache do this?
No, this requirement is not about running an SSL HTTP server. This
requirement is about *programmatically* sending data to a secured remote
server via POST and receiving a response. This is how authorize.net works.
This requirement does imply an SSL capable HTTP server in order to
securely obtain said data from the user (only if you do that though).
All of these requirements imply an HTTP server as well... :p
> [plenty of things snipped]
I am looking for suggestions on a platform which might provide all of
these things. A platform, using Python, which can compete well with PHP.
Paul Rubin
> I was not aware Python required trade-offs involving memory or cpu use in
> order to compete with PHP. In my experience Python is much much faster for
> basic scripting. I hoped that speed could be put to use on a web platform.
Benchmarks I've seen indicate PHP is considerably faster than Python.
That's not too surprising, since PHP is a simpler language and doesn't
have to do nearly so many dynamic lookups.
> I am looking for suggestions on a platform which might provide all of
> these things. A platform, using Python, which can compete well with PHP.
I suppose that's true, but PHP does a lot for you "out of the box"
that Python doesn't do unless you write or locate suitable add-on
modules, not only for HTML template expansion but also for things like
database access. A lot of such modules have of course already been
written and are floating around various places.
I would say Python is a far superior programming language to PHP and
is preferable for complicated software development. PHP, though, is
better suited for getting a basic site going with minimal fuss. I
know of some people who use PHP to implement the HTML front end of a
web application, and Python to do the backend logic. That seems
like a good approach, if you don't mind developing in two languages.
Afanasiy
wrote:
>Afanasiy <abeli...@hotmail.com> writes:
>> I was not aware Python required trade-offs involving memory or cpu use in
>> order to compete with PHP. In my experience Python is much much faster for
>> basic scripting. I hoped that speed could be put to use on a web platform.
>
>Benchmarks I've seen indicate PHP is considerably faster than Python.
>That's not too surprising, since PHP is a simpler language and doesn't
>have to do nearly so many dynamic lookups.
Benchmarks I've done indicate very much the opposite. Perhaps you saw
something for the zend optimizer, which I have not tested myself.
Anyway, it does look like PHP is in fact my best option overall.
Richard Jones
> I would love to use Python instead of PHP for web development, but
> I have so far been confused by the options, or perhaps lack thereof,
> given my strict requirements. I would appreciate suggestions.
>
> I've looked at some Python options and have experience in quite a
> few platforms previously. I have always disliked PHP as a language,
> but at the same time have found it very accessible and useful.
>
> Possible solutions should do 99% of the following :
>
> * Run on the same hardware I use currently
>
> * Be as fast or faster than Apache+PHP
> (I'm sorry to say Zope/Roxen are not)
> (This might mean it must use Apache)
Not much is going to be faster than Apache+PHP. Python can certainly reach the
same or similar speeds though.
> * Use a similar amount of memory
> (ie. not Java/.NET related)
This is pretty open-ended. Suffice to say though that Python doesn't consume
wads of memory on its own. The rest is up to your application.
> * Provide a fairly equivalent framework w/ sessions, cookies, get, post
> (Described in http://www.boddie.org.uk/python/web_frameworks.html)
> (eg. I should not have to write code to parse query strings, etc.)
> (PHP is somewhat unique in the ways it can bring in post variables)
I'm confused. A number of the frameworks on the page you link there support
all of the above. Are you stating that this is a requirement that isn't met
(which it clearly is) or that it _is_ met?
> * Allow all errors to be caught and handled so users never see them
> (I currently send them to an email address)
> (Users are notified an error occurred and a developer contacted)
Built into Python. try/except is part of the language itself.
> * Allow sending of email
Built into Python. Use the smtplib module.
> * Allow sending of raw binary data, for restricted file downloading
> (eg. http://example.com/sendfile.php?id=A6DE12FAB3...etc)
> (This requires header manipulation, specifically the mime type)
> (That sort of thing should be part of the framework 3 bullets up)
Any of the frameworks you reference above can do this. Python can do this by
itself with no framework:
print "Content-Type: image/png"
print
print open('/path/to/image.png').read()
> * Allow SSL secured authorize.net credit card processing
> (I currently use PHP's libcurl+ssl module)
I presume by the reference to libcurl you want to make a request of an SSL
website. This is built into python. Use the urllib2 module.
> * Allow similarly powerful regex capabilities
> (I currently use PHP's pcre module)
Built into Python. Use the re module.
> * Big plus, but optional, auto prepend/append files
> (eg. Apache+PHP has .htaccess directives like this )
> ( php_value auto_prepend_file "_header.php" )
> ( php_value auto_append_file "_footer.php" )
> (granular down to the directory
That would be dependent on the framework you chose, but they would all support
some version of this.
> * Finally, very optional plus, global application scoped variables
> (PHP does NOT allow this, but I know why)
> (I do not want to hear why you think this is bad)
> (I know why it is good, I know why it can be bad)
Most of the frameworks will support this in some manner. If they don't, then
using the stand-alone ZODB would be a trivial addition to your application.
Or you could use a relational database. It's up to you.
Richard
Afanasiy
>On Tue, 21 Jan 2003 4:16 am, Afanasiy wrote:
>> * Be as fast or faster than Apache+PHP
>
>Not much is going to be faster than Apache+PHP. Python can certainly reach the
>same or similar speeds though.
In my commandline scripting tests, Python always beats PHP...
What would stop a new Apache+Python integration project from being faster?
>> * Use a similar amount of memory
>> (ie. not Java/.NET related)
>
>This is pretty open-ended. Suffice to say though that Python doesn't consume
>wads of memory on its own. The rest is up to your application.
I primarily meant to exclude suggestions which would involve
anything using noticeably large amounts of memory, such as Java.
>> * Provide a fairly equivalent framework w/ sessions, cookies, get, post
>> (Described in http://www.boddie.org.uk/python/web_frameworks.html)
>> (eg. I should not have to write code to parse query strings, etc.)
>> (PHP is somewhat unique in the ways it can bring in post variables)
>
>I'm confused. A number of the frameworks on the page you link there support
>all of the above. Are you stating that this is a requirement that isn't met
>(which it clearly is) or that it _is_ met?
It is explicitly documented as one of many requirements. This is perhaps
the most basic requirement, but there are certainly solutions which do not
implement the basics of a web framework. However, yes, many do.
>> * Big plus, but optional, auto prepend/append files
>> (eg. Apache+PHP has .htaccess directives like this )
>> ( php_value auto_prepend_file "_header.php" )
>> ( php_value auto_append_file "_footer.php" )
>> (granular down to the directory
>
>That would be dependent on the framework you chose, but they would all support
>some version of this.
Not from what I have seen. I even imagine implicit vs. explicit would
cause many Python programmers to love writing cascading header/footer
includes and custom error handling on every new page, even if via paste.
>> * Finally, very optional plus, global application scoped variables
>> (PHP does NOT allow this, but I know why)
>> (I do not want to hear why you think this is bad)
>> (I know why it is good, I know why it can be bad)
>
>Most of the frameworks will support this in some manner. If they don't, then
I don't think they do, and you sound unsure.
Which platform do you recommend I test for all of these requirements?
Many tested so far fail relatively easily. I am looking for concrete
suggestions from people experienced in this sort of web development.
Jack Diederich
PHP is slow if you are doing an application engine. It must load all the
source for each hit. Writing a large app in PHP would also be ... hard.
It just isn't a first class programming language.
PHP is quick if you are doing a dynamic site without much application logic.
Alot more people know it, so it is easy to find or buy help.
It is fairly simple, so it is easy to bang pages out quickly.
I work on a largish web-based business app written in python.
But PHP was used to prototype it.
hope that helps,
-jackdied
Remi Delon
> I would love to use Python instead of PHP for web development, but
> I have so far been confused by the options, or perhaps lack thereof,
> given my strict requirements. I would appreciate suggestions.
>
> I've looked at some Python options and have experience in quite a
> few platforms previously. I have always disliked PHP as a language,
> but at the same time have found it very accessible and useful.
>
> Possible solutions should do 99% of the following :
>
> * Run on the same hardware I use currently
It is 100% pure python so if python runs on your hardware, CherryPy will
too.
> * Be as fast or faster than Apache+PHP
> (I'm sorry to say Zope/Roxen are not)
> (This might mean it must use Apache)
>
> * Use a similar amount of memory
> (ie. not Java/.NET related)
CherryPy uses a "compile source files and generate a single executable for
your website" approach, so it doesn't get much faster than that. And the
executable is very lightweight (only a few MB for a simple website)
>
> * Provide a fairly equivalent framework w/ sessions, cookies, get, post
> (Described in http://www.boddie.org.uk/python/web_frameworks.html)
> (eg. I should not have to write code to parse query strings, etc.)
> (PHP is somewhat unique in the ways it can bring in post variables)
CherryPy is also somewhat unique in the way it handles get and post
variables :-) Each page request is converted into a regular function call
and get and post variables become the arguments of the call:
http://localhost/index?var1=val1&var2=val2
is converted to:
index(var1="val1", var2=val2)
so you'll just have to write a function like this:
def index(var1, var2):
And it doesn't matter if var1 and var2 are sent via a get or a post or if
they're a short string or a large file that's being uploaded: they're
handled the same way.
Cookies are also automatically parsed and put in a nice python map for you.
Sessions are not implemented yet.
> * Allow all errors to be caught and handled so users never see them
> (I currently send them to an email address)
> (Users are notified an error occurred and a developer contacted)
That's how I set up my websites too :-) It's a trivial thing to do with
CherryPy. Chapter 14.3.4 of the tutorial shows how to do this.
> * Allow sending of raw binary data, for restricted file downloading
> (eg. http://example.com/sendfile.php?id=A6DE12FAB3...etc)
> (This requires header manipulation, specifically the mime type)
> (That sort of thing should be part of the framework 3 bullets up)
I'm not sure what you mean by that but CherryPy allows you to work on the
response header and body before it is sent back to the browser. So what you
want is probably quite easy to do with CherryPy.
> * Allow SSL secured authorize.net credit card processing
> (I currently use PHP's libcurl+ssl module)
Yep, it supports SSL.
> * Allow similarly powerful regex capabilities
> (I currently use PHP's pcre module)
Well, you have access to all python modules from your code, so just pick the
one you want.
> * Big plus, but optional, auto prepend/append files
> (eg. Apache+PHP has .htaccess directives like this )
> ( php_value auto_prepend_file "_header.php" )
> ( php_value auto_append_file "_footer.php" )
> (granular down to the directory
You can do that with CherryPy AOP feature. Check out this HowTo:
http://www.cherrypy.org/static/html/howto/node12.html
> * Finally, very optional plus, global application scoped variables
> (PHP does NOT allow this, but I know why)
> (I do not want to hear why you think this is bad)
> (I know why it is good, I know why it can be bad)
Yep, you can use global variables.
> Some of these things likely already exist in Python itself and thus
> in anything which uses Python in a web development solution, but I am
> mentioning them explicitly because they are indeed still requirements.
> Plus explicit is better than implicit, as I'm sure everyone knows. ;-)
>
> Some links for future and others' reference :
> http://www.boddie.org.uk/python/web_frameworks.html
> http://www.boddie.org.uk/python/web_modules.html
> http://modsnake.sf.net/ (dead? that's too bad)
> http://www.apache.org/dist/httpd/modpython/
> http://www.mems-exchange.org/software/quixote/
> http://skunkweb.sourceforge.net/
>
> I hope I have not forgotten anything which will lead this thread to hell.
Well, you've forgotten CherryPy :-)
I suggest you take a look at http://www.cherrypy.org, check out the small
online demo and take a quick look at the documentation (HowTo and tutorial).
If it doesn't meet your requirements, please tell me what is missing/you
don't like so I can improve it :-)
Regards.
Remi.
Fredrik Steen
> I would love to use Python instead of PHP for web development, but
> I have so far been confused by the options, or perhaps lack thereof,
> given my strict requirements. I would appreciate suggestions.
Have a look at draco.
It needs apache with mod_python: http://draco.boskant.nl/
./fs
Paul Boddie
> I have so far been confused by the options, or perhaps lack thereof,
> given my strict requirements. I would appreciate suggestions.
First of all, take a look at the PythonInfo Wiki:
http://www.python.org/cgi-bin/moinmoin/WebProgramming
This site is being updated and covers a lot more than the Web
Frameworks Overview. Certainly, the aim is to be able to help people
in situations like yours.
[...]
> Possible solutions should do 99% of the following :
>
> * Run on the same hardware I use currently
>
> * Be as fast or faster than Apache+PHP
> (I'm sorry to say Zope/Roxen are not)
> (This might mean it must use Apache)
Neither of these should be an issue.
> * Use a similar amount of memory
> (ie. not Java/.NET related)
Yes, we don't want the lights to dim when that application server
starts up. ;-)
> * Provide a fairly equivalent framework w/ sessions, cookies, get, post
> (Described in http://www.boddie.org.uk/python/web_frameworks.html)
> (eg. I should not have to write code to parse query strings, etc.)
> (PHP is somewhat unique in the ways it can bring in post variables)
I'm not familiar with PHP, but with most Python frameworks you don't
need to parse the raw HTTP inputs, and with many frameworks you have
convenient interfaces to those inputs. Webware provides a servlet-like
interface, although it's much more "sane" than the Java Servlet API
(in case you were wondering).
As for the "unique" aspects of PHP's variable handling, are you
referring to something like this...?
http://www.php.net/manual/en/security.registerglobals.php
> * Allow all errors to be caught and handled so users never see them
> (I currently send them to an email address)
> (Users are notified an error occurred and a developer contacted)
I've been thinking rather seriously about better solutions than "we'll
just log the error and send an e-mail"; it seems to me that most
people don't have enough time to sift through lengthy log files, and
when e-mails arrive I'm sure most developers just want to delete them.
> * Allow sending of email
Python provides a number of ways of doing this, although an integrated
approach to application communications would be a lot more attractive
than firing up sendmail. I'm not sure of the state of Python
frameworks in this respect, although I suspect that Twisted integrates
the different communications "channels" better than most.
> * Allow sending of raw binary data, for restricted file downloading
> (eg. http://example.com/sendfile.php?id=A6DE12FAB3...etc)
> (This requires header manipulation, specifically the mime type)
> (That sort of thing should be part of the framework 3 bullets up)
You mean full control over headers and the ability to bypass the
templating system?
> * Allow SSL secured authorize.net credit card processing
> (I currently use PHP's libcurl+ssl module)
This is presumably where your application connects out to another
application - some kind of back-end integration. This should be
possible, yes, but I'm not aware of any components for it that just
plug into a Python framework or application server, although there
could be a product available for Zope. Meanwhile, various Webware
people may have experience with this kind of thing even though the
Webware framework doesn't have a "checkbox feature" entitled "credit
card processing".
> * Allow similarly powerful regex capabilities
> (I currently use PHP's pcre module)
Python's re module should suffice.
> * Big plus, but optional, auto prepend/append files
> (eg. Apache+PHP has .htaccess directives like this )
> ( php_value auto_prepend_file "_header.php" )
> ( php_value auto_append_file "_footer.php" )
> (granular down to the directory
That's an Apache thing, surely, so you might want to check to see
whether various frameworks and templating systems are open to such
external content manipulation - ie. that they generate page fragments
rather than full pages. I would guess that most systems are.
> * Finally, very optional plus, global application scoped variables
> (PHP does NOT allow this, but I know why)
> (I do not want to hear why you think this is bad)
> (I know why it is good, I know why it can be bad)
If you're not talking about the various exploitable PHP features, but
are actually referring to "shared data" or "singletons" which are
available to all requests running within a given application, there
are various means of achieving this in the different frameworks. It
would surprise me if the Webware Wiki didn't cover this topic, for
example.
Paul
Tiberius Teng
Check it out. :)
Afanasiy
>> * Provide a fairly equivalent framework w/ sessions, cookies, get, post
>> (Described in http://www.boddie.org.uk/python/web_frameworks.html)
>> (eg. I should not have to write code to parse query strings, etc.)
>> (PHP is somewhat unique in the ways it can bring in post variables)
>
>I'm not familiar with PHP, but with most Python frameworks you don't
>need to parse the raw HTTP inputs, and with many frameworks you have
>convenient interfaces to those inputs. Webware provides a servlet-like
>interface, although it's much more "sane" than the Java Servlet API
>(in case you were wondering).
>
>As for the "unique" aspects of PHP's variable handling, are you
>referring to something like this...?
>
> http://www.php.net/manual/en/security.registerglobals.php
I am referring to things like PHP automatically splitting multi-valued
form fields (checkboxes, select/option) into arrays for you if you just
name them, in the html, with [] appended. Just a thoughtful addition.
Or perhaps Python developers consider it another horrible implicit.
>> * Allow sending of raw binary data, for restricted file downloading
>> (eg. http://example.com/sendfile.php?id=A6DE12FAB3...etc)
>> (This requires header manipulation, specifically the mime type)
>> (That sort of thing should be part of the framework 3 bullets up)
>
>You mean full control over headers and the ability to bypass the
>templating system?
To clarify, this requirement means I need to be able to explicitly replace
the request result such that it is in effect returning a binary file for
download. This is often used for secure downloading of protected files.
This requirement implies some method for customizing request result
headers, specifically Content-type/disposition/transfer-encoding/etc.
A good platform will simply allow customization of all headers.
(Header modification, as mentioned, is part of the framework requirement)
Additionally, some platforms might have a problem with including the
binary file as the actual result of the request (after the headers).
>> * Allow SSL secured authorize.net credit card processing
>> (I currently use PHP's libcurl+ssl module)
>
>This is presumably where your application connects out to another
>application - some kind of back-end integration. This should be
>possible, yes, but I'm not aware of any components for it that just
>plug into a Python framework or application server, although there
>could be a product available for Zope. Meanwhile, various Webware
>people may have experience with this kind of thing even though the
>Webware framework doesn't have a "checkbox feature" entitled "credit
>card processing".
You need to be able to automate a post from https to https (with the
ability to fake referer) and parse the result of your request.
I've done half of this with Python scripts already for an entirely
different end-effect, but never using SSL and never faking HTTP_REFERER.
>> * Big plus, but optional, auto prepend/append files
>> (eg. Apache+PHP has .htaccess directives like this )
>> ( php_value auto_prepend_file "_header.php" )
>> ( php_value auto_append_file "_footer.php" )
>> (granular down to the directory
>
>That's an Apache thing, surely, so you might want to check to see
>whether various frameworks and templating systems are open to such
>external content manipulation - ie. that they generate page fragments
>rather than full pages. I would guess that most systems are.
It's a mod_php thing. PHP, and mod_python, are optionally configurable
in Apache's rather flexible configuration system; allowing per directory
granularity if desired, and cascading/inheritance to subdirectories.
( Of course, there are a couple other ways to configure,
and security for the different methods and directives )
Anyway, the ability to implicitly include was the requirement.
I gave an example of how it is done in PHP for reference.
Thanks for the info, I will check out the Wiki.
So far, additional platforms I have found are strangely non-elegant.
A.M. Kuchling
> In my commandline scripting tests, Python always beats PHP...
> What would stop a new Apache+Python integration project from being faster?
Perhaps Richard knows the Zend optimizer for PHP is faster, or assumed
it would be. If you've measured Python at being faster, though,
that would settle the performance question for you.
>>>* Big plus, but optional, auto prepend/append files
>>> (eg. Apache+PHP has .htaccess directives like this )
>>> ( php_value auto_prepend_file "_header.php" )
>>> ( php_value auto_append_file "_footer.php" )
>>> (granular down to the directory
> Not from what I have seen. I even imagine implicit vs. explicit would
> cause many Python programmers to love writing cascading header/footer
> includes and custom error handling on every new page, even if via paste.
Indeed, I'd imagine support for this to be rare. Doing it explicitly
isn't a huge amount of work, though, and I'd hope it can be reduced
to a line or two. For example, once you've written header() and
footer() functions, pages look like this using Quixote's PTL:
from <wherever> import standard
def simple [html] ():
standard.header('Login')
'<h1>Welcome!</h1>'
... blah blah ...
standard.footer()
There are lots of different ways to tweak the HEAD element, though,
and to present additional things, so header() can get quite complicated:
def fancy [html] ():
standard.header(title='Application Login',
author='webm...@amk.ca',
description="Login page for a nifty application",
keywords="login, log-in, application, app,webapp..."
# titles for breadcrumb links
crumbs = ['Home', 'Application'],
)
It would be hard to make things this flexible with an automatic headera
and footer system, and in the common case (just supplying a title)
the overhead is low, so I find this an acceptable trade-off.
A toolkit certainly shouldn't require cut-and-paste programming, though,
but should support factoring out the common code into
sub-functions/methods/pages/<metaphor>s.
> Which platform do you recommend I test for all of these requirements?
> Many tested so far fail relatively easily. I am looking for concrete
> suggestions from people experienced in this sort of web development.
I think the problem with offering a recommendation is that you haven't
said in what respect they fail. Too low-level? Too many assumptions
that don't fit your situation?
There are so many different design approaches and requirements for web
applications that it's unlikely one toolkit could ever work for every
one, explaining why there are so many different ones out there.
--amk
Afanasiy
>Afanasiy wrote:
>> In my commandline scripting tests, Python always beats PHP...
>> What would stop a new Apache+Python integration project from being faster?
>
>Perhaps Richard knows the Zend optimizer for PHP is faster, or assumed
>it would be. If you've measured Python at being faster, though,
>that would settle the performance question for you.
That settles the performance question for standalone scripts.
This requirement targets the speed of the entire web solution.
As such, Zope fails the requirement even though it uses Python.
As far as whether or not he meant Zend, explicit clarification
is needed to be sure. I myself would not assume Zend is faster.
>>>>* Big plus, but optional, auto prepend/append files
>>>> (eg. Apache+PHP has .htaccess directives like this )
>>>> ( php_value auto_prepend_file "_header.php" )
>>>> ( php_value auto_append_file "_footer.php" )
>>>> (granular down to the directory
>>
>> Not from what I have seen. I even imagine implicit vs. explicit would
>> cause many Python programmers to love writing cascading header/footer
>> includes and custom error handling on every new page, even if via paste.
>
>There are lots of different ways to tweak the HEAD element, though,
>and to present additional things, so header() can get quite complicated:
>
>It would be hard to make things this flexible with an automatic headera
>and footer system, and in the common case (just supplying a title)
>the overhead is low, so I find this an acceptable trade-off.
This requirement has little to do with programmatic creation of <head>
>> Which platform do you recommend I test for all of these requirements?
>> Many tested so far fail relatively easily. I am looking for concrete
>> suggestions from people experienced in this sort of web development.
>
>I think the problem with offering a recommendation is that you haven't
>said in what respect they fail. Too low-level? Too many assumptions
>that don't fit your situation?
None of those researched meet enough of the documented requirements.
Paul Boddie
>
> I am referring to things like PHP automatically splitting multi-valued
> form fields (checkboxes, select/option) into arrays for you if you just
> name them, in the html, with [] appended. Just a thoughtful addition.
>
> Or perhaps Python developers consider it another horrible implicit.
Horribly flawed, more like. I'd rather my application know (through
various means, not necessarily hard-coded) what data type a particular
piece of information is supposed to be, rather than such information
arriving in an HTTP request. Certainly, the cgi module supports
multi-valued fields, and I do agree that frameworks could provide
better support for processing of field data. Indeed, I've done various
experiments aimed at this neglected side of Web programming.
Paul
Paul Boddie
>
[Per-directory content inclusion]
> Not from what I have seen. I even imagine implicit vs. explicit would
> cause many Python programmers to love writing cascading header/footer
> includes and custom error handling on every new page, even if via paste.
You're really milking this "implicit vs. explicit" thing, but
actually, even those frameworks which do this using mix-ins and
methods, for example, are avoiding copy/paste by encouraging such
approaches. Personally, I can see why you might want to do this stuff
"declaratively" - ie. putting it in an easy-to-understand
configuration file - and I do wonder whether Apache might still
support it, even if mod_python might not.
[Frameworks supporting "global application scoped variables"]
> I don't think they do, and you sound unsure.
Well, consider Webware. Since the application server is just a Python
program with a bunch of threads, you can easily imagine ways that
shared application state can be defined. Unfortunately, I can't easily
find any hints about that on the Webware Wiki, but since Webware is
meant to be Pythonic, any Pythonic approach, whether it be the
inclusion of a module to hold your "globals" or something else, is
likely to be a reasonable solution.
If you want a full persistence mechanism, then all you need to do is
to consult the WebProgramming section of the PythonInfo Wiki and check
out which frameworks offer persistence.
> Which platform do you recommend I test for all of these requirements?
> Many tested so far fail relatively easily. I am looking for concrete
> suggestions from people experienced in this sort of web development.
Perhaps the WebProgramming section needs a bit of restructuring if it
isn't that helpful. However, I'd suggest...
* Zope for a highly specified and flexible framework... that you've
already stated that you don't like. That's fine, though, because
I'd certainly be a bit nervous about deploying applications with
Zope without having a more complete understanding of all the
issues involved.
* Webware and Twisted for Pythonic frameworks which might leave you
with some additional higher-level framework effort. In other
words, the framework just stops at some point and then you have
to make some choices.
* SkunkWeb, Albatross and Quixote for Pythonic frameworks which
seem "battle-tested" but which offer integration with various
technological principles that you might not agree with (Quixote,
for example, promoting its own presentation strategy, for
example).
* Spyce and CherryPy for PHP-like frameworks.
I'm not saying that I've developed for all of these; indeed, I've only
ever really used Zope and Webware from the above. However, I've
investigated them and made notes based on my own experiences with Web
development. If you want a simple, single line "executive-level
recommendation", I'd suggest looking closely at CherryPy, particularly
since it seems to be actively developed and may be compatible with
what you have come to expect.
Paul
Afanasiy
Well they're all strings. Putting multi-valued fields into an array
at your discretion (by naming the form variable with [] appended)
is, in my opinion, useful. I guess if you routinely name form fields
with [] appended for some reason, this could indeed be a problem.
Ian Bicking
questions with relation to Webware:
On Mon, 2003-01-20 at 11:16, Afanasiy wrote:
> * Run on the same hardware I use currently
Yes, so long as you can run long-running processes. Most commercial
shared-hosting situations don't allow this. This applies to all Python
frameworks except when accessed strictly through CGI, which will give
you atrocious performance.
> * Be as fast or faster than Apache+PHP
> (I'm sorry to say Zope/Roxen are not)
> (This might mean it must use Apache)
Probably comparable speed. It's definitely faster than Zope. We
haven't encountered people who have felt limited by the speed of Webware
(as compared to bandwidth limitations, database, etc). But that might
just be because people aren't using in situations where it would be a
problem.
Webware also makes possible some significant caching, though you have to
do it in an ad hoc way. You'd do this through global data, which is
noted later.
> * Use a similar amount of memory
> (ie. not Java/.NET related)
Webware is fairly small, it should be fine.
> * Provide a fairly equivalent framework w/ sessions, cookies, get, post
> (Described in http://www.boddie.org.uk/python/web_frameworks.html)
> (eg. I should not have to write code to parse query strings, etc.)
> (PHP is somewhat unique in the ways it can bring in post variables)
Yep, they all do that. Of course it doesn't register variables as
globals, but that's a feature :)
> * Allow all errors to be caught and handled so users never see them
> (I currently send them to an email address)
> (Users are notified an error occurred and a developer contacted)
Yep.
> * Allow sending of email
Through smtplib. We don't duplicate anything that Python does fine on
its own.
> * Allow sending of raw binary data, for restricted file downloading
> (eg. http://example.com/sendfile.php?id=A6DE12FAB3...etc)
> (This requires header manipulation, specifically the mime type)
> (That sort of thing should be part of the framework 3 bullets up)
Certainly.
> * Allow SSL secured authorize.net credit card processing
> (I currently use PHP's libcurl+ssl module)
You'd do this using urllib, which is part of Python.
> * Allow similarly powerful regex capabilities
> (I currently use PHP's pcre module)
Again, using Python's re module (IMHO much easier to use than in PHP).
> * Big plus, but optional, auto prepend/append files
> (eg. Apache+PHP has .htaccess directives like this )
> ( php_value auto_prepend_file "_header.php" )
> ( php_value auto_append_file "_footer.php" )
> (granular down to the directory
I believe there's ways to do this with Apache while using Webware. But
you can (in effect) do this in Webware fairly easily. Each page of a
Webware application is a separate class. Generally there will be a
superclass that you write that is specific to your application. In it
you can put a header and footer, and those can be as dynamic as you
choose.
> * Finally, very optional plus, global application scoped variables
> (PHP does NOT allow this, but I know why)
> (I do not want to hear why you think this is bad)
> (I know why it is good, I know why it can be bad)
If I understand what you're thinking, yes, Webware does this well.
Webware runs in a single, multi-threaded process. So say you want to
keep a global list of everyone logged on: you'll have a global variable
in some module, and use .append() to add users or .remove() to remove
them. All servlets can access the same variable. Of course you have to
be careful to work with the data in a threadsafe manner, which is
inevitable (and of course the same concurrency issues exist in PHP as
well).
This data can be used for caching. IMHO the convenience of this caching
makes increasing performance easier in Webware than, say, SkunkWeb
(which uses multiple processes, and so makes data sharing harder).
SkunkWeb claims the multiple processes mean that it can better utilize
multi-processor systems because it is not pinned by the global
interpreter lock. This is true, but there are many ways to increase
performance, and I believe easy data sharing and caching is more of a
benefit than multiple processors would be. But anyway, I digress...
So, in conclusion, PHP is very expedient, and if you are making
applications where expedience is important, then PHP probably will work
well for you. If you are making larger more powerful applications where
design really matters, then Python has some benefits over PHP. And they
are big benefits -- PHP as a language is horrible, horrible, horrible.
And it's not even a matter of compromises -- PHP doesn't *have* to be
that horrible, most of its flaws aren't tied to its conveniences. It
just chooses to be horrible.
--
Ian Bicking Colorstudy Web Development
ia...@colorstudy.com http://www.colorstudy.com
PGP: gpg --keyserver pgp.mit.edu --recv-keys 0x9B9E28B7
4869 N Talman Ave, Chicago, IL 60625 / (773) 275-7241
Paul Rubin
> So, in conclusion, PHP is very expedient, and if you are making
> applications where expedience is important, then PHP probably will work
> well for you. If you are making larger more powerful applications where
> design really matters, then Python has some benefits over PHP. And they
> are big benefits -- PHP as a language is horrible, horrible, horrible.
> And it's not even a matter of compromises -- PHP doesn't *have* to be
> that horrible, most of its flaws aren't tied to its conveniences. It
> just chooses to be horrible.
I don't find PHP to be horrible. It's just kind of limited.
Ian Bicking
> I don't find PHP to be horrible. It's just kind of limited.
That might be how I'd describe Tcl, say, but PHP really has some stupid
properties. Like say a function foo() returns an object, and you want
to call a method on the object. You'd expect to be able to do
"foo()->method()". But you can't, you have to use an intermediate
temporary variable to hold the object. I don't know why this is the
case -- either the PHP parser is braindead or there's something very
weird going on with PHP objects.
And it has weird reference stuff going on, probably taken from Perl -- I
guess it's not surprising, but there's just no *point* to using that
technique. It gains you *nothing* except maybe some misguided notion of
safety across function calls (because the variables get copied), and it
adds a lot of inefficiency (because of unnecessary copying), and/or the
confusion of references.
The layout of the standard library is truly horrendous. Python isn't
great, we have has_key after all... but it's nothing like PHP.
And the string functions particularly annoy me. PHP is *meant* to
produce strings. That's *all* it's meant to produce. It should do that
really well, no? But it has no decent syntactic string handling, you
have to use functions for everything, and the functions are all over the
place. Regular expressions are even worse with the \'s doubled all over
the place and other nonsense.
Oh, and string literals are worse. Python doesn't have $ variable
substitution unlike PHP, but PHP is just idiotic and wastes this
feature. The thing that annoys me the most is that " (double-quotes)
gets used all the time in HTML, and ' (single-quote) doesn't. But if
you want to do variable substitution you have to use ", so you get all
these things like "<input type=\"text\" value=\"$value\">". Or even
worse you get people doing "<input type=text value='$value'>".
And the quoting is crazy. There's nothing to help you quote literal
text for HTML so you have to use the htmlentities function all the time
if you want to be thorough. Ditto SQL and shell. And that's why
there's all these security problems, once again PHP fails to make string
substitution reasonably easy to do right. And then they try to fix it
with magic quoting, which quite frankly is fucked up. Depending on your
installation you don't know if you have to unquote things, quote them,
or just leave them alone. So you get all these websites where the name
O'Connor gets turned into O\'Connor and other nonsense.
Well, you didn't really ask for it, but that's my PHP rant, or at least
the things I can think of off the top of my head this evening :)
Francois Pinard
> Ian Bicking <ia...@colorstudy.com> writes:
> > PHP as a language is horrible, horrible, horrible. [...]
> I don't find PHP to be horrible. It's just kind of limited.
As a tool, it might be usable. As a language, I agree with Ian that it is
not attractive. I did not evaluate it for long before putting it aside
as being a bit disgusting, and would prefer not having to use it ever.
Yet, I understand that circumstances do not always leave us a full choice.
--
François Pinard http://www.iro.umontreal.ca/~pinard
Afanasiy
>On Mon, 2003-01-20 at 11:16, Afanasiy wrote:
>> * Run on the same hardware I use currently
>
>Yes, so long as you can run long-running processes. Most commercial
>shared-hosting situations don't allow this. This applies to all Python
>frameworks except when accessed strictly through CGI, which will give
>you atrocious performance.
Webware is it's own daemon which would run behind Apache?
So..., stability depends on another daemon, one with much
less testing than Apache? Does it use lots of 3rd party
Python modules/code to provide this daemon?
>> * Be as fast or faster than Apache+PHP
>> (I'm sorry to say Zope/Roxen are not)
>> (This might mean it must use Apache)
>
>Probably comparable speed. It's definitely faster than Zope. We
>haven't encountered people who have felt limited by the speed of Webware
>(as compared to bandwidth limitations, database, etc). But that might
>just be because people aren't using in situations where it would be a
>problem.
Hmm.
>> * Allow all errors to be caught and handled so users never see them
>> (I currently send them to an email address)
>> (Users are notified an error occurred and a developer contacted)
>
>Yep.
With a global handler or pasting in your
try except handling block every page?
>> * Allow similarly powerful regex capabilities
>> (I currently use PHP's pcre module)
>
>Again, using Python's re module (IMHO much easier to use than in PHP).
Can you elaborate? I use PHP's heredoc syntax for my complex regex.
>I believe there's ways to do this with Apache while using Webware. But
>you can (in effect) do this in Webware fairly easily. Each page of a
>Webware application is a separate class. Generally there will be a
>superclass that you write that is specific to your application. In it
>you can put a header and footer, and those can be as dynamic as you
>choose.
Where does this class live? Do I have to edit .htaccess every time
I create a new page, I saw that in the mod_python documentation.
I am aware of different connectors, but mod_python looks likely.
A.M. Kuchling
> Webware is it's own daemon which would run behind Apache?
> So..., stability depends on another daemon, one with much
> less testing than Apache? Does it use lots of 3rd party
> Python modules/code to provide this daemon?
Depends how complicated the daemon is, though; if it's a simple daemon,
it's not too difficult to get it right and reliable. For example, SCGI
was originally written because about once a week Apache's mod_fastcgi
would hang, and attempts to replicate or debug the problem went nowhere.
I can't recall having any problems with SCGI once it went into
production use.
--amk (www.amk.ca)
Right, freeze! I mean, don't freeze, I mean stand still!
-- Ace, in "Dragonfire"
Jacob Smullyan
> This data can be used for caching. IMHO the convenience of this caching
> makes increasing performance easier in Webware than, say, SkunkWeb
> (which uses multiple processes, and so makes data sharing harder).
> SkunkWeb claims the multiple processes mean that it can better utilize
> multi-processor systems because it is not pinned by the global
> interpreter lock. This is true, but there are many ways to increase
> performance, and I believe easy data sharing and caching is more of a
> benefit than multiple processors would be. But anyway, I digress...
I'm a developer for SkunkWeb, and would make a different value
judgement, although I have no factual disagreement with Mr. Bicking.
I would agree that SkunkWeb's multiple-process design does not
necesarrily translate into a performance advantage in all scenarios,
and the published SkunkWeb propaganda may make too much of this design
decision (although I think it was an important factor for the
applications for which it was initially designed). There is a cost to
be paid in the heaviness of the application (which doesn't matter
much) and the relative difficulty in doing some kinds of resource
sharing that would be simpler or perhaps more efficient in a
multi-threaded context.
There are some pluses to a multi-process design beyond transcending
the limitations of the global interpreter lock. You never ever have
to worry about thread sychronization in writing for SkunkWeb, which
will make some developers' lives easier. You don't have to worry
about crashing the whole application server. It just fits with
SkunkWeb's rather open, un-boxed-in aesthetic; it is an easy design to
live with, for some folks. Let's just say, without starting a
religious war about threads (I'm agnostic!) that a multi-process
alternative is nice to have.
As for caching, SkunkWeb is I believe very strong in this area, as it
does have a very useful component cache, which stores component output
on disk; and this cache can be controlled with considerable finesse,
and furthermore with ease. While access to a disk cache is not going
to be as fast as access to memory, just as with the GIL lock issue,
there are other ways of improving performance. The convenience and
controllability of the SkunkWeb cache is a huge advantage, as it can
be utilized to the fullest with virtually no development effort.
(Incidentally, if anyone feels strongly that SkunkWeb should be able
to run in multi-threaded mode, feel free to submit patches :). The
main challenge would be to make the Configuration object in
multi-thread mode return thread-specific information, and of course to
provide an alternate bootloader.)
Cheers,
Jacob Smullyan
Paul Rubin
> As for caching, SkunkWeb is I believe very strong in this area, as it
> does have a very useful component cache, which stores component output
> on disk; and this cache can be controlled with considerable finesse,
> and furthermore with ease. While access to a disk cache is not going
> to be as fast as access to memory, just as with the GIL lock issue,
> there are other ways of improving performance. The convenience and
> controllability of the SkunkWeb cache is a huge advantage, as it can
> be utilized to the fullest with virtually no development effort.
>
> (Incidentally, if anyone feels strongly that SkunkWeb should be able
> to run in multi-threaded mode, feel free to submit patches :).
I'm not that crazy about threads either. I don't understand why more
multi-process servers don't use shared memory instead of disk files,
or even (gak) SQL databases and worse. Sometimes I wonder whether MPI
(the pseudo-shared memory used in scientific clusters like Beowulf)
might be an appropriate abstraction for programs like web servers.
There are very efficient MPI implementations that let processes on the
same machine communicate through shared memory, and can also let
processes on separate machines communicate through a LAN.
Jacob Smullyan
> I don't understand why more
> multi-process servers don't use shared memory instead of disk files,
> or even (gak) SQL databases and worse. Sometimes I wonder whether MPI
> (the pseudo-shared memory used in scientific clusters like Beowulf)
> might be an appropriate abstraction for programs like web servers.
> There are very efficient MPI implementations that let processes on the
> same machine communicate through shared memory, and can also let
> processes on separate machines communicate through a LAN.
A cache like SkunkWeb's needs to be able to hold a large amount of
data, and accessing it has to be faster than recalculating it (which
in the real world often involves going to a SQL database for it). A
naive implementation with shared memory and sql databases is unlikely
to meet those requirements; however, a naive implementation with disk
files works quite well. When a naive implementation works well
enough, it isn't unreasonable to continue using it, and innovate
elsewhere.
jacob smullyan
Karsten W. Rohrbach
> I'm not that crazy about threads either. I don't understand why more
> multi-process servers don't use shared memory instead of disk files,
SHM is an API which might have differences from platform to platform.
File I/O should be quite the same in the POSIX world. Furthermore, files
*are* RAM, in case of enough RAM, so they are pretty fast. On most
platforms you can mmap() them and, together with working file locking,
use them as sahred memory with a pretty similar API but with better
performance (cf. libmm).
> or even (gak) SQL databases and worse. Sometimes I wonder whether MPI
Abusing a RDBMS for cache data storage dosn't look like a good design
decision.
> (the pseudo-shared memory used in scientific clusters like Beowulf)
> might be an appropriate abstraction for programs like web servers.
MPI implementations are very (versatile|general)(?). In case of web
applications, a large number of problems is solved by separating
network I/O from execution logic from data I/O:
Apache/cache/proxy[i] -> App Server[j] -> DB Server[k]
where ( j > i > k ), at least in my experience ;-)
In this scenario, state and session information and shared data is held
in the back end DB server, but other mechanisms are possible. Cache data
is held on-disk in the front end Apache systems, which practically
increases the overall amount of cached data proportional to the number
of frontend systems, but this buys us network performance, so I consider
it a Good Thing[tm].
> There are very efficient MPI implementations that let processes on the
> same machine communicate through shared memory, and can also let
> processes on separate machines communicate through a LAN.
It all depends on the "amount of interactivity" you need between your
application's components, and in what stage of processing. If there's
just session IDs, cookies, user names and such (speak: "joe web app"),
it's pretty simple to implement that as a data-centric model in the
backend. For other (pseudo-realtime or interactive) applications, a
networked shared memory model would make sense.
Regards,
/k
Ian Bicking
> Ian Bicking <ia...@colorstudy.com> writes:
>
> > This data can be used for caching. IMHO the convenience of this caching
> > makes increasing performance easier in Webware than, say, SkunkWeb
> > (which uses multiple processes, and so makes data sharing harder).
> > SkunkWeb claims the multiple processes mean that it can better utilize
> > multi-processor systems because it is not pinned by the global
> > interpreter lock. This is true, but there are many ways to increase
> > performance, and I believe easy data sharing and caching is more of a
> > benefit than multiple processors would be. But anyway, I digress...
>
> I'm a developer for SkunkWeb, and would make a different value
> judgement, although I have no factual disagreement with Mr. Bicking.
> I would agree that SkunkWeb's multiple-process design does not
> necesarrily translate into a performance advantage in all scenarios,
> and the published SkunkWeb propaganda may make too much of this design
> decision (although I think it was an important factor for the
> applications for which it was initially designed). There is a cost to
> be paid in the heaviness of the application (which doesn't matter
> much) and the relative difficulty in doing some kinds of resource
> sharing that would be simpler or perhaps more efficient in a
> multi-threaded context.
I'm sorry, I was kind of randomly poking randomly at SkunkWeb, a product
for which I really have no antipathy. But anyway...
I guess to me performance is something that can *always* be improved
through a huge number of different techniques. The real limits come
from the amount of effort you can put into those techniques. So
theoretical limits aren't as important as the ease of incremental
improvements. There aren't any theoretical limits either, just limits
within certain bounds. SkunkWeb posits a situation where there's
multiple processors, and the application is bound by Python's execution
speed, which is not that general of a situation.
> There are some pluses to a multi-process design beyond transcending
> the limitations of the global interpreter lock. You never ever have
> to worry about thread sychronization in writing for SkunkWeb, which
> will make some developers' lives easier.
I don't see this as a real advantage. To the degree you share data and
resources you have to worry about concurrency issues. You may tend to
share data more often with threads, and so have to worry about
concurrency issues more often, but that just mitigates the advantages of
sharing. I don't think it makes multiple processes easier than threads.
Well, I suppose you can use globals in SkunkWeb in a way you couldn't in
Webware. Perhaps it's because I've always avoided globals, or perhaps
because I've internalized Webware's model, but I don't really desire to
use globals except when they are truly "global", i.e., shared.
> You don't have to worry
> about crashing the whole application server. It just fits with
> SkunkWeb's rather open, un-boxed-in aesthetic; it is an easy design to
> live with, for some folks. Let's just say, without starting a
> religious war about threads (I'm agnostic!) that a multi-process
> alternative is nice to have.
Certainly there can be issues in a single-process model. So far many of
them have come from buggy C modules, but there's other problems of
corruption as well. I must admit I remain a little wary of letting the
Webware server sit for months at a time -- cruft may start filling up
the cache, or other problems so far unforeseen may occur. This is a
problem that has always made me nervous of any stateful long-running
application, Webware included.
But I feel the problems are more connected with controlling persistence
then threads in particular. The partitioning required for multiple
processes tends to formalize some of that persistence, and I think the
advantage is as much one of enforced discipline as it is a quality of
the underlying technique.
> As for caching, SkunkWeb is I believe very strong in this area, as it
> does have a very useful component cache, which stores component output
> on disk; and this cache can be controlled with considerable finesse,
> and furthermore with ease. While access to a disk cache is not going
> to be as fast as access to memory, just as with the GIL lock issue,
> there are other ways of improving performance. The convenience and
> controllability of the SkunkWeb cache is a huge advantage, as it can
> be utilized to the fullest with virtually no development effort.
That is a definite advantage of SkunkWeb which Webware does not share.
We don't have any real abstract representation of the output in Webware
-- the output is almost opaque to the application server -- so it's not
clear where caching would go in. Cheetah has some significant caching
features, if you use that. Any caching done more directly in Webware is
usually from a pre-HTML perspective, caching calculated data and complex
objects, not necessarily the direct output.
Ian Bicking
> On 23 Jan 2003 15:57:33 -0600, Ian Bicking <ia...@colorstudy.com> wrote:
>
> >On Mon, 2003-01-20 at 11:16, Afanasiy wrote:
> >> * Run on the same hardware I use currently
> >
> >Yes, so long as you can run long-running processes. Most commercial
> >shared-hosting situations don't allow this. This applies to all Python
> >frameworks except when accessed strictly through CGI, which will give
> >you atrocious performance.
>
> Webware is it's own daemon which would run behind Apache?
> So..., stability depends on another daemon, one with much
> less testing than Apache? Does it use lots of 3rd party
> Python modules/code to provide this daemon?
Yes, it is essentially a daemon -- a program running in the background
that handles requests. This is true of almost all Python web
frameworks.
A daemon is just a process running in the background. There's not much
special code involved.
> >> * Allow all errors to be caught and handled so users never see them
> >> (I currently send them to an email address)
> >> (Users are notified an error occurred and a developer contacted)
> >
> >Yep.
>
> With a global handler or pasting in your
> try except handling block every page?
It's handled for you.
> >> * Allow similarly powerful regex capabilities
> >> (I currently use PHP's pcre module)
> >
> >Again, using Python's re module (IMHO much easier to use than in PHP).
>
> Can you elaborate? I use PHP's heredoc syntax for my complex regex.
Well, I find the quoting and handling of regexes in PHP to be annoying,
but I'm not an advanced PHP programmer and there might be conveniences
I'm missing. I think Python's re module works pretty well.
> >I believe there's ways to do this with Apache while using Webware. But
> >you can (in effect) do this in Webware fairly easily. Each page of a
> >Webware application is a separate class. Generally there will be a
> >superclass that you write that is specific to your application. In it
> >you can put a header and footer, and those can be as dynamic as you
> >choose.
>
> Where does this class live? Do I have to edit .htaccess every time
> I create a new page, I saw that in the mod_python documentation.
> I am aware of different connectors, but mod_python looks likely.
For Webware you wouldn't want to use mod_python. There's mod_webkit
which is a small module that just is meant to connect to the AppServer
(it's analogous to SCGI, and vaguely like a simplified FastCGI). You
can also use a CGI interface (which shouldn't be confused as meaning
Webware runs as a CGI -- just the adapter can run as such).
.htaccess files aren't really used for anything in Webware -- though you
can chain Webware with other Apache modules like mod_gzip. I haven't
experimented with this a great deal, though.
Your servlets are all plain classes in Webware. The classes can inherit
from other classes, so you might have a Login class (servlet) which
inherits from SitePage (your class that defines the look and other
convenience methods), which inherits from Page (a class that comes with
Webware).
Jonathan P.
Ok, I've checked the majority of them out and
Spyce gets my thumbs up. I looked especially
closely at CherryPy and Quixote because the
2 looked the most promising initially.
CherryPy 0.8
============
In spite of my enthusiastic first impression, I
found CherryPy a bit too verbose and was not
convinced that views/functions/masks were
essential abstractions. CherryPy also makes use
of templates which I later realized was something
I didn't want to deal with as much as possible.
CherryPy's unique feature is that your site
compiles down to a python script which is its
own server! Pretty neat. Now if they could
incorporate Medusa's ideas for boosting scalability
into the generated server...
Quixote 0.5.x
=============
I thought Quixote's code-centric style would be
to my liking. However, it had just enough complicated
details that I decided to try my luck somewhere else.
If there were not so many other choices, I'd probably
have stuck it out with Quixote. I didn't think it
was really all that hard to figure out and I had the
gut feel there was a LOT of power under the hood, but
I wanted something I could work with with as little
new studying as possible.
Webware and Skunkweb
====================
I then decided to check out these two mainstays again.
On doing so, I realized a key fact. No matter how
nice templates are, like I found PHP's Smarty
Templates to be, I realized in the end that they just
get in the way of an experienced programmer who will
always be able to do much better with straight code.
Skunkweb did not look like it had a good way of embedding
real Python code in HTML - relying pretty heavily on STML -
or at least that's the impression I got from its docs. With
Webware, I remember being less than fully satisfied with PSP
circa 0.4.x - the somewhat hard to read <% %> tags were one
reason - so I had an excuse to check out the other
alternatives.
Skunkweb's 'hype' makes it sound like it's the most reliably
scalable of all the solutions... although I suspect that the
principles it uses will not be hard to apply to the others
with a little work.
Spyce
=====
I was initially turned off by Spyce's "Hello World" example.
Those [[ and ]]s are not good for readability. However, I
found out that it had a [[\ operator which basically was an
escape to pure Python code and was exactly what I was looking
for.
Spyce gives me the vanilla PHP-like functionality I want(*) -
the kind of straight Python-code embedding that PyHP, PyML,
PMZ, Fenster provide except that these projects seem to have
been abandoned. Plus, upon closer inspection, Spyce seems
to be quite mature and has evolved quite a few really nifty
features.
(*) The only other 'living' alternative seems to be Webware's
PSP... pls. correct me if I'm wrong...
I was somewhat turned off by the fact that the docs asked
me to install fastcgi before running it but the fact is that
Spyce will work with plain cgi. Later on, I was sufficiently
enamoured with Spyce that I even installed modpython and now
that I plan to use it, may even upgrade to Apache 2.x so I
can use modpython with my thread-enabled Python interpreter
without worries.
Cheetah
=======
Another great bonus of Spyce is that it can work with Cheetah,
the one templating system which I felt like I could live with.
Instead of using hard-to-read < and >s, it uses $-prefixed
variables. Cheetah is its own templating project independent of
any Python server-side project and seems to have grown quite
sophisticated. It is intended to be a plug-in to Python
server-side solutions or stand-alone as a template processor
(say for static HTML generation).
I believe Cheetah is nice enough that it will prove useful one
day even for someone like me who eschews templates for real
code.
Performance and Reliability
===========================
My theory was that using a Python server-side solution meant
I could just use kinterbasdb to access Firebird databases
the way I've been doing in my Linux client apps along. This
is actually the reason I explored a Python server-side solution
- I didn't want to experiment anymore with interfacing between
PHP and Firebird. After I got Spyce running, I quickly and
joyously confirmed that theory.
Now that I have a 3-tier setup, I can realistically examine
performance and reliability issues.
I made a simple spyce script to fetch a big result from a
Firebird DB. It was a total cinch by the way, see code
below. Python's DB-API kicks the excellent, now _formerly_
excellent, PHP ADODB's butt any day of the week/month/year.
Let's not even talk about the native PHP MySQL or Interbase
APIs.
==== kinterbasdb.spy =====
[[\
import kinterbasdb
con=kinterbasdb.connect(dsn="localhost:/path/to/data.gdb",
user="foo",password="bar")
cur=con.cursor()
cur.execute("select * from big_table")
results=cur.fetchall() ]]
<html><body><pre>[[ for r in results: print x ]]</pre></body></html>
===== end of kinterbasdb.spy ======
With Firebird on the same server as Apache and using
mod_python, performance felt decent (no hard figures
yet) at the very least. mod_python looks like the way
to go (even better than fastcgi) and could be a good
excuse to upgrade to Apache 2.0.
Questions and more insights on issues - I'm especially
interested in python interpreter spawning differences
between mod_python and fastcgi, as well as Apache 1.3.x
vs Apache 2.x pros and cons - very happily entertained.
Mark Smith
>Spyce gives me the vanilla PHP-like functionality I want(*) -
>the kind of straight Python-code embedding that PyHP, PyML,
>PMZ, Fenster provide except that these projects seem to have
>been abandoned.
For simple templating and Python embedding, PWO (www.jamwt.com/pwo/)
has a clean, uncluttered syntax.
Alex Reid
news:o6m73vg64klj53r5r...@4ax.com...
If you don't mind your application being coupled to Java libraries, I've had
good results using JythonServlet. www.jython.org. Sometimes JSP can be used
to template pages. All of the Java libraries are there to use, not to
mention session handling. And it all runs inside a servlet container! For
me, it's the best compromise. I'd be interested to hear if anyone else is
using this approach widely.
Alex