ANNOUNCE Httpdapy 1.3a
Gregory (Grisha) Trubetskoy
Folks,
There is a new version of Httpdapy out there, that even supports Bobo
now. It still needs some work and I would still call it alpha, but it
already works really well.
Best of all, it now has Bobo support, so you get the best of both: The
ease of development with Bobo and the speed of Python embedded within
the http server.
The URL is http://www.ispol.com/home/grisha/httpdapy/
Here is an excerpt from the README:
OVERVIEW
Httpdapy allows embedding Python within a webserver for a considerable
boost in performance and added flexibility in designing web based
applications.
Currently the best (and in some respects only) supported http server is
Apache.
WHAT'S NEW IN THIS VERSION
1. Bobo support. The httpdapi_bobo module allows the use of Bobo
(http://www.digicool.com/site/Bobo/) with httpdapi. See the module
itself to find out how.
2. This version makes a major leap forward by introducing multiple
interpreters. Scripts running in different directories will run in
(almost) completely separate and clean namespace. At (apache) module
initialization, a dictionary of interpreters keyed by interpreter name
is created. Interpreter name can be any string. For every hit,
Httpdapy will use the file parent directory path from the URI as
interpreter name. If an interpreter by this name already exists, it
will be used, else it is created. You can also force an interpreter
name with PythonInterpreter directive (whose effects recurse into
subdirectories). This is useful if you want to share an interpreter
between separate directories.
3. Autoreload mode. It is on by default. It makes the server keep
track of module import time and forces a reload when the script file
change time is later than the time of last import. (Don't confuse this
with Python's default behaviour. In Python, once the module has been
imported, nothing but the "reload" command will make the interpreter
read the file again.)
3. mod_python.c will cd into the directory to which the URL points and
httpdapi.py will prepend a '.' to PYTHONPATH. This means that scripts
can be imported from the current directory. This is more intuitive in
my opinion than knowing that scripts are imported from somewhere in
PYTHONPATH.
4. PythonInitFunction directive is now obsolete. It still works for
backwards compatibility, but it forces the interpreter to function in
"single interpreter" mode - i.e. all scripts share the same global
namespace. Note that you can still use separate interpreters in single
mode by forcing an interpreter name with PythonInterpreter directive.
Laurent Szyster
Until now, mod_perl is my prefered choice as embedded interpreter,
although I like the Python language more (this is not a rational
choice and I don't plan to argue about such thing, "les goûts et les
couleurs ne se discute pas" as we say in French).
Having python embedded in Apache is definitively a nice thing. My
question is how different is httpdapy (or should I say mod_python)
from PyApache, and more important from mod_perl.
Andreas Jung
> Hello Gregory,
>
> Until now, mod_perl is my prefered choice as embedded interpreter,
> although I like the Python language more (this is not a rational
> choice and I don't plan to argue about such thing, "les goûts et les
> couleurs ne se discute pas" as we say in French).
>
> Having python embedded in Apache is definitively a nice thing. My
> question is how different is httpdapy (or should I say mod_python)
> from PyApache, and more important from mod_perl.
>
First the documentation of httpdapy is poor. I am not really sure
what is the main difference between httpdapy and PyApache. I assume
httpdady offers some kind of support for persistent Python scripts
because where is an option for autoreloading scripts when they
have changed. httdady is not a tool for embedding Python into HTML.
If httpdady offers support for persistent Python scripts this would be
a very nice solution for efficent Python scripting (you don't need to
load the interpreter and to load/compile the script). On the other hand
there a solutions available that do the same job however in a more general
fashion (-> FastCGI).
It would be nice to get more detailed information about what httpdady can do !
Andreas
Aaron_...@my-dejanews.com
Andreas Jung <aj...@sz-sb.de> wrote:
> First the documentation of httpdapy is poor. I am not really sure
> what is the main difference between httpdapy and PyApache.
Nor am I, but since it links into the server internals
a bit lower down it *might* be a bit faster (?). If you
want more info on it, the original version which has been
extended a great deal and fixed up but is basically similar
is explained in the embedding chapter of "Internet Programming
with Python." Thanks to Grisha for picking this up and
improving it! I think the IPwP chapter explains the basic
architecture pretty well.
BTW: Sean McGrath had a nice review of IPwP in Dr. Dobb's
Journal last month. Thanks!
-- Aaron Watters
===
"Man," said the Ghost, "...Will you decide what men
shall live and what men shall die? It may be that in
the sight of Heaven you are more worthless and less
fit to live than this poor man's child. Oh, God!
to hear the Insect on the leaf pronouncing on the
too much life among his hungry brethren in the dust!"
from "A Christmas Carol", Dickens.
-----------== Posted via Deja News, The Discussion Network ==----------
http://www.dejanews.com/ Search, Read, Discuss, or Start Your Own
Gregory (Grisha) Trubetskoy
>
> On Wed, Dec 02, 1998 at 04:09:40PM +0100, Laurent Szyster wrote:
>
> First the documentation of httpdapy is poor. I am not really sure
> what is the main difference between httpdapy and PyApache.
I don't have much information on the differences between PyApache and
Httpdapy because I myself don't know a lot about PyApache. I compiled it
and ran some test scripts, looked at the code a little bit, but I don't
know all the little details, certainly not enough to write a
comprehensive comparison.
May be we can start a discussion and with everyone's help soon I will
have a list. Please comment on this and correct any errors. I will then
add the end result to the README.
Here are the differences that I can come up with:
1. The development process for PyApache and Httpdapy is different. For
PyApache you for the most part write what look like CGI scripts. You get
your info from the environment and write to stdout. With Httpdapy you
have to inherit from the httpdapi.RequestHandler() class, and the
content is sent by returning a string rather than writing to stdout.
2. Httpdapy takes advantage of a new featrue in Python (since 1.5 I
think) that allows creation of multiple sub-interpreters each having its
own versions of imported modules, separate sys.modules, __builtin__,
__main__, stdin, stdout, etc. The Python C API documentation describes
it better here:
http://www.python.org/doc/api/initialization.html#l2h-2379
Httpdapy creates a separate sub-interpreter for different dircetories in
which scripts are located. So scripts /mydir/myscript.py and
/hisdir/hisscript.py will run in separate interpreters. (There is also a
way to share sub-interpreters between directories.)
As far as I understand mod_perl does something similar. PyApache does
not do this. In PyApache, the sub-interpreter is reset (destroyed and
recreated) for every hit, so you don't have different interpreters
running in parallel.
2. PyApache creates a sub-interpreter (Py_NewInterpreter()) for every
request and destroys it when done. This means that if your script begins
with "import HTMLgen", HTMLgen is imported (bytecode read from file) for
every hit.
Httpdapy keeps the interpreter around from the first hit and until the
process dies. So only the first hit will actually read HTMLgen.py(c),
all the subsequent won't.
PyApache has a persistent mode, however... Lele, could you comment on
this?
3. While PyApache is written in its entirety in C, Httpdapy only uses
enough C to provide a "link" between python and the web server. Most of
the actual functionality of Httpdapy is implemented in Python.
Httpdapy's C code imports the module, instantiates a Python objects and
from then on delegates handling all of the requests to that Python
object.
This is probably a tad slower than pure C, but it is more flexible this
way and allows for a tighter integration with the user scripts I think.
(I hope what I just said makes sense...)
4. Httpdapy has a couple of features convenient for developers. It can
write python traceback prints to the browser and will also re-import
(using "reload" statement) scripts whose file change date is newer than
the time of last import. Before I introduced this feature, one had to
restart the server every time a change to a script was made.
5. The httpdapi_bobo module provides plumbing for Bobo. This still needs
some work, but I think this is a very exciting feature. While Bobo
provides it's own tool for maintaining persistance which requires you to
run a separate "server" written in Python, I think embedding the
interpreter within the http server is a better solution.
Thanks,
Grisha
Paul Everitt
"Gregory (Grisha) Trubetskoy" wrote:
> 5. The httpdapi_bobo module provides plumbing for Bobo. This still needs
> some work, but I think this is a very exciting feature. While Bobo
> provides it's own tool for maintaining persistance which requires you to
> run a separate "server" written in Python, I think embedding the
> interpreter within the http server is a better solution.
Of course I agree that it's very exciting :^) At a minimum people get
the cool object publishing features (URLs that look like object
messages, etc.) and DocumentTemplate.
Bobo doesn't require you to run it as a separate process to manage the
persistence though. Some people don't use our object database and
rather use relational databases to manage the persistence. And Bobo is
pretty agnostic about how its objects get published -- for instance,
many people "embed" it in Medusa, quite similarly to what you are doing
here.
I think that many people in the Bobo community will be quite interested
in your project. If it's ok, we'll add it to our pages.
Finally, since I've said Bobo three times in this email, I need to go
ahead and announce that Bobo no longer exists. Instead, Bobo has merged
with our Principia and Aqueduct products to become the "Z Object
Publishing Environment", aka Zope. Check out www.zope.org for more
details. The first source release is tomorrow. We are positioning Zope
as a free, opensource competitor to Cold Fusion for web designers and
programmers.
What's in it for your project? First, Zope provides a web management
framework and an HTML GUI to go with it. Also, the C-ification pieces
in Principia are now available as free, opensource.
--Paul
Gregory (Grisha) Trubetskoy
Paul,
By the way, I think we're talking about different persistence here. I
meant persistence of Python environment from a request to request such
as what PCGI I think provides, not persistence the ability to save
objects to disk such as what BoboPOS does.
Now that I have this out of the way, this is very interesting news.
I have been developing web applications for a long time. What motivated
me to keep working on first Nsapy and then Httpdapy was performance of
database web applications. I found that using CGI for programs that need
to connect to relational databases (commercial or not) is too slow
because not only does every hit require loading of the Pyhton executable
which can be megabytes in size, but database libraries that need too be
loaded severely aggravate the situation. I did not have any faith in
MS's ASP, and was quite frustrated by Netscape's LiveWire slow
performance and bugginess. I was quite excited about ColdFusion when it
first came out, but then found that writing programs in cf tags makes
them as readable as assembly. Besides, I *really* wanted to write things
in Python.
So I thought if I could load the database libraries and connect to the
database once and then retain that connection throughout the life of the
server I will get a significant performance increase. Experience showed
that I was right. My tests with Nsapy showed that it beat LiveWire hands
down in performance and flexibility.
However, in the recent months the popularity of Netscape's servers took
a turn south, and so I set out to port Nsapy to (at least) Apache, so
was born Httpdapy.
Somewhere in the process I discovered Bobo and thought that Bobo's way
of publishing objects is the neatest way to develop WWW Python
applications I ever saw. I still have very little experience with Bobo,
but I think that Httpdapy could provide plumbing for Bobo (I know I
should call it Zope) that will immediately position it ahead of Cold
Fusion in performance and flexibility.
I don't see Httpdapy commercializing in the future, but to support a
product like Zope on which I understand the well-being of Digital
Creations will depend to a large extent, will require a tad better
support for Httpdapy than currently exists.
So I was wondering if you would be interested in combining efforts and
helping me develop it further so that it does not lag behind Zope's
development... Just something to think about, not sure what is the most
practical way to accomplish this, but we can discuss it later....
Grisha
Paul Everitt
Gregory (Grisha) Trubetskoy wrote:
>By the way, I think we're talking about different persistence here. I
>meant persistence of Python environment from a request to request such
>as what PCGI I think provides, not persistence the ability to save
>objects to disk such as what BoboPOS does.
Ahh. How is state communicated between Apache child processes?
>Now that I have this out of the way, this is very interesting news.
>
>I have been developing web applications for a long time. What motivated
>me to keep working on first Nsapy and then Httpdapy was performance of
>database web applications. I found that using CGI for programs that need
>to connect to relational databases (commercial or not) is too slow
>because not only does every hit require loading of the Pyhton executable
>which can be megabytes in size, but database libraries that need too be
>loaded severely aggravate the situation. I did not have any faith in
>MS's ASP, and was quite frustrated by Netscape's LiveWire slow
>performance and bugginess. I was quite excited about ColdFusion when it
>first came out, but then found that writing programs in cf tags makes
>them as readable as assembly. Besides, I *really* wanted to write things
>in Python.
I'm glad to hear that you have experience on these other architectures. You
should consider writing up a quick paper for the python.org site that
discusses your experiences. I'll help you with it if you'd like.
For any others out there that would like to contribute, let me know.
>So I thought if I could load the database libraries and connect to the
>database once and then retain that connection throughout the life of the
>server I will get a significant performance increase. Experience showed
>that I was right. My tests with Nsapy showed that it beat LiveWire hands
>down in performance and flexibility.
Right, that's what we have done as well, first with the ILU Requester, then
FastCGI, then with our PersistentCGI.
>However, in the recent months the popularity of Netscape's servers took
>a turn south, and so I set out to port Nsapy to (at least) Apache, so
>was born Httpdapy.
>
>Somewhere in the process I discovered Bobo and thought that Bobo's way
>of publishing objects is the neatest way to develop WWW Python
>applications I ever saw. I still have very little experience with Bobo,
>but I think that Httpdapy could provide plumbing for Bobo (I know I
>should call it Zope) that will immediately position it ahead of Cold
>Fusion in performance and flexibility.
Please go to www.zope.org and join the Zope mailing list. There is a lot of
talk about the next generation of PCGI. Some would like to have a Python
version, some would like to run it in Apache. Sounds like you can do both.
--Paul
Grisha Trubetskoy
Paul Everitt wrote:
>
> Gregory (Grisha) Trubetskoy wrote:
> >By the way, I think we're talking about different persistence here. I
> >meant persistence of Python environment from a request to request such
> >as what PCGI I think provides, not persistence the ability to save
> >objects to disk such as what BoboPOS does.
>
> Ahh. How is state communicated between Apache child processes?
It isn't.
I didn't find this to be a big problem though...
Aaron_...@my-dejanews.com
Paul Everitt <pa...@digicool.com> wrote:
> I think that many people in the Bobo community will be quite interested
> in your project. If it's ok, we'll add it to our pages.
Please mention that its based on the example from the "embedding"
chapter for "Internet Programming with Python", 1986 :).
-- Aaron Watters
===
"I weep for you," the Walrus said.
"I deeply sympathize!"
With sobs and tears he sorted out
Those of the largest size,
Holding his pocket handkerchief
Before his streaming eyes.
-- Lewis Carroll