Mod_Python
Harlin
benefit is there to use Mod_Python along with Karrigell in front of
Apache?
Thanks,
Harlin Seritt
Jack
Harlin Seritt
Thanks for the comment. Sorry to razz but scalability compared to? Do you mean compared to Karrigell or just Apache in general? The hosting thing does make sense I suppose.
Thanks,
Harlin
Jack <jli...@gmail.com> wrote:
We won't tell. Get more on shows you hate to love
(and love to hate): Yahoo! TV's Guilty Pleasures list.
Jack
When running karrigell as a standalone server, you get one
server process only, which does not run on multiple CPUs because
of GIL.
With mod_python, you can have multiple threads that could spread to
multiple processors. But I don't really know how to configure
karrigell with mod_python and I have not used mod_python myself
so this is just a theory, which could also be wrong, depending on
how karrigell is setup. If it's also one instance of karrigell,
I suppose it should be similar to running it separately?
--
Best regards,
Jack
Graham Dumpleton
> Hello Harlin,
>
> When running karrigell as a standalone server, you get one
> server process only, which does not run on multiple CPUs because
> of GIL.
>
> With mod_python, you can have multiple threads that could spread to
> multiple processors. But I don't really know how to configure
> karrigell with mod_python and I have not used mod_python myself
> so this is just a theory, which could also be wrong, depending on
> how karrigell is setup. If it's also one instance of karrigell,
> I suppose it should be similar to running it separately?
It is actually a bit more complicated than that.
The difference with mod_python when compared to a pure Python web
server is that in mod_python processing is split between the Apache C
code and the mod_python/karrigell Python code. Specifically, all the
initial acceptance of a HTTP request and determination of what handler
to send it to is in C code. Also, the read of request content off the
socket and writing back of content is also all in C code as well. When
in these parts of the code the GIL is not an issue nor is it an issue
when non Python handlers are being executed to server up static files
or pages dynamically generated by some other language such as PHP. It
is only when control is passed through into the Python code to process
the request and generate the response that GIL becomes an issue.
When you are using Apache prefork MPM on UNIX, none of this really
matters though as in that case there are multiple Apache child
processes for handling requests and a specific process only handles
one request at a time and there is no threading involved. Thus there
is no GIL contention issues within the context of any one Apache child
process and with the process only doing one thing at a time, distinct
processes can make use of different processors or cores.
When using Apache worker MPM on UNIX it is a different issue though.
You still have multiple Apache child processes and so at that level
multiple processors or cores can be utilised, but within a specific
Apache child process there can be multiple requests being handled
within distinct threads. In this case, the GIL will become an issue
and when there are multiple requests at the same time being handled
within that process which need to execute Python code, those threads
will not be able to run in parallel across different processors or
cores. This though only affects the threads running in Python code at
a specific time. That same process could be serving up static files at
the same time and the thread for that request could quite happily be
running on a different processor or core as the GIL isn't being used
as that is straight C code.
Thus, the GIL problem does not impact Apache on UNIX to as much degree
because it spreads requests across multiple child processes, plus,
even when multiple requests are run within the same process, there
could be a mix of Python requests, static file requests, plus requests
in other dynamic languages. Thus there can still be a fair degree of
parallelism because it isn't running only Python code.
Graham
lui...@gmail.com
I just read about your mod_wsgi project ( http://code.google.com/p/modwsgi/
) and, since you posted this message on the karrigell mailing list, I
wonder if you are familiarized with this framework.
What I'd like to know is:
Is it possible to integrate Karrigell with mod_wsgi?
Regarding your comment: "Thus, for a large Python web framework you
should only expect to see comparable performance to mod_python. Only
when using frameworks with less overhead or where no database access
is involved would you expect to see gains over using mod_python."
How does Karrigell stack up against these other frameworks? Do you
think it will have less or more ovehead?
In few words, do you think is posible to integrate Karrigell with
mod_wsgi and, if so, how much do you think it will improve its
performance?
Best regards,
Luis
On Apr 12, 8:48 pm, "Graham Dumpleton" <Graham.Dumple...@gmail.com>
wrote:
Graham Dumpleton
> ) and, since you posted this message on the karrigell mailing list, I
> wonder if you are familiarized with this framework.
>
> What I'd like to know is:
> Is it possible to integrate Karrigell withmod_wsgi?
> Regarding your comment: "Thus, for a large Python web framework you
> should only expect to see comparable performance to mod_python. Only
> when using frameworks with less overhead or where no database access
> is involved would you expect to see gains over using mod_python."
>
> How does Karrigell stack up against these other frameworks? Do you
> think it will have less or more ovehead?
>
> performance?
I actually know very little about Karrigell.
The question you really want to be asking is how hard would it be to
add to Karrigell support for being hosted using WSGI underneath. Do
that and you can use mod_wsgi, or any WSGI hosting mechanism for that
matter.
If I get some time I'll have a bit more of a look at Karrigell, but am
sure others more knowledgeable about Karrigell can answer a lot
quickly whether it could be hosted as a WSGI application.
Graham
Graham Dumpleton
mod_python, is that if Karrigell expects to be able to change the
working directory to a particular directory and for it to stay that
way, then you expose yourself to lots of potential problems given that
you share the server with lots of other stuff. In short, no web
application that wants to be runnable under Apache should be dependent
on being run out of a particular directory and should always use
absolute paths. This in some respects applies to any WSGI capable
application component as mounting two instances of such a component
where each requires a different working directory just isn't going to
work.
Now, the question is, what is there in Karrigell that makes it being
dependent on being run out of a specific directory. If there isn't it
shouldn't be too hard to create a WSGI adapter for it much like one
was done for mod_python as reposted recently.
Graham.
On Apr 26, 8:06 am, Graham Dumpleton <Graham.Dumple...@gmail.com>
wrote:
Graham Dumpleton
use on any multithreaded web server because of its use of os.chdir().
Thus, you couldn't use it on Windows with mod_python or mod_wsgi, nor
with worker MPM on UNIX. It would only be safe to use with prefork MPM
on UNIX. I would also suggest that it may also be unsafe to use with
the supplied multithreaded web server unless there is higher level
locking being performed, although doing that would make use of
mutilthreading a pointless exercise.
One example of unsafe code is in core/Template.py:
# before execution, chdir to script dir
saveDir=os.getcwd()
thisDir=os.path.dirname(fileName)
os.chdir(thisDir)
if not thisDir in sys.path:
sys.path.append(thisDir)
output=script.render(self.nameSpace,**args)
os.chdir(saveDir)
It is unsafe because another thread could change the working directory
between the call to os.chdir() and the rendering of the script.
Another possible example is in KarrigellRequestHandler.py:
os.chdir(os.path.dirname(fileName))
script = None
# cache
if self.cache(fileName):
script = self.loadedScripts[fileName][1]
# process the script according to its extension
elif extension.lower() in k_config.handled_extensions:
... deleted
# in any case, restore current directory
os.chdir(saveDir)
Again, another thread might change the current working directory
meaning that any code assuming that the current working directory was
a certain value and using relative paths could fail.
Hopefully someone will tell me that these calls to os.chdir() don't
actually occur as part of the process of handling requests and it is
all safe after all.
Graham
On Apr 26, 10:09 am, Graham Dumpleton <Graham.Dumple...@gmail.com>
Pierre Quentel
Thanks for the very detailed information. The conclusion is simple :
as it is today, Karrigell is not ready for use behind Apache /
mod_python because of this threading issue (and the built-in
multitheaded server is not reliable either). The current dir is
changed at every request to the script directory, so that relative
paths can be used to open files, and sys.path is updated so that
modules in the same directory can be imported
I suppose the solution would be to acquire a lock at the start of each
request and release it when the response has been sent. Granted, this
makes multithreading pointless, but I don't see any other solution
Well, yes, there is another one I am working on at the moment : see
the new thread on mod_cgi in this group
Regards,
Pierre
> > > Graham- Masquer le texte des messages précédents -
>
> - Afficher le texte des messages précédents -
makerjoe
has the pure modpython psphandler the same issue as karrigell
apachehandler ?
if not,.... why ?
by the way ,the same question regarding spyce2.1 apache handler
thanks
joseluis
> > - Afficher le texte des messages précédents -- Ocultar texto de la cita -
>
> - Mostrar texto de la cita -
Graham Dumpleton
change the working directory using os.chdir() at the start of every
request and expect it to stay that way for the life of the request
like Karrigell does.
Graham
> > > Thus, you couldn't use it on Windows with mod_python ormod_wsgi, nor
> > > > > that and you can usemod_wsgi, or any WSGI hosting mechanism for that
makerjoe
thanks for you answer
what about modpython + spyce2.1 ?
kind regards
joseluis
On 20 mayo, 19:22, Graham Dumpleton <Graham.Dumple...@gmail.com>
wrote:
> > > - Mostrar texto de la cita -- Ocultar texto de la cita -
Graham Dumpleton
using it with mod_python suggests though that they believe it works
okay. You are better off asking the spyce people if they know of any
issues.
Note that it is Karrigell here that has made a specific design
decision as to how they implemented things that causes problems in
multithreaded servers, it isn't is a general problem with mod_python
or multithreaded servers as such.
Graham
On May 21, 10:45 am, makerjoe <maker...@gmail.com> wrote:
> hi graham
> thanks for you answer
>
> what about modpython + spyce2.1 ?
>
> kind regards
> joseluis
>
> On 20 mayo, 19:22, Graham Dumpleton <Graham.Dumple...@gmail.com>
> wrote:
>
> > Themod_python.psp handler does not have a problem because it doesn't
> > change the working directory using os.chdir() at the start of every
> > request and expect it to stay that way for the life of the request
> > like Karrigell does.
>
> > Graham
>
> > On May 21, 4:13 am, makerjoe <maker...@gmail.com> wrote:
>
> > > hi pierre,graham
>
> > > has the pure modpython psphandler the same issue as karrigell
> > > apachehandler ?
> > > if not,.... why ?
>
> > > by the way ,the same question regarding spyce2.1 apache handler
>
> > > thanks
> > > joseluis
>
> > > On 8 mayo, 15:58, Pierre Quentel <quentel.pie...@wanadoo.fr> wrote:
>
> > > > Hi,
>
> > > > Thanks for the very detailed information. The conclusion is simple :
> > > > as it is today, Karrigell is not ready for use behind Apache /
> > > >mod_pythonbecause of this threading issue (and the built-in
> > > > multitheaded server is not reliable either). The current dir is
> > > > changed at every request to the script directory, so that relative
> > > > paths can be used to open files, and sys.path is updated so that
> > > > modules in the same directory can be imported
>
> > > > I suppose the solution would be to acquire a lock at the start of each
> > > > request and release it when the response has been sent. Granted, this
> > > > makes multithreading pointless, but I don't see any other solution
>
> > > > Well, yes, there is another one I am working on at the moment : see
> > > > the new thread on mod_cgi in this group
>
> > > > Regards,
> > > > Pierre
>
> > > > On 26 avr, 08:34, Graham Dumpleton <Graham.Dumple...@gmail.com> wrote:
>
> > > > > Looked a bit further it would seem to me that Karrigell is not safe to
> > > > > use on any multithreaded web server because of its use of os.chdir().
> > > > > Thus, you couldn't use it on Windows withmod_pythonormod_wsgi, nor
> > > > > > was done formod_pythonas reposted recently.
>
> > > > > > Graham.
>
> > > > > > On Apr 26, 8:06 am, Graham Dumpleton <Graham.Dumple...@gmail.com>
> > > > > > wrote:
>
> > > > > > > On Apr 26, 7:30 am, "luis...@gmail.com" <luis...@gmail.com> wrote:
>
> > > > > > > > Hello Graham,
>
> > > > > > > > I just read about yourmod_wsgiproject (http://code.google.com/p/modwsgi/
> > > > > > > > ) and, since you posted this message on the karrigell mailing list, I
> > > > > > > > wonder if you are familiarized with this framework.
>
> > > > > > > > What I'd like to know is:
> > > > > > > > Is it possible to integrate Karrigell withmod_wsgi?
> > > > > > > > Regarding your comment: "Thus, for a large Python web framework you
> > > > > > > > should only expect to see comparable performance tomod_python. Only
Harlin Seritt
This is not a flame, but I must ask: Why in the world all these mixes? I realize it's good to question the wheel every now and then, but re-inventing? or mash-ups only for the sake of mashing up? Why? Karrigell used with Apache will give you everything you need to create secure and functional dynamic web apps. Too many choices, too much redundancy (and not good redundancy as in clustered machines with multiple drives :-D ) leads to analysis paralysis. As always, if anyone can give me a good reason for it, I will listen with both ears. But I really don't see the need for Spyce and Karrigell to run together or Spyce be able to be run inside Karrigell. I'm still not sold too much on mod_python being used with Karrigell. Please forgive my ignorance if it shines through ;-)
Thanks,
Harlin Seritt
makerjoe <make...@gmail.com> wrote:
hi graham
thanks for you answer
what about modpython + spyce2.1 ?
kind regards
joseluis
On 20 mayo, 19:22, Graham Dumpleton
wrote:
> The mod_python.psp handler does not have a problem because it doesn't
> change the working directory using os.chdir() at the start of every
> request and expect it to stay that way for the life of the request
> like Karrigell does.
>
> Graham
>
> On May 21, 4:13 am, makerjoe wrote:
>
>
>
> > hi pierre,graham
>
> > has the pure modpython psphandler the same issue as karrigell
> > apachehandler ?
> > if not,.... why ?
>
> > by the way ,the same question regarding spyce2.1 apache handler
>
> > thanks
> > joseluis
>
> > On 8 mayo, 15:58, Pierre Quentel wrote:
>
> > > Hi,
>
> > > Thanks for the very detailed information. The conclusion is simple :
> > > as it is today, Karrigell is not ready for use behind Apache /
> > > mod_python because of this threading issue (and the built-in
> > > multitheaded server is not reliable either). The current dir is
> > > changed at every request to the script directory, so that relative
> > > paths can be used to open files, and sys.path is updated so that
> > > modules in the same directory can be imported
>
> > > I suppose the solution would be to acquire a lock at the start of each
> > > request and release it when the response has been sent. Granted, this
> > > makes multithreading pointless, but I don't see any other solution
>
> > > Well, yes, there is another one I am working on at the moment : see
> > > the new thread on mod_cgi in this group
>
> > > Regards,
> > > Pierre
>
> > > > wrote:
>
> > > > > BTW, one issue with Karrigell under Apache, ie., also applies to
> > > > > mod_python, is that if Karrigell expects to be able to change the
> > > > > working directory to a particular directory and for it to stay that
> > > > > way, then you expose yourself to lots of potential problems given that
> > > > > you share the server with lots of other stuff. In short, no web
> > > > > application that wants to be runnable under Apache should be dependent
> > > > > on being run out of a particular directory and should always use
> > > > > absolute paths. This in some respects applies to any WSGI capable
> > > > > application component as mounting two instances of such a component
> > > > > where each requires a different working directory just isn't going to
> > > > > work.
>
> > > > > Now, the question is, what is there in Karrigell that makes it being
> > > > > dependent on being run out of a specific directory. If there isn't it
> > > > > shouldn't be too hard to create a WSGI adapter for it much like one
> > > > > was done for mod_python as reposted recently.
>
> > > > > Graham.
>
> > > > > On Apr 26, 8:06 am, Graham Dumpleton
> > > > > wrote:
Get your own web address.
Have a HUGE year through Yahoo! Small Business.
lui...@gmail.com
1) Availability of shared hosting suppliers
2) Performance
Karrigell's default configuration is ok if you run your own server and
your site is not under heavy load.
However, nobody knows how it will perform if it gets too much traffic.
As far as I know, other framework authors strongly suggest using their
built-in-servers only for development, not for production.
Now if you want to use Karrigell in a shared hosting, it will be
extremelly difficult.
Hosting providers won't give you this service because it requires one
active server instance for each account, which makes it difficult to
maintain. It also requires more resources.
There used to be a supplier who hosted Karrigell apps (it is now
called webfaction and it is speciallized in python) but I don't know
if they still do. Their homepage doesn't mention Karrigell anymore.
Luis
On May 21, 8:37 am, Harlin Seritt <harlinser...@yahoo.com> wrote:
> To all:
>
> This is not a flame, but I must ask: Why in the world all these mixes? I realize it's good to question the wheel every now and then, but re-inventing? or mash-ups only for the sake of mashing up? Why? Karrigell used with Apache will give you everything you need to create secure and functional dynamic web apps. Too many choices, too much redundancy (and not good redundancy as in clustered machines with multiple drives :-D ) leads to analysis paralysis. As always, if anyone can give me a good reason for it, I will listen with both ears. But I really don't see the need for Spyce and Karrigell to run together or Spyce be able to be run inside Karrigell. I'm still not sold too much on mod_python being used with Karrigell. Please forgive my ignorance if it shines through ;-)
>
> Thanks,
>
> Harlin Seritt
>
> ---------------------------------
Harlin Seritt
Harlin
"lui...@gmail.com" <lui...@gmail.com> wrote:
Two main reasons:
1) Availability of shared hosting suppliers
2) Performance
Karrigell's default configuration is ok if you run your own server and
your site is not under heavy load.
However, nobody knows how it will perform if it gets too much traffic.
As far as I know, other framework authors strongly suggest using their
built-in-servers only for development, not for production.
Now if you want to use Karrigell in a shared hosting, it will be
extremelly difficult.
Hosting providers won't give you this service because it requires one
active server instance for each account, which makes it difficult to
maintain. It also requires more resources.
There used to be a supplier who hosted Karrigell apps (it is now
called webfaction and it is speciallized in python) but I don't know
if they still do. Their homepage doesn't mention Karrigell anymore.
Luis
On May 21, 8:37 am, Harlin Seritt wrote:
> To all:
>
> This is not a flame, but I must ask: Why in the world all these mixes? I realize it's good to question the wheel every now and then, but re-inventing? or mash-ups only for the sake of mashing up? Why? Karrigell used with Apache will give you everything you need to create secure and functional dynamic web apps. Too many choices, too much redundancy (and not good redundancy as in clustered machines with multiple drives :-D ) leads to analysis paralysis. As always, if anyone can give me a good reason for it, I will listen with both ears. But I really don't see the need for Spyce and Karrigell to run together or Spyce be able to be run inside Karrigell. I'm still not sold too much on mod_python being used with Karrigell. Please forgive my ignorance if it shines through ;-)
>
> Thanks,
>
> Harlin Seritt
>
Yahoo! oneSearch: Finally, mobile search that gives answers, not web links.