Hi again,

Ok, my goals are as follows, with the goals at the top being the more
important/most efficient ones (at least IMO):

   1. Run multiple instances of Django on the same IIS server with only
      one instance of PyISAPIe.dll.  This would allow a system admin to
      setup PyISAPIe once and let non-admin users configure their python
      setup by using a specially named file in the root of the virtual
      directory or website.
          * Problem: os.environ['DJANGO_SETTINGS_MODULE'] is required to
            get Django running correctly.  However, this value is set at
            the application level.  Therefore, even if that value is set
            multiple times by Http.Isapi (the PyISAPIe script), Django
            only cares about the first instance.  It is therefore
            currently impossible, as best I understand, to run multiple
            instances of Django under the same PyISAPIe.dll instance.
   2. Run multiple instances of Django on the same IIS server with
      multiple instances of PyISAPIe.dll on IIS 5.1 or IIS 6 (without
      multiple application pools).  This still requires admin
      intervention for every website installed, but would allow
      development testing on Windows XP with virtual directories and
      would require less resources on IIS.
         1. Problem 1: when configuring IIS 5.1 in the same setup as
            noted below using virtual directories, the second
            application to be initiated doesn't work.
         2. Problem 2: when configuring IIS 6.0 in the same setup as
            noted below using websites that share an application pool,
            the system becomes unstable
   3. Run multiple instances of Django on the same IIS server
      (completed).  It requires a website or virtual directory for each
      django instances.
          * Each website or virtual directory will need to have its own
            PyISAPIe.dll and HTTP folder
          * my directory structure is as follows
                o c:\websites\django1
                      + pyisapi
                            # PyISAPIe.dll
                            # HTTP (directory with HTTP files)
                      + python
                            # djtutorial1  (django project directory)
                      + wwwroot
                o c:\websites\django2
                      + pyisapi
                            # PyISAPIe.dll
                            # HTTP (directory with HTTP files)
                      + python
                            # djtutorial2  (django project directory)
                      + wwwroot
          * my websites are as follows
                o django1.example.com
                      + home directory is c:\websites\django1
                      + Has its own application pool named django1 (this
                        is required, if you try to run multiples
                        instances of PyISAPIe in the same application
                        pool the system becomes unstable and stops working)
                o django2.example.com
                      + home directory is c:\websites\django2
                      + Has its own application pool named django2

Ok, so the third goal has been accomplished as far as I can tell.  I was
initially concerned that having multiple application pools might consume
more resources than is necessary, bit it probably isn't an issue based
on this comment:

    Each application pool will consume resources, so i suppose that is the
    disadvantage. You should however compare this to the advantages of
    application pools.
    10 application pools on a server is however not in any way overkill.
    One
    application pool per website is not strange at all, instead it is a
    quite
    good configuration (of course this depends on the website(s) and its
    web
    applications).
    You have too many application pools when the server is running short on
    resources because the application pools consume too much resources.
    --
    Regards,
    Kristofer Gafvert (IIS MVP)

    found here:
    http://www.eggheadcafe.com/forumarchives/inetserveriis/Nov2005/post24...

However, I would still like to see the other two goals accomplished.  
Goal #1 probably doesn't have anything to do with PyISAPIe as far as I
can tell.  That is probably a Django issue which I have asked about here:

http://groups.google.com/group/django-developers/browse_thread/thread...

The second goal, however, may be a PyISAPIe goal.  Would it possible to
run multiple instances if PyISAPIe in the same application pool?  This
would use less resources on IIS 6 and enable IIS 5.1 to have multiple
Django applications using virtual directories.

Thoughts, comments, questions welcome.

Thanks.

--
--------------------------------------
Randy Syring
RCS Computers & Web Solutions
502-644-4776
http://www.rcs-comp.com

"Whether, then, you eat or drink or
whatever you do, do all to the glory
of God." 1 Cor 10:31

Randy,

Thanks for the detailed list; this really puts into perspective what
the important considerations are for getting PyISAPIe to be more
flexible with Django.

For now, I think I'll try to figure out why having multiple instances
of PyISAPIe in the same application pool causes trouble. Are the
worker processes just crashing on you when you request a page?

- Phillip

On May 10, 1:52 pm, Randy Syring <ra...@rcs-comp.com> wrote:

On May 13, 5:45 pm, Phillip <phillip.sit...@gmail.com> wrote:

Phillip,

Using the method described here:

http://groups.google.com/group/pyisapie/web/multiple-instances-of-dja...

On Windows XP and IIS 5.1 with two virtual directories the virtual
directory which has it's URL called first displays appropriately.
When the url of the second virtual directory is requested, the request
completes from the browser's perspective, but nothing is returned.  I
just get a completely white screen with no errors that I am aware of.

On Windows 2003 server with IIS 6.0 using two websites with the same
application pool I get a response from the first and second website.
However, the response from the second website is slow and eventually
that worker thread crashes.

If you need any more info, please let me know.

Ok, I've figured out the problem, which I managed to overlook while
debugging and crashing for some time :)

The issue is with Python. The way PyISAPIe uses Python is meant to be
simple: if you have Python installed, PyISAPIe will find it and use it
as the interpreter. Nowadays this works well because python25.dll, the
Python core, is in the system32 folder. When a worker process starts
up and loads the extension, it sees a dependency for python25 and
loads it. Later on, when you load your *second* copy of the extension,
the dependency for python25 is already satisfied, so the currently-in-
memory copy is used, along with all its global interpreter data.

The reason the worker processes crashed is because the DLL init checks
to see if Python is initialized and loads some important stuff in
there that doesn't exist later for the second instance, since Python
was already loaded.

Alas, my initial advice to use multiple copies of the DLL only applies
to multiple application pools, because that's the only way to
guarantee one Python interpreter per process. There are ways around
this, such as embedding the Python interpreter into the PyISAPIe DLL,
but then it turns into a "special" distribution of Python just for
this app, which I'd like to avoid (although I have successfully done
just that with the python sources).

Sorry I didn't spot that caveat sooner!

 - Phillip

On May 13, 7:41 pm, rcs_comp <rsyr...@gmail.com> wrote:

Just for my own curiosity, is the "global interpreter data" the area
of memory where the django settings are stored (i.e. in the
environment)?

If you embedded the Python interpreter into PyISAPIe, would that allow
you to run multiple instances of Django (or any other app that puts
values in the memory space shared by multiple processes) in a shared
environment without having to have the system admin configure a new
application pool for each instance?  What would that do to
performance?

Obviously I lack the conceptual framework to understand exactly how
Python is interacting with PyISAPIe and IIS.  PHP, which is what I
work with mostly, doesn't have a concept of "global interpreter
data."  Each request only has access to its own variables regardless
of where they are stored (even in $_GLOBAL) so I have never before run
into an issue like I am having with Django and it's shared settings.

Obviously, I think this would best be solved at the Django level by
moving the settings into a non-global context. But I have put up two
tickets on the developers mailing list and have yet to get a single
response.  I thought maybe it was because it was an IIS issue and they
are mostly Linux and Apache users.  However, I came across a post by
someone that is having similar issues with Apache and FastCGI where
they have to create seperate application pools for each instance.

Thanks Phillip!

Just a small update.  I found out that IIS 5.1 can use "High
(Isolated)" for the Application Protection setting of a virtual
directory.  This will allow multiple instances of Django on IIS 5.1
for testing purposes.  More info can be found here:

http://groups.google.com/group/pyisapie/web/multiple-instances-of-dja...

Phillip,

Can you comment any further on the possibility of modifying PyISAPIe
to accommodate multiple instances of django with just one global
PyISAPIe instance?  Maybe you could so something like allow a WSGI
application to indicate to PyISAPIe that it needs its own instance of
the Python interpreter?

Thanks.

Hi Randy,

Sorry for the delay, things have been a little hectic lately.

So, the Python global data problem might be solved by embedding the
interpreter into the DLL in such a way that we can run several
instances in the same process or app pool, having each with its own
Http package. I might even be able to use some tricks to do that
without comi

The global data I was talking about includes the global interpreter
lock (http://docs.python.org/api/threads.html) and the "main"
interpreter instance. The Python C API includes support for multiple
interpreters that provide mostly-isolated environments (if only it was
that simple), which I make use of, but right now I use one specific
interpreter for loading and initial request processing - this is where
two instances of PyISAPIe in the same process will clash. Each
interpreter stores things like the module search path. So I might be
able to fix this by creating interpreters with unique names.

About the GIL, I don't consider it too much of a problem right now
because other request threads run while data is sent to the client.
There is the more complicated issue of Django buffering output within
Python, which doesn't let other request threads run, but I'll save
that for later discussion.

Now, on to getting multiple instances of Django to work by compiling
the Python interpreter/DLL into PyISAPIe - it just won't work. We'll
get the ability to have multiple Python interpreters/DLL instances
coexist, but they will still use os.environ to read Django settings.
Under the hood, it's reading process-level data, so there's nothing we
can do that would allow multiple Django instances in the same process.

Hopefully that's all clear- let me know if it makes sense.

- Phillip

On May 15, 12:12 pm, rcs_comp <rsyr...@gmail.com> wrote:

On May 29, 3:38 pm, Phillip <phillip.sit...@gmail.com> wrote:

No problem at all.  I am just glad to have someone interested in
working through this stuff with me.  I really appreciate your efforts
and feedback.

So, if I understand you correctly, you are saying you can get multiple
interpreter instances to co-exist, but all of them would be running in
the same process?  If that is the case, I agree that won't help with
getting multiple instances of Django running.  As noted in the other
thread:

http://groups.google.com/group/pyisapie/browse_thread/thread/a01dd5d3...

Its not just the DJANGO_SETTINGS_MODULE that is the problem.  It would
appear that there are a lot of places that Django assumes one-instance-
per-process and so there is data and settings stored in the process
level modules.  It looks like Django is limited to one-instance-per-
process framework for now until someone more familiar with the
framework and Python can make some modifications to the core.

Most of it was clear as mud :)  But in the end, I think its wise to
keep PyISAPIe as-is, using IIS application isolation as necessary,
until changes to Django can be made.

Thanks again.