Python code issues with Apache - reload

14 views
Skip to first unread message

saran_ATD

unread,
Dec 23, 2009, 3:01:09 PM12/23/09
to modwsgi
1) I am using mod_wsgi module to load my python code with Apache.
However, I am having trouble with reloading the code. whenever I
modify the code I have to restart my apache. I am using WSGI Daemon
process load options in WSGI. I though that should solve the reload
problem. This is not a serious problem compared to that of my other
issue that I explained in section 2.

WSGIDaemonProcess borg-portal user=atd group=atd processes=1
threads=25
WSGIProcessGroup borg-portal

WSGIScriptAlias /borg-portal /home/skannan/borgTraining/atd/borg/
portal/portal.wsgi

2) More serious problem I have is when I fetch data from database, it
cached somewhere in Apache/mod_wsgi and at most of the time, the data
I see on the screen is different from what I have in database.
Please help.

Graham Dumpleton

unread,
Dec 23, 2009, 3:55:43 PM12/23/09
to mod...@googlegroups.com
2009/12/24 saran_ATD <saranka...@gmail.com>:

> 1) I am using mod_wsgi module to load my python code with Apache.
> However, I am having trouble with reloading the code. whenever I
> modify the code I have to restart my apache. I am using WSGI Daemon
> process load options in WSGI. I though that should solve the reload
> problem. This is not a serious problem compared to that of my other
> issue that I explained in section 2.
>
> WSGIDaemonProcess borg-portal user=atd group=atd processes=1
> threads=25
> WSGIProcessGroup borg-portal
>
> WSGIScriptAlias /borg-portal /home/skannan/borgTraining/atd/borg/
> portal/portal.wsgi

Have you read:

http://code.google.com/p/modwsgi/wiki/ReloadingSourceCode

> 2) More serious problem I have is when I fetch data from database, it
> cached somewhere in Apache/mod_wsgi and at most of the time, the data
> I see on the screen is different from what I have in database.
> Please help.

That is a caching issue for your specific application. If you are
using a framework, then ask on the mailing list for that framework.

Otherwise, you are going to have to explain more about what custom
framework you are using.

Graham

saran_ATD

unread,
Dec 29, 2009, 4:00:56 PM12/29/09
to modwsgi
Graham, Thank you for your response. I am not using custom framework.
It is just the latest django.

Here I have an issue, I don't know how to proceed.
Here what I have in models.py
I have a base class: for example say class base_class(models.Model)
In this base class I have definition for a db table, class Meta:
abstract = True
I have an manager class : class manager_class(models.Manager)
In this derived class I coded return reverse functionality

In view,
I am deriving a new class view_class(base_class)
Here I am setting Class Meta:db_table = my_table.

In this scenario, my_table is dynamic. Its physical structure is same
however, a new table created for every day. Hence, there will be more
than one table with same skeleton for every day. When I search for
based on date, it should search from corresponding date db table.
It is doing it only for the first request. Successive requests it is
searching from the same table. (Table name is part build using date
which user dynamically enters). I manipulated the table name with used
entered date however it is not recognized!!! in consecutive search.
When I restart my apache, it fetchs from the request table. How can I
tell this code to search from
different table for every request. (once again physical structures are
the same just one table for every day). In one sentence, When I print
the table name it prints correct table name however actual data is
fetched from first requested table.


On Dec 23, 3:55 pm, Graham Dumpleton <graham.dumple...@gmail.com>
wrote:
> 2009/12/24 saran_ATD <sarankannan2...@gmail.com>:

Graham Dumpleton

unread,
Dec 30, 2009, 5:10:47 AM12/30/09
to mod...@googlegroups.com
I do not know sufficient about Django to understand your issue. As I
said, you will be better off asking on the mailing list for the
framework you are using, ie., Django. So, perhaps try:

http://groups.google.com/group/django-users

Graham

2009/12/30 saran_ATD <saranka...@gmail.com>:

> --
>
> You received this message because you are subscribed to the Google Groups "modwsgi" group.
> To post to this group, send email to mod...@googlegroups.com.
> To unsubscribe from this group, send email to modwsgi+u...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/modwsgi?hl=en.
>
>
>

bran.name

unread,
Jan 9, 2010, 3:34:58 PM1/9/10
to modwsgi, Pascal Vree
Hey all,

I seem to be having exactly the same problems:

1. Sometimes getting old versions when refreshing a page, while I did
edit file.
- this first I did fix by writing a script that "touches" my wsgi
script when I save a file in my IDE, as read at:
http://code.google.com/p/modwsgi/wiki/ReloadingSourceCode

2. Some kind of caching of certain variables.
- this I dont know how to fix yet, but I got to reproduce the problem
fairly easy with the following code, executed as wsgi script:

import datetime
start = datetime.datetime.now()
def application(environment, start_response):
start_response('200 OK', [('Content-Type', 'text/html;
charset=utf8')])
return [str(datetime.datetime.now() - start)]


Exact Problem: Everytime I refresh the url to this page the time
printed grows bigger, if I wait exactly 1 second between 2 refreshes,
the time grows by exactly 1 second, there seems to be no end to it, I
waited 2 days, and it incremented with 2 days more. The moment I touch
the file or restart apache, the time printed is again the first
request parse time, like I need it to be every refresh. The problem
persists in both the mod_wsgi's embedded mode & daemon mode on all my
test boxes seen below.

Initially I was trying to write a simple page request time profiler
class for my homebrew python framework, but I have seen the problem
now in 3 different situations, with or without any framework, also
with a database connection.

I am running:
- Apache/2.2.12 on Ubuntu server 9.10 with mod_wsgi 3.1 in daemon mode
and python 2.6.4.

I am getting exactly the same results on these 2 other systems I
tested on:
- FreeBSD 6.1-RELEASE Apache/2.2.3 with python 2.4.3 and wsgi 3.1 with
same configuration.
- FreeBSD 7.0-RELEASE Apache/2.2.13 with python 2.5.4 and wsgi 3.1
with same configuration.

I am trying to setup a virtual hosting environment on various
application servers with a single source file storage server, but at
this moment I am only hosting 1 application, the problem persists on
all application servers within, I am pretty sure it has nothing todo
with concurrency or smth.


Tnx for all helpfull answers in advance,

Bran

On 30 dec 2009, 11:10, Graham Dumpleton <graham.dumple...@gmail.com>
wrote:


> I do not know sufficient about Django to understand your issue. As I
> said, you will be better off asking on the mailing list for the
> framework you are using, ie., Django. So, perhaps try:
>
>  http://groups.google.com/group/django-users
>
> Graham
>

> 2009/12/30 saran_ATD <sarankannan2...@gmail.com>:

GJ

unread,
Jan 9, 2010, 4:34:59 PM1/9/10
to modwsgi
According to the documentations, both issues are caused by the fact
mod_wsgi keeps the Python interpreter loaded all the time for a
specific application group. When you first time request a script from
your web browser, the script is *imported* by mod_wsgi, executing any
code on the global scope of the script. Then mod_wsgi looks for the
name "application" and calls it. On any subsequent calls mod_wsgi
simply calls "application" again from the already loaded script,
unless it detects that the script file has been edited.

1. Now if your script imports some other files, then those are cached
too. If you edit them and they happen to be imported again, then you
have two copies of the imported module, new and old, and it is quite
random which copy will be used. By using the daemon mode and
"touch"ing the main script this problem can be worked around since the
process restarts itself causing also all other imported modules be
removed from memory also.

2. This case is even simpler. The start variable is assigned in global
scope, so it is only set to some value when the script was first time
imported. If you need to set every time, you must do it inside the
called "application" since that is the only code mod_wsgi will call
again on each request. Now I am not sure if setting run_once has any
effect on this, but I would not count on using that.

bran.name

unread,
Jan 9, 2010, 5:05:57 PM1/9/10
to modwsgi
What would be the logic behind having two (or more when I edit&save a
file multiple times) copies of a imported module in memory, and "quite
random which copy will be used"..? That sounds pretty weird to me.

For issue 2: I didn't know the global scope was only executed once on
the first import, looks like a python feature I should accept :p

Graham Dumpleton

unread,
Jan 9, 2010, 10:56:25 PM1/9/10
to mod...@googlegroups.com
2010/1/10 GJ <godj...@gmail.com>:

> According to the documentations, both issues are caused by the fact
> mod_wsgi keeps the Python interpreter loaded all the time for a
> specific application group. When you first time request a script from
> your web browser, the script is *imported* by mod_wsgi, executing any
> code on the global scope of the script. Then mod_wsgi looks for the
> name "application" and calls it. On any subsequent calls mod_wsgi
> simply calls "application" again from the already loaded script,
> unless it detects that the script file has been edited.
>
> 1. Now if your script imports some other files, then those are cached
> too. If you edit them and they happen to be imported again, then you
> have two copies of the imported module, new and old, and it is quite
> random which copy will be used.

No you don't. At least, not within the same process.

You only get multiple copies where using multiprocess configuration
where a different process only first loads the module after the
subsequent code change.

In other words, different copies in different processes maybe, but not
the same process.

Do take head of the double import issue in the wiki documentation
about source code reloading however, albeit that that is a different
issue.

Graham

Reply all
Reply to author
Forward
0 new messages