Strange Problem Involving discount_price Template Tag and Dreaded "can't adapt" Error

[Stuart Laughlin]

Hello Everyone --

I have a recurring intermittent problem that I've been battling off
and on for the past few months. The error initially manifests itself
as a ProgrammingError in the product template on the bit that says "{{
product|discount_price:sale|currency }}". I've been dealing with it up
till now by commenting it out, avoiding using the discount_price
template tag, etc. Finally I resolved to get to the bottom of it and
added some logging to the get_product view. Well, the only effect that
had was to move the origin of the error from the template to my
logging statement in the view (but at least it seems like I get a
better stack trace when the error happens in the view!).

At any rate, I'm looking at my stack trace, and I think I understand
what's happening. My logging statement in the get_product() view
throws an error when I invoke the discount_price template-tag method:

log.debug("product|discount_price:sale is %s" %
discount_price(product, best_discount))

The error appears to actually occur in product/prices.py on line 14 here:

8. qty_discounts = product.price_set.exclude(
9. expires__isnull=False,
10. expires__lt=datetime.date.today()).filter(quantity__lte=qty)
11.
12. adjustments = None
13.
14. if qty_discounts.count() > 0:
15. # Get the price with the quantity closest to the one specified
without going over
16. adjustments = qty_discounts.order_by('price','-quantity',
'expires')[0].adjustments(product)

Line 14 executes the query defined in lines 8-10, and qty_discounts
ends up being set to "Error in formatting: can't adapt".

At the moment I'm getting this error every time I navigate to a
product (which obviously invokes my get_product view). I suspect that
when I restart apache, the error will go away.

But here's the really odd thing: I can use "manage.py" to fire up the
shell and run the discount_price template tag with no problems, even
though I'm running it exactly like I am in the logging statement. So I
find this very puzzling. Has anyone seen anything like this? I'm
really hoping someone can point me in the right direction.

Oh, and I'm on version 0.9.1 final.

Full traceback follows my signature.

Thanks very much!

--Stuart

Environment:

Request Method: GET
Request URL: http://10.10.20.20/vw/product/basic-black-center-console-304734/
Django Version: 1.1.1
Python Version: 2.5.2
Installed Applications:
['django.contrib.sites',
'satchmo_store.shop',
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.admindocs',
'django.contrib.contenttypes',
'django.contrib.comments',
'django.contrib.sessions',
'django.contrib.sitemaps',
'registration',
'keyedcache',
'livesettings',
'l10n',
'sorl.thumbnail',
'satchmo_store.contact',
'tax',
'tax.modules.no',
'tax.modules.area',
'tax.modules.percent',
'shipping',
'product',
'product.modules.configurable',
'payment',
'payment.modules.dummy',
'payment.modules.giftcertificate',
'satchmo_utils',
'app_plugins',
'satchmo_ext.productratings',
'satchmo_ext.recentlist',
'satchmo_ext.upsell',
'satchmo_ext.wishlist',
'satchel',
'south',
'ecomm',
'filer',
'tinymce',
'debug_toolbar']
Installed Middleware:
('django.middleware.cache.UpdateCacheMiddleware',
'django.middleware.common.CommonMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.locale.LocaleMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.middleware.doc.XViewMiddleware',
'threaded_multihost.middleware.ThreadLocalMiddleware',
'satchmo_store.shop.SSLMiddleware.SSLRedirect',
'debug_toolbar.middleware.DebugToolbarMiddleware',
'passion.middleware.FilterPersistMiddleware',
'django.middleware.cache.FetchFromCacheMiddleware')

Traceback:
File "/usr/local/python-envs/satch/lib/python2.5/site-packages/django/core/handlers/base.py"
in get_response
92. response = callback(request, *callback_args,
**callback_kwargs)
File "/usr/local/python-envs/satch/src/satchmo/satchmo/apps/product/views/__init__.py"
in get_product
161. log.debug("product|discount_price:sale is %s" %
discount_price(product, best_discount))
File "/usr/local/python-envs/satch/src/satchmo/satchmo/apps/product/templatetags/satchmo_discounts.py"
in discount_price
107. return untaxed_discount_price(product, discount)
File "/usr/local/python-envs/satch/src/satchmo/satchmo/apps/product/templatetags/satchmo_discounts.py"
in untaxed_discount_price
117. up = product.unit_price
File "/usr/local/python-envs/satch/src/satchmo/satchmo/apps/product/models.py"
in _get_fullPrice
925. price = get_product_quantity_price(self, Decimal('1'))
File "/usr/local/python-envs/satch/src/satchmo/satchmo/apps/product/prices.py"
in get_product_quantity_price
32. adjustments = get_product_quantity_adjustments(product,
qty=qty, parent=parent)
File "/usr/local/python-envs/satch/src/satchmo/satchmo/apps/product/prices.py"
in get_product_quantity_adjustments
14. if qty_discounts.count() > 0:
File "/usr/local/python-envs/satch/lib/python2.5/site-packages/django/db/models/query.py"
in count
292. return self.query.get_count()
File "/usr/local/python-envs/satch/lib/python2.5/site-packages/django/db/models/sql/query.py"
in get_count
376. number = obj.get_aggregation()[None]
File "/usr/local/python-envs/satch/lib/python2.5/site-packages/django/db/models/sql/query.py"
in get_aggregation
348. result = query.execute_sql(SINGLE)
File "/usr/local/python-envs/satch/lib/python2.5/site-packages/django/db/models/sql/query.py"
in execute_sql
2369. cursor.execute(sql, params)
File "/usr/local/python-envs/satch/lib/python2.5/site-packages/django/db/backends/util.py"
in execute
19. return self.cursor.execute(sql, params)

Exception Type: ProgrammingError at /product/basic-black-center-console-304734/
Exception Value: can't adapt

[Stuart Laughlin]

I fear I type too much. Let me try this a different way.

I have a template tag (discount_price) that intermittently breaks --
throws a ProgrammingError with "Error in formatting: couldn't adapt"
message (debian stable, apache2, mod-wsgi, satchmo 0.9.1. stable). I
don't know what causes it to happen. Restarting the server makes the
problem go away. While the problem is happening, I can fire up the
django shell and invoke the template tag with no problems. It seems it
must have something to do with the apache process's connection to the
postgresql database.

Has anyone seen anything like this before?

--Stuart

[Chris Moffitt]

Do you see anything in your postgres logs? Sounds like some sort of caching issue but I'm guessing.

-Chris

--
You received this message because you are subscribed to the Google Groups "Satchmo users" group.
To post to this group, send email to satchm...@googlegroups.com.
To unsubscribe from this group, send email to satchmo-user...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/satchmo-users?hl=en.

[Stuart Laughlin]

Chris, thanks for the response. Good idea about watching the postgres
logs; I'm going to look at that right now.

Caching sounds like a likely culprit to me, too. Do you have any ideas
for how I could prove/disprove that? Can I turn off caching entirely?
Presently I'm using memcached, in conjunction with the UpdateCache and
FetchFromCache middleware.

Another data point: the thing that seems to distinguish my satchmo
implementation from 'normal' ones is that I have two sites (two django
projects) on my apache server. They are both operating out of the same
virtualenv against a single installation of satchmo. I bring it up
because the 'random' nature of the error feels kind of like a
threading issue.

--Stuart

[Stuart]

Well, I still don't have this problem resolved, but I suppose I'm
getting closer all the time.

Today I found an old thread that may very well describe the identical
problem I'm having.

http://groups.google.com/group/satchmo-users/msg/e78f589100e722f8

Corey Oordt's reply indicates that what I'm up against might be a bug
in psycopg2 that manifests when running multiple django sites on the
same server against the same instance of psycopg2. Corey suggests that
he got around the problem by running mod_wsgi in daemon mode (rather
than default embedded mode, I gather) and using virtualenv. I've been
using virtualenv from the start, but I have both satchmo sites running
out of the same virtualenv. I wonder if what he's describing implies
that each site should have its own virtualenv..? Corey, if you're out
there, I'd be extremely grateful for any additional information.

Also, I'm having a difficult time figuring out how to get mod_wsgi
into daemon mode. It seems that all I really need to do is add
WSGIDaemonProcess and WSGIProcessGroup settings to my httpd.conf, but
I've done that and it doesn't appear to change anything. Need to read
more docs, I reckon.

I have to say, figuring out this error has been the low point of my
django career thus far, and incredibly boring (my love for django is
pretty much untarnished, though). I think I need a django mentor to
teach me things like how track down what's going wrong in situations
like this. The interactions between apache, wsgi, django, psycopg2,
etc are fuzzy to me. Bruce, is this why you run lighttpd+fastcgi?? If
switching would make this problem go away, I'm game and getting gamer
all the time. ;)


--Stuart

[Chris Moffitt]

To be perfectly honest I'm not that familiar with the details of setting up Apache and multiple hosts.

I was speaking to someone about this very same issue and this is what they said -

" made each apache VHOST that is running satchmo as as WSGI daemon process with unique process group and problem went away."

So, I think you're on the right track but can't give you anymore specific details since that's not my area of expertise.

-Chris

[lifewithryan]

I always use separate virtualenv instances per project.  That being said, if psycopg2 Indeed has some sort of bug that makes the sharing of environments not work so well (I doubt it) then try separate virtualenv instances.  However, I'd be willing to bet just using different wsgi process groups/instances would be fine.

While I haven't had to run multiple satchmo instances, I have had to run multiple django projects side by side but have always used separate wsgi process groups in my apache config for each virtual host.  (I've also always used separate virtualenv instances for each as well even when they share dependencies since my requirments could change or force an upgrade of any one package at any time.) 

Sent from my iPod

[Graham Dumpleton]

On Jun 15, 10:28 am, lifewithryan <lifewithr...@gmail.com> wrote:
> I always use separate virtualenv instances per project.  That being  
> said, if psycopg2 Indeed has some sort of bug that makes the sharing  
> of environments not work so well (I doubt it)

I wouldn't doubt it at all. The pyscopg2 module for a long time had
problems with being able to be used in multiple sub interpreters of
the same process. They eventually fixed this. There could well though
be other issues, or they have introduced a completely new issue.

The safest thing to do is to use multiple mod_wsgi daemon process
groups and delegate every WSGI application to a separate daemon
process group. Also, ensure that WSGI application runs in the main
interpreter of the process and not a sub interpreter as psycopg may
well for some things not work properly in sub interpreters, as various
C extensions are prone to do because they never consider that they
might be used in sub interpreters.

Thus use something like:

WSGIDaemonProcess group1
WSGIDaemonPprcess group2

WSGIScriptAlias /app1 /some/path/app1.wsgi process-group=group1
application-group=%{GLOBAL}
WSGIScriptAlias /app2 /some/path/app2.wsgi process-group=group2
application-group=%{GLOBAL}

Above example configuration expects you are using mod_wsgi 3.X.

Also ensure you are using recent psycopg2 and not some outdated
version that your specific Linux distribution is packaging up.

Graham

> ...
>
> read more »

[Stuart Laughlin]

Gentlemen --

I'm very thankful for your input! Graham, especially thanks for the
example config. I'm going to try it out this morning, and will
certainly report back with results!

--Stuart

[Stuart]

At the risk of jinxing myself, I'm here to report that my "can't
adapt" problem appears to be solved! Words fail to describe my
giddiness; I'm trying to be merely cautiously optimistic, but I'm too
excited. :)

I fixed it by doing exactly what you guys described -- I upgraded
psycopg2 and mod_wsgi to the latest versions and configured apache to
use daemon mode.

Some additional details, in case anyone is interested...

I'm now on Apache/2.2.9, mod_wsgi/3.2, Python/2.5.2 (on debian stable)
upgraded from mod_wsgi 2.5: uninstalled old version via apt; installed
apache2-threaded-dev via apt; installed mod_wsgi 3.2 from source.
upgraded from psycopg2 2.07 to 2.2.1: uninstalled old version via apt;
installed via "pip install --upgrade psycopg2".

I put mod_wsgi into daemon mode using Graham's suggested
configuration:

WSGIDaemonProcess site1
WSGIDaemonPprcess site2

WSGIScriptAlias /app1 /some/path/app1.wsgi process-group=site1
application-group=%{GLOBAL}
WSGIScriptAlias /app2 /some/path/app2.wsgi process-group=site2
application-group=%{GLOBAL}

Just for the archives, and to reiterate how odd this problem was (or
so it seems to me) this all began with a ProgrammingError, "can't
adapt" originating from the discount_price template filter.

Thanks again for the help on this one, gentlemen!

--Stuart

On Jun 15, 9:48 am, Stuart Laughlin <stu...@bistrotech.net> wrote:
> Gentlemen --
>
> I'm very thankful for your input! Graham, especially thanks for the
> example config. I'm going to try it out this morning, and will
> certainly report back with results!
>
> --Stuart
>
> On Tue, Jun 15, 2010 at 5:15 AM, Graham Dumpleton
>

> ...
>
> read more »

[Bruce Kroeze]

On Mon, Jun 14, 2010 at 3:56 PM, Stuart <stu...@bistrotech.net> wrote:

I have to say, figuring out this error has been the low point of my
django career thus far, and incredibly boring (my love for django is
pretty much untarnished, though). I think I need a django mentor to
teach me things like how track down what's going wrong in situations
like this. The interactions between apache, wsgi, django, psycopg2,
etc are fuzzy to me. Bruce, is this why you run lighttpd+fastcgi?? If
switching would make this problem go away, I'm game and getting gamer
all the time. ;)

This is *exactly* why I prefer lighttpd & fastcgi.  That way I can run my servers in completely separate virtualenv/buildout sandboxes, and be completely certain that memory is not being shared, threads aren't getting tangled, and ... most importantly *debugging is much easier*.

I happily sacrifice a tiny bit of theoretical performance from Apache in return for those benefits.  Also for being able to restart stores separately from one another.

 

--
Bruce Kroeze
http://www.ecomsmith.com
It's time to hammer your site into shape.