Django + pyodbc = MemoryError

[Greg Taylor]

This is a weird one, so bear with me.

I'm running Mac OS 10.5.8 with MacPorts Python 2.6, pyodbc '2.1.6-
beta0' with Actual ODBC SQL Driver pointing to a MS SQL 2000 server
across a LAN. We're working on a large Django application that needs
to have some views that show data from the remote MS SQL server. The
server is a brand new Xserve with 4 GB RAM and 3 TB worth of drive
space.

This is very irregular, but I am getting a MemoryError exception
thrown when running cursor.fetchone(). If I visit the page that's
generating the exception, it'll usually throw the exception again
until I refresh it a few times, then it's really random as to whether
it'll error or not. If I copy/paste the pyodbc code out into a
separate module and run it directly without Django, no exceptions are
thrown.

The view is extremely basic, so I'm very well stumped, and have no
idea how to even begin diagnosing this, other than that this appears
to be happening on the pyodbc side:

Traceback (most recent call last):

File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/
python2.6/site-packages/django/core/handlers/base.py", line 92, in
get_response
response = callback(request, *callback_args, **callback_kwargs)

File "/Users/admin/gchub_db/apps/workflow/views/job_views.py", line
1114, in etools_show_request
ejob = cursor.fetchone()

MemoryError

The query is a very basic SELECT * FROM some_table; Any ideas on how
to get to the root of this would be greatly appreciated.

[mkleehammer]

MemoryError means it was unable to allocate buffer space. Is one of
the values very large?

[Greg Taylor]

The values themselves are pretty small. There is one text field that
can get to be a few sentences long, but is generally below 500
characters. There are, however, about 50 columns on the table we're
querying, and we hit a lot of them. The tables are not under our
control, so we are not able to clean things up on that end.

What's odd is that this error doesn't happen every time the request is
sent. Sometimes it comes through just fine, and other times it throws
the MemoryError exception. I have pasted a traceback from the Python
crash while running this on my Snow Leopard 10.6.2 machine on Django's
dev server. This is very similar to what is seen when production
apache2 + mod_wsgi + pyodbc crashes. You may find the traceback at:
http://pastebin.com/m2d03d03d

Thanks in advance,
Greg

[Greg Taylor]

Is there anything else I can provide to assist in debugging? This
powers a pretty critical portion of our business, it's unfortunate
that it is so unreliable right now.

[Greg Taylor]

Is this thing on?

[ma...@unformatt.com]

Greg, did you figure this out?