Peeking into database queries from the unit-test framework
Adrian Holovaty
developers access to information about the underlying database queries
in a given test.
Specifically, I could see (and I currently *have*) needs for the
following assertions:
* Assert a given custom model method runs a certain, exact number of SQL queries
* Assert a given custom model method's SQL contains a certain string
The only way this is exposed in Django is through
django.db.connection.queries if DEBUG=True. The unit test framework
explicitly sets DEBUG=False on line 84 of django/test/simple.py, so
I've been doing the following in my tests:
def testFoo(self):
# Turn DEBUG on and reset queries, so we can keep track of queries.
# This is hackish.
from django.conf import settings
from django.db import connection
connection.queries = []
settings.DEBUG = True
MyModel.objects.some_method()
# Reset queries and turn DEBUG off again.
connection.queries = []
settings.DEBUG = False
There has to be a better way. :-)
The whole django.db.connection.queries thing has always been a bit of
a hack/wart. (Hackwart?) Maybe it's time to expose the underlying
queries in a more developer-friendly way. Off the top of my head, we
could allow the registration of callbacks with django.db.connection,
like so:
django.db.connection.register_query_callback(lambda query:
sys.stderr.write(query))
Adrian
--
Adrian Holovaty
holovaty.com | djangoproject.com
Russell Keith-Magee
>
> Specifically, I could see (and I currently *have*) needs for the
> following assertions:
>
> * Assert a given custom model method runs a certain, exact number of SQL queries
>
> * Assert a given custom model method's SQL contains a certain string
Sounds like a reasonable request.
> Maybe it's time to expose the underlying
> queries in a more developer-friendly way.
I can't say I have ever found myself unsatisfied with
connection.queries, but I don't have any particular objections to
replacing it.
> django.db.connection.register_query_callback(lambda query:
> sys.stderr.write(query))
A few elaboration questions:
- What happens to connection.queries in this plan? Does it remain, but
the callback to populate it only gets installed if DEBUG=True? Or are
you suggesting that we don't provide query logging by default, but
provide a utility callback that users can install on their own if they
ever need query logging?
- What happens to reset_queries? Does this listener still get
installed on the request_started signal? If so, how does it integrate
with the query callback?
Russ %-)
Amit Upadhyay
The whole django.db.connection.queries thing has always been a bit of
a hack/wart. (Hackwart?) Maybe it's time to expose the underlying
queries in a more developer-friendly way. Off the top of my head, we
could allow the registration of callbacks with django.db.connection,
like so:
django.db.connection.register_query_callback(lambda query:
sys.stderr.write(query))
Can't the same be done as a signal? Multiple parties can then listen for query execution.
Amit Upadhyay
Vakow! www.vakow.com
+91-9820-295-512
Malcolm Tredinnick
> I propose we add a hook to the unit-test framework that would give
> developers access to information about the underlying database queries
> in a given test.
>
> Specifically, I could see (and I currently *have*) needs for the
> following assertions:
>
> * Assert a given custom model method runs a certain, exact number of SQL queries
>
> * Assert a given custom model method's SQL contains a certain string
>
> The only way this is exposed in Django is through
> django.db.connection.queries if DEBUG=True.
One thing I've added in my local QuerySet rewrite is the ability to view
a query *before* it is executed. So you can view the same information
that goes into connection.queries directly, without it needing to go to
the database and back. Partly, this is to make debugging broken queries
easier -- presently, if they don't survive the trip to the database, you
sometimes don't see them in the debug log. However, it should be
possible to hook up some way of recording them in-situ in the test
framework, too.
I'm probably about two or three days away from publishing the first
patch of the QuerySet write-up (I'm going to be in airports and offline
a bit in the next couple of days). Once I get to New York and am able to
respond in a more timely fashion to requests, I'll put the patch up
somewhere and people can look at the API for things like that.
Regards,
Malcolm