db mock for unit testing
Andrey Khavryuchenko
database (even local one). Things get only worse when testcases start
demanding radically different datasets.
Then I've paused and wrote DbMock class for django that uses some black
magic to steal django db connection and substitute it with temporary sqlite
in-memory db.
Here it is: http://www.djangosnippets.org/snippets/345/
So far, it works w/o any issues on my pet project. Will be glad to hear
any comments and, especially, bugs found :)
--
Andrey V Khavryuchenko
Django NewGate - http://www.kds.com.ua/djiggit/
Development - http://www.kds.com.ua
Call akhavr1975 on www.gizmoproject.com
Russell Keith-Magee
>
> Then I've paused and wrote DbMock class for django that uses some black
> magic to steal django db connection and substitute it with temporary sqlite
> in-memory db.
How is this different to the default Django behavior if you specify
SQLite as your database? Can't you get exactly the same behavior by
creating a test_settings.py file that contains:
from settings import *
DATABASE_BACKEND='sqlite'
and then run:
./manage.py --settings=test_settings test
?
Yours,
Russ Magee %-)
Andrey Khavryuchenko
RK> On 7/27/07, Andrey Khavryuchenko <akh...@gmail.com> wrote:
>>
>> Then I've paused and wrote DbMock class for django that uses some black
>> magic to steal django db connection and substitute it with temporary sqlite
>> in-memory db.
RK> How is this different to the default Django behavior if you specify
RK> SQLite as your database? Can't you get exactly the same behavior by
RK> creating a test_settings.py file that contains:
RK> from settings import *
RK> DATABASE_BACKEND='sqlite'
RK> and then run:
RK> ./manage.py --settings=test_settings test
RK> ?
Just replied on django-developers: I need database mock to speedup tests.
Mocking them into in-memory sqlite was the simplest way to reach this w/o
losing flexibility.
Malcolm Tredinnick
> Russell,
>
> RK> On 7/27/07, Andrey Khavryuchenko <akh...@gmail.com> wrote:
> >>
> >> Then I've paused and wrote DbMock class for django that uses some black
> >> magic to steal django db connection and substitute it with temporary sqlite
> >> in-memory db.
>
> RK> How is this different to the default Django behavior if you specify
> RK> SQLite as your database? Can't you get exactly the same behavior by
> RK> creating a test_settings.py file that contains:
>
> RK> from settings import *
> RK> DATABASE_BACKEND='sqlite'
>
> RK> and then run:
>
> RK> ./manage.py --settings=test_settings test
>
> RK> ?
>
> Just replied on django-developers: I need database mock to speedup tests.
> Mocking them into in-memory sqlite was the simplest way to reach this w/o
> losing flexibility.
That isn't an answer to the question Russell asked, though. You can get
exactly the same end-effect (using in-memory SQLite) if you specify
SQLite as the database engine in the settings file you use for testing.
Deriving your testing settings file from the main settings file is just
a matter of importing and replacing the right variable.
The advantage of specifying SQLite directly (and the drawback to trying
to "fake it") is that Django won't inadvertently use any MySQL- or
PostgreSQL-specific SQL constructs under the covers when calling the
database. There are places we consider the value of
settings.DATABASE_ENGINE when constructing the SQL and we may leverage
that more in the future.
So what advantages are there to the mocking approach over just replacing
the setting?
Regards,
Malcolm
Andrey Khavryuchenko
MT> That isn't an answer to the question Russell asked, though. You can get
MT> exactly the same end-effect (using in-memory SQLite) if you specify
MT> SQLite as the database engine in the settings file you use for testing.
MT> Deriving your testing settings file from the main settings file is just
MT> a matter of importing and replacing the right variable.
MT> The advantage of specifying SQLite directly (and the drawback to trying
MT> to "fake it") is that Django won't inadvertently use any MySQL- or
MT> PostgreSQL-specific SQL constructs under the covers when calling the
MT> database. There are places we consider the value of
MT> settings.DATABASE_ENGINE when constructing the SQL and we may leverage
MT> that more in the future.
MT> So what advantages are there to the mocking approach over just replacing
MT> the setting?
Ok, I'll be more wordy.
I'm not using django's testing framework for several reasons:
- I'm using django from 0.91 days and wasn't following django-users all
this way
- I use nose and twill for testing and they have no ready-to-use plugs
into django testing (or I haven't been able to find it)
Also simply overriding DATABASE_ENGINE to sqlite leads to problems.
E.g. my code has
profiles = models.User.objects.extra(where=[
"'%s'" % identity_url + "like concat(login, '%%')"])
But sqlite has no concat function and straightforward approach will lead to
an exception. Thus
sqlite_conn.connection.create_function('concat', 2,
lambda *args: ''.join(args))
in DbMock setup.
So, back to business..
My intention was to speedup unit tests and make initial test data setup
easier. I am not proficient in django testing framework and couldn't
utilize it with nosetests. DbMock solves my issue.
Sure, I may be wrong and the proper way is to write nose test runner
and file patches that provide more mysql or postgresql compatability for
testing.
Russell Keith-Magee
>
> I'm not using django's testing framework for several reasons:
> - I'm using django from 0.91 days and wasn't following django-users all
> this way
> - I use nose and twill for testing and they have no ready-to-use plugs
> into django testing (or I haven't been able to find it)
There aren't any built-in mechanisms for supporting nose or twill
tests. There are so many 'alternative' testing frameworks - we're not
going to add support for every single one of them.
However, it is very simple to add support for an external testing
mechanism - this is one of the original design considerations. See the
following for details:
http://www.djangoproject.com/documentation/testing/#using-a-different-testing-framework
> But sqlite has no concat function and straightforward approach will lead to
> an exception. Thus
> sqlite_conn.connection.create_function('concat', 2,
> lambda *args: ''.join(args))
> in DbMock setup.
This is a workaround required by your specific SQL requirements.
Similar problems would exist if you used Postgres TSearch2 extensions
in your queries, or any other DB-specific extension. It's not really
Django's responsibility to normalize every possible SQL query across
every possible backend.
However, it probably is Django's responsibility to provide a generic
hook so that you (as an end user) can add whatever normalizations your
application may require. Any suggestions on how to approach this
problem would be greatfully accepted.
Yours,
Russ Magee %-)
Russell Keith-Magee
>
> So what advantages are there to the mocking approach over just replacing
> the setting?
Genuine mocking (as opposed to this proposal) has one really big
advantage - it's lightning fast. All the db-calls get faked using a
cache-like setup, so it takes pretty much no time to run any db query.
The cost comes in keeping the mock data source up to date.
I've been mentally ruminating on adding a mock framework to speed up
Django's tests, with some support framework to make generating the
mock data easier. When my ideas have congealed a little bit, I might
have some code to throw around. Watch this space.
Yours,
Russ Magee %-)
Andrey Khavryuchenko
RK> On 7/29/07, Malcolm Tredinnick <mal...@pointy-stick.com> wrote:
>>
>> So what advantages are there to the mocking approach over just replacing
>> the setting?
RK> Genuine mocking (as opposed to this proposal) has one really big
RK> advantage - it's lightning fast. All the db-calls get faked using a
RK> cache-like setup, so it takes pretty much no time to run any db query.
RK> The cost comes in keeping the mock data source up to date.
I first thought on logging sql queries with result and feed them via mock.
But the cost of supporting this in test-first development seems to high for
me. Having to think out everything down to sql level kills most
development benefits of Django ORM for me.
Andrey Khavryuchenko
RK> On 7/29/07, Andrey Khavryuchenko <akh...@gmail.com> wrote:
>>
>> I'm not using django's testing framework for several reasons:
>> - I'm using django from 0.91 days and wasn't following django-users all
>> this way
>> - I use nose and twill for testing and they have no ready-to-use plugs
>> into django testing (or I haven't been able to find it)
RK> There aren't any built-in mechanisms for supporting nose or twill
RK> tests. There are so many 'alternative' testing frameworks - we're not
RK> going to add support for every single one of them.
I know and accept that.
RK> However, it is very simple to add support for an external testing
RK> mechanism - this is one of the original design considerations. See the
RK> following for details:
RK> http://www.djangoproject.com/documentation/testing/#using-a-different-testing-framework
I know that section exists. When I mentioned 'ready-to-use plugs' above
I've meant exactly these hooks. Someday I'll write nosetests plugin.
>> But sqlite has no concat function and straightforward approach will lead to
>> an exception. Thus
>> sqlite_conn.connection.create_function('concat', 2,
>> lambda *args: ''.join(args))
>> in DbMock setup.
RK> This is a workaround required by your specific SQL requirements.
RK> Similar problems would exist if you used Postgres TSearch2 extensions
RK> in your queries, or any other DB-specific extension. It's not really
RK> Django's responsibility to normalize every possible SQL query across
RK> every possible backend.
I know.
RK> However, it probably is Django's responsibility to provide a generic
RK> hook so that you (as an end user) can add whatever normalizations your
RK> application may require. Any suggestions on how to approach this
RK> problem would be greatfully accepted.
Well, if I'm using sqlite as a test db backend, I can add nearly any
function I need in the manner described above.
What other usecases might be?