Less fixtures less inserts less time running tests

[Peter Bengtsson]

One of my goal is to make air mozilla more pleasant to work on (read: to
contribute to).

So I injected some code into the db backend to record every single INSERT
command sent when running the whole test suite.

I found that a lot of it was inserting fixtures. Some that I know aren't
needed in every test.
So I wrote this:
https://github.com/mozilla/airmozilla/commit/4aa24dac7ce92013ec962871b95a73219857b419#diff-3

Now instead of 9071 inserts, it only does 6357.

I also noticed that the most common insert is that for django_sessions so
switching to `SESSION_ENGINE = 'django.contrib.sessions.backends.cache'`
it's now down to 5747 inserts only.

This makes the test go from around 54 seconds down to 48 seconds (repeated
runs) which isn't a huge difference but it's a start. Every little counts.

--
Peter Bengtsson
Mozilla Web Engineering

[Michael Cooper]

Yay for making tests faster!

Could you share the code you injected to record the INSERT commands? That
sounds pretty useful, and I'd love to see what it says about SUMO.

On Tue, Jan 27, 2015 at 3:26 PM, Peter Bengtsson <pbeng...@mozilla.com>
wrote:

> _______________________________________________
> dev-webdev mailing list
> dev-w...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-webdev
>

[Peter Bengtsson]

It's crude but it works.

In django/db/backends/postgresql_psycopg2/base.py


class CursorWrapper(object):
"""
A thin wrapper around psycopg2's normal cursor class so that we can
catch
particular exception instances and reraise them with the right types.
"""

def __init__(self, cursor):
self.cursor = cursor

def execute(self, query, args=None):
#if 'INSERT' in query:
# with open('/tmp/sqls.log', 'a') as f:
# f.write(query+'\n')
...


Uncomment those lines and that'll create a file you can run this over
https://gist.github.com/peterbe/b857c7329ac23bef768d

[Jannis Leidel]

> On 28 Jan 2015, at 00:26, Peter Bengtsson <pbeng...@mozilla.com> wrote:
>
> One of my goal is to make air mozilla more pleasant to work on (read: to
> contribute to).
>
> So I injected some code into the db backend to record every single INSERT
> command sent when running the whole test suite.
>
> I found that a lot of it was inserting fixtures. Some that I know aren't
> needed in every test.
> So I wrote this:
> https://github.com/mozilla/airmozilla/commit/4aa24dac7ce92013ec962871b95a73219857b419#diff-3
>
> Now instead of 9071 inserts, it only does 6357.
>
> I also noticed that the most common insert is that for django_sessions so
> switching to `SESSION_ENGINE = 'django.contrib.sessions.backends.cache'`
> it's now down to 5747 inserts only.
>
> This makes the test go from around 54 seconds down to 48 seconds (repeated
> runs) which isn't a huge difference but it's a start. Every little counts.

To give a little insight in a soon to be released feature in Django 1.8:

The TestCase class gains a setUpTestData() method that is called once per class and can be used to create database content without worrying if it'll slow down tests:

https://docs.djangoproject.com/en/dev/topics/testing/tools/#django.test.TestCase.setUpTestData

Also, fixture loading is now finally being done once per test class.

https://docs.djangoproject.com/en/dev/releases/1.8/#testcase-data-setup

Jannis

[Peter Bengtsson]

Can't wait!
...to move to python 2.7 :)

[Andy McKay]

I’ve used https://github.com/andymckay/django-statsd <https://github.com/andymckay/django-statsd> for profiling tests, it has hooks for a bunch of things, including model changes, celery, caches and so on: https://django-statsd.readthedocs.org/en/latest/#django-model-save-and-delete-integration <https://django-statsd.readthedocs.org/en/latest/#django-model-save-and-delete-integration> and you can point the output anywhere you want.

I wrote about it in a series of blog posts here: http://www.agmweb.ca/2012-11-20-declaring-war-on-the-zamboni-test-suite-pt-1/

I’d also point out https://github.com/andymckay/nose-timing <https://github.com/andymckay/nose-timing> which tells you which tests are slow.

> On Jan 27, 2015, at 3:52 PM, Michael Cooper <mco...@mozilla.com> wrote:
>
> Yay for making tests faster!
>
> Could you share the code you injected to record the INSERT commands? That
> sounds pretty useful, and I'd love to see what it says about SUMO.
>

> On Tue, Jan 27, 2015 at 3:26 PM, Peter Bengtsson <pbeng...@mozilla.com>

> wrote:
>
>> One of my goal is to make air mozilla more pleasant to work on (read: to
>> contribute to).
>>
>> So I injected some code into the db backend to record every single INSERT
>> command sent when running the whole test suite.
>>
>> I found that a lot of it was inserting fixtures. Some that I know aren't
>> needed in every test.
>> So I wrote this:
>>
>> https://github.com/mozilla/airmozilla/commit/4aa24dac7ce92013ec962871b95a73219857b419#diff-3
>>
>> Now instead of 9071 inserts, it only does 6357.
>>
>> I also noticed that the most common insert is that for django_sessions so
>> switching to `SESSION_ENGINE = 'django.contrib.sessions.backends.cache'`
>> it's now down to 5747 inserts only.
>>
>> This makes the test go from around 54 seconds down to 48 seconds (repeated
>> runs) which isn't a huge difference but it's a start. Every little counts.
>>

>> --
>> Peter Bengtsson
>> Mozilla Web Engineering

[Peter Bengtsson]

Awesome stuff.
Just FYI, nose-timing is awesome but so is nose-timer.
It just gives you a sorted list of the times of each test. Makes it easy to
quickly find out which ones to work on first.

Did you write some tools that "visualize" those json files your plugin
creates?

On Wed, Jan 28, 2015 at 9:53 AM, Andy McKay <amc...@mozilla.com> wrote:

> I’ve used https://github.com/andymckay/django-statsd for profiling tests,

> it has hooks for a bunch of things, including model changes, celery, caches
> and so on:

> https://django-statsd.readthedocs.org/en/latest/#django-model-save-and-delete-integration and

> you can point the output anywhere you want.
>
> I wrote about it in a series of blog posts here:
> http://www.agmweb.ca/2012-11-20-declaring-war-on-the-zamboni-test-suite-pt-1/
>

> I’d also point out https://github.com/andymckay/nose-timing which tells