[Django] #28153: patch: it's now easier to populate data databases in test databases that run code in parallel

[Django]

#28153: patch: it's now easier to populate data databases in test databases that
run code in parallel
---------------------------------------------+------------------------
Reporter: Marcos Diez | Owner: nobody
Type: New feature | Status: new
Component: Testing framework | Version: master
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 1
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
---------------------------------------------+------------------------
Although Django makes very easy for one to extend `
django.test.runner.DiscoverRunner ` , it's ` setup_databases() ` does too
much.

Currently, it

- creates all the test databases (for single thread unit tests)
- duplicates all the test databases (in case of parallel unit tests)

In case I am running not running tests in parallel, I can just populate
the DB after running unit tests without any issues.

But if I care about my time and want to run tests in parallel, I can
either:

a) populate my data after setup_databases() is executed, once for each
thread of the parallel tests, which is slow
b) get my hands dirty and reimplement ` setup_databases() `

I propose (and I am sending the code to do so) a better solution. We just
have to break `setup_databases()` in 3 functions:

`DiscoverRunner.prepare_databases() `
`DiscoverRunner.populate_databases() # noop by default `
`DiscoverRunner.duplicate_databases_if_necessary()`

The idea is quite simple: in order to be backward compatible, `
setup_databases() `, will still exist but only call three functions above
in that order.

The first function will create all the test databases necessary for non
parallel tests to run.

` populate_databases() `, which should be a no op, can be overwritten by
the user who extends ` django.test.runner.DiscoverRunner ` so his/her
data can be populated

Afterwards, all the test DBs are copied as many times as necessary in case
parallel tests are run via `
DiscoverRunner.duplicate_databases_if_necessary() `

I believe this change on Django will have no downside, will be backward
compatible and help people who needs to populate real data on the DB for
their tests.

--
Ticket URL: <https://code.djangoproject.com/ticket/28153>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#28153: patch: it's now easier to populate data databases in test databases that
run code in parallel

-----------------------------------+--------------------------------------

Reporter: Marcos Diez | Owner: nobody
Type: New feature | Status: new
Component: Testing framework | Version: master

Severity: Normal | Resolution:

Keywords: | Triage Stage: Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

-----------------------------------+--------------------------------------
Description changed by Marcos Diez:

Old description:

New description:

Currently, it

The PR with the changes for this to work is available here:
https://github.com/django/django/pull/8437

--

--
Ticket URL: <https://code.djangoproject.com/ticket/28153#comment:1>

[Django]

#28153: patch: it's now easier to populate data in test databases that run code in
parallel
-----------------------------------+--------------------------------------

Reporter: Marcos Diez | Owner: nobody
Type: New feature | Status: new
Component: Testing framework | Version: master

Severity: Normal | Resolution:

Keywords: | Triage Stage: Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

-----------------------------------+--------------------------------------

--
Ticket URL: <https://code.djangoproject.com/ticket/28153#comment:2>

[Django]

#28153: Add a method to DiscoverRunner to allow populating a database with test
data
-----------------------------------+--------------------------------------

Reporter: Marcos Diez | Owner: nobody

Type: New feature | Status: closed

Component: Testing framework | Version: master

Severity: Normal | Resolution: wontfix

Keywords: | Triage Stage: Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

-----------------------------------+--------------------------------------
Changes (by Tim Graham):

* status: new => closed
* resolution: => wontfix

Comment:

The [https://groups.google.com/d/topic/django-developers/qjQgcsDT-
Ig/discussion django-developers discussion] hasn't yielded consensus about
this (we can reopen the ticket if that consensus changes). A suggested
alternative is to use the `post_migrate` handler during testing that loads
the data.

--
Ticket URL: <https://code.djangoproject.com/ticket/28153#comment:3>