[Django] #30636: mysqldump errors out with "unknown variable" when --parallel and --defaults-file
Django
--defaults-file
-------------------------------------+-------------------------------------
Reporter: fallen | Owner: nobody
Type: Bug | Status: new
Component: Database | Version: master
layer (models, ORM) |
Severity: Normal | Keywords:
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
What happens?
Running tests gives:
Creating test database for alias 'default'...
Cloning test database for alias 'default'...
mysqldump: unknown variable 'defaults-
file=/var/lib/jenkins/workspace/Pytition_master/my.cnf'
And then every test fails with 'E' (error).
How to reproduce?
1/ Configure your database like this:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'OPTIONS': {
'read_default_file': '/path/to/my.cnf', # file with
credentials and db name
},
}
}
2/ run tests with "manage.py test --parallel 2 appname"
The important here is "--parallel 2" (or more than 2)
Explanations?
This happens because Django tries to clone the database using mysqldump:
https://github.com/django/django/blob/master/django/db/backends/mysql/creation.py#L60
In my case I've printed the arguments, it gives:
dump_cmd: ['mysqldump', '--routines', '--events', '--defaults-
file=/var/lib/jenkins/workspace/Pytition_master/my.cnf', 'test_']
load_cmd: ['mysql', '--defaults-
file=/var/lib/jenkins/workspace/Pytition_master/my.cnf', 'test__1']
mysqldump: unknown variable 'defaults-
file=/var/lib/jenkins/workspace/Pytition_master/my.cnf'
However, it seems that, even if this is undocumented in mysqldump's man
page, it does only support --defaults-file parameter when in first
position.
See the Note on this StackOverflow answer:
https://stackoverflow.com/questions/10725209/specifying-separate-config-
file-on-mysqldump-command-line
Proposed solution:
Test if read_default_file is in settings, and in this case put it as first
argument to mysqldump
--
Ticket URL: <https://code.djangoproject.com/ticket/30636>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
--defaults-file
-------------------------------------+-------------------------------------
Reporter: fallen | Owner: nobody
Type: Bug | Status: new
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Old description:
New description:
Explanations?
--
--
Ticket URL: <https://code.djangoproject.com/ticket/30636#comment:1>
Django
--defaults-file
-------------------------------------+-------------------------------------
Reporter: fallen | Owner: nobody
Type: Bug | Status: new
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Old description:
> What happens?
New description:
Explanations?
}}}
--
--
Ticket URL: <https://code.djangoproject.com/ticket/30636#comment:2>
Django
--defaults-file
-------------------------------------+-------------------------------------
Reporter: fallen | Owner: nobody
Type: Bug | Status: new
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
Changes (by fallen):
* easy: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/30636#comment:3>
Django
--defaults-file
-------------------------------------+-------------------------------------
Reporter: fallen | Owner: nobody
Type: Bug | Status: new
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
Comment (by fallen):
It seems that the same requirement exists for "mariabackup" (the
equivalent tool to mysqldump for MariaDB).
See https://mariadb.com/kb/en/library/mariabackup-overview/
{{{
The following options relate to how MariaDB command-line tools handles
option files. They must be given as the first argument on the command-
line:
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/30636#comment:4>
Django
--defaults-file
-------------------------------------+-------------------------------------
Reporter: fallen | Owner: nobody
Type: Bug | Status: new
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
Changes (by fallen):
* has_patch: 0 => 1
Comment:
I propose this patch to fix the issue:
https://github.com/django/django/pull/11567
--
Ticket URL: <https://code.djangoproject.com/ticket/30636#comment:5>
Django
--defaults-file
-------------------------------------+-------------------------------------
Type: Bug | Status: assigned
Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
Changes (by fallen):
* owner: nobody => fallen
* status: new => assigned
--
Ticket URL: <https://code.djangoproject.com/ticket/30636#comment:6>
Django
-------------------------------------+-------------------------------------
Reporter: fallen | Owner: fallen
Type: Bug | Status: assigned
Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution:
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* keywords: => mysqldump
* easy: 1 => 0
* stage: Unreviewed => Accepted
--
Ticket URL: <https://code.djangoproject.com/ticket/30636#comment:7>
Django
-------------------------------------+-------------------------------------
Reporter: fallen | Owner: fallen
Component: Database layer | Version: master
(models, ORM) |
Keywords: mysqldump | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"e47b8293a793653bd7df795c1d9f0381f3f22cf7" e47b8293]:
{{{
#!CommitTicketReference repository=""
revision="e47b8293a793653bd7df795c1d9f0381f3f22cf7"
Fixed #30636 -- Fixed options ordering when cloning test database on
MySQL.
--defaults-file must be given before other options.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/30636#comment:8>
Django
-------------------------------------+-------------------------------------
| Sionneau
Type: Bug | Status: closed
Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution: fixed
Keywords: mysqldump | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Floréal Cabanettes):
I have still the problem with django 2.2.11, is the problem only solved on
django 3.0?
--
Ticket URL: <https://code.djangoproject.com/ticket/30636#comment:9>
Django
-------------------------------------+-------------------------------------
Reporter: Yann Sionneau | Owner: Yann
| Sionneau
Type: Bug | Status: closed
Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution: fixed
Keywords: mysqldump | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Floréal Cabanettes):
Effectively, the commit is only for 3.0. If I apply change to my 2.2.11
django it works. Is it possible to include this fix officially in the LTS
2.2 series?
--
Ticket URL: <https://code.djangoproject.com/ticket/30636#comment:10>
Django
-------------------------------------+-------------------------------------
Reporter: Yann Sionneau | Owner: Yann
| Sionneau
Type: Bug | Status: closed
Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution: fixed
Keywords: mysqldump | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Simon Charette):
This commit won't be backported to 2.2.x per
[https://docs.djangoproject.com/en/3.0/internals/release-process
/#supported-versions-policy our backporting policy].
--
Ticket URL: <https://code.djangoproject.com/ticket/30636#comment:11>