[Django] #33537: Cloning test database fails with mysql-client 8.x and older mysql-server or mariadb-server
Django
mariadb-server
-------------------------------------+-------------------------------------
Reporter: Stephen | Owner: nobody
Finucane |
Type: Bug | Status: new
Component: Database | Version: 3.2
layer (models, ORM) |
Severity: Normal | Keywords:
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
I've been seeing the following error message when attempting to run tests
in parallel using a MySQL backend:
{{{
py36-django22 run-test-pre: PYTHONHASHSEED='1711093702'
py36-django22 run-test: commands[0] | python
/home/patchwork/patchwork/manage.py test --noinput --parallel -- patchwork
mysqldump: Couldn't execute 'SELECT COLUMN_NAME,
JSON_EXTRACT(HISTOGRAM, '$."number-of-buckets-specified"')
FROM information_schema.COLUMN_STATISTICS WHERE SCHEMA_NAME
= 'test_patchwork' AND TABLE_NAME = 'auth_group';': Unknown table
'COLUMN_STATISTICS' in information_schema (1109)
mysqldump: Couldn't execute 'SELECT COLUMN_NAME,
JSON_EXTRACT(HISTOGRAM, '$."number-of-buckets-specified"')
FROM information_schema.COLUMN_STATISTICS WHERE SCHEMA_NAME
= 'test_patchwork' AND TABLE_NAME = 'auth_group';': Unknown table
'COLUMN_STATISTICS' in information_schema (1109)
System check identified no issues (0 silenced).
Creating test database for alias 'default'...
Cloning test database for alias 'default'...
Cloning test database for alias 'default'...
...
}}}
This is followed by a lot of test failures due to missing databases.
In my instance, I traced this back to the fact that I was using `mariadb-
server` with `mysql-client` rather than `mariadb-server`. Switching the
client packages resolved the issue. However, posts on StackOverflow
suggest the same issue can occur when using older MySQL servers (5.7?)
that don't provide these statistics with newer clients so it's not as
clear cut as this.
I'm not sure there's much to be done here from a Django perspective. We
could arguably add the `-column-statistics=0` flag to the `mysqldump` call
(in `django.db.backends.mysql.creation.DatabaseCreation._clone_db`), but
this is not compatible with the version of this tool provided by `mariadb-
client`. We might also want to make this a more serious error and fail the
test run rather than attempt to proceed, obscuring the failure in the
process (this could conceivably get flaky if you had a mix of DB and non-
DB tests running in different runner instances). Perhaps the best thing to
do here is close this and leave it here as a breadcrumb for others that
run into the issue.
--
Ticket URL: <https://code.djangoproject.com/ticket/33537>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
mariadb-server
-------------------------------------+-------------------------------------
Type: Bug | Status: closed
Component: Database layer | Version: 3.2
(models, ORM) |
Severity: Normal | Resolution: invalid
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: new => closed
* resolution: => invalid
Comment:
Thanks for this ticket, however Django 2.2 doesn't officially support
MariaDB. Moreover Django 2.2 is in extended support so it doesn't receive
bugfixes anymore (except security issues). Please reopen this ticket if
you can reproduce the issue on Django 4.0 (or the current `main` branch).
I couldn't reproduce it with MySQL 8+ or MariaDB.
--
Ticket URL: <https://code.djangoproject.com/ticket/33537#comment:1>
Django
mariadb-server
-------------------------------------+-------------------------------------
Type: Bug | Status: new
Component: Database layer | Version: 3.2
(models, ORM) |
Severity: Normal | Resolution:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: closed => new
* resolution: invalid =>
Comment:
Reopening per feedback on #34083
--
Ticket URL: <https://code.djangoproject.com/ticket/33537#comment:2>
Django
mariadb-server
-------------------------------------+-------------------------------------
Type: Bug | Status: new
Component: Database layer | Version: 4.1
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* version: 3.2 => 4.1
--
Ticket URL: <https://code.djangoproject.com/ticket/33537#comment:3>
Django
mariadb-server
-------------------------------------+-------------------------------------
Type: Bug | Status: new
Component: Database layer | Version: 4.1
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Stephen Finucane):
Reproducing from #34083.
I'm attempting to run tests in parallel using MariaDB. When running tests,
all access to the additional databases fail. Scrolling back to the top, I
see the following error messages:
{{{
❯ tox -e py38-django41 -- patchwork.tests.api.test_user
patchwork.tests.api.test_event
py38-django41 run-test-pre: PYTHONHASHSEED='631429049'
py38-django41 run-test: commands[0] | python
/home/patchwork/patchwork/manage.py test --noinput --parallel -v 2
--timing -- patchwork.tests.api.test_user patchwork.tests.api.test_event
Found 25 test(s).
Creating test database for alias 'default' ('test_patchwork')...
Operations to perform:
Synchronize unmigrated apps: django_filters, humanize, messages,
rest_framework, staticfiles
Apply all migrations: admin, auth, authtoken, contenttypes, patchwork,
sessions, sites
Synchronizing apps without migrations:
Creating tables...
Running deferred SQL...
Running migrations:
Applying contenttypes.0001_initial... OK
... {skipped} ...
Applying sites.0002_alter_domain_unique... OK
Cloning test database for alias 'default' ('test_patchwork')...
mysqldump: Couldn't execute 'SELECT COLUMN_NAME,
JSON_EXTRACT(HISTOGRAM, '$."number-of-buckets-specified"')
FROM information_schema.COLUMN_STATISTICS WHERE SCHEMA_NAME
= 'test_patchwork' AND TABLE_NAME = 'auth_group';': Unknown table
'COLUMN_STATISTICS' in information_schema (1109)
Cloning test database for alias 'default' ('test_patchwork')...
mysqldump: Couldn't execute 'SELECT COLUMN_NAME,
JSON_EXTRACT(HISTOGRAM, '$."number-of-buckets-specified"')
FROM information_schema.COLUMN_STATISTICS WHERE SCHEMA_NAME
= 'test_patchwork' AND TABLE_NAME = 'auth_group';': Unknown table
'COLUMN_STATISTICS' in information_schema (1109)
System check identified no issues (0 silenced).
...
}}}
The same issue happens on multiple Django versions. With a bit of hacking
on the code in the tox venv, I was able to inspect the command that Django
is actually executing:
{{{
Dumping! cmd = mysqldump --user=patchwork --host=localhost --routines
--events test_patchwork
}}}
Executing this locally, I see the same issue:
{{{
❯ MYSQL_PWD=password mysqldump --user=patchwork --host=localhost
--routines --events patchwork
-- MySQL dump 10.13 Distrib 8.0.30, for Linux (x86_64)
--
-- Host: localhost Database: patchwork
-- ------------------------------------------------------
-- Server version 5.5.5-10.5.16-MariaDB
/*!40101 SET @OLD_CHARACTER_SET_CLIENT=@@CHARACTER_SET_CLIENT */;
/*!40101 SET @OLD_CHARACTER_SET_RESULTS=@@CHARACTER_SET_RESULTS */;
/*!40101 SET @OLD_COLLATION_CONNECTION=@@COLLATION_CONNECTION */;
/*!50503 SET NAMES utf8mb4 */;
/*!40103 SET @OLD_TIME_ZONE=@@TIME_ZONE */;
/*!40103 SET TIME_ZONE='+00:00' */;
/*!40014 SET @OLD_UNIQUE_CHECKS=@@UNIQUE_CHECKS, UNIQUE_CHECKS=0 */;
/*!40014 SET @OLD_FOREIGN_KEY_CHECKS=@@FOREIGN_KEY_CHECKS,
FOREIGN_KEY_CHECKS=0 */;
/*!40101 SET @OLD_SQL_MODE=@@SQL_MODE, SQL_MODE='NO_AUTO_VALUE_ON_ZERO'
*/;
/*!40111 SET @OLD_SQL_NOTES=@@SQL_NOTES, SQL_NOTES=0 */;
--
-- Table structure for table `auth_group`
--
DROP TABLE IF EXISTS `auth_group`;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `auth_group` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(150) NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `name` (`name`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
/*!40101 SET character_set_client = @saved_cs_client */;
--
-- Dumping data for table `auth_group`
--
LOCK TABLES `auth_group` WRITE;
/*!40000 ALTER TABLE `auth_group` DISABLE KEYS */;
/*!40000 ALTER TABLE `auth_group` ENABLE KEYS */;
UNLOCK TABLES;
mysqldump: Couldn't execute 'SELECT COLUMN_NAME,
JSON_EXTRACT(HISTOGRAM, '$."number-of-buckets-specified"')
FROM information_schema.COLUMN_STATISTICS WHERE SCHEMA_NAME
= 'patchwork' AND TABLE_NAME = 'auth_group';': Unknown table
'COLUMN_STATISTICS' in information_schema (1109)
❯ echo $?
2
}}}
I'm using Fedora 36 with the default packages:
{{{
❯ cat /etc/system-release
Fedora release 36 (Thirty Six)
❯ sudo dnf list installed | grep mariadb
mariadb-backup.x86_64 3:10.5.16-1.fc36
@updates
mariadb-common.x86_64 3:10.5.16-1.fc36
@updates
mariadb-connector-c.x86_64 3.2.7-1.fc36
@updates
mariadb-connector-c-config.noarch 3.2.7-1.fc36
@updates
mariadb-connector-c-devel.x86_64 3.2.7-1.fc36
@updates
mariadb-connector-c-doc.noarch 3.2.7-1.fc36
@updates
mariadb-cracklib-password-check.x86_64 3:10.5.16-1.fc36
@updates
mariadb-devel.x86_64 3:10.5.16-1.fc36
@updates
mariadb-errmsg.x86_64 3:10.5.16-1.fc36
@updates
mariadb-gssapi-server.x86_64 3:10.5.16-1.fc36
@updates
mariadb-server.x86_64 3:10.5.16-1.fc36
@updates
mariadb-server-utils.x86_64 3:10.5.16-1.fc36
@updates
❯ sudo dnf list installed | grep mysql
community-mysql.x86_64 8.0.30-2.fc36
@updates
community-mysql-common.x86_64 8.0.30-2.fc36
@updates
mysql-selinux.noarch 1.0.5-1.fc36
@updates
}}}
I had previously reported a similar issue using a combination of `mysql-
client` with an older version of `mysql-server` or `mariadb-server`. This
was reported as #33537. However, I'm now seeing the issue with the latest
versions of both the server and client. As noted in the issue, I can work
around this on the command line by adding the `--column-statistics=0`
argument to `mysqldump`, however, there doesn't appear to be any mechanism
to do this in Django. Alternatively, I can add the following to `my.cnf`
(or equivalent):
{{{
/etc/my.cnf.d/mysqldump.cnf
[mysqldump]
column-statistics=0
}}}
There's a chance that someone has packaged something wrong in Fedora land
and that I'm using an incompatible client and server combo (MySQL-derived
client, MariaDB server). However, it's surprising that the tests proceeded
to run despite the DB clone failing. As you can see above, `mysqldump`
returned a non-zero exit code so there doesn't appear to be any reason
Django couldn't have stopped immediately. I think it would make sense to
fail fast at a minimum. If we wanted, we could also add the `--column-
statistics=0` argument to our call to `mysqldump` (or provide a mechanism
for users to do so), however, this isn't compatible with the 5.7.x version
of `mysqldump` or the version provided by e.g. the `mariadb-client`
package on Ubuntu) so this would have to be a version-specific check.
--
Ticket URL: <https://code.djangoproject.com/ticket/33537#comment:4>
Django
mariadb-server
-------------------------------------+-------------------------------------
Type: Bug | Status: new
Component: Database layer | Version: 4.1
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Mariusz Felisiak):
Your issue looks like a misconfiguration and such it is still invalid.
Do you want to propose adding `check=True` to subprocess calls and change
this ticket to a cleanup?
--
Ticket URL: <https://code.djangoproject.com/ticket/33537#comment:5>
Django
mariadb-server
-------------------------------------+-------------------------------------
Type: Bug | Status: new
Component: Database layer | Version: 4.1
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Stephen Finucane):
Replying to [comment:5 Mariusz Felisiak]:
> Your issue looks like a misconfiguration and such it is still invalid.
>
> Do you want to propose adding `check=True` to subprocess calls and
change this ticket to a cleanup?
Yes, exactly (though not with `check=True` as I think that's only
supported by `subprocess.run`?). I've submitted a patch that should
address this.
--
Ticket URL: <https://code.djangoproject.com/ticket/33537#comment:6>
Django
-------------------------------------+-------------------------------------
Reporter: Stephen Finucane | Owner: nobody
Cleanup/optimization |
Component: Database layer | Version: 4.1
(models, ORM) |
Severity: Normal | Resolution:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* type: Bug => Cleanup/optimization
* stage: Unreviewed => Accepted
--
Ticket URL: <https://code.djangoproject.com/ticket/33537#comment:7>
Django
-------------------------------------+-------------------------------------
Type: | Finucane
Cleanup/optimization | Status: assigned
Component: Database layer | Version: 4.1
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* owner: nobody => Stephen Finucane
* needs_better_patch: 0 => 1
* has_patch: 0 => 1
* status: new => assigned
Comment:
[https://github.com/django/django/pull/16170 PR]
--
Ticket URL: <https://code.djangoproject.com/ticket/33537#comment:8>
Django
-------------------------------------+-------------------------------------
Type: | Status: new
Cleanup/optimization |
Component: Database layer | Version: 4.1
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* owner: Stephen Finucane => (none)
* status: assigned => new
--
Ticket URL: <https://code.djangoproject.com/ticket/33537#comment:9>
Django
-------------------------------------+-------------------------------------
Reporter: Stephen Finucane | Owner: (none)
Type: | Status: new
Cleanup/optimization |
Component: Database layer | Version: 4.1
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 1
-------------------------------------+-------------------------------------
* cc: Sage Abdullah (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/33537#comment:10>
Django
-------------------------------------+-------------------------------------
Reporter: Stephen Finucane | Owner: (none)
Type: | Status: new
Cleanup/optimization |
Component: Database layer | Version: 4.1
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 1 => 0
Comment:
There is a new PR for this, and looks like it hasn't been reviewed. Please
correct me if I'm wrong. Thanks!
[https://github.com/django/django/pull/18436 PR]
--
Ticket URL: <https://code.djangoproject.com/ticket/33537#comment:11>
Django
-------------------------------------+-------------------------------------
Type: | Status: assigned
Component: Database layer | Version: 4.1
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* owner: (none) => Faakhir30
Ticket URL: <https://code.djangoproject.com/ticket/33537#comment:12>
Django
-------------------------------------+-------------------------------------
Reporter: Stephen Finucane | Owner: Faakhir30
Type: | Status: assigned
Cleanup/optimization |
Component: Database layer | Version: 4.1
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 1
-------------------------------------+-------------------------------------
* needs_better_patch: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/33537#comment:13>
Django
-------------------------------------+-------------------------------------
Type: | Felisiak
Cleanup/optimization | Status: assigned
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 1 => 0
Comment:
[https://github.com/django/django/pull/19276 New PR]
--
Ticket URL: <https://code.djangoproject.com/ticket/33537#comment:14>
Django
-------------------------------------+-------------------------------------
Reporter: Stephen Finucane | Owner: Mariusz
Type: | Felisiak
Cleanup/optimization | Status: assigned
Component: Database layer | Version: 4.1
(models, ORM) |
Severity: Normal | Resolution:
| checkin
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* stage: Accepted => Ready for checkin
--
Ticket URL: <https://code.djangoproject.com/ticket/33537#comment:15>
Django
-------------------------------------+-------------------------------------
Reporter: Stephen Finucane | Owner: Mariusz
Type: | Felisiak
(models, ORM) |
| checkin
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* resolution: => fixed
* status: assigned => closed
Comment:
In [changeset:"1823a8011346afc280e0efaf8cb7c086b148f881" 1823a801]:
{{{#!CommitTicketReference repository=""
revision="1823a8011346afc280e0efaf8cb7c086b148f881"
Fixed #33537 -- Made test database cloning on MySQL reraise unexpected
errors.
Thanks Faakhir Zahid and Stephen Finucane for the initial patch.
Thanks Simon Charette for the review.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/33537#comment:16>