[Django] #34083: Cloning test database fails with mariadb-server 8.x
Django
-------------------------------------+-------------------------------------
Reporter: Stephen | Owner: nobody
Finucane |
Type: Bug | Status: new
Component: Database | Version: 4.1
layer (models, ORM) |
Severity: Normal | Keywords:
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
I'm attempting to run tests in parallel using MariaDB. When running tests,
all access to the additional databases fail. Scrolling back to the top, I
see the following error messages:
{{{
❯ tox -e py38-django41 -- patchwork.tests.api.test_user
patchwork.tests.api.test_event
py38-django41 run-test-pre: PYTHONHASHSEED='631429049'
py38-django41 run-test: commands[0] | python
/home/patchwork/patchwork/manage.py test --noinput --parallel -v 2
--timing -- patchwork.tests.api.test_user patchwork.tests.api.test_event
Found 25 test(s).
Creating test database for alias 'default' ('test_patchwork')...
Operations to perform:
Synchronize unmigrated apps: django_filters, humanize, messages,
rest_framework, staticfiles
Apply all migrations: admin, auth, authtoken, contenttypes, patchwork,
sessions, sites
Synchronizing apps without migrations:
Creating tables...
Running deferred SQL...
Running migrations:
Applying contenttypes.0001_initial... OK
... {skipped} ...
Applying sites.0002_alter_domain_unique... OK
Cloning test database for alias 'default' ('test_patchwork')...
mysqldump: Couldn't execute 'SELECT COLUMN_NAME,
JSON_EXTRACT(HISTOGRAM, '$."number-of-buckets-specified"')
FROM information_schema.COLUMN_STATISTICS WHERE SCHEMA_NAME
= 'test_patchwork' AND TABLE_NAME = 'auth_group';': Unknown table
'COLUMN_STATISTICS' in information_schema (1109)
Cloning test database for alias 'default' ('test_patchwork')...
mysqldump: Couldn't execute 'SELECT COLUMN_NAME,
JSON_EXTRACT(HISTOGRAM, '$."number-of-buckets-specified"')
FROM information_schema.COLUMN_STATISTICS WHERE SCHEMA_NAME
= 'test_patchwork' AND TABLE_NAME = 'auth_group';': Unknown table
'COLUMN_STATISTICS' in information_schema (1109)
System check identified no issues (0 silenced).
...
}}}
The same issue happens on multiple Django versions. With a bit of hacking
on the code in the tox venv, I was able to inspect the command that Django
is actually executing:
{{{
Dumping! cmd = mysqldump --user=patchwork --host=localhost --routines
--events test_patchwork
}}}
Executing this locally, I see the same issue:
{{{
❯ MYSQL_PWD=password mysqldump --user=patchwork --host=localhost
--routines --events patchwork
-- MySQL dump 10.13 Distrib 8.0.30, for Linux (x86_64)
--
-- Host: localhost Database: patchwork
-- ------------------------------------------------------
-- Server version 5.5.5-10.5.16-MariaDB
/*!40101 SET @OLD_CHARACTER_SET_CLIENT=@@CHARACTER_SET_CLIENT */;
/*!40101 SET @OLD_CHARACTER_SET_RESULTS=@@CHARACTER_SET_RESULTS */;
/*!40101 SET @OLD_COLLATION_CONNECTION=@@COLLATION_CONNECTION */;
/*!50503 SET NAMES utf8mb4 */;
/*!40103 SET @OLD_TIME_ZONE=@@TIME_ZONE */;
/*!40103 SET TIME_ZONE='+00:00' */;
/*!40014 SET @OLD_UNIQUE_CHECKS=@@UNIQUE_CHECKS, UNIQUE_CHECKS=0 */;
/*!40014 SET @OLD_FOREIGN_KEY_CHECKS=@@FOREIGN_KEY_CHECKS,
FOREIGN_KEY_CHECKS=0 */;
/*!40101 SET @OLD_SQL_MODE=@@SQL_MODE, SQL_MODE='NO_AUTO_VALUE_ON_ZERO'
*/;
/*!40111 SET @OLD_SQL_NOTES=@@SQL_NOTES, SQL_NOTES=0 */;
--
-- Table structure for table `auth_group`
--
DROP TABLE IF EXISTS `auth_group`;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `auth_group` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(150) NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `name` (`name`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
/*!40101 SET character_set_client = @saved_cs_client */;
--
-- Dumping data for table `auth_group`
--
LOCK TABLES `auth_group` WRITE;
/*!40000 ALTER TABLE `auth_group` DISABLE KEYS */;
/*!40000 ALTER TABLE `auth_group` ENABLE KEYS */;
UNLOCK TABLES;
mysqldump: Couldn't execute 'SELECT COLUMN_NAME,
JSON_EXTRACT(HISTOGRAM, '$."number-of-buckets-specified"')
FROM information_schema.COLUMN_STATISTICS WHERE SCHEMA_NAME
= 'patchwork' AND TABLE_NAME = 'auth_group';': Unknown table
'COLUMN_STATISTICS' in information_schema (1109)
❯ echo $?
2
}}}
I'm using Fedora 36 with the default packages:
{{{
❯ cat /etc/system-release
Fedora release 36 (Thirty Six)
❯ sudo dnf list installed | grep mariadb
mariadb-backup.x86_64 3:10.5.16-1.fc36
@updates
mariadb-common.x86_64 3:10.5.16-1.fc36
@updates
mariadb-connector-c.x86_64 3.2.7-1.fc36
@updates
mariadb-connector-c-config.noarch 3.2.7-1.fc36
@updates
mariadb-connector-c-devel.x86_64 3.2.7-1.fc36
@updates
mariadb-connector-c-doc.noarch 3.2.7-1.fc36
@updates
mariadb-cracklib-password-check.x86_64 3:10.5.16-1.fc36
@updates
mariadb-devel.x86_64 3:10.5.16-1.fc36
@updates
mariadb-errmsg.x86_64 3:10.5.16-1.fc36
@updates
mariadb-gssapi-server.x86_64 3:10.5.16-1.fc36
@updates
mariadb-server.x86_64 3:10.5.16-1.fc36
@updates
mariadb-server-utils.x86_64 3:10.5.16-1.fc36
@updates
❯ sudo dnf list installed | grep mysql
community-mysql.x86_64 8.0.30-2.fc36
@updates
community-mysql-common.x86_64 8.0.30-2.fc36
@updates
mysql-selinux.noarch 1.0.5-1.fc36
@updates
}}}
I had previously reported a similar issue using a combination of `mysql-
client` with an older version of `mysql-server` or `mariadb-server`. This
was reported as #33537. However, I'm now seeing the issue with the latest
versions of both the server and client. As noted in the issue, I can work
around this on the command line by adding the `--column-statistics=0`
argument to `mysqldump`, however, there doesn't appear to be any mechanism
to do this in Django. Alternatively, I can add the following to `my.cnf`
(or equivalent):
{{{
/etc/my.cnf.d/mysqldump.cnf
[mysqldump]
column-statistics=0
}}}
There's a chance that someone has packaged something wrong in Fedora land
and that I'm using an incompatible client and server combo (MySQL-derived
client, MariaDB server). However, it's surprising that the tests proceeded
to run despite the DB clone failing. As you can see above, `mysqldump`
returned a non-zero exit code so there doesn't appear to be any reason
Django couldn't have stopped immediately. I think it would make sense to
fail fast at a minimum. If we wanted, we could also add the `--column-
statistics=0` argument to our call to `mysqldump` (or provide a mechanism
for users to do so), however, this isn't compatible with the 5.7.x version
of `mysqldump` or the version provided by e.g. the `mariadb-client`
package on Ubuntu) so this would have to be a version-specific check.
--
Ticket URL: <https://code.djangoproject.com/ticket/34083>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
-------------------------------------+-------------------------------------
Type: Bug | Status: closed
Component: Database layer | Version: 4.1
(models, ORM) |
Severity: Normal | Resolution: duplicate
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: new => closed
* resolution: => duplicate
Comment:
There is no reason to open a duplicate. Please add a comment to the
original ticket. Also, I don't think you've explained the issue in enough
detail to confirm a bug in Django. It looks like a misconfiguration.
Please reopen the ticket (#33537) if you can debug your issue and provide
details about why and where Django is at fault.
We can also consider your suggestion to check if `returncode` is non-zero.
Personally, I think it makes sense. Please suggest this in the original
ticket.
Duplicate of #33537.
--
Ticket URL: <https://code.djangoproject.com/ticket/34083#comment:1>