MariaDB, official support

[Mads Jensen]

Hi,

I got interest in contributing to Django. I started working on a bigger
ticket than the smaller contributions I've previously made.

Adam Chainz pointed out that MariaDB will include support for window
expressions (#26608) from 10.2. I asked on #django-dev what the official
policy for MariaDB is, and was told it's ad-hoc. Also, the test
environment does include MariaDB which to me would make sense to add. I
think the implemented features for MySQL is also available for MariaDB.

I know the existence of django-mysql.
--
Med venlig hilsen / Kind regards,
Mads Jensen

Saajan Fernandes: I think we forget things if there is nobody to
tell them.
-- The Lunchbox (2013)

[Josh Smeaton]

I believe Adam is (or is planning..) writing a DB backend that includes MySQL and MariaDB. Adam, did I remember correctly? If so, are you going to suggest adding this new backend to core as a replacement? Mads, perhaps you can help with this effort?

Regards,

[Adam Johnson]

Hi,

Sorry I missed this thread!

Adam is (or is planning..) writing a DB backend that includes MySQL and MariaDB

Not technically, I'm actually writing a backend to use the newer MySQL DBAPI library pymysql at https://github.com/adamchainz/django-pymysql-backend . For the first pass at least it won't do anything differently on MariaDB. I haven't worked on it since before DUTH unfortunately :(

As for whether Django core should have a support policy for MariaDB, I think it definitely should do. I also think it's probably best if it's done with switches *inside* the MySQL backend, rather than as a separate backend. Now here's why:

1. MariaDB is MySQL on some Linux distros, e.g. Red Hat. When you install the mysql package, you get MariaDB. These are listed at https://mariadb.com/kb/en/mariadb/distributions-which-include-mariadb/ . Thus some users don't even know they're using MariaDB instead of MySQL.
2. MariaDB may have a smaller user base at current, but it's growing pretty quickly compared to MySQL (though I guess it's hard to grow when you're so popular). See http://db-engines.com/en/ranking_trend/system/MariaDB versus http://db-engines.com/en/ranking_trend/system/MySQL .
3. The policy of creating one tool that works on both MySQL + MariaDB, with detection for which it is, is pretty much the standard practice in the community, e.g. this line in the popular mydumper utility: https://bazaar.launchpad.net/~max-bubenick/mydumper/0.9/view/head:/mydumper.c#L334
4. I don't expect there too be too many MariaDB specific checks in Django, since MariaDB still keeps MySQL compatibility. The TEXT / BLOB column defaults ( https://code.djangoproject.com/ticket/27676 ) are one example of a relatively small change.

[Claude Paroz]

Hi,

I would like to push a bit for that functionality in Django 2.1. Adam, any progress?

In https://github.com/django/django/pull/7778, you talked about a better plan. Show us your plan, please :-)

Claude

[Adam Johnson]

Hi Claude

Afraid I haven't made any progress, I don't get to contribute to Django at work these days which has slowed me down :(

So the sketchy plan I've had in my mind for some time is:

1) Get testing with all supported MariaDB versions (currently 5.5, 10.0, 10.1, 10.2) into Django CI, and fix any problems

2) Document that MariaDB is supported

3) Do not add a new database backend for MariaDB. Instead have the MySQL backend determine which it's talking to and act appropriately. I suspect a large percentage of MariaDB users aren't even aware they're using as they just did yum install mysql or w/e. Also the differences are generally minor, and the whole MySQL ecosystem is developing tools that work against both.

4) Add mysql_flavor (or something like that) to the MySQL database backend, and then update all uses of mysql_version to be like if mysql_flavor == 'oracle' and mysql_version >= (5, 6) or mysql_flavor == 'mariadb' and mysql_version >= (10, 1) - as appropriate for the feature in question.

5) Look at supporting pymysql instead of MySQLdb - it's better maintained, pure python so not compiled against libmysqlclient, and better tested with MariaDB. It should be fairly easy to drop in, but when I gave it a go >1 year ago it threw some non-simple errors.

Of course they can't really be done in order as they all interdepend, except #5 can probably be done separately.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscribe@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/2c525ee9-ed0c-4b4b-9dd2-f4750496b2c1%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Adam

[Carlton Gibson]

This seems a bit more pressing now: 2.0.7 broke MariaDB compatibility. 

https://code.djangoproject.com/ticket/29544

I'm sure that can be fixed. But we're in a tricky position now. 

Claude has marked the ticket as a Release Blocker, which is understandable, but I can't see how we can 

both support and not-support MariaDB, especially when we don't have test coverage on CI to support it. 

Adam, you came to mind immediately when I saw the ticket here, since you've talked about this. 

(So I'm glad you're already on this thread.) 

Where are we on MariaDB support? What timescale can we add it (if at all)? What can we realistically do in the interim? 

v2.2 is scheduled for the end of 2018. That's the earliest we can get it in now. 

What's a realistic workflow if we're going to consider MariaDB breaking changes as out of bounds before then? 

Thanks for the input all! 

C.  

[Tim Graham]

I added MariaDB 10.1.29 (as packaged by Ubuntu 18.04) to Jenkins:

https://djangoci.com/job/master-mariadb/

You can trigger the builder on PRs with the comment "buildbot, test on mariadb."

There are 30 failures, mainly related to windows expressions and select_for_update.

[Tom Forbes]

I think that given Adams plan it might be possible to get this done before 2.2 as there are not too many failures. I’m not entirely convinced that including MariaDB logic inside the MySQL backend is a permanent solution.

It might be the path of least resistance right now, and a good idea to get this in before 2.2, but if we care enough to make a fully supported backend then we might also care enough to remove any workarounds we have for MySQL (like 27676 + and others we discover), plus support any additional features or syntax that MariaDB includes. This could get pretty complex if we shoehorn it into one backend because we are going to be supporting a larger compatibility matrix.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.

To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.

Visit this group at https://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/4cf1e564-e0ad-4576-ba53-4d9276633cff%40googlegroups.com.