Recommending a Python 3-compatible MySQL connector

[Aymeric Augustin]

Hello,

To claim first-class Python 3 support in Django 1.6, we need to recommend a MySQL connector that works. This was discussed a few times on the mailing list, now's the time to make a decision. Here's a summary of the options.

Moist: https://github.com/farcepest/moist
- port by Andy, the maintainer of MySQLdb
- I don't think it works yet and development has stalled

PyMySQL: http://www.pymysql.org/
- pure-Python connector
- it doesn't appear to be widely used
- support for Python 3 looks a bit sketchy, eg. "run the build-py3k.sh script"
- it requires some modifications to the Django database backend, see https://github.com/clelland/django-mysql-pymysql
Related thread: https://groups.google.com/d/msg/django-developers/elQlGVnJ5b4/eoIZQ4kG-uIJ

MySQL-for-Python-3: https://github.com/davispuh/MySQL-for-Python-3 / https://github.com/clelland/MySQL-for-Python-3
- independent ports, there's a bit of fragmentation and no clear winner but at least they build upon MySQLdb's proven codebase.

So, unless someone has a better idea, or Ian objects, I'm going to link to his fork <https://github.com/clelland/MySQL-for-Python-3>.

Trac ticket: https://code.djangoproject.com/ticket/20025
Pull request: https://github.com/django/django/pull/1044

Any objections?

--
Aymeric.

[Ian Clelland]

So, unless someone has a better idea, or Ian objects, I'm going to link to his fork <https://github.com/clelland/MySQL-for-Python-3>.

I don't object at all -- as long as the code it still passing tests, that is :) It was originally developed as a proof of concept when python 3 support was bring added for Django 1.5, and I haven't touched the code since that was released.

I'll take a look and see if any work is going to be required to bring it in line with 1.6.

Ian

Trac ticket: https://code.djangoproject.com/ticket/20025
Pull request: https://github.com/django/django/pull/1044

Any objections?

--
Aymeric.

--
Regards,
Ian Clelland
<clel...@gmail.com>

[Aymeric Augustin]

On 5 mai 2013, at 21:15, Ian Clelland <clel...@gmail.com> wrote:

> I don't object at all -- as long as the code it still passing tests, that is :)

I just installed your version of MySQL-for-Python-3 on the CI server.

We'll have the results of the test runs in a few hours:
http://ci.djangoproject.com/job/Django/database=mysql,python=python3.2/
http://ci.djangoproject.com/job/Django/database=mysql,python=python3.3/
http://ci.djangoproject.com/job/Django/database=mysql_gis,python=python3.2/
http://ci.djangoproject.com/job/Django/database=mysql_gis,python=python3.3/
(There URLs return a 404 until the first build.)

--
Aymeric.

[Aymeric Augustin]

On 6 mai 2013, at 23:38, Travis Jensen <travis...@gmail.com> wrote:

On Sunday, May 5, 2013 2:46:43 PM UTC-6, Aymeric Augustin wrote:

On 5 mai 2013, at 21:15, Ian Clelland <clel...@gmail.com> wrote:

> I don't object at all -- as long as the code it still passing tests, that is :)

So, given that the tests aren't passing, what does that mean?  I've played around with removing that flag, but it just leads to "TypeError: 'OperationalError' object does not support indexing".  I spent the day playing with PyMySQL and MySQL-for-Python-3, and neither look like any work is being done on them.

I've reached roughly the same point.

Is Django, Python 3 and MySQL going to become non-supported?

These problems look fairly easy. I plan to work on them.

The only question is whether they should be fixed in the MySQL connector or in Django.

-- 

Aymeric.

[Aymeric Augustin]

On 7 mai 2013, at 08:34, Aymeric Augustin <aymeric....@polytechnique.org> wrote:

These problems look fairly easy. I plan to work on them.

I just updated the pull request and everything works except BinaryField:

https://code.djangoproject.com/ticket/20025#comment:5

Something forces the output to be a string, either in the MySQL connector or in the converters registered by Django.

I would appreciate some help on this.

Here's how to setup the MySQL connector with python 3.3:

$ pyvenv-3.3 mysql-py33

$ . mysql-py33/bin/activate

$ curl http://python-distribute.org/distribute_setup.py | python

$ git clone https://github.com/clelland/MySQL-for-Python-3

$ cd MySQL-for-Python-3

$ python setup.py install

Then run model_fields tests.

-- 

Aymeric.

[Travis Jensen]

Thanks, Aymeric. I grabbed the latest Django code and things worked like a charm.

Now, I have a decision to make: Stay on the Django/Python 3 bleeding edge or back off to Python 2.7.  My project's timeline says I won't release until after Django 1.6 (even with some slop for slipping releases), but it still feels dicey. On the other hand, on a brand new project, I'd prefer to be looking at Python's future rather than sticking in its past. :)

tj

[Mark Hughes]

On 5 May 2013 19:17, Aymeric Augustin

<aymeric....@polytechnique.org> wrote:
>
> To claim first-class Python 3 support in Django 1.6, we need to recommend a MySQL connector that works. This was discussed a few times on the mailing list, now's the time to make a decision. Here's a summary of the options.
>

Another option to consider could be mysql-connector-python

https://pypi.python.org/pypi/mysql-connector-python/1.0.9

It claims to support Python 2 and 3

http://dev.mysql.com/doc/refman/5.5/en/connector-python-versions.html

Also actively developed by @geertjanvdk at Oracle so he may be able to
help with any issues?

Mark.

[Aymeric Augustin]

Hi Mark,

On 10 mai 2013, at 10:16, Mark Hughes <ma...@cyki.com> wrote:
> Another option to consider could be mysql-connector-python
>
> https://pypi.python.org/pypi/mysql-connector-python/1.0.9

Thank you, it's the official library I was missing!

Unfortunately, it's licensed under the GPL. To the best of my understanding, importing modules in the same Python process triggers GPL contamination. Therefore, if Django officially supports this module, anyone distributing code that works with Django should either release it under the GPL or specify that it's illegal to use it with MySQL.

The Django project favors the BSD license to avoid inflicting such questions to its community. I'm afraid we won't compromise for such a central piece as a database backend.

(NB: Oracle's "FOSS License Exception" doesn't cover this case; it only allows the Django Software Foundation to bundle the MySQL client libraries in Django, if we wanted to.)

> Also actively developed by @geertjanvdk at Oracle so he may be able to
> help with any issues?

If he has the power to switch to a license that makes it possible to use his code, like the LGPL, that would be fantastic. I can't tell if he chose the GPL to make a point or by accident; I'm going to ask him.

--
Aymeric.

[Luke Plant]

On 10/05/13 14:12, Aymeric Augustin wrote:
> Hi Mark,
>
> On 10 mai 2013, at 10:16, Mark Hughes <ma...@cyki.com> wrote:
>> Another option to consider could be mysql-connector-python
>>
>> https://pypi.python.org/pypi/mysql-connector-python/1.0.9
>
> Thank you, it's the official library I was missing!
>
> Unfortunately, it's licensed under the GPL. To the best of my
> understanding, importing modules in the same Python process triggers
> GPL contamination. Therefore, if Django officially supports this
> module, anyone distributing code that works with Django should either
> release it under the GPL or specify that it's illegal to use it with
> MySQL.

Can the GPL really do this? My own thoughts on this are here:

http://lukeplant.me.uk/blog/posts/python-and-copyright/

I'm not saying that we should use GPL code if there is BSD alternative,
but as far as I can see, it's impossible for the GPL to say *anything*
in this situation, because there is nothing a typical Django project
would be doing with that library that would actually require them to
accept the terms of the license.

The GPL allows *using* of the code for any purpose. It's only when a
project becomes a distributor of the GPL code that it is required to
abide by the other terms.

Regards,

Luke

--
"God demonstrates his love towards us in this, that while we were
still sinners, Christ died for us." (Romans 5:8)

Luke Plant || http://lukeplant.me.uk/

[Donald Stufft]

On May 10, 2013, at 11:49 AM, Luke Plant <L.Pla...@cantab.net> wrote:

> On 10/05/13 14:12, Aymeric Augustin wrote:
>> Hi Mark,
>>
>> On 10 mai 2013, at 10:16, Mark Hughes <ma...@cyki.com> wrote:
>>> Another option to consider could be mysql-connector-python
>>>
>>> https://pypi.python.org/pypi/mysql-connector-python/1.0.9
>>
>> Thank you, it's the official library I was missing!
>>
>> Unfortunately, it's licensed under the GPL. To the best of my
>> understanding, importing modules in the same Python process triggers
>> GPL contamination. Therefore, if Django officially supports this
>> module, anyone distributing code that works with Django should either
>> release it under the GPL or specify that it's illegal to use it with
>> MySQL.
>
> Can the GPL really do this? My own thoughts on this are here:
>
> http://lukeplant.me.uk/blog/posts/python-and-copyright/
>
> I'm not saying that we should use GPL code if there is BSD alternative,
> but as far as I can see, it's impossible for the GPL to say *anything*
> in this situation, because there is nothing a typical Django project
> would be doing with that library that would actually require them to
> accept the terms of the license.
>
> The GPL allows *using* of the code for any purpose. It's only when a
> project becomes a distributor of the GPL code that it is required to
> abide by the other terms.
>

http://jacobian.org/writing/gpl-questions/

In particular look at VanL's responses.


-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

[Aymeric Augustin]

Hi Luke,

Your blog post nails the problem. The GPL wasn't written with dynamic
languages in mind and "linking" is a big question mark. As far as I know,
it has never been tested in court, and we won't be sure of anything until
it is.

Of course I could be wrong… I'm just repeating what I've heard from
others whom I trust.

In fact I would just like to skip this debate entirely and the easiest way
to do that is to avoid the GPL :)


That said, in my experience, people releasing Python libraries under
the GPL fall in two categories:
- people unfamiliar with licensing trying to do the right thing (I've done
that before!),
- people who don't want their code to be used in non-GPL apps (for
whatever reason).

In general, the first category doesn't object to switching to LGPL, which
is recognized as safe for our purposes (as far as I know).

It's hard to have a rational discussion with the second category because
they tend to go on a FSF jihad as soon as you try discussing licensing
with them ;)

Asking to change the license to LGPL is a good first step for libraries
where LGPL is obviously more suitable than GPL. If the author argues
that GPL is more suitable, that tells us about his motivations, and we
can make a decision to use or not use the code.

--
Aymeric.

[Aymeric Augustin]

2013/5/10 Aymeric Augustin <aymeric....@polytechnique.org>

> Also actively developed by @geertjanvdk at Oracle so he may be able to
> help with any issues?

If he has the power to switch to a license that makes it possible to use his code, like the LGPL, that would be fantastic. I can't tell if he chose the GPL to make a point or by accident; I'm going to ask him.

I got an answer today. The license will not be changed. I was asked not to publish our discussion.

Specifically, MySQL Connector/Python is released under GNU GPLv2 with FOSS exception.

The FOSS exception only applies to redistributing MySQL Connector/Python, which we don't intend to do anyway.

To sum up, the scenario I'm worried about is the following. A company develops a product based on Django + MySQL and distributes it to its customers. Currently, the product can be distributed under any license. If we switch to MySQL Connector/Python in Django, then the product must be licensed under the GPLv2 or a compatible license. Once again I'm not a lawyer and I may be wrong, I don't know where the line lies exactly, but that could create a legal trap for some users of Django.

I don't want to add a warning along the lines of "be careful, if you use MySQL, the licensing terms change!" in our docs. I'm not even sure how to write it without falling into FUD.

So at this point I see two solutions:

- a core dev (not me!) feels sufficiently confident that this isn't an issue to add such a backend to Django itself. That's Oracle's theory, as far as I understand.

- someone writes an external MySQL Connector/Python backend, probably based on the MyQSLdb backend. That keeps the licensing issue outside of Django.

I'm leaving this here, if someone wants to take over, the ground is yours!

--
Aymeric.

[Jacob Kaplan-Moss]

I've reached out to a lawyer friend to see if he can give us some
guidance. Until then, let's avoid making a recommendation either way.

Jacob

> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-develop...@googlegroups.com.
> To post to this group, send email to django-d...@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Daniel Sears]

I want to follow up on the issue of a Python 3 connector for MySQL.

Oracle has developed an open source Python driver for MySQL:

• PEP 249-compliant
• pure Python
• supports Python 3
• very simple installation:
  pip install mysql-connector 
• better support for asyncio than a C driver
• stable
• actively developed and supported by Oracle
• will likely track emerging MySQL features (in line with Oracle's other drivers, e.g. Java, PHP, ...)

I submitted a documentation patch for this a few months ago, but then I saw this closed ticket. So I want to revisit this issue in the hopes of getting a clarifying policy. Without that I should probably back out the patch.

I know the core issue of GPL remains. But since this discussion began Oracle has extended their driver to include their own Django back-end. Does this create enough separation to get us out of the GPL bind?

--Dan

[Russell Keith-Magee]

On Thu, Jan 23, 2014 at 2:51 PM, Daniel Sears <daniel...@gmail.com> wrote:

I want to follow up on the issue of a Python 3 connector for MySQL.

Oracle has developed an open source Python driver for MySQL:

• PEP 249-compliant
• pure Python
• supports Python 3
• very simple installation:
  pip install mysql-connector 
• better support for asyncio than a C driver
• stable
• actively developed and supported by Oracle
• will likely track emerging MySQL features (in line with Oracle's other drivers, e.g. Java, PHP, ...)

I submitted a documentation patch for this a few months ago, but then I saw this closed ticket. So I want to revisit this issue in the hopes of getting a clarifying policy. Without that I should probably back out the patch.

I know the core issue of GPL remains. But since this discussion began Oracle has extended their driver to include their own Django back-end. Does this create enough separation to get us out of the GPL bind?

The existence of a Django backend in MySQL Connector/Python doesn't change anything. The code is still released under the GPL, and the GPL has been specifically constructed to prevent people from "getting out of it". 

The issue is that the GPL is at odds with the type of community that the Django project has tried to foster. Django has been released under the terms of the BSD license specifically because we want to allow commercial activities without also requiring the release of source code.

If we recommended MySQL Connector/Python backend, anyone who chose to use that backend would need to release that project under the terms of the GPL - which means releasing source code.

This may be acceptable on a per project basis, and users are free to use Connector/Python in their own projects if they want -- but I don't think Django as a project should be encouraging this by documenting it as an option. 

Yours,

Russ Magee %-)

[Aymeric Augustin]

2014/1/23 Daniel Sears <daniel...@gmail.com>

I know the core issue of GPL remains. But since this discussion began Oracle has extended their driver to include their own Django back-end. Does this create enough separation to get us out of the GPL bind?

In the original discussion, I missed that MySQLdb is also released under the GPL. See https://pypi.python.org/pypi/MySQL-python.

Considering the current state of the MySQL ecosystem, I'm +0 for switching to Oracle's connector. I used to be -1.

You can use this email to ignore things I've said on this topic in the past.

-- 

Aymeric.