[Trac] Need help on trac migration 1.0.13 to 1.2.2: TracError: Unsupported database type "mysql"

[Torge Riedel]

Hi,

I've set up a new server Ubuntu 16.04.3 LTS and try to migrate my existing (and running) trac 1.0.13 installation to 1.2.2, but this fails with

TracError: Unsupported database type "mysql"

Setting up a new environment with the same connection string gives me:

Failed to create environment.
Unsupported database type "mysql"
Traceback (most recent call last):
  File "build/bdist.linux-x86_64/egg/trac/admin/console.py", line 500, in do_initenv
    options=options)
  File "build/bdist.linux-x86_64/egg/trac/core.py", line 136, in __call__
    self.__init__(*args, **kwargs)
  File "build/bdist.linux-x86_64/egg/trac/env.py", line 300, in __init__
    self.create(options)
  File "build/bdist.linux-x86_64/egg/trac/env.py", line 592, in create
    DatabaseManager(self).init_db()
  File "build/bdist.linux-x86_64/egg/trac/db/api.py", line 337, in init_db
    connector, args = self.get_connector()
  File "build/bdist.linux-x86_64/egg/trac/db/api.py", line 615, in get_connector
    scheme=scheme))
TracError: Unsupported database type "mysql"

So I think it is an environment/installation issue. I need help to point me to the right direction to find the cause. The python-mysqldb package is installed. But to be honest I'm not a python developer, so no idea from my site how to dig deeper in the code to find the root cause.

Thanks in advance
Torge

[Torge Riedel]

Ok, this switched to a general issue. I followed documentation on https://trac.edgewall.org/wiki/TracInstall and then the following call to initialize a new environment:

trac-admin /var/www/trac/env2 initenv

Creating a new Trac environment at /var/www/trac/env2

Trac will first ask a few questions about your environment
in order to initialize and prepare the project database.

 Please enter the name of your project.
 This name will be used in page titles and descriptions.

Project Name [My Project]> Test

 Please specify the connection string for the database to use.
 By default, a local SQLite database is created in the environment
 directory. It is also possible to use an existing MySQL or
 PostgreSQL database (check the Trac documentation for the exact
 connection string syntax).

Database connection string [sqlite:db/trac.db]> mysql://tracuser:MyPwd@localhost:3306/trac

Creating and Initializing Project
Initenv for '/var/www/trac/env2' failed.

Failed to create environment.
Unsupported database type "mysql"
Traceback (most recent call last):
  File "build/bdist.linux-x86_64/egg/trac/admin/console.py", line 500, in do_initenv
    options=options)
  File "build/bdist.linux-x86_64/egg/trac/core.py", line 136, in __call__
    self.__init__(*args, **kwargs)
  File "build/bdist.linux-x86_64/egg/trac/env.py", line 300, in __init__
    self.create(options)
  File "build/bdist.linux-x86_64/egg/trac/env.py", line 592, in create
    DatabaseManager(self).init_db()
  File "build/bdist.linux-x86_64/egg/trac/db/api.py", line 337, in init_db
    connector, args = self.get_connector()
  File "build/bdist.linux-x86_64/egg/trac/db/api.py", line 615, in get_connector
    scheme=scheme))
TracError: Unsupported database type "mysql"

Looks no good. I checked the following:

- mysql -utracuser -p      > giving the same password results in a successful login, db "trac" can be shown and tables too
- php -m shows following output

[PHP Modules]
calendar
Core
ctype
curl
date
dom
exif
fileinfo
filter
ftp
gd
gettext
hash
iconv
igbinary
intl
json
libxml
mbstring
mysqli
mysqlnd
openssl
pcntl
pcre
PDO
pdo_mysql
Phar
posix
readline
redis
Reflection
session
shmop
SimpleXML
sockets
SPL
standard
sysvmsg
sysvsem
sysvshm
tokenizer
wddx
xml
xmlreader
xmlwriter
xsl
Zend OPcache
zip
zlib

- apt-get install python-mysqldb displays

Reading package lists... Done
Building dependency tree
Reading state information... Done
python-mysqldb is already the newest version (1.3.7-1build2).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

Any ideas what to check further.

Thanks in advance
Torge

[Chris Shelton]

Are you sure that you have python 2.7 installed?  I believe that starting with Ubuntu server 16.04.1, the default python version is set to 3.5, not 2.7.  Trac still requires 2.7.x.  Try running:

python --version

to see what version of python that you are using.  

If you are using the 3.5 branch of python, you will very likely need to install python 2.7 for trac to work:

sudo apt-get install python2.7

should work to start installing python 2.7.

chris

Thanks in advance
Torge

--
You received this message because you are subscribed to the Google Groups "Trac Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to trac-users+unsubscribe@googlegroups.com.
To post to this group, send email to trac-...@googlegroups.com.
Visit this group at https://groups.google.com/group/trac-users.
For more options, visit https://groups.google.com/d/optout.

[Torge Riedel]

Ok, one more test, since trac is using MySQLdb I searched the code and extracted this short test code. And it works:

# python
Python 2.7.12 (default, Nov 19 2016, 06:48:10)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from trac.util import get_pkginfo
>>> import MySQLdb
>>> import MySQLdb.cursors
>>> mysqldb_version = get_pkginfo(MySQLdb).get('version', MySQLdb.__version__)
>>> print("MySQLdb.version = {}".format(mysqldb_version))
MySQLdb.version = 1.3.7
>>> con = MySQLdb.connect('localhost', 'tracuser', 'thung0hohF6no3ah', 'trac')
>>> cur = con.cursor()
>>> cur.execute("SHOW TABLES")
19L
>>> for (table_name) in cur:
...   print("- {}".format(table_name))
...
- ('attachment',)
- ('auth_cookie',)
- ('cache',)
- ('component',)
- ('enum',)
- ('milestone',)
- ('node_change',)
- ('permission',)
- ('report',)
- ('repository',)
- ('revision',)
- ('session',)
- ('session_attribute',)
- ('system',)
- ('ticket',)
- ('ticket_change',)
- ('ticket_custom',)
- ('version',)
- ('wiki',)
>>> cur.close()
>>> con.close()
>>> quit()

To unsubscribe from this group and stop receiving emails from it, send an email to trac-users+...@googlegroups.com.

[RjOllos]

Here is another command to try from the Python shell:

>>> from trac.db.mysql_backend import has_mysqldb

>>> has_mysqldb

True

>>> from trac.db.mysql_backend import mysqldb_version

>>> mysqldb_version

'1.2.5'

You might have the wrong MySQL database bindings installed. You need this one:

https://pypi.python.org/pypi/MySQL-python

Going from version numbers, it looks like you have this one:

https://pypi.python.org/pypi/mysqlclient/1.3.7

Side note, for the forthcoming Trac 1.4 release we've switched to PyMySQL:

https://trac.edgewall.org/wiki/MySqlDb#Trac1.3.2

and the following may be relevant to your situation:

https://trac.edgewall.org/ticket/12821#comment:13

- Ryan

[Jun Omae]

Could you please try initenv with logging using --inherit option?

$ cat /path/to/inherit.ini
[logging]
log_type = stderr
log_level = DEBUG
$ trac-admin /path/to/env initenv --inherit=/path/to/inherit.ini
my-project 'mysql://u:p@localhost/t'


In Ubuntu 16.04 on docker, I get the following 2 errors:

Error 1
=======
12:55:21 Trac[loader] DEBUG: Loading plugin "trac.db.mysql" from
"/tmp/venv/lib/python2.7/site-packages"
12:55:21 Trac[loader] ERROR: Skipping "trac.db.mysql =
trac.db.mysql_backend [mysql]":

Traceback (most recent call last):

File "/tmp/venv/local/lib/python2.7/site-packages/trac/loader.py",
line 68, in _load_eggs
entry.load(require=True)
File "/tmp/venv/local/lib/python2.7/site-packages/pkg_resources/__init__.py",
line 2228, in load
self.require(*args, **kwargs)
File "/tmp/venv/local/lib/python2.7/site-packages/pkg_resources/__init__.py",
line 2245, in require
items = working_set.resolve(reqs, env, installer)
File "/tmp/venv/local/lib/python2.7/site-packages/pkg_resources/__init__.py",
line 808, in resolve
if not req_extras.markers_pass(req):
File "/tmp/venv/local/lib/python2.7/site-packages/pkg_resources/__init__.py",
line 993, in markers_pass
return not req.marker or any(extra_evals) or req.marker.evaluate()
File "/tmp/venv/local/lib/python2.7/site-packages/pkg_resources/_vendor/packaging/markers.py",
line 278, in evaluate
return _evaluate_markers(self._markers, current_environment)
File "/tmp/venv/local/lib/python2.7/site-packages/pkg_resources/_vendor/packaging/markers.py",
line 203, in _evaluate_markers
lhs_value = _get_env(environment, lhs.value)
File "/tmp/venv/local/lib/python2.7/site-packages/pkg_resources/_vendor/packaging/markers.py",
line 185, in _get_env
"{0!r} does not exist in evaluation environment.".format(name)
UndefinedEnvironmentName: 'extra' does not exist in evaluation environment.
12:55:21 Trac[loader] DEBUG: Loading plugin "trac.db.postgres" from
"/tmp/venv/lib/python2.7/site-packages"
12:55:21 Trac[loader] ERROR: Skipping "trac.db.postgres =
trac.db.postgres_backend [postgresql]":

Traceback (most recent call last):

File "/tmp/venv/local/lib/python2.7/site-packages/trac/loader.py",
line 68, in _load_eggs
entry.load(require=True)
File "/tmp/venv/local/lib/python2.7/site-packages/pkg_resources/__init__.py",
line 2228, in load
self.require(*args, **kwargs)
File "/tmp/venv/local/lib/python2.7/site-packages/pkg_resources/__init__.py",
line 2245, in require
items = working_set.resolve(reqs, env, installer)
File "/tmp/venv/local/lib/python2.7/site-packages/pkg_resources/__init__.py",
line 808, in resolve
if not req_extras.markers_pass(req):
File "/tmp/venv/local/lib/python2.7/site-packages/pkg_resources/__init__.py",
line 993, in markers_pass
return not req.marker or any(extra_evals) or req.marker.evaluate()
File "/tmp/venv/local/lib/python2.7/site-packages/pkg_resources/_vendor/packaging/markers.py",
line 278, in evaluate
return _evaluate_markers(self._markers, current_environment)
File "/tmp/venv/local/lib/python2.7/site-packages/pkg_resources/_vendor/packaging/markers.py",
line 203, in _evaluate_markers
lhs_value = _get_env(environment, lhs.value)
File "/tmp/venv/local/lib/python2.7/site-packages/pkg_resources/_vendor/packaging/markers.py",
line 185, in _get_env
"{0!r} does not exist in evaluation environment.".format(name)
UndefinedEnvironmentName: 'extra' does not exist in evaluation environment.


Error 2 after setuptools is reinstalled
=======
12:57:49 Trac[loader] DEBUG: Loading plugin "trac.db.mysql" from
"/tmp/venv/lib/python2.7/site-packages"
12:57:49 Trac[loader] DEBUG: Skipping "trac.db.mysql =
trac.db.mysql_backend [mysql]": DistributionNotFound: The
'MySQL-python>=1.2.2; extra == "mysql"' distribution was not found and
is required by the application
12:57:49 Trac[loader] DEBUG: Loading plugin "trac.db.postgres" from
"/tmp/venv/lib/python2.7/site-packages"
12:57:49 Trac[loader] DEBUG: Skipping "trac.db.postgres =
trac.db.postgres_backend [postgresql]": DistributionNotFound: The
'psycopg2>=2.0; extra == "postgresql"' distribution was not found and
is required by the application


--
Jun Omae <jun...@gmail.com> (大前 潤)

[Torge Riedel]

--
You received this message because you are subscribed to the Google Groups "Trac Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to trac-users+...@googlegroups.com.
To post to this group, send email to trac-...@googlegroups.com.
Visit this group at https://groups.google.com/group/trac-users.
For more options, visit https://groups.google.com/d/optout.

Hi Ryan,

you are right it is the "wrong" database binding.

But taking a look at https://pypi.python.org/pypi/MySQL-python/1.2.5 I see, that this is really old (2014-01-02) and does not support MySql > 5.5 (as I understand). But Ubuntu 16.04 LTS comes with MySQL 5.7.

Taking a look at https://pypi.python.org/pypi/mysqlclient I read, that this is a fork of the other binding and supports MySQL >= 5.5 and py2.7.

That's why this binding (1.3.7) is installed when I execute apt-get install python-mysqldb and not the older one (1.2.5).

Imho my feeling is that there is currently no chance to get trac running with MySQL on Ubuntu 16.04? Is there any chance to patch "my" Trac 1.2.2 in a way to work with the newer binding?

Regards
Torge

[RjOllos]

To unsubscribe from this group and stop receiving emails from it, send an email to trac-users+unsubscribe@googlegroups.com.

To post to this group, send email to trac-...@googlegroups.com.
Visit this group at https://groups.google.com/group/trac-users.
For more options, visit https://groups.google.com/d/optout.

Hi Ryan,

you are right it is the "wrong" database binding.

But taking a look at https://pypi.python.org/pypi/MySQL-python/1.2.5 I see, that this is really old (2014-01-02) and does not support MySql > 5.5 (as I understand). But Ubuntu 16.04 LTS comes with MySQL 5.7.

Taking a look at https://pypi.python.org/pypi/mysqlclient I read, that this is a fork of the other binding and supports MySQL >= 5.5 and py2.7.

That's why this binding (1.3.7) is installed when I execute apt-get install python-mysqldb and not the older one (1.2.5).

Imho my feeling is that there is currently no chance to get trac running with MySQL on Ubuntu 16.04? Is there any chance to patch "my" Trac 1.2.2 in a way to work with the newer binding?

Regards
Torge

It might be possible to make mysqlclient work with Trac 1.2.x. If you follow Jun's guidance in generating debug output, the issue you are experiencing might become more obvious.

Your other option would be to install Trac 1.3.2 with PyMySQL, however the warnings about of running a 1.3.x release apply:

https://trac.edgewall.org/wiki/TracDownload#LatestDevRelease

- Ryan

[Torge Riedel]

Hi Jun,

I did the test Ryan suggest:

# python
Python 2.7.12 (default, Nov 19 2016, 06:48:10)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.

>>> from trac.db.mysql_backend import has_mysqldb
>>> has_mysqldb
True
>>> from trac.db.mysql_backend import mysqldb_version
>>> mysqldb_version

'1.3.7'
>>> quit()

So, to me it looks like trac can detect the MySQLdb package in this newer version. Running your test with debug output it prints like on your d-container:

07:36:16 Trac[loader] DEBUG: Skipping "trac.db.mysql = trac.db.mysql_backend [mysql]": DistributionNotFound: The 'MySQL-python>=1.2.2' distribution was not found and is required by the application

So somewhere in trac there is an additional check? I will look in the code (although I am not a python dev) and try to find the cause of this failing check. If you have any hints, it will be very helpful.

Thanks

Torge

[RjOllos]

Please try the following patch:

diff --git a/setup.py b/setup.py
index 5f6f2d7b1..1c93d9e41 100755
--- a/setup.py
+++ b/setup.py
@@ -97,7 +97,7 @@ setup(
     ],
     extras_require = {
         'babel': ['Babel>=0.9.5'],
-        'mysql': ['MySQL-python >= 1.2.2'],
+        'mysql': ['mysqlclient'],
         'postgresql': ['psycopg2 >= 2.0'],
         'pygments': ['Pygments>=0.6'],
         'rest': ['docutils>=0.3.9'],

All tests are passing for me. We might be able to add mysqlclient as a supported database backend in Trac 1.2.3.

- Ryan

[RjOllos]

Created a ticket: https://trac.edgewall.org/ticket/12952

- Ryan

[Torge Riedel]

--
You received this message because you are subscribed to the Google Groups "Trac Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to trac-users+...@googlegroups.com.

To post to this group, send email to trac-...@googlegroups.com.
Visit this group at https://groups.google.com/group/trac-users.
For more options, visit https://groups.google.com/d/optout.

Hi Ryan,

many thanks. This patch worked in a way for me, that Trac is now no more complaining about unsupported db scheme, it says (on initenv):

(1045, "Access denied for user 'u'@'localhost' (using password: YES)")

Looks good. I will continue this evening with setting up a new env with correct credentials and test the migration. I will give feedback here.

If it's possible to add a fix in Trac 1.2.3 to support both versions, I will be happy.

Thanks
Torge

[Torge Riedel]

Ok, I finished my checks and successfully migrated to Trac 1.2.2 with the patch. Everything works fine as far as I can see.

Thanks for your support.
Torge

[RjOllos]

To unsubscribe from this group and stop receiving emails from it, send an email to trac-users+unsubscribe@googlegroups.com.

To post to this group, send email to trac-...@googlegroups.com.
Visit this group at https://groups.google.com/group/trac-users.
For more options, visit https://groups.google.com/d/optout.

Hi Ryan,

many thanks. This patch worked in a way for me, that Trac is now no more complaining about unsupported db scheme, it says (on initenv):

(1045, "Access denied for user 'u'@'localhost' (using password: YES)")

Looks good. I will continue this evening with setting up a new env with correct credentials and test the migration. I will give feedback here.

If it's possible to add a fix in Trac 1.2.3 to support both versions, I will be happy.

Thanks
Torge

--
You received this message because you are subscribed to the Google Groups "Trac Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to trac-users+unsubscribe@googlegroups.com.

To post to this group, send email to trac-...@googlegroups.com.
Visit this group at https://groups.google.com/group/trac-users.
For more options, visit https://groups.google.com/d/optout.

Ok, I finished my checks and successfully migrated to Trac 1.2.2 with the patch. Everything works fine as far as I can see.

Thanks for your support.
Torge

Thanks for reporting back. I hope to have Trac 1.4 released by end of the year, which should resolve the issue by using PyMySQL. PyMySQL is available on Ubuntu 16.04.

https://trac.edgewall.org/wiki/TracDev/ApiChanges/1.3#CompatibleDistros

- Ryan

 

[Fred Leeflang]

I have the same problem on my Debian Stretch box. I checked but I don't have a setup.py that I can patch. I have installed:

ii  trac                                  1.2+dfsg-1                                  all          Enhanced wiki and issue tracking system for software development projects

Is there a file I can patch to make it work while waiting for the 1.2.3?

[Ryan Ollos]

On Sat, Nov 18, 2017 at 7:55 AM, Fred Leeflang <fr...@dutchie.org> wrote:

I have the same problem on my Debian Stretch box. I checked but I don't have a setup.py that I can patch. I have installed:

ii  trac                                  1.2+dfsg-1                                  all          Enhanced wiki and issue tracking system for software development projects

You'll need to install from source rather than using your package manager. See TracInstall for details.

You can install into a virtualenv if you wish to avoid adding packages to the system path that don't originate from your package manager.

 

Is there a file I can patch to make it work while waiting for the 1.2.3?

The patch was rejected for Trac 1.2.3. Trac 1.4 will be released by end of year and support PyMySQL.

- Ryan