Upgrading 1.0 to 1.4

[Frank R.]

Hi,

we are still running 1.0.1. I tried to grade to 1.2. Running

trac-admin PATH upgrade

I got an upgrade error (in German, hence I do not post it initially) regarding the collation of all Trac tables. Having searched the net I was able to change tables collation to utf8mb4_unicode_ci, but not to change the collation of columns. I always get an ERROR 1025: Error on rename... I found that

SET FOREIGN_KEY_CHECK=0

should solve my problem, but it does not. I'm stuck...

Best

Frank

[Jun Omae]

On Thu, May 4, 2023 at 11:36 PM Frank R. <frank.re...@gmail.com> wrote:
>
> Hi,
>
> we are still running 1.0.1. I tried to grade to 1.2. Running
>
> trac-admin PATH upgrade
>
> I got an upgrade error (in German, hence I do not post it initially) regarding the collation of all Trac tables.

Try the upgrade with English locale:

LC_ALL=en_US.UTF8 trac-admin PATH upgrade

Then, check $ENV/log/trac.log and provide upgrade errors in the log file.


--
Jun Omae <jun...@gmail.com> (大前 潤)

[Frank R.]

Try the upgrade with English locale:

  LC_ALL=en_US.UTF8 trac-admin PATH upgrade

Then, check $ENV/log/trac.log and provide upgrade errors in the log file.

The STDOUT message is.

The upgrade failed. Please fix the issue and try again.

TracError: All tables must be created with utf8_bin or utf8mb4_bin as collation. The following tables don't have the collations: attachment, auth_cookie, cache, component, enum, mil
estone, node_change, permission, report, repository, revision, session, session_attribute, system, ticket, ticket_change, ticket_custom, version, wiki

$ENV is the trac root, the same path I use when running trac-admin, isn't it? But the log folder below that path is empty.

[Jun Omae]

On Fri, May 5, 2023 at 4:04 PM Frank R. <frank.re...@gmail.com> wrote:
> TracError: All tables must be created with utf8_bin or utf8mb4_bin as collation. The following tables don't have the collations: attachment, auth_cookie, cache, component, enum, milestone, node_change, permission, report, repository, revision, session, session_attribute, system, ticket, ticket_change, ticket_custom, version, wiki

It seems the above collations of tables are not utf8_bin or utf8mb4_bin.
Please post output of the following query in your MySQL database:

select table_schema, table_name, table_collation
from information_schema.tables
where table_schema='trac' -- replace with your schema

Also, what version of the MySQL server are you using?

> $ENV is the trac root, the same path I use when running trac-admin, isn't it? But the log folder below that path is empty.

Check [logging] log_type option. It is needed to configure "log_type =
file" for logging to $ENV/log/trac.log.

[Mickaël Bucas]

Hi Frank

I've run into this problem in the past, and fixed it with the
following commands.

!! I recommend making a backup before trying these commands !!

=> Change database charset and collation
echo "alter database TRACDB charset utf8 collate utf8_bin;"|mysql

=> Change all tables charset and collation
for t in attachment auth_cookie cache component enum milestone node_change
permission report repository revision session session_attribute system ticket
ticket_change ticket_custom version wiki; do echo "alter table TRACDB.$t
convert to character set utf8 collate utf8_bin;"; done|mysql

You have to replace TRACDB with your database name.
You also should check the list of tables in the second command.

Best regards

Mickaël Bucas

> --
> You received this message because you are subscribed to the Google Groups "Trac Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to trac-users+...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/trac-users/CA%2ByhgXDgnk94YeSTRGRSrEfs-KGkStoxX33WHT8E9eADXtf9%2Bw%40mail.gmail.com.

[Frank R.]

  select table_schema, table_name, table_collation
  from information_schema.tables
  where table_schema='trac'  -- replace with your schema

MariaDB [(none)]> select table_schema, table_name, table_collation    from information_schema.tables where table_schema='trac';
+--------------+-----------------------+--------------------+
| table_schema | table_name            | table_collation    |
+--------------+-----------------------+--------------------+
| trac         | attachment            | utf8mb4_unicode_ci |
| trac         | auth_cookie           | utf8mb4_unicode_ci |
| trac         | cache                 | utf8mb4_unicode_ci |
| trac         | component             | utf8mb4_unicode_ci |
| trac         | enum                  | utf8mb4_unicode_ci |
| trac         | milestone             | utf8mb4_unicode_ci |
| trac         | node_change           | utf8mb4_unicode_ci |
| trac         | permission            | utf8mb4_unicode_ci |
| trac         | report                | utf8mb4_unicode_ci |
| trac         | repository            | utf8mb4_unicode_ci |
| trac         | revision              | utf8mb4_unicode_ci |
| trac         | session               | utf8mb4_unicode_ci |
| trac         | session_attribute     | utf8mb4_unicode_ci |
| trac         | system                | utf8mb4_unicode_ci |
| trac         | ticket                | utf8mb4_unicode_ci |
| trac         | ticket_change         | utf8mb4_unicode_ci |
| trac         | ticket_custom         | utf8mb4_unicode_ci |
| trac         | ticket_template_store | utf8mb4_unicode_ci |
| trac         | version               | utf8mb4_unicode_ci |
| trac         | wiki                  | utf8mb4_unicode_ci |
+--------------+-----------------------+--------------------+

 

Also, what version of the MySQL server are you using?

 Server version: 10.1.48-MariaDB-0ubuntu0.18.04.1 Ubuntu 18.04

> $ENV is the trac root, the same path I use when running trac-admin, isn't it? But the log folder below that path is empty.

Check [logging] log_type option. It is needed to configure "log_type =
file" for logging to $ENV/log/trac.log.

Ok, that was set to "syslog".  Switched to "file"

==> No errors in log file, but warnings

2023-05-05 10:54:06,654 Trac[api] INFO: Need to upgrade database_version from 29 to 41
2023-05-05 10:54:06,654 Trac[env] WARNING: Component <Component trac.env.EnvironmentSetup> requires environment upgrade
2023-05-05 10:54:06,655 Trac[api] INFO: Need to upgrade database_version from 29 to 41
2023-05-05 10:54:06,655 Trac[env] WARNING: Component <Component trac.env.EnvironmentSetup> requires environment upgrade
2023-05-05 10:54:06,655 Trac[api] INFO: Need to upgrade database_version from 29 to 41

[Jun Omae]

Trac requires utf8mb4_bin or utf8_bin for collation of the tables.

Please change collation of the tables to utf8mb4_bin.
Try the upgrade again after fixing the collation.

====
ALTER DATABASE trac COLLATE=utf8mb4_bin;
USE trac
ALTER TABLE attachment COLLATE=utf8mb4_bin;
ALTER TABLE auth_cookie COLLATE=utf8mb4_bin;
ALTER TABLE cache COLLATE=utf8mb4_bin;
ALTER TABLE component COLLATE=utf8mb4_bin;
ALTER TABLE enum COLLATE=utf8mb4_bin;
ALTER TABLE milestone COLLATE=utf8mb4_bin;
ALTER TABLE node_change COLLATE=utf8mb4_bin;
ALTER TABLE permission COLLATE=utf8mb4_bin;
ALTER TABLE report COLLATE=utf8mb4_bin;
ALTER TABLE repository COLLATE=utf8mb4_bin;
ALTER TABLE revision COLLATE=utf8mb4_bin;
ALTER TABLE session COLLATE=utf8mb4_bin;
ALTER TABLE session_attribute COLLATE=utf8mb4_bin;
ALTER TABLE system COLLATE=utf8mb4_bin;
ALTER TABLE ticket COLLATE=utf8mb4_bin;
ALTER TABLE ticket_change COLLATE=utf8mb4_bin;
ALTER TABLE ticket_custom COLLATE=utf8mb4_bin;
ALTER TABLE ticket_template_store COLLATE=utf8mb4_bin;
ALTER TABLE version COLLATE=utf8mb4_bin;
ALTER TABLE wiki COLLATE=utf8mb4_bin;
====

[Frank R.]

Thanks to both of you...

I did what you wrote, Jun, and then...

> LC_ALL=en_US.UTF8 trac-admin "/srv/www/trac" upgrade

The upgrade failed. Please fix the issue and try again.

OperationalError: (1025, 'Error on rename of \'./trac/#sql-458_c9f\' to \'./trac/session_attribute\' (errno: 184 "Tablespace already exists")')

Scanning the www I found that one has to go down to column level regarding collation change. But that's where I got stuck...

[Jun Omae]

On Fri, May 5, 2023 at 7:05 PM Frank R. <frank.re...@gmail.com> wrote:
> > LC_ALL=en_US.UTF8 trac-admin "/srv/www/trac" upgrade
> The upgrade failed. Please fix the issue and try again.
>
> OperationalError: (1025, 'Error on rename of \'./trac/#sql-458_c9f\' to \'./trac/session_attribute\' (errno: 184 "Tablespace already exists")')

Please share $ENV/log/trac.log.

[Frank R.]

[...]

2023-05-05 12:01:41,011 Trac[api] INFO: Upgraded database_version from 32 to 33
2023-05-05 12:01:41,329 Trac[api] INFO: Upgraded database_version from 33 to 34
2023-05-05 12:01:41,604 Trac[api] INFO: Upgraded database_version from 34 to 35
2023-05-05 12:01:41,605 Trac[api] INFO: Upgraded database_version from 35 to 36
2023-05-05 12:01:42,192 Trac[console] ERROR: Exception in trac-admin command: u'upgrade'
Traceback (most recent call last):
 File "build/bdist.linux-x86_64/egg/trac/admin/console.py", line 112, in onecmd
   rv = cmd.Cmd.onecmd(self, line) or 0
 File "/usr/lib/python2.7/cmd.py", line 220, in onecmd
   return self.default(line)
 File "build/bdist.linux-x86_64/egg/trac/admin/console.py", line 291, in default
   return self.cmd_mgr.execute_command(*args)
 File "build/bdist.linux-x86_64/egg/trac/admin/api.py", line 127, in execute_command
   return f(*fargs)
 File "build/bdist.linux-x86_64/egg/trac/env.py", line 1111, in _do_upgrade
   self.env.upgrade(backup=no_backup is None)
 File "build/bdist.linux-x86_64/egg/trac/env.py", line 848, in upgrade
   participant.upgrade_environment(*args)
 File "build/bdist.linux-x86_64/egg/trac/env.py", line 888, in upgrade_environment
   DatabaseManager(self.env).upgrade(db_default.db_version)
 File "build/bdist.linux-x86_64/egg/trac/db/api.py", line 576, in upgrade
   script.do_upgrade(self.env, i, cursor)
 File "build/bdist.linux-x86_64/egg/trac/upgrades/db37.py", line 40, in do_upgrade
   cursor.execute('ALTER TABLE %s %s' % (db.quote(tab), mods))
 File "build/bdist.linux-x86_64/egg/trac/db/util.py", line 73, in execute
   return self.cursor.execute(sql)
 File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 250, in execute
   self.errorhandler(self, exc, value)
 File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 50, in defaulterrorhandler
   raise errorvalue

[Frank R.]

 File "build/bdist.linux-x86_64/egg/trac/upgrades/db37.py", line 40, in do_upgrade
   cursor.execute('ALTER TABLE %s %s' % (db.quote(tab), mods))

 As far as I can tell, that code tries exactly what failed for me when trying manually, to change the collation of columns.

[Jun Omae]

On Fri, May 5, 2023 at 7:35 PM Frank R. <frank.re...@gmail.com> wrote:
> File "build/bdist.linux-x86_64/egg/trac/upgrades/db37.py", line 40, in do_upgrade
> cursor.execute('ALTER TABLE %s %s' % (db.quote(tab), mods))
> File "build/bdist.linux-x86_64/egg/trac/db/util.py", line 73, in execute
> return self.cursor.execute(sql)
> File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 250, in execute
> self.errorhandler(self, exc, value)
> File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 50, in defaulterrorhandler
> raise errorvalue
> OperationalError: (1025, 'Error on rename of \'./trac/#sql-458_c9f\' to \'./trac/session_attribute\' (errno: 184 "Tablespace already exists")')

The db37.py failed in your environment. The upgrade step alters tables
to modify columns which is TEXT type with MEDIUMTEXT type.

I tried to reproduce it with MariaDB 10.1.48, but it is unable to
reproduce it. I think that is an issue of MariaDB and/or your data
directory has something wrong.

Please report it to MariaDB bug tracker. Also, check
/var/lib/mysql/trac directory in your environment.

[Frank R.]

Thanks a lot for your efforts!

Also, check
/var/lib/mysql/trac directory in your environment.

Hmmm... but what to look for?

[Jun Omae]

On Mon, May 8, 2023 at 6:27 PM Frank R. <frank.re...@gmail.com> wrote:
>> Also, check
>> /var/lib/mysql/trac directory in your environment.
>
> Hmmm... but what to look for?

I suspect that /var/lib/mysql/trac/session.* files already exist
because of "Tablespace already exists" error. However, I'm not an
InnoDB expert so I don't know exactly.

Please ask on MariaDB community.

[RjOllos]

You may be better off just dropping the database and recreating it. For proper creation steps, see:

https://trac.edgewall.org/wiki/MySqlDb 

[Frank R.]

You may be better off just dropping the database and recreating it.

... which means to mysqldump the existing DB, create it according to

https://trac.edgewall.org/wiki/MySqlDb 

and then reimport it. But I wasn't able to find out how to reimport maintaing the new collation...?

[RjOllos]

I'm pretty sure that it's the database creation that describes the collation, so the import should change the collation. 

[Frank R.]

I'm pretty sure that it's the database creation that describes the collation, so the import should change the collation. 

Ok, thank you, seems to have worked out. But now trac-admin runs into

OperationalError: (1071, 'Specified key was too long; max key length is 767 bytes')

The character set change has seemingly blown up the index entries beyond the MariaDB field limit.

[Jun Omae]

* Use "innodb_large_prefix = 1".
* Provide commands which you invoked and the output.
* Check trac.log and provide the errors in the log, if you encounter errors.

> --
> You received this message because you are subscribed to the Google Groups "Trac Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to trac-users+...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/trac-users/CA%2ByhgXC6Z4RmNBYk1MWObcxSiAnHTLMqdRgU%3D%2BpGgDch7jPDqw%40mail.gmail.com.

[Frank R.]

Am Di., 16. Mai 2023 um 10:32 Uhr schrieb Jun Omae <jun...@gmail.com>:

* Use "innodb_large_prefix = 1".

I found several forum entries telling me that is not enough. One has to set

[mariadb]
innodb_file_format = Barracuda
innodb_file_per_table = on
innodb_default_row_format = dynamic
innodb_large_prefix = 1
innodb_file_format_max = Barracuda

But I am struggling with where to add that block. I've put it into /etc/mysql/mariadb.conf.d/50-server.cnf, into a new 60-fix-key-length.cnf, but none does work. After restarting mariadb I get

MariaDB [(none)]> SHOW GLOBAL VARIABLES LIKE 'innodb_large%';
+---------------------+-------+
| Variable_name       | Value |
+---------------------+-------+
| innodb_large_prefix | OFF   |
+---------------------+-------+
1 row in set (0.00 sec)

[Frank R.]

I had to add the above block directly to my.cnf, then global variables are set correctly and trac upgrade runs withour error. Now, starting trac in the browser, I get

Warnung: Fehler beim Navigations-Bereitsteller "AccountModule"

log:

WARNING: Error with navigation contributor AccountModule: ConfigurationError: Kann keine Implementierung der <code>IPasswordHashMethod</code> Sc
hnittstelle mit dem Namen <code>HtDigestHashMethod</code> finden. Überprüfen Sie, ob die Komponente aktiviert ist, oder aktualisieren Sie die Option <code>[account-manager] hash_met
hod</code> in trac.ini.

[Frank R.]

After adding

acct_mgr.pwhash.HtDigestHashMethod = enabled

to trac.ini the error disappeared.

[Frank R.]

Trying to display tickets in my upgraded trac I get

2023-05-17 11:52:34,014 Trac[main] ERROR: [192.1.4.222] Internal Server Error: <RequestWithSession "GET '/report/3'">, referrer 'http://192.1.8.223:8080/trac/report'

Traceback (most recent call last):

 File "build/bdist.linux-x86_64/egg/trac/web/main.py", line 647, in _dispatch_request
   dispatcher.dispatch(req)
 File "build/bdist.linux-x86_64/egg/trac/web/main.py", line 215, in dispatch
   chosen_handler = self._pre_process_request(req, chosen_handler)
 File "build/bdist.linux-x86_64/egg/trac/web/main.py", line 455, in _pre_process_request
   chosen_handler = filter_.pre_process_request(req, chosen_handler)
 File "build/bdist.linux-x86_64/egg/dynvars/web_ui.py", line 22, in pre_process_request
   self._validate_request(req, check_referer=True) # redirect if needed
 File "build/bdist.linux-x86_64/egg/dynvars/web_ui.py", line 56, in _validate_request
   vars = self._extract_vars(report)
 File "build/bdist.linux-x86_64/egg/dynvars/web_ui.py", line 98, in _extract_vars
   db = self.env.get_db_cnx()
AttributeError: 'Environment' object has no attribute 'get_db_cnx'

[Markus Rosjat]

Hi Frank,

if I remember correct get_db_cnx was deprecated in 1.2 or so so you might wanna check your Plugins to use a version that is compatible with the trac version you running now.

cheers

Markus

--
You received this message because you are subscribed to the Google Groups "Trac Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to trac-users+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/trac-users/CA%2ByhgXAD7_R5fh-v6a8VrXmHPW2OxC1zXMjjrnK3mB%2B7nuF2OQ%40mail.gmail.com.

[Frank R.]

Hi Markus,

if I remember correct get_db_cnx was deprecated in 1.2 or so so you might wanna check your Plugins to use a version that is compatible with the trac version you running now.

That's what I found out. And it was plugin TracDynamicVariables. Deactivating that plugin and the error was gone. But I don't remember or know whether other plugins do need this plugin... maybe my sneaky feeling prove true, because I now get

TypeError: childtickets() takes exactly 2 arguments (1 given)

Being forced to deactivate this plugin (ChildTickets) would break what's documented over years in Trac, and what we are used to.

[Markus Rosjat]

Hi Frank,

Am Mi., 17. Mai 2023 um 15:55 Uhr schrieb Frank R. <frank.re...@gmail.com>:

Hi Markus,

if I remember correct get_db_cnx was deprecated in 1.2 or so so you might wanna check your Plugins to use a version that is compatible with the trac version you running now.

That's what I found out. And it was plugin TracDynamicVariables. Deactivating that plugin and the error was gone. But I don't remember or know whether other plugins do need this plugin... maybe my sneaky feeling prove true, because I now get

TypeError: childtickets() takes exactly 2 arguments (1 given)

as i  said you might try to find a more recent version of the plugin or if you feel comfy enough try to fix the code. 

 

Being forced to deactivate this plugin (ChildTickets) would break what's documented over years in Trac, and what we are used to.

yeah i can understand the concern, we dont really use that many plugins beside the really common ones  and those get somehow an update by someone in the community or the developers.

 

--
You received this message because you are subscribed to the Google Groups "Trac Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to trac-users+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/trac-users/CA%2ByhgXA8KYfv8k%3DxgaRLuyP69PWMxzeXiGH2uBLTSAS871wxDw%40mail.gmail.com.

[Peter Suter]

On 17 May 2023 15:55, Frank R. wrote:

That's what I found out. And it was plugin TracDynamicVariables. Deactivating that plugin and the error was gone. But I don't remember or know whether other plugins do need this plugin... maybe my sneaky feeling prove true, because I now get

TypeError: childtickets() takes exactly 2 arguments (1 given)

(I would recommend new threads for new problems, and always include the full error message and related plugin versions etc.)

This error seems to  match the one in https://trac-hacks.org/ticket/13088
so you probably have a very old ChildTicketsPlugin and need the newer version mentioned there.

[RjOllos]

 You'll likely need to upgrade all of your plugins. I would recommend disabling all of them, upgrading one at a time, then re-enabling.

[Frank R.]

Thanks a lot, now it works.

@Peter Suter

Sorry, I am a trac...@googlegroups.com newbie, I promise I'll be better! :-)