Upgrade - 5.0.3 to latest

[Munish Joshi]

Hello,

I am trying to upgrade my bugzilla version from 5.0.3 to latest but while
running checksetup.pl, i am getting following error -

"Removing existing compiled templates...
Precompiling templates...done.
Fixing file permissions...
Not a reference at Bugzilla/DB/Schema.pm line 2916."

Please advise.

-thanks

[Thorsten Schöning]

Guten Tag Munish Joshi,
am Dienstag, 14. Mai 2019 um 22:39 schrieben Sie:

> I am trying to upgrade my bugzilla version from 5.0.3 to latest but while
> running checksetup.pl, i am getting following error -

[...]

> Not a reference at Bugzilla/DB/Schema.pm line 2916."

What does "latest" exactly mean? Are you sure not upgrading to some
unstable development version? I'm pretty sure to run the most recent
stable release 5.0.6 and in my case line 2916 of your mentioned file
is a comment. So I suggest checking your used branches first.

https://github.com/bugzilla/bugzilla/blob/5.0/Bugzilla/DB/Schema.pm#L2916

Mit freundlichen Grüßen,

Thorsten Schöning

--
Thorsten Schöning E-Mail: Thorsten....@AM-SoFT.de
AM-SoFT IT-Systeme http://www.AM-SoFT.de/

Telefon...........05151- 9468- 55
Fax...............05151- 9468- 88
Mobil..............0178-8 9468- 04

AM-SoFT GmbH IT-Systeme, Brandenburger Str. 7c, 31789 Hameln
AG Hannover HRB 207 694 - Geschäftsführer: Andreas Muchow

[Thorsten Schöning]

Guten Tag Munish Joshi,
am Freitag, 17. Mai 2019 um 01:17 schrieben Sie:

Please make sure to always repsond to the list, I will ignore further
private mails.

> Private Emails Do Not Help Others

https://www.bugzilla.org/support/

> Sorry I am currently running 5.0.3 and trying to upgrade to 5.0.6. 

Please read my answers more careful, your are most likely on master
instead of branch "5.0" providing version 5.0.6:

https://github.com/bugzilla/bugzilla/blob/master/Bugzilla/DB/Schema.pm#L2916

[munis...@gmail.com]

Error - Fixing file permissions...

Not a reference at Bugzilla/DB/Schema.pm line 2916.

and below is my Schema.pm snippet for lines 2916 - and i am running 5.0.3 version.


2902 =item C<columns_equal($col_one, $col_two)>
2903
2904 Description: Tells you if two columns have entirely identical definitions.
2905 The TYPE field's value will be compared case-insensitive.
2906 However, all other fields will be case-sensitive.
2907 Params: $col_one, $col_two - The columns to compare. Hash
2908 references, in C<ABSTRACT_SCHEMA> format.
2909 Returns: C<1> if the columns are identical, C<0> if they are not.
2910
2911 =back
2912
2913 =cut
2914
2915 my $self = shift;
2916 my $col_one = dclone(shift);
2917 my $col_two = dclone(shift);
2918
2919 $col_one->{TYPE} = uc($col_one->{TYPE});
2920 $col_two->{TYPE} = uc($col_two->{TYPE});
2921
2922 # We don't care about foreign keys when comparing column definitions.
2923 delete $col_one->{REFERENCES};
2924 delete $col_two->{REFERENCES};
2925
2926 my @col_one_array = %$col_one;
2927 my @col_two_array = %$col_two;
2928
2929 my ($removed, $added) = diff_arrays(\@col_one_array, \@col_two_array);
2930
2931 # If there are no differences between the arrays, then they are equal.
2932 return !scalar(@$removed) && !scalar(@$added) ? 1 : 0;
2933 }
2934
2935
2936 =head1 SERIALIZATION/DESERIALIZATION
2937
2938 =over 4

[Thorsten Schöning]

Guten Tag munis...@gmail.com,
am Samstag, 18. Mai 2019 um 07:22 schrieben Sie:

> and below is my Schema.pm snippet for lines 2916 - and i am running 5.0.3 version.

Which doesn't make any sense if you are writing you want to upgrade to
5.0.6 and are providing currently running code belonging to 5.0.3.
Agreed?

If you want to upgrade to 5.0.6 you obviously need to run
checksetup.pl from 5.0.6. If you provide code from 5.0.3 from
Schema.pm you are either running 5.0.3 at all, so you are not
upgrading to 5.0.6, or you you somehow mixed up your installation to
be a combination of 5.0.3 and 5.06 which is totally broken of course
and might explain the erorr you are getting.

So I suggest starting newly from scratch by following pretty much what
the docs say:

https://bugzilla.readthedocs.io/en/5.0/installing/upgrading.html

The only thing wrong in the docs are the mentioned branches and tags,
you either need to checkout "release-5.0.6" or preferred "5.0". The
latter will get updates merged into it for newer 5.0-releases the
former won't. These things have changed recently and the docs not
updated yet.

If you run into trouble again afterwards, please provide ALL and the
EXACT commands you have used, which configuration you have changed
where, which OS, Perl, web server, database etc. you are running and
all that basic stuff.

[Munijosh]

Thanks - I followed the upgrade guide and uses Git to pull new stable version 5.0.6.

After executing # git pull on correct directory.

Checksetup script showing this version
#./checksetup.pl --version
* This is Bugzilla 5.0.6 on perl 5.10.1
* Running on Linux 2.6.32-696.3.1.el6.x86_64 #1 SMP Tue May 30 19:52:55 UTC 2017

But if i tried to run #checksetup.pl now -

Following error is coming -

Bugzilla now uses the InnoDB storage engine in MySQL for most tables.
Converting tables to InnoDB:
Converting table bugs_fulltext... DBD::mysql::db do failed: The used table type doesn't support FULLTEXT indexes [for Statement "ALTER TABLE bugs_fulltext ENGINE = InnoDB"] at Bugzilla/DB/Mysql.pm line 391
Bugzilla::DB::Mysql::bz_setup_database('Bugzilla::DB::Mysql=HASH(0x56df8e8)') called at ./checksetup.pl line 123

I'm using MySql version - Server version: 5.1.73.

-Thanks.

[Munijosh]

I think this may be due to - Table - bugs_fulltext using - MyISAM Engine where as other all tables are using InnoDB.

If this is correct, than do you have any way to change the table type ?

If i tried to change using "Alter table"

it given me error = mysql> alter table bugs.bugs_fulltext engine=innodb;
ERROR 1214 (HY000): The used table type doesn't support FULLTEXT indexes
mysql>

Is this related to MySQL version 5.1.73 not supported ?

[Thorsten Schöning]

Guten Tag Munijosh,
am Sonntag, 19. Mai 2019 um 05:17 schrieben Sie:

> it given me error = mysql> alter table bugs.bugs_fulltext engine=innodb;
> ERROR 1214 (HY000): The used table type doesn't support FULLTEXT indexes
mysql>>

That is exactly what Bugzilla is trying to do itself.

> Is this related to MySQL version 5.1.73 not supported ?

Yes, 5.1.73 is too old, you need a version supporting FULLTEXT for
InnoDB. You need at least 5.6, which supports FULLTEXT INDEX for
tables using InnoDB, which is the default these days by Bugzilla.

https://dev.mysql.com/doc/refman/5.6/en/innodb-fulltext-index.html

[Munijosh]

Upgrade MySQL to 5.6.16 and re-run

# .\checksetup.pl

Script automatically converted MyISAM table into InnoDB.

But now, following error showing when i am running .\Checksetup.pl

Removing existing compiled templates...
Precompiling templates...done.

Fixing file permissions...
Not a reference at Bugzilla/DB/Schema.pm line 2889.


Below is the snippet of 2889 code lines -

2877 Description: Tells you if two columns have entirely identical definitions.
2878 The TYPE field's value will be compared case-insensitive.
2879 However, all other fields will be case-sensitive.
2880 Params: $col_one, $col_two - The columns to compare. Hash
2881 references, in C<ABSTRACT_SCHEMA> format.
2882 Returns: C<1> if the columns are identical, C<0> if they are not.
2883
2884 =back
2885
2886 =cut
2887
2888 my $self = shift;
2889 my $col_one = dclone(shift);
2890 my $col_two = dclone(shift);
2891
2892 $col_one->{TYPE} = uc($col_one->{TYPE});
2893 $col_two->{TYPE} = uc($col_two->{TYPE});


Let me know if this is OK - Or need further troubleshooting.

-Thanks

[Thorsten Schöning]

Guten Tag Munijosh,
am Sonntag, 19. Mai 2019 um 21:56 schrieben Sie:

> Not a reference at Bugzilla/DB/Schema.pm line 2889.

Now that we know you are running the correct version of Bugzilla, it
seems that your schema is broken for some reason. Someone might have
edited something on database level where one shouldn't have and things
got out of sync.

Bugzilla stores how the database should look like within the database
in the table bz_schema and updates that piece of information e.g. when
adding or removing custom fields, extensions etc. During upgrades, that
data is used to compare what the current definition of e.g. some table
is and how it should look like according to the version of Bugzilla
you are upgrading to. Afterwards Bugzilly decides what to do and in
your case it seems some piece of information is missing from the
stored schema.

You need to debug this further by looking at the stored schema for
e.g. extensions you are using and might not fully be reflected in the
schema or (more likely) custom fields you are using maybe not
available in the schema at all because they have wrongly been created
on the database level only.

https://bugzilla.mozilla.org/show_bug.cgi?id=1483226

[Munijosh]

Is there way to force Bz_schema to recreate with new database/tables information ?

Seems like going over each tables, their field values and types and compare with BZ_schema is almost impossible.

Please suggest some alternative.

[Thorsten Schöning]

Guten Tag Munijosh,
am Dienstag, 21. Mai 2019 um 00:08 schrieben Sie:

> Is there way to force Bz_schema to recreate with new database/tables information ?

No and it wouldn't make much sense anyway: Bugzilla can't work with an
arbitrary changed database schema, that's the whole point of keeping
track of changes itself does on the schema at runtime.

Just think of it: Bugzilla's code might assume that some colums are
NULLable and others are not and someone changes both at the database
level. What does it help that Bugzilla knows that because of a
regenerated schema? The code simply doesn't work that way and will
fail at some point.

> Please suggest some alternative.

First of all install a fresh new Bugzilla 5.0.6 without using your old
database and see if that works. If it does, it's another hint that
your current schema is broken for some reason. If it doesn't work as
well, my former educated guess might have been wrong and there's a
problem somewhere else in your concrete setup, maybe even a bug in
Bugzilla nobody came across yet.

But if it works, that's another hint that your current schema is broken
for some reason. If you don't want to debug your existing schema
further, the only thing I can think of is migrating your existing bugs
by exporting/importing to the newly installed Bugzilla and deal with
things like missing custom fields etc. as needed. If you recreate
everything using the official UI, you shouldn't be running into
inconsistent schema anymore in theory.

https://bugzilla.readthedocs.io/en/5.0/using/finding.html#bug-lists
https://www.bugzilla.org/docs/4.4/en/html/api/importxml.html
https://www.bugzilla.org/docs/4.4/en/html/api/contrib/bzdbcopy.html

If I was you, I would prefer XML, because that seems to be the best
seperation between your two schemas. The only downside AFAIK is that
things like attachments are only exported to XML if XML-generation is
invoked per bug instead of for a search result. The reason is that in
case of a search result "excludefield=attachmentdata" is forwarded by
the UI, which is not the case for individual bugs.

So, you have different options: Invoke the URL per individual bug or a
group of bugs at the shell like in the following example:

> http://bugzilla.example.org/show_bug.cgi?ctype=xml&id=243
> http://bugzilla.example.org/show_bug.cgi?ctype=xml&id=243&id=...&id=...

Or do the same per individual bug using the link "XML" in the bottom
right of each bug or use searches and the button "XML" in the bottom
left of the search results. Only in the last case attachments are not
provvidfed by default, if you want those as well, you need to use the
developer tools of your browser to change the containing form and
remove the hidden input "excludefield".

You need to test what works best for you, depending on the number of
bugs, how large the attachments are and stuff.