Questions on upgrading Gerrit.

220 views
Skip to first unread message

Joshua J. Kugler

unread,
Sep 27, 2012, 10:08:59 PM9/27/12
to repo-d...@googlegroups.com
I have two questions.

1) Clarification
https://gerrit.googlesource.com/gerrit/+/stable-2.5/ReleaseNotes/ReleaseNotes-2.5.txt
says:

"*WARNING:* Upgrading to 2.5.x requires the server be first upgraded to 2.1.7
(or a later 2.1.x version), and then to 2.5.x. If you are upgrading from
2.2.x.x or newer, you may ignore this warning and upgrade directly to 2.5.x."

So, I have 2.1.8 (which is later than 2.1.7), but then it says "If you are
upgrading from 2.2.x.x or newer..." What about releases between 2.1.7 and
2.2.x.x? Can I upgrade directly from 2.1.8 to 2.5?

2) Glitch (?) in upgrade
So, I downloaded gerrit-full-2.5-rc0.war, and ran it thusly:

java -jar gerrit-full-2.5-rc0.war init -d /path/to/existing/install

But, it instead of just walking through the upgrade, it 1) started presenting
me with questions about config, even though there is an existing config, and 2)
the defaults it offered me did not match with what was in the existing config
file. I can see presenting the current config as defaults to double-check the
config before upgrade, but the prompts not even matching the current config
seems a big odd. Is this designed behavior, or is it a bug?

Thanks!

j

--
Joshua J. Kugler - Fairbanks, Alaska
Azariah Enterprises - Programming and Website Design
jos...@azariah.com - Jabber: peda...@gmail.com
PGP Key: http://pgp.mit.edu/ ID 0x73B13B6A

Shawn Pearce

unread,
Sep 28, 2012, 12:42:28 AM9/28/12
to Joshua J. Kugler, repo-discuss
On Thu, Sep 27, 2012 at 7:08 PM, Joshua J. Kugler <jos...@azariah.com> wrote:
>
> I have two questions.
>
> 1) Clarification
> https://gerrit.googlesource.com/gerrit/+/stable-2.5/ReleaseNotes/ReleaseNotes-2.5.txt
> says:
>
> "*WARNING:* Upgrading to 2.5.x requires the server be first upgraded to 2.1.7
> (or a later 2.1.x version), and then to 2.5.x. If you are upgrading from
> 2.2.x.x or newer, you may ignore this warning and upgrade directly to 2.5.x."
>
> So, I have 2.1.8 (which is later than 2.1.7), but then it says "If you are
> upgrading from 2.2.x.x or newer..." What about releases between 2.1.7 and
> 2.2.x.x? Can I upgrade directly from 2.1.8 to 2.5?

Yes. What is missing is the ability to do 2.1.5 -> 2.2.x for example.
We deleted those ancient upgrade paths in 2.2.0.

> 2) Glitch (?) in upgrade
> So, I downloaded gerrit-full-2.5-rc0.war, and ran it thusly:
>
> java -jar gerrit-full-2.5-rc0.war init -d /path/to/existing/install
>
> But, it instead of just walking through the upgrade, it 1) started presenting
> me with questions about config, even though there is an existing config,

This is normal. init runs in an interactive mode and prompts with the
current configuration.

If you pass --batch it won't prompt, it will just do the upgrade. At
the end you will be given a block of SQL you have to run by hand to
clean up the database. If you omit --batch (as you did above) you will
be shown this block of SQL and asked if you want Gerrit to run it
immediately.

> and 2)
> the defaults it offered me did not match with what was in the existing config
> file. I can see presenting the current config as defaults to double-check the
> config before upgrade, but the prompts not even matching the current config
> seems a big odd. Is this designed behavior, or is it a bug?

This does seem to be a problem. Can the user you ran the init command
as actually read the file at
/path/to/existing/install/etc/gerrit.config ? It seems like maybe not
if the prompts aren't describing current settings.

Joshua J. Kugler

unread,
Sep 28, 2012, 7:58:36 PM9/28/12
to repo-d...@googlegroups.com
On Thursday, September 27, 2012 21:42:28 Shawn Pearce wrote:
> > So, I have 2.1.8 (which is later than 2.1.7), but then it says "If you are
> > upgrading from 2.2.x.x or newer..." What about releases between 2.1.7 and
> > 2.2.x.x? Can I upgrade directly from 2.1.8 to 2.5?
>
> Yes. What is missing is the ability to do 2.1.5 -> 2.2.x for example.
> We deleted those ancient upgrade paths in 2.2.0.

Thanks. It might be clearer if it said something along the lines of:

To upgrade to 2.5, gerrit must be at version 2.1.7 or later. If your Gerrit
version is older than 2.1.7, please upgrade to 2.1.7 first, then to 2.5.

> If you pass --batch it won't prompt, it will just do the upgrade. At
> the end you will be given a block of SQL you have to run by hand to
> clean up the database. If you omit --batch (as you did above) you will
> be shown this block of SQL and asked if you want Gerrit to run it
> immediately.
>
> This does seem to be a problem. Can the user you ran the init command
> as actually read the file at
> /path/to/existing/install/etc/gerrit.config ? It seems like maybe not
> if the prompts aren't describing current settings.

I ran the following (as 'gerrit', the user as which gerrit runs):

$ ls -l /opt/local/gerrit/review.whamcloud.com/etc/gerrit.config
-rw-r--r-- 1 gerrit root 1362 Jun 7 17:14
/opt/local/gerrit/review.domain.com/etc/gerrit.config

java -jar gerrit-full-2.5-rc0.war init -d /opt/local/gerrit/review.domain.com

And I get this:

$ java -jar gerrit-full-2.5-rc0.war init -d
/opt/local/gerrit/review.whamcloud.com

*** Gerrit Code Review 2.5-rc0
***


*** Git Repositories
***

Location of Git repositories [/srv/source/git]:

*** SQL Database
***

Database server type [H2/?]:

It gets the first one right (based on past preferences) but then doesn't
default to MySQL (the currently used DB).

If i do this:

$ java -jar gerrit-full-2.5-rc0.war init --batch -d
/opt/local/gerrit/review.domain.com
Upgrading database schema from version 52 to 53 ...
Exception in thread "main" com.google.gwtorm.server.OrmException: Cannot
create repository -- All Projects --

(I'll attach the full traceback)

So, as of right now, I'm stuck.

Ideas?
upgrade_traceback.txt

Shawn Pearce

unread,
Sep 28, 2012, 9:37:43 PM9/28/12
to Joshua J. Kugler, repo-discuss
I wonder how you have the database configured. If its not the standard
way its written into gerrit.config, maybe the init code is confused
and doesn't know what to do.

> If i do this:
>
> $ java -jar gerrit-full-2.5-rc0.war init --batch -d
> /opt/local/gerrit/review.domain.com
> Upgrading database schema from version 52 to 53 ...
> Exception in thread "main" com.google.gwtorm.server.OrmException: Cannot
> create repository -- All Projects --

"Creating directories for /srv/source/git/-- All Projects --.git failed"

Seems your user can't make this directory on your filesystem.

Chris Barrera

unread,
Oct 1, 2012, 10:49:02 AM10/1/12
to repo-d...@googlegroups.com, Joshua J. Kugler
We had this same problem going from 2.1.8->2.3 where All-Projects was attempted to be created twice, and removing it to restart the upgrade resulted in duplicate groups uuids in refs/meta/config... https://groups.google.com/forum/?fromgroups=#!searchin/repo-discuss/Barrera/repo-discuss/6pRno8Mwzhw/iRI1D42-hBwJ

Joshua J. Kugler

unread,
Oct 1, 2012, 7:19:32 PM10/1/12
to repo-d...@googlegroups.com, Shawn Pearce
On Friday, September 28, 2012 18:37:43 Shawn Pearce wrote:
> > $ java -jar gerrit-full-2.5-rc0.war init --batch -d
> > /opt/local/gerrit/review.domain.com
> > Upgrading database schema from version 52 to 53 ...
> > Exception in thread "main" com.google.gwtorm.server.OrmException: Cannot
> > create repository -- All Projects --
>
> "Creating directories for /srv/source/git/-- All Projects --.git failed"
>
> Seems your user can't make this directory on your filesystem.

So, yes, /srv/source/git was owned by root (as we usually create repos under
directories owned by gerrit). But the question is: why is trying to create
repos at all? There are "permission only" projects. I see it's trying to
create projects named after teh "permissions projects." Is that expected?
Guess I'll need to read back through the change log.

But, on to the errors. :) So, I get this (full traceback attached). Is there
a way to get a more complete error message (see Caused by: at the end of the
traceback). I googled, and didn't get anything relevant.

j

$ java -jar gerrit-full-2.5-rc0.war init --batch -d
/opt/local/gerrit/review.whamcloud.com
Upgrading database schema from version 52 to 53 ...
Upgrading database schema from version 53 to 54 ...
Upgrading database schema from version 54 to 55 ...
Renaming "-- All Projects --" to "All-Projects"
Upgrading database schema from version 55 to 56 ...
Upgrading database schema from version 56 to 57 ...
Upgrading database schema from version 57 to 58 ...
Upgrading database schema from version 58 to 59 ...
Upgrading database schema from version 59 to 60 ...
Exception in thread "main" com.google.gwtorm.server.OrmException: update
failure on change_messages
full_traceback.txt

Magnus Bäck

unread,
Oct 1, 2012, 7:30:25 PM10/1/12
to repo-d...@googlegroups.com
On Monday, October 01, 2012 at 19:19 EDT,
"Joshua J. Kugler" <jos...@azariah.com> wrote:

> So, yes, /srv/source/git was owned by root (as we usually create repos
> under directories owned by gerrit). But the question is: why is trying
> to create repos at all? There are "permission only" projects. I see
> it's trying to create projects named after teh "permissions projects."
> Is that expected? Guess I'll need to read back through the change log.

This is expected. As of Gerrit 2.2, project settings were moved
to the refs/meta/config branch of each git. This means that also
permissions-only projects need a git.

[...]

--
Magnus Bäck
ba...@google.com

Shawn Pearce

unread,
Oct 1, 2012, 8:47:11 PM10/1/12
to Joshua J. Kugler, repo-discuss
On Mon, Oct 1, 2012 at 4:19 PM, Joshua J. Kugler <jos...@azariah.com> wrote:
> But, on to the errors. :) So, I get this (full traceback attached). Is there
> a way to get a more complete error message (see Caused by: at the end of the
...
> Exception in thread "main" com.google.gwtorm.server.OrmException: update
> failure on change_messages

Looks like you have some disagreement between Gerrit and your database
about what character encoding should be used. Gerrit read something
from the database, then tried to store it back, and the database
doesn't like the encoding selected. This suggests a mismatch between
how the database is setup for character encoding, and maybe what the
JDBC driver Gerrit is using knows.

Joshua J. Kugler

unread,
Oct 1, 2012, 9:03:36 PM10/1/12
to repo-d...@googlegroups.com
Ah, crud. OK. The server is set to default to utf8:

default-collation=utf8_bin
character-set-server=utf8
collation-server=utf8_bin
default-character-set=utf8

But there are some tables with a latin1 encoding. Well...I'll give converting
those a go, and see if my results change.

Thanks for all your help so far. :)

j

Joshua J. Kugler

unread,
Oct 1, 2012, 9:52:03 PM10/1/12
to repo-d...@googlegroups.com
On Monday, October 01, 2012 17:03:36 Joshua J. Kugler wrote:
> Ah, crud. OK. The server is set to default to utf8:
>
> default-collation=utf8_bin
> character-set-server=utf8
> collation-server=utf8_bin
> default-character-set=utf8
>
> But there are some tables with a latin1 encoding. Well...I'll give
> converting those a go, and see if my results change.

So, I tried to convert t UTF8. But I hit this:

http://islandlinux.org/howto/mysql%2derror%2dspecified%2dkey%2dwas%2dtoo%2dlong%2dmax%2dkey%2dlength%2d1000%2dbytes

ARRRRGGH!

Well, I'm calling it a day. Will keep you updated on progress. :-D

I suppose I could convert all tables to latin1 and see if that fixes the issue,
but I really don't want to do that. Maybe I'll convert it to PostgreSQL. :P

Shawn Pearce

unread,
Oct 1, 2012, 10:20:13 PM10/1/12
to Joshua J. Kugler, repo-discuss
On Mon, Oct 1, 2012 at 6:52 PM, Joshua J. Kugler <jos...@azariah.com> wrote:
> On Monday, October 01, 2012 17:03:36 Joshua J. Kugler wrote:
>> Ah, crud. OK. The server is set to default to utf8:
>>
>> default-collation=utf8_bin
>> character-set-server=utf8
>> collation-server=utf8_bin
>> default-character-set=utf8
>>
>> But there are some tables with a latin1 encoding. Well...I'll give
>> converting those a go, and see if my results change.
>
> So, I tried to convert t UTF8. But I hit this:
>
> http://islandlinux.org/howto/mysql%2derror%2dspecified%2dkey%2dwas%2dtoo%2dlong%2dmax%2dkey%2dlength%2d1000%2dbytes
>
> ARRRRGGH!
>
> Well, I'm calling it a day. Will keep you updated on progress. :-D
>
> I suppose I could convert all tables to latin1 and see if that fixes the issue,
> but I really don't want to do that. Maybe I'll convert it to PostgreSQL. :P

Does switching to InnoDB increase the maximum primary key length?

Bailey, Darragh

unread,
Oct 2, 2012, 5:51:48 AM10/2/12
to repo-d...@googlegroups.com
On 02/10/12 02:52, Joshua J. Kugler wrote:
> So, I tried to convert t UTF8. But I hit this:
>
> http://islandlinux.org/howto/mysql%2derror%2dspecified%2dkey%2dwas%2dtoo%2dlong%2dmax%2dkey%2dlength%2d1000%2dbytes
>
> ARRRRGGH!
>
> Well, I'm calling it a day. Will keep you updated on progress. :-D
>
> I suppose I could convert all tables to latin1 and see if that fixes the issue,
> but I really don't want to do that. Maybe I'll convert it to PostgreSQL. :P
>
> j

No need to go that extreme :P


You might find the script attached to this launchpad bug useful
https://bugs.launchpad.net/openstack-ci/+bug/979227


As an added bonus, switching to utf8 & innodb fixes the following
problem as well:
http://code.google.com/p/gerrit/issues/detail?id=1101

--
Regards,
Darragh Bailey

Joshua J. Kugler

unread,
Nov 7, 2012, 8:02:01 PM11/7/12
to repo-d...@googlegroups.com, Shawn Pearce
On Monday, October 01, 2012 19:20:13 Shawn Pearce wrote:
> Does switching to InnoDB increase the maximum primary key length?

Yes, it does.

I converted all tables to InnoDB, then ran this:

echo "ALTER TABLE $T CONVERT TO CHARACTER SET utf8;"|mysql reviewdb2;

for all tables, and received no errors.

Now, trying to run the gerit upgrade via:

sudo -u gerrit java -jar gerrit-full-2.5.war init --batch -d
/opt/local/gerrit/review.whamcloud.com

I get the attached traceback, but the relevant parts are:

Caused by: java.sql.BatchUpdateException: Incorrect string value: '\xA0To
re...' for column 'message' at row 1
at
com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:2007)
at
com.mysql.jdbc.PreparedStatement.executeBatch(PreparedStatement.java:1443)
at com.google.gwtorm.schema.sql.SqlDialect.executeBatch(SqlDialect.java:370)
at com.google.gwtorm.jdbc.JdbcAccess.execute(JdbcAccess.java:438)
at com.google.gwtorm.jdbc.JdbcAccess.updateAsBatch(JdbcAccess.java:261)
at com.google.gwtorm.jdbc.JdbcAccess.update(JdbcAccess.java:214)

So, presumably, if that's what it pulls out of the database, it should be able
to put it back in the database, yes?

Does this indicate I simply have bad UTF data I will need to trace down
manually and fix?
full_traceback.txt
Reply all
Reply to author
Forward
0 new messages