Migrating gerrit (2.9.1) to a different server

108 views
Skip to first unread message

Pharthiphan Asokan

unread,
Jul 4, 2022, 1:44:44 AM7/4/22
to Repo and Gerrit Discussion
Hi,

I am trying to migrate the Gerrit server from one location to another (to a different server)

Gerrit server version is 2.9.1 (Eventually I want it to be upgraded to 3.6.1)

Steps I followed,

dump and restore PostgreSQL DB from old server to new
java -jar gerrit-2.9.1.war init -d git # new_server
java -jar gerrit-2.9.1.war reindex -d git #new_server


I see the following error on reindexing, is it safe to ignore or can I do something to fix this

Reindexing changes: projects: 63% (106/168), 14% (11148/77744) (\)[2022-07-04 04:21:11,681] ERROR com.google.gerrit.server.index.ChangeBatchIndexer : repository not found: /localuser/gerrit/git/git/woslcoud

java -jar gerrit-2.10.war init --batch -d git # successful and I am able to access gerrit

Upgrade from 2.10 to 2.11/ 2.12/ 2.13 getting following errors

```$ java -jar gerrit-2.11.war init --batch -d git
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.inject.internal.cglib.core.$ReflectUtils$2 (file:/localuser/gerrit/.gerritcodereview/tmp/gerrit_5805538292667440356_app/guice-4.0-beta5.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)
WARNING: Please consider reporting this to the maintainers of com.google.inject.internal.cglib.core.$ReflectUtils$2
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Upgrading schema to 99 ...
Upgrading schema to 100 ...
Upgrading schema to 101 ...
Upgrading schema to 102 ...
Upgrading schema to 103 ...
Upgrading schema to 104 ...
Upgrading schema to 105 ...
Upgrading schema to 106 ...
Upgrading schema to 107 ...
Migrating data to schema 99 ...
Migrating data to schema 100 ...
Migrating data to schema 101 ...
Migrating data to schema 102 ...
Exception in thread "main" com.google.gwtorm.server.OrmException: Cannot upgrade schema
    at com.google.gerrit.server.schema.SchemaUpdater.update(SchemaUpdater.java:110)
    at com.google.gerrit.pgm.init.BaseInit$SiteRun.upgradeSchema(BaseInit.java:339)
    at com.google.gerrit.pgm.init.BaseInit.run(BaseInit.java:120)
    at com.google.gerrit.pgm.util.AbstractProgram.main(AbstractProgram.java:64)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at com.google.gerrit.launcher.GerritLauncher.invokeProgram(GerritLauncher.java:166)
    at com.google.gerrit.launcher.GerritLauncher.mainImpl(GerritLauncher.java:93)
    at com.google.gerrit.launcher.GerritLauncher.main(GerritLauncher.java:50)
    at Main.main(Main.java:25)
Caused by: org.postgresql.util.PSQLException: ERROR: column i.indproc does not exist
  Hint: Perhaps you meant to reference the column "i.indpred".
  Position: 610
    at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2103)
    at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1836)
    at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257)
    at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:512)
    at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:374)
    at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:254)
    at org.postgresql.jdbc2.AbstractJdbc2DatabaseMetaData.getIndexInfo(AbstractJdbc2DatabaseMetaData.java:4127)
    at com.google.gwtorm.schema.sql.SqlDialect.listIndexes(SqlDialect.java:279)
    at com.google.gerrit.server.schema.Schema_102.migrateData(Schema_102.java:49)
    at com.google.gerrit.server.schema.SchemaVersion.migrateData(SchemaVersion.java:140)
    at com.google.gerrit.server.schema.SchemaVersion.upgradeFrom(SchemaVersion.java:79)
    at com.google.gerrit.server.schema.SchemaVersion.check(SchemaVersion.java:70)
    at com.google.gerrit.server.schema.SchemaUpdater.update(SchemaUpdater.java:108)
    ... 11 more```

Upgrade from 2.10 -> 2.14 -> 2.15 -> 2.16 is successful without any errors, after I reindex witn 2.16 (below error on reindex)

Reindexing changes: projects: 43% (123/285), 35% (27962/77738) (/)[2022-07-04 04:44:38,618] [Index-Batch-30] ERROR com.google.gerrit.index.Schema : error getting field added of ChangeData{Change{80 (Ie704c502f59df89899105cd2d2dec8c18e50ab03), dest=woscloud,refs/heads/master, status=M}}
com.google.gwtorm.server.OrmException: org.eclipse.jgit.errors.RepositoryNotFoundException: repository not found: Cannot open repository woscloud

2.16 startup error below and I had to add a java option to overcome this (javaOptions = "--add-opens=jdk.management/com.sun.management.internal=ALL-UNNAMED") and gerrit server start's ok now


java.lang.reflect.InaccessibleObjectException: Unable to make public long com.sun.management.internal.OperatingSystemImpl.getProcessCpuTime() accessible: module jdk.management does not "opens com.sun.management.internal" to unnamed module @50b5ac82

seeing this error message 

"Error 500 (Server Error): Internal server error" "Endpoint: /config/server/top-menus" repos are not showing up


Matthias Sohn

unread,
Jul 4, 2022, 4:25:27 AM7/4/22
to Pharthiphan Asokan, Repo and Gerrit Discussion
On Mon, Jul 4, 2022 at 7:44 AM Pharthiphan Asokan <phart...@gmail.com> wrote:
Hi,

I am trying to migrate the Gerrit server from one location to another (to a different server)

Gerrit server version is 2.9.1 (Eventually I want it to be upgraded to 3.6.1)

Steps I followed,

dump and restore PostgreSQL DB from old server to new
java -jar gerrit-2.9.1.war init -d git # new_server
java -jar gerrit-2.9.1.war reindex -d git #new_server

You don't need to reindex on every intermediate version, it should be sufficient to reindex when you reached the target version.
 
I see the following error on reindexing, is it safe to ignore or can I do something to fix this

Reindexing changes: projects: 63% (106/168), 14% (11148/77744) (\)[2022-07-04 04:21:11,681] ERROR com.google.gerrit.server.index.ChangeBatchIndexer : repository not found: /localuser/gerrit/git/git/woslcoud

I don't know if you expect this repository to be missing 

java -jar gerrit-2.10.war init --batch -d git # successful and I am able to access gerrit

Upgrade from 2.10 to 2.11/ 2.12/ 2.13 getting following errors

```$ java -jar gerrit-2.11.war init --batch -d git

Always use the latest service release of each minor version you are updating to in order to not miss any fixes.
So use 2.11.12 instead of 2.11 etc.
 
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.inject.internal.cglib.core.$ReflectUtils$2 (file:/localuser/gerrit/.gerritcodereview/tmp/gerrit_5805538292667440356_app/guice-4.0-beta5.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)
WARNING: Please consider reporting this to the maintainers of com.google.inject.internal.cglib.core.$ReflectUtils$2
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release

Which Java version do you use to run this ?
AFAIK Java 11 is supported since 3.2.0 (https://www.gerritcodereview.com/3.2.html)
    at com.google.gerrit.server.schema.SchemaVersion.migrateData(SchemaVersion.java:140)
    at com.google.gerrit.server.schema.SchemaVersion.upgradeFrom(SchemaVersion.java:79)
    at com.google.gerrit.server.schema.SchemaVersion.check(SchemaVersion.java:70)
    at com.google.gerrit.server.schema.SchemaUpdater.update(SchemaUpdater.java:108)
    ... 11 more```

Upgrade from 2.10 -> 2.14 -> 2.15 -> 2.16 is successful without any errors, after I reindex witn 2.16 (below error on reindex)

Reindexing changes: projects: 43% (123/285), 35% (27962/77738) (/)[2022-07-04 04:44:38,618] [Index-Batch-30] ERROR com.google.gerrit.index.Schema : error getting field added of ChangeData{Change{80 (Ie704c502f59df89899105cd2d2dec8c18e50ab03), dest=woscloud,refs/heads/master, status=M}}
com.google.gwtorm.server.OrmException: org.eclipse.jgit.errors.RepositoryNotFoundException: repository not found: Cannot open repository woscloud

2.16 startup error below and I had to add a java option to overcome this (javaOptions = "--add-opens=jdk.management/com.sun.management.internal=ALL-UNNAMED") and gerrit server start's ok now


java.lang.reflect.InaccessibleObjectException: Unable to make public long com.sun.management.internal.OperatingSystemImpl.getProcessCpuTime() accessible: module jdk.management does not "opens com.sun.management.internal" to unnamed module @50b5ac82

seeing this error message 

"Error 500 (Server Error): Internal server error" "Endpoint: /config/server/top-menus" repos are not showing up


--
--
To unsubscribe, email repo-discuss...@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en

---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/repo-discuss/4eb7e62d-16e1-4240-bcb0-0143e75227d3n%40googlegroups.com.

Pharthiphan Asokan

unread,
Jul 5, 2022, 8:59:21 AM7/5/22
to Matthias Sohn, Repo and Gerrit Discussion
I see the following error on reindexing, is it safe to ignore or can I do something to fix this

Reindexing changes: projects: 63% (106/168), 14% (11148/77744) (\)[2022-07-04 04:21:11,681] ERROR com.google.gerrit.server.index.ChangeBatchIndexer : repository not found: /localuser/gerrit/git/git/woslcoud

I don't know if you expect this repository to be missing 


Yes, we don't care about this repo, can we git rid of it and have a clean re-index. this is causing issue on notedb migration




Nasser Grainawi

unread,
Jul 6, 2022, 5:41:34 PM7/6/22
to Pharthiphan Asokan, Repo and Gerrit Discussion
On Sun, Jul 3, 2022 at 11:44 PM Pharthiphan Asokan <phart...@gmail.com> wrote:
Hi,

I am trying to migrate the Gerrit server from one location to another (to a different server)

Gerrit server version is 2.9.1 (Eventually I want it to be upgraded to 3.6.1)

Steps I followed,

You might want to take a look at https://gerrit.googlesource.com/summit/2021/+/master/sessions/2.7-to-3.4-upgrade-improvements.md and see if anything we mention there would be helpful/applicable to you. We're starting with 2.7, not 2.9, and ending at 3.5, but the rest should be very similar.
 

Pharthiphan Asokan

unread,
Jul 7, 2022, 2:24:05 PM7/7/22
to Nasser Grainawi, Repo and Gerrit Discussion
Thanks, Nasser, I think the issue I am facing is similar to these but I don't find a solution. 

Reindexing changes: projects: 63% (106/168), 14% (11148/77744) (\)[2022-07-04 04:21:11,681] ERROR com.google.gerrit.server.index.ChangeBatchIndexer : repository not found: /localuser/gerrit/git/git/woslcoud 

I just want to get rid of the repo being known to Gerrit

https://groups.google.com/g/repo-discuss/c/5U9foDktspA

Nasser Grainawi

unread,
Jul 7, 2022, 8:55:59 PM7/7/22
to Pharthiphan Asokan, Repo and Gerrit Discussion
On Thu, Jul 7, 2022 at 12:24 PM Pharthiphan Asokan <phart...@gmail.com> wrote:
Thanks, Nasser, I think the issue I am facing is similar to these but I don't find a solution. 

Please try to reply inline instead of top posting.
 

Reindexing changes: projects: 63% (106/168), 14% (11148/77744) (\)[2022-07-04 04:21:11,681] ERROR com.google.gerrit.server.index.ChangeBatchIndexer : repository not found: /localuser/gerrit/git/git/woslcoud 

I just want to get rid of the repo being known to Gerrit

You can remove the changes for that repo from the Gerrit database prior to upgrading. Be sure to test that with a copy of your database and not on your production server as it's a destructive action. It will be a SQL command like "DELETE FROM changes WHERE dest_project_name = 'woslcoud'".
 
Reply all
Reply to author
Forward
0 new messages