Gerrit fails to start after upgrading from v3.8.3 --> v3.9.1
267 views
Skip to first unread message
Clinton Wong
Jan 31, 2024, 8:48:59 PM1/31/24
to Repo and Gerrit Discussion
I attempted to start my gerrit server after running
1. java init
2. Verify the versions of the plugins
3. java reindex
I am not sure what is causing the failure. Does anyone have any tips?
The gerrit service failed to start, and I found the following in the error_log
[2024-01-31T18:37:25.493-06:00] [main] INFO com.google.gerrit.server.git.WorkQueue : Adding metrics for 'SshCommandStart' queue
[2024-01-31T18:37:25.761-06:00] [main] INFO com.google.gerrit.server.git.WorkQueue : Adding metrics for 'SSH-Stream-Worker' queue
[2024-01-31T18:37:25.763-06:00] [main] INFO com.google.gerrit.server.git.WorkQueue : Adding metrics for 'SSH-Interactive-Worker' queue
[2024-01-31T18:37:25.763-06:00] [main] INFO com.google.gerrit.server.git.WorkQueue : Adding metrics for 'SSH-Batch-Worker' queue
[2024-01-31T18:37:25.764-06:00] [main] INFO com.google.gerrit.server.git.WorkQueue : Adding metrics for 'MigrateExternalIdCase' queue
[2024-01-31T18:37:25.773-06:00] [main] WARN com.google.gerrit.server.config.GitwebCgiConfig : gitweb not installed (no /usr/lib/cgi-bin/gitweb.cgi found)
[2024-01-31T18:37:25.975-06:00] [main] INFO org.eclipse.jetty.util.log : Logging initialized @5242ms to org.eclipse.jetty.util.log.Slf4jLog
[2024-01-31T18:37:26.068-06:00] [main] INFO com.google.gerrit.server.git.SystemReaderInstaller : Set JGit's SystemReader to read system config from /sarc-app/gerrit/etc/jgit.config
[2024-01-31T18:37:26.071-06:00] [main] INFO com.google.gerrit.server.git.LocalDiskRepositoryManager : Defaulting core.streamFileThreshold to 2047m
[2024-01-31T18:37:26.305-06:00] [main] ERROR com.google.gerrit.pgm.Daemon : Unable to start daemon
com.google.inject.ProvisionException: Unable to provision, see the following errors:
1) [Guice/ErrorInjectingConstructor]: IndexFormatTooOldException: Format version is not supported (resource BufferedChecksumIndexInput(MMapIndexInput(path="/sarc-app/gerrit-3.9.1/index/projects_0004/segments_ju"))): This index was initially created with Lucene 6.x while the current version is 8.11.2 and Lucene only supports reading the current and previous major versions.. This version of Lucene only supports indexes created with release 7.0 and later.
at LuceneProjectIndex.<init>(LuceneProjectIndex.java:92)
while locating ProjectIndex annotated with @UniqueAnnotations$Internal(7)
Learn more:
https://github.com/google/guice/wiki/ERROR_INJECTING_CONSTRUCTOR
1 error
======================
Full classname legend:
======================
IndexFormatTooOldException: "org.apache.lucene.index.IndexFormatTooOldException"
LuceneProjectIndex: "com.google.gerrit.lucene.LuceneProjectIndex"
ProjectIndex: "com.google.gerrit.index.project.ProjectIndex"
UniqueAnnotations$Internal: "com.google.inject.internal.UniqueAnnotations$Internal"
========================
End of classname legend:
========================
at com.google.inject.internal.InternalProvisionException.toProvisionException(InternalProvisionException.java:251)
at com.google.inject.internal.InjectorImpl$1.get(InjectorImpl.java:1151)
at com.google.inject.assistedinject.FactoryProvider2.invoke(FactoryProvider2.java:907)
at jdk.proxy2/jdk.proxy2.$Proxy39.create(Unknown Source)
at com.google.gerrit.server.index.VersionManager.initIndex(VersionManager.java:176)
at com.google.gerrit.server.index.VersionManager.start(VersionManager.java:95)
at com.google.gerrit.lifecycle.LifecycleManager.start(LifecycleManager.java:95)
at com.google.gerrit.pgm.Daemon.start(Daemon.java:405)
at com.google.gerrit.pgm.Daemon.run(Daemon.java:298)
at com.google.gerrit.pgm.util.AbstractProgram.main(AbstractProgram.java:62)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at com.google.gerrit.launcher.GerritLauncher.invokeProgram(GerritLauncher.java:252)
at com.google.gerrit.launcher.GerritLauncher.mainImpl(GerritLauncher.java:148)
at com.google.gerrit.launcher.GerritLauncher.main(GerritLauncher.java:93)
at Main.main(Main.java:30)
Caused by: org.apache.lucene.index.IndexFormatTooOldException: Format version is not supported (resource BufferedChecksumIndexInput(MMapIndexInput(path="/sarc-app/gerrit-3.9.1/index/projects_0004/segments_ju"))): This index was initially created with Lucene 6.x while the current version is 8.11.2 and Lucene only supports reading the current and previous major versions.. This version of Lucene only supports indexes created with release 7.0 and later.
at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:322)
at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:291)
at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1037)
at com.google.gerrit.lucene.AutoCommitWriter.<init>(AutoCommitWriter.java:35)
at com.google.gerrit.lucene.AutoCommitWriter.<init>(AutoCommitWriter.java:31)
at com.google.gerrit.lucene.AbstractLuceneIndex.<init>(AbstractLuceneIndex.java:143)
at com.google.gerrit.lucene.LuceneProjectIndex.<init>(LuceneProjectIndex.java:92)
at com.google.gerrit.lucene.LuceneProjectIndex$$FastClassByGuice$$2f0699a1.GUICE$TRAMPOLINE(<generated>)
at com.google.gerrit.lucene.LuceneProjectIndex$$FastClassByGuice$$2f0699a1.apply(<generated>)
at com.google.inject.internal.DefaultConstructionProxyFactory$FastClassProxy.newInstance(DefaultConstructionProxyFactory.java:82)
at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:114)
at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:91)
at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:300)
at com.google.inject.internal.InjectorImpl$1.get(InjectorImpl.java:1148)
... 16 more
Suppressed: org.apache.lucene.index.CorruptIndexException: checksum passed (9f7d00e3). possibly transient resource issue, or a Lucene or JVM bug (resource=BufferedChecksumIndexInput(MMapIndexInput(path="/sarc-app/gerrit-3.9.1/index/projects_0004/segments_ju")))
at org.apache.lucene.codecs.CodecUtil.checkFooter(CodecUtil.java:466)
at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:434)
... 29 more
[2024-01-31T18:37:26.340-06:00] [ForkJoinPool-1-worker-1] WARN org.eclipse.jgit.internal.util.ShutdownHook : Cleanup org.eclipse.jgit.util.FS$FileStoreAttributes$$Lambda$71/0x00000008010ba910@445b295b during JVM shutdown
[2024-01-31T18:37:25.761-06:00] [main] INFO com.google.gerrit.server.git.WorkQueue : Adding metrics for 'SSH-Stream-Worker' queue
[2024-01-31T18:37:25.763-06:00] [main] INFO com.google.gerrit.server.git.WorkQueue : Adding metrics for 'SSH-Interactive-Worker' queue
[2024-01-31T18:37:25.763-06:00] [main] INFO com.google.gerrit.server.git.WorkQueue : Adding metrics for 'SSH-Batch-Worker' queue
[2024-01-31T18:37:25.764-06:00] [main] INFO com.google.gerrit.server.git.WorkQueue : Adding metrics for 'MigrateExternalIdCase' queue
[2024-01-31T18:37:25.773-06:00] [main] WARN com.google.gerrit.server.config.GitwebCgiConfig : gitweb not installed (no /usr/lib/cgi-bin/gitweb.cgi found)
[2024-01-31T18:37:25.975-06:00] [main] INFO org.eclipse.jetty.util.log : Logging initialized @5242ms to org.eclipse.jetty.util.log.Slf4jLog
[2024-01-31T18:37:26.068-06:00] [main] INFO com.google.gerrit.server.git.SystemReaderInstaller : Set JGit's SystemReader to read system config from /sarc-app/gerrit/etc/jgit.config
[2024-01-31T18:37:26.071-06:00] [main] INFO com.google.gerrit.server.git.LocalDiskRepositoryManager : Defaulting core.streamFileThreshold to 2047m
[2024-01-31T18:37:26.305-06:00] [main] ERROR com.google.gerrit.pgm.Daemon : Unable to start daemon
com.google.inject.ProvisionException: Unable to provision, see the following errors:
1) [Guice/ErrorInjectingConstructor]: IndexFormatTooOldException: Format version is not supported (resource BufferedChecksumIndexInput(MMapIndexInput(path="/sarc-app/gerrit-3.9.1/index/projects_0004/segments_ju"))): This index was initially created with Lucene 6.x while the current version is 8.11.2 and Lucene only supports reading the current and previous major versions.. This version of Lucene only supports indexes created with release 7.0 and later.
at LuceneProjectIndex.<init>(LuceneProjectIndex.java:92)
while locating ProjectIndex annotated with @UniqueAnnotations$Internal(7)
Learn more:
https://github.com/google/guice/wiki/ERROR_INJECTING_CONSTRUCTOR
1 error
======================
Full classname legend:
======================
IndexFormatTooOldException: "org.apache.lucene.index.IndexFormatTooOldException"
LuceneProjectIndex: "com.google.gerrit.lucene.LuceneProjectIndex"
ProjectIndex: "com.google.gerrit.index.project.ProjectIndex"
UniqueAnnotations$Internal: "com.google.inject.internal.UniqueAnnotations$Internal"
========================
End of classname legend:
========================
at com.google.inject.internal.InternalProvisionException.toProvisionException(InternalProvisionException.java:251)
at com.google.inject.internal.InjectorImpl$1.get(InjectorImpl.java:1151)
at com.google.inject.assistedinject.FactoryProvider2.invoke(FactoryProvider2.java:907)
at jdk.proxy2/jdk.proxy2.$Proxy39.create(Unknown Source)
at com.google.gerrit.server.index.VersionManager.initIndex(VersionManager.java:176)
at com.google.gerrit.server.index.VersionManager.start(VersionManager.java:95)
at com.google.gerrit.lifecycle.LifecycleManager.start(LifecycleManager.java:95)
at com.google.gerrit.pgm.Daemon.start(Daemon.java:405)
at com.google.gerrit.pgm.Daemon.run(Daemon.java:298)
at com.google.gerrit.pgm.util.AbstractProgram.main(AbstractProgram.java:62)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at com.google.gerrit.launcher.GerritLauncher.invokeProgram(GerritLauncher.java:252)
at com.google.gerrit.launcher.GerritLauncher.mainImpl(GerritLauncher.java:148)
at com.google.gerrit.launcher.GerritLauncher.main(GerritLauncher.java:93)
at Main.main(Main.java:30)
Caused by: org.apache.lucene.index.IndexFormatTooOldException: Format version is not supported (resource BufferedChecksumIndexInput(MMapIndexInput(path="/sarc-app/gerrit-3.9.1/index/projects_0004/segments_ju"))): This index was initially created with Lucene 6.x while the current version is 8.11.2 and Lucene only supports reading the current and previous major versions.. This version of Lucene only supports indexes created with release 7.0 and later.
at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:322)
at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:291)
at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1037)
at com.google.gerrit.lucene.AutoCommitWriter.<init>(AutoCommitWriter.java:35)
at com.google.gerrit.lucene.AutoCommitWriter.<init>(AutoCommitWriter.java:31)
at com.google.gerrit.lucene.AbstractLuceneIndex.<init>(AbstractLuceneIndex.java:143)
at com.google.gerrit.lucene.LuceneProjectIndex.<init>(LuceneProjectIndex.java:92)
at com.google.gerrit.lucene.LuceneProjectIndex$$FastClassByGuice$$2f0699a1.GUICE$TRAMPOLINE(<generated>)
at com.google.gerrit.lucene.LuceneProjectIndex$$FastClassByGuice$$2f0699a1.apply(<generated>)
at com.google.inject.internal.DefaultConstructionProxyFactory$FastClassProxy.newInstance(DefaultConstructionProxyFactory.java:82)
at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:114)
at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:91)
at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:300)
at com.google.inject.internal.InjectorImpl$1.get(InjectorImpl.java:1148)
... 16 more
Suppressed: org.apache.lucene.index.CorruptIndexException: checksum passed (9f7d00e3). possibly transient resource issue, or a Lucene or JVM bug (resource=BufferedChecksumIndexInput(MMapIndexInput(path="/sarc-app/gerrit-3.9.1/index/projects_0004/segments_ju")))
at org.apache.lucene.codecs.CodecUtil.checkFooter(CodecUtil.java:466)
at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:434)
... 29 more
[2024-01-31T18:37:26.340-06:00] [ForkJoinPool-1-worker-1] WARN org.eclipse.jgit.internal.util.ShutdownHook : Cleanup org.eclipse.jgit.util.FS$FileStoreAttributes$$Lambda$71/0x00000008010ba910@445b295b during JVM shutdown
Clark Boylan
Jan 31, 2024, 9:23:01 PM1/31/24
to Clinton Wong, Repo and Gerrit Discussion
On Wed, Jan 31, 2024 at 5:49 PM Clinton Wong <clinto...@samsung.com> wrote:
>
> I attempted to start my gerrit server after running
> 1. java init
> 2. Verify the versions of the plugins
> 3. java reindex
>
> I am not sure what is causing the failure. Does anyone have any tips?
>
> The gerrit service failed to start, and I found the following in the error_log
>
> [2024-01-31T18:37:25.493-06:00] [main] INFO com.google.gerrit.server.git.WorkQueue : Adding metrics for 'SshCommandStart' queue
> [2024-01-31T18:37:25.761-06:00] [main] INFO com.google.gerrit.server.git.WorkQueue : Adding metrics for 'SSH-Stream-Worker' queue
> [2024-01-31T18:37:25.763-06:00] [main] INFO com.google.gerrit.server.git.WorkQueue : Adding metrics for 'SSH-Interactive-Worker' queue
> [2024-01-31T18:37:25.763-06:00] [main] INFO com.google.gerrit.server.git.WorkQueue : Adding metrics for 'SSH-Batch-Worker' queue
> [2024-01-31T18:37:25.764-06:00] [main] INFO com.google.gerrit.server.git.WorkQueue : Adding metrics for 'MigrateExternalIdCase' queue
> [2024-01-31T18:37:25.773-06:00] [main] WARN com.google.gerrit.server.config.GitwebCgiConfig : gitweb not installed (no /usr/lib/cgi-bin/gitweb.cgi found)
> [2024-01-31T18:37:25.975-06:00] [main] INFO org.eclipse.jetty.util.log : Logging initialized @5242ms to org.eclipse.jetty.util.log.Slf4jLog
> [2024-01-31T18:37:26.068-06:00] [main] INFO com.google.gerrit.server.git.SystemReaderInstaller : Set JGit's SystemReader to read system config from /sarc-app/gerrit/etc/jgit.config
> [2024-01-31T18:37:26.071-06:00] [main] INFO com.google.gerrit.server.git.LocalDiskRepositoryManager : Defaulting core.streamFileThreshold to 2047m
> [2024-01-31T18:37:26.305-06:00] [main] ERROR com.google.gerrit.pgm.Daemon : Unable to start daemon
> com.google.inject.ProvisionException: Unable to provision, see the following errors:
>
> 1) [Guice/ErrorInjectingConstructor]: IndexFormatTooOldException: Format version is not supported (resource BufferedChecksumIndexInput(MMapIndexInput(path="/sarc-app/gerrit-3.9.1/index/projects_0004/segments_ju"))): This index was initially created with Lucene 6.x while the current version is 8.11.2 and Lucene only supports reading the current and previous major versions.. This version of Lucene only supports indexes created with release 7.0 and later.
Are you sure you are upgrading to version 3.9.1 and not 3.9.0? The
>
> I attempted to start my gerrit server after running
> 1. java init
> 2. Verify the versions of the plugins
> 3. java reindex
>
> I am not sure what is causing the failure. Does anyone have any tips?
>
> The gerrit service failed to start, and I found the following in the error_log
>
> [2024-01-31T18:37:25.493-06:00] [main] INFO com.google.gerrit.server.git.WorkQueue : Adding metrics for 'SshCommandStart' queue
> [2024-01-31T18:37:25.761-06:00] [main] INFO com.google.gerrit.server.git.WorkQueue : Adding metrics for 'SSH-Stream-Worker' queue
> [2024-01-31T18:37:25.763-06:00] [main] INFO com.google.gerrit.server.git.WorkQueue : Adding metrics for 'SSH-Interactive-Worker' queue
> [2024-01-31T18:37:25.763-06:00] [main] INFO com.google.gerrit.server.git.WorkQueue : Adding metrics for 'SSH-Batch-Worker' queue
> [2024-01-31T18:37:25.764-06:00] [main] INFO com.google.gerrit.server.git.WorkQueue : Adding metrics for 'MigrateExternalIdCase' queue
> [2024-01-31T18:37:25.773-06:00] [main] WARN com.google.gerrit.server.config.GitwebCgiConfig : gitweb not installed (no /usr/lib/cgi-bin/gitweb.cgi found)
> [2024-01-31T18:37:25.975-06:00] [main] INFO org.eclipse.jetty.util.log : Logging initialized @5242ms to org.eclipse.jetty.util.log.Slf4jLog
> [2024-01-31T18:37:26.068-06:00] [main] INFO com.google.gerrit.server.git.SystemReaderInstaller : Set JGit's SystemReader to read system config from /sarc-app/gerrit/etc/jgit.config
> [2024-01-31T18:37:26.071-06:00] [main] INFO com.google.gerrit.server.git.LocalDiskRepositoryManager : Defaulting core.streamFileThreshold to 2047m
> [2024-01-31T18:37:26.305-06:00] [main] ERROR com.google.gerrit.pgm.Daemon : Unable to start daemon
> com.google.inject.ProvisionException: Unable to provision, see the following errors:
>
> 1) [Guice/ErrorInjectingConstructor]: IndexFormatTooOldException: Format version is not supported (resource BufferedChecksumIndexInput(MMapIndexInput(path="/sarc-app/gerrit-3.9.1/index/projects_0004/segments_ju"))): This index was initially created with Lucene 6.x while the current version is 8.11.2 and Lucene only supports reading the current and previous major versions.. This version of Lucene only supports indexes created with release 7.0 and later.
3.9.0 release was not able to perform an online reindex of the lucene
indexes because it jumped too many versions ahead to have backward
compatibility with the lucene version in 3.8.x. This was corrected in
the 3.9.1 release.
Clinton Wong
Jan 31, 2024, 10:10:35 PM1/31/24
to Repo and Gerrit Discussion
I checked the war file, and it show version 3.9.1
Ran command:
zipgrep "^Version v" gerrit.war |grep -i install.html
Output:
Output:
Documentation/cmd-plugin-install.html:Version v3.9.1<br>
Documentation/install.html:Version v3.9.1<br>
Documentation/install.html:Version v3.9.1<br>
Sven Selberg
Feb 1, 2024, 4:16:59 AM2/1/24
to Repo and Gerrit Discussion
On Thursday, February 1, 2024 at 4:10:35 AM UTC+1 Clinton Wong wrote:
I checked the war file, and it show version 3.9.1Ran command:zipgrep "^Version v" gerrit.war |grep -i install.html
Output:Documentation/cmd-plugin-install.html:Version v3.9.1<br>
Documentation/install.html:Version v3.9.1<br>
Please stop top-posting, it is an inconvenience for those who are trying to help you.
F.Y.I. A more stable way of getting the Gerrit version from a war file is to use the `version` command:
https://gerrit-review.googlesource.com/Documentation/pgm-index.html
`java -jar bin/gerrit.war version`
`java -jar bin/gerrit.war version`
Clinton Wong
Feb 1, 2024, 2:43:19 PM2/1/24
to Repo and Gerrit Discussion
I am sorry about the top-posting. I was unaware of this convention. I hope that this post is better.
Thank you for point out a better way to determine the version of the war.
After looking at the error and the "Schema and index changes" in the 3.9 notes, I attempted to reindex the server.
I found error messages related to a set of repos that were mirrored by the admin before me. Since those repos were no longer in use, I mover them out of the git directory and ran the reindex again. The reindex completed without any "ERROR" messages.
My gerrit server now starts.
Reply all
Reply to author
Forward
0 new messages