Hi all,
We just upgraded from Gerrit 3.5.2 to 3.7.0. However, Gerrit no longer starts up. Any help would be much appreciated!
We're using Docker Gerrit, and ran first with the command init. However, upon starting up Gerrit, we see
gerrit_1 | Running Gerrit ...
gerrit_1 | Running Gerrit Code Review:
gerrit_1 | Nov 18, 2022 12:59:11 AM com.google.inject.assistedinject.FactoryProvider2 isValidForOptimizedAssistedInject
gerrit_1 | WARNING: AssistedInject factory com.google.gerrit.server.api.changes.ChangeApiImpl$Factory will be slow because class com.google.gerrit.server.api.changes.ChangeApiImpl has assisted Provider dependencies or injects the Injector. Stop injecting @Assisted Provider<T> (instead use @Assisted T) or Injector to speed things up. (It will be a ~6500% speed bump!) The exact offending deps are: [Key[type=com.google.inject.Injector, annotation=[none]]@com.google.gerrit.server.api.changes.ChangeApiImpl.<init>()[48]]
gerrit_1 | [2022-11-18 00:59:14,042] [main] INFO com.google.gerrit.server.cache.PersistentCacheBaseFactory : Enabling disk cache /var/gerrit/cache
gerrit_1 | [2022-11-18 00:59:14,349] [main] INFO com.google.gerrit.server.git.WorkQueue : Adding metrics for 'WorkQueue' queue
gerrit_1 | [2022-11-18 00:59:14,427] [main] INFO com.google.gerrit.server.git.WorkQueue : Adding metrics for 'Index-Interactive' queue
gerrit_1 | [2022-11-18 00:59:14,429] [main] INFO com.google.gerrit.server.git.WorkQueue : Adding metrics for 'Index-Batch' queue
gerrit_1 | [2022-11-18 00:59:14,494] [main] INFO com.google.gerrit.server.git.WorkQueue : Adding metrics for 'ReceiveCommits' queue
gerrit_1 | [2022-11-18 00:59:14,496] [main] INFO com.google.gerrit.server.git.WorkQueue : Adding metrics for 'SendEmail' queue
gerrit_1 | [2022-11-18 00:59:15,290] [main] INFO com.google.gerrit.server.rules.PrologEnvironment : reductionLimit: 100000, compileLimit: 1000000
gerrit_1 | [2022-11-18 00:59:15,297] [main] INFO com.google.gerrit.server.config.ScheduleConfig : No schedule configuration for "gc".
gerrit_1 | [2022-11-18 00:59:15,297] [main] INFO com.google.gerrit.server.config.ScheduleConfig : No schedule configuration for "changeCleanup".
gerrit_1 | Nov 18, 2022 12:59:15 AM com.google.inject.assistedinject.FactoryProvider2 isValidForOptimizedAssistedInject
gerrit_1 | WARNING: AssistedInject factory com.google.gerrit.sshd.DispatchCommand$Factory will be slow because class com.google.gerrit.sshd.DispatchCommand has assisted Provider dependencies or injects the Injector. Stop injecting @Assisted Provider<T> (instead use @Assisted T) or Injector to speed things up. (It will be a ~6500% speed bump!) The exact offending deps are: [Key[type=com.google.inject.Injector, annotation=[none]]@com.google.gerrit.sshd.BaseCommand.injector]
gerrit_1 | [2022-11-18T00:59:15.891Z] [main] INFO com.google.gerrit.server.git.WorkQueue : Adding metrics for 'SshCommandStart' queue
gerrit_1 | [2022-11-18T00:59:16.528Z] [main] INFO com.google.gerrit.server.git.WorkQueue : Adding metrics for 'SSH-Stream-Worker' queue
gerrit_1 | [2022-11-18T00:59:16.530Z] [main] INFO com.google.gerrit.server.git.WorkQueue : Adding metrics for 'SSH-Interactive-Worker' queue
gerrit_1 | [2022-11-18T00:59:16.531Z] [main] INFO com.google.gerrit.server.git.WorkQueue : Adding metrics for 'SSH-Batch-Worker' queue
gerrit_1 | [2022-11-18T00:59:16.532Z] [main] INFO com.google.gerrit.server.git.WorkQueue : Adding metrics for 'MigrateExternalIdCase' queue
gerrit_1 | [2022-11-18T00:59:16.548Z] [main] WARN com.google.gerrit.server.config.GitwebCgiConfig : gitweb not installed (no /usr/lib/cgi-bin/gitweb.cgi found)
gerrit_1 | [2022-11-18T00:59:17.530Z] [main] INFO org.eclipse.jetty.util.log : Logging initialized @8506ms to org.eclipse.jetty.util.log.Slf4jLog
gerrit_1 | [2022-11-18T00:59:17.698Z] [main] INFO com.google.gerrit.server.git.SystemReaderInstaller : Set JGit's SystemReader to read system config from /var/gerrit/etc/jgit.config
gerrit_1 | [2022-11-18T00:59:17.703Z] [main] INFO com.google.gerrit.server.git.LocalDiskRepositoryManager : Defaulting core.streamFileThreshold to 977m
gerrit_1 | [2022-11-18T00:59:20.789Z] [main] ERROR com.google.gerrit.pgm.Daemon : Unable to start daemon
gerrit_1 | com.google.inject.ProvisionException: Unable to provision, see the following errors:
gerrit_1 |
gerrit_1 | 1) No index versions for index 'changes' ready; run java -jar /var/gerrit/bin/gerrit.war reindex --index changes
gerrit_1 |
gerrit_1 | 1 error
gerrit_1 | at com.google.gerrit.server.index.VersionManager.initIndex(VersionManager.java:172)
gerrit_1 | at com.google.gerrit.server.index.VersionManager.start(VersionManager.java:95)
gerrit_1 | at com.google.gerrit.lifecycle.LifecycleManager.start(LifecycleManager.java:95)
gerrit_1 | at com.google.gerrit.pgm.Daemon.start(Daemon.java:386)
gerrit_1 | at com.google.gerrit.pgm.Daemon.run(Daemon.java:285)
gerrit_1 | at com.google.gerrit.pgm.util.AbstractProgram.main(AbstractProgram.java:61)
gerrit_1 | at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
gerrit_1 | at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
gerrit_1 | at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
gerrit_1 | at java.base/java.lang.reflect.Method.invoke(Method.java:566)
gerrit_1 | at com.google.gerrit.launcher.GerritLauncher.invokeProgram(GerritLauncher.java:224)
gerrit_1 | at com.google.gerrit.launcher.GerritLauncher.mainImpl(GerritLauncher.java:120)
gerrit_1 | at com.google.gerrit.launcher.GerritLauncher.main(GerritLauncher.java:65)
gerrit_1 | at Main.main(Main.java:29)
gerrit_gerrit_1 exited with code 1
===
We did try to reindex as well, and saw:
gerrit_1 | WARNING: An illegal reflective access operation has occurred
gerrit_1 | WARNING: Illegal reflective access by com.google.gerrit.common.IoUtil (file:/var/gerrit/.gerritcodereview/tmp/gerrit_17071708961186393414_app/com_google_gerrit_common_libserver.jar) to method java.net.URLClassLoader.addURL(java.net.URL)
gerrit_1 | WARNING: Please consider reporting this to the maintainers of com.google.gerrit.common.IoUtil
gerrit_1 | WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
gerrit_1 | WARNING: All illegal access operations will be denied in a future release
gerrit_1 | [2022-11-18 00:57:01,774] [main] INFO com.google.gerrit.server.git.SystemReaderInstaller : Set JGit's SystemReader to read system config from /var/gerrit/etc/jgit.config
gerrit_1 | [2022-11-18 00:57:01,802] [main] INFO com.google.gerrit.server.git.LocalDiskRepositoryManager : Defaulting core.streamFileThreshold to 977m
gerrit_1 | [2022-11-18 00:57:03,083] [main] INFO com.google.gerrit.server.cache.PersistentCacheBaseFactory : Enabling disk cache /var/gerrit/cache
gerrit_1 | [2022-11-18 00:57:03,156] [main] INFO com.google.gerrit.server.git.WorkQueue : Adding metrics for 'WorkQueue' queue
gerrit_1 | [2022-11-18 00:57:03,164] [main] INFO com.google.gerrit.server.git.WorkQueue : Adding metrics for 'Index-Interactive' queue
gerrit_1 | [2022-11-18 00:57:03,169] [main] INFO com.google.gerrit.server.git.WorkQueue : Adding metrics for 'Index-Batch' queue
gerrit_1 | [2022-11-18 00:57:03,339] [main] INFO com.google.gerrit.server.git.WorkQueue : Adding metrics for 'ReceiveCommits' queue
gerrit_1 | [2022-11-18 00:57:03,341] [main] INFO com.google.gerrit.server.git.WorkQueue : Adding metrics for 'SendEmail' queue
gerrit_1 | [2022-11-18 00:57:03,390] [main] INFO com.google.gerrit.server.rules.PrologEnvironment : reductionLimit: 100000, compileLimit: 1000000
gerrit_1 | [2022-11-18 00:57:04,123] [main] INFO com.google.gerrit.server.plugins.PluginLoader : Loading plugins from /var/gerrit/plugins
gerrit_1 | [2022-11-18 00:57:04,226] [main] INFO com.google.gerrit.server.plugins.PluginLoader : Loaded plugin avatars-gravatar, version 4d45f66597
gerrit_1 | [2022-11-18 00:57:04,248] [main] INFO com.google.gerrit.server.plugins.PluginLoader : Loaded plugin codemirror-editor, version v3.7.0
gerrit_1 | [2022-11-18 00:57:04,268] [main] INFO com.google.gerrit.server.plugins.PluginLoader : Loaded plugin commit-message-length-validator, version v3.7.0
gerrit_1 | [2022-11-18 00:57:04,288] [main] INFO com.google.gerrit.server.plugins.PluginLoader : Loaded plugin delete-project, version v3.7.0
gerrit_1 | [2022-11-18 00:57:04,306] [main] INFO com.google.gerrit.server.plugins.PluginLoader : Loaded plugin download-commands, version v3.7.0
gerrit_1 | [2022-11-18 00:57:04,347] [main] INFO com.google.gerrit.server.plugins.PluginLoader : Loaded plugin gerrit-oauth-provider, version 0f807f6
gerrit_1 | [2022-11-18 00:57:04,378] [main] INFO com.google.gerrit.server.plugins.PluginLoader : Loaded plugin gitiles, version v3.7.0
gerrit_1 | [2022-11-18 00:57:04,402] [main] INFO com.google.gerrit.server.plugins.PluginLoader : Loaded plugin hooks, version v3.7.0
gerrit_1 | [2022-11-18 00:57:04,426] [main] INFO com.google.gerrit.server.plugins.PluginLoader : Loaded plugin plugin-manager, version v3.7.0
gerrit_1 | [2022-11-18 00:57:04,450] [main] INFO com.google.gerrit.server.plugins.PluginLoader : Loaded plugin replication, version v3.7.0
gerrit_1 | [2022-11-18 00:57:04,473] [main] INFO com.google.gerrit.server.plugins.PluginLoader : Loaded plugin reviewnotes, version v3.7.0
gerrit_1 | [2022-11-18 00:57:04,496] [main] INFO com.google.gerrit.server.plugins.PluginLoader : Loaded plugin singleusergroup, version v3.7.0
gerrit_1 | [2022-11-18 00:57:04,520] [main] INFO com.google.gerrit.server.plugins.PluginLoader : Loaded plugin uploadvalidator, version v3.0.0-rc1-310-g7bf50a9ab1
gerrit_1 | [2022-11-18 00:57:04,544] [main] INFO com.google.gerrit.server.plugins.PluginLoader : Loaded plugin webhooks, version v3.7.0
gerrit_1 | [2022-11-18 00:57:04,561] [main] WARN com.google.gerrit.server.project.PeriodicProjectListCacheWarmer : project_list cache warmer is disabled
Reindexing changes: changes: 100% (11277/11277), project-slices: 100% (35/35), Slicing projects: 100% (38/38), done
gerrit_1 | Reindexed 11277 documents in changes index in 50.7s (222.6/s)
gerrit_1 | Index changes in version 79 is ready
gerrit_1 | Cache Statistics at the end of reindexing
gerrit_1 | Name |Entries | AvgGet |Hit Ratio|
gerrit_1 | | Mem Disk Space| |Mem Disk|
gerrit_1 | --------------------------------+---------------------+---------+---------+
gerrit_1 | groups_external | | | |
gerrit_1 | groups_bysubgroup | | | |
gerrit_1 | change_notes | 221 | 3.8ms | 0% |
gerrit_1 | groups_byuuid | | | |
gerrit_1 | default_preferences | | | |
gerrit_1 | prolog_rules | | | |
gerrit_1 | groups | | | |
gerrit_1 | projects | 26 | 266.3ms | 99% |
gerrit_1 | groups_byname | | | |
gerrit_1 | groups_external_persisted | | | |
gerrit_1 | project_list | 1 | 12.4ms | 0% |
gerrit_1 | groups_bymember | | | |
gerrit_1 | permission_sort | | | |
gerrit_1 | external_ids_map | | | |
gerrit_1 | D diff_summary | 7896 21773 13.05m| 2.4ms | 0% 100%|
gerrit_1 | D pure_revert | 100 160 18.01k| 25.2ms | 0% 100%|
gerrit_1 | D groups_byuuid_persisted | 27 11.29k| | |
gerrit_1 | D change_kind | 14201 1.72m| | |
gerrit_1 | D mergeability | 61437 9.28m| | |
gerrit_1 | D conflicts | 14834 2.00m| | |
gerrit_1 | D diff_intraline | 65504 89.14m| | |
gerrit_1 | D git_tags | 0.00k| | |
gerrit_1 | D gerrit_file_diff | 23860 41.70m| | |
gerrit_1 | D git_file_diff | 21175 38.67m| | |
gerrit_1 | D accounts | 341 164.88k| | |
gerrit_1 | D persisted_projects | 38 78.72k| | 100%|
gerrit_1 | D git_modified_files | 0.00k| | |
gerrit_1 | D modified_files | 0.00k| | |
gerrit_1 |
gerrit_1 | [2022-11-18 00:57:56,325] [main] INFO com.google.gerrit.server.plugins.PluginLoader : Unloading plugin replication, version v3.7.0
gerrit_1 | [2022-11-18 00:57:56,327] [main] INFO com.google.gerrit.server.plugins.PluginLoader : Unloading plugin codemirror-editor, version v3.7.0
gerrit_1 | [2022-11-18 00:57:56,327] [main] INFO com.google.gerrit.server.plugins.PluginLoader : Unloading plugin gitiles, version v3.7.0
gerrit_1 | [2022-11-18 00:57:56,327] [main] INFO com.google.gerrit.server.plugins.PluginLoader : Unloading plugin reviewnotes, version v3.7.0
gerrit_1 | [2022-11-18 00:57:56,328] [main] INFO com.google.gerrit.server.plugins.PluginLoader : Unloading plugin download-commands, version v3.7.0
gerrit_1 | [2022-11-18 00:57:56,328] [main] INFO com.google.gerrit.server.plugins.PluginLoader : Unloading plugin commit-message-length-validator, version v3.7.0
gerrit_1 | [2022-11-18 00:57:56,328] [main] INFO com.google.gerrit.server.plugins.PluginLoader : Unloading plugin webhooks, version v3.7.0
gerrit_1 | [2022-11-18 00:57:56,328] [main] INFO com.google.gerrit.server.plugins.PluginLoader : Unloading plugin avatars-gravatar, version 4d45f66597
gerrit_1 | [2022-11-18 00:57:56,329] [main] INFO com.google.gerrit.server.plugins.PluginLoader : Unloading plugin gerrit-oauth-provider, version 0f807f6
gerrit_1 | [2022-11-18 00:57:56,329] [main] INFO com.google.gerrit.server.plugins.PluginLoader : Unloading plugin delete-project, version v3.7.0
gerrit_1 | [2022-11-18 00:57:56,329] [main] INFO com.google.gerrit.server.plugins.PluginLoader : Unloading plugin plugin-manager, version v3.7.0
gerrit_1 | [2022-11-18 00:57:56,329] [main] INFO com.google.gerrit.server.plugins.PluginLoader : Unloading plugin singleusergroup, version v3.7.0
gerrit_1 | [2022-11-18 00:57:56,330] [main] INFO com.google.gerrit.server.plugins.PluginLoader : Unloading plugin hooks, version v3.7.0
gerrit_1 | [2022-11-18 00:57:56,330] [main] INFO com.google.gerrit.server.plugins.PluginLoader : Unloading plugin uploadvalidator, version v3.0.0-rc1-310-g7bf50a9ab1
gerrit_1 | [2022-11-18 00:57:56,557] [main] INFO com.google.gerrit.server.plugins.CleanupHandle : Cleaned plugin plugin_replication_221118_0057_12800627632096684444.jar
gerrit_1 | [2022-11-18 00:57:56,558] [main] INFO com.google.gerrit.server.plugins.CleanupHandle : Cleaned plugin plugin_codemirror-editor_221118_0057_572299070437956224.jar
gerrit_1 | [2022-11-18 00:57:56,558] [main] INFO com.google.gerrit.server.plugins.CleanupHandle : Cleaned plugin plugin_gitiles_221118_0057_5629862977742709338.jar
gerrit_1 | [2022-11-18 00:57:56,558] [main] INFO com.google.gerrit.server.plugins.CleanupHandle : Cleaned plugin plugin_reviewnotes_221118_0057_9055715645535632836.jar
gerrit_1 | [2022-11-18 00:57:56,558] [main] INFO com.google.gerrit.server.plugins.CleanupHandle : Cleaned plugin plugin_download-commands_221118_0057_422855622554322795.jar
gerrit_1 | [2022-11-18 00:57:56,559] [main] INFO com.google.gerrit.server.plugins.CleanupHandle : Cleaned plugin plugin_commit-message-length-validator_221118_0057_8135329960398565749.jar
gerrit_1 | [2022-11-18 00:57:56,559] [main] INFO com.google.gerrit.server.plugins.CleanupHandle : Cleaned plugin plugin_webhooks_221118_0057_8532048684970754343.jar
gerrit_1 | [2022-11-18 00:57:56,559] [main] INFO com.google.gerrit.server.plugins.CleanupHandle : Cleaned plugin plugin_avatars-gravatar_221118_0057_5890827554712529255.jar
gerrit_1 | [2022-11-18 00:57:56,559] [main] INFO com.google.gerrit.server.plugins.CleanupHandle : Cleaned plugin plugin_gerrit-oauth-provider_221118_0057_9476845543859991953.jar
gerrit_1 | [2022-11-18 00:57:56,560] [main] INFO com.google.gerrit.server.plugins.CleanupHandle : Cleaned plugin plugin_delete-project_221118_0057_7853976825348025118.jar
gerrit_1 | [2022-11-18 00:57:56,560] [main] INFO com.google.gerrit.server.plugins.CleanupHandle : Cleaned plugin plugin_plugin-manager_221118_0057_11551173459039953926.jar
gerrit_1 | [2022-11-18 00:57:56,561] [main] INFO com.google.gerrit.server.plugins.CleanupHandle : Cleaned plugin plugin_singleusergroup_221118_0057_4504760207368603079.jar
gerrit_1 | [2022-11-18 00:57:56,561] [main] INFO com.google.gerrit.server.plugins.CleanupHandle : Cleaned plugin plugin_hooks_221118_0057_6609462589917232378.jar
gerrit_1 | [2022-11-18 00:57:56,561] [main] INFO com.google.gerrit.server.plugins.CleanupHandle : Cleaned plugin plugin_uploadvalidator_221118_0057_16547060919853405600.jar
gerrit_gerrit_1 exited with code 0