Gerrit 3.4.0 Replication stopped working after upgrade from 3.3.4

Peter Bruin

unread,

May 31, 2021, 11:49:37 PM5/31/21

to Repo and Gerrit Discussion

I just noticed that my replication has stopped working since upgrading to 3.4.0 last week.

I run Gerrit from a the ubuntu 20 docker container.

The plugin is of the correct version and I see repeated errors that files are not found. When I check in the container these files are sometimes there and gerrit has read/write access to the folders.

Any ideas? Should I delete the while replication folder in the data dir?

[2021-05-27T15:04:54.554+08:00] [main] INFO com.google.gerrit.server.plugins.PluginLoader : Loaded plugin replication, version v3.4.0

[2021-05-27T18:23:31.303+08:00] [ReplicateTo-gitlab-projecta-1[java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@768c6096[Not completed, task = java.util.concurrent.Executors$RunnableAdapter@29945dfe[Wrapped task = com.google.gerrit.server.logging.LoggingContextAwareRunnable@709f26b2]]]] ERROR com.googlesource.gerrit.plugins.replication.ReplicationTasksStorage : Error while renaming task 41787746804b3a734fe4d559ce1d42196a5e5e94 [CONTEXT PLUGIN="replication" pushOneId="49db34a8" ]

java.nio.file.NoSuchFileException: /var/gerrit/data/replication/ref-updates/running/41787746804b3a734fe4d559ce1d42196a5e5e94 -> /var/gerrit/data/replication/ref-updates/waiting/41787746804b3a734fe4d559ce1d42196a5e5e94

at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)

at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)

at java.base/sun.nio.fs.UnixCopyFile.move(UnixCopyFile.java:417)

at java.base/sun.nio.fs.UnixFileSystemProvider.move(UnixFileSystemProvider.java:267)

at java.base/java.nio.file.Files.move(Files.java:1422)

at com.googlesource.gerrit.plugins.replication.ReplicationTasksStorage$Task.rename(ReplicationTasksStorage.java:269)

at com.googlesource.gerrit.plugins.replication.ReplicationTasksStorage$Task.reset(ReplicationTasksStorage.java:246)

at com.googlesource.gerrit.plugins.replication.ReplicationTasksStorage.reset(ReplicationTasksStorage.java:150)

at com.googlesource.gerrit.plugins.replication.Destination.reschedule(Destination.java:568)

at com.googlesource.gerrit.plugins.replication.PushOne.doRunPushOperation(PushOne.java:482)

at com.googlesource.gerrit.plugins.replication.PushOne.runPushOperation(PushOne.java:389)

at com.googlesource.gerrit.plugins.replication.PushOne.lambda$run$2(PushOne.java:375)

at com.google.gerrit.server.util.RequestScopePropagator.lambda$cleanup$1(RequestScopePropagator.java:182)

at com.google.gerrit.server.util.RequestScopePropagator.lambda$context$0(RequestScopePropagator.java:170)

at com.google.gerrit.server.git.PerThreadRequestScope$Propagator.lambda$scope$0(PerThreadRequestScope.java:70)

at com.googlesource.gerrit.plugins.replication.PushOne.run(PushOne.java:378)

at com.google.gerrit.server.logging.LoggingContextAwareRunnable.run(LoggingContextAwareRunnable.java:113)

at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)

at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)

at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)

at com.google.gerrit.server.git.WorkQueue$Task.run(WorkQueue.java:612)

at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)

at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)

at java.base/java.lang.Thread.run(Thread.java:829)

Peter Bruin

unread,

Jun 1, 2021, 9:31:09 AM6/1/21

to Repo and Gerrit Discussion

I delete the data/replication folder and it still does not work.

Straight after upgrading to 3.4.0 I see this in the replication logs:

Any help would be appreciated.

[2021-05-27 04:39:14,928] Cannot replicate to g...@code.domain.net:projectsa.git [CONTEXT PLUGIN="replication" pushOneId="d174bfb2" ]

org.eclipse.jgit.errors.TransportException: g...@code.domain.net:projecta.git: Session is being closed

at org.eclipse.jgit.transport.sshd.SshdSessionFactory.getSession(SshdSessionFactory.java:250)

at org.eclipse.jgit.transport.sshd.SshdSessionFactory.getSession(SshdSessionFactory.java:74)

at org.eclipse.jgit.transport.SshTransport.getSession(SshTransport.java:107)

at org.eclipse.jgit.transport.TransportGitSsh$SshPushConnection.<init>(TransportGitSsh.java:358)

at org.eclipse.jgit.transport.TransportGitSsh.openPush(TransportGitSsh.java:159)

at org.eclipse.jgit.transport.PushProcess.execute(PushProcess.java:127)

at org.eclipse.jgit.transport.Transport.push(Transport.java:1384)

at org.eclipse.jgit.transport.Transport.push(Transport.java:1430)

at com.googlesource.gerrit.plugins.replication.PushOne.pushVia(PushOne.java:567)

at com.googlesource.gerrit.plugins.replication.PushOne.runImpl(PushOne.java:538)

at com.googlesource.gerrit.plugins.replication.PushOne.doRunPushOperation(PushOne.java:421)