git-upload-pack failures on one repo.
306 views
Skip to first unread message
elzoc...@gmail.com
Feb 1, 2024, 12:14:49 PM2/1/24
to Repo and Gerrit Discussion
Hi,
In the past week we have been having some Jenkins fails on a specific repo.
Setup Gerrit 3.7.6, for investigations purpose, all traffic to that repo is sent to our Main Gerrit server (which receive UI/Clones/Pushes/...) (no replica).
I don't see any pattern of when it's happening, I had 3 failures in the past 12h out of a few thousands git-upload-pack success
Below, 2 different jobs with error from this morning, first git ls-remote, second git clone.
This is one of our most active repo, lots of branches, tags, changes. I usually run a git gc daily on it.
In the past week we have been having some Jenkins fails on a specific repo.
Setup Gerrit 3.7.6, for investigations purpose, all traffic to that repo is sent to our Main Gerrit server (which receive UI/Clones/Pushes/...) (no replica).
I don't see any pattern of when it's happening, I had 3 failures in the past 12h out of a few thousands git-upload-pack success
Below, 2 different jobs with error from this morning, first git ls-remote, second git clone.
This is one of our most active repo, lots of branches, tags, changes. I usually run a git gc daily on it.
git fsck --full shows some dangling commits (75), blobs (18) and tree(1) but no errors.
Any idea on what I can check / do ?
Thanks,
Cedric.
#Jenkins
git -c core.fsyncobjectfiles=0 ls-remote https://gerritserver.com/r/project/path/projname failed with exit code 128, output:
03:57:30 warning: core.fsyncObjectFiles is deprecated; use core.fsync instead
03:57:30 error: RPC failed; HTTP 500 curl 22 The requested URL returned error: 500
03:57:30 fatal: expected flush after ref listing
#Gerrit error_log
[2024-02-01T03:57:30.340Z] [HTTP POST /r/project/path/projname/git-upload-pack (N/A from x.x.x.x)] ERROR com.google.gerrit.httpd.GitOverHttpServlet.GerritUploadPackErrorHandler : Internal error during upload-pack from Repository[/gerrit/shared/git/project/path/projname.git] [CONTEXT project="project/path/projname" request="GIT_UPLOAD" ]
java.lang.IllegalArgumentException: Multiple entries with same key: 63895=ChangeData{Change{63895 (I49c898ae0c864326cd51ae619d54d1de5fbb771c), dest=project/path/projname,refs/heads/dev_sprint_21_1, status=A}} and 63895=ChangeData{Change{63895 (I49c898ae0c864326cd51ae619d54d1de5fbb771c), dest=project/path/projname,refs/heads/dev_sprint_21_1, status=A}}
at com.google.common.collect.ImmutableMap.conflictException(ImmutableMap.java:211)
at com.google.common.collect.ImmutableMap.checkNoConflict(ImmutableMap.java:205)
at com.google.common.collect.RegularImmutableMap.checkNoConflictInKeyBucket(RegularImmutableMap.java:146)
at com.google.common.collect.RegularImmutableMap.fromEntryArray(RegularImmutableMap.java:109)
at com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:390)
at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:582)
at com.google.gerrit.server.permissions.GitVisibleChangeFilter.getVisibleChanges(GitVisibleChangeFilter.java:90)
at com.google.gerrit.server.permissions.DefaultRefFilter.lambda$filter$1(DefaultRefFilter.java:147)
at com.google.common.base.Suppliers$NonSerializableMemoizingSupplier.get(Suppliers.java:167)
at com.google.gerrit.server.permissions.DefaultRefFilter.filterRefs(DefaultRefFilter.java:268)
at com.google.gerrit.server.permissions.DefaultRefFilter.filter(DefaultRefFilter.java:154)
at com.google.gerrit.server.permissions.ProjectControl$ForProjectImpl.filter(ProjectControl.java:425)
at com.google.gerrit.server.git.PermissionAwareReadOnlyRefDatabase.getRefsByPrefix(PermissionAwareReadOnlyRefDatabase.java:138)
at com.google.gerrit.server.git.PermissionAwareReadOnlyRefDatabase.getRefs(PermissionAwareReadOnlyRefDatabase.java:191)
at org.eclipse.jgit.transport.UploadPack.getAdvertisedOrDefaultRefs(UploadPack.java:919)
at org.eclipse.jgit.transport.UploadPack.getFilteredRefs(UploadPack.java:928)
at org.eclipse.jgit.transport.UploadPack.lsRefsV2(UploadPack.java:1143)
at org.eclipse.jgit.transport.UploadPack.serveOneCommandV2(UploadPack.java:1355)
at org.eclipse.jgit.transport.UploadPack.serviceV2(UploadPack.java:1420)
at org.eclipse.jgit.transport.UploadPack.uploadWithExceptionPropagation(UploadPack.java:871)
at org.eclipse.jgit.http.server.UploadPackServlet.upload(UploadPackServlet.java:192)
at org.eclipse.jgit.http.server.UploadPackServlet.lambda$doPost$0(UploadPackServlet.java:171)
at com.google.gerrit.httpd.GitOverHttpServlet$GerritUploadPackErrorHandler.upload(GitOverHttpServlet.java:493)
at org.eclipse.jgit.http.server.UploadPackServlet.doPost(UploadPackServlet.java:174)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.eclipse.jgit.http.server.glue.UrlPipeline$Chain.doFilter(UrlPipeline.java:212)
at com.google.gerrit.httpd.GitOverHttpServlet$UploadFilter.doFilter(GitOverHttpServlet.java:461)
at org.eclipse.jgit.http.server.glue.UrlPipeline$Chain.doFilter(UrlPipeline.java:210)
at org.eclipse.jgit.http.server.UploadPackServlet$Factory.doFilter(UploadPackServlet.java:137)
at org.eclipse.jgit.http.server.glue.UrlPipeline$Chain.doFilter(UrlPipeline.java:210)
at org.eclipse.jgit.http.server.RepositoryFilter.doFilter(RepositoryFilter.java:112)
at org.eclipse.jgit.http.server.glue.UrlPipeline$Chain.doFilter(UrlPipeline.java:210)
at org.eclipse.jgit.http.server.NoCacheFilter.doFilter(NoCacheFilter.java:53)
at org.eclipse.jgit.http.server.glue.UrlPipeline$Chain.doFilter(UrlPipeline.java:210)
at org.eclipse.jgit.http.server.glue.UrlPipeline.service(UrlPipeline.java:189)
at org.eclipse.jgit.http.server.glue.SuffixPipeline.service(SuffixPipeline.java:70)
at org.eclipse.jgit.http.server.glue.MetaFilter.doFilter(MetaFilter.java:151)
at org.eclipse.jgit.http.server.glue.MetaServlet.service(MetaServlet.java:109)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at com.google.inject.servlet.ServletDefinition.doServiceImpl(ServletDefinition.java:290)
at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:280)
at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:184)
at com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:89)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:85)
at com.google.gerrit.httpd.raw.StaticModule$PolyGerritFilter.doFilter(StaticModule.java:408)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.gerrit.httpd.GetUserFilter.doFilter(GetUserFilter.java:92)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.gerrit.httpd.RequireSslFilter.doFilter(RequireSslFilter.java:72)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.gerrit.httpd.RunAsFilter.doFilter(RunAsFilter.java:120)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.gerrit.httpd.SetThreadNameFilter.doFilter(SetThreadNameFilter.java:62)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.gerrit.httpd.AllRequestFilter$FilterProxy$1.doFilter(AllRequestFilter.java:139)
at com.googlesource.gerrit.plugins.readonly.ReadOnly.doFilter(ReadOnly.java:79)
at com.google.gerrit.httpd.AllRequestFilter$FilterProxy$1.doFilter(AllRequestFilter.java:135)
at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:239)
at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:215)
at com.googlesource.gerrit.plugins.javamelody.GerritMonitoringFilter.doFilter(GerritMonitoringFilter.java:66)
at com.google.gerrit.httpd.AllRequestFilter$FilterProxy$1.doFilter(AllRequestFilter.java:135)
at com.google.gerrit.httpd.AllowRenderInFrameFilter.doFilter(AllowRenderInFrameFilter.java:56)
at com.google.gerrit.httpd.AllRequestFilter$FilterProxy$1.doFilter(AllRequestFilter.java:135)
at com.google.gerrit.httpd.AllRequestFilter$FilterProxy.doFilter(AllRequestFilter.java:141)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.gerrit.httpd.RequestCleanupFilter.doFilter(RequestCleanupFilter.java:60)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.gerrit.pgm.http.jetty.ProjectQoSFilter.doFilter(ProjectQoSFilter.java:183)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.gerrit.httpd.ProjectBasicAuthFilter.doFilter(ProjectBasicAuthFilter.java:111)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.gerrit.httpd.RequestMetricsFilter.doFilter(RequestMetricsFilter.java:92)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.gerrit.httpd.RequestContextFilter.doFilter(RequestContextFilter.java:64)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:121)
at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:133)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:552)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:505)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.RequestLogHandler.handle(RequestLogHandler.java:54)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handleAsync(Server.java:559)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$2(HttpChannel.java:495)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:495)
at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:439)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
at java.base/java.lang.Thread.run(Thread.java:829)
#Jenkins
LANG=C git -c core.fsyncobjectfiles=0 clone --bare --mirror https://gerritserver.com/r/project/path/projname /home/jenkins/jenkinsroot/workspace --progress failed with exit code 128, no output
04:16:06 ERROR: lib32-projname-5-r0 do_fetch: Fetcher failure for URL: 'git://gerritserver.com/r/project/path/projname;protocol=https;branch=stable2;nobranch=1;name=projname'. Unable to fetch URL from any source.
#Gerrit error_log
[2024-02-01T04:16:06.082Z] [HTTP POST /r/project/path/projname/git-upload-pack (N/A from x.x.x.x)] ERROR com.google.gerrit.httpd.GitOverHttpServlet.GerritUploadPackErrorHandler : Internal error during upload-pack from Repository[/gerrit/shared/git/project/path/projname.git] [CONTEXT project="project/path/projname" request="GIT_UPLOAD" ]
java.lang.IllegalArgumentException: Multiple entries with same key: 85162=ChangeData{Change{85162 (I8390892f03caf0c5a686f81d57e50752f604b28a), dest=project/path/projname,refs/heads/stable2, status=M}} and 85162=ChangeData{Change{85162 (I8390892f03caf0c5a686f81d57e50752f604b28a), dest=project/path/projname,refs/heads/stable2, status=M}}
at com.google.common.collect.ImmutableMap.conflictException(ImmutableMap.java:211)
at com.google.common.collect.ImmutableMap.checkNoConflict(ImmutableMap.java:205)
at com.google.common.collect.RegularImmutableMap.checkNoConflictInKeyBucket(RegularImmutableMap.java:146)
at com.google.common.collect.RegularImmutableMap.fromEntryArray(RegularImmutableMap.java:109)
at com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:390)
at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:582)
at com.google.gerrit.server.permissions.GitVisibleChangeFilter.getVisibleChanges(GitVisibleChangeFilter.java:90)
at com.google.gerrit.server.permissions.DefaultRefFilter.lambda$filter$1(DefaultRefFilter.java:147)
at com.google.common.base.Suppliers$NonSerializableMemoizingSupplier.get(Suppliers.java:167)
at com.google.gerrit.server.permissions.DefaultRefFilter.filterRefs(DefaultRefFilter.java:268)
at com.google.gerrit.server.permissions.DefaultRefFilter.filter(DefaultRefFilter.java:154)
at com.google.gerrit.server.permissions.ProjectControl$ForProjectImpl.filter(ProjectControl.java:425)
at com.google.gerrit.server.git.PermissionAwareReadOnlyRefDatabase.getRefsByPrefix(PermissionAwareReadOnlyRefDatabase.java:138)
at com.google.gerrit.server.git.PermissionAwareReadOnlyRefDatabase.getRefs(PermissionAwareReadOnlyRefDatabase.java:191)
at org.eclipse.jgit.transport.UploadPack.getAdvertisedOrDefaultRefs(UploadPack.java:919)
at org.eclipse.jgit.transport.UploadPack.fetchV2(UploadPack.java:1187)
at org.eclipse.jgit.transport.UploadPack.serveOneCommandV2(UploadPack.java:1359)
at org.eclipse.jgit.transport.UploadPack.serviceV2(UploadPack.java:1420)
at org.eclipse.jgit.transport.UploadPack.uploadWithExceptionPropagation(UploadPack.java:871)
at org.eclipse.jgit.http.server.UploadPackServlet.upload(UploadPackServlet.java:192)
at org.eclipse.jgit.http.server.UploadPackServlet.lambda$doPost$0(UploadPackServlet.java:171)
at com.google.gerrit.httpd.GitOverHttpServlet$GerritUploadPackErrorHandler.upload(GitOverHttpServlet.java:493)
at org.eclipse.jgit.http.server.UploadPackServlet.doPost(UploadPackServlet.java:174)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.eclipse.jgit.http.server.glue.UrlPipeline$Chain.doFilter(UrlPipeline.java:212)
at com.google.gerrit.httpd.GitOverHttpServlet$UploadFilter.doFilter(GitOverHttpServlet.java:461)
at org.eclipse.jgit.http.server.glue.UrlPipeline$Chain.doFilter(UrlPipeline.java:210)
at org.eclipse.jgit.http.server.UploadPackServlet$Factory.doFilter(UploadPackServlet.java:137)
at org.eclipse.jgit.http.server.glue.UrlPipeline$Chain.doFilter(UrlPipeline.java:210)
at org.eclipse.jgit.http.server.RepositoryFilter.doFilter(RepositoryFilter.java:112)
at org.eclipse.jgit.http.server.glue.UrlPipeline$Chain.doFilter(UrlPipeline.java:210)
at org.eclipse.jgit.http.server.NoCacheFilter.doFilter(NoCacheFilter.java:53)
at org.eclipse.jgit.http.server.glue.UrlPipeline$Chain.doFilter(UrlPipeline.java:210)
at org.eclipse.jgit.http.server.glue.UrlPipeline.service(UrlPipeline.java:189)
at org.eclipse.jgit.http.server.glue.SuffixPipeline.service(SuffixPipeline.java:70)
at org.eclipse.jgit.http.server.glue.MetaFilter.doFilter(MetaFilter.java:151)
at org.eclipse.jgit.http.server.glue.MetaServlet.service(MetaServlet.java:109)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at com.google.inject.servlet.ServletDefinition.doServiceImpl(ServletDefinition.java:290)
at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:280)
at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:184)
at com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:89)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:85)
at com.google.gerrit.httpd.raw.StaticModule$PolyGerritFilter.doFilter(StaticModule.java:408)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.gerrit.httpd.GetUserFilter.doFilter(GetUserFilter.java:92)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.gerrit.httpd.RequireSslFilter.doFilter(RequireSslFilter.java:72)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.gerrit.httpd.RunAsFilter.doFilter(RunAsFilter.java:120)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.gerrit.httpd.SetThreadNameFilter.doFilter(SetThreadNameFilter.java:62)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.gerrit.httpd.AllRequestFilter$FilterProxy$1.doFilter(AllRequestFilter.java:139)
at com.googlesource.gerrit.plugins.readonly.ReadOnly.doFilter(ReadOnly.java:79)
at com.google.gerrit.httpd.AllRequestFilter$FilterProxy$1.doFilter(AllRequestFilter.java:135)
at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:239)
at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:215)
at com.googlesource.gerrit.plugins.javamelody.GerritMonitoringFilter.doFilter(GerritMonitoringFilter.java:66)
at com.google.gerrit.httpd.AllRequestFilter$FilterProxy$1.doFilter(AllRequestFilter.java:135)
at com.google.gerrit.httpd.AllowRenderInFrameFilter.doFilter(AllowRenderInFrameFilter.java:56)
at com.google.gerrit.httpd.AllRequestFilter$FilterProxy$1.doFilter(AllRequestFilter.java:135)
at com.google.gerrit.httpd.AllRequestFilter$FilterProxy.doFilter(AllRequestFilter.java:141)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.gerrit.httpd.RequestCleanupFilter.doFilter(RequestCleanupFilter.java:60)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.gerrit.pgm.http.jetty.ProjectQoSFilter.doFilter(ProjectQoSFilter.java:183)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.gerrit.httpd.ProjectBasicAuthFilter.doFilter(ProjectBasicAuthFilter.java:111)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.gerrit.httpd.RequestMetricsFilter.doFilter(RequestMetricsFilter.java:92)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.gerrit.httpd.RequestContextFilter.doFilter(RequestContextFilter.java:64)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:121)
at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:133)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:552)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:505)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.RequestLogHandler.handle(RequestLogHandler.java:54)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handleAsync(Server.java:559)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$2(HttpChannel.java:495)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:495)
at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:439)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
at java.base/java.lang.Thread.run(Thread.java:829)
Thanks,
Cedric.
#Jenkins
git -c core.fsyncobjectfiles=0 ls-remote https://gerritserver.com/r/project/path/projname failed with exit code 128, output:
03:57:30 warning: core.fsyncObjectFiles is deprecated; use core.fsync instead
03:57:30 error: RPC failed; HTTP 500 curl 22 The requested URL returned error: 500
03:57:30 fatal: expected flush after ref listing
#Gerrit error_log
[2024-02-01T03:57:30.340Z] [HTTP POST /r/project/path/projname/git-upload-pack (N/A from x.x.x.x)] ERROR com.google.gerrit.httpd.GitOverHttpServlet.GerritUploadPackErrorHandler : Internal error during upload-pack from Repository[/gerrit/shared/git/project/path/projname.git] [CONTEXT project="project/path/projname" request="GIT_UPLOAD" ]
java.lang.IllegalArgumentException: Multiple entries with same key: 63895=ChangeData{Change{63895 (I49c898ae0c864326cd51ae619d54d1de5fbb771c), dest=project/path/projname,refs/heads/dev_sprint_21_1, status=A}} and 63895=ChangeData{Change{63895 (I49c898ae0c864326cd51ae619d54d1de5fbb771c), dest=project/path/projname,refs/heads/dev_sprint_21_1, status=A}}
at com.google.common.collect.ImmutableMap.conflictException(ImmutableMap.java:211)
at com.google.common.collect.ImmutableMap.checkNoConflict(ImmutableMap.java:205)
at com.google.common.collect.RegularImmutableMap.checkNoConflictInKeyBucket(RegularImmutableMap.java:146)
at com.google.common.collect.RegularImmutableMap.fromEntryArray(RegularImmutableMap.java:109)
at com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:390)
at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:582)
at com.google.gerrit.server.permissions.GitVisibleChangeFilter.getVisibleChanges(GitVisibleChangeFilter.java:90)
at com.google.gerrit.server.permissions.DefaultRefFilter.lambda$filter$1(DefaultRefFilter.java:147)
at com.google.common.base.Suppliers$NonSerializableMemoizingSupplier.get(Suppliers.java:167)
at com.google.gerrit.server.permissions.DefaultRefFilter.filterRefs(DefaultRefFilter.java:268)
at com.google.gerrit.server.permissions.DefaultRefFilter.filter(DefaultRefFilter.java:154)
at com.google.gerrit.server.permissions.ProjectControl$ForProjectImpl.filter(ProjectControl.java:425)
at com.google.gerrit.server.git.PermissionAwareReadOnlyRefDatabase.getRefsByPrefix(PermissionAwareReadOnlyRefDatabase.java:138)
at com.google.gerrit.server.git.PermissionAwareReadOnlyRefDatabase.getRefs(PermissionAwareReadOnlyRefDatabase.java:191)
at org.eclipse.jgit.transport.UploadPack.getAdvertisedOrDefaultRefs(UploadPack.java:919)
at org.eclipse.jgit.transport.UploadPack.getFilteredRefs(UploadPack.java:928)
at org.eclipse.jgit.transport.UploadPack.lsRefsV2(UploadPack.java:1143)
at org.eclipse.jgit.transport.UploadPack.serveOneCommandV2(UploadPack.java:1355)
at org.eclipse.jgit.transport.UploadPack.serviceV2(UploadPack.java:1420)
at org.eclipse.jgit.transport.UploadPack.uploadWithExceptionPropagation(UploadPack.java:871)
at org.eclipse.jgit.http.server.UploadPackServlet.upload(UploadPackServlet.java:192)
at org.eclipse.jgit.http.server.UploadPackServlet.lambda$doPost$0(UploadPackServlet.java:171)
at com.google.gerrit.httpd.GitOverHttpServlet$GerritUploadPackErrorHandler.upload(GitOverHttpServlet.java:493)
at org.eclipse.jgit.http.server.UploadPackServlet.doPost(UploadPackServlet.java:174)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.eclipse.jgit.http.server.glue.UrlPipeline$Chain.doFilter(UrlPipeline.java:212)
at com.google.gerrit.httpd.GitOverHttpServlet$UploadFilter.doFilter(GitOverHttpServlet.java:461)
at org.eclipse.jgit.http.server.glue.UrlPipeline$Chain.doFilter(UrlPipeline.java:210)
at org.eclipse.jgit.http.server.UploadPackServlet$Factory.doFilter(UploadPackServlet.java:137)
at org.eclipse.jgit.http.server.glue.UrlPipeline$Chain.doFilter(UrlPipeline.java:210)
at org.eclipse.jgit.http.server.RepositoryFilter.doFilter(RepositoryFilter.java:112)
at org.eclipse.jgit.http.server.glue.UrlPipeline$Chain.doFilter(UrlPipeline.java:210)
at org.eclipse.jgit.http.server.NoCacheFilter.doFilter(NoCacheFilter.java:53)
at org.eclipse.jgit.http.server.glue.UrlPipeline$Chain.doFilter(UrlPipeline.java:210)
at org.eclipse.jgit.http.server.glue.UrlPipeline.service(UrlPipeline.java:189)
at org.eclipse.jgit.http.server.glue.SuffixPipeline.service(SuffixPipeline.java:70)
at org.eclipse.jgit.http.server.glue.MetaFilter.doFilter(MetaFilter.java:151)
at org.eclipse.jgit.http.server.glue.MetaServlet.service(MetaServlet.java:109)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at com.google.inject.servlet.ServletDefinition.doServiceImpl(ServletDefinition.java:290)
at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:280)
at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:184)
at com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:89)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:85)
at com.google.gerrit.httpd.raw.StaticModule$PolyGerritFilter.doFilter(StaticModule.java:408)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.gerrit.httpd.GetUserFilter.doFilter(GetUserFilter.java:92)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.gerrit.httpd.RequireSslFilter.doFilter(RequireSslFilter.java:72)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.gerrit.httpd.RunAsFilter.doFilter(RunAsFilter.java:120)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.gerrit.httpd.SetThreadNameFilter.doFilter(SetThreadNameFilter.java:62)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.gerrit.httpd.AllRequestFilter$FilterProxy$1.doFilter(AllRequestFilter.java:139)
at com.googlesource.gerrit.plugins.readonly.ReadOnly.doFilter(ReadOnly.java:79)
at com.google.gerrit.httpd.AllRequestFilter$FilterProxy$1.doFilter(AllRequestFilter.java:135)
at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:239)
at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:215)
at com.googlesource.gerrit.plugins.javamelody.GerritMonitoringFilter.doFilter(GerritMonitoringFilter.java:66)
at com.google.gerrit.httpd.AllRequestFilter$FilterProxy$1.doFilter(AllRequestFilter.java:135)
at com.google.gerrit.httpd.AllowRenderInFrameFilter.doFilter(AllowRenderInFrameFilter.java:56)
at com.google.gerrit.httpd.AllRequestFilter$FilterProxy$1.doFilter(AllRequestFilter.java:135)
at com.google.gerrit.httpd.AllRequestFilter$FilterProxy.doFilter(AllRequestFilter.java:141)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.gerrit.httpd.RequestCleanupFilter.doFilter(RequestCleanupFilter.java:60)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.gerrit.pgm.http.jetty.ProjectQoSFilter.doFilter(ProjectQoSFilter.java:183)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.gerrit.httpd.ProjectBasicAuthFilter.doFilter(ProjectBasicAuthFilter.java:111)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.gerrit.httpd.RequestMetricsFilter.doFilter(RequestMetricsFilter.java:92)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.gerrit.httpd.RequestContextFilter.doFilter(RequestContextFilter.java:64)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:121)
at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:133)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:552)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:505)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.RequestLogHandler.handle(RequestLogHandler.java:54)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handleAsync(Server.java:559)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$2(HttpChannel.java:495)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:495)
at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:439)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
at java.base/java.lang.Thread.run(Thread.java:829)
#Jenkins
LANG=C git -c core.fsyncobjectfiles=0 clone --bare --mirror https://gerritserver.com/r/project/path/projname /home/jenkins/jenkinsroot/workspace --progress failed with exit code 128, no output
04:16:06 ERROR: lib32-projname-5-r0 do_fetch: Fetcher failure for URL: 'git://gerritserver.com/r/project/path/projname;protocol=https;branch=stable2;nobranch=1;name=projname'. Unable to fetch URL from any source.
#Gerrit error_log
[2024-02-01T04:16:06.082Z] [HTTP POST /r/project/path/projname/git-upload-pack (N/A from x.x.x.x)] ERROR com.google.gerrit.httpd.GitOverHttpServlet.GerritUploadPackErrorHandler : Internal error during upload-pack from Repository[/gerrit/shared/git/project/path/projname.git] [CONTEXT project="project/path/projname" request="GIT_UPLOAD" ]
java.lang.IllegalArgumentException: Multiple entries with same key: 85162=ChangeData{Change{85162 (I8390892f03caf0c5a686f81d57e50752f604b28a), dest=project/path/projname,refs/heads/stable2, status=M}} and 85162=ChangeData{Change{85162 (I8390892f03caf0c5a686f81d57e50752f604b28a), dest=project/path/projname,refs/heads/stable2, status=M}}
at com.google.common.collect.ImmutableMap.conflictException(ImmutableMap.java:211)
at com.google.common.collect.ImmutableMap.checkNoConflict(ImmutableMap.java:205)
at com.google.common.collect.RegularImmutableMap.checkNoConflictInKeyBucket(RegularImmutableMap.java:146)
at com.google.common.collect.RegularImmutableMap.fromEntryArray(RegularImmutableMap.java:109)
at com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:390)
at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:582)
at com.google.gerrit.server.permissions.GitVisibleChangeFilter.getVisibleChanges(GitVisibleChangeFilter.java:90)
at com.google.gerrit.server.permissions.DefaultRefFilter.lambda$filter$1(DefaultRefFilter.java:147)
at com.google.common.base.Suppliers$NonSerializableMemoizingSupplier.get(Suppliers.java:167)
at com.google.gerrit.server.permissions.DefaultRefFilter.filterRefs(DefaultRefFilter.java:268)
at com.google.gerrit.server.permissions.DefaultRefFilter.filter(DefaultRefFilter.java:154)
at com.google.gerrit.server.permissions.ProjectControl$ForProjectImpl.filter(ProjectControl.java:425)
at com.google.gerrit.server.git.PermissionAwareReadOnlyRefDatabase.getRefsByPrefix(PermissionAwareReadOnlyRefDatabase.java:138)
at com.google.gerrit.server.git.PermissionAwareReadOnlyRefDatabase.getRefs(PermissionAwareReadOnlyRefDatabase.java:191)
at org.eclipse.jgit.transport.UploadPack.getAdvertisedOrDefaultRefs(UploadPack.java:919)
at org.eclipse.jgit.transport.UploadPack.fetchV2(UploadPack.java:1187)
at org.eclipse.jgit.transport.UploadPack.serveOneCommandV2(UploadPack.java:1359)
at org.eclipse.jgit.transport.UploadPack.serviceV2(UploadPack.java:1420)
at org.eclipse.jgit.transport.UploadPack.uploadWithExceptionPropagation(UploadPack.java:871)
at org.eclipse.jgit.http.server.UploadPackServlet.upload(UploadPackServlet.java:192)
at org.eclipse.jgit.http.server.UploadPackServlet.lambda$doPost$0(UploadPackServlet.java:171)
at com.google.gerrit.httpd.GitOverHttpServlet$GerritUploadPackErrorHandler.upload(GitOverHttpServlet.java:493)
at org.eclipse.jgit.http.server.UploadPackServlet.doPost(UploadPackServlet.java:174)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.eclipse.jgit.http.server.glue.UrlPipeline$Chain.doFilter(UrlPipeline.java:212)
at com.google.gerrit.httpd.GitOverHttpServlet$UploadFilter.doFilter(GitOverHttpServlet.java:461)
at org.eclipse.jgit.http.server.glue.UrlPipeline$Chain.doFilter(UrlPipeline.java:210)
at org.eclipse.jgit.http.server.UploadPackServlet$Factory.doFilter(UploadPackServlet.java:137)
at org.eclipse.jgit.http.server.glue.UrlPipeline$Chain.doFilter(UrlPipeline.java:210)
at org.eclipse.jgit.http.server.RepositoryFilter.doFilter(RepositoryFilter.java:112)
at org.eclipse.jgit.http.server.glue.UrlPipeline$Chain.doFilter(UrlPipeline.java:210)
at org.eclipse.jgit.http.server.NoCacheFilter.doFilter(NoCacheFilter.java:53)
at org.eclipse.jgit.http.server.glue.UrlPipeline$Chain.doFilter(UrlPipeline.java:210)
at org.eclipse.jgit.http.server.glue.UrlPipeline.service(UrlPipeline.java:189)
at org.eclipse.jgit.http.server.glue.SuffixPipeline.service(SuffixPipeline.java:70)
at org.eclipse.jgit.http.server.glue.MetaFilter.doFilter(MetaFilter.java:151)
at org.eclipse.jgit.http.server.glue.MetaServlet.service(MetaServlet.java:109)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at com.google.inject.servlet.ServletDefinition.doServiceImpl(ServletDefinition.java:290)
at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:280)
at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:184)
at com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:89)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:85)
at com.google.gerrit.httpd.raw.StaticModule$PolyGerritFilter.doFilter(StaticModule.java:408)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.gerrit.httpd.GetUserFilter.doFilter(GetUserFilter.java:92)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.gerrit.httpd.RequireSslFilter.doFilter(RequireSslFilter.java:72)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.gerrit.httpd.RunAsFilter.doFilter(RunAsFilter.java:120)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.gerrit.httpd.SetThreadNameFilter.doFilter(SetThreadNameFilter.java:62)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.gerrit.httpd.AllRequestFilter$FilterProxy$1.doFilter(AllRequestFilter.java:139)
at com.googlesource.gerrit.plugins.readonly.ReadOnly.doFilter(ReadOnly.java:79)
at com.google.gerrit.httpd.AllRequestFilter$FilterProxy$1.doFilter(AllRequestFilter.java:135)
at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:239)
at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:215)
at com.googlesource.gerrit.plugins.javamelody.GerritMonitoringFilter.doFilter(GerritMonitoringFilter.java:66)
at com.google.gerrit.httpd.AllRequestFilter$FilterProxy$1.doFilter(AllRequestFilter.java:135)
at com.google.gerrit.httpd.AllowRenderInFrameFilter.doFilter(AllowRenderInFrameFilter.java:56)
at com.google.gerrit.httpd.AllRequestFilter$FilterProxy$1.doFilter(AllRequestFilter.java:135)
at com.google.gerrit.httpd.AllRequestFilter$FilterProxy.doFilter(AllRequestFilter.java:141)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.gerrit.httpd.RequestCleanupFilter.doFilter(RequestCleanupFilter.java:60)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.gerrit.pgm.http.jetty.ProjectQoSFilter.doFilter(ProjectQoSFilter.java:183)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.gerrit.httpd.ProjectBasicAuthFilter.doFilter(ProjectBasicAuthFilter.java:111)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.gerrit.httpd.RequestMetricsFilter.doFilter(RequestMetricsFilter.java:92)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.gerrit.httpd.RequestContextFilter.doFilter(RequestContextFilter.java:64)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:121)
at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:133)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:552)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:505)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.RequestLogHandler.handle(RequestLogHandler.java:54)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handleAsync(Server.java:559)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$2(HttpChannel.java:495)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:495)
at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:439)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
at java.base/java.lang.Thread.run(Thread.java:829)
Luca Milanesio
Feb 1, 2024, 1:06:17 PM2/1/24
to Repo and Gerrit Discussion, Luca Milanesio, elzoc...@gmail.com
On 1 Feb 2024, at 12:14, elzoc...@gmail.com <elzoc...@gmail.com> wrote:Hi,
In the past week we have been having some Jenkins fails on a specific repo.
Setup Gerrit 3.7.6, for investigations purpose, all traffic to that repo is sent to our Main Gerrit server (which receive UI/Clones/Pushes/...) (no replica).
I don't see any pattern of when it's happening, I had 3 failures in the past 12h out of a few thousands git-upload-pack success
See the documentation on the index.paginationType configuration at [1].
I believe you’re using the default pagination type that doesn’t assure search results consistency and may lead to duplicates.
You need to set in gerrit.config:
[index]
paginationType = NONE
We had the same issue on our setup and then discovered the inconsistency in the search results and introduced the old legacy (none) pagination type.
Luca.
Below, 2 different jobs with error from this morning, first git ls-remote, second git clone.
This is one of our most active repo, lots of branches, tags, changes. I usually run a git gc daily on it.git fsck --full shows some dangling commits (75), blobs (18) and tree(1) but no errors.Any idea on what I can check / do ?
Thanks,
Cedric.
--
--
To unsubscribe, email repo-discuss...@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en
---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/repo-discuss/82f9eef1-89ce-463b-aca1-b0307e1263bcn%40googlegroups.com.
Cédric LE COZ
Feb 1, 2024, 2:51:10 PM2/1/24
to Repo and Gerrit Discussion
Le jeudi 1 février 2024 à 13:06:17 UTC, Luca Milanesio a écrit :
On 1 Feb 2024, at 12:14, elzoc...@gmail.com <elzoc...@gmail.com> wrote:Hi,
In the past week we have been having some Jenkins fails on a specific repo.
Setup Gerrit 3.7.6, for investigations purpose, all traffic to that repo is sent to our Main Gerrit server (which receive UI/Clones/Pushes/...) (no replica).
I don't see any pattern of when it's happening, I had 3 failures in the past 12h out of a few thousands git-upload-pack successSee the documentation on the index.paginationType configuration at [1].I believe you’re using the default pagination type that doesn’t assure search results consistency and may lead to duplicates.You need to set in gerrit.config:[index]paginationType = NONE
Correct, at the moment I have the following, I am not sure where the defaultMaxClauseCount comes from, I can't see in [1], nor in older versions...
I'll remove it as well as the type setting as [1] says it's deprecated.
[index]
type = lucene defaultMaxClauseCount = 2048
Just a restart of the service is enough after those parameter change? no need for a re-index or anything else?
Do you know if there is a way to easily reproduce a clone failure due that? I tried to load test that component on a dev instances, but could not reproduce.
Thanks!
Cedric.
Luca Milanesio
Feb 1, 2024, 3:26:32 PM2/1/24
to Repo and Gerrit Discussion, Luca Milanesio, Cédric LE COZ
On 1 Feb 2024, at 14:51, Cédric LE COZ <elzoc...@gmail.com> wrote:Le jeudi 1 février 2024 à 13:06:17 UTC, Luca Milanesio a écrit :On 1 Feb 2024, at 12:14, elzoc...@gmail.com <elzoc...@gmail.com> wrote:Hi,
In the past week we have been having some Jenkins fails on a specific repo.
Setup Gerrit 3.7.6, for investigations purpose, all traffic to that repo is sent to our Main Gerrit server (which receive UI/Clones/Pushes/...) (no replica).
I don't see any pattern of when it's happening, I had 3 failures in the past 12h out of a few thousands git-upload-pack successSee the documentation on the index.paginationType configuration at [1].I believe you’re using the default pagination type that doesn’t assure search results consistency and may lead to duplicates.You need to set in gerrit.config:[index]paginationType = NONECorrect, at the moment I have the following, I am not sure where the defaultMaxClauseCount comes from, I can't see in [1], nor in older versions...I'll remove it as well as the type setting as [1] says it's deprecated.[index]type = lucenedefaultMaxClauseCount = 2048Just a restart of the service is enough after those parameter change? no need for a re-index or anything else?
Yes, a Gerrit restart would suffice.
Do you know if there is a way to easily reproduce a clone failure due that? I tried to load test that component on a dev instances, but could not reproduce.
It’s a concurrency issue, therefore isn’t easy to reproduce.
Luca.
To view this discussion on the web visit https://groups.google.com/d/msgid/repo-discuss/97951c8b-9a48-4d3d-ac32-93c43a25c3edn%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages