Merge failed - missing blob?

[Nicholas Mucci]

Hello,
I've seen a few cases where someone submits a change, and it gets
stuck in "Submitted, Merge Pending" even though the logs show that the
merge has failed. The case I'm seeing now looks like this:

[2011-08-05 09:01:04,827] ERROR
com.google.gerrit.server.git.ChangeMergeQueue : Merge attempt for
foobar/repository,refs/heads/master failed
com.google.gerrit.server.git.MergeException: Cannot merge
509a77404c5a58959ec1eae733068cbd7b89c1f2
at
com.google.gerrit.server.git.MergeOp.cherryPickChanges(MergeOp.java:
766)
at com.google.gerrit.server.git.MergeOp.mergeImpl(MergeOp.java:
238)
at com.google.gerrit.server.git.MergeOp.merge(MergeOp.java:
213)
at
com.google.gerrit.server.git.ChangeMergeQueue.mergeImpl(ChangeMergeQueue.java:
179)
at
com.google.gerrit.server.git.ChangeMergeQueue.merge(ChangeMergeQueue.java:
99)
at com.google.gerrit.server.ChangeUtil.submit(ChangeUtil.java:
187)
at
com.google.gerrit.httpd.rpc.changedetail.SubmitAction.call(SubmitAction.java:
84)
at
com.google.gerrit.httpd.rpc.changedetail.SubmitAction.call(SubmitAction.java:
37)
at com.google.gerrit.httpd.rpc.Handler.to(Handler.java:65)
at
com.google.gerrit.httpd.rpc.changedetail.ChangeManageServiceImpl.submit(ChangeManageServiceImpl.java:
42)
at sun.reflect.GeneratedMethodAccessor71.invoke(Unknown
Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:
25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
com.google.gwtjsonrpc.server.MethodHandle.invoke(MethodHandle.java:91)
at
com.google.gwtjsonrpc.server.JsonServlet.doService(JsonServlet.java:
382)
at
com.google.gwtjsonrpc.server.JsonServlet.service(JsonServlet.java:268)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:
717)
at
com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:
231)
at
com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:
142)
at
com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:
93)
at
com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:
63)
at
com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:
137)
at
com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:
59)
at
com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:
137)
at
com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:
59)
at
com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:
137)
at
com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:
59)
at
com.google.gwtexpui.server.CacheControlFilter.doFilter(CacheControlFilter.java:
76)
at
com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:
132)
at
com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:
59)
at
com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:
137)
at
com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:
59)
at
com.google.gerrit.httpd.RequireSslFilter.doFilter(RequireSslFilter.java:
68)
at
com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:
132)
at
com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:
59)
at
com.google.gerrit.httpd.RequestCleanupFilter.doFilter(RequestCleanupFilter.java:
54)
at
com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:
132)
at
com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:
59)
at
com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:
122)
at
com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:110)
at org.eclipse.jetty.servlet.ServletHandler
$CachedChain.doFilter(ServletHandler.java:1322)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:
473)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:
921)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:
403)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:
856)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:
114)
at org.eclipse.jetty.server.Server.handle(Server.java:352)
at
org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:
596)
at org.eclipse.jetty.server.HttpConnection
$RequestHandler.content(HttpConnection.java:1069)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:
805)
at
org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:218)
at
org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:
426)
at
org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:
510)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint.access
$000(SelectChannelEndPoint.java:34)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint
$1.run(SelectChannelEndPoint.java:40)
at org.eclipse.jetty.util.thread.QueuedThreadPool
$2.run(QueuedThreadPool.java:450)
at java.lang.Thread.run(Thread.java:619)
Caused by: org.eclipse.jgit.errors.MissingObjectException: Missing
blob eff7175c77f4f183ccb6690fb19f8cffcf01ca6e
at
org.eclipse.jgit.storage.file.WindowCursor.open(WindowCursor.java:126)
at
org.eclipse.jgit.lib.ObjectDatabase.open(ObjectDatabase.java:176)
at org.eclipse.jgit.lib.Repository.open(Repository.java:270)
at
org.eclipse.jgit.merge.ResolveMerger.getRawText(ResolveMerger.java:
602)
at
org.eclipse.jgit.merge.ResolveMerger.contentMerge(ResolveMerger.java:
528)
at
org.eclipse.jgit.merge.ResolveMerger.processEntry(ResolveMerger.java:
451)
at
org.eclipse.jgit.merge.ResolveMerger.mergeImpl(ResolveMerger.java:194)
at org.eclipse.jgit.merge.Merger.merge(Merger.java:156)
at
org.eclipse.jgit.merge.ThreeWayMerger.merge(ThreeWayMerger.java:133)
at
org.eclipse.jgit.merge.ThreeWayMerger.merge(ThreeWayMerger.java:126)
at
com.google.gerrit.server.git.MergeOp.cherryPickChanges(MergeOp.java:
735)
... 57 more

What interests me is that I can't find a blob eff7175c in the
repository on the server. I ran a 'git show' on the full SHA1 and it
returned nothing, and looking in the objects directory, there was no
subdirectory starting with 'ef'. Running a 'git ls-tree 509a7740 |
grep eff7175c' on the repository on the server also provided
nothing . Could this be due to a user's repository being in an
invalid state or containing data that is not on the Gerrit server?
Thank you for any guidance on this.

[Nicholas Mucci]

After further investigation, I discovered that the "missing blob"
Gerrit is referring to is really the SHA1 of a commit in a submodule
that the parent project (in which the change that failed to merge
exists) refers to. What could cause Gerrit to get confused like this?

-Nick

> com.google.gwtexpui.server.CacheControlFilter.doFilter(CacheControlFilter.j ava:

> 76)
>         at
> com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:
> 132)
>         at
> com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocat ion.java:

> 59)
>         at
> com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:
> 137)
>         at
> com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocat ion.java:

[Martin Fick]

On Friday 05 August 2011 03:06:46 pm Nicholas Mucci wrote:
> After further investigation, I discovered that the
> "missing blob" Gerrit is referring to is really the SHA1
> of a commit in a submodule that the parent project (in
> which the change that failed to merge exists) refers to.
> What could cause Gerrit to get confused like this?

Strange, do they share Change-Ids by chance?

[Nicholas Mucci]

They do not share change ids. Using the above example, In change
509a7740, there is a line in the patch set referring to the submodule,
and the only content is a diff showing a change in the subproject
commit:

diff --git a/submodules/technology/foo/bar b/submodules/technology/foo/
bar
index eff7175..14e9fce 160000
--- a/submodules/technology/foo/bar
+++ b/submodules/technology/foo/bar
-Subproject commit eff7175c77f4f183ccb6690fb19f8cffcf01ca6e
+Subproject commit 14e9fce7b02e9e6f39a2cbf9f6be9a916a9aca34

Interestingly, eff7175c is what jgit is throwing a fit over, and that
change has been in for a few weeks.

Also, as a heads up, this is happening on Gerrit 2.1.7.

-Nick