gerrit 2.13: got internal server error when clone from slave

354 views
Skip to first unread message

Makson Lee

unread,
Sep 22, 2016, 7:10:58 PM9/22/16
to Repo and Gerrit Discussion
after upgraded to gerrit 2.13, we got internal server error when clone from slave, and then we turned slave mode off, problem gone, what's happened?

[2016-09-23 00:28:43,022] [SSH git-upload-pack '/platform/manifest' (xxxxxx)] ERROR com.google.gerrit.sshd.BaseCommand : Internal server error (user xxxxxx account 1000001) during git-upload-pack '/platform/manifest'
org.eclipse.jgit.transport.UploadPackInternalServerErrorException
        at org.eclipse.jgit.transport.UploadPack.service(UploadPack.java:768)
        at org.eclipse.jgit.transport.UploadPack.upload(UploadPack.java:667)
        at com.google.gerrit.sshd.commands.Upload.runImpl(Upload.java:92)
        at com.google.gerrit.sshd.AbstractGitCommand.service(AbstractGitCommand.java:101)
        at com.google.gerrit.sshd.AbstractGitCommand.access$000(AbstractGitCommand.java:32)
        at com.google.gerrit.sshd.AbstractGitCommand$1.run(AbstractGitCommand.java:70)
        at com.google.gerrit.sshd.BaseCommand$TaskThunk.run(BaseCommand.java:442)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at com.google.gerrit.server.git.WorkQueue$Task.run(WorkQueue.java:407)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: com.google.common.util.concurrent.UncheckedExecutionException: java.lang.NullPointerException
        at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2207)
        at com.google.common.cache.LocalCache.get(LocalCache.java:3953)
        at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3957)
        at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4875)
        at com.google.gerrit.server.git.SearchingChangeCacheImpl.getChangeData(SearchingChangeCacheImpl.java:102)
        at com.google.gerrit.server.git.VisibleRefFilter.visibleChangesBySearch(VisibleRefFilter.java:236)
        at com.google.gerrit.server.git.VisibleRefFilter.visible(VisibleRefFilter.java:216)
        at com.google.gerrit.server.git.VisibleRefFilter.filter(VisibleRefFilter.java:126)
        at com.google.gerrit.server.git.VisibleRefFilter.filter(VisibleRefFilter.java:208)
        at com.google.gerrit.server.git.VisibleRefFilter.getAdvertisedRefs(VisibleRefFilter.java:197)
        at org.eclipse.jgit.transport.AbstractAdvertiseRefsHook.advertiseRefs(AbstractAdvertiseRefsHook.java:62)
        at org.eclipse.jgit.transport.UploadPack.sendAdvertisedRefs(UploadPack.java:870)
        at org.eclipse.jgit.transport.UploadPack.service(UploadPack.java:718)
        ... 14 more
Caused by: java.lang.NullPointerException
        at com.google.gerrit.server.index.change.ChangeIndexRewriter.isIndexPredicate(ChangeIndexRewriter.java:235)
        at com.google.gerrit.server.index.change.ChangeIndexRewriter.rewriteImpl(ChangeIndexRewriter.java:185)
        at com.google.gerrit.server.index.change.ChangeIndexRewriter.rewriteImpl(ChangeIndexRewriter.java:157)
        at com.google.gerrit.server.index.change.ChangeIndexRewriter.rewrite(ChangeIndexRewriter.java:141)
        at com.google.gerrit.server.query.QueryProcessor.query(QueryProcessor.java:177)
        at com.google.gerrit.server.query.QueryProcessor.query(QueryProcessor.java:139)
        at com.google.gerrit.server.query.QueryProcessor.query(QueryProcessor.java:122)
        at com.google.gerrit.server.query.InternalQuery.query(InternalQuery.java:71)
        at com.google.gerrit.server.query.change.InternalChangeQuery.byProject(InternalChangeQuery.java:144)
        at com.google.gerrit.server.git.SearchingChangeCacheImpl$Loader.load(SearchingChangeCacheImpl.java:137)
        at com.google.gerrit.server.git.SearchingChangeCacheImpl$Loader.load(SearchingChangeCacheImpl.java:123)
        at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3542)
        at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2323)
        at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2286)
        at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2201)
        ... 26 more

Hugo Arès

unread,
Sep 23, 2016, 2:22:27 AM9/23/16
to Repo and Gerrit Discussion
It looks like VisibleRefFilter is using secondary index to evaluate the visibility of refs which cannot work in slave mode since there is no secondary index.

luca.mi...@gmail.com

unread,
Sep 23, 2016, 2:40:34 AM9/23/16
to Hugo Arès, Repo and Gerrit Discussion
Shall we start including a Docker-based master-slave scenario for our integration tests?

I believe we do not have master-slave scenarios in our suite ;-(

Luca

Sent from my iPhone
--
--
To unsubscribe, email repo-discuss...@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en

---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Makson Lee

unread,
Sep 23, 2016, 4:57:33 AM9/23/16
to Repo and Gerrit Discussion

On Friday, September 23, 2016 at 2:22:27 PM UTC+8, Hugo Arès wrote:
It looks like VisibleRefFilter is using secondary index to evaluate the visibility of refs which cannot work in slave mode since there is no secondary index.
 
before the issue is fixed, is it okay to just turn slave mode off?

Hugo Arès

unread,
Sep 23, 2016, 5:23:42 AM9/23/16
to Repo and Gerrit Discussion
If you turn slave mode off, users will be able to push to your slaves. As long as nobody push to slaves, I think it's ok.

Until --slave option is fixed, you can prevent users from pushing to slave by configuring receive.allowGroup to an empty(or set it to Administrators) group and use --headless to spawn gerrit without UI.

Makson Lee

unread,
Sep 23, 2016, 5:30:39 AM9/23/16
to Repo and Gerrit Discussion


On Friday, September 23, 2016 at 5:23:42 PM UTC+8, Hugo Arès wrote:
If you turn slave mode off, users will be able to push to your slaves. As long as nobody push to slaves, I think it's ok.
 
even the database used by slave is in standby mode which means it is read only, user can still push data to slave?

Hugo Arès

unread,
Sep 23, 2016, 5:53:09 AM9/23/16
to Repo and Gerrit Discussion
push will fail but I am not sure that commit will not make it to the repository.

I am testing the fix now, I will upload the change shortly.

Makson Lee

unread,
Sep 23, 2016, 6:00:59 AM9/23/16
to Repo and Gerrit Discussion


On Friday, September 23, 2016 at 5:53:09 PM UTC+8, Hugo Arès wrote:
push will fail but I am not sure that commit will not make it to the repository.

I am testing the fix now, I will upload the change shortly.
 
great :-) 

Hugo Arès

unread,
Sep 23, 2016, 6:13:29 AM9/23/16
to Repo and Gerrit Discussion

Hugo Arès

unread,
Sep 23, 2016, 7:29:43 AM9/23/16
to Repo and Gerrit Discussion
Fix is merged, you need to build tip of stable-2.13 branch if you want to get it before 2.13.1 is released.

Makson Lee

unread,
Sep 23, 2016, 9:34:44 AM9/23/16
to Repo and Gerrit Discussion


On Friday, September 23, 2016 at 7:29:43 PM UTC+8, Hugo Arès wrote:
Fix is merged, you need to build tip of stable-2.13 branch if you want to get it before 2.13.1 is released.
 
i have downloaded and tested [1], we can clone from slave without any problem now, thank for your help.

Reply all
Reply to author
Forward
0 new messages