Git Polling initiated by notifyCommit failing on remote since version 2.4.4

Michael Giroux

unread,

Mar 29, 2016, 2:44:15 PM3/29/16

to Jenkins Users

Since updating to Git plugin version 2.4.4, our jobs that execute on a remote slave are failing.  The full stack trace from the polling log is below.

Do we need to install additional software on the slaves to accommodate the update?

Michael Giroux

Polling for changes in
ERROR: Failed to record SCM polling for hudson.model.FreeStyleProject@22b7f935[CHCOM WEB Replatform - Pull Requests]
hudson.remoting.RemotingSystemException: java.io.IOException: Remote call on aws-slave-002-v2 failed
	at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:266)
	at com.sun.proxy.$Proxy79.getRemoteBranches(Unknown Source)
	at org.jenkinsci.plugins.gitclient.RemoteGitImpl.getRemoteBranches(RemoteGitImpl.java:485)
	at hudson.plugins.git.util.GitUtils.getAllBranchRevisions(GitUtils.java:47)
	at hudson.plugins.git.util.DefaultBuildChooser.getAdvancedCandidateRevisions(DefaultBuildChooser.java:204)
	at hudson.plugins.git.util.DefaultBuildChooser.getCandidateRevisions(DefaultBuildChooser.java:53)
	at hudson.plugins.git.GitSCM.compareRemoteRevisionWithImpl(GitSCM.java:706)
	at hudson.plugins.git.GitSCM.compareRemoteRevisionWith(GitSCM.java:571)
	at hudson.scm.SCM.compareRemoteRevisionWith(SCM.java:380)
	at hudson.scm.SCM.poll(SCM.java:397)
	at hudson.model.AbstractProject.pollWithWorkspace(AbstractProject.java:1471)
	at hudson.model.AbstractProject._poll(AbstractProject.java:1442)
	at hudson.model.AbstractProject.poll(AbstractProject.java:1353)
	at hudson.triggers.SCMTrigger$Runner.runPolling(SCMTrigger.java:510)
	at hudson.triggers.SCMTrigger$Runner.run(SCMTrigger.java:539)
	at hudson.util.SequentialExecutionQueue$QueueEntry.run(SequentialExecutionQueue.java:118)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
	at java.lang.Thread.run(Thread.java:682)
Caused by: java.io.IOException: Remote call on aws-slave-002-v2 failed
	at hudson.remoting.Channel.call(Channel.java:786)
	at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:250)
	... 21 more
Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.eclipse.jgit.util.FS
	at org.eclipse.jgit.lib.BaseRepositoryBuilder.setupWorkTree(BaseRepositoryBuilder.java:620)
	at org.eclipse.jgit.lib.BaseRepositoryBuilder.setup(BaseRepositoryBuilder.java:556)
	at org.eclipse.jgit.storage.file.FileRepositoryBuilder.build(FileRepositoryBuilder.java:92)
	at org.eclipse.jgit.storage.file.FileRepositoryBuilder.create(FileRepositoryBuilder.java:110)
	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.getRepository(CliGitAPIImpl.java:2312)
	at hudson.plugins.git.GitAPI.getRepository(GitAPI.java:269)
	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.getRemoteBranches(CliGitAPIImpl.java:1858)
	at hudson.plugins.git.GitAPI.getRemoteBranches(GitAPI.java:212)
	at sun.reflect.GeneratedMethodAccessor60.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:483)
	at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:608)
	at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:583)
	at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:542)
	at hudson.remoting.UserRequest.perform(UserRequest.java:121)
	at hudson.remoting.UserRequest.perform(UserRequest.java:49)
	at hudson.remoting.Request$2.run(Request.java:326)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
	at ......remote call to aws-slave-002-v2(Native Method)
	at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1413)
	at hudson.remoting.UserResponse.retrieve(UserRequest.java:221)
	at hudson.remoting.Channel.call(Channel.java:778)
	... 22 more

Michael Giroux

unread,

Mar 31, 2016, 12:33:21 PM3/31/16

to Jenkins Users

Seems another user has similar issue. https://issues.jenkins-ci.org/browse/JENKINS-33907

That is not exactly the same, but I've seen the same issue, and it could be related to my issue.

Mark Waite

unread,

Mar 31, 2016, 2:18:54 PM3/31/16

to Jenkins Users

I don't know how to duplicate either your issue or JENKINS-33907. I installed a Jenkins 1.609.3 last night with Java 6, then used a slave with Java 7, and confirmed that I was able to checkout on the master and the slave using the command line git implementation with git plugin 2.4.4 and git client plugin 1.19.6.

I confirmed that the referenced class is included in one of the jar files packaged with git client plugin 1.19.6.

Have you been able to duplicate the problem on a fresh installation?

Mark Waite

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/f0fb14b5-83a7-4236-9da9-2deb7d01e80e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Michael Giroux

unread,

Mar 31, 2016, 3:02:19 PM3/31/16

to Jenkins Users

Here are a few more facts that might give some clues.

Master is running Jenkins 1.609.3 in Oracle JDK 1.6.0_75

Slave is Linux running Oracle JDK 1.8.0_20

Git Plugin version 2.4.0

Git Client version 1.19.6

I modified the job configuration to restrict execution to a different slave (one that it had not run on previously) . Then attempted to build the job manually. Result was a slightly different exception (see stack). The interesting point here is the class version issue from the root cause frame.

Caused by: java.lang.UnsupportedClassVersionError: org/eclipse/jgit/util/Java7FSFactory : Unsupported major.minor version 51.0

This suggests that something is compiled with JDK 1.7 and being executed on the master which is JDK 1.6.  I'm guessing the git client plugin.

Will keep digging.

FATAL: java.io.IOException: Remote call on aws-slave-003-v2 failed
hudson.remoting.RemotingSystemException: java.io.IOException: Remote call on aws-slave-003-v2 failed
	at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:266)
	at com.sun.proxy.$Proxy74.getRemoteBranches(Unknown Source)
	at org.jenkinsci.plugins.gitclient.RemoteGitImpl.getRemoteBranches(RemoteGitImpl.java:485)
	at hudson.plugins.git.util.GitUtils.getAllBranchRevisions(GitUtils.java:47)
	at hudson.plugins.git.util.DefaultBuildChooser.getAdvancedCandidateRevisions(DefaultBuildChooser.java:204)
	at hudson.plugins.git.util.DefaultBuildChooser.getCandidateRevisions(DefaultBuildChooser.java:53)
	at hudson.plugins.git.GitSCM.determineRevisionToBuild(GitSCM.java:941)
	at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1044)
	at hudson.scm.SCM.checkout(SCM.java:484)
	at hudson.model.AbstractProject.checkout(AbstractProject.java:1274)
	at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:609)
	at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:531)
	at hudson.model.Run.execute(Run.java:1738)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:98)
	at hudson.model.Executor.run(Executor.java:381)
Caused by: java.io.IOException: Remote call on aws-slave-003-v2 failed
	at hudson.remoting.Channel.call(Channel.java:786)
	at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:250)
	... 16 more
Caused by: java.lang.LinkageError: Failed to load org.eclipse.jgit.util.Java7FSFactory
	at hudson.remoting.RemoteClassLoader$ClassLoaderProxy.fetch4(RemoteClassLoader.java:763)
	at hudson.remoting.RemoteClassLoader$ClassLoaderProxy.fetch3(RemoteClassLoader.java:803)
	at sun.reflect.GeneratedMethodAccessor71.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:608)
	at hudson.remoting.Request$2.run(Request.java:326)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
	at org.jenkinsci.remoting.CallableDecorator.call(CallableDecorator.java:18)
	at hudson.remoting.CallableDecoratorList$1.call(CallableDecoratorList.java:21)
	at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
	at java.lang.Thread.run(Thread.java:682)
	at ......remote call to channel(Native Method)
	at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1413)
	at hudson.remoting.Request.call(Request.java:172)
	at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:253)
	at com.sun.proxy.$Proxy5.fetch3(Unknown Source)
	at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:171)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:259)
	at org.eclipse.jgit.util.FS.detect(FS.java:152)
	at org.eclipse.jgit.util.FS.detect(FS.java:125)
	at org.eclipse.jgit.util.FS.<clinit>(FS.java:115)
	at org.eclipse.jgit.lib.BaseRepositoryBuilder.setupWorkTree(BaseRepositoryBuilder.java:620)
	at org.eclipse.jgit.lib.BaseRepositoryBuilder.setup(BaseRepositoryBuilder.java:556)
	at org.eclipse.jgit.storage.file.FileRepositoryBuilder.build(FileRepositoryBuilder.java:92)
	at org.eclipse.jgit.storage.file.FileRepositoryBuilder.create(FileRepositoryBuilder.java:110)
	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.getRepository(CliGitAPIImpl.java:2312)
	at hudson.plugins.git.GitAPI.getRepository(GitAPI.java:269)
	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.getRemoteBranches(CliGitAPIImpl.java:1858)
	at hudson.plugins.git.GitAPI.getRemoteBranches(GitAPI.java:212)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:483)
	at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:608)
	at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:583)
	at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:542)
	at hudson.remoting.UserRequest.perform(UserRequest.java:121)
	at hudson.remoting.UserRequest.perform(UserRequest.java:49)
	at hudson.remoting.Request$2.run(Request.java:326)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
	at ......remote call to aws-slave-003-v2(Native Method)
	at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1413)
	at hudson.remoting.UserResponse.retrieve(UserRequest.java:221)
	at hudson.remoting.Channel.call(Channel.java:778)
	... 17 more
Caused by: java.lang.UnsupportedClassVersionError: org/eclipse/jgit/util/Java7FSFactory : Unsupported major.minor version 51.0
	at java.lang.ClassLoader.defineClass1(Native Method)
	at java.lang.ClassLoader.defineClassCond(ClassLoader.java:637)
	at java.lang.ClassLoader.defineClass(ClassLoader.java:621)
	at jenkins.util.AntClassLoader.defineClassFromData(AntClassLoader.java:1138)
	at hudson.ClassicPluginStrategy$AntClassLoader2.defineClassFromData(ClassicPluginStrategy.java:799)
	at jenkins.util.AntClassLoader.getClassFromStream(AntClassLoader.java:1309)
	at jenkins.util.AntClassLoader.findClassInComponents(AntClassLoader.java:1365)
	at jenkins.util.AntClassLoader.findClass(AntClassLoader.java:1325)
	at jenkins.util.AntClassLoader.loadClass(AntClassLoader.java:1078)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
	at hudson.remoting.RemoteClassLoader$ClassLoaderProxy.fetch4(RemoteClassLoader.java:761)
	at hudson.remoting.RemoteClassLoader$ClassLoaderProxy.fetch3(RemoteClassLoader.java:803)
	at sun.reflect.GeneratedMethodAccessor71.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:608)
	at hudson.remoting.Request$2.run(Request.java:326)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
	at org.jenkinsci.remoting.CallableDecorator.call(CallableDecorator.java:18)
	at hudson.remoting.CallableDecoratorList$1.call(CallableDecoratorList.java:21)
	at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
	at java.lang.Thread.run(Thread.java:682)
[CHECKSTYLE] Skipping publisher since build result is FAILURE

Michael Giroux

unread,

Mar 31, 2016, 3:07:08 PM3/31/16

to Jenkins Users

In the interest of determining if the issue is related to the git client plugin, I would like to roll back to the previous version. We currently have 1.19.6 installed. Our previous version was 1.18.0.

Should this combination work? Git Plugin 2.4.0 (or 2.4.4) with Git Client 1.18.0.

Michael

On Thursday, March 31, 2016 at 11:18:54 AM UTC-7, Mark Waite wrote:

Michael Giroux

unread,

Mar 31, 2016, 3:22:27 PM3/31/16

to Jenkins Users

Reverted Git Client to 1.18.0 and problem is resolved. The notifyCommit no longer faults, and the job is running on the slave.

Current config: Git plugin 2.4.0, Git client 1.18.0.

Michael

On Thursday, March 31, 2016 at 11:18:54 AM UTC-7, Mark Waite wrote:

Mark Waite

unread,

Mar 31, 2016, 3:24:26 PM3/31/16

to Jenkins Users

Git plugin 2.4.0 requires at least git client plugin 1.18.0.

Git plugin 2.4.4 requires at least git client plugin 1.19.6.

Mark Waite

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/bf9f6ed5-5710-4fba-a283-16fa0d26e255%40googlegroups.com.

Michael

unread,

Mar 31, 2016, 3:32:28 PM3/31/16

to jenkins...@googlegroups.com

That is a shame. I wont be able to move up to Git plugin 2.4.4. until the git client issue is resolve. I realize you cannot reproduce, and I’m about to leave for two weeks of well deserved vacation, so I won’t be able to pick this up again for a while, but if someone else starts reporting issues, keep my results in mind.

Also, the git client seems to have one or more modules compiled with jdk 1.7. Perhaps one or more of the dependencies. Version 1.19.6 is throwing class version errors in a JDK 1.6 jenkins. I did not see any mention of this in docs, so I think this is an issue.

Michael

You received this message because you are subscribed to a topic in the Google Groups "Jenkins Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-users/XYr0XxjDMuE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAO49JtFzEeL7EFJxU%3DL3BvnCrBbBX7X4Tb%2B6q35xs79azVDobg%40mail.gmail.com.

Mark Waite

unread,

Mar 31, 2016, 3:43:07 PM3/31/16

to jenkins...@googlegroups.com

You're correct that git client plugin 1.19.6 includes a jar file that is compiled with JDK 7. The JGit support for symlinks requires it. However, that jar file should not be loaded if it is running on a Java 6 environment. The JGit implementation is designed to allow the Java 7 module to only be loaded if it detects it is running on JDK 7 or newer.

I definitely do not see class version errors in a JDK 1.6 environment. I configured a Docker instance yesterday with Java 6 master, Java 7 slaves, and confirmed that I could clone to both the master and the slaves using Git plugin 2.4.4 and Git client plugin 1.19.6.

See [https://github.com/jenkinsci/git-client-plugin/commit/cce0f8b30373ebce707aaf8457640cea7fe4489d] for the details

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/D8A87E5F-DE6E-43EE-99B3-3B770F2318FA%40gmail.com.

Michael Giroux

unread,

Mar 31, 2016, 4:06:14 PM3/31/16

to Jenkins Users

The git client 1.19.6 has been running since 3/22/2016 on the master without errors, so it seems the feature to detect the jdk works as desired, at least when run on the master. The class version issue only shows up when we configure the job to run on a slave, It is also interesting that the problem only shows up on the first execution on the slave. I got the error two ways:

1. deleted all jar files in .jenkins/cache/jars, then disabled and relaunched the slave. then kicked off a build. The first build got the class version error.

2. modified an existing job to build on a different slave, one it had not built on before. Again, the first build got the class version error.

For both cases, once the build had been attempted once, subsequent builds get the NoClassDefFoundError.

Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.eclipse.jgit.util.FS
	at org.eclipse.jgit.lib.BaseRepositoryBuilder.setupWorkTree(BaseRepositoryBuilder.java:620)
	at org.eclipse.jgit.lib.BaseRepositoryBuilder.setup(BaseRepositoryBuilder.java:556)
	at org.eclipse.jgit.sto

This leads me to think that for the first build, something is running on master that does not run for subsequent builds once the workspace has been created on the slave.  Just throwing that out since I have no other explanation for the class version errors.

Reply all

Reply to author

Forward