Build fails on webhook trigger but succeeds when triggered manually

[Ross Bender]

I have a build job configured for "GitHub hook trigger for GITScm polling". The build succeeds when triggered manually, but always fails when it is triggered via Github webhook!

Additionally, the other strange thing is I have a different build job pointing to the same repository which also is configured to be triggered on webhooks. However, this job doesn't have the same issue! It always succeeds whether triggered manually or through webhook.

The Github hook log on failing job shows the following:

Started on Nov 25, 2020 1:32:53 PM
Started by event from 192.168.51.102 ? https://build.mycompany.org/github-webhook/ on Wed Nov 25 13:32:50 CST 2020
Using strategy: Default
[poll] Last Built Revision: Revision fe90cf7fdeb7ba612241e5db5b938eeed8dc0aa7 (origin/master)
The recommended git tool is: NONE
using credential b6baca1a-e93c-48d5-bdd8-1035c64a3226
 > git --version # timeout=10
 > git --version # timeout=10
using GIT_SSH to set credentials Github
ERROR: Failed to record SCM polling
java.lang.RuntimeException: ssh executable not found. The git plugin only supports official git client http://git-scm.com/download/win
    at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.getSSHExecutable(CliGitAPIImpl.java:2319)
    at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.createWindowsGitSSH(CliGitAPIImpl.java:2325)
    at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1989)
    at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1951)
    at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1942)
    at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.getHeadRev(CliGitAPIImpl.java:3325)
    at hudson.plugins.git.GitSCM.compareRemoteRevisionWithImpl(GitSCM.java:718)
    at hudson.plugins.git.GitSCM.compareRemoteRevisionWith(GitSCM.java:677)
    at hudson.scm.SCM.compareRemoteRevisionWith(SCM.java:401)
    at hudson.scm.SCM.poll(SCM.java:418)
    at hudson.model.AbstractProject._poll(AbstractProject.java:1388)
    at hudson.model.AbstractProject.poll(AbstractProject.java:1291)
    at com.cloudbees.jenkins.GitHubPushTrigger$1.runPolling(GitHubPushTrigger.java:124)
    at com.cloudbees.jenkins.GitHubPushTrigger$1.run(GitHubPushTrigger.java:150)
    at hudson.util.SequentialExecutionQueue$QueueEntry.run(SequentialExecutionQueue.java:119)
    at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
    at java.util.concurrent.FutureTask.run(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

Both build jobs are running on a Linux Jenkins slave (master is Windows). We are using Jenkins 2.243 with the latest plugins (updated today).

Can anyone advise? Thank you!

[Mark Waite]

The hook is evaluated on the Jenkins controller.  Since you indicate that it is running on Windows, then you'll need command line git installed on your Jenkins controller.  We test regularly with Git for Windows using either the standard installer or the PortableGit installer.  Be sure that the location where command line git is being found has a copy of ssh.exe available in the bin directory adjacent to the location where git is located

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/1252faa7-9357-4d49-b5df-0d0b8d7520cen%40googlegroups.com.

[Ross Bender]

Thanks for the reply Mark.

We do have git installed/configured on the controller and the hook does succeed for some builds, but not all. The hook succeeds for other builds that also are configured to run on the slave and builds that run on the controller/master. We have only started seeing this problem with this particular type of build which is configured to run on a slave node (freestyle build with label).

In fact, for this particular hook, I can see the git verbose log show one build trigger successfully and the other fail. Both builds run on the slave and use same repo. I can't find any significant differences in those builds: besides a few additional build steps and one using git publisher as a post-build action, they have the same git executable configured (and as you mentioned, shouldn't apply for hook).

[Ross Bender]

I've noticed the same happening every time. When I push to Github:

• Job 1 successfully receives webhook and build is triggered.
• Job 2 errors receiving webhook and build is not triggered. Github hook log shows "ERROR: Failed to record SCM polling. java.lang.RuntimeException: ssh executable not found."

Both jobs are configured to run on the same build node. They both specify the same Git executable in Job config.

I tried disabling Job 1 and doing another push, but Job 2 still errors receiving the webhook.

Any recommendations or pointers on how I can troubleshoot the issue?

[Ross Bender]

Can anyone provide advice on how I can troubleshoot this issue further? I'm not able to log into JIRA so I'm not able to create an issue and communicate there.

I'm willing to dive deeper into the issue but am unsure how/where to start. Thanks!

[Ross Bender]

I've found if I use jgit instead of a system installed git, the webhook is working as expected

We did see a different issue where git publisher failed when trying to push tags during post-build:

The recommended git tool is: NONE

using credential xxx
Pushing tag v25.34 to repo origin
RefSpec is "refs/tags/v25.34:refs/heads/v25.34".
Opening connection
Counting objects
Finding sources
Writing objects
ERROR: Failed to push tag v25.34 to origin
hudson.plugins.git.GitException: failed REJECTED_OTHER_REASON for 'refs/tags/v25.34:refs/heads/v25.34' refspec 'v25.34' to git@xxx:team/app.git
    at org.jenkinsci.plugins.gitclient.JGitAPIImpl$8.execute(JGitAPIImpl.java:1887)
    at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:161)

It seems like this is a separate issue (JENKINS-37120), but I was able to use the suggested workaround and get it working (specify full tag path like refs/tags/xxx).

Summary:

* using jgit instead of system installed git

* both webhook and git publisher are working