using ssh agent and github SCM

[Jurgen Weber]

Can you use the ssh-agent plugin + github SCM using deploy keys/ssh?

I have been trying all day, Jenkins 2.8.3, 

Plugins:

      - build-token-root:1.4

      - credentials-binding:1.13

      - docker-commons:1.8

      - docker-build-step:1.43

      - Exclusion:0.12

      - git:3.6.0

      - git-client:2.5.0

      - github:1.28.0

      - google-login:1.3

      - kubernetes:1.0

      - preSCMbuildstep:0.3

      - promoted-builds:2.29.1

      - s3:0.10.12

      - ssh:2.5

      - ssh-agent:1.15

      - ssh-credentials:1.13

      - workflow-aggregator:2.5

      - workflow-job:2.14.1

I have the credntial parameter setup:

http://take.ms/bVKHr

the ssh-agent config setup in the job:

http://take.ms/vx8cf

but if github SCM is setup, it fails with failed credentials, if I leave the SCM out and get to a build.. if I `ssh-add -l` it has no entities.... if I logon to the slave... it has no entities.... any ideas?

[Jurgen Weber]

I am thinking I am hitting this; https://issues.jenkins-ci.org/browse/JENKINS-24750

once I went 'ignoremising' for the ssh agent plugin..

java.io.IOException: [ssh-agent] Could not find specified credentials

So the parameter expression option for ssh-agent is not finding the repo_name_deploy_key parameter which holds the name of the global security ssh private key to use.

[Jurgen Weber]

I have confirmed this, if I hard code the value and not use the credential parameter/parameter expression it all works fine.

Now if only I could use my jenkins.io account so I could make an issue. haha.

[Jurgen Weber]

ok, so.... What I have discovered.

If I manually in the UI, start the job and add the parameters... my build works, ssh-agent works, it finds the key, loads it and github SCM works.

If I run the job, with the same parameter values from curl/using the command line..... it does not work and can not find the key.

Build run from curl:

Building remotely on jenkins-slave-rn7w5 (jenkins-jenkins-slave) in workspace /home/jenkins/workspace/core-build
Running Prebuild steps
[core-build] $ /bin/bash -xe /tmp/jenkins3412092671259899558.sh
+ ssh-keyscan github.com
# github.com:22 SSH-2.0-libssh_0.7.0
# github.com:22 SSH-2.0-libssh_0.7.0
# github.com:22 SSH-2.0-libssh_0.7.0
+ ssh-keygen -lf githubKey
2048 SHA256:nThbg6kXUpJWGl7E1IGOCspRomTxdCARLviKw6E5SY8 github.com (RSA)
+ mkdir -p /home/jenkins/.ssh
+ cp githubKey /home/jenkins/.ssh/known_hosts
+ chmod 400 /home/jenkins/.ssh/known_hosts
+ cp -r /home/jenkins/.ssh /root/
Success build forhudson.tasks.Shell@4efa51a
FATAL: 
java.io.IOException: [ssh-agent] Could not find specified credentials
        at com.cloudbees.jenkins.plugins.sshagent.SSHAgentBuildWrapper.preCheckout(SSHAgentBuildWrapper.java:209)
        at jenkins.scm.SCMCheckoutStrategy.preCheckout(SCMCheckoutStrategy.java:76)
        at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:490)
        at hudson.model.Run.execute(Run.java:1724)
        at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
        at hudson.model.ResourceController.execute(ResourceController.java:97)
        at hudson.model.Executor.run(Executor.java:421)
FATAL: [ssh-agent] Could not find specified credentials
java.io.IOException: [ssh-agent] Could not find specified credentials
        at com.cloudbees.jenkins.plugins.sshagent.SSHAgentBuildWrapper.preCheckout(SSHAgentBuildWrapper.java:209)
        at jenkins.scm.SCMCheckoutStrategy.preCheckout(SCMCheckoutStrategy.java:76)
        at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:490)
        at hudson.model.Run.execute(Run.java:1724)
        at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
        at hudson.model.ResourceController.execute(ResourceController.java:97)
        at hudson.model.Executor.run(Executor.java:421)
Finished: FAILURE

Build where I manually run it in the UI:

Started by user Jurgen Weber
Building remotely on jenkins-slave-lxlqs (jenkins-jenkins-slave) in workspace /home/jenkins/workspace/core-build
Running Prebuild steps
[core-build] $ /bin/bash -xe /tmp/jenkins9161791465985411390.sh
+ ssh-keyscan github.com
# github.com:22 SSH-2.0-libssh_0.7.0
# github.com:22 SSH-2.0-libssh_0.7.0
# github.com:22 SSH-2.0-libssh_0.7.0
+ ssh-keygen -lf githubKey
2048 SHA256:nThbg6kXUpJWGl7E1IGOCspRomTxdCARLviKw6E5SY8 github.com (RSA)
+ mkdir -p /home/jenkins/.ssh
+ cp githubKey /home/jenkins/.ssh/known_hosts
+ chmod 400 /home/jenkins/.ssh/known_hosts
+ cp -r /home/jenkins/.ssh /root/
Success build forhudson.tasks.Shell@66cf5060
[ssh-agent] Looking for ssh-agent implementation...
[ssh-agent]   Exec ssh-agent (binary ssh-agent on a remote machine)
$ ssh-agent
SSH_AUTH_SOCK=/tmp/ssh-zU37OMCq2Skj/agent.127
SSH_AGENT_PID=129
[ssh-agent] Started.
$ ssh-add /home/jenkins/workspace/core-build@tmp/private_key_7170551602733961435.key
Identity added: /home/jenkins/workspace/core-build@tmp/private_key_7170551602733961435.key (/home/jenkins/workspace/core-build@tmp/private_key_7170551602733961435.key)
[ssh-agent] Using credentials git (trench-deploy-key)
Wiping out workspace first.
Cloning the remote Git repository
......................

I have tried varying formats of the curl with the same result:

curl -i -X POST https://jenkins:8080/buildByToken/buildWithParameters --data token=MYTOKEN --data job=core-build --data repo_name_deploy_key="MYREPO-deploy-key" --data repo_name=MYREPO --data repo_http_url=https://github.com/EXAMPLE/MYREPO --data repo_ssh_url=g...@github.com:EXAMPLE/MYREPO.git

[Jurgen Weber]

https://issues.jenkins-ci.org/browse/JENKINS-47463

[Slide]

What does the http request look like when you run the curl command? Can you get a wire shark trace it something?

On Mon, Oct 16, 2017, 17:21 'Jurgen Weber' via Jenkins Users <jenkins...@googlegroups.com> wrote:

https://issues.jenkins-ci.org/browse/JENKINS-47463

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/ae25cb00-ef03-4efa-8b95-c398c56022c1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jurgen Weber]

Here are some http requests:

[ingress-nginx-2796083105-n5qcr] 202.10.91.226 - [202.10.91.226] - - [17/Oct/2017:22:54:15 +0000] "POST /buildByToken/buildWithParameters?token=TOKENi&job=core-build&repo_name_deploy_key=trench-deploy-key&repo_name=trench&repo_http_url=https://github.com/example/trench&repo_ssh_url=g...@github.com:example/trench.git HTTP/1.1" 201 5 "-" "curl/7.54.0" 324 0.006 [devops-jenkins-jenkins-8080] 100.96.1.26:8080 0 0.006 201

[ingress-nginx-2796083105-trbzt] 202.10.91.226 - [202.10.91.226] - - [17/Oct/2017:22:54:33 +0000] "POST /buildByToken/buildWithParameters?token=TOKEN&job=core-build&repo_name_deploy_key=trench-deploy-key&repo_name=trench&repo_http_url=https://github.com/example/trench&repo_ssh_url=g...@github.com:example/trench.git HTTP/1.1" 302 5 "-" "curl/7.54.0" 324 0.003 [devops-jenkins-jenkins-8080] 100.96.1.26:8080 0 0.003 302

[ingress-nginx-2796083105-n5qcr] 202.10.91.226 - [202.10.91.226] - - [17/Oct/2017:22:54:35 +0000] "POST /buildByToken/buildWithParameters?token=TOKEN&job=core-build&repo_name_deploy_key=trench-deploy-key&repo_name=trench&repo_http_url=https://github.com/example/trench&repo_ssh_url=g...@github.com:example/trench.git HTTP/1.1" 302 5 "-" "curl/7.54.0" 324 0.003 [devops-jenkins-jenkins-8080] 100.96.1.26:8080 0 0.003 302

[ingress-nginx-2796083105-trbzt] 202.10.91.226 - [202.10.91.226] - - [17/Oct/2017:22:54:35 +0000] "POST /buildByToken/buildWithParameters?token=TOKEN&job=core-build&repo_name_deploy_key=trench-deploy-key&repo_name=trench&repo_http_url=https://github.com/example/trench&repo_ssh_url=g...@github.com:example/trench.git HTTP/1.1" 302 5 "-" "curl/7.54.0" 324 0.002 [devops-jenkins-jenkins-8080] 100.96.1.26:8080 0 0.002 302

[ingress-nginx-2796083105-trbzt] 202.10.91.226 - [202.10.91.226] - - [17/Oct/2017:22:54:35 +0000] "POST /buildByToken/buildWithParameters?token=TOKEN&job=core-build&repo_name_deploy_key=trench-deploy-key&repo_name=trench&repo_http_url=https://github.com/example/trench&repo_ssh_url=g...@github.com:example/trench.git HTTP/1.1" 302 5 "-" "curl/7.54.0" 324 0.002 [devops-jenkins-jenkins-8080] 100.96.1.26:8080 0 0.002 302

[ingress-nginx-2796083105-trbzt] 202.10.91.226 - [202.10.91.226] - - [17/Oct/2017:22:54:48 +0000] "POST /buildByToken/buildWithParameters?token=TOKEN&job=core-build&repo_name_deploy_key=trench-deploy-key&repo_name=trench&repo_http_url=https://github.com/example/trench&repo_ssh_url=g...@github.com:example/trench.git HTTP/1.1" 302 5 "-" "curl/7.54.0" 324 0.002 [devops-jenkins-jenkins-8080] 100.96.1.26:8080 0 0.002 302

A tcpdump -vvv port 8080

POST /buildByToken/buildWithParameters?token=TOKEN&job=core-build&repo_name_deploy_key=trench-deploy-key&repo_name=trench&repo_http_url=https://github.com/example/trench&repo_ssh_url=g...@github.com:example/trench.git HTTP/1.1

Host: jenkins.k8s.hipages.com.au

Connection: close

X-Real-IP: XXX

X-Forwarded-For: XXX

X-Forwarded-Host: jenkins

X-Forwarded-Port: 443

X-Forwarded-Proto: https

X-Original-URI: /buildByToken/buildWithParameters?token=TOKEN&job=core-build&repo_name_deploy_key=trench-deploy-key&repo_name=trench&repo_http_url=https://github.com/example/trench&repo_ssh_url=g...@github.com:example/trench.git

X-Scheme: https

User-Agent: curl/7.54.0

Accept: */*

[Jurgen Weber]

issue link: https://issues.jenkins-ci.org/browse/JENKINS-47463

[Jurgen Weber]

My final solution has been to bypass ssh-agent and not use it, since it does not work.

I have instead found a way using the Jenkins Script console (https://wiki.jenkins.io/display/JENKINS/Jenkins+Script+Console) to retrieve the private key and copy it to ~/.ssh/id_rsa and the SCM then works.

This works since I am running this in kubernetes and each pod is destroyed after its build.

On Friday, 20 October 2017 10:28:41 UTC+11, Jurgen Weber wrote:

issue link: https://issues.jenkins-ci.org/browse/JENKINS-47463