using ssh agent and github SCM
794 views
Skip to first unread message
Jurgen Weber
Oct 12, 2017, 5:41:14 AM10/12/17
to Jenkins Users
Can you use the ssh-agent plugin + github SCM using deploy keys/ssh?
I have been trying all day, Jenkins 2.8.3,
Plugins:
- build-token-root:1.4
- credentials-binding:1.13
- docker-commons:1.8
- docker-build-step:1.43
- Exclusion:0.12
- git:3.6.0
- git-client:2.5.0
- github:1.28.0
- google-login:1.3
- kubernetes:1.0
- preSCMbuildstep:0.3
- promoted-builds:2.29.1
- s3:0.10.12
- ssh:2.5
- ssh-agent:1.15
- ssh-credentials:1.13
- workflow-aggregator:2.5
- workflow-job:2.14.1
I have the credntial parameter setup:
the ssh-agent config setup in the job:
but if github SCM is setup, it fails with failed credentials, if I leave the SCM out and get to a build.. if I `ssh-add -l` it has no entities.... if I logon to the slave... it has no entities.... any ideas?
Jurgen Weber
Oct 12, 2017, 6:54:55 AM10/12/17
to Jenkins Users
I am thinking I am hitting this; https://issues.jenkins-ci.org/browse/JENKINS-24750
once I went 'ignoremising' for the ssh agent plugin..
java.io.IOException: [ssh-agent] Could not find specified credentials
So the parameter expression option for ssh-agent is not finding the repo_name_deploy_key parameter which holds the name of the global security ssh private key to use.
Jurgen Weber
Oct 12, 2017, 6:49:05 PM10/12/17
to Jenkins Users
I have confirmed this, if I hard code the value and not use the credential parameter/parameter expression it all works fine.
Now if only I could use my jenkins.io account so I could make an issue. haha.
Jurgen Weber
Oct 16, 2017, 7:38:56 PM10/16/17
to Jenkins Users
ok, so.... What I have discovered.
If I manually in the UI, start the job and add the parameters... my build works, ssh-agent works, it finds the key, loads it and github SCM works.
If I run the job, with the same parameter values from curl/using the command line..... it does not work and can not find the key.
Build run from curl:
Building remotely on jenkins-slave-rn7w5 (jenkins-jenkins-slave) in workspace /home/jenkins/workspace/core-build
Running Prebuild steps
[core-build] $ /bin/bash -xe /tmp/jenkins3412092671259899558.sh
+ ssh-keyscan github.com
# github.com:22 SSH-2.0-libssh_0.7.0
# github.com:22 SSH-2.0-libssh_0.7.0
# github.com:22 SSH-2.0-libssh_0.7.0
+ ssh-keygen -lf githubKey
2048 SHA256:nThbg6kXUpJWGl7E1IGOCspRomTxdCARLviKw6E5SY8 github.com (RSA)
+ mkdir -p /home/jenkins/.ssh
+ cp githubKey /home/jenkins/.ssh/known_hosts
+ chmod 400 /home/jenkins/.ssh/known_hosts
+ cp -r /home/jenkins/.ssh /root/
Success build forhudson.tasks.Shell@4efa51a
FATAL:
java.io.IOException: [ssh-agent] Could not find specified credentials
at com.cloudbees.jenkins.plugins.sshagent.SSHAgentBuildWrapper.preCheckout(SSHAgentBuildWrapper.java:209)
at jenkins.scm.SCMCheckoutStrategy.preCheckout(SCMCheckoutStrategy.java:76)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:490)
at hudson.model.Run.execute(Run.java:1724)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:421)
FATAL: [ssh-agent] Could not find specified credentials
java.io.IOException: [ssh-agent] Could not find specified credentials
at com.cloudbees.jenkins.plugins.sshagent.SSHAgentBuildWrapper.preCheckout(SSHAgentBuildWrapper.java:209)
at jenkins.scm.SCMCheckoutStrategy.preCheckout(SCMCheckoutStrategy.java:76)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:490)
at hudson.model.Run.execute(Run.java:1724)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:421)
Finished: FAILURE
Build where I manually run it in the UI:
Started by user Jurgen Weber
Building remotely on jenkins-slave-lxlqs (jenkins-jenkins-slave) in workspace /home/jenkins/workspace/core-build
Running Prebuild steps
[core-build] $ /bin/bash -xe /tmp/jenkins9161791465985411390.sh
+ ssh-keyscan github.com
# github.com:22 SSH-2.0-libssh_0.7.0
# github.com:22 SSH-2.0-libssh_0.7.0
# github.com:22 SSH-2.0-libssh_0.7.0
+ ssh-keygen -lf githubKey
2048 SHA256:nThbg6kXUpJWGl7E1IGOCspRomTxdCARLviKw6E5SY8 github.com (RSA)
+ mkdir -p /home/jenkins/.ssh
+ cp githubKey /home/jenkins/.ssh/known_hosts
+ chmod 400 /home/jenkins/.ssh/known_hosts
+ cp -r /home/jenkins/.ssh /root/
Success build forhudson.tasks.Shell@66cf5060
[ssh-agent] Looking for ssh-agent implementation...
[ssh-agent] Exec ssh-agent (binary ssh-agent on a remote machine)
$ ssh-agent
SSH_AUTH_SOCK=/tmp/ssh-zU37OMCq2Skj/agent.127
SSH_AGENT_PID=129
[ssh-agent] Started.
$ ssh-add /home/jenkins/workspace/core-build@tmp/private_key_7170551602733961435.key
Identity added: /home/jenkins/workspace/core-build@tmp/private_key_7170551602733961435.key (/home/jenkins/workspace/core-build@tmp/private_key_7170551602733961435.key)
[ssh-agent] Using credentials git (trench-deploy-key)
Wiping out workspace first.
Cloning the remote Git repository
......................
I have tried varying formats of the curl with the same result:
curl -i -X POST https://jenkins:8080/buildByToken/buildWithParameters --data token=MYTOKEN --data job=core-build --data repo_name_deploy_key="MYREPO-deploy-key" --data repo_name=MYREPO --data repo_http_url=https://github.com/EXAMPLE/MYREPO --data repo_ssh_url=g...@github.com:EXAMPLE/MYREPO.git
Jurgen Weber
Oct 16, 2017, 8:21:19 PM10/16/17
to Jenkins Users
Slide
Oct 17, 2017, 8:12:58 AM10/17/17
to jenkins...@googlegroups.com
What does the http request look like when you run the curl command? Can you get a wire shark trace it something?
On Mon, Oct 16, 2017, 17:21 'Jurgen Weber' via Jenkins Users <jenkins...@googlegroups.com> wrote:
--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/ae25cb00-ef03-4efa-8b95-c398c56022c1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Jurgen Weber
Oct 17, 2017, 7:03:41 PM10/17/17
to Jenkins Users
Here are some http requests:
A tcpdump -vvv port 8080
[ingress-nginx-2796083105-n5qcr] 202.10.91.226 - [202.10.91.226] - - [17/Oct/2017:22:54:15 +0000] "POST /buildByToken/buildWithParameters?token=TOKENi&job=core-build&repo_name_deploy_key=trench-deploy-key&repo_name=trench&repo_http_url=https://github.com/example/trench&repo_ssh_url=g...@github.com:example/trench.git HTTP/1.1" 201 5 "-" "curl/7.54.0" 324 0.006 [devops-jenkins-jenkins-8080] 100.96.1.26:8080 0 0.006 201
[ingress-nginx-2796083105-trbzt] 202.10.91.226 - [202.10.91.226] - - [17/Oct/2017:22:54:33 +0000] "POST /buildByToken/buildWithParameters?token=TOKEN&job=core-build&repo_name_deploy_key=trench-deploy-key&repo_name=trench&repo_http_url=https://github.com/example/trench&repo_ssh_url=g...@github.com:example/trench.git HTTP/1.1" 302 5 "-" "curl/7.54.0" 324 0.003 [devops-jenkins-jenkins-8080] 100.96.1.26:8080 0 0.003 302
[ingress-nginx-2796083105-n5qcr] 202.10.91.226 - [202.10.91.226] - - [17/Oct/2017:22:54:35 +0000] "POST /buildByToken/buildWithParameters?token=TOKEN&job=core-build&repo_name_deploy_key=trench-deploy-key&repo_name=trench&repo_http_url=https://github.com/example/trench&repo_ssh_url=g...@github.com:example/trench.git HTTP/1.1" 302 5 "-" "curl/7.54.0" 324 0.003 [devops-jenkins-jenkins-8080] 100.96.1.26:8080 0 0.003 302[ingress-nginx-2796083105-trbzt] 202.10.91.226 - [202.10.91.226] - - [17/Oct/2017:22:54:35 +0000] "POST /buildByToken/buildWithParameters?token=TOKEN&job=core-build&repo_name_deploy_key=trench-deploy-key&repo_name=trench&repo_http_url=https://github.com/example/trench&repo_ssh_url=g...@github.com:example/trench.git HTTP/1.1" 302 5 "-" "curl/7.54.0" 324 0.002 [devops-jenkins-jenkins-8080] 100.96.1.26:8080 0 0.002 302[ingress-nginx-2796083105-trbzt] 202.10.91.226 - [202.10.91.226] - - [17/Oct/2017:22:54:35 +0000] "POST /buildByToken/buildWithParameters?token=TOKEN&job=core-build&repo_name_deploy_key=trench-deploy-key&repo_name=trench&repo_http_url=https://github.com/example/trench&repo_ssh_url=g...@github.com:example/trench.git HTTP/1.1" 302 5 "-" "curl/7.54.0" 324 0.002 [devops-jenkins-jenkins-8080] 100.96.1.26:8080 0 0.002 302
[ingress-nginx-2796083105-trbzt] 202.10.91.226 - [202.10.91.226] - - [17/Oct/2017:22:54:48 +0000] "POST /buildByToken/buildWithParameters?token=TOKEN&job=core-build&repo_name_deploy_key=trench-deploy-key&repo_name=trench&repo_http_url=https://github.com/example/trench&repo_ssh_url=g...@github.com:example/trench.git HTTP/1.1" 302 5 "-" "curl/7.54.0" 324 0.002 [devops-jenkins-jenkins-8080] 100.96.1.26:8080 0 0.002 302A tcpdump -vvv port 8080
POST /buildByToken/buildWithParameters?token=TOKEN&job=core-build&repo_name_deploy_key=trench-deploy-key&repo_name=trench&repo_http_url=https://github.com/example/trench&repo_ssh_url=g...@github.com:example/trench.git HTTP/1.1 Connection: close X-Real-IP: XXX X-Forwarded-For: XXX X-Forwarded-Host: jenkins X-Forwarded-Port: 443 X-Forwarded-Proto: https X-Original-URI: /buildByToken/buildWithParameters?token=TOKEN&job=core-build&repo_name_deploy_key=trench-deploy-key&repo_name=trench&repo_http_url=https://github.com/example/trench&repo_ssh_url=g...@github.com:example/trench.git X-Scheme: https User-Agent: curl/7.54.0 Accept: */*Jurgen Weber
Oct 19, 2017, 7:28:41 PM10/19/17
to Jenkins Users
Jurgen Weber
Oct 22, 2017, 9:47:07 PM10/22/17
to Jenkins Users
My final solution has been to bypass ssh-agent and not use it, since it does not work.
I have instead found a way using the Jenkins Script console (https://wiki.jenkins.io/display/JENKINS/Jenkins+Script+Console) to retrieve the private key and copy it to ~/.ssh/id_rsa and the SCM then works.
This works since I am running this in kubernetes and each pod is destroyed after its build.
On Friday, 20 October 2017 10:28:41 UTC+11, Jurgen Weber wrote:
This works since I am running this in kubernetes and each pod is destroyed after its build.
On Friday, 20 October 2017 10:28:41 UTC+11, Jurgen Weber wrote:
Reply all
Reply to author
Forward
0 new messages