[JIRA] (JENKINS-37899) Git client does not call CredentialsProvider.snapshot() when adding credentials to a SmartCredentialsProvider that will be used on a remote instance

[stephenconnolly@java.net (JIRA)]

stephenconnolly created an issue

Jenkins / JENKINS-37899 Git client does not call CredentialsProvider.snapshot() when adding credentials to a SmartCredentialsProvider that will be used on a remote instance Issue Type: Bug Assignee: Mark Waite Components: git-client-plugin Created: 2016/Sep/01 2:57 PM Priority: Critical Reporter: stephenconnolly Some properties of a credential may need to be resolved at point of use rather than being stored in the credential itself. Observed the following as an example stacktraces: With JGit as the client: java.lang.NullPointerException at jenkins.security.ConfidentialStore.get(ConfidentialStore.java:65) at jenkins.security.ConfidentialKey.load(ConfidentialKey.java:46) at jenkins.security.CryptoConfidentialKey.getKey(CryptoConfidentialKey.java:32) at jenkins.security.CryptoConfidentialKey.decrypt(CryptoConfidentialKey.java:67) at hudson.util.Secret.decrypt(Secret.java:151) at hudson.util.Secret.fromString(Secret.java:200) at REDACTED.getPassword(REDACTED.java:136) at org.jenkinsci.plugins.gitclient.trilead.SmartCredentialsProvider.get(SmartCredentialsProvider.java:132) at org.eclipse.jgit.transport.HttpAuthMethod.authorize(HttpAuthMethod.java:219) at org.eclipse.jgit.transport.TransportHttp.connect(TransportHttp.java:502) at org.eclipse.jgit.transport.TransportHttp.openFetch(TransportHttp.java:309) at org.eclipse.jgit.transport.FetchProcess.executeImp(FetchProcess.java:136) at org.eclipse.jgit.transport.FetchProcess.execute(FetchProcess.java:122) at org.eclipse.jgit.transport.Transport.fetch(Transport.java:1138) at org.eclipse.jgit.api.FetchCommand.call(FetchCommand.java:130) at org.jenkinsci.plugins.gitclient.JGitAPIImpl$5.execute(JGitAPIImpl.java:1448) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:152) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:145) at hudson.remoting.UserRequest.perform(UserRequest.java:153) at hudson.remoting.UserRequest.perform(UserRequest.java:50) at hudson.remoting.Request$2.run(Request.java:332) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) at ......remote call to mac-os(Native Method) at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1416) at hudson.remoting.UserResponse.retrieve(UserRequest.java:253) at hudson.remoting.Channel.call(Channel.java:781) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:145) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:131) at com.sun.proxy.$Proxy133.execute(Unknown Source) at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1046) at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1086) at hudson.scm.SCM.checkout(SCM.java:495) at hudson.model.AbstractProject.checkout(AbstractProject.java:1269) at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:604) at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:529) at hudson.model.Run.execute(Run.java:1741) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:98) at hudson.model.Executor.run(Executor.java:410) with CLI Git as the client FATAL: null java.lang.NullPointerException at jenkins.security.ConfidentialStore.get(ConfidentialStore.java:65) at jenkins.security.ConfidentialKey.load(ConfidentialKey.java:46) at jenkins.security.CryptoConfidentialKey.getKey(CryptoConfidentialKey.java:32) at jenkins.security.CryptoConfidentialKey.decrypt(CryptoConfidentialKey.java:67) at hudson.util.Secret.decrypt(Secret.java:151) at hudson.util.Secret.fromString(Secret.java:200) at REDACTED.getPassword(REDACTED.java:136) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.getGitCredentialsURL(CliGitAPIImpl.java:2635) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1383) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:63) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:314) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$2.execute(CliGitAPIImpl.java:506) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:152) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:145) at hudson.remoting.UserRequest.perform(UserRequest.java:153) at hudson.remoting.UserRequest.perform(UserRequest.java:50) at hudson.remoting.Request$2.run(Request.java:332) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) at ......remote call to mac-os(Native Method) at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1416) at hudson.remoting.UserResponse.retrieve(UserRequest.java:253) at hudson.remoting.Channel.call(Channel.java:781) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:145) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:131) at com.sun.proxy.$Proxy133.execute(Unknown Source) at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1046) at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1086) at hudson.scm.SCM.checkout(SCM.java:495) at hudson.model.AbstractProject.checkout(AbstractProject.java:1269) at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:604) at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:529) at hudson.model.Run.execute(Run.java:1741) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:98) at hudson.model.Executor.run(Executor.java:410) Finished: FAILURE In this case the REDACTED class was attempting to make a call to retrieve the password from an external password store... but that call cannot take place when working on a remote system... so in that case it returns null as the password string that then has to be encrypted into a Secret... Now there seems to be another issues here also... Namely a remote agent can never decrypt a Secret so that is a set-up for failure... So I think what should be happening is that the SmartCredentialsProvider should be Channel.exported() rather than transferred over the wire... that way it will access the credentials on the master and perform the password decryption on the one node that can decrypt a secret... then the plain text can be sent to the remote node (of course for security you want to ensure you are using either SSH agents or JNLP protocol 3/4+)... that removes the need to CredentialsProvider.snapshot() and should fix issues. Similarly CliGitAPIImpl.getGitCredentialsURL seems to have the same broken assumption. I suspect also that I have hit upon the root cause as to Password protected SSH Keys not working with the Git on remote agents... namely that the getPassword().getPlainText() is being called on the remote agent and not on the master... and hence the password cannot be decrypted and hence the ssh key cannot be used. Add Comment

This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c)

[mark.earl.waite@gmail.com (JIRA)]

Mark Waite commented on JENKINS-37899

Re: Git client does not call CredentialsProvider.snapshot() when adding credentials to a SmartCredentialsProvider that will be used on a remote instance Thanks for the insight. Passphrase protected private keys in the git plugin have been a source of puzzlement to me for a very long time. There are several bug reports that they don't work in various ways. Oddly, I have a docker instance which I run in two very different environments where passphrase protected private keys work on slaves, but don't work on the master (of that docker instance). I've spent time creating various intentionally interesting ssh passphrases (trivial characters, interesting special characters, etc.), yet still haven't understood the failure cases well enough. I can't explain why my passphrase protected slave cases work, since I think they are the precise case you're describing should fail. I'll investigate further while evaluating git client PR207

Add Comment

[cleclerc@cloudbees.com (JIRA)]

Cyrille Le Clerc commented on JENKINS-37899

Re: Git client does not call CredentialsProvider.snapshot() when adding credentials to a SmartCredentialsProvider that will be used on a remote instance

Pull Request submitted: https://github.com/jenkinsci/git-client-plugin/pull/235

Add Comment

[mark.earl.waite@gmail.com (JIRA)]

Mark Waite commented on JENKINS-37899

Re: Git client does not call CredentialsProvider.snapshot() when adding credentials to a SmartCredentialsProvider that will be used on a remote instance

I'm still unable to make the git plugin work with a passphrase protected ssh private key on a local or a remote agent. Were you able to confirm that the code was not working before your change, and that it is now working after your change? Can you provide more details of the configuration steps you're using?

Add Comment

[cleclerc@cloudbees.com (JIRA)]

Cyrille Le Clerc commented on JENKINS-37899

Re: Git client does not call CredentialsProvider.snapshot() when adding credentials to a SmartCredentialsProvider that will be used on a remote instance

> I'm still unable to make the git plugin work with a passphrase protected ssh private key on a local or a remote agent.

I think I understand. Here are my tests SSH Key Build on Result  Private Key=Enter directly, passphrase=no  master    Private Key=Enter directly, passphrase=yes  master    Private Key=Enter directly, passphrase=yes  remote build agent via ssh    Private Key=From a file on Jenkins master, passphrase=yes  remote build agent via ssh   Environment Git client plugin (git-client): 2.3.0-SNAPSHOT -> latest from github Ant Plugin (ant): 1.2 OWASP Markup Formatter Plugin (antisamy-markup-formatter): 1.1 Credentials Plugin (credentials): 2.1.12 CVS Plug-in (cvs): 2.11 Display URL API (display-url-api): 1.1.1 External Monitor Job Type Plugin (external-monitor-job): 1.4 GIT server Plugin (git-server): 1.7 Git plugin (git): 3.0.5 Javadoc Plugin (javadoc): 1.1 JUnit Plugin (junit): 1.20 LDAP Plugin (ldap): 1.11 Mailer Plugin (mailer): 1.19 Matrix Authorization Strategy Plugin (matrix-auth): 1.1 Matrix Project Plugin (matrix-project): 1.8 Maven Integration plugin (maven-plugin): 2.7.1 PAM Authentication plugin (pam-auth): 1.1 SCM API Plugin (scm-api): 2.0.7 Script Security Plugin (script-security): 1.13 SSH Credentials Plugin (ssh-credentials): 1.12 SSH Slaves plugin (ssh-slaves): 1.9 Structs Plugin (structs): 1.5 Subversion Plug-in (subversion): 1.54 Translation Assistance plugin (translation): 1.10 Windows Slaves Plugin (windows-slaves): 1.0 Pipeline: SCM Step (workflow-scm-step): 1.14.2 Pipeline: Step API (workflow-step-api): 1.14.2

Add Comment

[cleclerc@cloudbees.com (JIRA)]

Cyrille Le Clerc edited a comment on JENKINS-37899

Re: Git client does not call CredentialsProvider.snapshot() when adding credentials to a SmartCredentialsProvider that will be used on a remote instance

> I'm still unable to make the git plugin work with a passphrase protected ssh private key on a local or a remote agent. I think I understand.

Here are my tests , is it what you mean?

|| SSH Key || Build on || Result ||

| Private Key=Enter directly, passphrase=no | master | (/) | | Private Key=Enter directly, passphrase=yes | master | (/) | | Private Key=Enter directly, passphrase=yes | remote build agent via ssh | (/) | | Private Key=From a file on Jenkins master, passphrase=yes | remote build agent via ssh | (x) | | Private Key=From a file on Jenkins master, passphrase=no | remote build agent via ssh | (/) | Environment {noformat}

{noformat}

Add Comment

[cleclerc@cloudbees.com (JIRA)]

Cyrille Le Clerc edited a comment on JENKINS-37899

Re: Git client does not call CredentialsProvider.snapshot() when adding credentials to a SmartCredentialsProvider that will be used on a remote instance

> I'm still unable to make the git plugin work with a passphrase protected ssh private key on a local or a remote agent.

Here are my tests, is it what you mean?

|| SSH Key || Build on || Result || | Private Key=Enter directly, passphrase=no | master | (/) | | Private Key=Enter directly, passphrase=yes | master | (/) | | Private Key=Enter directly, passphrase=yes | remote build agent via ssh | (/) |

| Private Key=From a file on Jenkins master, passphrase=yes | remote build agent via ssh | ( x / ) |

[scm_issue_link@java.net (JIRA)]

SCM/JIRA link daemon commented on JENKINS-37899

Re: Git client does not call CredentialsProvider.snapshot() when adding credentials to a SmartCredentialsProvider that will be used on a remote instance

Code changed in jenkins User: Cyrille Le Clerc Path: src/main/java/org/jenkinsci/plugins/gitclient/RemoteGitImpl.java http://jenkins-ci.org/commit/git-client-plugin/0381412bdd3ea7639cc6cdbb708fd03dc6a38339 Log: JENKINS-37899 RemoteGitImpl: Take a snapshot of the credentials before passing them to the git client proxy We need to snapshot of the credentials before passing them to the git client proxy so that the credentials are snapshotted before being serialised and transferred to the build agent through Jenkins remoting.

Add Comment

[scm_issue_link@java.net (JIRA)]

SCM/JIRA link daemon commented on JENKINS-37899

Re: Git client does not call CredentialsProvider.snapshot() when adding credentials to a SmartCredentialsProvider that will be used on a remote instance

Code changed in jenkins User: Mark Waite Path: src/main/java/org/jenkinsci/plugins/gitclient/RemoteGitImpl.java http://jenkins-ci.org/commit/git-client-plugin/e75b299c645b2155f693897754bb6a16501a3117 Log: Merge pull request #235 from cyrille-leclerc/JENKINS-37899

JENKINS-37899 RemoteGitImpl: Take a snapshot of the credentials before passing them to the git client proxy

Compare: https://github.com/jenkinsci/git-client-plugin/compare/196ed5dbb394...e75b299c645b

Add Comment

[mark.earl.waite@gmail.com (JIRA)]

Mark Waite commented on JENKINS-37899

Re: Git client does not call CredentialsProvider.snapshot() when adding credentials to a SmartCredentialsProvider that will be used on a remote instance

Change will be included in git client plugin 2.3.0, slated to be released within the next week

Add Comment

[mark.earl.waite@gmail.com (JIRA)]

Mark Waite resolved as Fixed

git client plugin 2.3.0 has released with this fix.

Jenkins / JENKINS-37899

Git client does not call CredentialsProvider.snapshot() when adding credentials to a SmartCredentialsProvider that will be used on a remote instance

Change By: Mark Waite Status: Open Resolved Resolution: Fixed

Add Comment

[mark.earl.waite@gmail.com (JIRA)]

Mark Waite closed an issue as Fixed

Jenkins / JENKINS-37899 Git client does not call CredentialsProvider.snapshot() when adding credentials to a SmartCredentialsProvider that will be used on a remote instance

Change By: Mark Waite Status: Resolved Closed

Add Comment

This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)