SSH Agent, z/OS USS, and git authentication problems

[Randall Becker]

I'm having issues trying to get an agent to authenticate using the SSH Agent plugin on a R2.4 z/OS USS agent with a Docker Jenkins controller. The goal is to convince GitSCM to actually fetch properly. We get SSH authentication errors no matter what happens. This is using Pipelines.

I've tried 

                        sshagent (credentials: ['mvs-randall']) {
                            checkout([$class: 'GitSCM',
                                branches: [[name: '*/development']],
                                extensions: [
                                    [$class: 'CleanBeforeCheckout'],
                                    [$class: 'SubmoduleOption', disableSubmodules: false, parentCredentials: true,
                                        recursiveSubmodules: true, reference: '', trackingSubmodules: false]],
                                    doGenerateSubmoduleConfigurations: false, submoduleCfg: [],
                                userRemoteConfigs: [[url: 'g...@xx.xxx.xxx.xxx:proj/repo.git'']]])
                        }

and

                        checkout([$class: 'GitSCM',
                            branches: [[name: '*/development']],
                            extensions: [
                                [$class: 'CleanBeforeCheckout'],
                                [$class: 'SubmoduleOption', disableSubmodules: false, parentCredentials: true,
                                    recursiveSubmodules: true, reference: '', trackingSubmodules: false]],
                                doGenerateSubmoduleConfigurations: false, submoduleCfg: [],
                            userRemoteConfigs: [[credentialsId: 'mvs-randall',url: 'g...@xx.xxx.xxx.xxx:proj/repo.git']]])

Both result in Permission denied (publickey).

I've done the same thing on many other platforms with no problem. This seems very R2.4 specific. There was a change in the supported file encodings as well - we used to use -Dfile.encoding=utf8 in the agent config (because this is an IBM that likes EBCDIC), but had to move to -Dfile.encoding=ISO8859-1 and everything seems messed up now. IBM had this funky script they recommend that massages the key into an IBM-1047 encoding but that does not help at all - in fact the GitSCM agent cannot process any results if that script is used.

Help!

TIA,

Randall

[Ivan Fernandez Calvo]

I think that this is the reason why it does not work https://support.cloudbees.com/hc/en-us/articles/224910467-Why-am-I-unable-to-authenticate-via-sshagent-inside-docker-

[Randall Becker]

I wish it was that simple. The issue definitely appears to be the encoding of the private key during a key exchange. When using SSH-Agent and git commands from within a shell in the pipeline, the authentication works fine. So this is likely an interaction with the GitSCM plugin not being aware of IBM-1047 encodings.

[Mark Waite]

If the operation you're performing is a checkout, why use the ssh-agent wrapper?  Why not use the same credentials ID as an argument to checkout rather than wrapping checkout in ssh-agent?

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/1ece555d-921d-4a66-ba9d-2afe1cf212fao%40googlegroups.com.

[Randall Becker]

That's what we were trying to do originally. There is a problem. When GitSCM creates the GIT_SSH content on z/OS agent the file name is encoded as an IBM1047 EBCDIC regardless of the -Dfile.encoding argument, even though the original private key and passphrase are coming from a UTF8/US-ASCII controller. When this goes to git and then SSH, the file is still encoded as IBM1047 and is when it hits the KEX code, fails. When we use the SSH Agent, this problem does not occur. I want to use the correct using GitSCM credential ID, but it does not work. I do not have a decent debug environment that would clearly demonstrate this, isolate the section of code where this is (not) happening, or allow me to easily fix this. The most important bit is that the private key should be encoded in UTF8 or US-ASCII when supplied to GIT_SSH, not the default encoding. Somehow, the SSH Agent Plugin does this correctly.

On Monday, 13 July 2020 17:41:07 UTC-4, Mark Waite wrote:

If the operation you're performing is a checkout, why use the ssh-agent wrapper?  Why not use the same credentials ID as an argument to checkout rather than wrapping checkout in ssh-agent?

To unsubscribe from this group and stop receiving emails from it, send an email to jenkins...@googlegroups.com.

[Mark Waite]

https://github.com/jenkinsci/git-client-plugin/blob/eeec334af0b6447f3db9fb88d55728911a092d73/src/main/java/org/jenkinsci/plugins/gitclient/CliGitAPIImpl.java#L2411 has specific code for z/OS.  I do not have access to z/OS so cannot test that code.  It was merged based on a submission from people that said it works.  

You're welcome to suggest improvements in that area with the understanding that I can only evaluate that code by inspection, not by execution.

Mark Waite

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/b071384e-8160-4522-9237-d036af9a6c15o%40googlegroups.com.

[Randall Becker]

I'm going to try the ibm.system.encoding property to see whether that makes a difference. It would almost make sense to have the encoding externalized as a plugin property available in a pipeline. I think part of our issue is that we run multi-platform and multi-encoding, which seems out of scope for the change. The ibm.system.encoding seems to be coming from the agent without explicit action from the controller. I'm wondering whether starting the agent with -Dibm.system.encoding=ISO8859-1 might fix our situation (or make it worse). Will try this. The bigger issue is that this is a more pervasive situation than just the GitSCM but really applies to any String instantiated by the agent.

On Friday, 17 July 2020 09:47:19 UTC-4, Mark Waite wrote:

https://github.com/jenkinsci/git-client-plugin/blob/eeec334af0b6447f3db9fb88d55728911a092d73/src/main/java/org/jenkinsci/plugins/gitclient/CliGitAPIImpl.java#L2411 has specific code for z/OS.  I do not have access to z/OS so cannot test that code.  It was merged based on a submission from people that said it works.  

You're welcome to suggest improvements in that area with the understanding that I can only evaluate that code by inspection, not by execution.

Mark Waite

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/b071384e-8160-4522-9237-d036af9a6c15o%40googlegroups.com.

[Randall Becker]

I'm going to have to dig deeper and probably debug the GitSCM plugin on the agent. -Dibm.file.encoding does not help the situation. I have a call later today that might shed some light on the situation.

On Friday, 17 July 2020 09:47:19 UTC-4, Mark Waite wrote:

https://github.com/jenkinsci/git-client-plugin/blob/eeec334af0b6447f3db9fb88d55728911a092d73/src/main/java/org/jenkinsci/plugins/gitclient/CliGitAPIImpl.java#L2411 has specific code for z/OS.  I do not have access to z/OS so cannot test that code.  It was merged based on a submission from people that said it works.  

You're welcome to suggest improvements in that area with the understanding that I can only evaluate that code by inspection, not by execution.

Mark Waite

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/b071384e-8160-4522-9237-d036af9a6c15o%40googlegroups.com.

[Randall Becker]

Turns out, you cannot use a key-pair with a passphrase in this situation. SSH key-pairs without a passphrase works correctly (but passphrases are mandatory in our shop).

[jeremy mordkoff]

CI and passphrases never mix :) You're going to need some password-less keys even if they have limited access.