[JIRA] (JENKINS-61356) Trilead SSH does not support PKCS#8 encoded private keys

20 views
Skip to first unread message

msicker@cloudbees.com (JIRA)

unread,
Apr 27, 2020, 11:58:02 AM4/27/20
to jenkinsc...@googlegroups.com
Matt Sicker updated an issue
 
Jenkins / Bug JENKINS-61356
Trilead SSH does not support PKCS#8 encoded private keys
Change By: Matt Sicker
Summary: Unable to connect to slave via Launch Agent mode( Trilead SSH ) does not support PKCS#8 encoded private keys
Add Comment Add Comment
 
This message was sent by Atlassian Jira (v7.13.12#713012-sha1:6e07c38)
Atlassian logo

msicker@cloudbees.com (JIRA)

unread,
Apr 27, 2020, 12:04:04 PM4/27/20
to jenkinsc...@googlegroups.com
Matt Sicker updated an issue
The Jenkins SSH agent plugin uses trilead-ssh which only supports the legacy PEM and new SSH2 private key file formats. In certain hardened environments, SSH keys are mandated to be encoded in PKCS#8 (presumably because they're more universal and allow for password protection and encrypting the key) to the point where OpenSSH and {{ssh-keygen}} are patched to only support PKCS#8 private keys.

Attached to this ticket are various encodings of private keys as we troubleshooted the initial issue described below. Note that private keys that have a header line with {{BEGIN PRIVATE KEY}} are indicative of the file being encoded in PKCS#8. If the file header contains {{BEGIN RSA PRIVATE KEY}} (or {{EC}} or whatever other algorithm), that is the legacy PEM format. If the file header contains {{BEGIN OPENSSH PRIVATE KEY}}, then that is the new SSH2 format. The latter two encoding formats are already supported in trilead-ssh.

h2. Original Details

* My Jenkins exists on an Amazon EC2 instance.
* I'm trying to add Red Hat 7.7 slave via ssh.
* I can ssh directly using terminal from my master to slave and vice versa using that key file

{code:java}
jenkins@master:/var/lib/jenkins
$ ssh -i .ssh/id_rsa -q 10.193.177.232

jenkins@slave:/var/lib/jenkins
$ ssh -i .ssh/id_rsa -q 10.193.177.209
{code}
I’m also able to make a successful SSH connection when I perform a SSH connection test from the *Manage Jenkins -> Configure System* section of the Master (_FYI - see attached screenshot_).

But when I try to configure my slave via launch agent method, it gives me the following error:
{code:java}
caused by: java.io.IOException: PEM problem: it is of unknown type
        at com.trilead.ssh2.crypto.PEMDecoder.decpdeKeyPair(PEMDecoder.java:500)
{code}
* How I've configured the slave (one of the method)?
** I've saved slave machine credentials as "*ssh key username with private key*"
** Launch method: Launch agents via SSH
** Hostname: slave machine private ip address.
** Host Key Verification Strategy: *Manually trusted key Verification Strategy*

Note:
* I cannot regenerate ssh keys as I'll not be able to access my machines again and the infrastructure is very strict. 
* All SSH key are 4096 bits in length and RSA.
* I cannot use credentials as username/password as either there is no password.
* I've tried everything whatever mentioned in other jenkins tickets for same issue or on internet, but my issue is not getting resolved.

Please see the following attachments for log details and screenshots.

 

This is a total blocker. Please advise. Thank you.

msicker@cloudbees.com (JIRA)

unread,
Apr 27, 2020, 12:05:03 PM4/27/20
to jenkinsc...@googlegroups.com

msicker@cloudbees.com (JIRA)

unread,
Apr 27, 2020, 12:07:02 PM4/27/20
to jenkinsc...@googlegroups.com
Matt Sicker commented on Improvement JENKINS-61356
 
Re: Trilead SSH does not support PKCS#8 encoded private keys

I've updated the original ticket to better reflect our understanding of the underlying issue.

msicker@cloudbees.com (JIRA)

unread,
Apr 27, 2020, 12:07:03 PM4/27/20
to jenkinsc...@googlegroups.com
Matt Sicker assigned an issue to Ivan Fernandez Calvo
 
Change By: Matt Sicker
Assignee: Matt Sicker Ivan Fernandez Calvo

kuisathaverat@gmail.com (JIRA)

unread,
Apr 27, 2020, 12:53:54 PM4/27/20
to jenkinsc...@googlegroups.com
Ivan Fernandez Calvo commented on Improvement JENKINS-61356
 
Re: Trilead SSH does not support PKCS#8 encoded private keys

For me is not a critical issue, there are several formats supported, those formats can be generated with any modern version of OpenSSL or OpenSSH. The issue of the user is thigh attached to the restrictions and versions they use in their environment, so the overall impact on users is minimum and has a solution in most of the cases.
I've documented the issue on the troubleshooting guide and in the configuration documents.

msicker@cloudbees.com (JIRA)

unread,
Apr 27, 2020, 1:44:02 PM4/27/20
to jenkinsc...@googlegroups.com

msicker@cloudbees.com (JIRA)

unread,
Apr 27, 2020, 1:45:03 PM4/27/20
to jenkinsc...@googlegroups.com
Matt Sicker commented on Improvement JENKINS-61356
 
Re: Trilead SSH does not support PKCS#8 encoded private keys

Makes sense. I've updated it to minor. I've also started talking with some other Jenkins developers to try to find out whether or not we can migrate to Apache MINA SSH as our SSH client library as it's still actively developed.

msicker@cloudbees.com (JIRA)

unread,
Apr 27, 2020, 1:49:01 PM4/27/20
to jenkinsc...@googlegroups.com
Matt Sicker edited a comment on Improvement JENKINS-61356
Makes sense. I've updated it to minor. I've also started talking with some other Jenkins developers to try to find out whether or not we can migrate to Apache MINA SSH as our SSH client library as it's still actively developed. That may or may not be relevant in the future as default OpenSSH ciphers change over time. For example, there's no AES/GCM support, and I don't remember if ChaCha20-Poly1305 is already supported or not. Then there will be updated key exchanges and all the other supporting primitives. Not sure how much of that we want to continue maintaining if we can use an OSS dependency with a community around it.
Reply all
Reply to author
Forward
0 new messages