[JIRA] (JENKINS-42959) Failed known_hosts verification for SSH agent

Issue Type:	Bug
Assignee:	Stephen Connolly
Components:	ssh-credentials-plugin, ssh-slaves-plugin
Created:	2017/Mar/21 10:19 AM
Environment:	Versions: Jenkins: 2.51 SSH Agent Plugin: 1.14 SSH Credentials plugin: 1.13 SSH Slaves Plugin : 1.15 Ubuntu 14.04, 16.04
Priority:	Major
Reporter:	Vladimir Eremeev

SSH agent isn't launched after the latest update, complaining about missing records in the known hosts file.

Nevertheless, the records do exist. I've tried to connect manually with ssh, everything was fine, ssh reports that it has found the host in known_hosts.

Probable reason is the hashed host name in known_hosts.

Here is the log from the Jenkins slave launch page:

 
                                                                [03/21/17 11:40:34] [SSH] Opening SSH connection to xxx.xxx.xxx.xxx:22
[03/21/17 11:40:34] [SSH] WARNING: No entry currently exists in the Known Hosts file for this host.      Connections will be denied until this new host and its associated key is added to the Known Hosts file.
     Key exchange was not finished, connection is closed.
     java.io.IOException: There was a problem while connecting to xxx.xxx.xxx.xxx:22
	at com.trilead.ssh2.Connection.connect(Connection.java:818)
	at com.trilead.ssh2.Connection.connect(Connection.java:687)
	at com.trilead.ssh2.Connection.connect(Connection.java:601)
	at hudson.plugins.sshslaves.SSHLauncher.openConnection(SSHLauncher.java:1265)
	at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:790)
	at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:785)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
     Caused by: java.io.IOException: Key exchange was not finished, connection is closed.
	at com.trilead.ssh2.transport.KexManager.getOrWaitForConnectionInfo(KexManager.java:93)
	at com.trilead.ssh2.transport.TransportManager.getConnectionInfo(TransportManager.java:230)
	at com.trilead.ssh2.Connection.connect(Connection.java:770)
	... 9 more
     Caused by: java.io.IOException: The server hostkey was not accepted by the verifier callback
	at com.trilead.ssh2.transport.KexManager.handleMessage(KexManager.java:591)
	at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:777)
	at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:489)
	... 1 more
     [03/21/17 11:40:34] Launch failed - cleaning up connection
     [03/21/17 11:40:34] [SSH] Connection closed.

 
                                                            

And this is the debug output from ssh, showing that it has found a record in the seknown_hosts.

 
                                                                  $ sudo -u jenkins -g jenkins ssh -v jen...@xxx.xxx.xxx.xxx
    OpenSSH_6.6.1, OpenSSL 1.0.1f 6 Jan 2014
    debug1: Reading configuration data /etc/ssh/ssh_config
    debug1: /etc/ssh/ssh_config line 19: Applying options for *
    debug1: Connecting to xxx.xxx.xxx.xxx [xxx.xxx.xxx.xxx] port 22.
    debug1: Connection established.
    debug1: identity file /var/lib/jenkins/.ssh/id_rsa type 1
...
    debug1: sending SSH2_MSG_KEX_ECDH_INIT
    debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
    debug1: Server host key: ECDSA xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx
    debug1: Host 'xxx.xxx.xxx.xxx.xxx' is known and matches the ECDSA host key.
    debug1: Found key in /var/lib/jenkins/.ssh/known_hosts:2
    debug1: ssh_ecdsa_verify: signature correct
...
    Welcome to Ubuntu 16.04.2 LTS (GNU/Linux 4.4.0-67-generic x86_64)
 
                                                            

Additional logs from Jenkins system log:

 
                                                                Mar 21, 2017 12:06:08 PM FINER com.trilead.ssh2.transport.KexManager
    kex_algo=diffie-hellman-group14-sha1
Mar 21, 2017 12:06:08 PM FINER com.trilead.ssh2.transport.KexManager
    server_host_key_algo=ssh-rsa
    Mar 21, 2017 12:06:08 PM FINER com.trilead.ssh2.transport.KexManager
    enc_algo_client_to_server=aes256-ctr
    Mar 21, 2017 12:06:08 PM FINER com.trilead.ssh2.transport.KexManager
    enc_algo_server_to_client=aes256-ctr
    Mar 21, 2017 12:06:08 PM FINER com.trilead.ssh2.transport.KexManager
    mac_algo_client_to_server=hmac-sha1
    Mar 21, 2017 12:06:08 PM FINER com.trilead.ssh2.transport.KexManager
    mac_algo_server_to_client=hmac-sha1
    Mar 21, 2017 12:06:08 PM FINER com.trilead.ssh2.transport.KexManager
    comp_algo_client_to_server=none
    Mar 21, 2017 12:06:08 PM FINER com.trilead.ssh2.transport.KexManager
    comp_algo_server_to_client=none
    Mar 21, 2017 12:06:08 PM FINE com.trilead.ssh2.transport.TransportManager
    Receive thread: error in receiveLoop
    java.io.IOException: The server hostkey was not accepted by the verifier callback
            at com.trilead.ssh2.transport.KexManager.handleMessage(KexManager.java:591)
            at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:777)
            at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:489)
            at java.lang.Thread.run(Thread.java:745)

    Mar 21, 2017 12:06:08 PM FINER com.trilead.ssh2.transport.TransportManager
    Receive thread: back from receiveLoop
    Mar 21, 2017 12:06:10 PM FINER com.trilead.ssh2.transport.KexManager
    kex_algo=diffie-hellman-group14-sha1
    Mar 21, 2017 12:06:10 PM FINER com.trilead.ssh2.transport.KexManager
    server_host_key_algo=ssh-rsa
    Mar 21, 2017 12:06:10 PM FINER com.trilead.ssh2.transport.KexManager
    enc_algo_client_to_server=aes256-ctr
    Mar 21, 2017 12:06:10 PM FINER com.trilead.ssh2.transport.KexManager
    enc_algo_server_to_client=aes256-ctr
    Mar 21, 2017 12:06:10 PM FINER com.trilead.ssh2.transport.KexManager
    mac_algo_client_to_server=hmac-sha1
    Mar 21, 2017 12:06:10 PM FINER com.trilead.ssh2.transport.KexManager
    mac_algo_server_to_client=hmac-sha1
    Mar 21, 2017 12:06:10 PM FINER com.trilead.ssh2.transport.KexManager
    comp_algo_client_to_server=none
    Mar 21, 2017 12:06:10 PM FINER com.trilead.ssh2.transport.KexManager
    comp_algo_server_to_client=none
    Mar 21, 2017 12:06:10 PM FINE com.trilead.ssh2.transport.TransportManager
    Receive thread: error in receiveLoop
    java.io.IOException: The server hostkey was not accepted by the verifier callback
            at com.trilead.ssh2.transport.KexManager.handleMessage(KexManager.java:591)
            at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:777)
            at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:489)
            at java.lang.Thread.run(Thread.java:745)

    Mar 21, 2017 12:06:10 PM FINER com.trilead.ssh2.transport.TransportManager
    Receive thread: back from receiveLoop
 
                                                            

File /var/lib/jenkins/.ssh/known_hosts contains strings, looking like being base64-encoded, delimited by '|'. Here is the sample.

 
                                                                |1|DAg  ...   o... 1ll9wI=| ...  ....  tIrM= ecdsa-sha2-nistp256 xxxxxx..... bmlzdHAyNTYAAAAIbm................. .................xxxxxxxxxxxoKEHF3Vr0q685jI2+6vWjvAAG4lz5Ckujy9k=

[Github issue|github.com/jenkinsci/ssh-slaves-plugin/issues/48]

This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e)

wl2776@gmail.com (JIRA)

unread,

Mar 21, 2017, 6:21:02 AM3/21/17

to jenkinsc...@googlegroups.com

Vladimir Eremeev updated an issue

Jenkins /

Change By:	Vladimir Eremeev

SSH agent isn't launched after the latest update, complaining about missing records in the known hosts file.

Nevertheless, the records do exist. I've tried to connect manually with ssh, everything was fine, ssh reports that it has found the host in known_hosts.

Probable reason is the hashed host name in known_hosts.

Here is the log from the Jenkins slave launch page:

{noformat}

And this is the debug output from ssh, showing that it has found a record in the seknown_hosts.

{noformat}

  $ sudo -u jenkins -g jenkins ssh -v jen...@xxx.xxx.xxx.xxx
    OpenSSH_6.6.1, OpenSSL 1.0.1f 6 Jan 2014
    debug1: Reading configuration data /etc/ssh/ssh_config
    debug1: /etc/ssh/ssh_config line 19: Applying options for *
    debug1: Connecting to xxx.xxx.xxx.xxx [xxx.xxx.xxx.xxx] port 22.
    debug1: Connection established.
    debug1: identity file /var/lib/jenkins/.ssh/id_rsa type 1
...
    debug1: sending SSH2_MSG_KEX_ECDH_INIT
    debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
    debug1: Server host key: ECDSA xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx
    debug1: Host 'xxx.xxx.xxx.xxx.xxx' is known and matches the ECDSA host key.
    debug1: Found key in /var/lib/jenkins/.ssh/known_hosts:2
    debug1: ssh_ecdsa_verify: signature correct
...
    Welcome to Ubuntu 16.04.2 LTS (GNU/Linux 4.4.0-67-generic x86_64)

{noformat}

Additional logs from Jenkins system log:

{noformat}

File /var/lib/jenkins/.ssh/known_hosts contains strings, looking like being base64-encoded, delimited by '|'. Here is the sample.

{noformat}

|1|DAg ... o... 1ll9wI=| ... .... tIrM= ecdsa-sha2-nistp256 xxxxxx..... bmlzdHAyNTYAAAAIbm................. .................xxxxxxxxxxxoKEHF3Vr0q685jI2+6vWjvAAG4lz5Ckujy9k=

{noformat}

[Github issue| https:// github.com/jenkinsci/ssh-slaves-plugin/issues/48]

wl2776@gmail.com (JIRA)

unread,

Mar 21, 2017, 6:23:01 AM3/21/17

to jenkinsc...@googlegroups.com

And this is the debug output from ssh, showing that it has found a record in the seknown_hosts second line of known_hosts .

michael.m.clarke@gmail.com (JIRA)

unread,

Mar 21, 2017, 6:36:01 AM3/21/17

to jenkinsc...@googlegroups.com

Michael Clarke commented on

This actually looks like it might be being caused by your host using ECDSA, but Trilead only supporting RSA or DSA keys, rather than the hostnames being hashed.

wl2776@gmail.com (JIRA)

unread,

Mar 21, 2017, 6:45:01 AM3/21/17

to jenkinsc...@googlegroups.com

Vladimir Eremeev commented on

How can I change it?

wl2776@gmail.com (JIRA)

unread,

Mar 21, 2017, 6:51:03 AM3/21/17

to jenkinsc...@googlegroups.com

Vladimir Eremeev edited a comment on

How can I change it?

Looks like SSH settings are default.

jglick@cloudbees.com (JIRA)

unread,

Mar 21, 2017, 10:13:02 AM3/21/17

to jenkinsc...@googlegroups.com

Jesse Glick updated an issue

Jenkins /

Change By:	Jesse Glick
Component/s:	ssh-credentials-plugin

jglick@cloudbees.com (JIRA)

unread,

Mar 21, 2017, 10:14:02 AM3/21/17

to jenkinsc...@googlegroups.com

Jesse Glick assigned an issue to Michael Clarke

Jenkins /

Change By:	Jesse Glick
Assignee:	Stephen Connolly Michael Clarke

jglick@cloudbees.com (JIRA)

unread,

Mar 21, 2017, 10:14:02 AM3/21/17

to jenkinsc...@googlegroups.com

Jesse Glick updated an issue

Jenkins /

Change By:	Jesse Glick
Labels:	regression

peter.vohmann@de.bosch.com (JIRA)

unread,

Mar 21, 2017, 1:32:09 PM3/21/17

to jenkinsc...@googlegroups.com

Peter Vohmann commented on

I had the same issue, cygwin ssh added the ecdsa-sha2-nistp256 type key to known hosts.

One can get all known types with ssh-keyscan <HOSTNAME> >>known_hosts to satisfy ssh-slave.

Observed with cygwin ssh (OpenSSH_6.8p1, OpenSSL 1.0.2a 19 Mar 2015)

peter.vohmann@de.bosch.com (JIRA)

unread,

Mar 21, 2017, 1:34:01 PM3/21/17

to jenkinsc...@googlegroups.com

Peter Vohmann edited a comment on

I had the same issue, cygwin ssh added the ecdsa-sha2-nistp256 type key to known hosts.

One can get add all known types with * ssh-keyscan <HOSTNAME> >>known_hosts to satisfy *
this did include the RSA key for ssh-slave.

Observed with cygwin ssh (OpenSSH_6.8p1, OpenSSL 1.0.2a 19 Mar 2015)

wl2776@gmail.com (JIRA)

unread,

Mar 22, 2017, 3:49:02 AM3/22/17

to jenkinsc...@googlegroups.com

Vladimir Eremeev updated an issue

Jenkins /

Change By:	Vladimir Eremeev
Attachment:	J42959.png

wl2776@gmail.com (JIRA)

unread,

Mar 22, 2017, 3:50:01 AM3/22/17

to jenkinsc...@googlegroups.com

Vladimir Eremeev commented on

Peter Vohmann: Thank you, I've obtained one more key with ssh-keyscan, having ssh-rsa in it, and the slave now launches fine.

Michael Clarke: I think, more information in the help message will be useful.
I mean the following: - this is a help on the slave configuration page. It says nothing about supported key encodings.

christian.beushausen@continental-corporation.com (JIRA)

unread,

Mar 22, 2017, 4:51:05 AM3/22/17

to jenkinsc...@googlegroups.com

Christian Beushausen commented on

I see the same issue now on one of our Jenkins 2.32.3 installations, slave was connected fine but I had to reconnect it for reasons and since then I get this same error message.

Following Peter Vohmann's advice I added the RSA key to my known_hosts file, but was not able to solve the issue. Error still persists.

christian.beushausen@continental-corporation.com (JIRA)

unread,

Mar 22, 2017, 8:51:04 AM3/22/17

to jenkinsc...@googlegroups.com

Christian Beushausen edited a comment on

I see the same issue now on one of our Jenkins 2.32.3 installations, slave was connected fine but I had to reconnect it for reasons and since then I get this same error message.

Following [~pvohmann]'s advice I added the RSA key to my known_hosts file, but was not able to solve the issue. Error still persists.

Update: Nevermind. I did not read carefully and only executed this on the slave and not on the master ... after properly updating the known_hosts file on the Jenkins master connection can be established again. Sorry for any confusion.

Would also propose to update the documentation to make it easier understandable.

john@keyba.se (JIRA)

unread,

Mar 22, 2017, 1:51:01 PM3/22/17

to jenkinsc...@googlegroups.com

John Zila commented on

It seems like there's a legitimate bug here, where the SSH Slave Plugin SSH client doesn't report which host key types it supports. My hosts all have ssh-rsa and ecdsa-sha2-nistp256 host keys, but because the client doesn't tell the host which one it wants, the host responds only with the ecdsa-sha2-nistp266 key. This causes the plugin to fail to connect.

john@keyba.se (JIRA)

unread,

Mar 22, 2017, 1:51:02 PM3/22/17

to jenkinsc...@googlegroups.com

John Zila edited a comment on

It seems like there's a legitimate bug here, where the SSH Slave Plugin SSH client doesn't report which host key types it supports. My hosts all have {{ssh-rsa}} and {{ecdsa-sha2-nistp256}} host keys, but because the client doesn't tell the host which one it wants, the host responds only with the {{ecdsa-sha2- nistp266 nistp256 }} key. This causes the plugin to fail to connect.

john@keyba.se (JIRA)

unread,

Mar 22, 2017, 1:53:03 PM3/22/17

to jenkinsc...@googlegroups.com

John Zila commented on

FWIW a workaround is to switch the Host Key Verification Strategy to "Manually trusted key Verification Strategy". According to the description, this automatically verifies the key on first-time connections, and subsequently requires the key to match. That is exactly what I need because we use dynamically created hosts via the EC2 Spot Fleet plugin.

it.support@misatravel.com (JIRA)

unread,

Mar 23, 2017, 12:03:02 AM3/23/17

to jenkinsc...@googlegroups.com

Misa Travel commented on

Hello! I also want to join my case:

I have Jenkins running in a Windows 10 machine, no cygwin, and after the upgrade I get these messages:

[03/23/17 11:46:57] [SSH] Opening SSH connection to fedora1.misatravel.org:22. C:\Users\dev\.ssh\known_hosts [SSH] No Known Hosts file was found at C:\Users\dev\.ssh\known_hosts. Please ensure one is created at this path and that Jenkins can read it. Key exchange was not finished, connection is closed. java.io.IOException: There was a problem while connecting to fedora1.misatravel.org:22 at com.trilead.ssh2.Connection.connect(Connection.java:818) at com.trilead.ssh2.Connection.connect(Connection.java:687) at com.trilead.ssh2.Connection.connect(Connection.java:601) at hudson.plugins.sshslaves.SSHLauncher.openConnection(SSHLauncher.java:1265) at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:790) at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:785) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Caused by: java.io.IOException: Key exchange was not finished, connection is closed. at com.trilead.ssh2.transport.KexManager.getOrWaitForConnectionInfo(KexManager.java:93) at com.trilead.ssh2.transport.TransportManager.getConnectionInfo(TransportManager.java:230) at com.trilead.ssh2.Connection.connect(Connection.java:770) ... 9 more Caused by: java.io.IOException: The server hostkey was not accepted by the verifier callback at com.trilead.ssh2.transport.KexManager.handleMessage(KexManager.java:591) at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:777) at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:489) ... 1 more [03/23/17 11:46:57] Launch failed - cleaning up connection [03/23/17 11:46:57] [SSH] Connection closed.

Again, it is a Windows machine and it doesn't need to follow the architecture of a Linux machine.

Why is the SSH know relying in what the system may have instead of Jenkins' configuration? After of all, the manager of Jenkins doesn't need to be the same manager than the manager for the system account where Jenkins is running.

michael.m.clarke@gmail.com (JIRA)

unread,

Mar 24, 2017, 9:00:02 AM3/24/17

to jenkinsc...@googlegroups.com

Michael Clarke commented on

Misa Travel I'm not sure what your point in your final paragraph is saying. Jenkins looks for a known_hosts file in $user_home/,ssh/known_hosts which is the default location for a known_hosts file for most SSH clients, regardless of whether they're running on Linux or Windows. I could update the Known Hosts strategy to take a path for the known_hosts file but I'm not clear if that's the issue here.

michael.m.clarke@gmail.com (JIRA)

unread,

Mar 24, 2017, 9:01:01 AM3/24/17

to jenkinsc...@googlegroups.com

Michael Clarke commented on

For the issue other people have mentioned here: Trilead is no longer an actively maintained library, although I'll raise a Pull Requests against the Jenkins fork to allow it to handle ECDSA host keys. This should fix the issue where validation works on a command prompt but fails in Jenkins.

I'd rather not add arguments to the SSH command to force a specific key format to be used, as some hosts may not have that format, and it then causes upgrade issues if we ever chose to move format and some user's machines don't support newer formats.

it.support@misatravel.com (JIRA)

unread,

Mar 26, 2017, 10:39:02 PM3/26/17

to jenkinsc...@googlegroups.com

Misa Travel commented on

Michael Clarke The fact is that I already created the file manually and I have tried to make it run. I would like to highlight again that I am running Jenkins on Windows, which doesn't have OpenSSH, and as Jenkins manager, I may not have access to Windows to operate neither remotely nor locally. At the end, I have created the known_hosts using reference other known_hosts, but this is what I get:

[03/27/17 10:24:43] [SSH] Opening SSH connection to fedora3.misatravel.org:22. [03/27/17 10:24:44] [SSH] WARNING: No entry currently exists in the Known Hosts file for this host. Connections will be denied until this new host and its associated key is added to the Known Hosts file. Key exchange was not finished, connection is closed. java.io.IOException: There was a problem while connecting to fedora3.misatravel.org:22 at com.trilead.ssh2.Connection.connect(Connection.java:818) at com.trilead.ssh2.Connection.connect(Connection.java:687) at com.trilead.ssh2.Connection.connect(Connection.java:601) at hudson.plugins.sshslaves.SSHLauncher.openConnection(SSHLauncher.java:1265) at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:790) at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:785) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Caused by: java.io.IOException: Key exchange was not finished, connection is closed. at com.trilead.ssh2.transport.KexManager.getOrWaitForConnectionInfo(KexManager.java:93) at com.trilead.ssh2.transport.TransportManager.getConnectionInfo(TransportManager.java:230) at com.trilead.ssh2.Connection.connect(Connection.java:770) ... 9 more Caused by: java.io.IOException: The server hostkey was not accepted by the verifier callback at com.trilead.ssh2.transport.KexManager.handleMessage(KexManager.java:591) at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:777) at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:489) ... 1 more [03/27/17 10:24:44] Launch failed - cleaning up connection [03/27/17 10:24:44] [SSH] Connection closed.

Now, it is when I realize about the box "Host Key Verification Strategy" which was set up as "Known hosts file Verification Strategy", but you can see that it didn't work for me.

So I change it to "Manually provided key Verification Strategy", and I put in the "SSH Key" the public key of the computer+user I want to access, but I get

[03/27/17 10:32:10] [SSH] Opening SSH connection to fedora3.misatravel.org:22. [03/27/17 10:32:10] [SSH] WARNING: The SSH key for this host does not match the key required in the connection configuration. Connections will be denied until until the host key matches the configuration key. Key exchange was not finished, connection is closed. java.io.IOException: There was a problem while connecting to fedora3.misatravel.org:22 at com.trilead.ssh2.Connection.connect(Connection.java:818) at com.trilead.ssh2.Connection.connect(Connection.java:687) at com.trilead.ssh2.Connection.connect(Connection.java:601) at hudson.plugins.sshslaves.SSHLauncher.openConnection(SSHLauncher.java:1265) at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:790) at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:785) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Caused by: java.io.IOException: Key exchange was not finished, connection is closed. at com.trilead.ssh2.transport.KexManager.getOrWaitForConnectionInfo(KexManager.java:93) at com.trilead.ssh2.transport.TransportManager.getConnectionInfo(TransportManager.java:230) at com.trilead.ssh2.Connection.connect(Connection.java:770) ... 9 more Caused by: java.io.IOException: The server hostkey was not accepted by the verifier callback at com.trilead.ssh2.transport.KexManager.handleMessage(KexManager.java:591) at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:777) at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:489) ... 1 more [03/27/17 10:32:10] Launch failed - cleaning up connection [03/27/17 10:32:10] [SSH] Connection closed.

At this point I also realize there is a new option called "Trust SSH Host Key" in the left, and after clicking on it, and accept the host key, I am still getting this exception:

[03/27/17 10:35:10] [SSH] Opening SSH connection to fedora3.misatravel.org:22. [03/27/17 10:35:11] [SSH] WARNING: The SSH key for this host does not match the key required in the connection configuration. Connections will be denied until until the host key matches the configuration key. Key exchange was not finished, connection is closed. java.io.IOException: There was a problem while connecting to fedora3.misatravel.org:22 at com.trilead.ssh2.Connection.connect(Connection.java:818) at com.trilead.ssh2.Connection.connect(Connection.java:687) at com.trilead.ssh2.Connection.connect(Connection.java:601) at hudson.plugins.sshslaves.SSHLauncher.openConnection(SSHLauncher.java:1265) at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:790) at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:785) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Caused by: java.io.IOException: Key exchange was not finished, connection is closed. at com.trilead.ssh2.transport.KexManager.getOrWaitForConnectionInfo(KexManager.java:93) at com.trilead.ssh2.transport.TransportManager.getConnectionInfo(TransportManager.java:230) at com.trilead.ssh2.Connection.connect(Connection.java:770) ... 9 more Caused by: java.io.IOException: The server hostkey was not accepted by the verifier callback at com.trilead.ssh2.transport.KexManager.handleMessage(KexManager.java:591) at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:777) at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:489) ... 1 more [03/27/17 10:35:11] Launch failed - cleaning up connection [03/27/17 10:35:11] [SSH] Connection closed.

Finally, I have changed it to "Manually trusted key Verification Strategy" and now I am able to connect with the slave again.

mindlessconsumer@gmail.com (JIRA)

unread,

Mar 27, 2017, 5:27:01 PM3/27/17

to jenkinsc...@googlegroups.com

apotek commented on

Have the exact same issue as reported after latest round of updates.

{quote}Finally, I have changed it to "Manually trusted key Verification Strategy" and now I am able to connect with the slave again.{quote}

This workaround allowed me to the disconnected slave nodes again.

mindlessconsumer@gmail.com (JIRA)

unread,

Mar 27, 2017, 5:28:01 PM3/27/17

to jenkinsc...@googlegroups.com

apotek edited a comment on

Have Corroborating:

I have the exact same issue as reported after latest round of updates.

\ {quote}Finally, I have changed it to "Manually trusted key Verification Strategy" and now I am able to connect with the slave again. \
{quote}

This workaround allowed me to connect to the disconnected slave nodes again.

srl295@gmail.com (JIRA)

unread,

Mar 29, 2017, 1:25:02 PM3/29/17

to jenkinsc...@googlegroups.com

Steven Loomis commented on

https://groups.google.com/d/msgid/jenkinsci-users/7006ab93-7ca4-4063-baf6-7c844be60165%40googlegroups.com

I needed this workaround from the mailing list:
> ssh -o HostKeyAlgorithms=ssh-rsa slave2.example.com

The implication below is that Jenkins is using weaker encryption.

adrian@smop.co.uk (JIRA)

unread,

Mar 31, 2017, 4:06:05 AM3/31/17

to jenkinsc...@googlegroups.com

Adrian Bridgett commented on

Steven Loomis's fix solved it for me as well. I purged the other lines from ~/.ssh/known_hosts just in case.

wilson@ds.net (JIRA)

unread,

Mar 31, 2017, 1:43:05 PM3/31/17

to jenkinsc...@googlegroups.com

Brian Wilson commented on

I also used the "Manually trusted key Verification Strategy" and the Agent came back online with no further issues. I then switched back to the "Known hosts file Verification Strategy", took the Agent offline and brought it back online with no issues.

Its almost as if the known_hosts file being checked isn't the file for the user id executing the Jenkins war file.

wilson@ds.net (JIRA)

unread,

Mar 31, 2017, 2:02:05 PM3/31/17

to jenkinsc...@googlegroups.com

Brian Wilson edited a comment on

I also used logged in to the Master server and sudo (sudo -su <user>) to the user running the Jenkins war file. I ran the ssh command to connect to each of the Jenkins Agent machines and had no issue connecting. I did this with both the machine name and the fully qualified domain name (e.g. machine1, and machine1.company-name.com). From what I could see the ssh ~/.ssh/known_hosts file contained valid information on the Agent machines and had correct permissions of 644.

On the Jenkins Master, I went to the Nodes, Agent, Configure page and switched from the " Known hosts file Verification Strategy" to the " Manually trusted key Verification Strategy" and then brought the Agent came back Agents online with no further issues. I then switched the Agent configuration back to the "Known hosts file Verification Strategy", took the Agent Agents offline and brought it them back online again with no issues.

I looked at the time stamp on the ~/.ssh/known_hosts file and verified its contents hadn't changed. Its almost as if the known_hosts file being checked isn't the file for the user id executing the Jenkins war file. Either way, this is an issue that needs to be addressed sooner rather than later.

wilson@ds.net (JIRA)

unread,

Mar 31, 2017, 2:27:05 PM3/31/17

to jenkinsc...@googlegroups.com

Brian Wilson edited a comment on

I logged in to the Master server in a command line shell and sudo 'd (sudo -su <user>) to the user running the Jenkins war file. I ran the ssh command to connect to each of the Jenkins Agent machines and had no issue connecting. I did this with both the machine name and the fully qualified domain name (e.g. machine1, and machine1.company-name.com). From what I could see the ssh ~/.ssh/known_hosts file contained valid information on the Agent machines and had correct permissions of 644.

On the Jenkins Master web page , I went to the Nodes, Agent, Configure page (http://<master>:8080/computer/<agent>/) and switched from the "Known hosts file Verification Strategy" to the "Manually trusted key Verification Strategy" then brought the Agents online with no issues. I then switched the Agent configuration back to the "Known hosts file Verification Strategy", took the Agents offline and brought them back online again with no issues.

I looked at the time stamp on the ~/.ssh/known_hosts file and verified its contents hadn't changed. Its almost as if the known_hosts file being checked isn't the file for the user id executing the Jenkins war file. Either way, this is an issue that needs to be addressed sooner rather than later.

mindlessconsumer@gmail.com (JIRA)

unread,

Mar 31, 2017, 2:53:04 PM3/31/17

to jenkinsc...@googlegroups.com

apotek commented on

I'll try to summarize what we already know here from reading the original post, and the first few comments.

Jenkins uses a Java ssh library. It does not use the same ssh as is used on the command line. The exception found in the original issue description makes this clear

 
                                                                The server hostkey was not accepted by the verifier callback at com.trilead.ssh2.transport.KexManager.handleMessage(KexManager.java:591)  
                                                            

2. As stated by Steven Loomis above:

I needed this workaround from the mailing list:
> ssh -o HostKeyAlgorithms=ssh-rsa slave2.example.com

The implication below is that Jenkins is using weaker encryption.

https://groups.google.com/d/msgid/jenkinsci-users/7006ab93-7ca4-4063-baf6-7c844be60165%40googlegroups.com

3. The workaround (not fix) seems to be to switch the node over to "Manually trusted key Verification Strategy".

The actual fix, then, seems to be for the com.trilead.ssh2 library to be updated to handle advances in which kinds of ssh keys are considered secure at this point. But a search on the internet makes it clear it is no longer being worked on by the original developers, though someone appears to be maintaining it somewhat here: https://github.com/jenkinsci/trilead-ssh2

If that Java library is not going to be updated, then perhaps there needs to be a push to find another Java library for managing ssh connections.

sshj seems to be the best option at the moment: https://github.com/hierynomus/sshj every thing else I am finding seems pretty dormant.

mindlessconsumer@gmail.com (JIRA)

unread,

Mar 31, 2017, 2:56:02 PM3/31/17

to jenkinsc...@googlegroups.com

apotek edited a comment on

I'll try to summarize what we already know here from reading the original post, and the first few comments.

# Jenkins uses a Java ssh library. It does not use the same ssh as is used on the command line. The exception found in the original issue description makes this clear

{code:java}
The server hostkey was not accepted by the verifier callback at com.trilead.ssh2.transport.KexManager.handleMessage(KexManager.java:591) {code}

2. As stated by [~srl295] above:

{quote}I needed this workaround from the mailing list:
> ssh -o HostKeyAlgorithms=ssh-rsa [slave2.example.com|http://slave2.example.com/]
{quote}
{quote}The implication below is that Jenkins is using weaker encryption.

[https://groups.google.com/d/msgid/jenkinsci-users/7006ab93-7ca4-4063-baf6-7c844be60165%40googlegroups.com|https://groups.google.com/d/msgid/jenkinsci-users/7006ab93-7ca4-4063-baf6-7c844be60165%40googlegroups.com?utm_medium=email&utm_source=footer]
{quote}

3. The workaround (not fix) seems to be to switch the node over to "Manually trusted key Verification Strategy".

The actual fix, then, seems to be for the com.trilead.ssh2 library to be updated to handle advances in which kinds of ssh keys are considered secure at this point. But a search on the internet makes it clear it is no longer being worked on by the original developers, though someone appears to be maintaining it somewhat here:

[ https://github.com/jenkinsci/trilead-ssh2 ]

If that Java library is not going to be updated, then perhaps there needs to be a push to find another Java library for managing ssh connections.

sshj seems to be the best option at the moment: [https://github.com/hierynomus/sshj] every thing else I am finding seems pretty dormant.

Apache Mina SSHD might also be an option: http://mina.apache.org/sshd-project/