jenkins slave node running debian 7 will not launch

126 views
Skip to first unread message

Jaap de Jong

unread,
May 2, 2017, 3:17:12 AM5/2/17
to Jenkins Users
Hi All!

Since a recent update my slave jenkins node will not launch.
It is running debian 7 with oracle jre 1.8.0_131.
The master runs debian 8 with the same jvm.
Launching the slave fails on the first ssh connection the master tries to make.
In the /var/log/jenkins/jenkins.log on the master:
[05/01/17 08:09:14] SSH Launch of OpenEmbedded Buildserver on mySlave failed in 4,899 ms
May 01, 2017 8:09:14 AM hudson.remoting.SynchronousCommandTransport$ReaderThread run
SEVERE: I/O error in channel OpenEmbedded Buildserver
java.io.IOException: Unexpected termination of the channel
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:73)
Caused by: java.io.EOFException
at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2335)
at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:2804)
at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:802)
at java.io.ObjectInputStream.<init>(ObjectInputStream.java:299)
at hudson.remoting.ObjectInputStreamEx.<init>(ObjectInputStreamEx.java:48)
at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:59)
On the slave in /var/log/auth.log:
May  1 14:03:27 nvs0559 sshd[31795]: fatal: dh_gen_key: group too small: 1024 (2*need 1024) [preauth]
Upgrading the slave to debian 8 fixes it, but I really need debian 7 for the project to be able to build it.
What are my options?

Thanks!
Jaap de Jong 

Richard Ginga

unread,
May 2, 2017, 7:48:31 AM5/2/17
to jenkins...@googlegroups.com
depending on how you are starting the slave, there are ways to designate which version of Java to use different from any default java. for ssh connections, see the advanced button. for webstart, see the slave.xml file. (or something like that)

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/fec29bdf-e118-47ef-9f97-4f4552db0d63%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.



--
Dick Ginga
Build Engineer

Mark Waite

unread,
May 2, 2017, 8:30:39 AM5/2/17
to jenkins...@googlegroups.com
I think you've detected a bug (or likely multiple bugs).  I think they may be due to the recent updates to the trilead ssh library in the weekly build.  Thanks for detecting them!

Passing Test Configuration

I have a Docker configuration that running Jenkins 2.46.2 with "Manually provided Key Verification Strategy".  That master uses ssh agents connected to various physical machines for platform testing, including Debian 7, Debian 8, Ubuntu 14, Ubuntu 16, CentOS 6, CentOS 7, Windows 7, and Windows 10.  Each of the Linux agents in that configuration includes the host key in the agent configuration and connects successfully no matter where I run that docker instance inside my network.

Failing Test Configuration

If I take that same configuration and replace Jenkins 2.46.2 with Jenkins 2.58, I see multiple failures.
  • None of the ssh agents connect at startup with Jenkins 2.58.  The log reports "WARNING: The SSH key for this host does not match the key required in the connection configuration" and reports "java.io.IOException: Key exchange was not finished, connection is closed" and reports "The server hostkey was not accepted by the verifier callback"
  • If I reconfigure each agent to switch from "Manually provided Key Verification Strategy" to "Non verifying Verification Strategy", then the Debian 8, Ubuntu 14, Ubuntu 16, CentOS 6, and CentOS 7 ssh based agents are able to connect.  The Debian 7 ssh based agent is not able to connect.  It reports "IOException: There was a problem while connecting to wheezy64b.markwaite.net:22" and "IOException: Key exchange was not finished, connection is closed" and "IOException: Cannot read full block, EOF reached".  Note, the "Cannot read full block" is a different failure message than the other platforms
I suspect the ssh implementation on Debian 7 is too old for the changes that were made to the trilead ssh implementation recently.  That would explain the failure you (and I) are seeing with Debian 7.  If that is correct, then there may be nothing to be done about the issue, since Debian 7 is rapidly approaching its end of life.  The Debian project will likely stop supporting Debian 7 as soon as they release Debian 9.  Your likely best solution (for now) is to stay with Jenkins 2.46.2 long term support release.

I can't explain why the "Manually provided Key Verification Strategy" no longer works.

There was a surprising stack trace in the startup of my docker instance using 2.58 which isn't there with 2.46.2.  It says:

May 02, 2017 6:26:03 AM hudson.init.impl.InstallUncaughtExceptionHandler$DefaultUncaughtExceptionHandler uncaughtException
SEVERE: A thread (Thread-30/307) died unexpectedly due to an uncaught exception, this may leave your Jenkins in a bad way and is usually indicative of a bug in the code.
java.lang.NullPointerException
        at com.trilead.ssh2.transport.KexManager.handleMessage(KexManager.java:447)
        at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:790)
        at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:502)
        at java.lang.Thread.run(Thread.java:745)

That may have something to do with at least one of the failures.

Could you log a bug so that this can be tracked?  We want some time to review the behavior before the base Jenkins version is chosen for the next long term support release.

Mark Waite

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
--
Dick Ginga
Build Engineer

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAL3PpaXMYkUoqTL2%2Bd6-6GTWgdD9dASOnus-0yk-U9Tb4jsQkg%40mail.gmail.com.

Jaap de Jong

unread,
May 3, 2017, 8:08:21 AM5/3/17
to Jenkins Users
Hi Mark,
thanks for your reply.
Cheers,
Jaap

Op dinsdag 2 mei 2017 14:30:39 UTC+2 schreef Mark Waite:
Reply all
Reply to author
Forward
0 new messages