Jenkins service going down on Windows Slave

[gotvi...@gmail.com]

Hello,

We have a Windows Server 2016 Jenkins Slave.  The Jenkins service in this slave goes down at times.  We then go back and start it manually though the service is set to start automatically.  Need help figuring out what's causing it to go down and why it isn't coming back online on its own.

[Slide]

Did you check the logs on the agent or via the Jenkins UI? (FYI, agent is the preferred term instead of "slave").

On Mon, Nov 26, 2018 at 8:06 AM <gotvi...@gmail.com> wrote:

Hello,

We have a Windows Server 2016 Jenkins Slave.  The Jenkins service in this slave goes down at times.  We then go back and start it manually though the service is set to start automatically.  Need help figuring out what's causing it to go down and why it isn't coming back online on its own.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/fd7cf10d-fd69-4c20-be36-804f6256890a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Website: http://earl-of-code.com

[John Mellor]

We have that problem all the time. IMHO the JNLP connectivity to the master is not robust enough to be used.

I have tried setting up the agent connection as a service per the Jenkins doc, but it still cannot survive momentary network or performance outages. The only way that I have managed to get this machine consistent, is to reboot the Windows machine and the service will shortly re-establish connectivity. JNLP connections are just-plain defective.

[gotvi...@gmail.com]

This is all I see in the log section of the Slave.  Is there anywhere else to see?

JNLP agent connected from IP/IP
Remoting version: 3.17
This is a Windows agent
Agent successfully connected and online

[Slide]

Weird, I have had ZERO issues with JNLP agents. It works flawlessly for me.

--

You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/0643151142ab642b3a0117ea0b42ab1e41ccfda3.camel%40esentire.com.

For more options, visit https://groups.google.com/d/optout.

--

Website: http://earl-of-code.com

[John Mellor]

Hi Slide,

We run our worker nodes mostly on sometimes-overloaded VMware hosts. We also interrupt the undersized network fairly routinely, for other reasons. Can you try hooking in a network degradation service, so that you see network outages, for instance.

JNLP connections fail in these circumstances, while ssh connections do not. Unfortunately, Windows essentially requires JNLP-only connections.

[gotvi...@gmail.com]

Any idea on where else I can look for details?

[gotvi...@gmail.com]

We have couple of more Windows Slaves, but we don't have any issue with those.  Just this one slave going down often.

[gotvi...@gmail.com]

When I looked in Windows Server event logs, I see this error - 

sigint to 81192 failed - killing as feedback

I see this message multiple times and the number (81192) is defferent in every error.

[gotvi...@gmail.com]

Any help/suggestion on this?

[Pranav Kv]

Did you checked your port number?My be it is an arbitrary selection. So better to configure the port as fixed number on your master configuration.

Regards

Pranav

--

You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/317ec954-f56e-4990-b183-55f1327aa9ae%40googlegroups.com.

[gotvi...@gmail.com]

The agent port is selected as 'fixed' in the Jenkins master configuration.

[gotvi...@gmail.com]

Found a log file on the slave machine.  It's Java heap size error.

INFO: HttpClient configured for $URL, authenticating as USER

Nov 20, 2018 6:25:09 PM com.microsoft.tfs.core.ws.runtime.transport.IdleHTTPConnectionCloser run

SEVERE: unexpected error

java.lang.OutOfMemoryError: Java heap space

at java.util.Arrays.copyOf(Unknown Source)

at java.util.Arrays.copyOf(Unknown Source)

at java.util.ArrayList.grow(Unknown Source)

at java.util.ArrayList.ensureExplicitCapacity(Unknown Source)

at java.util.ArrayList.ensureCapacityInternal(Unknown Source)

at java.util.ArrayList.add(Unknown Source)

at com.microsoft.tfs.core.ws.runtime.transport.IdleHTTPConnectionCloser.run(IdleHTTPConnectionCloser.java:109)

Nov 20, 2018 6:25:12 PM org.jenkinsci.remoting.protocol.impl.BIONetworkLayer$Reader run

SEVERE: [JNLP4-connect connection to JENKINS_URL/IP:35351] Reader thread killed by OutOfMemoryError

java.lang.OutOfMemoryError: Java heap space

Exception in thread "pool-1-thread-3" java.lang.OutOfMemoryError: Java heap space

Nov 20, 2018 6:29:32 PM hudson.slaves.ChannelPinger$1 onDead

INFO: Ping failed. Terminating the channel JNLP4-connect connection to JENKINS_URL/IP:35351.

java.util.concurrent.TimeoutException: Ping started at 1542756332316 hasn't completed by 1542756572331

at hudson.remoting.PingThread.ping(PingThread.java:134)

at hudson.remoting.PingThread.run(PingThread.java:90)

Nov 20, 2018 6:34:32 PM hudson.slaves.ChannelPinger$1 onDead

INFO: Ping failed. Terminating the channel JNLP4-connect connection to JENKINS_URL/IP:35351.

java.util.concurrent.TimeoutException: Ping started at 1542756632868 hasn't completed by 1542756872880

at hudson.remoting.PingThread.ping(PingThread.java:134)

at hudson.remoting.PingThread.run(PingThread.java:90)

Nov 20, 2018 6:39:32 PM hudson.slaves.ChannelPinger$1 onDead

INFO: Ping failed. Terminating the channel JNLP4-connect connection to JENKINS_URL/IP:35351.

java.util.concurrent.TimeoutException: Ping started at 1542756932704 hasn't completed by 1542757172726

at hudson.remoting.PingThread.ping(PingThread.java:134)

at hudson.remoting.PingThread.run(PingThread.java:90)

Nov 20, 2018 8:38:34 PM hudson.remoting.jnlp.Main createEngine

INFO: Setting up agent: HOST_NAME

Nov 20, 2018 8:38:34 PM hudson.remoting.jnlp.Main$CuiListener <init>

INFO: Jenkins agent is running in headless mode.

Trying to figure out way to increase the heap size.

Thanks to all those threw in suggestion.

[Slide]

There should be a way to pass parameters to the JVM in the .xml file (slave.xml or something similar).

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/0e4cd39b-77c0-4ef8-b783-f197bda9b86a%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Website: http://earl-of-code.com

[gotvi...@gmail.com]

Just found it.  In the Slave configuration section, below "Launch method" in "Advanced" there is a section called "JVM options".  We can pass it as a parameter here.  Need to test it and see if it works.