Jenkins become unresponsive

[Rupali]

Hi,

We are facing one weird issue.
Intermittently, our Jenkins URL becomes inaccessible. Only restarting Jenkins service resolves the issue.

We are using Jenkins 1.565. We tried to analyses memory and CPU usage when such state appears. But CPU usage looked very low around 2% and memory usage by Jenkins Java process also was around 1.7 GB which is the case usually. We have Master-Slave setup containing 8 slave machines of which 2 slaves have dedicated jobs to run.

Can you advise on how to analyze this issue to find out the root cause of Jenkins becoming inaccessible?

Regards,
Rupali

[Dunnigan, Terrence J]

Are you using the Jenkins Monitoring plugin?

 

https://wiki.jenkins-ci.org/display/JENKINS/Monitoring

 

It might help to provide some context and additional information when Jenkins is not responding.

 

Terry

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Rupali Chorghe]

Hi Terrence,

Thanks for reply.
Problem we are facing is when Jenkins becomes unresponsive, everything that needs Jenkins URL, becomes inaccessible. So we decided to use Jconsole. But we didn't find abnormalities in CPU and memory usage when monitored using jconsole.

Regards,
Rupali

You received this message because you are subscribed to a topic in the Google Groups "Jenkins Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-users/FLJye7YDTr8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-use...@googlegroups.com.

[James Nord (jnord)]

Are you on Linux and are you using the embedded container?

If so have you upped the number of file descriptors available from Jenkins – I have see in the past that is was not uncommon for the HTTP receiving thread to die due to not being able to handle a connection correctly.

 

Otherwise – jstack will likely be your friend – will show any deadlocks and show what threads are running.

 

/james

[Rupali]

Hi James,

Thanks for your reply.
We are using Windows server.
I will try jstack as you suggested and will post my findings.

Thanks,
Rupali

[Isaac Cohen]

Attached is a thread dump taken when jenkins was unresponsive. I couldnt find anything suspicious, but maybe I missed something?

Thanks,

Isaac

[Daniel Beck]

While the thread is not blocked, it still looks suspiciously like
http://bugs.java.com/bugdatabase/view_bug.do?bug_id=7130796

> <thread_dump_24OCT2014.txt>

[Cohen, Isaac S]

David - Thanks for the quick reply. How did you come to this conclusion? Sorry, I am still new to thread dumps!

This email is confidential and subject to important disclaimers and conditions including on offers for the purchase or sale of securities, accuracy and completeness of information, viruses, confidentiality, legal privilege, and legal entity disclaimers, available at http://www.jpmorgan.com/pages/disclosures/email.

[Rupali]

Hi,

We are still facing this issue.
Attached are is the list of threads that were running when we found this issue.
The blocked thread do not look to be blocking on

sun.nio.ch.WindowsSelectorImpl.discardUrgentData()

as stated in the bug link here:http://bugs.java.com/bugdatabase/view_bug.do?bug_id=7130796
Could you advise on what could be causing the issue?

Regards,
Rupali

[Rupali]

Hi,

Another instance of Jenkins hanging. Thread dump attached.
We are using Jenkins version 1.565
Can you advise what could be happening here?
Does Jenkins has any pause times where it becomes inaccessible?

Regards,
Rupali