[JIRA] (JENKINS-61023) Jenkins slave threads deadlock due to remoting.jar

66 views
Skip to first unread message

doubleknd26@gmail.com (JIRA)

unread,
Feb 9, 2020, 9:53:03 AM2/9/20
to jenkinsc...@googlegroups.com
Kideok Kim created an issue
 
Jenkins / Bug JENKINS-61023
Jenkins slave threads deadlock due to remoting.jar
Issue Type: Bug Bug
Assignee: Jeff Thompson
Attachments: channel thread.png, cpu usage of remoting_jar.png, remoting_jar ps.png, stats.png
Components: core, remoting
Created: 2020-02-09 14:52
Priority: Blocker Blocker
Reporter: Kideok Kim

descriptiion:

we setup jenkins master and several slaves. one of slaves periodically deadlocked and cpu spiked. I found It because remoting.jar. Here is a screenshot.

 

slave server stats:

 

cpu usage:

 

tracking PID 15699: ( It was remoting.jar )

 

slave thread dump: 

I really appreciate if you take a look and help us to find root cause.

Thanks!

Add Comment Add Comment
 
This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)
Atlassian logo

doubleknd26@gmail.com (JIRA)

unread,
Feb 9, 2020, 9:53:03 AM2/9/20
to jenkinsc...@googlegroups.com
Kideok Kim updated an issue
Change By: Kideok Kim
Attachment: channel thread.png
Attachment: cpu usage of remoting_jar.png
Attachment: remoting_jar ps.png
Attachment: stats.png

doubleknd26@gmail.com (JIRA)

unread,
Feb 9, 2020, 9:54:03 AM2/9/20
to jenkinsc...@googlegroups.com
Kideok Kim updated an issue
*descriptiion:*

we setup jenkins master and several slaves. one of slaves periodically deadlocked and cpu spiked. I found It because *remoting.jar*. Here is a screenshot.

 

*slave server stats:*

!
image-2020-02-09-23-46-42-173 stats .png| width=925,height=312 thumbnail !

 

*cpu usage:*

* ! image-2020-02-09-23-48-03-663 cpu usage of remoting_jar .png |thumbnail ! *

 

*tracking PID 15699: (* It was remoting.jar )

* ! image-2020-02-09-23-48-19-132 remoting_jar ps .png |thumbnail ! *

 

*slave thread dump:* 
 

! image-2020-02-09-23-47-26-493 channel thread .png| width=849,height=291 thumbnail !

 

 

I really appreciate if you take a look and help us to find root cause.

Thanks!

doubleknd26@gmail.com (JIRA)

unread,
Feb 9, 2020, 9:54:03 AM2/9/20
to jenkinsc...@googlegroups.com
Kideok Kim updated an issue
* descriptiion description :*


we setup jenkins master and several slaves. one of slaves periodically deadlocked and cpu spiked. I found It because *remoting.jar*. Here is a screenshot.

 

*slave server stats:*

!stats.png|thumbnail!

 

*cpu usage:*

!cpu usage of remoting_jar.png|thumbnail!


 

*tracking PID 15699: (* It was remoting.jar )

!remoting_jar ps.png|thumbnail!

 

*slave thread dump:*  

!channel thread.png|thumbnail!


 

 

I really appreciate if you take a look and help us to find root cause.

Thanks!

doubleknd26@gmail.com (JIRA)

unread,
Feb 9, 2020, 9:58:02 AM2/9/20
to jenkinsc...@googlegroups.com
Kideok Kim updated an issue
ref) [https://wiki.jenkins.io/display/JENKINS/Remoting+issue]

 

*description:*

we setup jenkins master and several slaves. one of slaves periodically deadlocked and cpu spiked. I found It because *
SynchronousCommandTransport$ReaderThread.run* from * remoting.jar*. Here is (ref: [https://wiki.jenkins.io/display/JENKINS/Remoting+issue])

I added some screenshots. I really appreciate if you take
a screenshot look and help us to find root cause .


 

*slave server stats:*

!stats.png|thumbnail!

 

*cpu usage:*

!cpu usage of remoting_jar.png|thumbnail!

 

*tracking PID 15699: (* It was remoting.jar )

!remoting_jar ps.png|thumbnail!

 

*slave thread dump:*  

!channel thread.png|thumbnail!

 

 

I really appreciate if you take a look and help us to find root cause.

Thanks!

jthompson@cloudbees.com (JIRA)

unread,
Feb 10, 2020, 1:27:03 PM2/10/20
to jenkinsc...@googlegroups.com
Jeff Thompson commented on Bug JENKINS-61023
 
Re: Jenkins slave threads deadlock due to remoting.jar

As described on the page you link to, you'll need to provide further information before anything can be done with this report. There is insufficient here for any action.

Things you will need to investigate include: versions used, types of agents used, operations of the different threads, what the agents are doing, etc.

Most of the time, issues like this are caused by something external to the Remoting library. These things can be system, network, or configuration. It might be a misbehaving plugin or interaction between multiple plugins.

Issues like this usually cannot be addressed without providing a reproducible scenario or sufficient diagnostics. Best of luck in troubleshooting and isolating the problem.

Reply all
Reply to author
Forward
0 new messages