[JIRA] [tfs-plugin] (JENKINS-35432) Scheduled builds crash after running for more than 8 hours

65 views
Skip to first unread message

albert.chu@gdmpipelines.com (JIRA)

unread,
Jun 7, 2016, 1:08:01 PM6/7/16
to jenkinsc...@googlegroups.com
albert chu created an issue
 
Jenkins / Bug JENKINS-35432
Scheduled builds crash after running for more than 8 hours
Issue Type: Bug Bug
Assignee: Unassigned
Components: tfs-plugin
Created: 2016/Jun/07 5:07 PM
Labels: tfs-plugin
Priority: Major Major
Reporter: albert chu

I am using VS online, TFS plugin 4.1.

I have scheduled builds (every 5 minutes) which checkcode code using the TFS plugin. After 1 or 2 days the builds start failing and I need to restart jenkins.

From my initial searches, I believe this is an issue with how the plugin interacts with the TFS Java sdk.

https://social.msdn.microsoft.com/Forums/vstudio/en-US/2ace3851-8e1c-4aee-8e2d-192b0810213a/tfs-sdk-leak?forum=tee

The workaround is to close the TFSConfigurationServer first and then close the TFSTeamProjectCollection. Future versions of the TFS SDK will take care of closing the TFSConfigurationServer AND throw an exception if you close them the other way around.

Here is the log:

FATAL: null
java.lang.StackOverflowError
at com.microsoft.tfs.util.listeners.Comparators$IdentityComparator.compare(Comparators.java:57)
at com.microsoft.tfs.util.listeners.StandardListenerList$ListenerNode.addListener(StandardListenerList.java:330)
at com.microsoft.tfs.util.listeners.StandardListenerList$ListenerNode.addListener(StandardListenerList.java:343)
at com.microsoft.tfs.util.listeners.StandardListenerList$ListenerNode.addListener(StandardListenerList.java:343)
at com.microsoft.tfs.util.listeners.StandardListenerList$ListenerNode.addListener(StandardListenerList.java:343)
at com.microsoft.tfs.util.listeners.StandardListenerList$ListenerNode.addListener(StandardListenerList.java:343)

Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265)
Atlassian logo

mikael@dubratz.net (JIRA)

unread,
Sep 2, 2016, 4:02:01 AM9/2/16
to jenkinsc...@googlegroups.com
Mikael Norée commented on Bug JENKINS-35432
 
Re: Scheduled builds crash after running for more than 8 hours

I have the same problem using Jenkins 2.20 and TFS-plugin 5.1.0. After a 2-3 days all jobs start failing with same message in output. Fixed by restarting Jenkins service on the windows server.

15:31:06 FATAL: null
15:31:06
java.lang.StackOverflowError
15:31:06 at com.microsoft.tfs.util.listeners.StandardListenerList$ListenerNode.addListener(StandardListenerList.java:295)
15:31:06 at com.microsoft.tfs.util.listeners.StandardListenerList$ListenerNode.addListener(StandardListenerList.java:304)
15:31:06 at com.microsoft.tfs.util.listeners.StandardListenerList$ListenerNode.addListener(StandardListenerList.java:304)
15:31:06 at com.microsoft.tfs.util.listeners.StandardListenerList$ListenerNode.addListener(StandardListenerList.java:304)
...

This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c)
Atlassian logo

supreet.singh86@gmail.com (JIRA)

unread,
Dec 28, 2016, 12:51:02 AM12/28/16
to jenkinsc...@googlegroups.com

I am also facing similar issue with Jenkins ver. 1.642.18.3 (CloudBees Jenkins Enterprise 15.11) and TFS plugin 5.2.1. After every 3-4 days all jobs start failing on windows slave and need a restart of slave service to fix the issue.

Does anyone has a permanent fix for this issue?

adactuslatem@gmail.com (JIRA)

unread,
Jan 26, 2017, 6:22:01 PM1/26/17
to jenkinsc...@googlegroups.com

This was supposed to be fixed in the Team Explorer Everywhere SDK v14.0.3 Release. This was included in version 5.1.0 of the TFS-plugin.

I'm not sure why this is still happening, unless it wasn't actually fixed in the Team Explorer Anywhere 14.0.3.

I'm on plugin version 5.3 and we still get this error occasionally.

The only permanent fix would be to get it fixed in the TFS SDK.

florian@fpepping.de (JIRA)

unread,
Jun 20, 2018, 2:34:04 AM6/20/18
to jenkinsc...@googlegroups.com
Florian Pepping reopened an issue
 

Hi all,

unfortunately I can still confirm the bug/problem with the latest Jenkins and TFS-Versions.
Environment is:

  • Windows Server 2012 R2
    Java HotSpot(TM) 64-Bit Server VM (1.8.0_151)
  • Jenkins ver. 2.121.1
  • TFS Plugin 5.133.0

So this topic is not solved and still being pending. If some further input or traces are required please let me know. I'll try to provide them as far as possible.

Thanks and best regards
Flocki

Change By: Florian Pepping
Resolution: Done
Status: Closed Reopened
Assignee: Kellie Jos
This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e)
Atlassian logo

mmclaughlin@cloudbees.com (JIRA)

unread,
Nov 8, 2018, 1:02:02 PM11/8/18
to jenkinsc...@googlegroups.com
Mitch McLaughlin commented on Bug JENKINS-35432
 
Re: Scheduled builds crash after running for more than 8 hours

Kellie Jos

I have just seen this recreated as well using:

  • Jenkins Version: 2.89.4.2
  • TFS Plugin Version: 5.133.0
  • Java Version: 1.8.0_181
  • Operating System: RedHat Linux 3.10.0-862.14.4.el7.x86_64

Is there any plan to address this? Or a potential workaround to allow for jobs to continue running?

This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)

joe.middour@milliken.com (JIRA)

unread,
Nov 9, 2018, 10:08:02 AM11/9/18
to jenkinsc...@googlegroups.com

Mitch,

The workaround I settled on was to schedule a nightly job to restart Jenkins.

This is the command I use:
java -jar "%JENKINS_HOME%/war/WEB-INF/jenkins-cli.jar" -s "http://localhost" safe-restart --username <user> --password <pass>

I recommend using one of the credential add-ins to protect the credentials for the job.

owen@nerdnetworks.org (JIRA)

unread,
Jan 5, 2020, 11:07:13 PM1/5/20
to jenkinsc...@googlegroups.com
Owen Mehegan assigned an issue to Jason Prickett
 
Change By: Owen Mehegan
Assignee: Kellie Jos Jason Prickett
This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)
Atlassian logo
Reply all
Reply to author
Forward
0 new messages