Javamelody causing memory leak on undeploying/deploying to tomcat

[selman...@gmail.com]

Hi All,

I have this problem with javamelody. When I deploy my .war application where I use javamelody, I get this error in tomcat logs:"SEVERE: The web application [/XX] appears to have started a thread named [javamelody XX] but has failed to stop it. This is very likely to create a memory leak."

JavaMelody Version:1.60.0

Tomcat Version:7.0.39

More logs available below:

INFO: Undeploying context [/XX]

Aug 17, 2017 6:14:47 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads

SEVERE: The web application [/XX] appears to have started a thread named [javamelody XX] but has failed to stop it. This is very likely to create a memory leak.

Aug 17, 2017 6:14:48 PM org.apache.catalina.startup.HostConfig deployWAR

INFO: Deploying web application archive /opt/apache-tomcat-7.0.39/webapps/XX.war

Thanks in advance

[evernat]

Hi,

Can you debug the init and destroy methods of the net.bull.javamelody.MonitoringFilter class with breakpoints?
You probably have a severe exception or an error somewhere causing some problems.

bye
Emeric

[selman...@gmail.com]

Hi,

I could not somehow debug tomcat core classes (where some suspicuous things could have happened) . However,when I increase the tomcat log level,I see the logs below:

The mentioned jar file is in the war. Does it say anything to you?

Aug 18, 2017 1:18:44 PM org.apache.catalina.core.ApplicationFilterConfig unregisterJMX

FINE: JMX de-registration complete for filter of type [net.bull.javamelody.MonitoringFilter] and name [javamelody]

Aug 18, 2017 1:18:44 PM org.apache.catalina.loader.WebappClassLoader loadClass

FINE: loadClass(net.bull.javamelody.MonitoringController, false)

Aug 18, 2017 1:18:44 PM org.apache.catalina.loader.WebappClassLoader loadClass

FINE:   Searching local repositories

Aug 18, 2017 1:18:44 PM org.apache.catalina.loader.WebappClassLoader findClass

FINE:     findClass(net.bull.javamelody.MonitoringController)

Aug 18, 2017 1:18:44 PM org.apache.catalina.loader.WebappClassLoader openJARs

FINE: Failed to open JAR

java.io.FileNotFoundException: /opt/tomcat/webapps/XX/WEB-INF/lib/XmlSchema-1.4.7.jar (No such file or directory)

at java.util.zip.ZipFile.open(Native Method)

at java.util.zip.ZipFile.<init>(ZipFile.java:220)

at java.util.zip.ZipFile.<init>(ZipFile.java:150)

at java.util.jar.JarFile.<init>(JarFile.java:166)

at java.util.jar.JarFile.<init>(JarFile.java:130)

at org.apache.catalina.loader.WebappClassLoader.openJARs(WebappClassLoader.java:2788)

at org.apache.catalina.loader.WebappClassLoader.findResourceInternal(WebappClassLoader.java:3041)

at org.apache.catalina.loader.WebappClassLoader.findClassInternal(WebappClassLoader.java:2823)

at org.apache.catalina.loader.WebappClassLoader.findClass(WebappClassLoader.java:1172)

at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1680)

at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1558)

at net.bull.javamelody.FilterContext.destroy(FilterContext.java:363)

at net.bull.javamelody.MonitoringFilter.destroy(MonitoringFilter.java:139)

at org.apache.catalina.core.ApplicationFilterConfig.release(ApplicationFilterConfig.java:315)

at org.apache.catalina.core.StandardContext.filterStop(StandardContext.java:4782)

at org.apache.catalina.core.StandardContext.stopInternal(StandardContext.java:5565)

at org.apache.catalina.util.LifecycleBase.stop(LifecycleBase.java:232)

at org.apache.catalina.core.ContainerBase.removeChild(ContainerBase.java:1028)

at org.apache.catalina.startup.HostConfig.deleteRedeployResources(HostConfig.java:1300)

at org.apache.catalina.startup.HostConfig.checkResources(HostConfig.java:1251)

at org.apache.catalina.startup.HostConfig.check(HostConfig.java:1440)

at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:301)

at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:119)

at org.apache.catalina.util.LifecycleBase.fireLifecycleEvent(LifecycleBase.java:90)

at org.apache.catalina.core.ContainerBase.backgroundProcess(ContainerBase.java:1374)

at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1530)

at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1540)

at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1519)

at java.lang.Thread.run(Thread.java:745)

[evernat]

Hi,

In fact, it is trying to load a class while undeploying the webapp. It is supposed to work, but the problem is that some files of the webapp were already deleted by Tomcat.
Your javamelody version seems rather old given your stack-trace, so I suggest to upgrade the javamelody version.
If that's not enough to fix the issue, you should upgrade your Tomcat version. For example, the Tomcat issue 56321 is fixed in Tomcat 7.0.54 and in Tomcat 8.0.6, and there are other related issues which were fixed in later Tomcat versions.

bye,
Emeric

[Selman Tayyar]

Hi,

I have upgraded Javamelody from 1.60.0 to 1.69.0 and the memorly leak error message disappeared.

Thanks a lot for your support!

Cheers

--

Selman Tayyar

http://www.selmantayyar.com

[Selman Tayyar]

Hi,

Unfortunately I have to get back to this.

I am still having the error that i have described.I had the impression that it is gone,and since this was a test environment I didn't pay that much attention.

So the issue persists. I have tried the folllowing:

-move the java-melody and jrobin jar to tomcat/lib folder.

But in this case,I see the issue below in the logs:

Jan 04, 2018 3:29:00 PM net.bull.javamelody.internal.common.JavaLogger warn

WARNING: exception while collecting data: java.lang.IllegalStateException: Timer already cancelled.

java.lang.IllegalStateException: Timer already cancelled.

        at java.util.Timer.sched(Timer.java:397)

        at java.util.Timer.schedule(Timer.java:248)

        at net.bull.javamelody.internal.model.RrdNioBackend.<init>(RrdNioBackend.java:71)

        at net.bull.javamelody.internal.model.RrdNioBackendFactory.open(RrdNioBackendFactory.java:84)

        at org.jrobin.core.RrdDb.<init>(RrdDb.java:229)

        at org.jrobin.core.RrdDb.<init>(RrdDb.java:203)

        at org.jrobin.core.RrdDb.<init>(RrdDb.java:266)

        at org.jrobin.core.RrdDbPool.requestRrdDb(RrdDbPool.java:103)

        at net.bull.javamelody.internal.model.JRobin.addValue(JRobin.java:362)

        at net.bull.javamelody.internal.model.Collector.addJRobinValue(Collector.java:626)

        at net.bull.javamelody.internal.model.Collector.collectJRobinValues(Collector.java:548)

        at net.bull.javamelody.internal.model.Collector.collectJavaInformations(Collector.java:410)

        at net.bull.javamelody.internal.model.Collector.collect(Collector.java:338)

        at net.bull.javamelody.internal.model.Collector.collectWithoutErrors(Collector.java:319)

        at net.bull.javamelody.internal.model.Collector.collectLocalContextWithoutErrors(Collector.java:308)

        at net.bull.javamelody.FilterContext$CollectTimerTask.run(FilterContext.java:79)

        at java.util.TimerThread.mainLoop(Timer.java:555)

        at java.util.TimerThread.run(Timer.java:505)

I don't know whether this means that memory leak is not happening anymore,but it is annoying in any case.

Any help appreciated.

JavaMelody Version:1.69.0

Tomcat Version:7.0.39

[evernat]

Hi,

As said before, upgrade your Tomcat version.
And also, do not move javamelody and jrobin in tomcat/lib.

bye,
Emeric