[groovy-user] Groovy objects/classes not GCed after GroovyShell.evaluate()

830 views
Skip to first unread message

Bob_...@mcafee.com

unread,
Apr 8, 2013, 6:12:31 PM4/8/13
to us...@groovy.codehaus.org

Hi,

 

I’ve been looking to use Groovy on an upcoming project, but Groovy objects not getting GCed (and the resulting classes not being cleared out of PermGen) is proving to be a showstopper.

 

The following code sufficiently demonstrates the problem:

 

import groovy.lang.GroovyShell;

 

public class DemoGroovy {

    public static void main(String[] args) throws Exception {

        GroovyShell gs = new GroovyShell();

        Object result = gs.evaluate("println 'Hello, World';");

        Thread.sleep(5000L);

        gs = null;

        result = null;

        System.gc();

        System.gc();

        Thread.sleep(600000L);

    }

}

 

Running the above with these options (on Linux-x64, Java 1.7.0_15):

 

java -verbose:gc -XX:+UnlockDiagnosticVMOptions -XX:+UnlockExperimentalVMOptions -XX:+CMSClassUnloadingEnabled -XX:+ExplicitGCInvokesConcurrentAndUnloadsClass -XX:+TraceClassUnloading -XX:+CMSPermGenSweepingEnabled  -cp .:groovy-2.1.2.jar:asm-4.0.jar:antlr-2.7.7.jar DemoGroovy

 

shows zero classes being removed from PermGen.  Looking at a heap dump of the process with VisualVM shows many Groovy objects still in the VM’s existence.  These seem unable to be garbage collected, meaning that all the corresponding classes need to remain in PermGen.  (I’ve tried many variations of the above, including writing my own classloader and offloading the GroovyShell evaluation into its own thread that dies – and verifying that my own classloader and thread are properly disposed of – and the problem still occurs; I have a separate test thread that uses a couple of test classes/objects which shows that those test classes/objects are GCed and unloaded from the PermGen space just fine.)

 

The code that would normally be running the Groovy script exists as loadable module to a Java application that runs under Tomcat.  This application loads our module with an isolated classloader.  Our extension runs fine and runs Groovy scripts as expected, until it comes time to unload the extension on Windows systems: since all the Groovy objects are still on the heap, and the classes are still in the PermGen, the Windows JVM ends up holding a file lock on the Groovy JAR file.  Trying to remove the module from disk bombs out since the Groovy JAR is locked.  I’ve thought about trying to use OSGi and using Groovy as a bundle, but that is not an option for me (for a variety of reasons.)

 

I’d be terribly disappointed not to be able to embed Groovy, since the DSL/builder features are perfect for a feature I am trying to create, but the JAR locking issue is a complete showstopper.  If anyone has any ideas, I’d really appreciate them.

 

Thanks,

-Bob

Cédric Champeau

unread,
Apr 9, 2013, 4:09:11 AM4/9/13
to us...@groovy.codehaus.org
Hi!

System.gc() doesn't guarantee at all that classes will be removed from PermGen. Actually, a better test is:

long id=0
while (true) {

        GroovyShell gs = new GroovyShell();
        def result = gs.evaluate("class Foo$id{ }; new Foo$id()");
        id++
}

Use JConsole to connect to the process and you would see that permgen space increases and is periodically cleaned up. I'm using an id here just for you to make sure the classes are distinct, but there's no need for it, the behaviour would be the same.
-- 
Cédric Champeau
SpringSource - A Division Of VMware
http://www.springsource.com/
http://twitter.com/CedricChampeau

Bob_...@mcafee.com

unread,
Apr 9, 2013, 10:26:08 AM4/9/13
to us...@groovy.codehaus.org

Actually, that’s incorrect:

 

-XX:+CMSClassUnloadingEnabled -XX:+ExplicitGCInvokesConcurrentAndUnloadsClass pretty much guarantees (with OpenJDK 7) that unused classes are GCed from PermGen.  (I have code that demonstrates that exact behavior and have verified it using  –XX:+TraceClassUnloading.)

 

Regardless of whether they are or not, that’s not the fundamental problem: Groovy appears to be allocating objects on the heap that cannot be garbage collected, so the underlying class definitions can’t be swept out of PermGen, anyway.

 

I would expect that if I’m able to embed Groovy in my Java application via (new GroovyShell()).evaluate() that after the allocated GroovyShell object was disposed of, all of the related allocated Groovy objects would be disposed of.

 

To summarize: my use case is “I need all of these allocated Groovy objects to go away after evaluation,” not “Groovy doesn’t have a memory leak from repeated calls to GroovyShell.evaluate().”  So, is there a method call or set of method calls in Groovy where I can make that happen?

 

Thanks,

-Bob

Cédric Champeau

unread,
Apr 9, 2013, 5:17:36 PM4/9/13
to us...@groovy.codehaus.org
What indicates to you that Groovy behaves so? My sample shows the exact contrary: you can create as many classes as you want, on heap, and you will never run out of permgen space. Using JConsole, you will see a lot of classes created (which is normal since I'm using a shell in a loop) then deallocated when the VM runs out of permgen. The loop is a very fast way to demonstrate that you can run fine for hours without permgen leak.

Do you have anycode that throws a heap space error or a permgen space error after running for a while?  If you retain a result evaluated from a GroovyShell, you can obviously have a leak, but for a simple use case like the one you've shown (and my test too), I'm very confident this cannot happen (we tracked down a lot of those issues in the past).

Whatever the option ExplicitGCInvokesConcurrentAndUnloadsClass says (btw, it's not well documented), it doesn't work as expected. At least, here, I only managed to see unloaded classes using the following trick, which is basically limiting the perm gen space:

$ cat > DemoGroovy.java << DELIM

import groovy.lang.GroovyShell;

public class DemoGroovy {
    public static void main(String[] args) throws Exception {
   
    GroovyShell gs;
    Object result;
    for (int i=0;i<10000000;i++) {       
        gs = new GroovyShell();
            result = gs.evaluate(" 'Hello, World';");   
    }
    System.out.println("Sleeping before gc");

        Thread.sleep(5000L);

        gs = null;
        result = null;
        System.gc();
        System.gc();
        Thread.sleep(5000L);
    }
}
DELIM

$ javac -cp groovy-all.jar DemoGroovy.java
$ /opt/jdk1.7.0_17/bin/java -cp .:groovy-all.jar -verbose:class -XX:+UnlockDiagnosticVMOptions -XX:+UnlockExperimentalVMOptions -XX:+CMSClassUnloadingEnabled -XX:+ExplicitGCInvokesConcurrentAndUnloadsClasses -XX:+TraceClassUnloading -XX:MaxPermSize=16M -XX:+UseConcMarkSweepGC -XX:+ExplicitGCInvokesConcurrent -XX:+PrintGCDetails -XX:-UseParNewGC DemoGroovy

[Loaded Script1 from file:/groovy/shell]
[Loaded Script1 from file:/groovy/shell]
[Loaded Script1 from file:/groovy/shell]
[Loaded Script1 from file:/groovy/shell]
[Loaded Script1 from file:/groovy/shell]
[Loaded Script1 from file:/groovy/shell]
[Loaded Script1 from file:/groovy/shell]
// snip
[Unloading class Script1]
[Unloading class Script1]
[Unloading class Script1]
[Unloading class Script1]
[Unloading class Script1]
[Unloading class Script1]
[Unloading class Script1]
[Unloading class Script1]
[Unloading class Script1]
[Unloading class Script1]
// snip

Hope this helps

Bob_...@mcafee.com

unread,
Apr 10, 2013, 9:50:42 AM4/10/13
to us...@groovy.codehaus.org

Your test case below does not test my use case.  I don’t think you’re understanding what I’m actually saying here:

 

Step 1: Allocate a GroovyShell

Step 2: Execute something with GroovyShell.evaluate()

Step 3: Dispose of the allocated GroovyShell

Step 4: Force a GC (with VisualVM or System.gc())

 

After Step 4, with any normal system, all of the objects allocated starting at the top of the tree with GroovyShell should be gone from the heap – entirely.  At this point, I should be able to reclaim PermGen space because there are no more Groovy object instances on the heap.

 

What *you* are saying is that there is no PermGen/heap memory leak, because you can run a million iterations and the memory usage stays fixed.  The problem with that test is that Groovy classes/objects occupy a fixed amount of PermGen/heap space on the first allocation, and that allocation never changes: there is no *accumulating* leak, but there is an *initial* leak.

 

In any case, this is trivial to test: run a demo that is guaranteed to throw away the GroovyShell and all of its classes.  Set the PermGen size so that the demo has just enough space to allocate all of its classes and no more.  Once you have the PermGen size set correctly, attach VisualVM with the flags I originally outlined.  VisualVM will force the JVM to load RMI classes, so if what you think is true, the Groovy classes will get forced out – after all, if there’s no PermGen leak, all of the Groovy classes should be unloaded, right?

 

I can tell you what you’ll see, though: you might see a single Groovy class unloaded, but then you’ll see errors from the demo program because the JVM can’t allocate the RMI classes, because the Groovy classes are stuck in PermGen.

 

Whatever the option ExplicitGCInvokesConcurrentAndUnloadsClass says (btw, it's not well documented), it doesn't work as expected.”

 

It works exactly as expected.  I have code that demonstrates that it works exactly as expected, *on code that doesn’t load the Groovy JAR and run GroovyShell.evaluate()*.  If you’re running it on a demo that runs GroovyShell.evaluate(), it doesn’t work at all, for the reasons I’ve mentioned above.

Brian Gardner

unread,
Apr 10, 2013, 11:29:55 AM4/10/13
to us...@groovy.codehaus.org
I don't know if this is related, but I have a Groovlet that makes a database connection, runs some queries, builds some HTML with the MarkupBuilder, and returns it to the browser. It leaks memory, forcing a Tomcat restart every couple of days. PermGen is filling up with objects that don't get GCd. 

The guts of this code is SQL, HTMLMarkupBuilder, and JSONBuilder.

When I do a jmap on the process, I get a whole bunch of class loaders that are dead. It almost looks like a class loader is created to dynamically build something, and is left in the permgen map.

Here's the output (some of it) from jmap -permstat

class_loader    classes bytes   parent_loader   alive?  type
 
<bootstrap>     1927    11151976          null          live    <internal>
0x00002aaab8048350      44      437568  0x00002aaab8090ec8      dead    groovy/lang/GroovyClassLoader$InnerLoader@0x00002aaae13e5ef0
0x00002aaabf278680      1       3152    0x00002aaabf274de0      dead    sun/reflect/DelegatingClassLoader@0x00002aaaddaf51b8
0x00002aaab80489e8      1       3096      null          dead    sun/reflect/DelegatingClassLoader@0x00002aaaddaf51b8
0x00002aaabe3aa8c8      1       3112    0x00002aaabf274860      dead    sun/reflect/DelegatingClassLoader@0x00002aaaddaf51b8
0x00002aaabf278c30      1       3120    0x00002aaabf274de0      dead    sun/reflect/DelegatingClassLoader@0x00002aaaddaf51b8
0x00002aaabf2784a8      1       3112    0x00002aaabf274860      dead    sun/reflect/DelegatingClassLoader@0x00002aaaddaf51b8
0x00002aaabf276758      19      206584  0x00002aaabf274860      dead    org/apache/derby/impl/services/reflect/ReflectLoaderJava2@0x00002aaadf404660
0x00002aaabe38ce50      1       3112    0x00002aaabf274860      dead    sun/reflect/DelegatingClassLoader@0x00002aaaddaf51b8
0x00002aaabee3bec0      1       3128    0x00002aaabf274860      dead    sun/reflect/DelegatingClassLoader@0x00002aaaddaf51b8
0x00002aaab8048708      1       1968      null          dead    sun/reflect/DelegatingClassLoader@0x00002aaaddaf51b8
0x00002aaabf277880      1       3128    0x00002aaabf274de0      dead    sun/reflect/DelegatingClassLoader@0x00002aaaddaf51b8
0x00002aaabe747040      1       3120    0x00002aaabf274860      dead    sun/reflect/DelegatingClassLoader@0x00002aaaddaf51b8
0x00002aaabe39ce48      1       3112    0x00002aaabf274860      dead    sun/reflect/DelegatingClassLoader@0x00002aaaddaf51b8
0x00002aaabe2bafa0      1       3112    0x00002aaabf274860      dead    sun/reflect/DelegatingClassLoader@0x00002aaaddaf51b8
0x00002aaabe2a44d8      1       3112    0x00002aaabf274860      dead    sun/reflect/DelegatingClassLoader@0x00002aaaddaf51b8
0x00002aaabf277cf8      1       3112    0x00002aaabf274860      dead    sun/reflect/DelegatingClassLoader@0x00002aaaddaf51b8
0x00002aaabe2fd848      1       3112    0x00002aaabf274860      dead    sun/reflect/DelegatingClassLoader@0x00002aaaddaf51b8

I'm not that sure how to interpret this output, or if this is the same kind of thing. I'm forced to do daily Tomcat restarts, which is not the way it's supposed to be.

Brian

Cédric Champeau

unread,
Apr 10, 2013, 11:39:36 AM4/10/13
to us...@groovy.codehaus.org
We definitely don't understand each other, but I don't see how Groovy behaves differently from any other library here. Heap space and permgen space are two very different beasts. When you create a GroovyShell then you call evaluate(), you:

    * create a new class at runtime, loaded by the underlying GroovyClassLoader, which is linked to the GroovyShell. The class is allocated in the permgen space.
    * store the result into "result", which means that you have a strong reference to the result of the shell evaluation. Here, the result is *null*, since you don't return null. This means that at this point, the class which has been generated by the shell is only referenced through the GroovyClassLoader itself referenced by GroovyShell. Since the shell is not set to null, you still have the class around.
    * Thread.sleep() here is useless, for the above reason
    * gs=null makes the shell removable by the GC
    * result=null is useless since result is already null at this point
    * System.gc() will remove:
        ** the GroovyShell
        ** the GroovyClassLoader referenced by the shell
    * at this point, the class generated by the shell is removable from PermGen. Without any exotic VM option, it's up to the VM to decide whether to remove the class from PermGen or not.

Now let's emulate this without Groovy, to show you that this behaviour is not specific to our code. I will create a class loader that loads a class which is in a different directory, to make sure that it's not loaded through an import and an explicit use in code.

Follow those instructions:

$ mkdir dep
$ echo "public class Dependency {}" > Dependency.java
$ cd dep; javac Dependency.java ; cd ..
$ cat > NoDifferent.java << DELIM
import java.io.File;
import java.net.URL;
import java.net.URLClassLoader;


public class NoDifferent {

    public static void main(String[] args) throws Throwable {
        ClassLoader cl = new URLClassLoader(new URL[]{new File("dep").toURI().toURL()}, null);
        Class clazz = cl.loadClass("Dependency");
        System.out.println("clazz = " + clazz);
        System.gc(); // useless
        Thread.sleep(5000);
        System.out.println("Nulling references, sleeping 5s");
        cl = null;
        clazz = null; // hey, it's gone, it's now impossible to create an instance of this class again!
        Thread.sleep(5000);
        System.out.println("Calling gc()");
        System.gc();
        System.gc();
    }

}
DELIM
$ javac NoDifferent
$ java -verbose:class -XX:+UnlockDiagnosticVMOptions -XX:+UnlockExperimentalVMOptions -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled -XX:+ExplicitGCInvokesConcurrentAndUnloadsClasses -XX:+TraceClassUnloading NoDifferent

[Loaded ...] (tons of)
[Loaded Dependency from file:/tmp/foo/dep/] (once)

And no single [Unloading] appears. There's no leak here. It's just the way the VM works. It doesn't guarantee that classes will be unloaded even if you cannot use them anymore, until it really runs out of PermGen. Note that I used ExplicitGCInvokesConcurrentAndUnloadsClasses...

Bob_...@mcafee.com

unread,
Apr 10, 2013, 2:25:54 PM4/10/13
to us...@groovy.codehaus.org

Look, you can try to demonstrate to me that I’m not seeing Unloaded class notifications until you’re blue in the face, but the fact is that I see them.  I can easily trigger all unused classes getting swept out of PermGen, with the corresponding Unloaded notification for each class, using OpenJDK 7 on Linux-x64.

 

Now, read this carefully: the classes locked in PermGen are not the fundamental problem.  (Yes, classes locked in PermGen cause the JAR to be locked on Windows because the JVM is keeping the JAR open for the class definitions.  However, this is the *symptom* we see, and I am debugging the problem by addressing the symptom - trying to get classes swept out of PermGen, because if I can get classes swept out of PermGen, that means the classes are unused, the JAR can be closed by the JVM and Windows should unlock it).  The fundamental problem is that Groovy spews a static set of objects all over the heap, which are not GCed when the root Groovy object is disposed of.

 

Further, I actually found out what the problem in Groovy is: a series of SoftReference objects that are keeping the instances (and underlying classes) loaded.  Forcing all the SoftReference objects to be GCed after disposing of GroovyShell has resulted in every single Groovy instance being destroyed and all of the underlying classes being forced out of PermGen, with the correct ClassLoader/Thread code and the correct JVM options set.

 

Unfortunately, there is no good way of clearing SoftReferences out of a single loaded package hierarchy.  SoftReferences are only unloaded when memory pressure is very high (such as trying to allocate more RAM than is available to the JVM heap), and this results in ALL SoftReferences being cleared.  This is undesirable in a system that loads/unloads modules, as it will result in performance degradation of the core system or other modules that are also using SoftReferences.

 

I’ll probably be filing a JIRA issue about this, but I’m going to verify first that the JAR locking issue is resolved by forcing the JVM to evict all of the SoftReferences that Groovy is creating (instead of something fundamentally broken like ClassLoader.getResource().openStream() getting called somewhere in Groovy.)

Marc Paquette

unread,
Apr 10, 2013, 3:33:40 PM4/10/13
to us...@groovy.codehaus.org
Maybe it is just me, but it sounds as if your frustrations are showing through ?

You seem to be very knowledgeable of low-level stuff, but from my experience, garbage collection and memory management in java is far from trivial and jumping to conclusion from a proof-by-example point of view is like skating on thin ice. 

I'm sure everybody appreciate any documented bug report on this subject, but If the tone could go down a little, that would be appreciated. 


Marc Paquette

Jochen Theodorou

unread,
Apr 10, 2013, 4:06:34 PM4/10/13
to us...@groovy.codehaus.org
Am 10.04.2013 20:25, schrieb Bob_...@McAfee.com:
...
> Further, I actually found out what the problem in Groovy is: a series of
> SoftReference objects that are keeping the instances (and underlying
> classes) loaded.

I just wanted to point to this ;)

[...]
> Unfortunately, there is no good way of clearing SoftReferences out of a
> single loaded package hierarchy. SoftReferences are only unloaded when
> memory pressure is very high (such as trying to allocate more RAM than
> is available to the JVM heap), and this results in ALL SoftReferences
> being cleared. This is undesirable in a system that loads/unloads
> modules, as it will result in performance degradation of the core system
> or other modules that are also using SoftReferences.

sadly I don't really think there is a "one fits them all" solution here.
The SoftReferences you see are for example the meta class registry
keeping soft references to meta classes, which reference the class. As
long as the meta class exists, the class cannot be unloaded. Producing
the meta class is expensive, that's why we keep it around. You could
register a listener in the meta class, that tracks the created meta
classes and remove them from there after.

> I�ll probably be filing a JIRA issue about this, but I�m going to verify
> first that the JAR locking issue is resolved by forcing the JVM to evict
> all of the SoftReferences that Groovy is creating (instead of something
> fundamentally broken like ClassLoader.getResource().openStream() getting
> called somewhere in Groovy.)

out of interest.. how do you force those reference to null?

bye blackdrag


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email


Bob_...@mcafee.com

unread,
Apr 10, 2013, 4:29:05 PM4/10/13
to us...@groovy.codehaus.org

If there’s any frustration there, it’s with Cedric for refusing to believe the observations I had made.  (I don’t mind being told I’m wrong with the reasons why I’m wrong, but I do take some offense to being called a liar.)  It’s doubly frustrating when I get responses like that when I’m trying to promote the use of a great tool – Groovy – in a corporate environment, because now I have to temper that enthusiasm with the extra time cost it might take to discuss possible issues with Groovy on the ML. 

 

In any case, I apologize if that frustration came through in the email.

 

I agree with your experience on GC issues, but I feel like I’ve got a pretty solid case and evidence here.  (The VM options for probing GC issues in Java 7 help a lot.) 

 

-Bob

Cédric Champeau

unread,
Apr 10, 2013, 4:31:08 PM4/10/13
to us...@groovy.codehaus.org
Hi,

I'm staying polite, I would appreciate you keep calm.

 

Now, read this carefully: the classes locked in PermGen are not the fundamental problem.  (Yes, classes locked in PermGen cause the JAR to be locked on Windows because the JVM is keeping the JAR open for the class definitions.  However, this is the *symptom* we see, and I am debugging the problem by addressing the symptom - trying to get classes swept out of PermGen, because if I can get classes swept out of PermGen, that means the classes are unused, the JAR can be closed by the JVM and Windows should unlock it).  The fundamental problem is that Groovy spews a static set of objects all over the heap, which are not GCed when the root Groovy object is disposed of.

Ah, so you are probably talking about the metaclass registry. I understood that your problem was that you thought the classes generated at runtime weren't unloaded. So, this has nothing to do with the classes generated through the shell. It's about the Groovy language runtime itself. In that case, that's not a bug, it's how things works. Currently the metaclass registry relies on a singleton. There are various reasons for that, both historical and technical (performance, slow intialization, avoid loosing metaclass changes, ...).

 

Further, I actually found out what the problem in Groovy is: a series of SoftReference objects that are keeping the instances (and underlying classes) loaded.  Forcing all the SoftReference objects to be GCed after disposing of GroovyShell has resulted in every single Groovy instance being destroyed and all of the underlying classes being forced out of PermGen, with the correct ClassLoader/Thread code and the correct JVM options set.

 

Actually the problem is a bit more complex. There are both issues with the metaclass registry singleton, which can be solved using a dedicated class loader to load the groovy classes, but you would still have a problem with the reflection cache of Groovy which makes use of a thread local. Again, that can be worked around easily loading the class from a dedicated thread. But one problem that we cannot solve (and that I've just discovered, to be honest), is that Groovy makes use of a custom subclass of SoftReference. What I see, using heap snapshots and pathes from GC roots, it that the root of all groovy classes loaded using the technique I described earlier (class loader, separate thread), is this subclass:

 

So my understanding here, is that our custom subclass of SoftReference causes an unexpected behaviour in the JVM. Even if the reference itself is not used anymore, the fact of being a subclass of SoftReference makes it held by the JVM (probably because a SoftReference is something internally used for GC so has a specific behaviour). It's held because it has been loaded once (which is the meaning of <loader> of SoftRef). This is problematic indeed, and totally unexpected.

Anyway, if you consider Groovy as a runtime that needs to be initialized, this in practice should not be a problem.

Bob_...@mcafee.com

unread,
Apr 10, 2013, 5:10:08 PM4/10/13
to us...@groovy.codehaus.org
I figured the use of SoftReference was for performance. I'll admit this is somewhat selfish, but for my use case, I don't care about performance: my rationale for using Groovy revolves around constructing an easily-to-read and easy-to-document domain-specific language that's also easy to maintain; the actual usage of the DSL and associated Groovy code would be relatively infrequent, so if the DSL takes 3.2 seconds to execute instead of 2.0 seconds that's ok. (Additionally, in my use case it's likely that the MetaClasses would be unique per GroovyShell.evaluate() invocation, so caching the MetaClasses doesn't buy me anything.)

Re: clearing SoftReferences:

http://stackoverflow.com/questions/3785713/how-to-make-the-java-system-release-soft-references

which definitely works using OpenJDK 7 on Linux.

Related (differences between Soft/Weak References): http://stackoverflow.com/questions/6968141/is-there-a-way-to-force-weak-and-or-soft-referenced-objects-to-be-gcd-in-java

OpenJDK 7 on Linux definitely falls in the "clear WeakReferences on GC, keep SoftReferences until memory pressure forces them out" camp.

I did take a quick look to see how many times SoftReference was used in Groovy's src/main/java tree, but ended up seeing around 10 files as starting points. I can certainly investigate the MetaClass listener idea, but since you seem to be familiar with the Groovy internals, is the MetaClass listener going to be enough or will I have to track down half a dozen different sources for clearing objects/SoftReferences (or, possibly, finding a place where the stock Groovy JAR doesn't even have the hooks so that I can clear them out?)

-Bob

-----Original Message-----
From: Jochen Theodorou [mailto:blac...@gmx.org]
Sent: Wednesday, April 10, 2013 3:07 PM
To: us...@groovy.codehaus.org
Subject: Re: [groovy-user] Groovy objects/classes not GCed after GroovyShell.evaluate()

Am 10.04.2013 20:25, schrieb Bob_...@McAfee.com:
...
> Further, I actually found out what the problem in Groovy is: a series
> of SoftReference objects that are keeping the instances (and
> underlying
> classes) loaded.

I just wanted to point to this ;)

[...]
> Unfortunately, there is no good way of clearing SoftReferences out of
> a single loaded package hierarchy. SoftReferences are only unloaded
> when memory pressure is very high (such as trying to allocate more RAM
> than is available to the JVM heap), and this results in ALL
> SoftReferences being cleared. This is undesirable in a system that
> loads/unloads modules, as it will result in performance degradation of
> the core system or other modules that are also using SoftReferences.

sadly I don't really think there is a "one fits them all" solution here.
The SoftReferences you see are for example the meta class registry keeping soft references to meta classes, which reference the class. As long as the meta class exists, the class cannot be unloaded. Producing the meta class is expensive, that's why we keep it around. You could register a listener in the meta class, that tracks the created meta classes and remove them from there after.

> I'll probably be filing a JIRA issue about this, but I'm going to

Cédric Champeau

unread,
Apr 10, 2013, 5:24:44 PM4/10/13
to us...@groovy.codehaus.org
Le 10/04/2013 22:29, Bob_...@McAfee.com a écrit :

If there’s any frustration there, it’s with Cedric for refusing to believe the observations I had made.  (I don’t mind being told I’m wrong with the reasons why I’m wrong, but I do take some offense to being called a liar.)

I could take offense (but I will not) for the fact you say I told you were a liar. I never did, nor did I refuse to see anything. We were not talking of the same thing. As I explained in my latest email, you were talking about the Groovy system itself while I was talking about the classes that Groovy generates at runtime. The latter *are* collected. The Groovy runtime itself is not, for the reasons I gave. Now as to why SoftRef is not released, I have right now absolutely no idea, since "simple" cases I try to recreate from scratch do not seem to produce the same behaviour. As we're on edge cases here, it's pretty difficult.

Also forgive me but I have very long days of work, to be able to fix bugs and answer questions like yours. I'm not complaining at all, but I'm groovying since 8 am and it's already 11pm here and it can be difficult to stay focused all day :)
Reply all
Reply to author
Forward
0 new messages