How to search for leaks
Christian
I have got a problem ... I have a java Application that seems to leak
memory.
And I am not talking about Packratting.
My heap and non Heap memory looks fine.
Though the heap + non heap memory that jConsole shows up only make up
less than one third of my totally used memory (the memory Task Manager
shows).
heap + nonheap using together about 150 MiB in average (maximum size is
280MiB of the two together)
Though taskmanager shows the application to use up more than 700 MiB.
(Actually the usage rises constantly...)
How can I find out what is causing this lost Ram outside of normal Java
memory ?
Some Native libs?
Some not properly closed ressources?
Christian
Joshua Cranmer
> heap + nonheap using together about 150 MiB in average (maximum size is
> 280MiB of the two together)
> Though taskmanager shows the application to use up more than 700 MiB.
> (Actually the usage rises constantly...)
See something like <http://shsc.info/WindowsMemoryManagement> for starters.
There are several different numbers that represent "memory usage," such
as the amount of memory requested via malloc and friends, there's the
physical memory in play, there's the blocks of memory reserved by malloc
but not granted to the program, there's the blocks of memory that the
program has requested from malloc but not used, etc.
The best way to put it is this: the Windows Task Manager is not very
accurate when it comes to memory usage.
--
Beware of bugs in the above code; I have only proved it correct, not
tried it. -- Donald E. Knuth
John B. Matthews
Christian <fake...@xyz.de> wrote:
My favorite clog: neglecting to dispose() offscreen Graphics contexts in
a timely manner.
--
John B. Matthews
trashgod at gmail dot com
<http://sites.google.com/site/drjohnbmatthews>
Christian
I seem to have a correlation of the memory used with downloads done.
Application sends and receives files .. hashes them ..shows transfers in
the GUI...
Though uploads seem not to correlate with the sent files at least not as
measureable as with downloads.
Graphics of up and downloads are pretty much the same. Downloads and
uploads don't differ by much. Most important difference is probably that
on download a java.util.concurrent.Semaphore is used to limit the
transfer speed... Though I doubt such a tremendous problem lies in the
java source. I would have suspected some native libs like the zlib
stream java offers. Though it seems not to depend on these also.
Many many other possibilitys remain ... for example after a finished
download how the file is handeld.
Are there any tools I could use to analyze a dump file of the whole
process not just of heap and nonheap memory of the jvm?
Would be good to have at least some sort of hint for what to look..
Christian
Norbert Ziegler
>
> Are there any tools I could use to analyze a dump file of the whole
> process not just of heap and nonheap memory of the jvm?
What kind of "dump file" are you talking about? If you can explain that,
you may get an advice.
> Would be good to have at least some sort of hint for what to look..
First look for an profiler-tool. E.g. I can recommend JProfiler.
Possible reasons for mem-leaking may be
- abuse of Observer-Pattern: forgetting to detach the Observer
- references to inner classes combined with cloning
- there was something concerning phantom / weak-references, I have
forgotten the details.
Christian
> Christian wrote:
>
>>
>> Are there any tools I could use to analyze a dump file of the whole
>> process not just of heap and nonheap memory of the jvm?
>
> What kind of "dump file" are you talking about? If you can explain that,
> you may get an advice.
>
process to a file.
>> Would be good to have at least some sort of hint for what to look..
> First look for an profiler-tool. E.g. I can recommend JProfiler.
> Possible reasons for mem-leaking may be
> - abuse of Observer-Pattern: forgetting to detach the Observer
Should result in Packratting not in Leaking memory.
> - references to inner classes combined with cloning
I am not using cloning... somehow I am Rather a fan of
CopyConstructors... Actually I also would assume this to rsult in
packratting..
> - there was something concerning phantom / weak-references, I have
> forgotten the details.
Should also Result in Memory in the Heap being used?
My Problem is that the memory is used outside of what jConsole shows as
Heap and non-heap..
Christian
John B. Matthews
Christian <fake...@xyz.de> wrote:
Sorry, I don't know much about Windows memory management. I mentioned
the Graphics context as an example of a native resource that might leak
outside the Java heap. Any suspicious JNI in the mix?
If your application is pure Java, it might be worth testing on another
implementation to see if it's OS specific. On Mac OS, you can use
Dtrace, which has a convenient GUI named Instruments. It's a nice
complement to a typical IDE profiler.
<http://www.apple.com/macosx/developertools/instruments.html>
<http://arstechnica.com/reviews/os/mac-os-x-10-5.ars/5>
<http://www.mactech.com/articles/mactech/Vol.23/23.11/ExploringLeopardwit
hDTrace/index.html>
Christian
The application uses a lot of native code though none of my own.
It uses eclipse rich client platform. Together with SWT this is a lot
of potential for leak.
Also some other native libs code is called like Java's ZLib
implementation.
There is no code by me that is written in native libs.
>
> If your application is pure Java, it might be worth testing on another
> implementation to see if it's OS specific. On Mac OS, you can use
> Dtrace, which has a convenient GUI named Instruments. It's a nice
> complement to a typical IDE profiler.
>
> <http://www.apple.com/macosx/developertools/instruments.html>
> <http://arstechnica.com/reviews/os/mac-os-x-10-5.ars/5>
> <http://www.mactech.com/articles/mactech/Vol.23/23.11/ExploringLeopardwit
> hDTrace/index.html>
>
I could test it under Linux... though I don't have a Mac.
The Program is bound to Linux/Windows/Mac due to usage of eclipse RCP I
am restricted to these platforms.. (not that I ever needed more..
(Solaris should be possible too))
regards
Christian
Roedy Green
quoted or indirectly quoted someone who said :
>I seem to have a correlation of the memory used with downloads done.
>Application sends and receives files .. hashes them ..shows transfers in
>the GUI..
Changes are it is going to be something you don't dispose or close.
These may be small in themselves but cling on to large hunks of native
resources. These native resources are not directly under Java's
control so I doubt nothing in Java will tell you about them directly.
Do a study of what objects you are hanging onto, being especially
careful with any sort of object that has a dispose method associated
with it.
See http://mindprod.com/jgloss/profiler.html
http://mindprod.com/jgloss/packratting.html
When you get rid of those objects the fat native resources will go
too.
Have a look at http://mindprod.com/products1.html#HTML
and study the code, scanning particularly for "close" and make sure
you do just as much closing.
There is one gotcha that comes to mind. With exec you must close the
streams even if you don't use them.
--
Roedy Green Canadian Mind Products
http://mindprod.com
We are almost certainly going to miss our [global warming] deadline.
We cannot get the 10 lost years back, and by the time a new global agreement to
replace the Kyoto accord is negotiated and put into effect, there will probably
not be enough time left to stop the warming short of the point where we must not
go. ~ Gwynne Dyer
Knute Johnson
> On Thu, 15 Jan 2009 18:27:22 +0100, Christian <fake...@xyz.de> wrote,
> quoted or indirectly quoted someone who said :
>
>> I seem to have a correlation of the memory used with downloads done.
>> Application sends and receives files .. hashes them ..shows transfers in
>> the GUI..
>
> Changes are it is going to be something you don't dispose or close.
> These may be small in themselves but cling on to large hunks of native
> resources. These native resources are not directly under Java's
> control so I doubt nothing in Java will tell you about them directly.
>
> Do a study of what objects you are hanging onto, being especially
> careful with any sort of object that has a dispose method associated
> with it.
>
> See http://mindprod.com/jgloss/profiler.html
> http://mindprod.com/jgloss/packratting.html
>
> When you get rid of those objects the fat native resources will go
> too.
>
> Have a look at http://mindprod.com/products1.html#HTML
> and study the code, scanning particularly for "close" and make sure
> you do just as much closing.
>
> There is one gotcha that comes to mind. With exec you must close the
> streams even if you don't use them.
I got caught not disposing some JDialogs. Took about two and a half
weeks and it would blow up. A few dispose() statements and it runs for
months now.
--
Knute Johnson
email s/nospam/knute2009/
--
Posted via NewsDemon.com - Premium Uncensored Newsgroup Service
------->>>>>>http://www.NewsDemon.com<<<<<<------
Unlimited Access, Anonymous Accounts, Uncensored Broadband Access
Mark Space
> I got caught not disposing some JDialogs. Took about two and a half
> weeks and it would blow up. A few dispose() statements and it runs for
> months now.
>
I thought most Swing components had a finalize() method to prevent this,
but I don't see it. So add your own finalize() for child classes, or
don't forget to call dispose(). Either way, a yucky state of affairs,
as the programmer has to add the clean-up manually.
Lew
> I thought most Swing components had a finalize() method to prevent this,
> but I don't see it. So add your own finalize() for child classes, or
> don't forget to call dispose(). Either way, a yucky state of affairs,
> as the programmer has to add the clean-up manually.
Good advice, except for the part about using 'finalize()'.
Avoid 'finalize()' except for cleanup of native memory that is outside
the ken of GC. This basically only comes up in JNI code. Otherwise,
'finalize()' is evil.
--
Lew
Mark Space
Isn't that what dispose() does, clean up memory allocated by the
(native) windowing system? I thought that's the memory that Knute was
talking about when he said his app would leak memory and blow up after
two and one half weeks.
John B. Matthews
Mark Space <mark...@sbcglobal.net> wrote:
In both Graphics and JDialog, finalize() calls dispose(). If you don't
call dispose() explicitly, the finalize() method should do it for you.
IIUC, you may run out of memory in the interim. I've seen it described
as more of a clog than a leak.
Arne Vajhøj
I would try to avoid relying on finalizer if possible.
Arne
Lew
>>>> Avoid 'finalize()' except for cleanup of native memory that is outside
>>>> the ken of GC. This basically only comes up in JNI code. Otherwise,
>>>> 'finalize()' is evil.
Mark Space wrote:
>>> Isn't that what dispose() does, clean up memory allocated by the
>>> (native) windowing system? I thought that's the memory that Knute
>>> was talking about when he said his app would leak memory and blow up
>>> after two and one half weeks.
John B. Matthews wrote:
>> In both Graphics and JDialog, finalize() calls dispose(). If you don't
>> call dispose() explicitly, the finalize() method should do it for you.
"Should"?
Only if the Java object becomes eligible for garbage collection. That could
be a long time coming.
>> IIUC, you may run out of memory in the interim. I've seen it described
>> as more of a clog than a leak.
Arne Vajhøj wrote:
> I would try to avoid relying on finalizer if possible.
That's part of my point. You never can tell when or if a finalizer will run.
Since the JVM does not monitor native memory, it will not know that there is
memory pressure to call the finalizers that release native memory.
The other part is not to add finalizers to one's own classes unless such
classes *explicitly* interact with JNI. The point of finalizers that work
with JNI is to free the native memory that has the same lifetime as the Java
object, so that it goes away when the Java object does.
With graphics components, there might be a whole lot of native memory in use
but not necessarily a whole lot of Java heap, so it could take GC a long time
to get around to freeing the objects and thus the native memory. Calling
'dispose()' explicitly gives native memory allocations a shorter lifetime than
the managed objects'.
--
Lew
John B. Matthews
wrote:
Calling dispose() explicitly is clearly best practice; the problem
occurs when such a call is missing. The index shows a pageful of
eponymous methods, some in frequently used classes. Is there a
convention about using the name "dispose" to indicate the release of
system resources? Are there other likely prospects?
Lew
>>>> In both Graphics and JDialog, finalize() calls dispose(). If you don't
>>>> call dispose() explicitly, the finalize() method should do it for you.
> occurs when such a call is missing. The index shows a pageful of
> eponymous methods, some in frequently used classes. Is there a
> convention about using the name "dispose" to indicate the release of
> system resources? Are there other likely prospects?
OK, yes, indeed, 'finalize()' as a last-ditch cleanup is appropriate
sometimes. I had quite the argument with a co-worker about this some years
ago - he excoriated my use of a finalizer to clean up database connections and
I justified it as a hack for just in case. This is somewhat supported in
/Effective Java/ as a valid use case, as long as one doesn't ignore that
"calling dispose() [or the equivalent] explicitly is clearly best practice".
--
Lew
John B. Matthews
wrote:
Speaking of equivalents, I had thought to make a list of explicit
termination methods. Of course, it can't be exhaustive, but it might
serve as a guide:
java.awt.Component.BltBufferStrategy#dispose()
java.awt.Component.FlipBufferStrategy#dispose()
java.awt.CompositeContext#dispose()
java.awt.Graphics#dispose()
java.awt.im.InputContext#dispose()
java.awt.im.spi.InputMethod#dispose()
java.awt.image.BufferStrategy#dispose()
java.awt.Image#flush()
java.awt.PaintContext#dispose()
java.awt.Window#dispose()
java.io.InputStream#close()*
java.io.OutputStream#close()*
java.sql.Connection#close()
java.util.Timer#cancel()
javax.imageio.ImageReader#dispose()
javax.imageio.ImageWriter#dispose()
javax.print.StreamPrintService#dispose()
javax.security.sasl.SaslClient#dispose()
javax.security.sasl.SaslServer#dispose()
javax.swing.DebugGraphics#dispose()
javax.swing.JInternalFrame#dispose()
org.ietf.jgss.GSSContext#dispose()
org.ietf.jgss.GSSCredential#dispose()
* Includes subclasses
Mark Space
>
> In both Graphics and JDialog, finalize() calls dispose(). If you don't
You know, I'm sure I've read this too. However, I can't find it anywhere
in the Java docs right now. Do you have a source for this? I find it
odd that this isn't documented anywhere.
John B. Matthews
Mark Space <mark...@sbcglobal.net> wrote:
The Graphics API says, "Although the finalization process of the garbage
collector also disposes of the same system resources, it is preferable
to manually free the associated resources..."
<http://java.sun.com/javase/6/docs/api/java/awt/Graphics.html#dispose()>
As it's not entirely dispositive, I couldn't resist peeking behind the
curtain over my vendor's implementation of Graphics:
public void finalize() {
dispose();
}
Joshua Bloch mentions a colleague who ran afoul of neglecting dispose()
in _Effective_Java_, Item 7.
Lew
> As it's not entirely dispositive,
Good one!
> I couldn't resist peeking behind the
> curtain over my vendor's implementation of Graphics:
> public void finalize() {
> dispose();
> }
>
> Joshua Bloch mentions a colleague who ran afoul of neglecting dispose()
> in _Effective_Java_, Item 7.
"Failing to release resources is a negative habit," Joshua said dispositively.
--
Lew
Christian
I think I found at least one large culprit...
FileSystemView.getFileSystemView()
I thought that was kind of a singleton that I could just call every time
to get the single instance of fileSystem view...
Though it seems like each time you call this one FileSystemView object
is created which is never disposed. And it does not come with a dispose
method...
I still need to test if this is really responsible for the memory lost
outside of java heapand non heap memory...
thoughg the class has the potential to this... (native code in it
suspected..)
Also comparison between 2 heap dumps shows that nearly 30.000
FileSystemViews more were alive then shortly after start (compared to 16
hours running)...
(eclipse mat is a great tool!!!)
greetings
Christian
Lew
> Also comparison between 2 heap dumps shows that nearly 30.000
> FileSystemViews more were alive then shortly after start (compared to 16
> hours running)...
I assume that you mean that there were 30K more 'FileSystemView'
objects in the heap after 16 hours than just after the start of the
program.
That sure is labor intensive - how in the world were you able to
invoke 'JFIleChooser' 30 thousand times in 16 hours? Perhaps you used
a test harness? If so, that could be the source of your packratting.
(More on this in a moment.)
Regardless, there is no evidence that 'FileSystemView' is a graphical
object, and I expect that it isn't, nor that it hangs on to native
(i.e., unmanaged) memory. That would explain its lack of a 'dispose
()' method.
Since 'FileSystemView' is simply a "gateway to the file system", it
probably doesn't use much heap for each instance. It doesn't document
that it has a 'finalize()' method, although implementing classes might
(and should, if they do use unmanaged memory). It is quite possible
that implementing classes have no need to allocate unmanaged memory,
relying on system calls that use only transient memory to perform its
functions.
The presence of all those objects in the heap is only evidence that GC
didn't feel any memory pressure, so didn't bother to sweep away the
unreferenced objects. It's also possible that something in your code
is packratting the 'JFileChooser's that created the 'FileSystemView's,
particularly if you are using a test harness to run the app. If GC
doesn't need to run, you shouldn't expect the objects to disappear.
Are you getting 'OutOfMemoryError's?
I suspect that 'FileSystemView' does not cause your memory "leak".
--
Lew
Mark Space
> In article <gkq8m4$o2f$1...@news.motzarella.org>,
> Mark Space <mark...@sbcglobal.net> wrote:
>
>> John B. Matthews wrote:
>>> In both Graphics and JDialog, finalize() calls dispose(). If you don't
>> You know, I'm sure I've read this too. However, I can't find it anywhere
>> in the Java docs right now. Do you have a source for this? I find it
>> odd that this isn't documented anywhere.
>
> The Graphics API says, "Although the finalization process of the garbage
> collector also disposes of the same system resources, it is preferable
> to manually free the associated resources..."
Yeah, that's Graphics. I was thinking primarily of JDialog and
super-classes, like Window. I don't see the behavior of finalize()
discussed at all for that class hierarchy. Possibly so that folks won't
rely on finalize() and just call dispose() instead. But still,
documentation is good...
John B. Matthews
<979ab8a4-e293-4524...@e18g2000yqo.googlegroups.com>,
Lew <l...@lewscanon.com> wrote:
> Christian wrote:
> > Also comparison between 2 heap dumps shows that nearly 30.000
> > FileSystemViews more were alive then shortly after start (compared to 16
> > hours running)...
[...]
> Regardless, there is no evidence that 'FileSystemView' is a graphical
> object, and I expect that it isn't, nor that it hangs on to native
> (i.e., unmanaged) memory. That would explain its lack of a 'dispose
> ()' method.
[...]
> I suspect that 'FileSystemView' does not cause your memory "leak".
Interesting, the abstract class FileSystemView has a single abstract
method, createNewFolder(). When I instantiate the class repeatedly with
an implementation returning null, there is no leak. In contrast,
repeatedly invoking the static factory leaks instances of
FileSystemView$1 linearly over time. This may be required by the
implementation, which isn't usually invoked tens of thousands of times
per day.
import javax.swing.filechooser.FileSystemView;
/** @author John B. Matthews */
public class Test {
public static void main(String[] args) {
while (true) {
FileSystemView.getFileSystemView();
}
}
}
Perhaps the OP can use a single instance.
John B. Matthews
Mark Space <mark...@sbcglobal.net> wrote:
> John B. Matthews wrote:
> > In article <gkq8m4$o2f$1...@news.motzarella.org>,
> > Mark Space <mark...@sbcglobal.net> wrote:
> >
> >> John B. Matthews wrote:
> >>> In both Graphics and JDialog, finalize() calls dispose().
> >>
> >> You know, I'm sure I've read this too. However, I can't find it
> >> anywhere in the Java docs right now. Do you have a source for
> >> this? I find it odd that this isn't documented anywhere.
> >
> > The Graphics API says, "Although the finalization process of the
> > garbage collector also disposes of the same system resources, it is
> > preferable to manually free the associated resources..."
>
> Yeah, that's Graphics. I was thinking primarily of JDialog and
> super-classes, like Window. I don't see the behavior of finalize()
> discussed at all for that class hierarchy. Possibly so that folks
> won't rely on finalize() and just call dispose() instead. But still,
> documentation is good...
Sadly, I have a slight predilection for over-optimistic interpretation
of the API. :-)
As you note, JDialog inherits finalize() from Window. I can find nothing
helpful except "Subclasses that override this method should call
super.finalize()." Absent automatic finalizer chaining, Bloch discusses
the Finalizer Guardian idiom to protect against careless (or malicious)
failure to do so. The source for my implementation mentions this very
problem in the body of dispose().
Mark Space
> As you note, JDialog inherits finalize() from Window. I can find nothing
> helpful except "Subclasses that override this method should call
> super.finalize()." Absent automatic finalizer chaining, Bloch discusses
> the Finalizer Guardian idiom to protect against careless (or malicious)
> failure to do so. The source for my implementation mentions this very
> problem in the body of dispose().
>
I just checked this in my IDE:
JDialog d = new JDialog() {
@Override
public void finalize() throws Throwable {
super.finalize();
}
};
If I ask my IDE to go to the source of super.finalize(), it takes me to
Object. So yeah, I think there's no finalize() method for JDialog.
I think my solution to disposing resources was to always use
"DISPOSE_ON_CLOSE". That way, as soon as the window (or any subclass of
window) is removed from the screen, it gets dispose() called. That's as
close to RAII as I think you can get there. I think you might have to
re-implement this behavior (or at least chain to it) if you set
WindowListener. Not sure though.
Martin Gregorie
Jumping in (apologies if I repeat something already said), but I was
trying to use WindowListener with JFrame today by extending
WindowAdapter. The first thing I found was that windowClosing(),
windowClosed() and windowDeactivation() methods weren't firing. I
eventually worked out that I had to add:
setDefaultCloseOperation(WindowConstants.DO_NOTHING_ON_CLOSE);
in the JFrame extension class constructor to get the WindowAdapter to
fire. The JFrame extension class overrides processWindowEvent() along
these lines:
protected void processWindowEvent(WindowEvent e)
{
if (e.getID() == WindowEvent.WINDOW_CLOSING)
{
dispose();
// Do application specific stuff.
// This doesn't change any JFrame states
}
super.processWindowEvent(e);
}
but as this only does non-default stuff for the WINDOW-CLOSING event I
don't think it can mess with either the windowClosed() or
windowDeactivated() WindowAdapter methods.
This is in Java 1.6.0_07.
--
martin@ | Martin Gregorie
gregorie. | Essex, UK
org |
Mark Space
>
> in the JFrame extension class constructor to get the WindowAdapter to
> fire. The JFrame extension class overrides processWindowEvent() along
> these lines:
>
> protected void processWindowEvent(WindowEvent e)
> {
> if (e.getID() == WindowEvent.WINDOW_CLOSING)
> {
> dispose();
It's interesting to me that they do this in the CLOSING part, not
CLOSED. I think dispose() is equivalent setVisible(false) in terms of
it's outward behavior. Maybe they're just killing two birds with one stone.
Good information though.
John B. Matthews
Martin Gregorie <mar...@see.sig.for.address.invalid> wrote:
Thanks Mark & Martin. I erred above when I said, "In both Graphics and
JDialog, finalize() calls dispose()." I believe this is true for
Graphics, but not for JDialog.
<http://groups.google.com/group/comp.lang.java.programmer/msg/5b22fbaaf6a
7409b>
Christian
I did...
Changed that..
About my usage:
FileSystemView is used at 2 points in my application.
The more often used one is:
Skimming through the disc of a user below a certain diretory to find any
file/directory that is not hidden.
Now for each directory/subdirectory the FileSystemView$1 is created once..
And a refresh of this list of files is done every hour.
In the end that gives lots and lots of Objects...
Though they are not my culprit.
They are barely noticeable on the heap in the end.
Also they seem not to carry any native storage with them.
Christian
Christian
> On Thu, 15 Jan 2009 18:27:22 +0100, Christian <fake...@xyz.de> wrote,
> quoted or indirectly quoted someone who said :
>
>> I seem to have a correlation of the memory used with downloads done.
>> Application sends and receives files .. hashes them ..shows transfers in
>> the GUI..
>
> Changes are it is going to be something you don't dispose or close.
> These may be small in themselves but cling on to large hunks of native
> resources. These native resources are not directly under Java's
> control so I doubt nothing in Java will tell you about them directly.
>
> Do a study of what objects you are hanging onto, being especially
> careful with any sort of object that has a dispose method associated
> with it.
>
> See http://mindprod.com/jgloss/profiler.html
> http://mindprod.com/jgloss/packratting.html
>
> When you get rid of those objects the fat native resources will go
> too.
>
> Have a look at http://mindprod.com/products1.html#HTML
> and study the code, scanning particularly for "close" and make sure
> you do just as much closing.
>
> There is one gotcha that comes to mind. With exec you must close the
> streams even if you don't use them.
hmm
I have no exec call to run subprocesses in my Program.
MAT (you have that one listed as SAP on
http://mindprod.com/jgloss/profiler.html) shows no Objects ratting
together..
Comparing heapdumps from start to 10 hours old only helped me to find
some problem that was not the culprit.
I am really a bit lost.
Usually FindBugs warns me of unclosed streams. So I am rather optimistic
that I don't have any of these.
Christian
John B. Matthews
Christian <fake...@xyz.de> wrote:
> John B. Matthews schrieb:
> > In article
> > <979ab8a4-e293-4524...@e18g2000yqo.googlegroups.com>,
> > Lew <l...@lewscanon.com> wrote:
> >
> >> Christian wrote:
> >>> Also comparison between 2 heap dumps shows that nearly 30.000
> >>> FileSystemViews more were alive then shortly after start (compared to 16
> >>> hours running)...
> > [...]
> >> Regardless, there is no evidence that 'FileSystemView' is a graphical
> >> object, and I expect that it isn't, nor that it hangs on to native
> >> (i.e., unmanaged) memory. That would explain its lack of a 'dispose
> >> ()' method.
> > [...]
> >> I suspect that 'FileSystemView' does not cause your memory "leak".
> >
> > Interesting, the abstract class FileSystemView has a single abstract
> > method, createNewFolder(). When I instantiate the class repeatedly with
> > an implementation returning null, there is no leak. In contrast,
> > repeatedly invoking the static factory leaks instances of
> > FileSystemView$1 linearly over time. This may be required by the
> > implementation, which isn't usually invoked tens of thousands of times
> > per day.
[...]
> > Perhaps the OP can use a single instance.
>
> I did...
> Changed that..
>
> About my usage:
> FileSystemView is used at 2 points in my application.
>
> The more often used one is:
> Skimming through the disc of a user below a certain diretory to find any
> file/directory that is not hidden.
>
> Now for each directory/subdirectory the FileSystemView$1 is created once..
> And a refresh of this list of files is done every hour.
>
> In the end that gives lots and lots of Objects...
>
> Though they are not my culprit.
> They are barely noticeable on the heap in the end.
> Also they seem not to carry any native storage with them.
I see what you mean. The first chart shows a nearly linear growth in
surviving generations, while the second shows that those thousands of
instances take almost no memory:
<http://sites.google.com/site/trashgod/garbageheap>
I look forward to reading more on any culprits you find.
Christian
>
> I see what you mean. The first chart shows a nearly linear growth in
> surviving generations, while the second shows that those thousands of
> instances take almost no memory:
>
> <http://sites.google.com/site/trashgod/garbageheap>
>
> I look forward to reading more on any culprits you find.
>
As you were looking forward to it.. I have created a SSCCE that shows
what I have finally found out to be my culprit...
It is the FileSystemView... though not in the expected way:
import java.io.File;
import javax.swing.filechooser.FileSystemView;
public class LeakTest {
private static long counter = 0;
private static FileSystemView view = FileSystemView.getFileSystemView();
public static void main(String[] args) throws InterruptedException {
while (true) {
for (File f : File.listRoots()) {
if (f.isDirectory()) {
countFiles(f);
System.out.println("found: "+counter);
}
}
System.out.println("found Total: "+counter);
counter = 0;
Thread.sleep(1000);
}
}
private static void countFiles(File root) {
for (File f :view.getFiles(root, true)) {
if (f.isDirectory()) {
countFiles(f);
} else {
counter++;
}
}
}
}
This little Program lets my Memory usage rise easily to 200 MiB in about
one Minute. And Heap as well as non-Heap memory stays nearly constant.
So well ... I have written a method that does nearly the same (just
without the memory leak..):
though its not exactly the same as File.isHidden() behaves not the same
as FileSystemView on hidden Files:
private static File[] getFiles(File parent,boolean useHidden) {
File[] files = parent.listFiles();
if (files == null) return new File[0];
if (useHidden) {
int length = files.length;
for (int i = 0; i < length; i++) {
if (files[i].isHidden()) {
length--;
System.arraycopy(files, i+1, files, i, length-i );
}
}
if (length != files.length) {
File[] onlyVisible = new File[length];
System.arraycopy(files, 0, onlyVisible, 0, length);
return onlyVisible;
}
}
return files;
}
Christian
Mark Space
Here's what your program did for me running in NetBeans:
run:
#
# An unexpected error has been detected by Java Runtime Environment:
#
# EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x763b03fb, pid=4964,
tid=1336
#
# Java VM: Java HotSpot(TM) Client VM (1.6.0-b105 mixed mode, sharing)
# Problematic frame:
# C [SHLWAPI.dll+0x203fb]
#
# An error report file with more information is saved as hs_err_pid4964.log
#
# If you would like to submit a bug report, please visit:
# http://java.sun.com/webapps/bugreport/crash.jsp
#
Java Result: 1
BUILD SUCCESSFUL (total time: 3 minutes 1 second)
blue indigo
*cough* Successful, my ass. It looks like the vm segged out. That,
obviously, constitutes a bug. And FileSystemView looks to be leaking
native memory, probably because of lack of a finalizer to clean it up, or
else it's packratting FDs or streams under the hood. That makes two bugs.
I'd use the above link to report the crash, and put a new bug in
separately "FileSystemView leaks native memory", both with the SSCCE and
possible a link to this thread on (ugh) google groups.
It's odd it crashed only in NetBeans. Maybe the debugging hooks are the
cause, interacting with the leaking native memory in some way. If the vm
is what crashed, though, the vm necessarily contains a bug.
This, and the "successful" message after a massive failure, have done
little to convince me to use NetBeans in preference to gvim. Nevermind
that NetBeans gobbles literally a hundred times as much disk and memory. :-)
--
blue indigo
UA Telecom since 1987
Christian
It works "fine" for me within eclipse...
Though I only tested till about 1 GiB of used RAM. Never got even close
to my system limits. (have 8 GiB of RAM/no pagefile)
I already reported the bug at sun... Though it has not yet a bugID ..
just a reviewID ..
Christian
John B. Matthews
Christian <fake...@xyz.de> wrote:
> John B. Matthews schrieb:
> >
> > I see what you mean. The first chart shows a nearly linear growth in
> > surviving generations, while the second shows that those thousands of
> > instances take almost no memory:
> >
> > <http://sites.google.com/site/trashgod/garbageheap>
> >
> > I look forward to reading more on any culprits you find.
>
> As you were looking forward to it.. I have created a SSCCE that shows
> what I have finally found out to be my culprit... It is the
> FileSystemView... though not in the expected way:
[...]
Thank you. On my system, file.isHidden() and view.isHiddenFile(file)
return the same result. Are the two results different on your system?
Perhaps I misunderstand, but why use FileSystemView at all?
import java.io.File;
import javax.swing.filechooser.FileSystemView;
public class LeakTest {
private static FileSystemView view =
FileSystemView.getFileSystemView();
private static int hiddenCount = 0;
private static int visibleCount = 0;
private static int anomalyCount = 0;
public static void main(String args[]) {
File root;
if (args.length > 0) root = new File(args[0]);
else root = new File(System.getProperty("user.dir"));
ls(root);
System.out.println("Visible: " + visibleCount);
System.out.println("Hidden: " + hiddenCount);
System.out.println("Anomalous: " + anomalyCount);
}
// traverse f recursively
private static void ls(File f) {
File list[] = f.listFiles();
for (File file : list) {
if (file.isDirectory()) ls(file);
else if (file.isHidden()) hiddenCount++;
else visibleCount++;
if (file.isHidden() != view.isHiddenFile(file))
anomalyCount++;
}
}
}
I see no anomalies in hundreds of thousands of files.
Christian
I used the following code for comparing:
public static File[] getFiles(File parent,boolean useHidden) {
File[] files = parent.listFiles();
if (files == null) return new File[0];
if (useHidden) {
int length = files.length;
for (int i = 0; i < length; i++) {
if (files[i].isHidden()) {
length--;
System.arraycopy(files, i+1, files, i, length-i );
}
}
if (length != files.length) {
File[] onlyVisible = new File[length];
System.arraycopy(files, 0, onlyVisible, 0, length);
return onlyVisible;
}
}
return files;
}
for me this getFiles returns different ammount of files than the
FileSystemView implementation...
Might be a Windows only issue... things like the trashcan are not
detected as hidden by File.isHidden() though getFiles() of file
system view does not return them...
Christian