Memory leak in kmeans.Compute?

[jfcote....@gmail.com]

Hi César,

I think there may be a memory leak somewhere in the Compute function of the object KMeans. I didn't look at the code itself since I'm just using the DLL.

In my application, I run a KMeans on a lot of picture in a loop to cluster them (like 800 pictures). When I put the kMeans.Compute fonction in comments, I don't have memory leak. When I remove the comment, the memory consumption start around 80 000 K and can grow up to 500 000 K, depending on the number of picture in the set. When the loop is finished, the memory don't go down, it stay there.

I've searched for a Dispose in the kmeans object but there is none. Here is the code (very simple). Put that in a loop and you should got the bug.

// Retrieve the number of clusters
int k = 3;

// Load original image
Bitmap image = _image.ToBitmap();

// Create conversors
ImageToArray imageToArray = new ImageToArray(min: 0, max: +255);
ArrayToImage arrayToImage = new ArrayToImage(image.Width, image.Height, min: 0, max: +255);

// Transform the image into an array of pixel values
double[][] pixels;
imageToArray.Convert(image, out pixels);

// Create a K-Means algorithm using given k and a
// square euclidean distance as distance metric.
KMeans kmeans = new KMeans(k, Accord.Math.Distance.SquareEuclidean);

// Compute the K-Means algorithm until the difference in
// cluster centroids between two iterations is below 0.05
kmeans.Randomize(pixels, true);
int[] idx = kmeans.Compute(pixels, 0.1);

// Replace every pixel with its corresponding centroid
pixels.ApplyInPlace((x, i) => kmeans.Clusters.Centroids[idx[i]]);

Since my application will need to work on video frame of very long video, it can't work if there is memory leak. Let me know what you think about it and if it's a real bug, I can make an entry in the bug system.

[César]

Hi there,

I took the K-Means sample application and put the call to compute inside a tight loop, but I couldn't reproduce the issue! May I ask if you can send a demo project so I can better visualize what is going on? If you would rather not post the project publicly, you can send it to me directly as well!

By the way, this could be totally unrelated, but if you are using a ToBitmap() method to extract frames from your video, make sure to call .Dispose on this bitmap before you exit the loop. Perhaps this could be producing some side effect as well!

Best regards,

Cesar

[jfcote....@gmail.com]

I'm doing the dispose() but I will revalidate everything.

When I'll get the time, I will create a sample project to reproduce the problem.

I don't know when I'll have the time, maybe next week!

Thanks!

[Jfcote]

Hi César!

 

I did a project on the side with the same code and the memory leak didn't show up.

 

Then, I though that the only difference between this little demo and my real application was the complexity. So maybe the Garbage Collector didn't have the time to collect. So I put a GC.Collect() at the right place in my loop and it fix everything!

 

Thanks a lot for your help!

[César]

Hi there!

I am glad you got it working! But well, I guess this would still qualify as a bug. Perhaps the framework is holding more managed references than necessary. I will try to investigate again, but the best way to diagnose this would surely be by using a memory profiler. Is there any chance I can get a look at your full project (or perhaps at a smaller version of it)?

Best regards,

Cesar

[Jfcote]

Hi César,

 

I'm not sure it's related entirely to Accord.NET framework. I also use EmguCV and use the ToBitmap() function. I do a "Dispose()" but like I said, it seems that sometime the GC just takes a LOT of time before being called. But maybe you are right too because most of the memory consumption were on the line of the kmean function and the line before where it's output to an array of pixel.

 

The little demo I've done doesn't have any problem at all, even without call to the GC.

 

Unfortunately, I can't send you my main project because it's part of a R&D project in the company I work for and I can't disclose any information about it. Sorry about that! But I'm pretty confident that your code is ok and maybe it's the complexity of mixing different framework together that created this "leak".

 

Thanks and have a nice weekend.