matlab/mex function doesn't free memory
Alex Berkovich
I'm running a mex function that calls for c code (compiled in the same file as the mex function itself).
when a simulation is done, it seems matlab doesn't release the memory it uses since i see a growing amount of memory being used by matlab (via task manager).
it reaches the limit of my computer and then either crashes or simply continues to run forever.
any suggestion on how to solve this problem is very appriciated.
Bruno Luong
Bruno
Alex Berkovich
should I free memory / variables / do other actions, other that free to any variable that i did memory allocation to?
thanks a lot!
Bruno Luong
> should I free memory / variables / do other actions, other that free to any variable that i did memory allocation to?
>
Call free and use MxDestroyArray, and you should be fine. I believe the later is even not necessary because Matlab should automatically destroy the one use forget to do. I always do that manually though.
Bruno
Rune Allnor
Any data structures you allocate with malloc-type functions
(or with new in C++) should be freed by the appropriate
corresponding calls to free or delete before your MEX function
returns.
As for mxCreateXXXX, only use them for whatever you intend to
return back to the matlab workspace. I've never understood how
the matlab memory managment system works (it seems to be based
on some Java-type garbage collector) and I've always ended up
with segmentation faults when I used mxFree on local variables
inside the MEX function.
Rune
Praetorian
If you're using mxFree on variables created using mxCreate* functions
it'll always result in a memory leak. mxFree will only free the
mxArray header and not the data pointed to by the header. Use
mxDestroyArray to free variables allocated using mxCreate* and mxFree
for those allocated using mxCalloc, mxMalloc and mxRealloc. I only use
the MEX API functions to manage memory in C mex functions and it works
just fine.
However, in case of C++ files it's better to use new except for
mxArray variables as you mentioned, and then free these using delete
in the class destructor. The MEX API does do it's own garbage
collection whenever MATLAB terminates a MEX file (because of a call to
mexErrMsgIdAndTxt for instance) before the mexFunction stack is
unwound. In this case trying to free memory using mxDestroyArray or
mxFree in the destructor causes a segv (since the memory has already
been freed by the garbage collector).
Ronan
> On Dec 14, 2:35=A0pm, Rune Allnor <all...@tele.ntnu.no> wrote:
> > On 14 Des, 19:09, "Alex Berkovich" <alex...@tx.technion> wrote:
> >
> > > Hello all,
> >
> file as the mex function itself).
> t uses since i see a growing amount of memory being used by matlab (via tas=
> k manager).
> >
> > > it reaches the limit of my computer and then either crashes or simply c=
I have a question close to that one..
You said that "The MEX API does do it's own garbage collection whenever MATLAB terminates a MEX file", fine, but what about a function called in a shared library (via calllib) that return a mxArray* and thus, alloc via mxCreate functions ? Is this mxArray destoyed in C++ workspace ?
Thanks..
gabo
Hi all,
It DOES seem that MatLab leaks memory when running mex files. Take a look at the following: Below is a c-mex file (trace2plane.c) that finds intersections between hyperplanes and polygonal-line traces. Run the test code (test_trace2plane.m, down at the bottom) several times and you'll discover MatLab is taking up more and more memory in the task manager. I've been trying to figure out the problem for almost a month now, any insigths or help of any kind will be much appreciated.
Gabriel
/* trace2plane.c */
#include "mex.h"
#include <math.h>
#define min(a, b) (((a) < (b)) ? (a) : (b))
#define max(a, b) (((a) > (b)) ? (a) : (b))
#define sqr(a) ((a)*(a))
#define abs(a) (((a) > (0)) ? (a) : (-a))
#define sgn(a) (((a) > (0)) ? (1) : (-1))
/* Gateway foutine. */
void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[]) {
/* Declare variables. */
int i, j, k;
double *center, *direction, *trace;
double **position, **velocity, *location, *distance;
int **member;
double *pointer, dot, norm, delta;
int length, dimension, total, product, index;
int size = 0, count = 0, increment;
char buffer[100];
/* Check argument count. */
if (nrhs < 3)
mexErrMsgTxt("Not enough inputs.");
if (nlhs > 5)
mexErrMsgTxt("Too many outputs.");
/* Check input class. */
for (i = 0; i < nrhs; i++) {
if (!(mxIsNumeric(prhs[i]))) {
sprintf(buffer, "Input %d must be numeric.", i + 1);
mexErrMsgTxt(buffer);
}
if (mxIsComplex(prhs[i])) {
sprintf(buffer, "Input %d must be real.", i + 1);
mexErrMsgTxt(buffer);
}
if (!(mxIsDouble(prhs[i]))) {
sprintf(buffer, "Input %d must be of class double.", i + 1);
mexErrMsgTxt(buffer);
}
}
/* Check input size. */
dimension = mxGetN(prhs[0]);
for (i = 0; i < nrhs; i++) {
if (mxGetNumberOfDimensions(prhs[i]) > 2) {
sprintf(buffer, "Input %d must be a matrix.", i + 1);
mexErrMsgTxt(buffer);
}
if (mxGetN(prhs[i]) != dimension) {
sprintf(buffer, "Input %d must have %d columns.", i + 1, dimension);
mexErrMsgTxt(buffer);
}
}
total = mxGetM(prhs[0]);
if (mxGetM(prhs[1]) != total) {
sprintf(buffer, "Input 2 must have %d rows.", total);
mexErrMsgTxt(buffer);
}
product = mxGetNumberOfElements(prhs[0]);
/* Check for early return. */
if (mxIsEmpty(prhs[0]) || mxIsEmpty(prhs[2])) {
plhs[0] = mxCreateNumericMatrix(count, dimension, mxDOUBLE_CLASS, mxREAL);
if (nlhs > 1) {
plhs[1] = mxCreateNumericMatrix(count, dimension, mxDOUBLE_CLASS, mxREAL);
if (nlhs > 2) {
plhs[2] = mxCreateNumericMatrix(count, 2, mxDOUBLE_CLASS, mxREAL);
if (nlhs > 3) {
plhs[3] = mxCreateNumericMatrix(count, 1, mxDOUBLE_CLASS, mxREAL);
if (nlhs > 4) {
plhs[4] = mxCreateNumericMatrix(count, 1, mxDOUBLE_CLASS, mxREAL);
}
}
}
}
return;
}
/* Allocate space. */
increment = nrhs - 2;
size = increment;
position = mxMalloc(size*sizeof(double*));
velocity = mxMalloc(size*sizeof(double*));
member = mxMalloc(size*sizeof(int*));
for (i = 0; i < size; i++) {
position[i] = mxMalloc(dimension*sizeof(double));
velocity[i] = mxMalloc(dimension*sizeof(double));
member[i] = mxMalloc(2*sizeof(int));
}
location = mxMalloc(size*sizeof(double));
distance = mxMalloc(size*sizeof(double));
/* Get pointer. */
center = mxGetPr(prhs[0]);
direction = mxGetPr(prhs[1]);
for (i = 0; i < total; i++) {
/* Calculate projections. */
for (j = 2; j < nrhs; j++) {
trace = mxGetPr(prhs[j]);
length = mxGetM(prhs[j]);
for (k = 0; k < length - 1; k++) {
dot = 0.0;
delta = 0.0;
pointer = trace + k;
for (index = i; index < product; index += total, pointer += length) {
dot += direction[index]*(center[index] - *pointer);
delta += direction[index]*(*(pointer + 1) - *pointer);
}
if (delta != 0.0) {
dot /= delta;
if ((dot > 0.0) && (dot <= 1.0)) {
/* Allocate more space if necessary. */
if (count >= size) {
size += increment;
position = mxRealloc(position, size*sizeof(double*));
velocity = mxRealloc(velocity, size*sizeof(double*));
member = mxRealloc(member, size*sizeof(int*));
for (index = size - increment; index < size; index++) {
position[index] = mxMalloc(dimension*sizeof(double));
velocity[index] = mxMalloc(dimension*sizeof(double));
member[index] = mxMalloc(2*sizeof(int));
}
location = mxRealloc(location, size*sizeof(double));
distance = mxRealloc(distance, size*sizeof(double));
}
/* Append. */
norm = 0.0;
pointer = trace + k;
for (index = 0; index < dimension; index++, pointer += length) {
delta = *(pointer + 1) - *pointer;
position[count][index] = *pointer + dot*delta;
velocity[count][index] = delta;
delta = center[i + index*dimension] - position[count][index];
norm += sqr(delta);
}
member[count][0] = i + 1;
member[count][1] = j - 1;
location[count] = dot + (double) (k + 1);
distance[count] = sqrt(norm);
count++;
}
}
}
}
/* Trim. */
for (i = size - count; k < size; k++) {
mxFree(position[i]);
mxFree(velocity[i]);
mxFree(member[i]);
}
position = mxRealloc(position, count*sizeof(double*));
velocity = mxRealloc(velocity, count*sizeof(double*));
member = mxRealloc(member, count*sizeof(int*));
location = mxRealloc(location, count*sizeof(double));
distance = mxRealloc(distance, count*sizeof(double));
}
/* Fill in position. */
plhs[0] = mxCreateNumericMatrix(count, dimension, mxDOUBLE_CLASS, mxREAL);
pointer = mxGetPr(plhs[0]);
for (j = 0; j < dimension; j++)
for (i = 0; i < count; i++) {
*pointer = position[i][j];
pointer++;
}
if (nlhs > 1) {
/* Fill in velocity. */
plhs[1] = mxCreateNumericMatrix(count, dimension, mxDOUBLE_CLASS, mxREAL);
pointer = mxGetPr(plhs[1]);
for (j = 0; j < dimension; j++)
for (i = 0; i < count; i++) {
*pointer = velocity[i][j];
pointer++;
}
if (nlhs > 2) {
/* Fill in member. */
plhs[2] = mxCreateNumericMatrix(count, 2, mxDOUBLE_CLASS, mxREAL);
pointer = mxGetPr(plhs[2]);
for (j = 0; j < 2; j++)
for (i = 0; i < count; i++) {
*pointer = (double) member[i][j];
pointer++;
}
if (nlhs > 3) {
/* Fill in location. */
plhs[3] = mxCreateNumericMatrix(count, 1, mxDOUBLE_CLASS, mxREAL);
pointer = mxGetPr(plhs[3]);
for (i = 0; i < count; i++)
pointer[i] = location[i];
if (nlhs > 4) {
/* Fill in distance. */
plhs[4] = mxCreateNumericMatrix(count, 1, mxDOUBLE_CLASS, mxREAL);
pointer = mxGetPr(plhs[4]);
for (i = 0; i < count; i++)
pointer[i] = distance[i];
}
}
}
}
/* Free space. */
for (i = 0; i < size; i++) {
mxFree(position[i]);
mxFree(velocity[i]);
mxFree(member[i]);
}
mxFree(position);
mxFree(velocity);
mxFree(member);
mxFree(location);
mxFree(distance);
}
function test_trace2plane
%-------------------------------------------------------------------------%
% First case %
%-------------------------------------------------------------------------%
% Generate data.
length = 5;
dimension = 2;
center = zeros(1, dimension);
direction = randn(1, dimension);
trace = randn(length, dimension);
trace = cumsum(trace, 1);
trace = bsxfun(@minus, trace, mean(trace, 1));
trace = bsxfun(@times, trace, 1./std(trace, 1, 1));
% Project.
[position, velocity, member, location, distance] = ...
trace2plane(center, direction, trace);
% Create line.
x = transpose(linspace(-1.0, 1.0, length));
[q, r] = qr(transpose(direction));
i = eq(r, 0.0);
x = bsxfun(@plus, x*q(i, :), center);
% Plot.
figure
set(gca, ...
'Color', get(gcf, 'Color'), ...
'XGrid', 'on', ...
'YGrid', 'on', ...
'DataAspectRatio', ones(1, 3), ...
'NextPlot', 'add')
quiver(center(:, 1), center(:, 2), ...
direction(:, 1), direction(:, 2), ...
'Color', 'r', ...
'Marker', '.', ...
'AutoScale', 'off')
plot(x(:, 1), x(:, 2), ...
'Color', 'r', ...
'LineStyle', '--')
plot(trace(:, 1), trace(:, 2),...
'Color', 'b', ...
'Marker', '.')
plot(transpose(cat(2, ...
center(ones(size(position, 1), 1), 1), ...
position(:, 1))), ...
transpose(cat(2, ...
center(ones(size(position, 1), 1), 2), ...
position(:, 2))), ...
'Color', 'y', ...
'Marker', 'none', ...
'LineStyle', '-')
for i = 1:length
text(trace(i, 1), trace(i, 2), ...
sprintf('%d', i), ...
'HorizontalAlignment', 'Left', ...
'VerticalAlignment', 'Bottom')
end
for i = 1:size(position, 1)
text(position(i, 1), position(i, 2), ...
sprintf('%4.4f', location(i)), ...
'Color', 'g', ...
'HorizontalAlignment', 'Right', ...
'VerticalAlignment', 'Top')
text(0.5*(center(1) + position(i, 1)), ...
0.5*(center(2) + position(i, 2)), ...
sprintf('%4.4f', distance(i)), ...
'Color', 'y', ...
'HorizontalAlignment', 'Right', ...
'VerticalAlignment', 'Top')
end
plot(position(:, 1), position(:, 2), ...
'Color', 'g', ...
'Marker', '.', ...
'LineStyle', 'none')
%-------------------------------------------------------------------------%
% Second case %
%-------------------------------------------------------------------------%
% Generate data.
length = 5;
dimension = 3;
center = zeros(1, dimension);
direction = randn(1, dimension);
trace = randn(length, dimension);
trace = cumsum(trace, 1);
trace = bsxfun(@minus, trace, mean(trace, 1));
trace = bsxfun(@times, trace, 1./std(trace, 1, 1));
% Project.
[position, velocity, member, location, distance] = ...
trace2plane(center, direction, trace);
% Create plane.
x = cell(1, 2);
[x{:}] = meshgrid(transpose(linspace(-1.0, 1.0, length)));
for i = 1:2
x{i} = x{i}(:);
end
x = cat(2, x{:});
[q, r] = qr(transpose(direction));
i = eq(r, 0.0);
x = bsxfun(@plus, x*q(i, :), center);
x = mat2cell(x, length^2, ones(1, 3));
for i = 1:3
x{i} = reshape(x{i}, length, length);
end
% Plot.
figure
set(gca, ...
'Color', get(gcf, 'Color'), ...
'XGrid', 'on', ...
'YGrid', 'on', ...
'ZGrid', 'on', ...
'DataAspectRatio', ones(1, 3), ...
'NextPlot', 'add')
quiver3(center(:, 1), center(:, 2), center(:, 3), ...
direction(:, 1), direction(:, 2), direction(:, 3), ...
'Color', 'r', ...
'Marker', '.', ...
'AutoScale', 'off')
surf(x{1}, x{2}, x{3}, ...
'EdgeColor', 'r', ...
'FaceColor', 'none')
plot3(trace(:, 1), trace(:, 2), trace(:, 3), ...
'Color', 'b', ...
'Marker', '.')
plot3(transpose(cat(2, ...
center(ones(size(position, 1), 1), 1), ...
position(:, 1))), ...
transpose(cat(2, ...
center(ones(size(position, 1), 1), 2), ...
position(:, 2))), ...
transpose(cat(2, ...
center(ones(size(position, 1), 1), 3), ...
position(:, 3))), ...
'Color', 'y', ...
'Marker', 'none', ...
'LineStyle', '-')
for i = 1:length
text(trace(i, 1), trace(i, 2), trace(i, 3), ...
sprintf('%d', i), ...
'HorizontalAlignment', 'Left', ...
'VerticalAlignment', 'Bottom')
end
for i = 1:size(position, 1)
text(position(i, 1), position(i, 2), position(i, 3), ...
sprintf('%4.4f', location(i)), ...
'Color', 'g', ...
'HorizontalAlignment', 'Right', ...
'VerticalAlignment', 'Top')
text(0.5*(center(1) + position(i, 1)), ...
0.5*(center(2) + position(i, 2)), ...
0.5*(center(3) + position(i, 3)), ...
sprintf('%4.4f', distance(i)), ...
'Color', 'y', ...
'HorizontalAlignment', 'Right', ...
'VerticalAlignment', 'Top')
end
plot3(position(:, 1), position(:, 2), position(:, 3), ...
'Color', 'g', ...
'Marker', '.', ...
'LineStyle', 'none')
return
James Tursa
> "Alex Berkovich" <ale...@tx.technion> wrote in message <gi3i3u$m5q$1...@fred.mathworks.com>...
> > Hello all,
> >
> > I'm running a mex function that calls for c code (compiled in the same file as the mex function itself).
> > when a simulation is done, it seems matlab doesn't release the memory it uses since i see a growing amount of memory being used by matlab (via task manager).
> >
> > it reaches the limit of my computer and then either crashes or simply continues to run forever.
> >
> > any suggestion on how to solve this problem is very appriciated.
> >
>
> Hi all,
>
> It DOES seem that MatLab leaks memory when running mex files. Take a look at the following: Below is a c-mex file (trace2plane.c) that finds intersections between hyperplanes and polygonal-line traces. Run the test code (test_trace2plane.m, down at the bottom) several times and you'll discover MatLab is taking up more and more memory in the task manager. I've been trying to figure out the problem for almost a month now, any insigths or help of any kind will be much appreciated.
>
> Gabriel
>
Here are my results:
>> mex trace2plane.c
>> feature memstats
1505 MB (5e1d1000)
>> test_trace2plane
>> feature memstats
1505 MB (5e1d1000)
>> test_trace2plane
>> feature memstats
1505 MB (5e1d1000)
>> test_trace2plane
>> feature memstats
1505 MB (5e1d1000)
>> test_trace2plane
>> feature memstats
1505 MB (5e1d1000)
>> test_trace2plane
>> feature memstats
1505 MB (5e1d1000)
>> test_trace2plane
>> feature memstats
1505 MB (5e1d1000)
>> test_trace2plane
>> feature memstats
1505 MB (5e1d1000)
>>
Other than a bunch of figures building up, I don't see any large increases in memory usage from call to call. So I can't reproduce your problem. WinXP with MSVC 8.0. How many times do you have to run it to see a problem? How much memory are you losing with each call?
James Tursa
Praetorian
wrote:
> "gabo " <pelusacora...@hotmail.com> wrote in message <grgne9$1n...@fred.mathworks.com>...
> > "Alex Berkovich" <alex...@tx.technion> wrote in message <gi3i3u$m5...@fred.mathworks.com>...
James,
I was hoping you'd see this thread. Take a look at the post from Ronan
on the 4th of February. It was directed to me but I didn't see it
until last night. Do you have any experience with calling a DLL that
uses the mxCreate* functions to allocate memory? And does the MATLAB
memory manager garbage collect that memory assuming that the DLL is
loaded and used by a mex file?
Regards,
Ashish.
James Tursa
>
> James,
> I was hoping you'd see this thread. Take a look at the post from Ronan
> on the 4th of February. It was directed to me but I didn't see it
> until last night. Do you have any experience with calling a DLL that
> uses the mxCreate* functions to allocate memory? And does the MATLAB
> memory manager garbage collect that memory assuming that the DLL is
> loaded and used by a mex file?
>
> Regards,
> Ashish.
Well, I didn't have any experience with this when I first read this post, but now I do. The answer is no, MATLAB does *not* do garbage collection on mx___ allocated memory in a loaded dll. This is different from mex functions, where garbage collection *is* done. Example files are below. Commands I used to show the memory leak are:
mex mexmem.c
mexmem
feature memstats
This will show the garbage collection being done for mex functions. Inside the function, 100MB is allocated but not explicitly deallocated. The feature memstats done inside the mex routine show the allocation. But upon returning to MATLAB and doing a feature memstats reveals that the memory is in fact freed.
Now build the dllmem dll (I used a mex command with a custom mexopts.bat file to build the dll library) and then do this:
loadlibrary('dllmem.mexw32','dllmem.h')
feature memstats
calllib('dllmem','memallocate')
feature memstats
unloadlibrary('dllmem')
feature memstats
You can see that the 100MB has been leaked. Garbage collection did *not* take place when the dll library was unloaded.
Bottom line: Free the memory manually, no safety net here.
James Tursa
--------------------------------------------------------------------------------------------------
mexmem.c
#include "mex.h"
double *dp = NULL;
void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[])
{
mexEvalString("feature memstats");
if( dp == NULL ) {
dp = mxMalloc(13107200*sizeof(double)); // allocate 100MB
}
mexEvalString("feature memstats");
}
-------------------------------------------------------------------------------------------------------
dllmem.c
#include "mex.h"
double *dp = NULL;
void memallocate(void)
{
if( dp == NULL ) {
dp = mxMalloc(13107200*sizeof(double)); // allocate 100MB
}
}
void memdeallocate(void)
{
if( dp ) {
mxFree(dp);
}
dp = NULL;
}
-----------------------------------------------------------------------------------------------------
dllmem.h
void memallocate(void);
void memdeallocate(void);
----------------------------------------------------------------------------------------------------
dllmem.def
EXPORTS
memallocate
memdeallocate
Praetorian
wrote:
> Praetorian <ashish.sadanan...@gmail.com> wrote in message <251c8005-d072-419f-ad9a-28680bdb1...@g19g2000yql.googlegroups.com>...
Thanks for coding up that example! So I went ahead and did things a
little differently; instead of using CALLLIB from the command line I
loaded the DLL using the Windows API LoadLibrary function from within
mexmem. And what do you know, MATLAB garbage collects the memory in
that case. The only other difference was that I created a Visual
Studio project to compile my DLL instead of modifying mexopts.bat but
I can't think of any project setting that would cause this change in
behavior. Here's my modified mexmem.c
------------------------------------------------------------------------------------------------------
mexmem.c
#include "mex.h"
#include "windows.h"
double *dp = NULL;
/* mexmem - allocates memory within mexFunction, does not
free it
* mexmem( false ) - same as first call
* mexmem( true ) - calls dllmem.dll which allocates memory, does
not free it
* mexmem( true, ... ) - calls dllmem.dll which deallocates memory
*/
void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray
*prhs[])
{
mexEvalString("feature memstats");
if( ( nrhs == 0 ) || ( !mxIsLogicalScalarTrue(prhs[0]) ) ) {
if( dp == NULL ) {
dp = mxMalloc(100*1024*1024*sizeof(double)); //100MB * sizeof
(double)
mexPrintf( "Memory allocated at: %px\n", dp );
} else {
mexPrintf( "Memory not allocated\n" );
}
} else {
HMODULE hLib = LoadLibrary( "dllmem.dll" );
mexPrintf( "Loading dllmem\n" );
if( hLib == NULL ) {
mexPrintf( "Failed to load dllmem" );
} else {
FARPROC pAlloc = GetProcAddress( hLib, "memallocate" );
FARPROC pDealloc = GetProcAddress( hLib, "memdeallocate" );
if( pAlloc == NULL ) {
mexPrintf( "Failed to get memallocate address" );
} else if( pDealloc == NULL ) {
mexPrintf( "Failed to get memdeallocate address" );
}
if( nrhs > 1 ) {
(pDealloc)();
} else {
(pAlloc)();
}
if( FreeLibrary( hLib ) == 0 ) {
mexPrintf( "Failed to unload dllmem" );
}
}
}
mexEvalString("feature memstats");
}
------------------------------------------------------------------------------------
The calls I made at the command line were:
%To test allocation within mexmem
mexmem
feature memstats
[x x] = inmem;
any(ismember('mexmem', x))
%To test allocation within dllmem
mexmem( true )
feature memstats
[x x] = inmem;
any(ismember('mexmem', x))
I added the INMEM check because I was always under the impression that
MATLAB didn't do any mex garbage collection until the mex file was
cleared from memory. But these tests prove otherwise! Also, I tried
commenting out the FreeLibrary call to see what would happen if the
DLL remained in memory but that gave the same results.
Any thoughts on why CALLLIB doesn't work the same way?
Regards,
Ashish.
Praetorian
Oh and I forgot to mention that any attempt to have dllmem.dll
deallocate the memory results in a segv, which is to be expected.
James Tursa
>
> Thanks for coding up that example! So I went ahead and did things a
> little differently; instead of using CALLLIB from the command line I
> loaded the DLL using the Windows API LoadLibrary function from within
> mexmem. And what do you know, MATLAB garbage collects the memory in
> that case. The only other difference was that I created a Visual
> Studio project to compile my DLL instead of modifying mexopts.bat but
> I can't think of any project setting that would cause this change in
> behavior. Here's my modified mexmem.c
>
>
> I added the INMEM check because I was always under the impression that
> MATLAB didn't do any mex garbage collection until the mex file was
> cleared from memory. But these tests prove otherwise! Also, I tried
> commenting out the FreeLibrary call to see what would happen if the
> DLL remained in memory but that gave the same results.
>
> Any thoughts on why CALLLIB doesn't work the same way?
>
> Regards,
> Ashish.
Ashish,
I haven't had a chance to try out your example yet ... maybe this weekend. But I did modify some other dll code that I have that is a basic memory sharing example. Called it using Windows API functions from within a mex routine. The dll has functions for allocating memory, copying data to the memory, displaying the memory, and deallocating the memory. The dll works great using malloc and free, but when I switch to mxMalloc and mxFree the dll craps out with a seg fault. This doesn't always happen right away, but it always happens eventually. So I suspect that there is a fundamental restriction here that you can't use MATLAB API functions inside a shared library dll because it messes up the MATLAB memory manager in some way ... they are probably walking on each other. You apparently experienced seg faults as well when deallocating with one of your examples. So the garbage collection issue
is probably a moot point for shared library dll's since using any of the MATLAB API functions that allocate or deallocate memory will screw up the MATLAB memory manager and cause a seg fault.
James Tursa
Praetorian
wrote:
> Praetorian <ashish.sadanan...@gmail.com> wrote in message <a018942f-5682-44ca-82bf-ea96eb58c...@a23g2000vbl.googlegroups.com>...
I'm not so sure you can't use the mxMalloc and mxFree from within a
shared library. I modified the code so that I was calling the DLL's
allocate function followed immediately by the deallocate function; I
then called the mex file from the command line 1000 times and didn't
get any seg faults. The reason I got one yesterday was because I
allocated memory using the DLL in one call to the mex file and then
tried to deallocate it during the next call. But my tests from
yesterday show that the Matlab memory manager does garbage collection
on the memory allocated by the DLL when the calling mex file exits. So
the memory allocated in the first call had already been deallocated
(by the memory manager) and I tried to explicitly deallocate it in the
next call resulting in the segv.
Ashish.
James Tursa
>
> I'm not so sure you can't use the mxMalloc and mxFree from within a
> shared library. I modified the code so that I was calling the DLL's
> allocate function followed immediately by the deallocate function; I
> then called the mex file from the command line 1000 times and didn't
> get any seg faults. The reason I got one yesterday was because I
> allocated memory using the DLL in one call to the mex file and then
> tried to deallocate it during the next call. But my tests from
> yesterday show that the Matlab memory manager does garbage collection
> on the memory allocated by the DLL when the calling mex file exits. So
> the memory allocated in the first call had already been deallocated
> (by the memory manager) and I tried to explicitly deallocate it in the
> next call resulting in the segv.
>
> Ashish.
Ah, got it. Thanks! I put in a mexMakeMemoryPersistent call in the dll and now everything works ok. That puts the burdon of clearing the memory on the dll programmer, but that's really as it should be. I tested my example using both API calls from within a mex and from the MATLAB prompt using loadlibrary and calllib. Works in both cases, no memory leak and no memory corruption. The dll detects when it is being detached from MATLAB, so the programmer can put code in the dll to automatically clear the memory in that event.
James Tursa
Xueming
Currently I meet exactly the same problem. I wrote a function in C which returns mxArray* to Matlab, and this function is called in a shared library as well, and it will be called continuously. To free memory which occupied by return mxArray I wrote a function ( using mxDestroyArray), however, Matlab crashed, and seems it cannot find the location of returned mxArray. Is that becaues Matlab has it own garbage collection and after every call of function, it frees the memory occupied by returned mxArray?
Have you figured this issue out? I will appreciate a lot if you can give me any hint:)
James Tursa
> Currently I meet exactly the same problem. I wrote a function in C which returns mxArray* to Matlab, and this function is called in a shared library as well, and it will be called continuously. To free memory which occupied by return mxArray I wrote a function ( using mxDestroyArray), however, Matlab crashed, and seems it cannot find the location of returned mxArray. Is that becaues Matlab has it own garbage collection and after every call of function, it frees the memory occupied by returned mxArray?
> Have you figured this issue out? I will appreciate a lot if you can give me any hint:)
James Tursa
Xueming
Now I have two issues,
1) I create a C function, which will be called by Matlab, and when everytime it's called, it create a mxArray and return this pointer to Matlab. With called more and more times, it creates more and more mxArray. I guess the memory must be significantly occupied. I'm thinking if we can mxDestory a mxArray (by writing another function) as soon as it's returned to Matlab, so that we only allocate memory for current mxArray.
2)How about inside C code, if declaring a global mxArray*, how to make this global pointer point to a mxArray which is returned by a C function.
I tried to declare a global mxArray*, and set it point to a mxArray and returned to Matlab. This global mxArray* is used in other functions. But it seems in other functions, the global mxArray pointer could not be found where it points to. I got some strange result. Could anyone tell me the reason. Here is a simple code.
if run this code, calllib('mxArrayTest', 'create'), calllib('mxArrayTest','test'); the result of retruing mxCell1 and mxCell_g are completely different. The free_mxelem(mxCell_g) works pretty well just following mxCell_g=mxCell1, however, outside the "create" function, the free_mxelem() shows pretty strange result. I assume this is because after C returns mxArray to Matlab, memory of mxArray will be free automatically.
here is my code:
/////////////////mxArrayTest.c
#include "mex.h" /* only needed because of mexFunction below and mexPrintf */
#include <math.h>
#include "mxArrayTest.h"
#include "string.h"
/*******************function declare***********************************/
#define mxNULL mxCreateDoubleMatrix(0,0,mxREAL)
mxArray *mxCell_g;
/*type check*/
void type_check(void)
{
int num;
int i;
mxCell_g=mxNULL;
num=mxGetN(mxCell_g);
if(mxIsEmpty(mxCell_g))
{
mexPrintf("Empty");
}
else
{
for(i=0;i<num;i++)
{
mexPrintf("type is: %s",mxGetClassName(mxCell_g));
}
}
}
/********************get element************************************/
void getelement(int pos,mxArray* mxCell_g)
{
if(mxIsEmpty(mxCell_g))
{
mexPrintf("No element!\n");
}
else if(mxIsCell(mxGetCell(mxCell_g,pos)))
{
mexPrintf("This element is a cell!\n ");
}
else
{
mexPrintf("This elem is other type!\n");
}
}
/********************************************************/
int free_mxelem(mxArray* mxelem)
{
if(mxIsEmpty(mxelem))
{
mexWarnMsgTxt("Empty Element!\n");
return 0;
}
else if(mxIsChar(mxelem)||mxIsDouble(mxelem))
{
mxDestroyArray(mxelem);
mexPrintf("Single Successful!\n");
return 1;
}
// else if(mxIsCell(mxelem))
else if(mxIsCell(mxelem))
{
int n;//number of element
int i;//counter
n=mxGetN(mxelem);
mexPrintf("Free Cell!\n");
for(i=0;i<n;i++)
{
free_mxelem(mxGetCell(mxelem,i));
}
mxDestroyArray(mxelem);
mexPrintf("Cell Successful!\n");
return 1;
}
return 0;
}
/**************test******************************/
DLL_EXPORT mxArray* create(void)
{
mxArray *mxDouble;
mxArray *mxCell;
mxArray *mxCell1;
mxArray *mxString;
double *a;
char *str;
mxDouble=mxCreateDoubleMatrix(1,2,mxREAL);
a=mxGetPr(mxDouble);
a[0]=1;
a[1]=2;
str=(char*)malloc(sizeof(char)*20);
strcpy(str,"mxArray Test");
mxString=mxCreateString(str);
mxCell=mxCreateCellMatrix(1,2);
mxSetCell(mxCell,0,mxDouble);
mxSetCell(mxCell,1,mxString);
mxCell1=mxCreateCellMatrix(1,2);
mxSetCell(mxCell1,0,mxCell);
mxSetCell(mxCell1,1,mxString);
mxCell_g=mxCell1;
// free_mxelem(mxCell_g);
mexPrintf("is this empty? %d\nClass: %s\n",mxIsEmpty(mxCell_g),mxGetClassName(mxCell_g));
// return mxCell_g;
return mxCell1;
}
DLL_EXPORT void test(void)
{
int free_mxelem(mxArray* mxelem);
mexPrintf("is this empty? %d\n%s\n",mxIsEmpty(mxCell_g),mxGetClassName(mxCell_g));
free_mxelem(mxCell_g);
}
////////////////////////mxArrayTest.h
#ifndef _MXARRAYTEST_H_
#define _MXARRAYTEST_H_
#define DLL_EXPORT __declspec(dllexport)
#ifdef __cplusplus
extern "C" {
#endif
#include "mex.h"
//free_element
//DLL_EXPORT void free_elem(void);
/*test*/
DLL_EXPORT void test(void);
DLL_EXPORT mxArray* create(void);
DLL_EXPORT void type_check(void);
#ifdef __cplusplus
}
#endif
#endif
/////////////////end
If this piece of code and this message are messy for, I do apologize, since I'm a really beginer with this.
Emily
James Tursa
> Hi James,
> Now I have two issues,
> 1) I create a C function, which will be called by Matlab, and when everytime it's called, it create a mxArray and return this pointer to Matlab. With called more and more times, it creates more and more mxArray. I guess the memory must be significantly occupied. I'm thinking if we can mxDestory a mxArray (by writing another function) as soon as it's returned to Matlab, so that we only allocate memory for current mxArray.
> 2)How about inside C code, if declaring a global mxArray*, how to make this global pointer point to a mxArray which is returned by a C function.
> I tried to declare a global mxArray*, and set it point to a mxArray and returned to Matlab. This global mxArray* is used in other functions. But it seems in other functions, the global mxArray pointer could not be found where it points to. I got some strange result. Could anyone tell me the reason. Here is a simple code.
> if run this code, calllib('mxArrayTest', 'create'), calllib('mxArrayTest','test'); the result of retruing mxCell1 and mxCell_g are completely different. The free_mxelem(mxCell_g) works pretty well just following mxCell_g=mxCell1, however, outside the "create" function, the free_mxelem() shows pretty strange result. I assume this is because after C returns mxArray to Matlab, memory of mxArray will be free automatically.
> mxCell1=mxCreateCellMatrix(1,2);
> mxSetCell(mxCell1,0,mxCell);
> mxSetCell(mxCell1,1,mxString);
>
> mxCell_g=mxCell1;
Your mxCell_g is your global mxArray that you want to persist between calls. So you need to follow up the above code with this line which will remove mxCell_g from the VAL:
mexMakeArrayPersistent(mxCell_g);
CAUTION: Once you make the above call, mxCell_g is permanently removed from the VAL and there is no way to get it back on the VAL that I know of. So you risk a permanent memory leak if you don't have adequate code in place to destroy this mxArray. E.g., a function within the library to destroy this mxArray (so you can destroy it manually), and code in place that detects when the library is being detached from MATLAB (e.g., unloaded from memory) so that it can destroy the mxArray prior to being unloaded. Etc.
(btw, There is a similar function, mexMakeMemoryPersistent, for raw memory allocated with mxMalloc and friends that removes it from garbage collection. In that case, there *is* an undocumented function to get it back on the garbage collection list.)
Another suggestion I will make is to keep your global pointers in a valid state at all times. E.g., do this declaration instead of the one you have:
mxArray *mxCell_g = NULL;
That way it starts in a testable state indicating there is no variable present. When you create a variable, always test mxCell_g first, and if it is not NULL then destroy it before replacing it with another allocated mxArray. And whenever you destroy the variable in mxCell_g, set mxCell_g = NULL. Etc. That way you can always test the value of mxCell_g anywhere in your code to determine its state and if there is a variable present.
Also, I would note that the following code:
> str=(char*)malloc(sizeof(char)*20);
> strcpy(str,"mxArray Test");
> mxString=mxCreateString(str);
mxString=mxCreateString("mxArray Test");
The code you have is cumbersome and leaks the memory behind the str pointer.
James Tursa
Xueming
Btw, I tried to put mexMakeArrayPersistent(mxCell_g);
following
mxSetCell(mxCell1,1,mxString);
mxCell_g=mxCell1;
but it did work. I was wondering since there is a cell in mxCell1, should I keep it from being destroyed as well?
Xueming
James Tursa
> James,hank you very much for pointing out my misunderstanding of garbage collection mechanism, and my careless about pointer initialization. I found the memory management issues on Mathwork website, and gonna take a look very carefully.
> Btw, I tried to put mexMakeArrayPersistent(mxCell_g);
> following
> mxSetCell(mxCell1,1,mxString);
> mxCell_g=mxCell1;
> but it did work. I was wondering since there is a cell in mxCell1, should I keep it from being destroyed as well?
James Tursa
Xueming
my code was no big difference for the previous on, I just add mexMakeArrayPersistent(mxCell_g); and the code is like below:
{
...
mxCell1=mxCreateCellMatrix(1,2);
mxSetCell(mxCell1,0,mxCell);
mxSetCell(mxCell1,1,mxString);
mxCell_g=mxCell1;
mxCell_g=mxCell1;
mexMakeArrayPersistent(mxCell1);
return mxCell_g;
}
The result turned out as the same as without mexMakeArrayPersistent(mxCell1).
Emily