vxworks leaks memory
Dewan RASHID
Hi all:
I am facing a strange problem in vxworks when I
run the following simple code:
#include <fcntl.h>
#include <new.h>
#include <ctype.h>
#include <iostream.h>
#include <fstream.h>
ifdef VXWORKS
extern "C" int Start( int X )
else
main()
endif
{
ofstream* pX = (ofstream*)new char[sizeof(ofstream)];
int fd = open("BM.txt", O_RDWR, 644);
new (pX) ofstream(fd);
ostream& Outfile = *pX;
for ( int i = 0; i < X ; i++ )
{
Outfile << "THIS MAY BE A TEST" << endl;
if ( i % 400 )
{
pX->flush();
}
}
return 0;
}
The problem is that it consistently leaks memory
in vxworks machine but not in HP machines running
UNIX. The target processor is MC68040 running
vxworks HP9000 series as the host. The compiler
is progressive 95q3 from Cygnus.
-> ld<memoryLeak.o
value = 15931584 = 0xf318c0
-> memShow
status bytes blocks avg block max block
------ --------- -------- ---------- ----------
current
free 14581120 34 428856 12864932
alloc 1267392 5463 231 -
cumulative
alloc 3460228 8100 427 -
value = 0 = 0x0
-> sp Start, 10000
task spawned: id = 0xe63d3c, name = t2
value = 15088956 = 0xe63d3c
-> memShow
status bytes blocks avg block max block
------ --------- -------- ---------- ----------
current
free 14381920 36 399497 12864932
alloc 1466592 5840 251 -
cumulative
alloc 3687980 8486 434 -
value = 0 = 0x0
-> sp Start, 10000
task spawned: id = 0xe32908, name = t3
value = 14887176 = 0xe32908
-> memShow
status bytes blocks avg block max block
------ --------- -------- ---------- ----------
current
free 14182720 38 373229 12864932
alloc 1665792 6217 267 -
cumulative
alloc 3916940 8873 441 -
value = 0 = 0x0
It will be of great help if anyone can help me in this
regard.
Thanks,
Dewan
GEC Marconi Systems
Sydney, Australia.
Ted Marz
d...@f111.iassf.easams.com.au (Dewan RASHID) wrote:
>Hi all:
>I am facing a strange problem in vxworks when I
>run the following simple code:
snip
>The problem is that it consistently leaks memory
>in vxworks machine but not in HP machines running
>UNIX. The target processor is MC68040 running
>vxworks HP9000 series as the host. The compiler
>is progressive 95q3 from Cygnus.
snip
>It will be of great help if anyone can help me in this
>regard.
>Thanks,
>Dewan
>GEC Marconi Systems
>Sydney, Australia.
A while ago (I just don't have it in my news history) there was a
discussion here about memory allocation, and more importantly, freeing
allocated memory and garbage collection in VxWorks. You may want to
check DejaNews.
I wasn't paying a lot of attention, but I believe that the general
answer was that VxWorks does not garbage collect well, if at all.
Consequently, dynamic memory applications that are started multiple
times may be somewhat problematic.
Unix systems run in a virtual memory context and all have virtual
memory management, and consequently, task exit garbage collection (as
well as some at other times, which may be unpredictable, in a time
sense). VxWorks tends to run in the physical memory space (unless you
go thru quite a bit of relatively un-documented territory and use
VxVMI) and does not (I think) garbage collect, making it more time
predictable.
In short, VxWorks ain't Unix, doesn't want to be Unix, and isn't ever
going to be Unix. The philosophy seems to be more one of embedded
systems, which do one thing, in real-time, for a long time. Not a
system where multiple tasks are loaded, run, run again, unloaded, etc.
Sorry if this isn't the answer you were looking for.
Ted Marz
Kevin Bradley
Excerpts from netnews.comp.os.vxworks: 28-May-97 vxworks leaks memory by
Dewan RAS...@f111.iassf.
> int fd = open("BM.txt", O_RDWR, 644);
> new (pX) ofstream(fd);
>
>
> The problem is that it consistently leaks memory
> in vxworks machine but not in HP machines running
> UNIX. The target processor is MC68040 running
> vxworks HP9000 series as the host. The compiler
> is progressive 95q3 from Cygnus.
The problem is that under UNIX all memory not free()'d by the
process is cleaned up by the OS on exit. This protects the
internal workings of the heap etc. Ditto file closing -- if
a UNIX process leaves files open, they're closed at exit.
VxWorks, since it operates in a global space and everything
is shared, cannot assume that memory allocated by a task will
not be used after that task is deallocated. For example, driver
init tasks routinely allocate memory that's used later on.
Multi-task file sharing is also possible, so the same set of
memory and file pointers is accessible to different tasks.
Since you've put the pointer on the stack, allocated it, and
then thrown the pointer away with the stack on exit, you're
creating the leak internally. If you make a global pointer
"pX" (eek) and check to see if it's already allocated on task
init, then you don't need to re-allocate it or re-open the file.
E.g.:
ofstream *pX = NULL;
extern int Start()
{
if(pX == NULL) {
// Allocate, open file
}
// Rest of code
}
pX will be initialized on download, as it would be under UNIX on
process start (really a download from disk followed by _main).
Subsequent calls to Start() without download will not re-init it.
You can also check pX with the shell after Start() terminates.
Alternatively, if you put a "delete pX" at the end of your code,
part of your leak will go away. Also, closing the file will
certainly help as well.
You can check the open file pointers with "iosFdShow" and see if
you've left any hanging around. I certainly see 'em after my
task dies inappropriately, but then I can close them manually later
to protect the file integrity.
-- Kevin
Rajesh
Hi,
I am sorry if this question has been already answered or looks like a silly
question. I am new to VxWorks OS and we are in confused state in relation
with the following question.
Does VxWorks takes care of memory fragmentation?
I mean, if number of tasks use malloc() and free() several times, it would
definately lead to memory fragmentation. How does VxWorks handle it?? Does
it collect the fragmented memory areas at one place and use it for
further malloc() calls!!.
Thanks in advance for your comments!!
Rajesh.
Dewan RASHID
Dear all:
In a recent mail I wrote about memory leak problem
in vxworks. Probably the example was not the best
example that I could give. The following example
is a better one (I hope).
--------------
#include <fcntl.h>
#include <new.h>
#include <ctype.h>
#include <iostream.h>
#include <fstream.h>
#ifdef VXWORKS
extern "C" int Start( int X )
#else
main()
#endif
{
ofstream Outfile("BM.txt");
Outfile << "THIS MAY BE A TEST" << endl;
Outfile << "THIS MAY BE A TEST" << endl;
Outfile << "THIS MAY BE A TEST" << endl;
Outfile.close();
return 0;
}
-------------
Once I compile and run it, I see a memory leak of
304 bytes.
---------------
-> ld<memoryLeak.o
-> memShow
status bytes blocks avg block max block
------ --------- -------- ---------- ----------
current
free 14392344 59 243938 12733736
alloc 1456136 5497 264 -
cumulative
alloc 47130296 58864 800 -
value = 0 = 0x0
-> sp Start
task spawned: id = 0xe922fc, name = t6
value = 15278844 = 0xe922fc
-> memShow
status bytes blocks avg block max block
------ --------- -------- ---------- ----------
current
free 14392040 60 239867 12733736
alloc 1456440 5498 264 -
cumulative
alloc 47161836 58879 800 -
value = 0 = 0x0
---------------
In our case, the consequence of this memory leak is
severe. Because when we try to retrieve data from
local hard disk (the data is saved in a raw file system
and the size can be of the order of 50 MB)
and save it in an NFS file, either ASCII or binary,
the *memory leak* becomes accumulative and eventually
the processor crashes. The processor is MC68040 with
16 MB of memory.
In conclusion I like to ask the following questions:
a) Why does this memory leak occurs in vxworks?
b) Is there any way to prevent this?
b) Is there any way to download the local hard disk
data into an UNIX file system without crashing
the processor?
Any help will be greatly apprecited.
Thanks,
Dewan M Rashid
John Finley
Rajesh wrote:
....
> Does VxWorks takes care of memory fragmentation?
>
> I mean, if number of tasks use malloc() and free() several times, it would
> definately lead to memory fragmentation. How does VxWorks handle it?? Does
> it collect the fragmented memory areas at one place and use it for
> further malloc() calls!!.
The only thing VxWorks will do is coalesce free blocks that
happen to be next to each other into larger free blocks.
It does not (cannot) move free blocks next to each other to
reduce fragmentation.
John
--------------------------------------------------
John Finley Engineering Consultant
jo...@finley.com Real Time Systems
(619) 689-0032 Networking Protocols
--------------------------------------------------
Rajesh
In article , John says...
>
>Rajesh wrote:
>....
>> Does VxWorks takes care of memory fragmentation?
>> ** Skip ***
>
>The only thing VxWorks will do is coalesce free blocks that
>happen to be next to each other into larger free blocks.
>It does not (cannot) move free blocks next to each other to
>reduce fragmentation.
>
>John
In such a case how to manage the fragmentation? My program uses lot
of dynamic memory allocation. I am afraid that this would result in unnecessary malloc() fail.
Is any "Garbage Collector" (GC) sort of routine available on VxWorks (or as third part s/w)?? Or the user is supposed to write its own GC routine!!
thanks,
Rajesh.
John Finley
Garbage collection and memory fragmentation are separate
issues, both of which have generated lengthy threads in
this newsgroup in the past.
I don't think garbage collection can be done using a
standard-c-library compatible memory manager. You would
have to add additional hooks into the memory manager
so it knows when pointers are assigned etc. There may
be third party memory managers that provide this, but
I don't know what they are. Perhaps your application
could be rewritten in lisp to run on a Vax. :)
Memory fragmentation *in practice* also requires
application awareness to avoid. You have to access
blocks by pointers-to-pointers, and lock them at
appropriate times, so the memory manager can move
them if it wants to. (*in practice* means there are
ways to do it otherwise, but I don't think any will
appear for VxWorks any time soon.)
There are some simple things you can do in your
application to reduce memory fragmentation. For
example, you can use buffer pools if you allocate
and free a certain structure a lot. You can define
several memory partitions, and use them for
different purposes by using memPartAlloc instead
of malloc.
Either automatic garbage collection or memory
defragmentation will generally cost you some
predictability in your system. Nothing's free,
except the lunch I had last Saturday. It was good,
too.
h.j. bae
John Finley wrote:
>
> I don't think garbage collection can be done using a
> standard-c-library compatible memory manager. You would
> have to add additional hooks into the memory manager
> so it knows when pointers are assigned etc. There may
> be third party memory managers that provide this, but
> I don't know what they are. Perhaps your application
> could be rewritten in lisp to run on a Vax. :)
>
a good garbage collector for C and C++ is available
http://reality.sgi.com/employees/boehm_mt/gc.html.
this is the boehm-demers-weiser conservative
garbage collector, which is used as part of toba,
a java->C translating system.
> Memory fragmentation *in practice* also requires
> application awareness to avoid. You have to access
> blocks by pointers-to-pointers, and lock them at
> appropriate times, so the memory manager can move
> them if it wants to. (*in practice* means there are
> ways to do it otherwise, but I don't think any will
> appear for VxWorks any time soon.)
>
the memory allocator subsystem can be as simple as
a set of fixed sized bucket allocator if you just
want to avoid fragmentation. a good system that
will avoid the fragmentation and accommodate your
application memory usage behavior can be written
easily. or better yet, it can be ported (there
are many such allocators). vxworks versions have
been ported by many vxworks users.
> Either automatic garbage collection or memory
> defragmentation will generally cost you some
> predictability in your system. Nothing's free,
> except the lunch I had last Saturday. It was good,
> too.
>
actually, vxworks memLib does to a very trivial
defragmentation. it only happens when a block
that is consecutive to other blocks in the free
pool gets freed. the problem with this is that it
is a natural for fragmentation for most real applications.
a much simpler bucket oriented scheme works much better
for most embedded applications. and the original
design and implementation of memLib opened doors for
many hours of wasted efforts to fix it by numerous
vxworks users.