loadModuleAt() allocating large blocks of memory

[Zac]

I am hoping that I am not the first to run into this problem...
perhaps there is a thread on this already, but I haven't found it.

I use VxWorks 5.5 and our application frequently uses loadModule() and
unldByModuleId() to change the functionality of our target. This has
caused us a lot of grief with memory fragmentation. In order to avoid
some of the fragmentation problems, I decided to try pre-allocated
memory blocks for the text, data and bss blocks that we load and
unload. This appeared to work well and it may solve most of our
fragmentation problems, except for one thing...

I pass in the address of the pointers to the pre-allocated blocks of
memory when calling the loadModuleAt() function. So, I would not
expect the load function to allocate any memory... I have already done
that. However, it looks like the loadModuleAt() function is
allocating very large blocks of memory (on the order of about 10x the
size of the largest memory block required by the module) even though I
have already allocated the memory. I only know this because I get a
"memPartAlloc... block too big" error when calling the function. The
error message tells me the size of the block that it is attempting to
allocate and I cannot imagine the purpose of such a large block.

So, my questions are:
1. Has anybody run into a similar problem in the past?
2. Why is the loadModuleAt() function allocating so much memory?
3. How can I avoid this when loading/unloading modules from memory?

Thanks in advance for your help.

[PAD]

Do you have a lot of symbols in your loaded modules? C++ code is known
to generate an incredible amount of symbols. Each symbol information
stored in the system symbol table take room. You could also test this
by turning off the registration of symbols (check the 'ld', loadModule
(), and loadModuleAt() manual for the specific option).

Another possibility is the memory used to store the entire file in
memory when the RSH or FTP file access methods are used to load the
modules. Note that the loader uses by default the same file access
method as the one indicated via the boot monitor to find and load the
VxWorks image in order to boot the board. You could test this by using
the NFS client to access files.

--
PAD

[Zac]

We are using C++ for a good portion of our application development.
Now that you mention the symbol table, that brings up some interesting
questions.

Could it be that the OS is allocating a larger space for the global
symbol table when I load a new module? Maybe that is why the
loadModuleAt() function is allocating such a large block of memory. I
can imagine that the OS may need to allocate a whole new block of
memory when new symbols are added because there is not enough space in
the memory block that was initially allocated for it. At that point,
the existing symbols in the table may be copied to the new space
before the original symbol table space is freed.

When we load our modules, we use the option to load global symbols
into the system symbol table. I will look into either reducing the
number of global symbols defined in our applications or perhaps
eliminating the need to add the global symbols to the system symbol
table.

[PAD]

On Nov 3, 9:36 am, Zac <zac.i...@gmail.com> wrote:
[...]

> Could it be that the OS is allocating a larger space for the global
> symbol table when I load a new module? Maybe that is why the
> loadModuleAt() function is allocating such a large block of memory. I
> can imagine that the OS may need to allocate a whole new block of
> memory when new symbols are added because there is not enough space in
> the memory block that was initially allocated for it. At that point,
> the existing symbols in the table may be copied to the new space
> before the original symbol table space is freed.

You are making a confusion regarding what the block of memory you
allocated is for: it is only used to store the loaded module's text,
data and bss sections. Nothing more. All the symbols are separately
allocated and added to the system's symbol table. This symbol table is
different from the one that comes with the object module (the file's
symbol table section is ultimately discarded).

So, assuming you want to register only the global symbols and your
module has 10,000 global symbols, then the loader will do 10,000
memory allocations (16 bytes each in 5.5) to create a VxWorks symbol
for each of the global symbols entries found in the file.

> When we load our modules, we use the option to load global symbols
> into the system symbol table. I will look into either reducing the
> number of global symbols defined in our applications or perhaps
> eliminating the need to add the global symbols to the system symbol
> table.

Yes, either approaches will help.

Again, keep in mind that, depending on what file access method is
being used, the entire object module might be copied in memory prior
to being "loaded" by the loader (essentially the memory is used as a
temporary RAM file system) . This means that file content like debug
symbols and associated strings for which the loader does not allocated
memory will still need memory to hold them. With C++ the amount of
debug symbols and associated strings can be literally enormous... That
memory is freed when the loader is done but, in the mean time, the
memory consumption reaches peak values.

Cheers,

--
PAD