Memory fragmentation

[Richard L.]

I am writing some software which allocates many small
objects of different sizes, some of them in an std::list.

It seems that the memory is being fragmented very quickly
to the point where WinCe will not give us the requested
memory even though there should be enough available.

The function HeapCompact is not supported on WinCE, so I
was wondering what alternatives there are, and/or what is
the best way to use HeapCreate to make lots of little
memort allocations?

Advice would be greatly appreciated.
:)

[Michael J. Salamone [eMVP]]

How are you determining "should be enough available"?

The CE heap manager does coalesce free blocks.

For CE 3.0, it doesn't do it with every free, however. The rule for
determining when to compact seems a bit complex so I'm not exactly sure when
it does it.

It seems to do it with every free for CE 4.2.

Note that your app may be fragmenting the heap such that it cannot be
compacted efficiently.

Also note that corrupting the heap could lead to problems compacting (as
well as other fun problems).

Even if the heap weren't compacted, if the heap manager can't find a chunk
big enough for the request, it will try to grow the heap and give you a
piece of the new part.

Run the app under the debugger and set a break just before attempting an
allocation that would fail. Verify the size being requested. You can also
run the Remote Viewer (from the eVC tools menu) and take a look at the
outstanding allocations and free blocks. Does it look like, if it were
compacted, the allocation would succeed?

--

Michael Salamone [eMVP]
Entrek Software, Inc.
www.entrek.com

"Richard L." <rich...@pixology.com> wrote in message
news:000801c3a22c$f4bddc50$a401...@phx.gbl...

[Daniel Pfeffer]

"Richard L." <rich...@pixology.com> wrote in message
news:000801c3a22c$f4bddc50$a401...@phx.gbl...

Many programs typically allocate blocks of a small number of different
sizes. Writing a global new() operator that uses a separate heap for each
block size is not that difficult. An even better solution would be to write
your fixed block allocator so that it works over the VirtualAlloc()
mechanism, thereby improving memory usage for tiny blocks and getting rid of
the extra overhead of the HeapXXX() APIs.

[Hints: reserve memory regions according to the VirtualAlloc() allocation
granularity, commit memory by page, and use a bitmap at the start of the
memory region to keep track of the available blocks in the memory region.
Multiple memory regions may be linked using a linked list. Note that
VirtualAlloc() always returns addresses that are a multiple of the
granularity, so any address in a memory region whose size is <= granularity
size can be converted to the block address with a simple masking operation.]

Alternatively, you can write class-specific new() operators that allocate
memory from class-specific heaps. For instantiations of std::list, this will
require you to create a custom allocator class.

HTH,
Daniel Pfeffer

[Jeremy Sherrill]

The doccumentation for HeapAlloc states that the actual size of an
allocation may exceed the size requested. To get the size of each
allocation, use HeapSize. Try checking the size each time to see if
allocations are exceeding the requested size.

Jeremy

"Richard L." <rich...@pixology.com> wrote in message
news:000801c3a22c$f4bddc50$a401...@phx.gbl...

[Richard L.]

Thanks, Daniel, that's very helpful.

I have had no problem creating a global 'operator new []',
but the eVC4 compiler complains that 'operator new' is
already defined in the core dll, even if my implementation
is inline.
Any ideas how to get around this problem?

[Daniel Pfeffer]

"Richard L." <anon...@discussions.microsoft.com> wrote in message
news:088f01c3a37d$a1011cb0$a601...@phx.gbl...

> Thanks, Daniel, that's very helpful.
>
> I have had no problem creating a global 'operator new []',
> but the eVC4 compiler complains that 'operator new' is
> already defined in the core dll, even if my implementation
> is inline.

The following code compiles and links perfectly under eVC 4.0 SP2
(STANDARDSDK targeting the emulator). Are you using an earlier version?

#include "stdafx.h"
#include <new>

void * ::operator new(unsigned int cbBlock) throw(std::bad_alloc)
{
void *retval = ::HeapAlloc(::GetProcessHeap(),0,cbBlock);

if (!retval)
throw std::bad_alloc("");

return retval;
}

void ::operator delete(void *pvBlock) throw()
{
::HeapFree(::GetProcessHeap(),0,pvBlock);
}

int WINAPI WinMain( HINSTANCE hInstance,
HINSTANCE hPrevInstance,
LPTSTR lpCmdLine,
int nCmdShow)
{
// TODO: Place code here.
int *pi = new int;

delete pi;

return 0;
}

[Richard L.]

Thanks.
:O)