Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

need help for fast page heap

57 views
Skip to first unread message

JJ

unread,
Jul 31, 2005, 10:26:02 PM7/31/05
to
Hi:

When I try to enable page heap using gflags, at the beginning, it looks like
only fast page heap was enabled. So I turn on heap checking by "!heap -e all
" and "!heap -E all", but after that, I saw the warning of "!heap allocation
size error (exceed bffdeff), what does this warning mean? windbg still
continue to run and is it a fatal warning that means the memory is already
bad?

I have tried to enable full page heap with size range of 0 200, but
application just can't start (consume too much memories, using the fast page
heap, can it detect memory corruption?

Ivan Brugiolo [MSFT]

unread,
Jul 31, 2005, 11:38:12 PM7/31/05
to
For page-heap status, `!heap -p` and `!heap -p -h <heap_handle>`
are the best commands.
Can you see if you have warnings after you use proper commands ?

If you have at least 512Megs of physical memory and adequate page-file
space,
your application should be able to at least start.
I have not yest seen an application that exhaust
memory just before even executing `WinMain`.
Moreover, full-pageheap is designed to fall-back to light-pageheap under
machine
memory pressure, so, at the end, you should see the same behavior
as if light-pageheap had been enabled.
What exactly do you see in the debugger when the application fails to start
?

Light PageHeap can detect the same classes of heap corruption of
Full-PageHeap,
with the difference that the detection of problem happens
when it may be harder to debug them, normally when the free operation
happens, or when the delayed free-queue is empty-ed.
This may or may not be satisfactory for the kind of problem you are chasing.


--
This posting is provided "AS IS" with no warranties, and confers no rights.
Use of any included script samples are subject to the terms specified at
http://www.microsoft.com/info/cpyright.htm


"JJ" <J...@discussions.microsoft.com> wrote in message
news:A6F70E00-8FCD-4F4C...@microsoft.com...

JJ

unread,
Aug 1, 2005, 1:02:01 AM8/1/05
to
The program without enabling page heap the memory already took 1.5GB after
successfully start up. When I try to enable full page heap, it gives an error
of "Application unwind error" when the VM reaches to 3 GB (maybe it's due to
the program has override the default "new" and use its own custom multiheap
implementation, and program will throw this error when new return null
pointer). This program has 1500 threads, and some of the threads have
approximately 1MB for stack size (due to we didn't set the
STACK_SIZE_IS_RESERVATION flag when ew do CreateThread). I can try to change
the program to reduce these unnesasary stack size before I startup to reduce
the size of memory.

You are right, when I use fast fill heap and when the application stops with
access violation, it's already too late, here is the output when it stops, at
this point, I found we already have several "new" failure, but application
just reached the VM of 2 GB (perfmon virtual bytes output).

0:712> !heap -x -v 42e5f594
Walking the segment exceeded the 100000 limit
Walking the segment exceeded the 100000 limit
Walking the segment exceeded the 100000 limit
Walking the segment exceeded the 100000 limit
ERROR: Block 87ff0000 has signature corrupted 0x0 (expected 0xab)
HEAP 01d70000 (Seg 7fff0000) At 87ff0000 Error: block signature corrupted

Search VM for address range 42e5f594 - 42e5f594 :
0:712> !heap
Index Address Name Debugging options enabled
1: 00180000 tail checking free checking validate
parameters validate on call
2: 00380000 tail checking free checking validate
parameters validate on call
3: 00390000 tail checking free checking validate
parameters validate on call
4: 003d0000 tail checking free checking validate
parameters validate on call
5: 01010000 tail checking free checking validate
parameters validate on call
6: 01130000 tail checking free checking validate
parameters validate on call
7: 01870000 tail checking free checking validate
parameters validate on call
8: 019b0000 tail checking free checking validate
parameters validate on call
9: 01af0000 tail checking free checking validate
parameters validate on call
10: 01c30000 tail checking free checking validate
parameters validate on call
11: 01d70000 tail checking free checking validate
parameters validate on call
12: 027b0000 tail checking free checking validate
parameters validate on call
13: 02c20000 tail checking free checking validate
parameters validate on call
14: 02d50000 tail checking free checking validate
parameters validate on call
15: 02e90000 tail checking free checking validate
parameters validate on call
16: 02fd0000 tail checking free checking validate
parameters validate on call
17: 03110000 tail checking free checking validate
parameters validate on call
18: 03250000 tail checking free checking validate
parameters validate on call
19: 38270000 tail checking free checking validate
parameters validate on call
20: 38280000 tail checking free checking validate
parameters validate on call
21: 38290000 tail checking free checking validate
parameters validate on call
Application verifier settings (00048005):

- fast fill heap (a.k.a light page heap)
- lock checks (critical section verifier)
- handle checks

!ext.heap -p

Active GlobalFlag bits:
vrf - Enable application verifier
hpa - Place heap allocations at ends of pages

StackTraceDataBase @ 00410000 of size 00800000 with 00000afe traces

PageHeap enabled with options:
ENABLE_PAGE_HEAP
COLLECT_STACK_TRACES

active heaps:

+ 80000
ENABLE_PAGE_HEAP COLLECT_STACK_TRACES
NormalHeap - 180000
HEAP_GROWABLE HEAP_TAIL_CHECKING_ENABLED HEAP_FREE_CHECKING_ENABLED
+ 280000
ENABLE_PAGE_HEAP COLLECT_STACK_TRACES
NormalHeap - 380000
HEAP_GROWABLE HEAP_TAIL_CHECKING_ENABLED
HEAP_FREE_CHECKING_ENABLED HEAP_CLASS_1
+ c10000
ENABLE_PAGE_HEAP COLLECT_STACK_TRACES
NormalHeap - 390000
HEAP_GROWABLE HEAP_TAIL_CHECKING_ENABLED
HEAP_FREE_CHECKING_ENABLED HEAP_CLASS_1
+ f10000
ENABLE_PAGE_HEAP COLLECT_STACK_TRACES
NormalHeap - 1010000
HEAP_GROWABLE HEAP_TAIL_CHECKING_ENABLED
HEAP_FREE_CHECKING_ENABLED HEAP_CLASS_1
+ 1030000
ENABLE_PAGE_HEAP COLLECT_STACK_TRACES
NormalHeap - 1130000
HEAP_GROWABLE HEAP_TAIL_CHECKING_ENABLED
HEAP_FREE_CHECKING_ENABLED HEAP_CLASS_1
+ 1770000
ENABLE_PAGE_HEAP COLLECT_STACK_TRACES
NormalHeap - 1870000
HEAP_GROWABLE HEAP_TAIL_CHECKING_ENABLED
HEAP_FREE_CHECKING_ENABLED HEAP_CLASS_1
+ 18b0000
ENABLE_PAGE_HEAP COLLECT_STACK_TRACES
NormalHeap - 19b0000
HEAP_GROWABLE HEAP_TAIL_CHECKING_ENABLED
HEAP_FREE_CHECKING_ENABLED HEAP_CLASS_1
+ 19f0000
ENABLE_PAGE_HEAP COLLECT_STACK_TRACES
NormalHeap - 1af0000
HEAP_GROWABLE HEAP_TAIL_CHECKING_ENABLED
HEAP_FREE_CHECKING_ENABLED HEAP_CLASS_1
+ 1b30000
ENABLE_PAGE_HEAP COLLECT_STACK_TRACES
NormalHeap - 1c30000
HEAP_GROWABLE HEAP_TAIL_CHECKING_ENABLED
HEAP_FREE_CHECKING_ENABLED HEAP_CLASS_1
+ 1c70000
ENABLE_PAGE_HEAP COLLECT_STACK_TRACES
NormalHeap - 1d70000
HEAP_GROWABLE HEAP_TAIL_CHECKING_ENABLED
HEAP_FREE_CHECKING_ENABLED HEAP_CLASS_1
+ 26b0000
ENABLE_PAGE_HEAP COLLECT_STACK_TRACES
NormalHeap - 27b0000
HEAP_GROWABLE HEAP_TAIL_CHECKING_ENABLED
HEAP_FREE_CHECKING_ENABLED HEAP_CLASS_1
+ 2b20000
ENABLE_PAGE_HEAP COLLECT_STACK_TRACES
NormalHeap - 2c20000
HEAP_GROWABLE HEAP_TAIL_CHECKING_ENABLED
HEAP_FREE_CHECKING_ENABLED HEAP_CLASS_1
+ 2c50000
ENABLE_PAGE_HEAP COLLECT_STACK_TRACES
NormalHeap - 2d50000
HEAP_GROWABLE HEAP_TAIL_CHECKING_ENABLED
HEAP_FREE_CHECKING_ENABLED HEAP_CLASS_1
+ 2d90000
ENABLE_PAGE_HEAP COLLECT_STACK_TRACES
NormalHeap - 2e90000
HEAP_GROWABLE HEAP_TAIL_CHECKING_ENABLED
HEAP_FREE_CHECKING_ENABLED HEAP_CLASS_1
+ 2ed0000
ENABLE_PAGE_HEAP COLLECT_STACK_TRACES
NormalHeap - 2fd0000
HEAP_GROWABLE HEAP_TAIL_CHECKING_ENABLED
HEAP_FREE_CHECKING_ENABLED HEAP_CLASS_1
+ 3010000
ENABLE_PAGE_HEAP COLLECT_STACK_TRACES
NormalHeap - 3110000
HEAP_GROWABLE HEAP_TAIL_CHECKING_ENABLED
HEAP_FREE_CHECKING_ENABLED HEAP_CLASS_1
+ 3150000
ENABLE_PAGE_HEAP COLLECT_STACK_TRACES
NormalHeap - 3250000
HEAP_GROWABLE HEAP_TAIL_CHECKING_ENABLED
HEAP_FREE_CHECKING_ENABLED HEAP_CLASS_1
+ 38170000
ENABLE_PAGE_HEAP COLLECT_STACK_TRACES
NormalHeap - 38270000
HEAP_GROWABLE HEAP_TAIL_CHECKING_ENABLED
HEAP_FREE_CHECKING_ENABLED HEAP_CLASS_1
+ 3ea20000
ENABLE_PAGE_HEAP COLLECT_STACK_TRACES
NormalHeap - 38280000
HEAP_GROWABLE HEAP_TAIL_CHECKING_ENABLED
HEAP_FREE_CHECKING_ENABLED HEAP_CLASS_1
+ 3eb20000
ENABLE_PAGE_HEAP COLLECT_STACK_TRACES
NormalHeap - 38290000
HEAP_GROWABLE HEAP_TAIL_CHECKING_ENABLED
HEAP_FREE_CHECKING_ENABLED HEAP_CLASS_1


No verifier stop active.

Note. Sometimes bugs found by verifier manifest themselves
as raised exceptions (access violations, stack overflows, invalid handles
and it is not always necessary to have a verifier stop.
0:712> !avrf -hp 01d70000
Dumping last 1 entries from tracker @ 00d10040 with 8192 valid entries ...
--------------------------------------------------------------
HeapFree: 0 0 0 0

77e42853: kernel32!LocalFree+0x27
6ff466fd: <dll>!SystemUtility::GetSystemErrorString+0x15D
0:712> !avrf -hp 01d70000 10
Dumping last 1 entries from tracker @ 00d10040 with 8192 valid entries ...
--------------------------------------------------------------
HeapFree: 0 0 0 0

77e42853: kernel32!LocalFree+0x27
6ff466fd: <dll>!SystemUtility::GetSystemErrorString+0x15D

Ivan Brugiolo [MSFT]

unread,
Aug 1, 2005, 11:15:56 AM8/1/05
to
> You are right, when I use fast fill heap and when the application stops
> with
> access violation, it's already too late, here is the output when it stops,
> at
> this point, I found we already have several "new" failure, but application
> just reached the VM of 2 GB (perfmon virtual bytes output).

Can you report the output of `r;kb` at the time
the application takes an access violation ?
It's not cleat where you fished out the value for this command
`0:712> !heap -x -v 42e5f594`,
and probably, `!heap -p -a 42e5f594` would have given
more meaningful results.

Can you also run `!address -summary` and see
how is your address space usage ?

--
This posting is provided "AS IS" with no warranties, and confers no rights.
Use of any included script samples are subject to the terms specified at
http://www.microsoft.com/info/cpyright.htm


"JJ" <J...@discussions.microsoft.com> wrote in message

news:1F82CE0E-634A-4422...@microsoft.com...

JJ

unread,
Aug 1, 2005, 1:37:09 PM8/1/05
to
It looks like I pick up the wrong address, the 42e5f594 is actually the child
EBP when the access violation happened.

0:709> .ecxr;kb
eax=00000000 ebx=02a7ff0f ecx=00000009 edx=00000001 esi=00000000 edi=42e5f574
eip=6e9a64f9 esp=42e5f568 ebp=42e5fe54 iopl=0 nv up ei pl nz na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00010202

m1!f1:GetCurrDateTime+0x29:

6e9a64f9 f3a5 rep movsd ds:00000000=???????? es:42e5f574=42e5f5e4
*** Stack trace for last set context - .thread/.cxr resets it

ChildEBP RetAddr Args to Child
42e5f594 6e916685 42e5f5b4 42e5fe48 42e5fc70 m1!f1::GetCurrDateTime+0x29
[s1.cpp]
42e5f5cc 6e916501 3ac031a0 0000001d 42e5fe48 m3!f2::WriteMessage+0x85 [s2.cpp]
42e5f5fc 6e9165af 00000000 42e5fc70 02a7ff3d m3!f3::LogMessage+0x91 [s3.cpp]
42e5f608 02a7ff3d 00000000 00000000 42e5fc70 m3!f3::LogMessage+0xf [s3.cpp]
00000000 00000000 00000000 00000000 00000000 m4!f4::Main+0x10d [s4.cpp]

But which address I should pick in this case to input for the !heap -p -h ?

I have the address summary as follows:
0:709> !address -summary

-------------------- Usage SUMMARY --------------------------
TotSize Pct(Tots) Pct(Busy) Usage
03613000 : 1.76% 2.69% : RegionUsageIsVAD
422b0000 : 34.46% 0.00% : RegionUsageFree
035b1000 : 1.75% 2.67% : RegionUsageImage
2d780000 : 23.68% 36.14% : RegionUsageStack
002d9000 : 0.09% 0.14% : RegionUsageTeb
48333000 : 37.60% 57.38% : RegionUsageHeap
013ec000 : 0.65% 0.99% : RegionUsagePageHeap
00001000 : 0.00% 0.00% : RegionUsagePeb
00001000 : 0.00% 0.00% : RegionUsageProcessParametrs
00002000 : 0.00% 0.00% : RegionUsageEnvironmentBlock
Tot: bfff0000 Busy: 7dd40000

-------------------- Type SUMMARY --------------------------
TotSize Pct(Tots) Usage
422b0000 : 34.46% : <free>
035b1000 : 1.75% : MEM_IMAGE
00ae1000 : 0.35% : MEM_MAPPED
79cae000 : 63.43% : MEM_PRIVATE

-------------------- State SUMMARY --------------------------
TotSize Pct(Tots) Usage
2a46b000 : 22.02% : MEM_COMMIT
422b0000 : 34.46% : MEM_FREE
538d5000 : 43.52% : MEM_RESERVE

Largest free region: Base 8fff0000 - Size 2fcec000

Ivan Brugiolo [MSFT]

unread,
Aug 1, 2005, 3:25:43 PM8/1/05
to
The stack below looks a memcpy()/memmov() with NULL destination pointer.

Your process is LargeAddressAware.
Your stack usage is about 1/3 of the used address space, that is bad in
general
(I have never seen an application that by-design uses 1500 threads,
probably there are better ways of doing what these threads are doing,
such as using thread pools and completion ports).
The Full-Page-Heap usage is neglectible (looks like you have
Light-PageHeap),
and heap usage is the biggest users of the address space.
However, you have about 800 Megs of contiguous address space availalbe,
and, that should allow HEAP_SEGMENT expansion.

Can you identify where does the NULL pointer used below come from ?

--
This posting is provided "AS IS" with no warranties, and confers no rights.
Use of any included script samples are subject to the terms specified at
http://www.microsoft.com/info/cpyright.htm


"JJ" <J...@discussions.microsoft.com> wrote in message

news:EB3A1AA7-2667-42FF...@microsoft.com...

JJ

unread,
Aug 3, 2005, 6:33:09 PM8/3/05
to
Actually I found that the null pointer comes from our exception handler of
"new", so there is already some HeapAlloc error returned from previous call
(and in the handler, it tries to log a message then failed bcz to log a
message it requires another new).

Now, the question is, why does "new" fail if there is still adquet memory
(800 M left). Is it possible that virtual address is fragmented? I have the
following output printed from !heap -s
Heap Flags Reserv Commit Virt Free List UCR Virt Lock Fast
(k) (k) (k) (k) length blocks cont. heap
-----------------------------------------------------------------------------
00090000 00000002 2048 1380 1380 5 0 1 0 0 L
00190000 00001002 64 28 28 0 0 1 0 0 L
001a0000 00008000 64 12 12 10 1 1 0 0
Virtual block: 6be60000 - 6be60000 (size 00000000)
00270000 00001002 1047616 620636 621764 1452 269 117 1 39d7 L
00290000 00001002 256 72 72 3 1 1 0 0 L
003c0000 00001003 64768 55780 59032 5384 11 12 0 bad
006a0000 00001003 15616 7304 11640 1267 255 130 0 bad
External fragmentation 17 % (255 free blocks)
Virtual address fragmentation 37 % (130 uncommited ranges)
006e0000 00001003 15616 7112 12376 882 162 122 0 bad
External fragmentation 12 % (162 free blocks)
Virtual address fragmentation 42 % (122 uncommited ranges)
00720000 00001003 15616 7496 13156 909 126 115 0 bad
External fragmentation 12 % (126 free blocks)
Virtual address fragmentation 43 % (115 uncommited ranges)
00760000 00001003 15616 7936 13308 538 55 45 0 bad
007a0000 00001003 32000 9604 17140 326 10 30 0 bad
007e0000 00001003 15616 7248 11808 709 233 22 0 bad
00820000 00001003 64768 5396 8872 319 30 35 0 bad
00860000 00001003 32000 6064 14584 614 62 40 0 bad
008a0000 00001003 32000 14856 27716 454 64 143 0 bad
Virtual address fragmentation 46 % (143 uncommited ranges)
Virtual block: 36120000 - 36120000 (size 00000000)
008e0000 00001003 64768 14284 14324 1399 34 8 1 bad
02920000 00011002 256 12 12 4 1 1 0 1c L
02c90000 00001002 64 16 16 1 0 1 0 0 L
02cc0000 00001003 256 20 20 5 1 1 0 bad
02d00000 00001003 256 4 4 2 1 1 0 bad
02d40000 00001003 256 4 4 2 1 1 0 bad
02d80000 00001003 256 4 4 2 1 1 0 bad
02dc0000 00001003 256 4 4 2 1 1 0 bad
02e00000 00001003 256 4 4 2 1 1 0 bad
02e40000 00001003 256 4 4 2 1 1 0 bad
02e80000 00001003 256 4 4 2 1 1 0 bad
02ec0000 00001003 256 4 4 2 1 1 0 bad
02f00000 00001003 256 4 4 2 1 1 0 bad
02f40000 00001003 256 4 4 2 1 1 0 bad
03460000 00001002 64 40 40 2 1 1 0 0 L
11fa0000 00001002 64 12 12 4 1 1 0 0 L
11fb0000 00001002 256 60 60 3 1 1 0 0 L
53bd0000 00001002 1088 208 208 8 2 2 0 0 LFH

Ivan Brugiolo [MSFT]

unread,
Aug 3, 2005, 9:39:53 PM8/3/05
to
"why does HeapAlloc/malloc/new fails" is a question for which
I have tried to give reasonable answers in the newsgroups in the past,
and, at the end, ita takes some expert look at the output of `!address`
to get a good answer.

I guess the heap that has problem is this one


00270000 00001002 1047616 620636 621764 1452 269 117 1 39d7 L

But, without a full address-space layout dump it would not
be easy to give a sensible answer.

--
This posting is provided "AS IS" with no warranties, and confers no rights.
Use of any included script samples are subject to the terms specified at
http://www.microsoft.com/info/cpyright.htm


"JJ" <J...@discussions.microsoft.com> wrote in message

news:5878BDEF-2B93-42A8...@microsoft.com...

0 new messages