When the hugepage size is 1G, it will fail to start.

192 views
Skip to first unread message

gchen....@gmail.com

unread,
Mar 29, 2019, 7:33:07 AM3/29/19
to stressapptest-discuss
hi nick
   I found a issue, when the hugepage=1G on my system, the stressapptest will fail to start.
   I checked the code for stressapptest and found that it was calculated in OsLayer::FindFreeMemSize()  by Assume  hugepage=2M。
   
thanks

Below is a detail log:
#stressapptest -s 86400 -W
2019/03/29-19:18:18(CST) Log: Commandline - ./src/stressapptest -s 86400 -W
2019/03/29-19:18:18(CST) Stats: SAT revision 1.0.9_autoconf, 64 bit binary
2019/03/29-19:18:18(CST) Log: root @ kvm_9_16_47_105 on Fri Mar 29 19:17:45 CST 2019 from open source release
2019/03/29-19:18:18(CST) Log: 1 nodes, 80 cpus.
2019/03/29-19:18:18(CST) Log: Defaulting to 80 copy threads
2019/03/29-19:18:18(CST) Log: Total 772004 MB. Free 677051 MB. Hugepages 100 MB. Targeting 100 MB (0%)
2019/03/29-19:18:18(CST) Log: Prefer using hugepage allocation.
2019/03/29-19:18:18(CST) Log: Using shared hugepage object 0x102a0032 at 0x2aaac0000000.
2019/03/29-19:18:18(CST) Stats: Starting SAT, 100M, 86400 seconds
2019/03/29-19:18:18(CST) Process Error: freepages < neededpages.
2019/03/29-19:18:18(CST) Stats: Total: 100, Needed: 80, Marked free: 40
2019/03/29-19:18:18(CST) Process Error: Initialize Pages failed
2019/03/29-19:18:18(CST) Process Error: Sat::Initialize() failed
2019/03/29-19:18:18(CST) 
2019/03/29-19:18:18(CST) Status: FAIL - test encountered procedural errors
2019/03/29-19:18:18(CST) 
2019/03/29-19:18:18(CST) Process Error: Fatal issue encountered. See above logs for details.
 

#cat /proc/meminfo  
MemTotal:       790532556 kB
MemFree:        693281824 kB
MemAvailable:   733498712 kB
Buffers:          653100 kB
Cached:         39227068 kB
SwapCached:            0 kB
Active:         29910840 kB
Inactive:       10325392 kB
Active(anon):     371172 kB
Inactive(anon):    25796 kB
Active(file):   29539668 kB
Inactive(file): 10299596 kB
Unevictable:       13840 kB
Mlocked:           13840 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Dirty:              4652 kB
Writeback:             0 kB
AnonPages:        367888 kB
Mapped:           112556 kB
Shmem:             37040 kB
Slab:            1996912 kB
SReclaimable:    1705424 kB
SUnreclaim:       291488 kB
KernelStack:       17136 kB
PageTables:        11756 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    369051876 kB
Committed_AS:    1341252 kB
VmallocTotal:   34359738367 kB
VmallocUsed:     1973980 kB
VmallocChunk:   33954721788 kB
HardwareCorrupted:     0 kB
AnonHugePages:    151552 kB
HugePages_Total:      50
HugePages_Free:       40
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:    1048576 kB        ==> hugepage = 1G  
DirectMap4k:      391744 kB
DirectMap2M:     7735296 kB
DirectMap1G:    797966336 kB

#lscpu
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                80
On-line CPU(s) list:   0-79
Thread(s) per core:    2
Core(s) per socket:    20
Socket(s):             2
NUMA node(s):          2
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 85
Model name:            Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz
Stepping:              4
CPU MHz:               2401.000
BogoMIPS:              4806.33
Virtualization:        VT-x
L1d cache:             32K
L1i cache:             32K
L2 cache:              1024K
L3 cache:              28160K
NUMA node0 CPU(s):     0-19,40-59
NUMA node1 CPU(s):     20-39,60-79


Nick Sanders

unread,
Mar 30, 2019, 2:36:06 PM3/30/19
to stressappt...@googlegroups.com
stressapptest has a usecase where hugepages are used for the majority of memory for testing, and will prefer hugepage allocation over malloc by default. So if you have any hugepages, it will want to run in that region. 

However, you seem to only have 100M free hugepages, which stressapptest allocated. Stressapptest requires ~3MB per thread to run, and with 80 cores will default to 80 threads. So the failure is that stressapptest needs 240M of memory allocation to run, and only has 100M.

The straightforwards fix would be to increase your hugepage allocation to 600G while running stressapptest, or specify memory size on the commandline "-M 600000" to avoid hugepages.

--

---
You received this message because you are subscribed to the Google Groups "stressapptest-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to stressapptest-di...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Nick Sanders

unread,
Mar 30, 2019, 2:44:06 PM3/30/19
to stressappt...@googlegroups.com
Oh, I see what you mean.. 

Yes, stressapptest is assuming 2MB hugepage size, which isn't correct these days. I don't have access to systems that supports hugepages anymore, but I think it should be possible to query size and do a better job of probing and allocation. If you can add it I'd be happy to pull it upstream.

gchen chen

unread,
Apr 21, 2019, 9:36:55 AM4/21/19
to stressappt...@googlegroups.com
hi Nick
Yes, I have write a function to query the hugepage size, and test it ok.
The attachment is my patch.

thanks and regards

Nick Sanders <pud...@gmail.com> 于2019年3月31日周日 上午2:44写道：
0001-Support-and-detection-of-hugepagesize-in-the-runtime.patch
Reply all
Reply to author
Forward
0 new messages