Sybase ASE 15.0.3 and RedHat AS 5 Hugepage setup

[Amer]

I did not find any thing well documented anywhere, so I spent some
time and came up with the working setup of ASE with linux hugepage. I
have not tested its performance yet, but I am able to configure and
utilize huge pages in linux with ASE. Hope this helps other Admins out
there.

=========================================================================================
Sybase ASE 15.0.3 and RedHat Hugepage setup

Hardware: HP Proliant DL585 G1
ASE Version: Adaptive Server Enterprise/15.0.3/EBF 17687 ESD#1.1
RedHat Version: Red Hat Enterprise Linux Server release 5.4 (Tikanga)
RedHat Kernel: [2.6.18-164.el5 #1 SMP Tue Aug 18 15:51:48 EDT 2009
x86_64 x86_64 x86_64 GNU/Linux]

The following steps will setup 25GB of hugepages for use with ASE

HP Server Total Physical Memory: 32GB
ASE Server Total Physical Memory: 25GB (As shown below in ASE cfg
file)
********************************************************************
[Physical Memory]
max memory = 12800000
additional network memory = 8192
shared memory starting address = DEFAULT
allocate max shared memory = 1
dynamic allocation on demand = DEFAULT
lock shared memory = 1
heap memory per user = DEFAULT
engine memory log size = DEFAULT
compression memory size = DEFAULT
*********************************************************************
==============================================================================

Following lines were added in /etc/sysctl.conf
*********************************************************************
# Controls the maximum shared segment size, in bytes -- 32GB
kernel.shmmax = 32212254720

# Controls the maximum number of shared memory segments, in pages
kernel.shmall = 4294967296

# Huge pagesetup example: For 25GB, (25*1024*1024)/2048=12800
vm.nr_hugepages = 12800

# Add the gid to the hugetlb_shm_group to give access to the sybase
users
vm.hugetlb_shm_group = 502

kernel.exec-shield=0
**********************************************************************
Good reference for hugepage info: http://unixfoo.blogspot.com/2007/10/hugepages.html
Good reference for hugepage setup: https://bugzilla.redhat.com/show_bug.cgi?id=426877

=================================================================================

Following lines were added in /etc/security/limits.conf
**********************************************************************
#For sybase user... Setting memlock value of 25GB:
25*1024*1024=26214400
sybase hard memlock 26214400
sybase soft memlock 26214400

# End of file
**********************************************************************

************************
REBOOT MACHINE
************************

=================================================================================

The following is a very good script to monitor the usage of hugepages
while you
are configuring ASE for the use. Save in a .sh file and run it. Make
sure that
the first and second values are either the same or your ASE is
configured to use
not more than the value in second column. If your ASE attempts to
allocate more
than available hugepages, the whole system will freeze and you will
then have to
hard reboot to fix the issue.
Here is the script: First value is total configured hugepage, second
is available
hugepages, third value is the reserved hugepage
**********************************************************************
[root@cpsybtest ~]# cat checkMeminfo.sh
while true; do
for i in $(grep ^Huge /proc/meminfo | head -3 | awk '{print $2}'); do
echo -n "$i "
done
echo ""
sleep 5
done
**********************************************************************
================================================================================

If all goes well, you ASE server log will say something like this:
**********************************************************************
00:00000:00000:2010/03/11 03:34:56.14 kernel Checked out license
ASE_CORE
00:00000:00000:2010/03/11 03:34:56.14 kernel Adaptive Server
Enterprise (Enterprise Edition)
00:00000:00000:2010/03/11 03:34:56.21 kernel Using config area from
primary master device.
00:00000:00000:2010/03/11 03:34:56.25 kernel Locking shared memory
into physical memory.
00:00000:00000:2010/03/11 03:35:14.26 kernel Internal run-time model
set for Linux - Native
00:00000:00000:2010/03/11 03:35:14.27 kernel Using 1024 file
descriptors.
00:00000:00000:2010/03/11 03:35:14.27 kernel Adaptive Server
Enterprise/15.0.3/EBF 17687 ESD#1.1 RELSE/P/x86_64/Enterprise Linux/
ase1503/2681/64-bit/FBO/Thu Aug 20 17:17:06 2009
0
***********************************************************************

================================================================================
if it says something like the following, then you configured something
wrong and
you should check the output of sysctl -a, ulimit -l for sybase and max
memory in
ASE cfg.
***********************************************************************
00:00000:00000:2010/03/11 03:20:19.41 kernel Checked out license
ASE_CORE
00:00000:00000:2010/03/11 03:20:19.41 kernel Adaptive Server
Enterprise (Enterprise Edition)
00:00000:00000:2010/03/11 03:20:19.42 kernel Using config area from
primary master device.
00:00000:00000:2010/03/11 03:20:19.43 kernel Locking shared memory
into physical memory.
00:00000:00000:2010/03/11 03:20:19.43 kernel Could not allocate
memory using Huge Pages. Allocated using regular pages. For better
performance, reboot the server after configuri
ng enough Huge Pages.
************************************************************************
Please use the information here at your own risk.
Good luck! Originally created by Amer Khan -- Sybase Technology
Administrator, Canpar Transport Ltd.
=================================================================

[shalinip...@gmail.com]

Hi..,
I have installed Sybase ASE 15.0.3 on my RHEL machine 5.7.And whenever am starting my ASE server am getting the message :

00:00000:00000:2010/03/11 03:20:19.43 kernel Could not allocate
memory using Huge Pages. Allocated using regular pages. For better
performance, reboot the server after configuring enough Huge Pages.

Tried like you have said Above.But it still keep giving me the same message.How to go with it?

Have max_memory allocated for sybase :
1. sp_configure 'max memory', 250000
go

2. Linux MEM INFO:
Have enough free memory:

[root@mgmt-cm-rhl datadir]# cat /proc/meminfo
MemTotal: 2057672 kB
MemFree: 559784 kB
.....................
HugePages_Total: 250
HugePages_Free: 250
HugePages_Rsvd: 0
Hugepagesize: 2048 kB
[root@mgmt-cm-rhl datadir]#

3. [root@mgmt-cm-rhl datadir]# tail -10 /etc/sysctl.conf
kernel.shmmax = 512000000
# sysctl -w vm.nr_hugepages=250
vm.nr_hugepages = 250

# Add the gid to the hugetlb_shm_group to give access to the sybase
users

vm.hugetlb_shm_group = 510

kernel.exec-shield=0
[root@mgmt-cm-rhl datadir]#

4.[root@mgmt-cm-rhl datadir]# tail -3 /etc/security/limits.conf
sybase hard memlock 512000
sybase soft memlock 512000
# End of file
[root@mgmt-cm-rhl datadir]#

Rebooted the server after these setting.But nothing worked for me.
I found sybase doc , saying this concept exist only for p-series machine.
How to find the same on RHEL machine?I couldn't find that either.

Thanks.

[ser...@gmail.com]

Are you sure that your numbers are correct?

Because for 24GB i have higher values for hugepage.

Hugepage Size: cat /proc/meminfo|grep Hugepagesize

Memory based on ~27GB (Rounded to more 3GB)

Reserved Memory(27648Mb) /Hupegasize(2Mb) = 13824

Added to /etc/sysctl.conf
#SYBASE hugepage reserved
vm.nr_hugepages = 13824
vm.hugetlb_shm_group = 1002

Add limits to system
Reserved Memory(13824Mb) * Hupegasize(2048K) = 28311552

vim /etc/security/limits.conf
sybase soft memlock 28311552
sybase hard memlock 28311552

Best Regards,
SM