Sun box runs out of swap when starting Oracle

[vsevolod afanassiev]

We have legacy system running Oracle 9.2.0.8.
It is being migrated to new M5000 (Solaris 10). We are running Oracle
on the physical server, not on Solaris Zone. The server has 128 GB
memory, Oracle is configured with 50 GB SGA, 30 GB buffer cache, and 3
GB shared pool.
"top" shows that around 80 GB memory is free. Swap is 8 GB, all free.
Following Solaris tradition we have several tmpfs filesystems:

Filesystem kbytes used avail capacity Mounted on
swap 18452376 448 18451928 1% /etc/svc/
volatile
swap 18451928 0 18451928 0% /tmp
swap 18451992 64 18451928 1% /var/run

This is test server and it is idle, there are no connections to the
database, I am the only person connected to the server.

When I tried to start Oracle with 70 GB SGA it started but server run
out of swap.

/var/adm/messages contains:

tmpfs: [ID 518458 kern.warning] WARNING: /etc/svc/volatile: File
system full, swap space limit exceeded
genunix: [ID 470503 kern.warning] WARNING: Sorry, no swap space to
grow stack for pid 2291 (scopeux)

alert.log contains:

ORA-04030: out of process memory when trying to allocate 48000 bytes
(pga heap,non-vector post process batch)

The server stopped responding and we had to restart it.

Any suggestion what could be done to avoid this problem? (other than
increasing swap to 128 GB)

[John Hurley]

On Sep 26, 6:36 pm, vsevolod afanassiev

Open an SR with Oracle?

[Mladen Gogala]

On Mon, 26 Sep 2011 15:36:59 -0700, vsevolod afanassiev wrote:

> We have legacy system running Oracle 9.2.0.8. It is being migrated to
> new M5000 (Solaris 10). We are running Oracle on the physical server,
> not on Solaris Zone. The server has 128 GB memory, Oracle is configured
> with 50 GB SGA, 30 GB buffer cache, and 3 GB shared pool.
> "top" shows that around 80 GB memory is free. Swap is 8 GB, all free.

Swap looks like 18GB to me.

> Following Solaris tradition we have several tmpfs filesystems:

Tradition? Why do you need more than a single swap space? What is that
good for?

>
> Filesystem kbytes used avail capacity Mounted on swap
> 18452376 448 18451928 1% /etc/svc/ volatile
> swap 18451928 0 18451928 0% /tmp swap
> 18451992 64 18451928 1% /var/run
>
> This is test server and it is idle, there are no connections to the
> database, I am the only person connected to the server.
>
> When I tried to start Oracle with 70 GB SGA it started but server run
> out of swap.

I dunno about Slowaris, but Linux has a facility called huge pages which
enables DBA people to allocate RAM which is not being paged. Slowaris
must have the same capability. I read something about zones long time
ago, but Slowaris is not really my forte.

> Any suggestion what could be done to avoid this problem? (other than
> increasing swap to 128 GB)

Get a decent Slowaris system administrator who uses logic and knowledge
to administer the machines. This is really a Slowaris question which
would probably be better answered on one of the Slowaris newsgroups. As
for identifying a good Slowaris SA, it's simple. Good Slowaris SA knows
the airspeed velocity of an unladen swallow.




--
http://mgogala.byethost5.com

[vsevolod afanassiev]

After reading a bit more on the subject I've found that Oracle uses
DISM (Dynamic ISM) pages,
this can be seen using "pmap -sx":

0000000380000000 4096 4096 - - 4M
rwxs- [ dism shmid=0x16 ]
0000000380400000 15822848 - - - -
rwxs- [ dism shmid=0x16 ]
0000000746000000 180224 180224 - - 4M
rwxs- [ dism shmid=0x16 ]
0000000751000000 8192 8192 - - -
rwxs- [ dism shmid=0x16 ]
0000000751800000 8192 8192 - - 4M
rwxs- [ dism shmid=0x16 ]

There is Solaris paper "Dynamic Reconfiguration and Oracle9i
Dynamically Resizable SGA",
it states that Oracle 9i uses DISM if SGA size > size of its
components which is true in our case.

Experimenting with another database that has 1 GB SGA I was able to
get it to use ISM instead of DISM
after increasing db_cache_size:

000000010362C000 8 8 8 - 8K
rwx-- [ heap ]
0000000380000000 1069056 1069056 - 1069056 4M
rwxsR [ ism shmid=0x2000036 ]
FFFFFFFF7D100000 8 8 8 - 8K
rwx-- [ anon ]
FFFFFFFF7D102000 8 - - - -
rwx-- [ anon ]

[joel garry]

On Sep 26, 5:24 pm, vsevolod afanassiev

This is such a common issue I've been telling people to search for
DISM on MOS for any similar memory issue before even bothering with OS
tools. I was burned by the similar ISM issue 15 years ago.

jg
--
@home.com is bogus.
"This is a very big machine."
http://www.computerworlduk.com/news/infrastructure/3306349/oracle-sparc-supercluster-due-by-end-of-the-year/