Dynamically allocated new virtual shared memory segment repeatdly

[marcelo jaime]

Since a long time we have some troubles with Shared Memory segments assigned by the engine.

When we run an onstat - g seg, this one shows that is always assigning new segments and it seems that they never release them, because sometimes appears the following error:

[ENOMEM][12]: out of available data space, check system memory
parameters (e.g. MAXMEM)

How can we know the process these segments are generating? How can we release them, in case we could do it, without shut down the base?

Thanks in advance

[Art Kagel]

You can release shared memory segments (except for the RESIDENT segment, Message segments, and the first virtual segment) by running "onmode -F".  You don't state what version you are running, so I'll warn you that on some IDS 7.3x/9.30 and earlier releases running onmode -F can sometimes crash the engine especially if you run it often, so try it the first few times during off times to make sure it is safe if you have an older version of Informix.

To prevent allocation of new memory resources, you have two choices: 

• Expand the initial virtual segment size to include all of the extra memory that's being allocated over time (SHMVIRTSIZE) or
  
• find the complex queries and sorts that are making the requests for more memory and eliminate or simplify them. 

Now, hitting the error 12 means that the OS would not give Informix any more memory.  That can be caused by several factors:

• The kernel parameter that controls how much memory a single process can have is set too low.  Solution: tune he kernel parameter (usually MAXMEM)
  
• The kernel parameter that controls how much shared memory a single process can have or the number of shared memory segments a process can attache to is set too low.  Solution: tune the kernel (usually SHMMAX or SHMNMN)
  
• You do not have enough physical memory plus swap to permit the engine to allocate any more memory.  Solution: buy and install more memory and/or create more swap space.
  
• You are running a 32bit version of Informix and require access to more than the 2-3 GB of memory that a 32bit process can address.  Solution: Upgrade to a 64bit version of Informix.
  

Art

Art S. Kagel
Advanced DataTools (www.advancedatatools.com)
IIUG Board of Directors (a...@iiug.org)

Disclaimer: Please keep in mind that my own opinions are my own opinions and do not reflect on my employer, Advanced DataTools, the IIUG, nor any other organization with which I am associated either explicitly, implicitly, or by inference.  Neither do those opinions reflect those of other individuals affiliated with any entity with which I am affiliated nor those of the entities themselves.

_______________________________________________
Informix-list mailing list
Inform...@iiug.org
http://www.iiug.org/mailman/listinfo/informix-list

[marcelo jaime]

Dear ART,

 

Thank you for your answer.

 

My Informix version is 10.00 UC6 (i know 9/2010 end of support...) On Linux Box (RHL).

 

I would like to know, besides complex queries and sort, which other actions could generate the increase on these segments? Views?
Is there any tool, besides Set Explain On (4GL), Trace on (SPL) that allows me  to see why these segments keep on increasing and why these virtual memory segments are not liberated?.

 

Thank you,

Marcelo

[Art Kagel]

The engine will not free a virtual segment once it has been created under the assumption that if the space was needed once, it will be needed again.  Some of the reasons for the multiple segments can be:

• SHMADD is too small
• Multiple sessions needed space simultaneously where if sequentially less would be needed
• The segments space became fragmented so that later queries couldn't get enough contiguous memory and so allocated another segment.

Realistically, these all resolve back to SHMADD is too small.  And of course, their presence at all indicates - as I already said - that probably you should increase SHMVIRTSIZE.

If you SET EXPLAIN ON in some or all of your applications you can see which queries are sorting, creating temporary indexes, or creating Hash Tables and which are performing GROUP BY operations.  These are the operations, in general, which will require additional memory to complete.

There's no command that will tell you instantly which sessions allocated memory.  You could query the SMI tables about sessions and threads and gather some of this to narrow down the search, but that's so time consuming that normally the session that caused the problem is gone before you can drill down that far.

Art

Art S. Kagel
Advanced DataTools (www.advancedatatools.com)
IIUG Board of Directors (a...@iiug.org)

Disclaimer: Please keep in mind that my own opinions are my own opinions and do not reflect on my employer, Advanced DataTools, the IIUG, nor any other organization with which I am associated either explicitly, implicitly, or by inference.  Neither do those opinions reflect those of other individuals affiliated with any entity with which I am affiliated nor those of the entities themselves.

[Superboer]

hmm long shot..

you could look at onstat -g ses and see who eats memory;
then you could do a onstat -g ses <sid>
in the past a huge ralloc was most of the time caused by not closeing/
freeing cursors/statements.
if i recall correctly you can do a onstat -g stm <sid>
which displays the sql for the session, maybe it is a pointer which
can be used to
kick a developer who forgets to close and free cursors/statements....

Superboer

On 23 jul, 17:30, Art Kagel <art.ka...@gmail.com> wrote:
> The engine will not free a virtual segment once it has been created under
> the assumption that if the space was needed once, it will be needed again.
> Some of the reasons for the multiple segments can be:
>

>    - SHMADD is too small
>    - Multiple sessions needed space simultaneously where if sequentially
>    less would be needed
>    - The segments space became fragmented so that later queries couldn't get

>    enough contiguous memory and so allocated another segment.
>
> Realistically, these all resolve back to SHMADD is too small.  And of
> course, their presence at all indicates - as I already said - that probably
> you should increase SHMVIRTSIZE.
>
> If you SET EXPLAIN ON in some or all of your applications you can see which
> queries are sorting, creating temporary indexes, or creating Hash Tables and
> which are performing GROUP BY operations.  These are the operations, in
> general, which will require additional memory to complete.
>
> There's no command that will tell you instantly which sessions allocated
> memory.  You could query the SMI tables about sessions and threads and
> gather some of this to narrow down the search, but that's so time consuming
> that normally the session that caused the problem is gone before you can
> drill down that far.
>
> Art
>
> Art S. Kagel
> Advanced DataTools (www.advancedatatools.com)
> IIUG Board of Directors (a...@iiug.org)
>
> Disclaimer: Please keep in mind that my own opinions are my own opinions and
> do not reflect on my employer, Advanced DataTools, the IIUG, nor any other
> organization with which I am associated either explicitly, implicitly, or by
> inference.  Neither do those opinions reflect those of other individuals
> affiliated with any entity with which I am affiliated nor those of the
> entities themselves.
>

> >    - Expand the initial virtual segment size to include all of the extra

> >    memory that's being allocated over time (SHMVIRTSIZE) or

> >    - find the complex queries and sorts that are making the requests for

> >    more memory and eliminate or simplify them.
>
> > Now, hitting the error 12 means that the OS would not give Informix any
> > more memory.  That can be caused by several factors:
>

> >    - The kernel parameter that controls how much memory a single process

> >    can have is set too low.  Solution: tune he kernel parameter (usually
> >    MAXMEM)

> >    - The kernel parameter that controls how much shared memory a single

> >    process can have or the number of shared memory segments a process can
> >    attache to is set too low.  Solution: tune the kernel (usually SHMMAX or
> >    SHMNMN)

> >    - You do not have enough physical memory plus swap to permit the engine

> >    to allocate any more memory.  Solution: buy and install more memory and/or
> >    create more swap space.

> >    - You are running a 32bit version of Informix and require access to

> >    more than the 2-3 GB of memory that a 32bit process can address.  Solution:
> >    Upgrade to a 64bit version of Informix.
>
> > Art
>
> > Art S. Kagel
> > Advanced DataTools (www.advancedatatools.com)
> > IIUG Board of Directors (a...@iiug.org)
>
> > Disclaimer: Please keep in mind that my own opinions are my own opinions
> > and do not reflect on my employer, Advanced DataTools, the IIUG, nor any
> > other organization with which I am associated either explicitly, implicitly,
> > or by inference.  Neither do those opinions reflect those of other
> > individuals affiliated with any entity with which I am affiliated nor those
> > of the entities themselves.
>

> > On Thu, Jul 22, 2010 at 4:02 PM, marcelo jaime <emetej...@gmail.com>wrote:
>
> >> Since a long time we have some troubles with Shared Memory segments
> >> assigned by the engine.
> >> When we run an onstat - g seg, this one shows that is always assigning new
> >> segments and it seems that they never release them, because sometimes
> >> appears the following error:
> >> [ENOMEM][12]: out of available data space, check system memory
> >> parameters (e.g. MAXMEM)
> >> How can we know the process these segments are generating? How can we
> >> release them, in case we could do it, without shut down the base?
>
> >> Thanks in advance
>
> >> _______________________________________________
> >> Informix-list mailing list

> >> Informix-l...@iiug.org

> >>http://www.iiug.org/mailman/listinfo/informix-list
>
> > _______________________________________________
> > Informix-list mailing list

> > Informix-l...@iiug.org
> >http://www.iiug.org/mailman/listinfo/informix-list

[Brian Amstutz]

We're running

Informix 10.00.FC9
HPUX 11iv2 (Itanium)

Last weekend, I added a second DUAL core processor to our HP rx2260 server
(so now we have a total of 4 processors - i.e. 2 dual core).

I changed the following:

MULTIPROCESSOR from 0 to 1
SINGLE_CPU_VP from 1 to 0
VPCLASS from cpu,num=1,max=2,noage to cpu,num=3,max=4,noage

Anything else I should consider changing?

Thanks!

Brian

~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Brian Amstutz
Enterprise Systems Manager
Asbury Theological Seminary
204 North Lexington Ave.
Wilmore, KY 40390
859.858.2321
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

[Art Kagel]

Nope, you've got it nailed.

Art

Art S. Kagel
Advanced DataTools (www.advancedatatools.com)
IIUG Board of Directors (a...@iiug.org)

Disclaimer: Please keep in mind that my own opinions are my own opinions and do not reflect on my employer, Advanced DataTools, the IIUG, nor any other organization with which I am associated either explicitly, implicitly, or by inference.  Neither do those opinions reflect those of other individuals affiliated with any entity with which I am affiliated nor those of the entities themselves.

[Brian Amstutz]

Thanks for the confirmation Art!

Brian

---