Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
Informix 11.70.FC4 (Innovator-C edition) crash on Linux
671 views
Skip to first unread message
checc...@libero.it
Dec 18, 2012, 4:37:34 AM12/18/12
to
Informix 11.70.FC4IE
O.S. HP-UX Version B.11.31
The crash happened after 6 months without instance stops.
Memory limitation forced to 2 GB (Innovator-C max accepted):
RESIDENT 1
SHMBASE 0x0
SHMVIRTSIZE 1041176
SHMADD 32768
EXTSHMADD 32768
SHMTOTAL 2097152
SHMVIRT_ALLOCSEG 0.000000
SHMNOACCESS cpu,num=3,noage
Below some details from the af generated by the crash:
03:04:01 Warning: The IBM IDS Innovator-C Edition license restriction limits
03:04:01 the total shared memory size for this server to 2097152 KB.
03:04:01 Size of resident + virtual segments 1055428 KB + 1074492 KB > 2097152 KB.
03:04:01 Size of resident + virtual segments 1055428KB + 1074492KB > 2097152KB
03:04:01 total allowed by configuration parameter SHMTOTAL
03:04:01 out of virtual shared memory
03:04:01 Warning: The IBM IDS Innovator-C Edition license restriction limits
03:04:01 the total shared memory size for this server to 2097152 KB.
03:04:01 Size of resident + virtual segments 1055428 KB + 1074492 KB > 2097152 KB.
03:04:01 Warning: The IBM IDS Innovator-C Edition license restriction limits
03:04:01 the total shared memory size for this server to 2097152 KB.
03:04:01 Size of resident + virtual segments 1055428 KB + 1074492 KB > 2097152 KB.
03:04:01 Warning: The IBM IDS Innovator-C Edition license restriction limits
03:04:01 the total shared memory size for this server to 2097152 KB.
03:04:01 Size of resident + virtual segments 1055428 KB + 1074492 KB > 2097152 KB.
03:04:01 Grid Command Failed
GRID
SQL Statement Error SQLERR:-208 ISAM:0
select * from grid_cdr_node_tab where gn_need_sync = 'y'
03:04:01 ER Monitor received an error from the Internal Replicate Sync subcomponent.
Disabling Monitoring of this subcomponent.
03:04:01 stack trace for pid 17874 written to /opt/IBM/informix/tmp/af.e0638911
03:04:02 Assert Failed: No Exception Handler
03:04:02 IBM Informix Dynamic Server Version 11.70.FC4IE
03:04:02 Who: Session(1696417, replica@rp34, -1, c00000009d1df468)
Thread(1891451, sqlexec, c0000000976ed918, 9)
File: mtex.c Line: 498
03:04:02 Results: Exception Caught. Type: MT_EX_OS, Context: mem
03:04:02 Action: Please notify IBM Informix Technical Support.
03:04:02 See Also: /opt/IBM/informix/tmp/af.e0638911, shmem.e0638911.0
03:04:14 Error writing '/opt/IBM/informix/tmp/shmem.e0638911.0' errno = 28
03:04:14 mtex.c, line 498, thread 1891451, proc id 17874, No Exception Handler.
03:04:15 Fatal error in ADM VP at mt.c:14022
03:04:15 Unexpected virtual processor termination, pid = 17874, exit = 0x100
03:04:15 PANIC: Attempting to bring system down
08:17:36 IBM Informix Dynamic Server Started.
08:17:37 Segment locked: addr=c000000040000000, size=1080758272
Fri Dec 14 08:17:38 2012
08:17:38 Event alarms enabled. ALARMPROG = '/opt/IBM/informix/etc/alarmprogram.sh'
08:17:38 Booting Language <c> from module <>
08:17:38 Loading Module <CNULL>
08:17:38 Booting Language <builtin> from module <>
08:17:38 Loading Module <BUILTINNULL>
08:17:44 DR: DRAUTO is 0 (Off)
08:17:44 DR: ENCRYPT_HDR is 0 (HDR encryption Disabled)
08:17:44 Fast poll /dev/poll enabled.
08:17:44 IBM Informix Dynamic Server Version 11.70.FC4IE Software Serial Number AAA#B000000
08:17:44 HPUX Version B.11.31 -> Using flag/select style KAIO
08:17:44 HP KAIO concurrent requests changed from 1000 to 4000
08:17:45 (11) connection rejected - no calls allowed for sqlexec
08:17:46 (12) connection rejected - no calls allowed for sqlexec
08:17:47 (13) connection rejected - no calls allowed for sqlexec
08:17:47 (14) connection rejected - no calls allowed for sqlexec
08:17:47 (15) connection rejected - no calls allowed for sqlexec
08:17:47 (16) connection rejected - no calls allowed for sqlexec
08:17:47 (17) connection rejected - no calls allowed for sqlexec
08:17:47 (18) connection rejected - no calls allowed for sqlexec
08:17:47 (19) connection rejected - no calls allowed for sqlexec
08:17:47 (20) connection rejected - no calls allowed for sqlexec
08:17:47 (21) connection rejected - no calls allowed for sqlexec
08:17:47 (22) connection rejected - no calls allowed for sqlexec
08:17:47 (23) connection rejected - no calls allowed for sqlexec
08:17:48 (24) connection rejected - no calls allowed for sqlexec
08:17:48 (25) connection rejected - no calls allowed for sqlexec
08:17:48 (26) connection rejected - no calls allowed for sqlexec
08:17:48 (27) connection rejected - no calls allowed for sqlexec
08:17:48 (28) connection rejected - no calls allowed for sqlexec
08:17:48 (29) connection rejected - no calls allowed for sqlexec
08:17:48 (30) connection rejected - no calls allowed for sqlexec
08:17:48 (31) connection rejected - no calls allowed for sqlexec
08:17:48 (32) connection rejected - no calls allowed for sqlexec
08:17:48 (33) connection rejected - no calls allowed for sqlexec
08:17:48 (34) connection rejected - no calls allowed for sqlexec
08:17:48 (35) connection rejected - no calls allowed for sqlexec
08:17:48 (36) connection rejected - no calls allowed for sqlexec
08:17:48 (37) connection rejected - no calls allowed for sqlexec
08:17:48 (38) connection rejected - no calls allowed for sqlexec
08:17:48 IBM Informix Dynamic Server Initialized -- Shared Memory Initialized.
08:17:48 Physical Recovery Started at Page (24:545022).
08:17:49 Physical Recovery Complete: 2719 Pages Examined, 2719 Pages Restored.
08:17:49 Logical Recovery Started.
08:17:49 10 recovery worker threads will be started.
08:17:52 Logical Recovery has reached the transaction cleanup phase.
08:17:52 Logical Recovery Complete.
13212 Committed, 0 Rolled Back, 0 Open, 0 Bad Locks
------------------------------------------------------------------------------
03:04:02 Stack for thread: 1891451 sqlexec
base: 0xc0000000aa3fb000
len: 102400
pc: 0x0000000000000000
tos: 0xc0000000aa40d990
state: running
vp: 9
0x40000000008a9f00 (ormix11.70IE/bin/oninit)afstack
0x400000000082d390 (ormix11.70IE/bin/oninit)mt_ex_throw_sig
0x4000000002bb35c0 (ormix11.70IE/bin/oninit)afsig_handler
0x400000000134fad0 (ormix11.70IE/bin/oninit)mkridnode
0x400000000212aa00 (ormix11.70IE/bin/oninit)rowidlabel2expr
0x4000000000db79a0 (ormix11.70IE/bin/oninit)col2expr
0x4000000000db56d0 (ormix11.70IE/bin/oninit)dotted2expr
0x4000000000c25d70 (ormix11.70IE/bin/oninit)expr2expr
0x4000000002294d00 (ormix11.70IE/bin/oninit)op_bind_rtn
0x400000000228c7c0 (ormix11.70IE/bin/oninit)buildexpr
0x4000000000c26640 (ormix11.70IE/bin/oninit)expr2expr
0x4000000001988a30 (ormix11.70IE/bin/oninit)cond2expr
0x4000000002157e60 (ormix11.70IE/bin/oninit)upd2cb
0x4000000000da4500 (ormix11.70IE/bin/oninit)stmt2cb
0x40000000010e6b20 (ormix11.70IE/bin/oninit)p_sp2cb
0x40000000022ae530 (ormix11.70IE/bin/oninit)parsecmd
0x40000000022ada40 (ormix11.70IE/bin/oninit)sqcmd
0x4000000000bee1a0 (ormix11.70IE/bin/oninit)sq_cmnd
0x40000000022b31f0 (ormix11.70IE/bin/oninit)sq_prepare
0x40000000010ae0c0 (ormix11.70IE/bin/oninit)sqmain
0x4000000002723f70 (ormix11.70IE/bin/oninit)listen_verify
0x400000000271ef90 (ormix11.70IE/bin/oninit)spawn_thread
0x4000000000870750 (ormix11.70IE/bin/oninit)startup
0x4000000003111100 (ormix11.70IE/bin/oninit)coroutine_start
Any idea about this problem?
Thanks
Francesco
O.S. HP-UX Version B.11.31
The crash happened after 6 months without instance stops.
Memory limitation forced to 2 GB (Innovator-C max accepted):
RESIDENT 1
SHMBASE 0x0
SHMVIRTSIZE 1041176
SHMADD 32768
EXTSHMADD 32768
SHMTOTAL 2097152
SHMVIRT_ALLOCSEG 0.000000
SHMNOACCESS cpu,num=3,noage
Below some details from the af generated by the crash:
03:04:01 Warning: The IBM IDS Innovator-C Edition license restriction limits
03:04:01 the total shared memory size for this server to 2097152 KB.
03:04:01 Size of resident + virtual segments 1055428 KB + 1074492 KB > 2097152 KB.
03:04:01 Size of resident + virtual segments 1055428KB + 1074492KB > 2097152KB
03:04:01 total allowed by configuration parameter SHMTOTAL
03:04:01 out of virtual shared memory
03:04:01 Warning: The IBM IDS Innovator-C Edition license restriction limits
03:04:01 the total shared memory size for this server to 2097152 KB.
03:04:01 Size of resident + virtual segments 1055428 KB + 1074492 KB > 2097152 KB.
03:04:01 Warning: The IBM IDS Innovator-C Edition license restriction limits
03:04:01 the total shared memory size for this server to 2097152 KB.
03:04:01 Size of resident + virtual segments 1055428 KB + 1074492 KB > 2097152 KB.
03:04:01 Warning: The IBM IDS Innovator-C Edition license restriction limits
03:04:01 the total shared memory size for this server to 2097152 KB.
03:04:01 Size of resident + virtual segments 1055428 KB + 1074492 KB > 2097152 KB.
03:04:01 Grid Command Failed
GRID
SQL Statement Error SQLERR:-208 ISAM:0
select * from grid_cdr_node_tab where gn_need_sync = 'y'
03:04:01 ER Monitor received an error from the Internal Replicate Sync subcomponent.
Disabling Monitoring of this subcomponent.
03:04:01 stack trace for pid 17874 written to /opt/IBM/informix/tmp/af.e0638911
03:04:02 Assert Failed: No Exception Handler
03:04:02 IBM Informix Dynamic Server Version 11.70.FC4IE
03:04:02 Who: Session(1696417, replica@rp34, -1, c00000009d1df468)
Thread(1891451, sqlexec, c0000000976ed918, 9)
File: mtex.c Line: 498
03:04:02 Results: Exception Caught. Type: MT_EX_OS, Context: mem
03:04:02 Action: Please notify IBM Informix Technical Support.
03:04:02 See Also: /opt/IBM/informix/tmp/af.e0638911, shmem.e0638911.0
03:04:14 Error writing '/opt/IBM/informix/tmp/shmem.e0638911.0' errno = 28
03:04:14 mtex.c, line 498, thread 1891451, proc id 17874, No Exception Handler.
03:04:15 Fatal error in ADM VP at mt.c:14022
03:04:15 Unexpected virtual processor termination, pid = 17874, exit = 0x100
03:04:15 PANIC: Attempting to bring system down
08:17:36 IBM Informix Dynamic Server Started.
08:17:37 Segment locked: addr=c000000040000000, size=1080758272
Fri Dec 14 08:17:38 2012
08:17:38 Event alarms enabled. ALARMPROG = '/opt/IBM/informix/etc/alarmprogram.sh'
08:17:38 Booting Language <c> from module <>
08:17:38 Loading Module <CNULL>
08:17:38 Booting Language <builtin> from module <>
08:17:38 Loading Module <BUILTINNULL>
08:17:44 DR: DRAUTO is 0 (Off)
08:17:44 DR: ENCRYPT_HDR is 0 (HDR encryption Disabled)
08:17:44 Fast poll /dev/poll enabled.
08:17:44 IBM Informix Dynamic Server Version 11.70.FC4IE Software Serial Number AAA#B000000
08:17:44 HPUX Version B.11.31 -> Using flag/select style KAIO
08:17:44 HP KAIO concurrent requests changed from 1000 to 4000
08:17:45 (11) connection rejected - no calls allowed for sqlexec
08:17:46 (12) connection rejected - no calls allowed for sqlexec
08:17:47 (13) connection rejected - no calls allowed for sqlexec
08:17:47 (14) connection rejected - no calls allowed for sqlexec
08:17:47 (15) connection rejected - no calls allowed for sqlexec
08:17:47 (16) connection rejected - no calls allowed for sqlexec
08:17:47 (17) connection rejected - no calls allowed for sqlexec
08:17:47 (18) connection rejected - no calls allowed for sqlexec
08:17:47 (19) connection rejected - no calls allowed for sqlexec
08:17:47 (20) connection rejected - no calls allowed for sqlexec
08:17:47 (21) connection rejected - no calls allowed for sqlexec
08:17:47 (22) connection rejected - no calls allowed for sqlexec
08:17:47 (23) connection rejected - no calls allowed for sqlexec
08:17:48 (24) connection rejected - no calls allowed for sqlexec
08:17:48 (25) connection rejected - no calls allowed for sqlexec
08:17:48 (26) connection rejected - no calls allowed for sqlexec
08:17:48 (27) connection rejected - no calls allowed for sqlexec
08:17:48 (28) connection rejected - no calls allowed for sqlexec
08:17:48 (29) connection rejected - no calls allowed for sqlexec
08:17:48 (30) connection rejected - no calls allowed for sqlexec
08:17:48 (31) connection rejected - no calls allowed for sqlexec
08:17:48 (32) connection rejected - no calls allowed for sqlexec
08:17:48 (33) connection rejected - no calls allowed for sqlexec
08:17:48 (34) connection rejected - no calls allowed for sqlexec
08:17:48 (35) connection rejected - no calls allowed for sqlexec
08:17:48 (36) connection rejected - no calls allowed for sqlexec
08:17:48 (37) connection rejected - no calls allowed for sqlexec
08:17:48 (38) connection rejected - no calls allowed for sqlexec
08:17:48 IBM Informix Dynamic Server Initialized -- Shared Memory Initialized.
08:17:48 Physical Recovery Started at Page (24:545022).
08:17:49 Physical Recovery Complete: 2719 Pages Examined, 2719 Pages Restored.
08:17:49 Logical Recovery Started.
08:17:49 10 recovery worker threads will be started.
08:17:52 Logical Recovery has reached the transaction cleanup phase.
08:17:52 Logical Recovery Complete.
13212 Committed, 0 Rolled Back, 0 Open, 0 Bad Locks
------------------------------------------------------------------------------
03:04:02 Stack for thread: 1891451 sqlexec
base: 0xc0000000aa3fb000
len: 102400
pc: 0x0000000000000000
tos: 0xc0000000aa40d990
state: running
vp: 9
0x40000000008a9f00 (ormix11.70IE/bin/oninit)afstack
0x400000000082d390 (ormix11.70IE/bin/oninit)mt_ex_throw_sig
0x4000000002bb35c0 (ormix11.70IE/bin/oninit)afsig_handler
0x400000000134fad0 (ormix11.70IE/bin/oninit)mkridnode
0x400000000212aa00 (ormix11.70IE/bin/oninit)rowidlabel2expr
0x4000000000db79a0 (ormix11.70IE/bin/oninit)col2expr
0x4000000000db56d0 (ormix11.70IE/bin/oninit)dotted2expr
0x4000000000c25d70 (ormix11.70IE/bin/oninit)expr2expr
0x4000000002294d00 (ormix11.70IE/bin/oninit)op_bind_rtn
0x400000000228c7c0 (ormix11.70IE/bin/oninit)buildexpr
0x4000000000c26640 (ormix11.70IE/bin/oninit)expr2expr
0x4000000001988a30 (ormix11.70IE/bin/oninit)cond2expr
0x4000000002157e60 (ormix11.70IE/bin/oninit)upd2cb
0x4000000000da4500 (ormix11.70IE/bin/oninit)stmt2cb
0x40000000010e6b20 (ormix11.70IE/bin/oninit)p_sp2cb
0x40000000022ae530 (ormix11.70IE/bin/oninit)parsecmd
0x40000000022ada40 (ormix11.70IE/bin/oninit)sqcmd
0x4000000000bee1a0 (ormix11.70IE/bin/oninit)sq_cmnd
0x40000000022b31f0 (ormix11.70IE/bin/oninit)sq_prepare
0x40000000010ae0c0 (ormix11.70IE/bin/oninit)sqmain
0x4000000002723f70 (ormix11.70IE/bin/oninit)listen_verify
0x400000000271ef90 (ormix11.70IE/bin/oninit)spawn_thread
0x4000000000870750 (ormix11.70IE/bin/oninit)startup
0x4000000003111100 (ormix11.70IE/bin/oninit)coroutine_start
Any idea about this problem?
Thanks
Francesco
checc...@libero.it
Dec 18, 2012, 4:43:34 AM12/18/12
to
Sorry about the title...the operating system is HP-UX, not Linux.
However, below the onstat -g ses of the session that generated the crash:
/opt/IBM/informix/bin/onstat -g ses 1696417:
IBM Informix Dynamic Server Version 11.70.FC4IE -- On-Line -- Up 122 days 11:57:26 -- 2097152 Kbytes
session effective #RSAM total used dynamic
id user user tty pid hostname threads memory memory explain
1696417 replica - - -1 rp34 1 241664 232864 off
Program :
-
tid name rstcb flags curstk status
1891451 sqlexec c0000000976ed918 ---P--- 1774272832 running-
Memory pools count 2
name class addr totalsize freesize #allocfrag #freefrag
1696417 V c0000000a6786040 237568 8016 292 29
1696417*O0 V c0000000b4f6f040 4096 784 1 1
name free used name free used
overhead 0 6624 scb 0 176
opentable 0 10160 filetable 0 1840
ru 0 608 log 0 16544
temprec 0 21696 keys 0 640
ralloc 0 124688 gentcb 0 1728
ostcb 0 4048 sqscb 0 27280
sql 0 176 hashfiletab 0 560
osenv 0 2816 sqtcb 0 10928
fragman 0 1552 sapi 0 80
udr 0 192 rsam_seqscan 0 448
sqscb info
scb sqscb optofc pdqpriority optcompind directives
c0000000c105cc70 c0000000983f2030 0 0 2 1
Sess SQL Current Iso Lock SQL ISAM F.E.
Id Stmt type Database Lvl Mode ERR ERR Vers Explain
1696417 SELECT imxr CR Wait 3000 -208 0 9.28 Off
Current SQL statement (34593) :
update MMFG:MN_ORDINI_ARCHIVIO set id_mn_ord_archivio = ? , trasmissione =
? , anno = ? , stagione = ? , collezione = ? , tipo_ordine = ? ,
nazione_mn = ? , canale = ? , modello_mn = ? , variante_mn = ? , nome = ?
, listino_acquisto = ? , prezzo_acquisto_eu = ? , listino_vendita = ? ,
prezzo_vendita_eu = ? , data_saldo = ? , stato_chiusura = ? , dta_chiusura
= ? , stato = ? , qta_or1 = ? , qta_or2 = ? , qta_or3 = ? , qta_or4 = ? ,
qta_or5 = ? , qta_or6 = ? , qta_or7 = ? , qta_or8 = ? , dta_ins = ? ,
ute_ins = ? , dta_upd = ? , ute_upd = ? , raggr_saldo = ? , qta_op1 = ? ,
qta_op2 = ? , qta_op3 = ? , qta_op4 = ? , qta_op5 = ? , qta_op6 = ? ,
qta_op7 = ? , qta_op8 = ? , flg_annullo = ? , modulo = ? , destinazione =
? where rowid = ? ;
Host variables :
address type flags value
-----------------------------------------
0xc0000000cf6b0040 CHAR 0x002
0xc0000000cf6b00d0 CHAR 0x002
0xc0000000cf6b0160 CHAR 0x002
0xc0000000cf6b01f0 CHAR 0x002
0xc0000000cf6b0280 CHAR 0x002
0xc0000000cf6b0310 CHAR 0x002
0xc0000000cf6b03a0 CHAR 0x002
0xc0000000cf6b0430 CHAR 0x002
0xc0000000cf6b04c0 CHAR 0x002
0xc0000000cf6b0550 CHAR 0x002
0xc0000000cf6b05e0 CHAR 0x002
0xc0000000cf6b0670 CHAR 0x002
0xc0000000cf6b0700 CHAR 0x002
0xc0000000cf6b0790 CHAR 0x002
0xc0000000cf6b0820 CHAR 0x002
0xc0000000cf6b08b0 CHAR 0x002
0xc0000000cf6b0940 CHAR 0x002
0xc0000000cf6b09d0 CHAR 0x002
0xc0000000cf6b0a60 CHAR 0x002
0xc0000000cf6b0af0 CHAR 0x002
0xc0000000cf6b0b80 CHAR 0x002
0xc0000000cf6b0c10 CHAR 0x002
0xc0000000cf6b0ca0 CHAR 0x002
0xc0000000cf6b0d30 CHAR 0x002
0xc0000000cf6b0dc0 CHAR 0x002
0xc0000000cf6b0e50 CHAR 0x002
0xc0000000cf6b0ee0 CHAR 0x002
0xc0000000cf6b0f70 CHAR 0x002
0xc0000000cf6b1000 CHAR 0x002
0xc0000000cf6b1090 CHAR 0x002
0xc0000000cf6b1120 CHAR 0x002
0xc0000000cf6b11b0 CHAR 0x002
0xc0000000cf6b1240 CHAR 0x002
0xc0000000cf6b12d0 CHAR 0x002
0xc0000000cf6b1360 CHAR 0x002
0xc0000000cf6b13f0 CHAR 0x002
0xc0000000cf6b1480 CHAR 0x002
0xc0000000cf6b1510 CHAR 0x002
0xc0000000cf6b15a0 CHAR 0x002
0xc0000000cf6b1630 CHAR 0x002
0xc0000000cf6b16c0 CHAR 0x002
0xc0000000cf6b1750 CHAR 0x002
0xc0000000cf6b17e0 CHAR 0x002
0xc0000000cf6b1870 CHAR 0x002
Last parsed SQL statement :
update MMFG:MN_ORDINI_ARCHIVIO set id_mn_ord_archivio = ? , trasmissione =
? , anno = ? , stagione = ? , collezione = ? , tipo_ordine = ? ,
nazione_mn = ? , canale = ? , modello_mn = ? , variante_mn = ? , nome = ?
, listino_acquisto = ? , prezzo_acquisto_eu = ? , listino_vendita = ? ,
prezzo_vendita_eu = ? , data_saldo = ? , stato_chiusura = ? , dta_chiusura
= ? , stato = ? , qta_or1 = ? , qta_or2 = ? , qta_or3 = ? , qta_or4 = ? ,
qta_or5 = ? , qta_or6 = ? , qta_or7 = ? , qta_or8 = ? , dta_ins = ? ,
ute_ins = ? , dta_upd = ? , ute_upd = ? , raggr_saldo = ? , qta_op1 = ? ,
qta_op2 = ? , qta_op3 = ? , qta_op4 = ? , qta_op5 = ? , qta_op6 = ? ,
qta_op7 = ? , qta_op8 = ? , flg_annullo = ? , modulo = ? , destinazione =
? where rowid = ? ;
-----------------------------------------------------------------------------
The architecture is composed by 2 nodes only: one (primary) replicate some tables on a secondary-server // the replication is mono-directional. Can the message below be related to the panic?
03:04:01 Grid Command Failed
GRID
SQL Statement Error SQLERR:-208 ISAM:0
select * from grid_cdr_node_tab where gn_need_sync = 'y'
03:04:01 ER Monitor received an error from the Internal Replicate Sync subcomponent.
Disabling Monitoring of this subcomponent.
Still thanks
However, below the onstat -g ses of the session that generated the crash:
/opt/IBM/informix/bin/onstat -g ses 1696417:
IBM Informix Dynamic Server Version 11.70.FC4IE -- On-Line -- Up 122 days 11:57:26 -- 2097152 Kbytes
session effective #RSAM total used dynamic
id user user tty pid hostname threads memory memory explain
1696417 replica - - -1 rp34 1 241664 232864 off
Program :
-
tid name rstcb flags curstk status
1891451 sqlexec c0000000976ed918 ---P--- 1774272832 running-
Memory pools count 2
name class addr totalsize freesize #allocfrag #freefrag
1696417 V c0000000a6786040 237568 8016 292 29
1696417*O0 V c0000000b4f6f040 4096 784 1 1
name free used name free used
overhead 0 6624 scb 0 176
opentable 0 10160 filetable 0 1840
ru 0 608 log 0 16544
temprec 0 21696 keys 0 640
ralloc 0 124688 gentcb 0 1728
ostcb 0 4048 sqscb 0 27280
sql 0 176 hashfiletab 0 560
osenv 0 2816 sqtcb 0 10928
fragman 0 1552 sapi 0 80
udr 0 192 rsam_seqscan 0 448
sqscb info
scb sqscb optofc pdqpriority optcompind directives
c0000000c105cc70 c0000000983f2030 0 0 2 1
Sess SQL Current Iso Lock SQL ISAM F.E.
Id Stmt type Database Lvl Mode ERR ERR Vers Explain
1696417 SELECT imxr CR Wait 3000 -208 0 9.28 Off
Current SQL statement (34593) :
update MMFG:MN_ORDINI_ARCHIVIO set id_mn_ord_archivio = ? , trasmissione =
? , anno = ? , stagione = ? , collezione = ? , tipo_ordine = ? ,
nazione_mn = ? , canale = ? , modello_mn = ? , variante_mn = ? , nome = ?
, listino_acquisto = ? , prezzo_acquisto_eu = ? , listino_vendita = ? ,
prezzo_vendita_eu = ? , data_saldo = ? , stato_chiusura = ? , dta_chiusura
= ? , stato = ? , qta_or1 = ? , qta_or2 = ? , qta_or3 = ? , qta_or4 = ? ,
qta_or5 = ? , qta_or6 = ? , qta_or7 = ? , qta_or8 = ? , dta_ins = ? ,
ute_ins = ? , dta_upd = ? , ute_upd = ? , raggr_saldo = ? , qta_op1 = ? ,
qta_op2 = ? , qta_op3 = ? , qta_op4 = ? , qta_op5 = ? , qta_op6 = ? ,
qta_op7 = ? , qta_op8 = ? , flg_annullo = ? , modulo = ? , destinazione =
? where rowid = ? ;
Host variables :
address type flags value
-----------------------------------------
0xc0000000cf6b0040 CHAR 0x002
0xc0000000cf6b00d0 CHAR 0x002
0xc0000000cf6b0160 CHAR 0x002
0xc0000000cf6b01f0 CHAR 0x002
0xc0000000cf6b0280 CHAR 0x002
0xc0000000cf6b0310 CHAR 0x002
0xc0000000cf6b03a0 CHAR 0x002
0xc0000000cf6b0430 CHAR 0x002
0xc0000000cf6b04c0 CHAR 0x002
0xc0000000cf6b0550 CHAR 0x002
0xc0000000cf6b05e0 CHAR 0x002
0xc0000000cf6b0670 CHAR 0x002
0xc0000000cf6b0700 CHAR 0x002
0xc0000000cf6b0790 CHAR 0x002
0xc0000000cf6b0820 CHAR 0x002
0xc0000000cf6b08b0 CHAR 0x002
0xc0000000cf6b0940 CHAR 0x002
0xc0000000cf6b09d0 CHAR 0x002
0xc0000000cf6b0a60 CHAR 0x002
0xc0000000cf6b0af0 CHAR 0x002
0xc0000000cf6b0b80 CHAR 0x002
0xc0000000cf6b0c10 CHAR 0x002
0xc0000000cf6b0ca0 CHAR 0x002
0xc0000000cf6b0d30 CHAR 0x002
0xc0000000cf6b0dc0 CHAR 0x002
0xc0000000cf6b0e50 CHAR 0x002
0xc0000000cf6b0ee0 CHAR 0x002
0xc0000000cf6b0f70 CHAR 0x002
0xc0000000cf6b1000 CHAR 0x002
0xc0000000cf6b1090 CHAR 0x002
0xc0000000cf6b1120 CHAR 0x002
0xc0000000cf6b11b0 CHAR 0x002
0xc0000000cf6b1240 CHAR 0x002
0xc0000000cf6b12d0 CHAR 0x002
0xc0000000cf6b1360 CHAR 0x002
0xc0000000cf6b13f0 CHAR 0x002
0xc0000000cf6b1480 CHAR 0x002
0xc0000000cf6b1510 CHAR 0x002
0xc0000000cf6b15a0 CHAR 0x002
0xc0000000cf6b1630 CHAR 0x002
0xc0000000cf6b16c0 CHAR 0x002
0xc0000000cf6b1750 CHAR 0x002
0xc0000000cf6b17e0 CHAR 0x002
0xc0000000cf6b1870 CHAR 0x002
Last parsed SQL statement :
update MMFG:MN_ORDINI_ARCHIVIO set id_mn_ord_archivio = ? , trasmissione =
? , anno = ? , stagione = ? , collezione = ? , tipo_ordine = ? ,
nazione_mn = ? , canale = ? , modello_mn = ? , variante_mn = ? , nome = ?
, listino_acquisto = ? , prezzo_acquisto_eu = ? , listino_vendita = ? ,
prezzo_vendita_eu = ? , data_saldo = ? , stato_chiusura = ? , dta_chiusura
= ? , stato = ? , qta_or1 = ? , qta_or2 = ? , qta_or3 = ? , qta_or4 = ? ,
qta_or5 = ? , qta_or6 = ? , qta_or7 = ? , qta_or8 = ? , dta_ins = ? ,
ute_ins = ? , dta_upd = ? , ute_upd = ? , raggr_saldo = ? , qta_op1 = ? ,
qta_op2 = ? , qta_op3 = ? , qta_op4 = ? , qta_op5 = ? , qta_op6 = ? ,
qta_op7 = ? , qta_op8 = ? , flg_annullo = ? , modulo = ? , destinazione =
? where rowid = ? ;
-----------------------------------------------------------------------------
The architecture is composed by 2 nodes only: one (primary) replicate some tables on a secondary-server // the replication is mono-directional. Can the message below be related to the panic?
03:04:01 Grid Command Failed
GRID
SQL Statement Error SQLERR:-208 ISAM:0
select * from grid_cdr_node_tab where gn_need_sync = 'y'
03:04:01 ER Monitor received an error from the Internal Replicate Sync subcomponent.
Disabling Monitoring of this subcomponent.
jrenaut
Dec 18, 2012, 5:31:48 PM12/18/12
to
On Tuesday, December 18, 2012 3:43:34 AM UTC-6, checc...@libero.it wrote:
<stuff cut>
03:04:01 Warning: The IBM IDS Innovator-C Edition license restriction limits
03:04:01 the total shared memory size for this server to 2097152 KB.
03:04:01 Size of resident + virtual segments 1055428 KB + 1074492 KB > 2097152 KB.
03:04:01 Size of resident + virtual segments 1055428KB + 1074492KB > 2097152KB
03:04:01 total allowed by configuration parameter SHMTOTAL
03:04:01 out of virtual shared memory
03:04:01 Warning: The IBM IDS Innovator-C Edition license restriction limits
03:04:01 the total shared memory size for this server to 2097152 KB.
03:04:01 Size of resident + virtual segments 1055428 KB + 1074492 KB > 2097152 KB.
03:04:01 Warning: The IBM IDS Innovator-C Edition license restriction limits
03:04:01 the total shared memory size for this server to 2097152 KB.
03:04:01 Size of resident + virtual segments 1055428 KB + 1074492 KB > 2097152 KB.
03:04:01 Warning: The IBM IDS Innovator-C Edition license restriction limits
03:04:01 the total shared memory size for this server to 2097152 KB.
03:04:01 Size of resident + virtual segments 1055428 KB + 1074492 KB > 2097152 KB.
So it looks like you are out of or nearly out of memory available to the server and are using most/all of it. So the asserting thread (which also showed a -208 error in the onstat -g ses output which is a memory allocation failed error) likely tried to allocate memory for a structure, got a 208 error, but then the code path didn't correctly check to see if the structure was allocated or not, and then drove on and later tried to reference it and crashed. So it's probably a defect, but to run into it you need to have little to no free memory available to the running threads on the instance.
Jacques Renaut
IBM Informix Advanced Support
APD Team
<stuff cut>
>
> The architecture is composed by 2 nodes only: one (primary) replicate some tables on a secondary-server // the replication is mono-directional. Can the message below be related to the panic?
>
>
>
> 03:04:01 Grid Command Failed
>
> GRID
>
> SQL Statement Error SQLERR:-208 ISAM:0
>
> select * from grid_cdr_node_tab where gn_need_sync = 'y'
>
> 03:04:01 ER Monitor received an error from the Internal Replicate Sync subcomponent.
>
> Disabling Monitoring of this subcomponent.
>
>
>
> Still thanks
Actually, I would more likely guess the problem had to do with these messages:
> The architecture is composed by 2 nodes only: one (primary) replicate some tables on a secondary-server // the replication is mono-directional. Can the message below be related to the panic?
>
>
>
> 03:04:01 Grid Command Failed
>
> GRID
>
> SQL Statement Error SQLERR:-208 ISAM:0
>
> select * from grid_cdr_node_tab where gn_need_sync = 'y'
>
> 03:04:01 ER Monitor received an error from the Internal Replicate Sync subcomponent.
>
> Disabling Monitoring of this subcomponent.
>
>
>
> Still thanks
03:04:01 Warning: The IBM IDS Innovator-C Edition license restriction limits
03:04:01 the total shared memory size for this server to 2097152 KB.
03:04:01 Size of resident + virtual segments 1055428 KB + 1074492 KB > 2097152 KB.
03:04:01 Size of resident + virtual segments 1055428KB + 1074492KB > 2097152KB
03:04:01 total allowed by configuration parameter SHMTOTAL
03:04:01 out of virtual shared memory
03:04:01 Warning: The IBM IDS Innovator-C Edition license restriction limits
03:04:01 the total shared memory size for this server to 2097152 KB.
03:04:01 Size of resident + virtual segments 1055428 KB + 1074492 KB > 2097152 KB.
03:04:01 Warning: The IBM IDS Innovator-C Edition license restriction limits
03:04:01 the total shared memory size for this server to 2097152 KB.
03:04:01 Size of resident + virtual segments 1055428 KB + 1074492 KB > 2097152 KB.
03:04:01 Warning: The IBM IDS Innovator-C Edition license restriction limits
03:04:01 the total shared memory size for this server to 2097152 KB.
03:04:01 Size of resident + virtual segments 1055428 KB + 1074492 KB > 2097152 KB.
Jacques Renaut
IBM Informix Advanced Support
APD Team
Art Kagel
Dec 18, 2012, 6:21:32 PM12/18/12
to jrenaut, inform...@iiug.org
So, what Jacques is saying is to resolve this issue finally, you have to upgrade to an edition that will permit you to support more complex queries and/or more users. That will mean paying for licenses for Growth Edition or Ultimate Edition.
Art
Art
Art S. Kagel
Advanced DataTools (www.advancedatatools.com)
Blog: http://informix-myview.blogspot.com/
Disclaimer: Please keep in mind that my own opinions are my own opinions and do not reflect on my employer, Advanced DataTools, the IIUG, nor any other organization with which I am associated either explicitly, implicitly, or by inference. Neither do those opinions reflect those of other individuals affiliated with any entity with which I am affiliated nor those of the entities themselves.
Advanced DataTools (www.advancedatatools.com)
Blog: http://informix-myview.blogspot.com/
Disclaimer: Please keep in mind that my own opinions are my own opinions and do not reflect on my employer, Advanced DataTools, the IIUG, nor any other organization with which I am associated either explicitly, implicitly, or by inference. Neither do those opinions reflect those of other individuals affiliated with any entity with which I am affiliated nor those of the entities themselves.
_______________________________________________
Informix-list mailing list
Inform...@iiug.org
http://www.iiug.org/mailman/listinfo/informix-list
checc...@libero.it
Dec 19, 2012, 5:19:31 AM12/19/12
to jrenaut, inform...@iiug.org
> So, what Jacques is saying is to resolve this issue finally, you have to upgrade to an edition that will permit you to support more complex queries and/or more users. That will mean paying for licenses for Growth Edition or Ultimate Edition.
>
>
>
> Art
>
>
> Art S. Kagel
> Advanced DataTools (www.advancedatatools.com)
> Blog: http://informix-myview.blogspot.com/
>
Sure this is a way; is there another one, trying to arrange the $ONCONFIG parameters (see above) quite differently?
>
>
>
> Art
>
>
> Art S. Kagel
> Advanced DataTools (www.advancedatatools.com)
> Blog: http://informix-myview.blogspot.com/
>
Art Kagel
Dec 19, 2012, 7:13:39 AM12/19/12
to checc...@libero.it, inform...@iiug.org, jrenaut
Take a look at all parameters that allocate memory in the virtual and resident memory segments. That includes all of the SHM* parameters the BUFFERPOOL parameters, LOCKS (128 bytes per lock in a 64bit server), DS_NONPDQ_QUERY_MEM, DS_TOTAL_MEMORY, others. Also the environment variable DBUPSPACE which affects user session memory allocation. Try to minimize the memory taken by various features. Try to eliminate sorting. SImplify complex queries where possible.
Art
Art
Disclaimer: Please keep in mind that my own opinions are my own opinions and do not reflect on my employer, Advanced DataTools, the IIUG, nor any other organization with which I am associated either explicitly, implicitly, or by inference. Neither do those opinions reflect those of other individuals affiliated with any entity with which I am affiliated nor those of the entities themselves.
Ian Goddard
Dec 23, 2012, 8:42:16 AM12/23/12
to
jrenaut wrote:
> On Tuesday, December 18, 2012 3:43:34 AM UTC-6, checc...@libero.it wrote:
> <stuff cut>
>>
>> The architecture is composed by 2 nodes only: one (primary) replicate some tables on a secondary-server // the replication is mono-directional. Can the message below be related to the panic?
>>
>>
>>
>> 03:04:01 Grid Command Failed
>>
>> GRID
>>
>> SQL Statement Error SQLERR:-208 ISAM:0
>>
>> select * from grid_cdr_node_tab where gn_need_sync = 'y'
>>
>> 03:04:01 ER Monitor received an error from the Internal Replicate Sync subcomponent.
>>
>> Disabling Monitoring of this subcomponent.
>>
>>
>>
>> Still thanks
>
> Actually, I would more likely guess the problem had to do with these messages:
>
> 03:04:01 Warning: The IBM IDS Innovator-C Edition license restriction limits
> 03:04:01 the total shared memory size for this server to 2097152 KB.
> 03:04:01 Size of resident + virtual segments 1055428 KB + 1074492 KB > 2097152 KB.
> 03:04:01 Size of resident + virtual segments 1055428KB + 1074492KB > 2097152KB
> 03:04:01 total allowed by configuration parameter SHMTOTAL
> 03:04:01 out of virtual shared memory
It might be as well to check the application code.
> On Tuesday, December 18, 2012 3:43:34 AM UTC-6, checc...@libero.it wrote:
> <stuff cut>
>>
>> The architecture is composed by 2 nodes only: one (primary) replicate some tables on a secondary-server // the replication is mono-directional. Can the message below be related to the panic?
>>
>>
>>
>> 03:04:01 Grid Command Failed
>>
>> GRID
>>
>> SQL Statement Error SQLERR:-208 ISAM:0
>>
>> select * from grid_cdr_node_tab where gn_need_sync = 'y'
>>
>> 03:04:01 ER Monitor received an error from the Internal Replicate Sync subcomponent.
>>
>> Disabling Monitoring of this subcomponent.
>>
>>
>>
>> Still thanks
>
> Actually, I would more likely guess the problem had to do with these messages:
>
> 03:04:01 Warning: The IBM IDS Innovator-C Edition license restriction limits
> 03:04:01 the total shared memory size for this server to 2097152 KB.
> 03:04:01 Size of resident + virtual segments 1055428 KB + 1074492 KB > 2097152 KB.
> 03:04:01 Size of resident + virtual segments 1055428KB + 1074492KB > 2097152KB
> 03:04:01 total allowed by configuration parameter SHMTOTAL
> 03:04:01 out of virtual shared memory
A long time back - Informix 5ish - I had a weekly invoice run crash two
weeks in a row (Friday lunchtime, just to add to the aggravation).
After the second crash I found it was the back-end exceeding its OS
imposed allocation. Although it was a 3rd party package we had been
provided with the source so I spent the afternoon digging through the
it. I eventually found a statement being prepared & executed but with
no release of the resources afterwards and buried deep in nested loops.
It didn't hit until the level of business being invoiced became
sufficient to exhaust the memory limit. As the OP's system has been
running for some time it's possible that some such growth of job size is
happening here.
--
Ian
The Hotmail address is my spam-bin. Real mail address is iang
at austonley org uk
checc...@libero.it
Dec 28, 2012, 12:14:24 PM12/28/12
to
Thanks all for the answers!
Happy new year!
Francesco
Happy new year!
Francesco
0 new messages