openresty-gdb-utils 探测内存溢出问题失败

ushar...@gmail.com

unread,

Jan 18, 2018, 8:37:43 AM1/18/18

to openresty

我发现程序报 not enough memory 错误信息。然后通过openresty-gdb-utils来探测gc情况，并未发现异常。下边我罗列下我的操作步骤

环境：centos，openresty/1.11.2.2

1、从新编译openresty，使用 ./configure --with-cc-opt="-g -O0 -D DDEBUG=0"

2、安装gdb

3、设置PYTHONPATH环境变量

4、使用wrk对api进行压测

5、gdb -p 4915 进入

6、执行gbb分析

GNU gdb (GDB) Amazon Linux (7.6.1-64.33.amzn1)
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-amazon-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Attaching to process 22113
Reading symbols from /usr/local/openresty/nginx/sbin/nginx...done.
Reading symbols from /lib64/libdl.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/libdl.so.2
Reading symbols from /lib64/libpthread.so.0...(no debugging symbols found)...done.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Loaded symbols for /lib64/libpthread.so.0
Reading symbols from /lib64/libcrypt.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/libcrypt.so.1
Reading symbols from /usr/local/openresty/luajit/lib/libluajit-5.1.so.2...done.
Loaded symbols for /usr/local/openresty/luajit/lib/libluajit-5.1.so.2
Reading symbols from /lib64/libm.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib64/libm.so.6
Reading symbols from /lib64/libpcre.so.0...(no debugging symbols found)...done.
Loaded symbols for /lib64/libpcre.so.0
Reading symbols from /usr/lib64/libssl.so.10...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libssl.so.10
Reading symbols from /lib64/libcrypto.so.10...(no debugging symbols found)...done.
Loaded symbols for /lib64/libcrypto.so.10
Reading symbols from /lib64/libz.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/libz.so.1
Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib64/libc.so.6
Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
Reading symbols from /lib64/libfreebl3.so...(no debugging symbols found)...done.
Loaded symbols for /lib64/libfreebl3.so
Reading symbols from /lib64/libgcc_s.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/libgcc_s.so.1
Reading symbols from /usr/lib64/libgssapi_krb5.so.2...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libgssapi_krb5.so.2
Reading symbols from /usr/lib64/libkrb5.so.3...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libkrb5.so.3
Reading symbols from /usr/lib64/libcom_err.so.2...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libcom_err.so.2
Reading symbols from /usr/lib64/libk5crypto.so.3...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libk5crypto.so.3
Reading symbols from /usr/lib64/libkrb5support.so.0...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libkrb5support.so.0
Reading symbols from /lib64/libkeyutils.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/libkeyutils.so.1
Reading symbols from /lib64/libresolv.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/libresolv.so.2
Reading symbols from /usr/lib64/libselinux.so.1...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libselinux.so.1
Reading symbols from /lib64/libnss_files.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/libnss_files.so.2
Reading symbols from /usr/local/openresty/lualib/cjson.so...done.
Loaded symbols for /usr/local/openresty/lualib/cjson.so
Reading symbols from /usr/local/lib/lua/5.1/socket/core.so...(no debugging symbols found)...done.
Loaded symbols for /usr/local/lib/lua/5.1/socket/core.so
Reading symbols from /usr/local/lib/lua/5.1/lpeg.so...(no debugging symbols found)...done.
Loaded symbols for /usr/local/lib/lua/5.1/lpeg.so
0x00007f4431d058c3 in __epoll_wait_nocancel () from /lib64/libc.so.6
Missing separate debuginfos, use: debuginfo-install openresty-1.11.2.5-2.amzn1.x86_64(gdb) source luajit21.py(gdb) lgcstat
2425 str        objects: max=129375434, avg = 53380, min=18, sum=129447211
 804 upval      objects: max=24, avg = 24, min=24, sum=19296
   1 thread     objects: max=600, avg = 600, min=600, sum=600
 653 proto      objects: max=2982, avg = 274, min=74, sum=179558
1383 func       objects: max=144, avg = 29, min=20, sum=41176
   5 trace      objects: max=2296, avg = 1000, min=160, sum=5004
  33 cdata      objects: max=72, avg = 17, min=12, sum=564
 522 tab        objects: max=6176, avg = 220, min=32, sum=114872
  64 udata      objects: max=1376, avg = 242, min=32, sum=15544

 sizeof strhash 16384
 sizeof g->tmpbuf 16777216
 sizeof ctype_state 8664
 sizeof jit_state 2464

total sz 146633841
g->strnum 2425, g->gc.total 146633841
elapsed: 0.320000 sec
(gdb) lbt
(gdb)

在这样的gc状态的时候，服务已经报not enough memory错误信息了。

请问如何解决呢？

SanJiang Liu

unread,

Jan 19, 2018, 1:37:59 AM1/19/18

to openresty

shared dict不够用也会报 not enough memory

在 2018年1月18日星期四 UTC+8下午9:37:43，ushar...@gmail.com写道：

ushar...@gmail.com

unread,

Jan 19, 2018, 4:13:48 AM1/19/18

to openresty

shared dict 报的错误是："no memory".

在 2018年1月19日星期五 UTC+8下午2:37:59，SanJiang Liu写道：

Reply all

Reply to author

Forward