stapxx火焰图错误

[王立]

hi，agentzh：

我们团队架构从apache+python转向openResty之后，接口的响应时间比之前快了很多。但是，最近在上线一个新的业务之后（我们的业务是混合部署的），调用上游的redis等资源时，发现了大量的504错误。与此同时机器的负载和cpu占用率等都不高，redis端也都没发现问题。根据论坛中春哥和各位的回复，认为原因在于“nginx 阻塞在了某些操作上使得它的事件循环不能及时响应网络连接上的事件，而导致超时错误”。为了找到新业务的哪些操作占住了cpu，我们研究了下春哥提供的stap++工具，但是在使用lj-lua-stacks.sxx时，出现了如下的错误：

semantic error: unable to find member 'cur_L' for struct global_State (alternatives: strhash strmask strnum allocf allocd gc tmpbuf nilnode strempty stremptyz hookmask dispatchmode vmevmask mainthref registrytv tmptv tmptv2 uvhead hookcount hookcstart hookf wrapf panic vmstate bc_cfunc_int bc_cfunc_ext jit_L jit_base ctype_state gcroot): operator '->' at stapxx-3PGBbS8f/luajit.stp:631:93
        source:     gco = @cast(g, "global_State", "/usr/local/openresty/luajit/lib/libluajit-5.1.so.2.0.2")->cur_L->gcptr32
                                                                                                            ^

Pass 2: analysis failed.  [man error::pass2]

请问是什么原因导致的？谢谢。

[zheng.x...@zte.com.cn]

这个我替春哥答复，请使用Luajit2.1版本

王立 <wangl...@gmail.com> 发件人:  open...@googlegroups.com 2015-03-10 16:41 请答复 给 open...@googlegroups.com

收件人 open...@googlegroups.com 抄送 主题 [openresty] stapxx火焰图错误

--
--
邮件来自列表“openresty”,专用于技术讨论!
订阅: 请发空白邮件到 openresty...@googlegroups.com
发言: 请发邮件到 open...@googlegroups.com
退订: 请发邮件至 openresty+...@googlegroups.com
归档: http://groups.google.com/group/openresty
官网: http://openresty.org/
仓库: https://github.com/agentzh/ngx_openresty
教程: http://openresty.org/download/agentzh-nginx-tutorials-zhcn.html

--------------------------------------------------------
ZTE Information Security Notice: The information contained in this mail (and any attachment transmitted herewith) is privileged and confidential and is intended for the exclusive use of the addressee(s).  If you are not an intended recipient, any disclosure, reproduction, distribution or other dissemination or use of the information contained is strictly prohibited.  If you have received this mail in error, please delete it and notify us immediately.

--------------------------------------------------------
ZTE Information Security Notice: The information contained in this mail (and any attachment transmitted herewith) is privileged and confidential and is intended for the exclusive use of the addressee(s).  If you are not an intended recipient, any disclosure, reproduction, distribution or other dissemination or use of the information contained is strictly prohibited.  If you have received this mail in error, please delete it and notify us immediately.

[Yichun Zhang (agentzh)]

Hello!

2015-03-10 1:41 GMT-07:00 王立:

> 我们团队架构从apache+python转向openResty之后，接口的响应时间比之前快了很多。
> 但是，最近在上线一个新的业务之后（我们的业务是混合部署的），调用上游的redis等
> 资源时，发现了大量的504错误。与此同时机器的负载和cpu占用率等都不高，redis端
> 也都没发现问题。

根据你的描述，很可能是你们新上线的业务代码中使用了阻塞的系统调用（比如阻塞 IO）。这种问题很容易使用 off-CPU 火焰图进行定位：

https://github.com/openresty/nginx-systemtap-toolkit#sample-bt-off-cpu

同时下面这个工具的分析结果也具有参考意义：

https://github.com/openresty/stapxx#epoll-loop-blocking-distr

值得一提的是，为避免各种已知的不必要的麻烦，推荐使用最新版本的 OpenResty 软件包：

http://openresty.org/#Download

Regards,
-agentzh