Segmentation fault when importing trac.web.main
Dan Connolly
I'm trying to run trac with mod_wsgi. I can run trac as a CGI script, but when I try to run it as a WSGI app, I get a SEGV:
[Mon Jun 21 11:42:36 2010] [error] [client 10.17.8.76] Premature end of script headers: trac.wsgi
[Mon Jun 21 11:42:36 2010] [notice] child pid 1854 exit signal Segmentation fault (11)
Using the very helpful http://code.google.com/p/modwsgi/wiki/DebuggingTechniques , I narrowed it down to:
from trac.web.main import dispatch_request
that line, added to a hello-world.cgi, produces the end-of-script/SEGV.
I'm puzzled because I got hgwebdir running under mod_wsgi just fine a la http://mercurial.selenic.com/wiki/modwsgi
I suspected an .egg-cache permissions issue, but running as CGI relies on the same permissions, so I don't think that's the cause.
Details of my setup:
1. Which version of mod_wsgi you are using and if using a packaged distribution, who provided the distribution.
I installed mod_wsgi from source:
http://modwsgi.googlecode.com/files/mod_wsgi-3.2.tar.gz
2. Which version of Python you are using and if using a packaged distribution, who provided the distribution.
python 2.6 from SuSE
3. Which version of Apache you are using and if using a packaged distribution, who provided the distribution.
apache 2.2 from source
4. What operating system you are using.
SUSE Linux Enterprise Server 11 (x86_64)
5. Details on any third party packages being used and what versions of those packages.
trac from source: Trac-0.12.tar.gz
6. The mod_wsgi configuration you are using from Apache configuration files.
mod-wsgi.con and trac.conf are attached.
(for my reference: e3ebf751fd82 2010-06-21 10:57 -0600)
wsgi-env.txt is the output of the script from http://code.google.com/p/modwsgi/wiki/DebuggingTechniques plus sys.path
I'm not certain which MPM apache has been compiled for; how do I check?
7. Relevant error messages from the Apache error logs.
This is a complete log from starting apache to the problem to when apache is quiescent again:
[Mon Jun 21 11:42:15 2010] [notice] Apache/2.2.15 (Unix) PHP/5.2.13 mod_jk/1.2.28 mod_wsgi/3.2 Python/2.6 configured -- resuming normal operations
[Mon Jun 21 11:42:15 2010] [info] Server built: Apr 7 2010 13:23:50
[Mon Jun 21 11:42:15 2010] [info] mod_wsgi (pid=1880): Initializing Python.
[Mon Jun 21 11:42:15 2010] [info] mod_wsgi (pid=1880): Attach interpreter ''.
[Mon Jun 21 11:42:35 2010] [info] [client 10.17.8.76] mod_wsgi (pid=1854, process='bioinformatics.kumc.edu', application=''): Loading WSGI script '/usr/local/apache2/trac-share/cgi-bin/trac.wsgi'.
[Mon Jun 21 11:42:36 2010] [error] [client 10.17.8.76] Premature end of script headers: trac.wsgi
[Mon Jun 21 11:42:36 2010] [notice] child pid 1854 exit signal Segmentation fault (11)
[Mon Jun 21 11:42:36 2010] [info] mod_wsgi (pid=1854): Process 'bioinformatics.kumc.edu' has died, restarting.
[Mon Jun 21 11:42:36 2010] [info] mod_wsgi (pid=1885): Starting process 'bioinformatics.kumc.edu' with uid=2, gid=2 and threads=1.
[Mon Jun 21 11:42:36 2010] [info] mod_wsgi (pid=1885): Initializing Python.
[Mon Jun 21 11:42:36 2010] [info] mod_wsgi (pid=1885): Attach interpreter ''.
Graham Dumpleton
Graham
> --
> You received this message because you are subscribed to the Google Groups
> "modwsgi" group.
> To post to this group, send email to mod...@googlegroups.com.
> To unsubscribe from this group, send email to
> modwsgi+u...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/modwsgi?hl=en.
>
Dan Connolly
> Can you temporarily disable mod_php and try again?
Graham Dumpleton
Nothing obviously wrong in configuration. Can only suggest you have a go at:
http://code.google.com/p/modwsgi/wiki/DebuggingTechniques#Debugging_Crashes_With_GDB
to see if gdb can tell you where it is crashing.
Graham
Dan Connolly
I seem to be hitting the pyexpat issue...
Anybody want to remind me of the fix or work-around?
Perhaps I can find it myself...
(gdb) run -X
Starting program: /usr/local/apache2/bin/httpd -X
[Thread debugging using libthread_db enabled]
Program received signal SIGSEGV, Segmentation fault.
0x00007ffff69d1590 in strlen () from /lib64/libc.so.6
(gdb) bt
#0 0x00007ffff69d1590 in strlen () from /lib64/libc.so.6
#1 0x00007ffff364ad80 in PyString_FromString ()
from /usr/lib64/libpython2.6.so.1.0
#2 0x00007ffff36a82d2 in PyModule_AddStringConstant ()
from /usr/lib64/libpython2.6.so.1.0
#3 0x00007ffff076f067 in initpyexpat ()
from /usr/lib64/python2.6/site-packages/_xmlplus/parsers/pyexpat.so
#4 0x00007ffff36a540e in _PyImport_LoadDynamicModule ()
from /usr/lib64/libpython2.6.so.1.0
#5 0x00007ffff36a43da in ?? () from /usr/lib64/libpython2.6.so.1.0
#6 0x00007ffff36a4668 in ?? () from /usr/lib64/libpython2.6.so.1.0
#7 0x00007ffff36a490f in ?? () from /usr/lib64/libpython2.6.so.1.0
#8 0x00007ffff36a52bc in PyImport_ImportModuleLevel ()
from /usr/lib64/libpython2.6.so.1.0
#9 0x00007ffff368a67d in ?? () from /usr/lib64/libpython2.6.so.1.0
#10 0x00007ffff3607ac9 in PyObject_Call ()
from /usr/lib64/libpython2.6.so.1.0
#11 0x00007ffff368b138 in PyEval_CallObjectWithKeywords ()
from /usr/lib64/libpython2.6.so.1.0
#12 0x00007ffff368d4d9 in PyEval_EvalFrameEx ()
from /usr/lib64/libpython2.6.so.1.0
---Type <return> to continue, or q <return> to quit---
#13 0x00007ffff36926c0 in PyEval_EvalCodeEx ()
from /usr/lib64/libpython2.6.so.1.0
#14 0x00007ffff368a822 in PyEval_EvalCode ()
from /usr/lib64/libpython2.6.so.1.0
#15 0x00007ffff36a147c in PyImport_ExecCodeModuleEx ()
from /usr/lib64/libpython2.6.so.1.0
#16 0x00007ffff36a16af in ?? () from /usr/lib64/libpython2.6.so.1.0
#17 0x00007ffff36a43da in ?? () from /usr/lib64/libpython2.6.so.1.0
#18 0x00007ffff36a215a in ?? () from /usr/lib64/libpython2.6.so.1.0
#19 0x00007ffff36a4c0d in ?? () from /usr/lib64/libpython2.6.so.1.0
#20 0x00007ffff36a52bc in PyImport_ImportModuleLevel ()
from /usr/lib64/libpython2.6.so.1.0
#21 0x00007ffff368a67d in ?? () from /usr/lib64/libpython2.6.so.1.0
#22 0x00007ffff3607ac9 in PyObject_Call ()
from /usr/lib64/libpython2.6.so.1.0
#23 0x00007ffff368b138 in PyEval_CallObjectWithKeywords ()
from /usr/lib64/libpython2.6.so.1.0
#24 0x00007ffff368d4d9 in PyEval_EvalFrameEx ()
from /usr/lib64/libpython2.6.so.1.0
#25 0x00007ffff36926c0 in PyEval_EvalCodeEx ()
from /usr/lib64/libpython2.6.so.1.0
---Type <return> to continue, or q <return> to quit---
#26 0x00007ffff368a822 in PyEval_EvalCode ()
from /usr/lib64/libpython2.6.so.1.0
#27 0x00007ffff36a147c in PyImport_ExecCodeModuleEx ()
from /usr/lib64/libpython2.6.so.1.0
#28 0x00007ffff36cc947 in ?? () from /usr/lib64/libpython2.6.so.1.0
#29 0x00007ffff3607ac9 in PyObject_Call ()
from /usr/lib64/libpython2.6.so.1.0
#30 0x00007ffff3604501 in ?? () from /usr/lib64/libpython2.6.so.1.0
#31 0x00007ffff360462e in PyObject_CallMethod ()
from /usr/lib64/libpython2.6.so.1.0
#32 0x00007ffff36a43da in ?? () from /usr/lib64/libpython2.6.so.1.0
#33 0x00007ffff36a4668 in ?? () from /usr/lib64/libpython2.6.so.1.0
#34 0x00007ffff36a4b86 in ?? () from /usr/lib64/libpython2.6.so.1.0
#35 0x00007ffff36a52bc in PyImport_ImportModuleLevel ()
from /usr/lib64/libpython2.6.so.1.0
#36 0x00007ffff368a67d in ?? () from /usr/lib64/libpython2.6.so.1.0
#37 0x00007ffff3607ac9 in PyObject_Call ()
from /usr/lib64/libpython2.6.so.1.0
#38 0x00007ffff368b138 in PyEval_CallObjectWithKeywords ()
from /usr/lib64/libpython2.6.so.1.0
#39 0x00007ffff368d4d9 in PyEval_EvalFrameEx ()
from /usr/lib64/libpython2.6.so.1.0
---Type <return> to continue, or q <return> to quit---
#40 0x00007ffff36926c0 in PyEval_EvalCodeEx ()
from /usr/lib64/libpython2.6.so.1.0
#41 0x00007ffff368a822 in PyEval_EvalCode ()
from /usr/lib64/libpython2.6.so.1.0
#42 0x00007ffff36a147c in PyImport_ExecCodeModuleEx ()
from /usr/lib64/libpython2.6.so.1.0
#43 0x00007ffff36cc947 in ?? () from /usr/lib64/libpython2.6.so.1.0
#44 0x00007ffff3607ac9 in PyObject_Call ()
from /usr/lib64/libpython2.6.so.1.0
#45 0x00007ffff3604501 in ?? () from /usr/lib64/libpython2.6.so.1.0
#46 0x00007ffff360462e in PyObject_CallMethod ()
from /usr/lib64/libpython2.6.so.1.0
#47 0x00007ffff36a43da in ?? () from /usr/lib64/libpython2.6.so.1.0
#48 0x00007ffff36a4719 in ?? () from /usr/lib64/libpython2.6.so.1.0
#49 0x00007ffff36a490f in ?? () from /usr/lib64/libpython2.6.so.1.0
#50 0x00007ffff36a52bc in PyImport_ImportModuleLevel ()
from /usr/lib64/libpython2.6.so.1.0
#51 0x00007ffff368a67d in ?? () from /usr/lib64/libpython2.6.so.1.0
#52 0x00007ffff3607ac9 in PyObject_Call ()
from /usr/lib64/libpython2.6.so.1.0
#53 0x00007ffff368b138 in PyEval_CallObjectWithKeywords ()
from /usr/lib64/libpython2.6.so.1.0
---Type <return> to continue, or q <return> to quit---
#54 0x00007ffff368d4d9 in PyEval_EvalFrameEx ()
from /usr/lib64/libpython2.6.so.1.0
#55 0x00007ffff36926c0 in PyEval_EvalCodeEx ()
from /usr/lib64/libpython2.6.so.1.0
#56 0x00007ffff368a822 in PyEval_EvalCode ()
from /usr/lib64/libpython2.6.so.1.0
#57 0x00007ffff36a147c in PyImport_ExecCodeModuleEx ()
from /usr/lib64/libpython2.6.so.1.0
#58 0x00007ffff36cc947 in ?? () from /usr/lib64/libpython2.6.so.1.0
#59 0x00007ffff3607ac9 in PyObject_Call ()
from /usr/lib64/libpython2.6.so.1.0
#60 0x00007ffff3604501 in ?? () from /usr/lib64/libpython2.6.so.1.0
#61 0x00007ffff360462e in PyObject_CallMethod ()
from /usr/lib64/libpython2.6.so.1.0
#62 0x00007ffff36a43da in ?? () from /usr/lib64/libpython2.6.so.1.0
#63 0x00007ffff36a4668 in ?? () from /usr/lib64/libpython2.6.so.1.0
#64 0x00007ffff36a4b86 in ?? () from /usr/lib64/libpython2.6.so.1.0
#65 0x00007ffff36a52bc in PyImport_ImportModuleLevel ()
from /usr/lib64/libpython2.6.so.1.0
#66 0x00007ffff368a67d in ?? () from /usr/lib64/libpython2.6.so.1.0
#67 0x00007ffff3607ac9 in PyObject_Call ()
from /usr/lib64/libpython2.6.so.1.0
---Type <return> to continue, or q <return> to quit---
#68 0x00007ffff368b138 in PyEval_CallObjectWithKeywords ()
from /usr/lib64/libpython2.6.so.1.0
#69 0x00007ffff368d4d9 in PyEval_EvalFrameEx ()
from /usr/lib64/libpython2.6.so.1.0
#70 0x00007ffff36926c0 in PyEval_EvalCodeEx ()
from /usr/lib64/libpython2.6.so.1.0
#71 0x00007ffff368a822 in PyEval_EvalCode ()
from /usr/lib64/libpython2.6.so.1.0
#72 0x00007ffff36a147c in PyImport_ExecCodeModuleEx ()
from /usr/lib64/libpython2.6.so.1.0
#73 0x00007ffff36cc947 in ?? () from /usr/lib64/libpython2.6.so.1.0
#74 0x00007ffff3607ac9 in PyObject_Call ()
from /usr/lib64/libpython2.6.so.1.0
#75 0x00007ffff3604501 in ?? () from /usr/lib64/libpython2.6.so.1.0
#76 0x00007ffff360462e in PyObject_CallMethod ()
from /usr/lib64/libpython2.6.so.1.0
#77 0x00007ffff36a43da in ?? () from /usr/lib64/libpython2.6.so.1.0
#78 0x00007ffff36a4668 in ?? () from /usr/lib64/libpython2.6.so.1.0
#79 0x00007ffff36a4b86 in ?? () from /usr/lib64/libpython2.6.so.1.0
#80 0x00007ffff36a52bc in PyImport_ImportModuleLevel ()
from /usr/lib64/libpython2.6.so.1.0
---Type <return> to continue, or q <return> to quit---
#81 0x00007ffff368a67d in ?? () from /usr/lib64/libpython2.6.so.1.0
#82 0x00007ffff3607ac9 in PyObject_Call ()
from /usr/lib64/libpython2.6.so.1.0
#83 0x00007ffff368b138 in PyEval_CallObjectWithKeywords ()
from /usr/lib64/libpython2.6.so.1.0
#84 0x00007ffff368d4d9 in PyEval_EvalFrameEx ()
from /usr/lib64/libpython2.6.so.1.0
#85 0x00007ffff36926c0 in PyEval_EvalCodeEx ()
from /usr/lib64/libpython2.6.so.1.0
#86 0x00007ffff368a822 in PyEval_EvalCode ()
from /usr/lib64/libpython2.6.so.1.0
#87 0x00007ffff36a147c in PyImport_ExecCodeModuleEx ()
from /usr/lib64/libpython2.6.so.1.0
#88 0x00007ffff36cc947 in ?? () from /usr/lib64/libpython2.6.so.1.0
#89 0x00007ffff3607ac9 in PyObject_Call ()
from /usr/lib64/libpython2.6.so.1.0
#90 0x00007ffff3604501 in ?? () from /usr/lib64/libpython2.6.so.1.0
#91 0x00007ffff360462e in PyObject_CallMethod ()
from /usr/lib64/libpython2.6.so.1.0
#92 0x00007ffff36a43da in ?? () from /usr/lib64/libpython2.6.so.1.0
#93 0x00007ffff36a4668 in ?? () from /usr/lib64/libpython2.6.so.1.0
#94 0x00007ffff36a4b86 in ?? () from /usr/lib64/libpython2.6.so.1.0
---Type <return> to continue, or q <return> to quit---
#95 0x00007ffff36a52bc in PyImport_ImportModuleLevel ()
from /usr/lib64/libpython2.6.so.1.0
#96 0x00007ffff368a67d in ?? () from /usr/lib64/libpython2.6.so.1.0
#97 0x00007ffff3607ac9 in PyObject_Call ()
from /usr/lib64/libpython2.6.so.1.0
#98 0x00007ffff368b138 in PyEval_CallObjectWithKeywords ()
from /usr/lib64/libpython2.6.so.1.0
#99 0x00007ffff368d4d9 in PyEval_EvalFrameEx ()
from /usr/lib64/libpython2.6.so.1.0
#100 0x00007ffff36926c0 in PyEval_EvalCodeEx ()
from /usr/lib64/libpython2.6.so.1.0
#101 0x00007ffff368a822 in PyEval_EvalCode ()
from /usr/lib64/libpython2.6.so.1.0
#102 0x00007ffff36a147c in PyImport_ExecCodeModuleEx ()
from /usr/lib64/libpython2.6.so.1.0
#103 0x00007ffff36cc947 in ?? () from /usr/lib64/libpython2.6.so.1.0
#104 0x00007ffff3607ac9 in PyObject_Call ()
from /usr/lib64/libpython2.6.so.1.0
#105 0x00007ffff3604501 in ?? () from /usr/lib64/libpython2.6.so.1.0
#106 0x00007ffff360462e in PyObject_CallMethod ()
from /usr/lib64/libpython2.6.so.1.0
---Type <return> to continue, or q <return> to quit---
#107 0x00007ffff36a43da in ?? () from /usr/lib64/libpython2.6.so.1.0
#108 0x00007ffff36a4668 in ?? () from /usr/lib64/libpython2.6.so.1.0
#109 0x00007ffff36a4b86 in ?? () from /usr/lib64/libpython2.6.so.1.0
#110 0x00007ffff36a52bc in PyImport_ImportModuleLevel ()
from /usr/lib64/libpython2.6.so.1.0
#111 0x00007ffff368a67d in ?? () from /usr/lib64/libpython2.6.so.1.0
#112 0x00007ffff3607ac9 in PyObject_Call ()
from /usr/lib64/libpython2.6.so.1.0
#113 0x00007ffff368b138 in PyEval_CallObjectWithKeywords ()
from /usr/lib64/libpython2.6.so.1.0
#114 0x00007ffff368d4d9 in PyEval_EvalFrameEx ()
from /usr/lib64/libpython2.6.so.1.0
#115 0x00007ffff36926c0 in PyEval_EvalCodeEx ()
from /usr/lib64/libpython2.6.so.1.0
#116 0x00007ffff368a822 in PyEval_EvalCode ()
from /usr/lib64/libpython2.6.so.1.0
#117 0x00007ffff36a147c in PyImport_ExecCodeModuleEx ()
from /usr/lib64/libpython2.6.so.1.0
#118 0x00007ffff36cc947 in ?? () from /usr/lib64/libpython2.6.so.1.0
#119 0x00007ffff3607ac9 in PyObject_Call ()
---Type <return> to continue, or q <return> to quit---
from /usr/lib64/libpython2.6.so.1.0
#120 0x00007ffff3604501 in ?? () from /usr/lib64/libpython2.6.so.1.0
#121 0x00007ffff360462e in PyObject_CallMethod ()
from /usr/lib64/libpython2.6.so.1.0
#122 0x00007ffff36a43da in ?? () from /usr/lib64/libpython2.6.so.1.0
#123 0x00007ffff36a4668 in ?? () from /usr/lib64/libpython2.6.so.1.0
#124 0x00007ffff36a4b86 in ?? () from /usr/lib64/libpython2.6.so.1.0
#125 0x00007ffff36a51e8 in PyImport_ImportModuleLevel ()
from /usr/lib64/libpython2.6.so.1.0
#126 0x00007ffff368a67d in ?? () from /usr/lib64/libpython2.6.so.1.0
#127 0x00007ffff3607ac9 in PyObject_Call ()
from /usr/lib64/libpython2.6.so.1.0
#128 0x00007ffff368b138 in PyEval_CallObjectWithKeywords ()
from /usr/lib64/libpython2.6.so.1.0
#129 0x00007ffff368d4d9 in PyEval_EvalFrameEx ()
from /usr/lib64/libpython2.6.so.1.0
#130 0x00007ffff36924c2 in PyEval_EvalCodeEx ()
from /usr/lib64/libpython2.6.so.1.0
#131 0x00007ffff362a098 in ?? () from /usr/lib64/libpython2.6.so.1.0
#132 0x00007ffff3607ac9 in PyObject_Call ()
---Type <return> to continue, or q <return> to quit---
from /usr/lib64/libpython2.6.so.1.0
#133 0x00007ffff368b138 in PyEval_CallObjectWithKeywords ()
from /usr/lib64/libpython2.6.so.1.0
#134 0x00007ffff396bcda in wsgi_execute_script (r=0xb56f20) at
mod_wsgi.c:3829
#135 0x000000000043c073 in ap_run_handler (r=0xb56f20) at config.c:158
#136 0x000000000043f58f in ap_invoke_handler (r=0xb56f20) at
config.c:372
#137 0x000000000045604e in ap_process_request (r=0xb56f20)
at http_request.c:282
#138 0x0000000000453038 in ap_process_http_connection (c=0x975b00)
at http_core.c:190
#139 0x00000000004435b3 in ap_run_process_connection (c=0x975b00)
at connection.c:43
#140 0x00000000004693b0 in child_main (child_num_arg=<value optimized
out>)
at prefork.c:662
#141 0x0000000000469638 in make_child (s=0x6a7150, slot=0) at
prefork.c:702
#142 0x0000000000469c50 in ap_mpm_run (_pconf=<value optimized out>,
plog=<value optimized out>, s=<value optimized out>) at
prefork.c:978
#143 0x0000000000428aa5 in main (argc=2, argv=0x7fffffffecc8) at
main.c:740
Dan Connolly
> I seem to be hitting the pyexpat issue...
>
> Anybody want to remind me of the fix or work-around?
> Perhaps I can find it myself...
OK, so I found
http://code.google.com/p/modwsgi/wiki/IssuesWithExpatLibrary
and I re-built apache with --disable-expat.
Now lsof shows no libexpat.so at all.
But I'm still getting a SEGV in roughly the same place:
dconnolly@bmidev1:/usr/local/apache2> ps aux | grep http | head -3
root 24665 0.0 0.1 82316 6376 ? Ss 11:16
0:00 /usr/local/apache2/bin/httpd -k start
daemon 24694 0.0 0.2 84348 8656 ? S 11:16
0:00 /usr/local/apache2/bin/httpd -k start
daemon 24725 0.0 0.1 84348 7828 ? S 11:16
0:00 /usr/local/apache2/bin/httpd -k start
dconnolly@bmidev1:/usr/local/apache2> sudo lsof -p 82316 | grep
expatdconnolly@bmidev1:/usr/local/apache2> sudo apachectl stop
dconnolly@bmidev1:/usr/local/apache2> sudo gdb bin/httpd
GNU gdb (GDB; SUSE Linux Enterprise 11) 6.8.50.20081120-cvs
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show
copying"
and "show warranty" for details.
This GDB was configured as "x86_64-suse-linux".
For bug reporting instructions, please see:
<http://bugs.opensuse.org/>...
(gdb) run -X
Starting program: /usr/local/apache2/bin/httpd -X
[Thread debugging using libthread_db enabled]
Program received signal SIGSEGV, Segmentation fault.
0x00007ffff61f0590 in strlen () from /lib64/libc.so.6
(gdb) where
#0 0x00007ffff61f0590 in strlen () from /lib64/libc.so.6
#1 0x00007ffff364ad80 in PyString_FromString ()
from /usr/lib64/libpython2.6.so.1.0
#2 0x00007ffff36a82d2 in PyModule_AddStringConstant ()
from /usr/lib64/libpython2.6.so.1.0
#3 0x00007ffff2298067 in initpyexpat ()
#28 0x00007ffff36a16af in ?? () from /usr/lib64/libpython2.6.so.1.0
#29 0x00007ffff36a43da in ?? () from /usr/lib64/libpython2.6.so.1.0
#30 0x00007ffff36a4668 in ?? () from /usr/lib64/libpython2.6.so.1.0
#31 0x00007ffff36a4b1a in ?? () from /usr/lib64/libpython2.6.so.1.0
#32 0x00007ffff36a51e8 in PyImport_ImportModuleLevel ()
from /usr/lib64/libpython2.6.so.1.0
#33 0x00007ffff368a67d in ?? () from /usr/lib64/libpython2.6.so.1.0
#34 0x00007ffff3607ac9 in PyObject_Call ()
from /usr/lib64/libpython2.6.so.1.0
#35 0x00007ffff368b138 in PyEval_CallObjectWithKeywords ()
from /usr/lib64/libpython2.6.so.1.0
#36 0x00007ffff368d4d9 in PyEval_EvalFrameEx ()
from /usr/lib64/libpython2.6.so.1.0
#37 0x00007ffff36926c0 in PyEval_EvalCodeEx ()
from /usr/lib64/libpython2.6.so.1.0
#38 0x00007ffff368a822 in PyEval_EvalCode ()
from /usr/lib64/libpython2.6.so.1.0
---Type <return> to continue, or q <return> to quit---
#39 0x00007ffff36a147c in PyImport_ExecCodeModuleEx ()
from /usr/lib64/libpython2.6.so.1.0
#40 0x00007ffff3966cc1 in wsgi_load_source (pool=0xb930c8, r=0xb93140,
name=0xb98300 "_mod_wsgi_006a4a376ac2ebb09dfe1a204af8a289",
exists=<value optimized out>,
filename=0xb94d38 "/usr/local/apache2/htdocs/hello-world.wsgi",
process_group=0x7ffff3973538 "",
application_group=0xb976c0 "bmidev1.kumc.edu|/hello") at
mod_wsgi.c:6125
#41 0x00007ffff396be2c in wsgi_execute_script (r=0xb93140) at
mod_wsgi.c:6482
#42 0x0000000000443363 in ap_run_handler (r=0xb93140) at config.c:158
#43 0x000000000044687f in ap_invoke_handler (r=0xb93140) at config.c:372
#44 0x000000000047186e in ap_process_request (r=0xb93140)
at http_request.c:282
#45 0x000000000046e858 in ap_process_http_connection (c=0x9afd10)
at http_core.c:190
#46 0x000000000044a8a3 in ap_run_process_connection (c=0x9afd10)
at connection.c:43
#47 0x00000000004875d0 in child_main (child_num_arg=<value optimized
out>)
at prefork.c:662
#48 0x0000000000487858 in make_child (s=0x6d55d0, slot=0) at
prefork.c:702
---Type <return> to continue, or q <return> to quit---
#49 0x0000000000487e70 in ap_mpm_run (_pconf=<value optimized out>,
plog=<value optimized out>, s=<value optimized out>) at
prefork.c:978
#50 0x000000000042fd95 in main (argc=2, argv=0x7fffffffecc8) at
main.c:740
Dan Connolly
> On Mon, 2010-06-28 at 10:51 -0500, Dan Connolly wrote:
> > I seem to be hitting the pyexpat issue...
> >
> > Anybody want to remind me of the fix or work-around?
> > Perhaps I can find it myself...
>
> OK, so I found
> http://code.google.com/p/modwsgi/wiki/IssuesWithExpatLibrary
>
> and I re-built apache with --disable-expat.
> Now lsof shows no libexpat.so at all.
Not util it's dynamically loaded, that is.
tee hee.
> But I'm still getting a SEGV in roughly the same place:
The fix was:
1) sudo zypper install libexpat-devel
2) add --with-expat=/usr/lib to apache config options and recompile.
That IssuesWithExpatLibrary doc is really quite nice. Thanks!
Graham Dumpleton
There is a slight problem with that. The expat issues shouldn't apply
to Python 2.5 and later as from then on the pyexpat bundled with
Python namespace prefixed the symbols so they didn't clash with any
separate expat shared library used by something else linked into the
same executable.
The only way there could be a problem is if pyexpat was updated to
newer expat code and they forgot to properly namespace prefix some
symbols that were being used. Alternatively if the pyexpat was
explicitly compiled so as to use system libexpat instead in contained
expat code.
It seems that for MacOS X Snow Leopard supplied Python 2.6 as example,
it does use separate expat library. You can find this out by going:
nm /System/Library/Frameworks/Python.framework/Versions/Current/lib/python2.6/lib-dynload/pyexpat.so
| grep XML
yielding:
U _XML_ErrorString
U _XML_ExpatVersion
U _XML_ExpatVersionInfo
U _XML_ExternalEntityParserCreate
U _XML_FreeContentModel
U _XML_GetBase
U _XML_GetBuffer
U _XML_GetCurrentByteIndex
U _XML_GetCurrentColumnNumber
U _XML_GetCurrentLineNumber
U _XML_GetErrorCode
U _XML_GetFeatureList
U _XML_GetInputContext
U _XML_GetSpecifiedAttributeCount
U _XML_Parse
U _XML_ParseBuffer
U _XML_ParserCreate
U _XML_ParserCreateNS
U _XML_ParserCreate_MM
U _XML_ParserFree
U _XML_SetAttlistDeclHandler
U _XML_SetBase
U _XML_SetCharacterDataHandler
U _XML_SetCommentHandler
U _XML_SetDefaultHandler
U _XML_SetDefaultHandlerExpand
U _XML_SetElementDeclHandler
U _XML_SetElementHandler
U _XML_SetEndCdataSectionHandler
U _XML_SetEndDoctypeDeclHandler
U _XML_SetEndElementHandler
U _XML_SetEndNamespaceDeclHandler
U _XML_SetEntityDeclHandler
U _XML_SetExternalEntityRefHandler
U _XML_SetNamespaceDeclHandler
U _XML_SetNotStandaloneHandler
U _XML_SetNotationDeclHandler
U _XML_SetParamEntityParsing
U _XML_SetProcessingInstructionHandler
U _XML_SetReturnNSTriplet
U _XML_SetSkippedEntityHandler
U _XML_SetStartCdataSectionHandler
U _XML_SetStartDoctypeDeclHandler
U _XML_SetStartElementHandler
U _XML_SetStartNamespaceDeclHandler
U _XML_SetUnknownEncodingHandler
U _XML_SetUnparsedEntityDeclHandler
U _XML_SetUserData
U _XML_SetXmlDeclHandler
U _XML_StopParser
U _XML_UseForeignDTD
Whereas if I check hand compiled Python 2.6.5 on same OS I get:
000000000000f6f0 T _PyExpat_XML_DefaultCurrent
00000000000091a0 T _PyExpat_XML_ErrorString
00000000000091c0 T _PyExpat_XML_ExpatVersion
00000000000091d0 T _PyExpat_XML_ExpatVersionInfo
000000000000ca30 T _PyExpat_XML_ExternalEntityParserCreate
0000000000009120 T _PyExpat_XML_FreeContentModel
00000000000089f0 T _PyExpat_XML_GetBase
000000000000af10 T _PyExpat_XML_GetBuffer
0000000000008fc0 T _PyExpat_XML_GetCurrentByteCount
0000000000008f90 T _PyExpat_XML_GetCurrentByteIndex
00000000000090b0 T _PyExpat_XML_GetCurrentColumnNumber
0000000000009040 T _PyExpat_XML_GetCurrentLineNumber
0000000000008f80 T _PyExpat_XML_GetErrorCode
00000000000091f0 T _PyExpat_XML_GetFeatureList
0000000000008a10 T _PyExpat_XML_GetIdAttributeIndex
0000000000009000 T _PyExpat_XML_GetInputContext
0000000000008f70 T _PyExpat_XML_GetParsingStatus
0000000000008a00 T _PyExpat_XML_GetSpecifiedAttributeCount
0000000000009180 T _PyExpat_XML_MemFree
0000000000009140 T _PyExpat_XML_MemMalloc
0000000000009160 T _PyExpat_XML_MemRealloc
000000000000b0f0 T _PyExpat_XML_Parse
0000000000008c90 T _PyExpat_XML_ParseBuffer
000000000000c610 T _PyExpat_XML_ParserCreate
000000000000c660 T _PyExpat_XML_ParserCreateNS
000000000000c5c0 T _PyExpat_XML_ParserCreate_MM
000000000000a480 T _PyExpat_XML_ParserFree
000000000000bb90 T _PyExpat_XML_ParserReset
0000000000008e80 T _PyExpat_XML_ResumeParser
0000000000008c20 T _PyExpat_XML_SetAttlistDeclHandler
000000000000b6c0 T _PyExpat_XML_SetBase
0000000000008a80 T _PyExpat_XML_SetCdataSectionHandler
0000000000008a50 T _PyExpat_XML_SetCharacterDataHandler
0000000000008a70 T _PyExpat_XML_SetCommentHandler
0000000000008ac0 T _PyExpat_XML_SetDefaultHandler
0000000000008ae0 T _PyExpat_XML_SetDefaultHandlerExpand
0000000000008b00 T _PyExpat_XML_SetDoctypeDeclHandler
0000000000008c10 T _PyExpat_XML_SetElementDeclHandler
0000000000008a20 T _PyExpat_XML_SetElementHandler
000000000000b780 T _PyExpat_XML_SetEncoding
0000000000008ab0 T _PyExpat_XML_SetEndCdataSectionHandler
0000000000008b30 T _PyExpat_XML_SetEndDoctypeDeclHandler
0000000000008a40 T _PyExpat_XML_SetEndElementHandler
0000000000008b90 T _PyExpat_XML_SetEndNamespaceDeclHandler
0000000000008c30 T _PyExpat_XML_SetEntityDeclHandler
0000000000008bb0 T _PyExpat_XML_SetExternalEntityRefHandler
0000000000008bc0 T _PyExpat_XML_SetExternalEntityRefHandlerArg
0000000000008b60 T _PyExpat_XML_SetNamespaceDeclHandler
0000000000008ba0 T _PyExpat_XML_SetNotStandaloneHandler
0000000000008b50 T _PyExpat_XML_SetNotationDeclHandler
0000000000008c50 T _PyExpat_XML_SetParamEntityParsing
0000000000008a60 T _PyExpat_XML_SetProcessingInstructionHandler
00000000000089a0 T _PyExpat_XML_SetReturnNSTriplet
0000000000008be0 T _PyExpat_XML_SetSkippedEntityHandler
0000000000008aa0 T _PyExpat_XML_SetStartCdataSectionHandler
0000000000008b20 T _PyExpat_XML_SetStartDoctypeDeclHandler
0000000000008a30 T _PyExpat_XML_SetStartElementHandler
0000000000008b80 T _PyExpat_XML_SetStartNamespaceDeclHandler
0000000000008bf0 T _PyExpat_XML_SetUnknownEncodingHandler
0000000000008b40 T _PyExpat_XML_SetUnparsedEntityDeclHandler
00000000000089c0 T _PyExpat_XML_SetUserData
0000000000008c40 T _PyExpat_XML_SetXmlDeclHandler
0000000000008df0 T _PyExpat_XML_StopParser
0000000000008970 T _PyExpat_XML_UseForeignDTD
0000000000008960 T _PyExpat_XML_UseParserAsHandlerArg
See how they are part of object and prefixed.
You might do that same nm on pyexpat.so in your Python installation
and work out if local symbols or undefined, thus meaning they come
from external libexpat.so.
It almost seems that your pyexpat.so is using libexpat.so and so you
can get problems mentioned in that document.
Seems I will need to update that document to indicate that can occur
for latter Python versions if it was configured to use libexpat.so
instead of builtin code.
Graham
Graham Dumpleton
http://code.google.com/p/modwsgi/issues/detail?id=203
to remind me to update documentation. :-)
Graham