I am author of a Flask application running under Linux/Apache mod_wsgi that is experiencing intermittent, random segmentation faults.
What is unusual is that the mod_wsgi process segfaults are occurring not at startup when mod_wsgi is loaded, or at when an incoming request accesses the app, but when the wsgi processes are just sitting there, quiescent.
From a user's point of view, everything looks fine, the mod_wsgi processes and the app respond with the right results with no sign of trouble at the client's browser. But looking at the Apache logs shows the wsgi processes periodically segfaulting and getting restarted with no correlated incoming requests. They die sometimes after running for a few minutes, sometimes after a few hours. There are no incoming requests to the the wsgi app logged near the time of these crashes.
For example:
[Mon May 30 22:35:43.040387 2022] [wsgi:info] [pid 2575903:tid 139929303559104] mod_wsgi (pid=2575903): Initializing Python.
[Mon May 30 22:35:43.099053 2022] [wsgi:info] [pid 2575903:tid 139929303559104] mod_wsgi (pid=2575903): Attach interpreter ''.
[Tue
May 31 01:29:06.434000 2022] [core:notice] [pid 2876203:tid
139929303559104] AH00052: child pid 2511562 exit signal Segmentation
fault (11)
[Tue May 31 01:29:07.466268 2022] [wsgi:info] [pid 2605661:tid 139929303559104] mod_wsgi (pid=2605661): Initializing Python.
[Tue May 31 01:29:07.517413 2022] [wsgi:info] [pid 2605661:tid 139929303559104] mod_wsgi (pid=2605661): Attach interpreter ''.
[Tue
May 31 04:14:59.405491 2022] [core:notice] [pid 2876203:tid
139929303559104] AH00052: child pid 2575903 exit signal Segmentation
fault (11)
My wsgi app is still being tested so other than infrequent requests generated by me and a few other people there is very little traffic to it. However the web server itself is handling some continuous moderate volume of traffic to other apps including to C, Python and PHP CGI apps.
What I know about the environment (if any other info would be useful I'll try and dig it up):
$ cat /etc/*release
PRETTY_NAME="Debian GNU/Linux 11 (bullseye)
Apache, mod_wsgi, python were all built from source by the site's administrator.
There are (at least) two Python's on the system:
/usr/bin/python3 -- 3.9.2
/usr/local/bin/python3 -- 3.10.1
Apachche/mod_wsgi is was supposedly built against python-3.10. From
the http server header:
Apache/2.4.54 (Unix) OpenSSL/1.1.1n mod_wsgi/4.9.4 Python/3.10 PHP/7.4.23
The Apache .conf file uses:
WSGIDaemonProcess myapp processes=2 threads=10 \
display-name=apache2-myapp locale=en_US.UTF-8 lang=en_US.UTF-8
$ /usr/local/apache2/bin/httpd -V
Server version: Apache/2.4.54 (Unix)
Server built: Oct 13 2022 00:07:38
Server's Module Magic Number: 20120211:124
Server loaded: APR 1.6.5, APR-UTIL 1.6.1, PCRE 10.36 2020-12-04
Compiled using: APR 1.6.5, APR-UTIL 1.6.1, PCRE 10.36 2020-12-04
Architecture: 64-bit
Server MPM: event
threaded: yes (fixed thread count)
forked: yes (variable process count)
Server compiled with....
-D APR_HAS_SENDFILE
-D APR_HAS_MMAP
-D APR_HAVE_IPV6 (IPv4-mapped addresses enabled)
-D APR_USE_SYSVSEM_SERIALIZE
-D APR_USE_PTHREAD_SERIALIZE
-D SINGLE_LISTEN_UNSERIALIZED_ACCEPT
-D APR_HAS_OTHER_CHILD
-D AP_HAVE_RELIABLE_PIPED_LOGS
-D DYNAMIC_MODULE_LIMIT=256
-D HTTPD_ROOT="/usr/local/apache2"
-D SUEXEC_BIN="/usr/local/apache2/bin/suexec"
-D DEFAULT_PIDLOG="logs/httpd.pid"
-D DEFAULT_SCOREBOARD="logs/apache_runtime_status"
-D DEFAULT_ERRORLOG="logs/error_log"
-D AP_TYPES_CONFIG_FILE="conf/mime.types"
-D SERVER_CONFIG_FILE="conf/httpd.conf"
$ bin/httpd -M
Loaded Modules:
core_module (static)
so_module (static)
http_module (static)
mpm_event_module (static)
authz_core_module (shared)
authz_host_module (shared)
unixd_module (shared)
dir_module (shared)
access_compat_module (shared)
env_module (shared)
alias_module (shared)
log_config_module (shared)
ssl_module (shared)
mime_module (shared)
socache_shmcb_module (shared)
setenvif_module (shared)
cgid_module (shared)
userdir_module (shared)
headers_module (shared)
rewrite_module (shared)
autoindex_module (shared)
negotiation_module (shared)
dav_module (shared)
deflate_module (shared)
info_module (shared)
status_module (shared)
wsgi_module (shared)
evasive24_module (shared)
php7_module (shared)