Tracking down semaphore leak in k8s (OpenShift)

[John Anderson]

I am running mod_wsgi (v4.6.7) + django in an OpenShift environment on a RHEL7 base image.

I have noticed a semaphore leak due to restarts of mod_wsgi because of container restarts. This is k8s so the good old pause container with shared IPC is in play for pods.

When my application containers restart they are allocating new semaphores each time mod_wsgi starts and eventually this results in IPC semaphore exhaustion.

This originally manifested because of a cluster platform issue which caused a high rate of container restarts in a short period of time.

This is my mod_wsgi invocation:

python manage.py runmodwsgi --https-port=$FOO_SERVICE_PORT --server-name=foo  --server-alias=*.example.com --server-alias=localhost --allow-localhost --https-only --log-to-terminal --log-level=info --ssl-certificate=/etc/tls-certs/tls --trust-proxy-header=X-Forwarded-For --trust-proxy-header=X-Forwarded-Host --trust-proxy-header=X-Forwarded-Port --trust-proxy-header=X-Forwarded-Server --trust-proxy-header=X-Forwarded-Proto

Is there anything glaring I could try to mitigate this? Perhaps with apache mutex settings?

Thanks,

John

[Graham Dumpleton]

Can’t give a proper reply right now but use —include-file to include an Apache config snippet which overrides ApacheMutex type and see if it helps. I can’t access what mutex type you could use right now and going to be offline for a while now.

Graham

On 8 Jan 2020, at 08:03, John Anderson <lamp...@gmail.com> wrote:



--
You received this message because you are subscribed to the Google Groups "modwsgi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to modwsgi+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/modwsgi/ac5a77ca-9ace-4f34-acf0-6a6444ab3928%40googlegroups.com.

[Graham Dumpleton]

Did you get a chance to try this?

The fallback is to usually use:

    Mutex file:/path/to/mutex

From memory the daemon processes should inherit the main Apache setting. If not you may also need to set:

    WSGIAcceptMutex file:/path/to/mutex

Graham