Correct SSL config for use in OpenShift with mod_wsgi-express

[John Anderson]

I am trying to get SSL working in an OpenShift environment for service to service communication. Bellow is a setup that is working for me but is producing some strange behavior and logs.

The OpenShift service name is "foo" in this case and FOO_SERVICE_PORT will eval to 8443.

python manage.py runmodwsgi --https-port=$FOO_SERVICE_PORT --server-name=foo  --server-alias=*.example.com --server-alias=localhost --allow-localhost --https-only --log-to-terminal --log-level=info

Successfully ran command.

Server URL         : http://foo:8000/

Server URL (HTTPS) : https://foo:8443/

Server Root        : /tmp/mod_wsgi-localhost:8000:1001230000

Server Conf        : /tmp/mod_wsgi-localhost:8000:1001230000/httpd.conf

Error Log File     : /dev/stderr (info)

Request Capacity   : 5 (1 process * 5 threads)

Request Timeout    : 60 (seconds)

Startup Timeout    : 15 (seconds)

Queue Backlog      : 100 (connections)

Queue Timeout      : 45 (seconds)

Server Capacity    : 20 (event/worker), 20 (prefork)

Server Backlog     : 500 (connections)

Locale Setting     : en_US.UTF-8

[Thu Sep 26 07:19:46.932499 2019] [ssl:info] [pid 31:tid 140489790564544] AH01887: Init: Initializing (virtual) servers for SSL

[Thu Sep 26 07:19:46.932559 2019] [ssl:info] [pid 31:tid 140489790564544] AH01914: Configuring server foo:443 for SSL protocol

[Thu Sep 26 07:19:46.932968 2019] [ssl:warn] [pid 31:tid 140489790564544] AH01909: foo:443:0 server certificate does NOT include an ID which matches the server name

[Thu Sep 26 07:19:46.932977 2019] [ssl:info] [pid 31:tid 140489790564544] AH02568: Certificate and private key foo:443:0 configured from /etc/tls-certs/tls.crt and /etc/tls-certs/tls.key

[Thu Sep 26 07:19:46.933045 2019] [ssl:info] [pid 31:tid 140489790564544] AH01914: Configuring server localhost:8443 for SSL protocol

[Thu Sep 26 07:19:46.933237 2019] [ssl:warn] [pid 31:tid 140489790564544] AH01909: localhost:8443:0 server certificate does NOT include an ID which matches the server name

[Thu Sep 26 07:19:46.933244 2019] [ssl:info] [pid 31:tid 140489790564544] AH02568: Certificate and private key localhost:8443:0 configured from /etc/tls-certs/tls.crt and /etc/tls-certs/tls.key

[Thu Sep 26 07:19:46.933311 2019] [ssl:info] [pid 31:tid 140489790564544] AH01876: mod_ssl/2.4.34 compiled against Server: Apache/2.4.34, Library: OpenSSL/1.0.2k

[Thu Sep 26 07:19:46.942518 2019] [ssl:warn] [pid 31:tid 140489790564544] AH01873: Init: Session Cache is not configured [hint: SSLSessionCache]

[Thu Sep 26 07:19:46.942527 2019] [ssl:info] [pid 31:tid 140489790564544] AH01887: Init: Initializing (virtual) servers for SSL

[Thu Sep 26 07:19:46.942535 2019] [ssl:info] [pid 31:tid 140489790564544] AH01914: Configuring server foo:443 for SSL protocol

[Thu Sep 26 07:19:46.942864 2019] [ssl:warn] [pid 31:tid 140489790564544] AH01909: foo:443:0 server certificate does NOT include an ID which matches the server name

[Thu Sep 26 07:19:46.942883 2019] [ssl:info] [pid 31:tid 140489790564544] AH02568: Certificate and private key foo:443:0 configured from /etc/tls-certs/tls.crt and /etc/tls-certs/tls.key

[Thu Sep 26 07:19:46.942972 2019] [ssl:info] [pid 31:tid 140489790564544] AH01914: Configuring server localhost:8443 for SSL protocol

[Thu Sep 26 07:19:46.943144 2019] [ssl:warn] [pid 31:tid 140489790564544] AH01909: localhost:8443:0 server certificate does NOT include an ID which matches the server name

[Thu Sep 26 07:19:46.943151 2019] [ssl:info] [pid 31:tid 140489790564544] AH02568: Certificate and private key localhost:8443:0 configured from /etc/tls-certs/tls.crt and /etc/tls-certs/tls.key

[Thu Sep 26 07:19:46.943211 2019] [ssl:info] [pid 31:tid 140489790564544] AH01876: mod_ssl/2.4.34 compiled against Server: Apache/2.4.34, Library: OpenSSL/1.0.2k

[Thu Sep 26 07:19:46.944517 2019] [wsgi:info] [pid 34:tid 140489790564544] mod_wsgi (pid=34): Starting process 'localhost:8000' with threads=5.

[Thu Sep 26 07:19:46.944776 2019] [wsgi:info] [pid 34:tid 140489790564544] mod_wsgi (pid=34): Python home /opt/app-root.

[Thu Sep 26 07:19:46.944840 2019] [wsgi:info] [pid 34:tid 140489790564544] mod_wsgi (pid=34): Initializing Python.

[Thu Sep 26 07:19:46.945195 2019] [mpm_event:notice] [pid 31:tid 140489790564544] AH00489: Apache/2.4.34 (Red Hat) mod_wsgi/4.6.7 Python/3.6 OpenSSL/1.0.2k-fips configured -- resuming normal operations

[Thu Sep 26 07:19:46.945210 2019] [mpm_event:info] [pid 31:tid 140489790564544] AH00490: Server built: Nov  7 2018 07:25:25

[Thu Sep 26 07:19:46.945232 2019] [core:notice] [pid 31:tid 140489790564544] AH00094: Command line: 'httpd (mod_wsgi-express)         -f /tmp/mod_wsgi-localhost:8000:1001230000/httpd.conf -D MOD_WSGI_VIRTUAL_HOST -D MOD_WSGI_WITH_HTTPS -D MOD_WSGI_HTTPS_ONLY -D MOD_WSGI_SERVER_ALIAS -D MOD_WSGI_ALLOW_LOCALHOST -D MOD_WSGI_WITH_PROXY_HEADERS -D MOD_WSGI_MPM_ENABLE_EVENT_MODULE -D MOD_WSGI_MPM_EXISTS_EVENT_MODULE -D MOD_WSGI_MPM_EXISTS_WORKER_MODULE -D MOD_WSGI_MPM_EXISTS_PREFORK_MODULE -D FOREGROUND'

[Thu Sep 26 07:19:46.968551 2019] [wsgi:info] [pid 34:tid 140489790564544] mod_wsgi (pid=34): Attach interpreter ''.

[Thu Sep 26 07:19:46.972513 2019] [wsgi:info] [pid 34:tid 140489790564544] mod_wsgi (pid=34): Imported 'mod_wsgi'.

[Thu Sep 26 07:19:46.972580 2019] [wsgi:info] [pid 34:tid 140489790564544] mod_wsgi (pid=34, process='localhost:8000', application=''): Loading Python script file '/tmp/mod_wsgi-localhost:8000:1001230000/handler.wsgi'.

[Thu Sep 26 07:20:08.707321 2019] [ssl:info] [pid 35:tid 140489790293760] [client 1.1.1.1:59788] AH01964: Connection to child 0 established (server localhost:8443)

[Thu Sep 26 07:20:08.707663 2019] [ssl:info] [pid 35:tid 140489789761280] [client 2.2.2.2:36512] AH01964: Connection to child 2 established (server localhost:8443)

[Thu Sep 26 07:20:08.711239 2019] [ssl:info] [pid 35:tid 140489790293760] (104)Connection reset by peer: [client 1.1.1.1:59788] AH01991: SSL input filter read failed.

[Thu Sep 26 07:20:08.711296 2019] [ssl:info] [pid 35:tid 140489789761280] (104)Connection reset by peer: [client 2.2.2.2:36512] AH01991: SSL input filter read failed.

[Thu Sep 26 07:20:13.740839 2019] [ssl:info] [pid 35:tid 140489790027520] [client 2.2.2.2:36866] AH01964: Connection to child 1 established (server localhost:8443)

[Thu Sep 26 07:20:13.743444 2019] [ssl:info] [pid 35:tid 140489790027520] (104)Connection reset by peer: [client 2.2.2.2:36866] AH01991: SSL input filter read failed.

[Thu Sep 26 07:20:13.748342 2019] [ssl:info] [pid 35:tid 140489789495040] [client 1.1.1.1:60518] AH01964: Connection to child 3 established (server localhost:8443)

[Thu Sep 26 07:20:13.751069 2019] [ssl:info] [pid 35:tid 140489789495040] (104)Connection reset by peer: [client 1.1.1.1:60518] AH01991: SSL input filter read failed.

[Thu Sep 26 07:20:18.742468 2019] [ssl:info] [pid 35:tid 140489789228800] [client 2.2.2.2:37190] AH01964: Connection to child 4 established (server localhost:8443)

I understand and am okay with the ssl warnings at the top that the ID of the cert does not actually contain "foo". This is expected as I am using the automated TLS service cert creation API in OpenShift. Again, things are working. It is the bottom set of messages about SSL input filter read failed that I don't get. I know this has something to do with the virtual server name config for SSL, but given the mod_wsgi-express command-line options, I am not sure what to use. Also, why is it saying in the logs that foo is listening on 443 while localhost is listening on 8443?

I appreciate any help here,

John

[Graham Dumpleton]

If working, where are the options telling mod_wsgi (runmodwsgi) where the generated certificates are?

And can you clarify whether you are talking about the generated certificates you can inject into the pod when using re-encrpyt traffic with the OpenShift route, or something else?

If it is re-encrypt, I would still expect to see options pointing at the certificates. For example:

https://github.com/openshift-katacoda/blog-django-py/blob/master/app.sh#L16

That said, I can't quite remember how you inject those certificates in. You mount them in, but I don't remember the config require in OpenShift.

It is the bottom set of messages about SSL input filter read failed that I don't get. I know this has something to do with the virtual server name config for SSL, but given the mod_wsgi-express command-line options, I am not sure what to use.

I am not so concerned about the input filter read errors. My recollection is that is normal to see those in some cases where connection is closed off especially if keep alive is enabled, but then I don't see you enabling keep alive. Does it happen on every request?

I am actually not even sure why mod_ssl is being so noisy in the logs for info level. I would not regard messages about each and every connection to be info level details you want. You may want to consider using:

    --log-level "info ssl:notice"

to reduce how much it logs.

Also, why is it saying in the logs that foo is listening on 443 while localhost is listening on 8443?

That I am not sure. Can you confirm that there is no Listen in:

    /tmp/mod_wsgi-localhost:8000:1001230000/httpd.conf

for port 443.

I appreciate any help here,

John

--
You received this message because you are subscribed to the Google Groups "modwsgi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to modwsgi+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/modwsgi/CADOYVe33LFiv_nE076GrQQmYezxrr%3DbTtPmhNtoJdaLc2EWjxA%40mail.gmail.com.

[John Anderson]

My apologies looks like I only copy/pasted half the arguments being used... Sorry about that. Here is the full command:

python manage.py runmodwsgi --https-port=$FOO_SERVICE_PORT --server-name=foo  --server-alias=*.example.com --server-alias=localhost --allow-localhost --https-only --log-to-terminal --log-level=info --ssl-certificate=/etc/tls-certs/tls --trust-proxy-header=X-Forwarded-For --trust-proxy-header=X-Forwarded-Host --trust-proxy-header=X-Forwarded-Port --trust-proxy-header=X-Forwarded-Server --trust-proxy-header=X-Forwarded-Proto

Yes, I am using the OpenShift feature that allows you to place the bellow annotation on a service and have it automatically create a secret (of name foo-tls) with the contents of tls.crt and tls.key:

annotations:

  service.alpha.openshift.io/service-cert-sercet-name: foo-tls

Then as you pointed out, in the pod config you mount that secret to /etc/tls-certs, which then makes --ssl-certificate=/etc/tls-certs/tls valid.

One of my use cases is for reencrypted inbound traffic from the router, which is why I use --server-alias=*.example.com, but I am also setting this up for service to service tls within the cluster.

I took a look at the rendered config and confirmed it is not listening on 443, just 8443 and 8000. Is there any way to disable HTTP on 8000 altogether? It's not a huge concern as I am only exposing 8443 on the pod anyway, but just curious.

Barring further conversation, it sounds like these messages aren't really a concern. I don't plan on running this at info level anyway, just doing it here to get things going.

Thanks,

John

To view this discussion on the web visit https://groups.google.com/d/msgid/modwsgi/2D7EF0B8-B730-4249-8EE8-AC311D01A60A%40gmail.com.