there seems to be a problem getting the uri module to play nicely with two way TLS. I have managed to make client validation OR server validation work, but not both at the same time, which seems strange. The same setup works without problem with cURL, as demonstrated below. If you have some experience with the uri module and TLS, maybe you can help.
THE SETUP
I have set up Apache httpd to accept TLS connections on localhost, with a self signed root certificate issued for 127.0.0.1, an intermediate CA certificate, a server and a client cert. I can test this setup with cURL, and everything works fine, see below.
The file host.127.0.0.1.cert.pem contains three certificates: the root, the intermediate, and the server cert.
The file ca.chain.pem contains two certificates: the root and the intermediate.
The file client.SomeUser.cert.pem contains a single certificate: the client cert
As you can see from the output below, I'm running python version = 3.7.0
From httpd.conf:
=========================================
Listen 443
<VirtualHost *:443>
ServerName 127.0.0.1
SSLEngine on
SSLCertificateFile "/path/to/host.127.0.0.1.cert.pem"
SSLCertificateKeyFile "/path/to/host.127.0.0.1.privkey.nopass.pem"
SSLVerifyClient require
SSLVerifyDepth 2
SSLCACertificateFile "/path/to/ca.chain.pem"
</VirtualHost>
=========================================
TESTING THE SETUP WITH CURL
The cURL command and output:
=========================================
$ curl -vvv --cert client.SomeUser.cert.pem --key client.SomeUser.privkey.pem --cacert ca.chain.pem -X GET https://127.0.0.1 * Connected to 127.0.0.1 (127.0.0.1) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
Enter PEM pass phrase:
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
* CAfile: /Users/mans/devel/phenixid/test-ca/src/test/resources/fixtures/tls/ca1/ca.chain.pem
CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Request CERT (13):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS handshake, CERT verify (15):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-CHACHA20-POLY1305
* ALPN, server accepted to use http/1.1
* Server certificate:
* subject: C=SE; ST=Nacka; L=Nacka Strand; O=Example; OU=Testers; CN=127.0.0.1
* start date: Dec 14 11:45:08 2018 GMT
* expire date: Dec 21 11:45:08 2018 GMT
* subjectAltName: host "127.0.0.1" matched cert's IP address!
* issuer: C=SE; ST=Nacka; O=Example; OU=Testers; CN=Example Intermediate CA 1
* SSL certificate verify ok.
> GET / HTTP/1.1
> Host: 127.0.0.1
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Fri, 14 Dec 2018 15:03:31 GMT
< Server: Apache/2.4.33 (Unix) LibreSSL/2.2.7
< Content-Location: index.html.en
< Vary: negotiate
< TCN: choice
< Last-Modified: Mon, 11 Jun 2007 18:53:14 GMT
< ETag: "2d-432a5e4a73a80"
< Accept-Ranges: bytes
< Content-Length: 45
< Content-Type: text/html
<
<html><body><h1>It works!</h1></body></html>
=========================================
TESTING THE SETUP WITH ANSIBLE
My ansible task looks like this:
=========================================
- name: "GET"
uri:
method: GET
validate_certs: yes
client_cert: "client.SomeUser.cert.pem"
client_key: "client.SomeUser.privkey.pem"
=========================================
So here is what I tried running the playbook:
- If I turn off cert validation on the server side, the client accepts the server certificate, if I include the CA chain in the SSL_CERT_FILE env var. Good.
- If I turn on cert validation on the server side, but turn it off client side (validate_certs: no), the server accepts the client cert. Good.
- But if I turn on cert validation on both sides, the client fails to accept the server certificate. Bad.
This is the ansible command and output when it fails:
=========================================
$ SSL_CERT_FILE=/path/to/ca.chain.pem ansible-playbook my-playbook.yml -vvv
ansible-playbook 2.7.4
config file = /Users/xxx/.ansible.cfg
configured module search path = ['/Users/xxx/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /Users/xxx/virtualenvs/p3/lib/python3.7/site-packages/ansible
executable location = /Users/xxx/virtualenvs/p3/bin/ansible-playbook
python version = 3.7.0 (default, Aug 22 2018, 15:22:33) [Clang 9.1.0 (clang-902.0.39.2)]
Using /Users/xxx/.ansible.cfg as config file
/etc/ansible/hosts did not meet host_list requirements, check plugin documentation if this is unexpected
/etc/ansible/hosts did not meet script requirements, check plugin documentation if this is unexpected
Parsed /etc/ansible/hosts inventory source with ini plugin
......output omitted......
The full traceback is:
File "/var/folders/78/n3ytyhrs2415sl3h8h1fms000000gn/T/ansible_uri_payload_bus4199a/ansible_uri_payload.zip/ansible/module_utils/urls.py", line 1259, in fetch_url
client_key=client_key, cookies=cookies)
File "/var/folders/78/n3ytyhrs2415sl3h8h1fms000000gn/T/ansible_uri_payload_bus4199a/ansible_uri_payload.zip/ansible/module_utils/urls.py", line 1162, in open_url
client_cert=client_cert, client_key=client_key, cookies=cookies)
File "/var/folders/78/n3ytyhrs2415sl3h8h1fms000000gn/T/ansible_uri_payload_bus4199a/ansible_uri_payload.zip/ansible/module_utils/urls.py", line 1070, in open
r = urllib_request.urlopen(*urlopen_args)
File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 523, in open
req = meth(req)
File "/var/folders/78/n3ytyhrs2415sl3h8h1fms000000gn/T/ansible_uri_payload_bus4199a/ansible_uri_payload.zip/ansible/module_utils/urls.py", line 779, in http_request
build_ssl_validation_error(self.hostname, self.port, paths_checked, e)
File "/var/folders/78/n3ytyhrs2415sl3h8h1fms000000gn/T/ansible_uri_payload_bus4199a/ansible_uri_payload.zip/ansible/module_utils/urls.py", line 580, in build_ssl_validation_error
raise SSLValidationError(' '.join(msg) % (hostname, port, ", ".join(paths)))
fatal: [127.0.0.1]: FAILED! => {
"changed": false,
"invocation": {
"module_args": {
"attributes": null,
"backup": null,
"body": null,
"body_format": "raw",
"client_cert": "/path/to/client.SomeUser.cert.pem",
"client_key": "/path/to/client.SomeUser.privkey.pem",
"content": null,
"creates": null,
"delimiter": null,
"dest": null,
"directory_mode": null,
"follow": false,
"follow_redirects": "safe",
"force": false,
"force_basic_auth": false,
"group": null,
"headers": {},
"http_agent": "ansible-httpget",
"method": "GET",
"mode": null,
"owner": null,
"regexp": null,
"remote_src": null,
"removes": null,
"return_content": false,
"selevel": null,
"serole": null,
"setype": null,
"seuser": null,
"src": null,
"status_code": [
200
],
"timeout": 30,
"unsafe_writes": null,
"url_password": null,
"url_username": null,
"use_proxy": true,
"validate_certs": true
}
},
"msg": "Failed to validate the SSL certificate for 127.0.0.1:443. Make sure your managed systems have a valid CA certificate installed. You can use validate_certs=False if you do not need to confirm the servers identity but this is unsafe and not recommended. Paths checked for this platform: /etc/ssl/certs, /etc/ansible, /usr/local/etc/openssl. The exception msg was: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:1045)." }