Does not run against windows targets anymore

[YvanM]

Hi,

I am running Ansible 2.7.6 on Debian testing/sid and regularly run a playbook against Windows 7 desktops. However, I can not connect anymore and receive the following error (see below for a more verbose output):

ntlm: HTTPSConnectionPool(host='my-host', port=5986): Max retries exceeded with url: /wsman (Caused by SSLError(SSLError(1, '[SSL: UNSUPPORTED_PROTOCOL] unsupported protocol (_ssl.c:1056)')))

I have been able to do a WinRM connection from another Windows computer to the same target (as explained somewhere in Ansible documentation), so the target is probably OK. I did not find any similar issue on this forum, nor Ansible bug tracker, nor pywinrm bug tracker.

I know that debian recently moved to python3 for Ansible, and it seems that pywinrm has many issues whith python3 (see for example https://github.com/diyan/pywinrm/pull/222). Are there other people running Ansible with python3 against Windows host using self-signed certificates and NTLM ? Do you have any idea to help debugging this ?

Best regards,

Yvan

The more verbose output:

TASK [Gathering Facts] ***************************************************************************************************************************************************************************
task path: /home/yvan/Nextcloud/Lycee/Ansible/windows-desktops.yml:1
Using module file /usr/lib/python3/dist-packages/ansible/modules/windows/setup.ps1
<some-host> ESTABLISH WINRM CONNECTION FOR USER: .\administrateur on PORT 5986 TO my-host
checking if winrm_host my-host is an IPv6 address
<my-host> WINRM CONNECT: transport=ntlm endpoint=https://my-host:5986/wsman
<my-host> WINRM CONNECTION ERROR: HTTPSConnectionPool(host='my-host', port=5986): Max retries exceeded with url: /wsman (Caused by SSLError(SSLError(1, '[SSL: UNSUPPORTED_PROTOCOL] unsupported protocol (_ssl.c:1056)')))
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 343, in _make_request
    self._validate_conn(conn)
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 841, in _validate_conn
    conn.connect()
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 344, in connect
    ssl_context=context)
  File "/usr/lib/python3/dist-packages/urllib3/util/ssl_.py", line 342, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "/usr/lib/python3.7/ssl.py", line 412, in wrap_socket
    session=session
  File "/usr/lib/python3.7/ssl.py", line 853, in _create
    self.do_handshake()
  File "/usr/lib/python3.7/ssl.py", line 1117, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: UNSUPPORTED_PROTOCOL] unsupported protocol (_ssl.c:1056)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 638, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 398, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='my-host', port=5986): Max retries exceeded with url: /wsman (Caused by SSLError(SSLError(1, '[SSL: UNSUPPORTED_PROTOCOL] unsupported protocol (_ssl.c:1056)')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/ansible/plugins/connection/winrm.py", line 400, in _winrm_connect
    self.shell_id = protocol.open_shell(codepage=65001)  # UTF-8
  File "/usr/lib/python3/dist-packages/winrm/protocol.py", line 157, in open_shell
    res = self.send_message(xmltodict.unparse(req))
  File "/usr/lib/python3/dist-packages/winrm/protocol.py", line 234, in send_message
    resp = self.transport.send_message(message)
  File "/usr/lib/python3/dist-packages/winrm/transport.py", line 265, in send_message
    response = self._send_message_request(prepared_request, message)
  File "/usr/lib/python3/dist-packages/winrm/transport.py", line 270, in _send_message_request
    response = self.session.send(prepared_request, timeout=self.read_timeout_sec)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 637, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/adapters.py", line 514, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='my-host', port=5986): Max retries exceeded with url: /wsman (Caused by SSLError(SSLError(1, '[SSL: UNSUPPORTED_PROTOCOL] unsupported protocol (_ssl.c:1056)')))

fatal: [my-host]: UNREACHABLE! => {
    "changed": false,
    "msg": "ntlm: HTTPSConnectionPool(host='my-host', port=5986): Max retries exceeded with url: /wsman (Caused by SSLError(SSLError(1, '[SSL: UNSUPPORTED_PROTOCOL] unsupported protocol (_ssl.c:1056)')))",
    "unreachable": true
}

[Jordan Borean]

That sounds like your Windows host has been updated recently and it's list of supported cipher suites or TLS protocols has changed. Unfortunately there's now no common cipher suites and TLS protocol versions between your linux and Windows host hence the message unsupported protocol.

Ansible or pywinrm has no control over this as this all happens down deep in the Python layers and really even further into the OpenSSL provider. It sounds like your Python 3 has been compiled against an old OpenSSL version which doesn't support the protocol or cipher suites that is required. Are you able to run

python3 -c "import ssl; print(ssl.OPENSSL_VERSION)"

and post back the results here.

Thanks

Jordan

[YvanM]

Thanks for this very informative answer! But it seems the SSL library is not very old:

$ python3 -c "import ssl; print(ssl.OPENSSL_VERSION)"
OpenSSL 1.1.1a  20 Nov 2018

Maybe it is the contrary (SSL is too new on my side and does not accept old protocol/cypher)?

[YvanM]

Other information which might be useful (or not):

Packages versions: openssl 1.1.1a-1, python-openssl 19.0.0-1

$ openssl ciphers
TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_GCM_SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:DHE-RSA-AES256-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES256-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES128-SHA:RSA-PSK-AES256-GCM-SHA384:DHE-PSK-AES256-GCM-SHA384:RSA-PSK-CHACHA20-POLY1305:DHE-PSK-CHACHA20-POLY1305:ECDHE-PSK-CHACHA20-POLY1305:AES256-GCM-SHA384:PSK-AES256-GCM-SHA384:PSK-CHACHA20-POLY1305:RSA-PSK-AES128-GCM-SHA256:DHE-PSK-AES128-GCM-SHA256:AES128-GCM-SHA256:PSK-AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:ECDHE-PSK-AES256-CBC-SHA384:ECDHE-PSK-AES256-CBC-SHA:SRP-RSA-AES-256-CBC-SHA:SRP-AES-256-CBC-SHA:RSA-PSK-AES256-CBC-SHA384:DHE-PSK-AES256-CBC-SHA384:RSA-PSK-AES256-CBC-SHA:DHE-PSK-AES256-CBC-SHA:AES256-SHA:PSK-AES256-CBC-SHA384:PSK-AES256-CBC-SHA:ECDHE-PSK-AES128-CBC-SHA256:ECDHE-PSK-AES128-CBC-SHA:SRP-RSA-AES-128-CBC-SHA:SRP-AES-128-CBC-SHA:RSA-PSK-AES128-CBC-SHA256:DHE-PSK-AES128-CBC-SHA256:RSA-PSK-AES128-CBC-SHA:DHE-PSK-AES128-CBC-SHA:AES128-SHA:PSK-AES128-CBC-SHA256:PSK-AES128-CBC-SHA

Regards,

Yvan

[YvanM]

I suppose I will need to check on the Windows side the supported protocols/cyphers. Do you think running the following command would be sufficient (I can not do it today but maybe tomorrow) ?

$ nmap --script ssl-enum-ciphers -p 5986 my-host

[YvanM]

I think I found the solution, but maybe you can confirm: the python-openssl package on Debian is only for python2, and I am currently missing the python3-openssl package.





I will install it and and let you know as soon as possible.

[Jordan Borean]

pyOpenSSL (python-openssl) is not used for standard SSL connections in Python. It is a requirement of CredSSP auth in Ansible but that's unrelated to the problem here.

I think you still have a mismatch of compatible TLS protocols just in the opposite direction we initially thought. I believe your Debian host only supports TLS 1.2 but your Windows 7 host only supports up to TLS 1.0. TLS 1.0 is old and potentially insecure which is why more and more distributions are disabling TLS 1.0 and TLS 1.1 and older, see https://lists.debian.org/debian-devel-announce/2017/08/msg00004.html.

Luckily for you, Windows 7 does support TLS 1.2 just not by default. You need to make sure you have installed the latest updates on your Windows host then create the following registry keys, you can use the below script to do so;

Function Enable-TLS12 {
    param(
        [ValidateSet("Server", "Client")]
        [String]$Component = "Server"
    )

    $protocols_path = "HKLM:\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols"
    New-Item -Path "$protocols_path\TLS 1.2\$Component" -Force
    New-ItemProperty -Path  "$protocols_path\TLS 1.2\$Component" -Name Enabled -Value 1 -Type DWORD -Force
    New-ItemProperty -Path  "$protocols_path\TLS 1.2\$Component" -Name DisabledByDefault -Value 0 -Type DWORD -Force
}
Enable-TLS12 -Component Server

# Not required but highly recommended to enable the Client side TLS 1.2 components
Enable-TLS12 -Component Client

You definitely need to enable the Server component but I also highly recommend you enable the Client component as well. Once you've created the registry keys you need to reboot the host and try again.

To verify independently what protocol is being negotiated by OpenSSL you can run the comman below;

openssl s_client -connect <hostname>:5986

Near the bottom of the output you can see something like the following;

SSL handshake has read 1884 bytes and written 293 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-GCM-SHA384
    Session-ID: E61400004F2F2662F404CDCD3203CB6AE5F53C36B0129AF615A016D404F1C155
    Session-ID-ctx:
    Master-Key: 123F15364A949A03DD75E7841EDD395304A2568B32124206A5E36BDDD10AF7837E74746DED16356972D318169DD81B5E
    Start Time: 1550089169
    Timeout   : 7200 (sec)
    Verify return code: 21 (unable to verify the first certificate)
---

We can see that the Protocol negotiated was TLSv1.2 and it negotiated the cipher suite 'ECDHE-RSA-AES256-GCM-SHA384'. Try running that before and after you enable TLS 1.2 on your Windows host and reboot to check for differences.

Thanks

Jordan

[YvanM]

THANKS! I would never have found this myself!

If it can help others, here is what I plan to do: as I am on a secure network, Il will allow TLS 1.0 (see the end end of /etc/ssl/openssl.cnf on Debian), run your script using ansible_shell module to enable TLS 1.2 on Windows 7, and finally disallow again TLS 1.0 on my computer.

I will try to find the time to submit a bug report to add what you explained to the documentation.

Regards,

Yvan