SSL Issues in the New THREDDS Tutorial

[swainn]

I had the following question sent to me recently and I thought the answer would be helpful for many of you:

We were trying to add the data from https://thredds.servirglobal.net/thredds/catalog/Locusts/catalog.html to the thredds tutorial and we get the following error.

 

requests.exceptions.SSLError: HTTPSConnectionPool(host='thredds.servirglobal.net', port=443): Max retries exceeded with url: /thredds/catalog/Locusts/catalog.xml (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')])")))

 

I looked in the logs and it seems to be making the get request

 

./tethys_apps/base/app_base.py", line 1039, in get_spatial_dataset_service

 

We were wondering if you could take a look at it and see if you could make it work?  The certificate is a valid cert,  you can navigate to the xml referred to in the error message and see it https://thredds.servirglobal.net/thredds/catalog/Locusts/catalog.xml

[swainn]

This error is caused by an issue with the certificate authority that issued your THREDDS certificate not being recognized. This can and does happen with valid certificates often. Tethys uses a library called Siphon to interact with THREDDS services and Siphon uses the Requests library behind the scenes to issue the HTTP requests to the server. Requests uses a default Certificate Authority (CA) Bundle to validate SSL requests. The bundle is called something like cacert.pem and is installed as part of the certifi package in site-packages. This CA default bundle is not all-inclusive and is provided more for convenience of development.

You have a few options if the authority that issued your certificate is not included in the default certifi package:

1. You can disable SSL checks. If you were using the Requests library directly, this is easily done by setting the verify parameter to False (see: https://2.python-requests.org/en/v1.1.0/user/advanced/#ssl-cert-verification). Since we don't have direct control over how Siphon is making Requests calls, this can be done using the tip in the tutorial (see Tip box at the end of step 3.2 of the Visualize THREDDS Services with Leaflet). I ONLY recommend this as a temporary workaround for development and not in production (see next suggestion).

2. For production, you will need to provide your own CA bundle that contains the PEM key for the certificate authority that issued your certificate for your THREDDS server. You can override the default CA bundle used by Requests using the REQUESTS_CA_BUNDLE environment variable. Set this variable to the path to your custom CA bundle. A few suggestions for custom CA bundles:

    - Try using the system certificates. On Ubuntu / Debian these are usually located at /etc/ssl/certs/ca-certficiates.crt

    - Copy the default CA bundle from certifi and append the key for your certificate authority to the end of it. How to get the key for your certificate authority depends on the authority. You'll need to look it up in their documentation, likely.

Note that this same discussion applies to SSL issues with using owslib.