Journalist in Atlanta seeks help with SSL certificate error while trying to upload to Overview server from command line

[Jennifer Peebles]

AP Overview friends,

Hello from Atlanta, and I'd like to ask if anyone out there could help me.

I'm at the Atlanta Journal-Constitution, and we're using AP Overview to poke into a large dump of government PDFs. 

We have the multi-user version of Overview running on a Linux AWS instance and our local Overview thingy is accessible only inside our home network here in our office. We have already been able to use it to look at two small document sets and it's very cool. (Honest disclosure: Our AWS instance was set up by people much smarter than I about these things.) 

Our problem is that we really need to be able to upload documents from the command line, because there are just too many files in some of these folders to upload them through the browser -- but we keep running into an error message when we try to do it on the command line. 

I've followed the instructions in the Overview wiki on Github for uploading documents on the command line. We've got Python 3 installed, check. We got overview-upload installed, check. 

My problem is that when we run the command to create an empty document set, we get an error message that talks about the SSL and saying "CERTIFICATE_VERIFY_FAILED." (Note that our Overview thingy is on an "https:" URL, and when you first try to go there, even from within our office, Chrome warns you about the security certificate.)

Please forgive me for my stupidity, but there must be something I'm doing wrong here. What do I need to tell Overview to tell it that it's OK to upload? I'm giving it the Global API Token we generated. I'm giving it the server URL, and the name for the empty document set we'd like created. 

Does anyone have any ideas on what I'm doing wrong? 

Is there someplace in the command for creating the document set that I'm supposed to put in our PEM/PPK key to make this work? 

Is there something in the Overview variables files where I need to change a setting? 

Am I supposed to have some kind of key sitting on the AWS server that I'll have to ask one of our more tech-friendly folks to help me with? 

I would be most grateful if anyone could help. I'm a middle-aged newspaper reporter with a liberal arts degree who finds stories in data but who really only speaks so much Computer and only speaks pidgin Server and even less Networking. 

And if you can help me resolve this, you'll be helping the cause of democracy and will generally be a great American and I will love you forever. 

Thank you,

Jennifer Peebles
normally jennifer...@ajc.com for work purposes

[Adam Hooper]

On Wed, Jan 31, 2018 at 8:28 PM Jennifer Peebles <jpi...@gmail.com> wrote:

My problem is that when we run the command to create an empty document set, we get an error message that talks about the SSL and saying "CERTIFICATE_VERIFY_FAILED." (Note that our Overview thingy is on an "https:" URL, and when you first try to go there, even from within our office, Chrome warns you about the security certificate.)

Hi Jennifer,

Here's the deal: your Overview instance has a bad SSL certificate. That's why Chrome gives a warning. Our upload script doesn't let you work around this the way Chrome does.

For now, you don't need to worry about API tokens, PEMs, PPKs, or anything else. The only way to progress is to fix that certificate.

We can't see your network, so there's not much we can do to help. You may need to enlist a sysadmin-type person in your office. Here's a list of hopeful solutions:

• The SSL certificate might be self-signed -- meaning it's for intra-office use only. If that's the case, your sysadmin(s) probably publishes a "root certificate" file: a file Chrome and overview-upload can use to verify the SSL certificate. If this is the case, and you have access to that file, you should be able to prepend REQUESTS_CA_BUNDLE to your command, like this: REQUESTS_CA_BUNDLE=/path/to/my/root-certificate.crt overview-upload ....
• The SSL certificate might be tied to a different server name. For instance, maybe you're typing https://my-overview.intranet but the server is returning a certificate that's only valid for "192.168.1.100". If that's the case, you may be able to just browse to whatever the server name is -- for instance (in this hopeful hypothetical example), "https://192.168.1.100". Firefox (not Chrome) has a helpful certificate inspection popup you can get by clicking or right-clicking the lock icon when you browse to Overview. Check out the "common name" and "alternate names" for the certificate and make sure you're browsing to one of them. (That same Firefox dialog might have other clues as to why the SSL certificate isn't working.)
• The SSL certificate might be invalid in some other way. Maybe it's expired; maybe it uses an obsolete cipher; maybe it's tied to the wrong server name. The only solution in that case is to convince your system administrator to replace the SSL certificate.

If you can make your web browser reach Overview without a warning, you can make overview-upload work.

Enjoy life,

Adam

--

Adam Hooper

http://adamhooper.com

+1-929-332-3503