py-certbot still with openssl errors after upgrade

[johns.tablet20]

Hi,

context:

root@v007:/root# freebsd-version -ku
11.2-STABLE-HBSD
11.2--HBSD

root@v007:/root# sysctl hardening.version
hardening.version: 1100056

I upgraded like this:

1. pkg delete -af
2. hbsd-update
3. reboot
4. pkg bootstrap
5. pkg-static clean -y
6. pkg-static upgrade -f

Then I went to install apache and py-certbot (the py27 flavour):

7. pkg install apache24-2.4.33_1
8. pkg install py27-certbot-0.25.1,1

Then I try to run certbot, I get the error which looks like an openssl
error, but I don't know. Should I take this up with the port maintainer?

root@v007:/root# certbot renew
Traceback (most recent call last):
File "/usr/local/bin/certbot", line 11, in <module>
load_entry_point('certbot==0.25.1', 'console_scripts', 'certbot')()
File
"/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line
476, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File
"/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line
2700, in load_entry_point
return ep.load()
File
"/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line
2318, in load
return self.resolve()
File
"/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line
2324, in resolve
module = __import__(self.module_name, fromlist=['__name__'], level=0)
File "/usr/local/lib/python2.7/site-packages/certbot/main.py", line
10, in <module>
import josepy as jose
File "/usr/local/lib/python2.7/site-packages/josepy/__init__.py",
line 44, in <module>
from josepy.interfaces import JSONDeSerializable
File "/usr/local/lib/python2.7/site-packages/josepy/interfaces.py",
line 8, in <module>
from josepy import errors, util
File "/usr/local/lib/python2.7/site-packages/josepy/util.py", line 4,
in <module>
import OpenSSL
File "/usr/local/lib/python2.7/site-packages/OpenSSL/__init__.py",
line 8, in <module>
from OpenSSL import crypto, SSL
File "/usr/local/lib/python2.7/site-packages/OpenSSL/crypto.py", line
16, in <module>
from OpenSSL._util import (
File "/usr/local/lib/python2.7/site-packages/OpenSSL/_util.py", line
6, in <module>
from cryptography.hazmat.bindings.openssl.binding import Binding
File
"/usr/local/lib/python2.7/site-packages/cryptography/hazmat/bindings/openssl/binding.py",
line 13, in <module>
from cryptography.hazmat.bindings._openssl import ffi, lib
ImportError:
/usr/local/lib/python2.7/site-packages/cryptography/hazmat/bindings/_openssl.so:
Undefined symbol "X509_up_ref"

thanks,
--
johns.t...@gmail.com

[Shawn Webb]

What's the output of:

/usr/bin/openssl version

Thanks,

--
Shawn Webb
Cofounder and Security Engineer
HardenedBSD

Tor-ified Signal: +1 443-546-8752
Tor+XMPP+OTR: lat...@is.a.hacker.sx
GPG Key ID: 0x6A84658F52456EEE
GPG Key Fingerprint: 2ABA B6BD EF6A F486 BE89 3D9E 6A84 658F 5245 6EEE

[johns.tablet20]

On 05/07/2018 14:49, Shawn Webb wrote:
> What's the output of:
>
> /usr/bin/openssl version

john@v007:~ % /usr/bin/openssl version
OpenSSL 1.0.2o-freebsd 27 Mar 2018
john@v007:~ %

--
johns.t...@gmail.com

[johns.tablet20]

On 05/07/2018 15:35, johns.tablet20 wrote:
> On 05/07/2018 14:49, Shawn Webb wrote:
>> What's the output of:
>>
>> /usr/bin/openssl version
>
> john@v007:~ % /usr/bin/openssl version
> OpenSSL 1.0.2o-freebsd  27 Mar 2018
> john@v007:~ %

any thoughts?

What I'd like to do is to try spinning up a 12-current amd64 image, then
just installing pkg and then py-certbot to see if I get the same result,
but I can't seem to find an installer for 12-current.
--
johns.t...@gmail.com

[Shawn Webb]

On Fri, Jul 06, 2018 at 12:22:16PM +0100, johns.tablet20 wrote:
> On 05/07/2018 15:35, johns.tablet20 wrote:
> > On 05/07/2018 14:49, Shawn Webb wrote:
> > > What's the output of:
> > >
> > > /usr/bin/openssl version
> >
> > john@v007:~ % /usr/bin/openssl version

> > OpenSSL 1.0.2o-freebsd? 27 Mar 2018

> > john@v007:~ %
>
> any thoughts?

I'd probably ping py-certbot. Now that HardenedBSD 11-STABLE is back
to using OpenSSL, it ought to work.

The thing I wonder about, though, is if the pre-built openssl module
that pip (yuck!) installed is linking against the OpenSSL in FreeBSD
10.*-RELEASE, which is not the same as the OpenSSL in FreeBSD 11.2-RELEASE.

So, this is still a problem of Python's own stupidness. The real fix
would be for Python developers to throw pip in the trash where it
belongs and rely on the package management systems of the operating
systems.

>
> What I'd like to do is to try spinning up a 12-current amd64 image, then
> just installing pkg and then py-certbot to see if I get the same result, but
> I can't seem to find an installer for 12-current.

The latest 12-CURRENT/amd64 build is
here:http://jenkins.hardenedbsd.org/builds/HardenedBSD-CURRENT-amd64-LATEST/

Remember, though, that we're still in the LibreSSL->OpenSSL transition
period for 12-CURRENT. Packages are still being rebuilt. So you'll
need to install from ports yourself.

[johns.tablet20]

On 06/07/2018 16:54, Shawn Webb wrote:
> The thing I wonder about, though, is if the pre-built openssl module
> that pip (yuck!) installed is linking against the OpenSSL in FreeBSD
> 10.*-RELEASE, which is not the same as the OpenSSL in FreeBSD 11.2-RELEASE.

Would this still be the case if py-certbot was built from the port ?

thanks,
--
johns.t...@gmail.com

[Shawn Webb]

Yup. py-certbot instructs python to download remote code, install, and
run it. This includes a custom _openssl.so by the looks of it.

[johns.tablet20]

On 06/07/2018 17:05, Shawn Webb wrote:
> Yup. py-certbot instructs python to download remote code, install, and
> run it. This includes a custom _openssl.so by the looks of it.

OK. Would you expect it therefore to *not* work on FreeBSD
11.2-PRERELEASE ? Because it does.

This 11.2-PRERELEASE is a VM. I can deinstall all ports then upgrade
this to latest 11.2-stable, then try py-certbot again if you're interested.

--
johns.t...@gmail.com

[Shawn Webb]

I'm not sure what to tell you, then. Since I'm not a Python developer,
I'm going to need help from the community to resolve this issue.

[johns.tablet20]

On 06/07/2018 18:06, Shawn Webb wrote:
> I'm not sure what to tell you, then. Since I'm not a Python developer,
> I'm going to need help from the community to resolve this issue.

Fair enough.

In the meantime I'm trying to use the hbsd-12 memstick installer to
install a hbsd-12 guest vm, but I can't as I get dropped to the
mountroot prompt error 19. I guess this is because bhyve expects an iso
rather than an image. I usually install a VM like this:

sh /usr/share/examples/bhyve/vmrun.sh -c 2 -m 4096M -t tap0 -d
fbsd12c.img -i -I freebsd-dvd1.iso guestname

(obviously replacing the dvd1 in this case with the hardenedbsd-12.img)

Do you know of a way to either convert the memstick to disk1.iso or to
tell bhyve to use the memstick as install media?

If I manage to install a hbsd-12 VM, I'll try the py-certbot thing again

thanks,
--
johns.t...@gmail.com

[Shawn Webb]

You can use the -d option with the memstick image. There's currently a
bug with how we're building the 12-CURRENT installation media. It's
skipping the iso files. I just wish there were more than 24 hours in a
day and perhaps also more than 7 days in a week.

[johns.tablet20]

On 06/07/2018 18:06, Shawn Webb wrote:

> I'm not sure what to tell you, then. Since I'm not a Python developer,
> I'm going to need help from the community to resolve this issue.

I think it's an 11-stable hardenedbsd issue, this particular thing.

I managed to spin up a vm made from the 12-installer you linked earlier.
py27-certbot works, even though openssl version reports LibreSSL and
py27-certbot was installed from pkg, and pkg was installed via pkg
bootstrap. So it doesn't seem to be a Libre/openssl issue.

I've noticed other recent-ish 11-stable-only oddness with bhyve, in that
when the VM spins up, you won't get console output. But you do get it
with 12-current. It doesn't matter if the host system runs (FreeBSD)
-stable or -current. I'm not sure how recently this started happening,
but I think it's over a year. It happens with both Free and HardenedBSD.

While investigating this, I tried spinning up
HardenedBSD-11-STABLE-v46.2-amd64-disc1.iso. This one produces the
expected output. I haven't checked to see whether it happens in
10-stable yet.

--
johns.t...@gmail.com

[johns.tablet20]

Hi,

I have a 12-current hbsd system (amd64) brand new install just for
testing purposes. I installed py27-certbot (via pkg) and it worked fine.

I have a 11-stable hbsd system (amd64), the first problematic system. It
(certbot) didn't work, so I grabbed 12 hbsd sources and installed those,
rebooted, did make delete-old and delete-old-libs, rebooted, ensured I
was using openssl, I was. Deleted all ports. Grabbed a new ports tree.
Installed py-certbot (it installed py27-certbot) from the ports.
Installs fine, same error. Not tried with just pkg.

I installed the latest hbsd arm64 for rpi3. There are no packages for
this as you've mentioned. Installed the ports tree. Installed
py-certbot. Installed fine. Same error when it runs.

Here is the error:

root@rpi3:~ # certbot certonly -d rpi3.[REDACTED], www.[REDACTED],
[REDACTED]

Next step is to install freebsd/rpi3, then try the same as above.

--
johns.t...@gmail.com

[Shawn Webb]

Looping Bernard in. Hopefully he has some non-xkcd[1] pointers.

[1]: https://www.xkcd.com/138/

[johns.tablet20]

On 10/07/2018 02:47, Shawn Webb wrote:
> Looping Bernard in. Hopefully he has some non-xkcd[1] pointers.
>
> [1]:https://www.xkcd.com/138/

;)
--
johns.t...@gmail.com