Bug#844018: libcurl3: Building with OpenSSL 1.1 breaks packages using both OpenSSL 1.0 and curl

[Adrian Bunk]

Control: tags -1 patch

On Fri, Nov 11, 2016 at 09:30:07PM +0100, Jan Niehusmann wrote:
> Package: libcurl3
> Version: 7.51.0-1
> Severity: serious
> Justification: Policy 8.1
>
> Dear Maintainer,
>
> the curl ABI contains structs inherited from OpenSSL, e.g. in calls
> like:
>
> curl_easy_setopt(easy, CURLOPT_SSL_CTX_FUNCTION, &sslCtxFunction_cb);
>
> Here, sslCtxFunction_cb is a function which takes an SSL_CTX * as a
> parameter.
>
> (This is from zurl, one example of a package affected by this bug.)
>
> Since 7.51.0-1, curl links against OpenSSL 1.1 instead of OpenSSL 1.0
> (implicitly caused by an update of libssl-dev, not by a change to the
> curl package). This changes the structure of SSL_CTX, which in turn
> changes the above mentioned ABI and breaks zurl (and possibly other
> packages).
>
> Such ABI changes require a SONAME change, according to policy 8.1,
> exactly to avoid breaking other packages which use the library.
>
> Therefore, please consider changing the SONAME (and the name of the
> binary package).

For 3rd party software doing anything Debian-specific here would be bad.

Debian should really follow whatever upstream and other distributions
are doing when they switch to 1.1 (hopefully a new soname from upstream).

For stretch it would also not solve the problem that there is
libcurl-using software that is using OpenSSL 1.0.2

> Alternatively, build-depend on libssl1.0-dev, to link
> against OpenSSL 1.0 and keep the old ABI.
>...

Unfortunately this is not sufficient.

libcurl4-openssl-dev also has to depend on libssl1.0-dev,
otherwise you will have the same problem the other way around.

I know it is not pretty, but the following seems to be required:

--- debian/control.old 2016-11-20 08:19:55.000000000 +0000
+++ debian/control 2016-11-20 08:20:32.000000000 +0000
@@ -16,7 +16,7 @@
libnss3-dev,
librtmp-dev (>= 2.4+20131018.git79459a2-3~),
libssh2-1-dev,
- libssl-dev,
+ libssl1.0-dev,
libtool,
openssh-server <!nocheck>,
python:native,
@@ -121,14 +121,13 @@
libcurl4-dev,
libcurl3-dev
Conflicts: libcurl4-gnutls-dev, libcurl4-nss-dev
-Depends: ${misc:Depends}, libcurl3 (= ${binary:Version})
+Depends: ${misc:Depends}, libcurl3 (= ${binary:Version}), libssl1.0-dev
Suggests: libcurl4-doc, libcurl3-dbg,
libidn11-dev,
libkrb5-dev,
libldap2-dev,
librtmp-dev,
libssh2-1-dev,
- libssl-dev,
pkg-config,
zlib1g-dev
Multi-Arch: same


cu
Adrian

--

"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon Seed

[Stepan Golosunov]

11.11.2016 в 21:30:07 +0100 Jan Niehusmann написал:

> the curl ABI contains structs inherited from OpenSSL, e.g. in calls
> like:
>
> curl_easy_setopt(easy, CURLOPT_SSL_CTX_FUNCTION, &sslCtxFunction_cb);
>
> Here, sslCtxFunction_cb is a function which takes an SSL_CTX * as a
> parameter.
>
> (This is from zurl, one example of a package affected by this bug.)
>
> Since 7.51.0-1, curl links against OpenSSL 1.1 instead of OpenSSL 1.0
> (implicitly caused by an update of libssl-dev, not by a change to the
> curl package). This changes the structure of SSL_CTX, which in turn
> changes the above mentioned ABI and breaks zurl (and possibly other
> packages).

And libcurl3 in testing links with libssl1.0.2 while libcurl3 in
jessie links with libssl1.0.0. And while structure of SSL_CTX is
probably identical in this case, global state of libssl is different,
as well as code and supported features are different.

Also, when libcurl is linked with OpenSSL older then 1.1 many
applications need to be linked with the same OpenSSL libraries as curl
as they need to initialize libssl as described in
https://lists.debian.org/debian-devel/2016/11/msg00718.html
With OpenSSL 1.1 this requirement goes away, as newer OpenSSL
initializes itself automatically (and if application initializes old
version of OpenSSL no harm is done).

> Such ABI changes require a SONAME change, according to policy 8.1,
> exactly to avoid breaking other packages which use the library.
>
> Therefore, please consider changing the SONAME (and the name of the

> binary package). Alternatively, build-depend on libssl1.0-dev, to link

> against OpenSSL 1.0 and keep the old ABI.

If SONAME change is needed it is needed when linking libcurl with
libssl1.0.2 too. (But when linking libcurl with libssl1.0.2 more
applications are affected due to the need of explicit initialization.)

[Jan Niehusmann]

On Sun, Nov 20, 2016 at 07:40:22PM +0400, Stepan Golosunov wrote:
> If SONAME change is needed it is needed when linking libcurl with
> libssl1.0.2 too. (But when linking libcurl with libssl1.0.2 more
> applications are affected due to the need of explicit initialization.)

Upstream documentation suggests that this is not true:

"Minor releases that change the last digit, e.g. 1.0.1 vs. 1.0.2, can
and are likely to contain new features, but in a way that does not break
binary compatibility. This means that an application compiled and
dynamically linked with 1.0.0 does not need to be recompiled when the
shared library is updated to 1.0.2."
(https://www.openssl.org/policies/releasestrat.html)

But of course, in practice, there may have been unintended ABI changes
- so yes, may be a SONAME change is nessary for 1.0.2 as well. I just
don't know.

Still, for 1.1 the SONAME change is definitely necessary.

Jan

[Stepan Golosunov]

20.11.2016 в 20:48:44 +0100 Jan Niehusmann написал:

> On Sun, Nov 20, 2016 at 11:38:49PM +0400, Stepan Golosunov wrote:
> > > Still, for 1.1 the SONAME change is definitely necessary.
> >

> > Breaks: against applications using CURLOPT_SSL_CTX_FUNCTION and
> > compiled with OpenSSL 1.0 will probably suffice in that case.
>
> Wouldn't this still be a serious bug as it would - explicitly - break
> those applications?

It would be a grave bug in such application if it does not have a
working version, yes. Whether or not it would be a serious bug in
lubcurl3 depends on how many and how important such applications are.
(And how difficult they are to fix.)

[Stepan Golosunov]

20.11.2016 в 23:38:48 +0400 Stepan Golosunov написал:
> 20.11.2016 в 20:01:25 +0100 Jan Niehusmann написал:

> > Still, for 1.1 the SONAME change is definitely necessary.
>

> Breaks: against applications using CURLOPT_SSL_CTX_FUNCTION and
> compiled with OpenSSL 1.0 will probably suffice in that case.

No, that won't be sufficient. One must also ensure that applications
compiled with libssl1.1-using libcurl3 also depend on such
libcurl3 (both for the sake of applications using
CURLOPT_SSL_CTX_FUNCTION and applications dropping OpenSSL
initialization).

[Jan Niehusmann]

On Mon, Nov 21, 2016 at 12:02:45AM +0400, Stepan Golosunov wrote:
> It would be a grave bug in such application if it does not have a
> working version, yes. Whether or not it would be a serious bug in
> lubcurl3 depends on how many and how important such applications are.
> (And how difficult they are to fix.)

As there are applications which depend on libcurl and qt, and it seems
like qt can't be ported to openssl 1.1 in time for stretch, fixing such
applications would mean uploading a version of curl linked to openssl
1.0. This could be a new package, or a statically linked version etc. -
IMHO all worse than just linking the curl package with openssl 1.0 until
(close to) all packages are ready for a transition to openssl 1.1.

Jan

[Stepan Golosunov]

20.11.2016 в 21:29:35 +0100 Jan Niehusmann написал:

So far I do not know why using libssl1.1 together with a
libssl1.0.2-using Qt wouldn't work. So far I can imagine the
following reasons:

1. Application relies on Qt initializing the library. That was the
cause of breakage during libssl1.0.0 -> libssl1.0.2 transition, but
is not applicable now as libssl1.1 does not require explicit
initialization.

2. Application passes OpenSSL objects from libssl1.1 to Qt (or vice
versa). Why one would do it?

3. dlsym(3) is used with RTLD_DEFAULT or RTLD_NEXT. Does not seem to
be the case.

4. dlsym(3) finds wrong library. Manual page suggests that should not
be the case.

…

Is any of the above applicable?

[Jan Niehusmann]

On Mon, Nov 21, 2016 at 01:03:19AM +0400, Stepan Golosunov wrote:
> So far I do not know why using libssl1.1 together with a
> libssl1.0.2-using Qt wouldn't work.

Well I don't know enough about the dynamic linker and about the internals
of openssl to know if (indirectly) linking to both libraries at the same
time is fine.

If it was, that would be great news. Many mails in the thread "OpenSSL
1.1.0" on debian-devel seem to be based on the assumption that such
linking could cause bugs, and therefore packages can only transition in
clusters of packages linking to the same version of openssl.

Still, qt is only an example - the same holds true for other libraries
linking to openssl1.0-dev. There may be cases where your 2nd case
('Application passes OpenSSL objects from libssl1.1 to ...') is more
probable than with qt.

The safest way to avoid hidden bugs would still be changing SONAME and
package name, so package maintainers would be aware of the change and
could check their packages for compatibility.

But yes, it may be less work to somehow identify affected packages and
handle them directly instead of forcing all packages depending on curl
through a transition. Identifying those packages in a reliable way could
be difficult, though.

Jan

[Stepan Golosunov]

22.11.2016 в 22:57:30 +0200 Adrian Bunk написал:
> 23:14 < bunk> Q_: If you come up with a better patch than mine in #844018,
> please post to that bug. I know that my patch is not pretty, but
> I did not find any better short-term solution.

Search for CURLOPT_SSL_CTX_FUNCTION on codesearch.debian.net produces
the following list of potentially affected packages:
cargo
chromium-browser
cmake
criticalmass
curl
curlpp
firefox
firefox-esr
fpc
hhvm
icedove
lastpass-cli
libapache2-mod-auth-cas
libwww-curl-perl
lua-curl
netcdf
netsurf
openjfx
r-cran-curl
r-cran-rcurl
ruby-curb
slcurl
sx
tclcurl
wpa
xmltooling
zurl

So the the alternative to you patch looks like:

fixing #828564 (fixed-upstream, new upstream version available),
fixing #828608 or removing xmltooling out of testing,
checking whether last apache2 upload fixed #844799,
fixing or ignoring #828259 (not in testing, fixed upstream version
available),
fixing #828371 (untested patch available) or removing lastpass-cli,
removing 3 characters from zurl's debian/control,
binnmu-ing affected packages that still depend on libssl1.0.2,
no need to ensure that applications (even if they are linked with
libcurl3 indirectly) are linked with the same libssl as libcurl3
(unlike with the patch),
more compatibility with applications from jessie than with the patch.