Implementation plan : inclusion of OpenSSL

[Emmanuel Charpentier]

Proposal for implementation of OpenSSL inclusion in Sage.

The inclusion of OpenSSL in Sage has been decided after a long and fruitful discussion. Now remains to implement it…

The case of installing Sage without OpenSSL is addressed in a separate thread (to be posted RealSoonNow).

Since the OpenSSL relicensing is still underway, the “obvious” way (make openssl a standard package, and be done with it) canot be done (yet : that will be the end of the story…). The following proposal deals with the transitional period until this relicensing.

The ticket #24107 has been opened on Track. However, it seems more useful to keep the discussion here, dedicating the exchanges on Trac to technical implementation issues.

Licensing clarification

The incompatibility between GPL and OpenSSL Licenses does not seem to amount to much

• R, which is GPL-licensed, depends on an https-enabled SSL (i. e. OpenSSL), without discussing it.
• Wget, a GPL-licensed Gnu utility, depends on OpenSSL, and has added the following language to its license :

"Additional permission under GNU GPL version 3 section 7

If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work."

That (modulo sed -e"s/Free Foftware Foundation/Sage/g", of course) seems to cover all possible recourse against us except possibly their use of GPL version 3 rather than 2. GPL exegetes, there is your opportunity to speak up…

However, in the relevant pieces of documentation (mostly README, I think), we may add the following blurb :

"Sage depends on the availability of the OpenSSL software library (a piece of software that allows secure 
communications between your computer and servers used to download parts of the system) in your system software.
It  is likely that this library is already present on your system. If you are using a pre-compiled packaging of Sage, such 
as the Windows distribution, a Debian, Red Hat or Gentoo package or a Macintosh package, these packages depend on 
the necessary library, that will be automatically installed on your computer from the official source of your software 
package."

In the Installation Guide, the same language could be inserted in the relevant “binary installation” part. In the “compiling from sources” part, ditto, but add :

"However, due to its current licensing terms, Sage can be linked against (i. e. built in order to work using)  this library,
but we cannot host it on our servers.

If this library and its header files are not currently installed, the installation script will offer you the choice of
* downloading OpenSSL sources from the OpenSSL repository, for installation in the Sage tree only, or
* aborting the installation, in order to let you install OpenSSL systemwide
In the first case, OpenSSL will be used only by Sage and its hosted software, such as Python and R. In the second
case, OpenSSL will become available to all your installed programs."

Actual implementation

Binary packages

• Distributions with a reliable dependency system (all Linux and freeBSD major distributions, to the best of my knowledge) : they can simply depend of the distribution’s OpenSSL library package.
• Cygwin : Erik Bray tells that Cygwin’s openssl package is installed by default ; therefor, his binary packaging can ensure that the relevant library is present.
• Mac OS : This seems possible for most supported OS X versions ; I defer to the expertise of our Mac developpers.
• Other Unices : I don’t know…

In short, we need practical advice on how to cope with various versions of Mac OS and non-Linux, non-FreeBSD unices (which are a bit exotic, nowadays…). Advice welcome.

Source tarball

There, things begin to be interesting :

• We can’t (yet) “just” make a “normal” standard package of openssl : that would entail hosting it on the Sage mirrors, which is, in the opinion of some, “illegal” (in which legal context ?).
• Furthermore, the existence of a systemwide openssl makes this inclusion futile.

We can use a procedure that I’m said to be used for some of our standard packages : the toplevel configure script :

1. tests for the existence (and, if necessary, the characteristics and abilities) of a package ;
2. if not found : adds it to the list of packages to be installed ;
3. if found : marks it as installed (with the real version).
   The “normal installation process takes care of :

• if necessary, downloading the necessary source tarball from (one of) the Sage mirror(s) ;
• building and installing the package in the $SAGE_ROOT tree ;
• optionally running its testsuite.

The step 2. cannot be used “raw” in openssl’s case : the normal installation process would fail at the download stage. We can :

• “Blindly” depend on openssl libraries and headers being present systemwide, fail if not present with a (loud) error message for a missing dependency ;
• Download the original, pristine, OpenSSL tarball from OpenSSL site directly to $SAGE_ROOT/upstream, then proceed as for a “normal’ package ;

I propose that we combine those steps in the following manner :

• create a standard package for openssl, but do NOT upload its tarball to the Sage mirrors ;
• replace step 2. above (i. e. what happens when a systemwide OpenSSL isn’t found) with the following :
  • Query the user with the choice :
    • “Download OpenSSL from its original site, then do the installation in the local Sage tree”
    • “Abort this installation, to be restarted after installation of a systemwide OpenSSL (recommended)” ;
    • “Abort, and download a tarball allowing for installation without OpenSSL”.
  • According to the choice of the user :
    • Download from OpenSSL site directly to $SAGE_ROOT/upstream then proceed ;
    • Display a hint for systemwide installation, then abort ;
    • Display the adress of an OpenSSL-less Sage tarball, then abort.

Important corollary

All traces of our current, outdated and probably (pseudo-)illegal openssl package must disappear.

Also, looking for optional packages, I see :

charpent@p-202-021:~$ sage -optional | grep openssl
openssl.................................1.0.2j (not_installed)
pyopenssl...............................17.3.0 (not_installed)

I do not understand why. Is my last installation too old ? The questoin goes to mirrors administrators and possibly present and past release managers.

---

What do you think ? Advice, criticisms, lazzi and even koans welcome.

HTH,

—
Emmanuel Charpentier

​

[Emmanuel Charpentier]

I need an example of a standard package not installed if present systemwide : I know there are some, but can't retrieve it right now...

--
Emmanuel Charpentier

[Dr. David Kirkby (Kirkby Microwave Ltd)]

On 25 October 2017 at 16:38, Emmanuel Charpentier <emanuel.c...@gmail.com> wrote:

Proposal for implementation of OpenSSL inclusion in Sage.

The inclusion of OpenSSL in Sage has been decided after a long and fruitful discussion. Now remains to implement it…

That's a very dubious statement, based on a flawed poll.

Dave

[Erik Bray]

On Wed, Oct 25, 2017 at 5:42 PM, Emmanuel Charpentier
<emanuel.c...@gmail.com> wrote:
> I need an example of a standard package not installed if present systemwide
> : I know there are some, but can't retrieve it right now...

gcc

> --
> You received this message because you are subscribed to the Google Groups
> "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sage-devel+...@googlegroups.com.
> To post to this group, send email to sage-...@googlegroups.com.
> Visit this group at https://groups.google.com/group/sage-devel.
> For more options, visit https://groups.google.com/d/optout.

[Jeroen Demeyer]

On 2017-10-25 17:38, Emmanuel Charpentier wrote:
> The incompatibility between GPL and OpenSSL Licenses does not seem to
> amount to much

This is a very dubious statement. And the rest of your implementation
plan depends on this, so we should really think about this.

For example, a lot depends on whether you consider Sage to be a single
program or an "aggregate" of programs.

If it is a single program, we cannot just change the license: that would
require approval of *all* GPL packages in Sage.

For binary packages, we can get away with the system library exception
to link against a system OpenSSL.

[Emmanuel Charpentier]

Can you explain how the Wget case is different from ours ?

--
Emmanuel Charpentier

[Emmanuel Charpentier]

Le mercredi 25 octobre 2017 18:10:10 UTC+2, Erik Bray a écrit :

On Wed, Oct 25, 2017 at 5:42 PM, Emmanuel Charpentier
<emanuel.c...@gmail.com> wrote:
> I need an example of a standard package not installed if present systemwide
> : I know there are some, but can't retrieve it right now...

gcc

<Slaps forehead> Of course !

Thanks a lot !

--
Emmanuel Charpentier
 

[Michael Orlitzky]

On 10/25/2017 12:14 PM, Emmanuel Charpentier wrote:
> Can you explain how the Wget case is different from ours ?

A single entity (the FSF) owned the copyright on all of the code in wget
when they changed the license to add the exception. The same is not true
of SageMath: we don't have copyright assignment, and we bundle a lot of
code that is copyrighted by complete strangers.

[Emmanuel Charpentier]

I see...

I'll try to think of an alternative, if that turns out to be an unpassable obstacle.

--
Emmanuel Charpentier

[Emmanuel Charpentier]

Le mercredi 25 octobre 2017 21:37:59 UTC+2, Michael Orlitzky a écrit :

Okay : I'm not still convinced, but I'll follow you on a "primum non nocere" basis.

If we follow you, we have to take back any talk of *inclusion* right now. The solution is to depend *unconditionally* on a systemwide OpenSSL and stop building if not found, possibly hinting at steps to that effect.

*After* OpenSSL relicensing, we can introduce a standard openssl package, whith no huffing and puffing.

Now, all that I miss is a hint to give to people wanting to build sage *withut* OpenSSL. Since we get rid oif the current anti-OpenSSL patches, We ncan't let these users hang dry...

--
Emmanuel Charpentier

--
Emmanuel Charpentier

[Michael Orlitzky]

On 10/27/2017 10:53 AM, Emmanuel Charpentier wrote:
>
> Okay : I'm not still convinced, but I'll follow you on a "primum non
> nocere" basis.
>
> If we follow you, we have to take back any talk of *inclusion* right
> now. The solution is to depend *unconditionally* on a systemwide OpenSSL
> and stop building if not found, possibly hinting at steps to that effect.

That's what I would recommend. Otherwise, anyone distributing the
SageMath code or binaries is putting themselves at risk. Or more likely,
we're putting them at risk -- because I doubt many mirror operators are
following this discussion.

There are two separate issues with bundling:

a) distributing SageMath binaries linked against a non-system copy of
OpenSSL

b) distributing the source for "SageMath the distribution," which
sort-of includes a copy of the OpenSSL code

Linking creates a derivative work (most people agree on this), so the
resulting binaries in (a) cannot be distributed: doing so would violate
the GPL, because anyone receiving a copy would be subject to the
additional restrictions from the OpenSSL license.

The OpenSSL source code is no longer in our git repo; we do however ship
an SPKG that automatically downloads and patches the OpenSSL source to
integrate it with the rest of SageMath. Whether or not that creates a
non-distributable derivative work, I don't know -- but I would err on
the side of caution.

And don't forget that end users are free to do whatever they want. If we
tell them to download the OpenSSL tarball, patch it, and to put the
result somewhere in $SAGE_ROOT before running "make", then neither (a)
nor (b) is happening. That's perfectly legal, even though it makes the
argument in the previous paragraph look stupid (I went into math and not
law for a reason). The only "gotcha" there is that end users would not
be able to redistribute the resulting SageMath binaries, for the same
reason that we can't

> *After* OpenSSL relicensing, we can introduce a standard openssl
> package, whith no huffing and puffing.

Right, but I'm sceptical that it will happen. If it does, how long are
we willing to wait?

Requiring a system copy is the best solution, if for no other reason,
then because nobody will have to read any more long emails about it.