VOTE: inclusion of OpenSSL in Sage

447 views
Skip to first unread message

Emmanuel Charpentier

unread,
Oct 16, 2017, 5:47:09 AM10/16/17
to sage-devel
Following numerous discussions on this list an various tickets, the issue of maintaining Sage-specific patches to various components of Sage emerged again about the proposed upgrade of R to 3.4.2 (discussed here).

Since Trac#22189, installation of a systemwide opennssl is recommended (may be too strongly, in the taste of some respectable Sage developers...).

The proposed inclusion would entail :
Deprecation of our OpenSSL-avidance patches
Sta
At compilation, research of a systemwide OpenSSL
If found : do nothing

Emmanuel Charpentier

unread,
Oct 16, 2017, 6:08:52 AM10/16/17
to sage-devel
[ The first post started too fast... Sorry for the interruption ! ]

Following numerous discussions on this list and various Trac tickets*, the issue of maintaining Sage-specific patches to various components of Sage emerged again about the proposed upgrade of R to 3.4.2 (discussed here). William again raises the issue of security.

Since Trac#22189, installation of a systemwide opennssl is recommended (may be too strongly, in the taste of some respectable Sage developers...). The ongoing relicensing of OpenSSL should lift the last barriers to its inclusion in sage. A discussed here,, the probability of a legal problem related to the incusion of this library in Sage seems infinitesimal.

It has beeen furthermore suggested to add to our licensing (an adaptatin of) the following language, used in Gnu Wget License (GPL) :

"Additional permission under GNU GPL version 3 section 7

If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work."


The proposed inclusion would entail :
  • Deprecation of our OpenSSL-avidance patches
  • Standardization of SSL communications on OpenSSL
      • At compilation, research of a systemwide OpenSSL
      • If found : do nothing
      • In not found : installation of OpenSSL in the Sage tree from a Sage-specific repository (as for most of our standard and optional packages...).
    • Licensing clarification
    In short, we have two options : include OpenSSL now (using language clarification), or wait for the complete OpenSSL relicensing. The exact terms of the vote are therefore :

    |_| Yes, we should fully support OpenSSL now, and clarify the licensing issue.

    |_| No, we should wait until OpenSSL finishes fixing their license situation formally.

    The vote will take place as answers to this post, and will be open until Monday October 23, 14h UTC.

    Sincerely yours,

                                                                            Emmanuel Charpentier

    * Perusing the results of searching Trac and sage-devel Google group is enlightening...
    --
    Emmanuel Charpentier

    Erik Bray

    unread,
    Oct 16, 2017, 10:58:07 AM10/16/17
    to sage-devel
    On Mon, Oct 16, 2017 at 12:08 PM, Emmanuel Charpentier
    <emanuel.c...@gmail.com> wrote:
    > In short, we have two options : include OpenSSL now (using language
    > clarification), or wait for the complete OpenSSL relicensing. The exact
    > terms of the vote are therefore :
    >
    > |X| Yes, we should fully support OpenSSL now, and clarify the licensing
    > issue.
    >
    > |_| No, we should wait until OpenSSL finishes fixing their license situation
    > formally.

    Yes, but it's terrible in the first place that we have to include
    OpenSSL "in Sage" at all. As long as the system library is used by
    default though then I don't mind.

    Dima Pasechnik

    unread,
    Oct 16, 2017, 11:22:16 AM10/16/17
    to sage-devel


    On Monday, October 16, 2017 at 11:08:52 AM UTC+1, Emmanuel Charpentier wrote:
    [ The first post started too fast... Sorry for the interruption ! ]

    Following numerous discussions on this list and various Trac tickets*, the issue of maintaining Sage-specific patches to various components of Sage emerged again about the proposed upgrade of R to 3.4.2 (discussed here). William again raises the issue of security.

    Since Trac#22189, installation of a systemwide opennssl is recommended (may be too strongly, in the taste of some respectable Sage developers...). The ongoing relicensing of OpenSSL should lift the last barriers to its inclusion in sage. A discussed here,, the probability of a legal problem related to the incusion of this library in Sage seems infinitesimal.

    It has beeen furthermore suggested to add to our licensing (an adaptatin of) the following language, used in Gnu Wget License (GPL) :

    "Additional permission under GNU GPL version 3 section 7

    If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work."


    The proposed inclusion would entail :
    • Deprecation of our OpenSSL-avidance patches
    • Standardization of SSL communications on OpenSSL
      • At compilation, research of a systemwide OpenSSL
      • If found : do nothing
      • In not found : installation of OpenSSL in the Sage tree from a Sage-specific repository (as for most of our standard and optional packages...).
    • Licensing clarification
    In short, we have two options : include OpenSSL now (using language clarification), or wait for the complete OpenSSL relicensing. The exact terms of the vote are therefore :

    |X| Yes, we should fully support OpenSSL now, and clarify the licensing issue.

    |_| No, we should wait until OpenSSL finishes fixing their license situation formally.

    William Stein

    unread,
    Oct 16, 2017, 11:24:49 AM10/16/17
    to sage-...@googlegroups.com
    |X| Yes, we should fully support OpenSSL now, and clarify the licensing issue.
    --
    You received this message because you are subscribed to the Google Groups "sage-devel" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
    To post to this group, send email to sage-...@googlegroups.com.
    Visit this group at https://groups.google.com/group/sage-devel.
    For more options, visit https://groups.google.com/d/optout.
    --
    -- William Stein

    John Cremona

    unread,
    Oct 16, 2017, 12:06:42 PM10/16/17
    to SAGE devel

    David Roe

    unread,
    Oct 16, 2017, 1:08:25 PM10/16/17
    to sage-devel

    Jeroen Demeyer

    unread,
    Oct 16, 2017, 2:07:59 PM10/16/17
    to sage-...@googlegroups.com
    On 2017-10-16 12:08, Emmanuel Charpentier wrote:
    > |_| Yes, we should fully support OpenSSL now, and clarify the
    > licensing issue.

    What does "clarifying" the licensing issue even mean? The fact that
    OpenSSL is *in the process of* relicensing does not help us at the
    moment. And you don't need a mailing list vote to change the Sage
    license: you need approval from every author of every GPL package in Sage.

    William Stein

    unread,
    Oct 16, 2017, 3:44:57 PM10/16/17
    to sage-...@googlegroups.com
    To me, "clarify the license situations means":

    1. At a mimum:  make it crytstal clear in our LICENSE/README file and binaries download page that we are distribution OpenSSL (or at least something that depends on OpenSSL), and that -- depending on interpretations of the system license exception -- this may violate the GPL.  I think making this situation *clear* is absolutely essential, rather than say just "sneaking" openssl into Sage.

    2. Also: explain that the risk is minimal, since it is the intention of the OpenSSL authors to relicense, and several of us significant copyright holders (e.g., me) can at least make a clear statement that **WE** are not going to complain or sue anybody for combining Sage with OpenSSL.

    Until OpenSSL is properly relicensed there is a small but real risk of some problem arising from this copyright situation.    That has to be balanced with the very real risk that shipping a crippled security stack directly results in users of Sage having their computers and personal information compromised.    

    A legally safer approach would be to never include openssl in sage, but instead make a system-wide install of openssl a hard requirement for building or installing sage.  We then still link to openssl.   The build fails if it libopenssl-dev (or whatever) is not available.  A binary install doesn't work (in some cases?) if it isn't available.    Maybe this should be a third option for the vote?   It seems like what Eric wanted...

    William



    --
    You received this message because you are subscribed to the Google Groups "sage-devel" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
    To post to this group, send email to sage-...@googlegroups.com.
    Visit this group at https://groups.google.com/group/sage-devel.
    For more options, visit https://groups.google.com/d/optout.
    --
    -- William Stein

    Dima Pasechnik

    unread,
    Oct 16, 2017, 6:40:31 PM10/16/17
    to sage-devel

    Nathan Dunfield

    unread,
    Oct 17, 2017, 12:31:02 AM10/17/17
    to sage-devel
    |X| Yes, we should fully support OpenSSL now, and clarify the licensing issue.  


    Jeroen Demeyer

    unread,
    Oct 17, 2017, 2:55:28 AM10/17/17
    to sage-...@googlegroups.com
    So basically you want to add OpenSSL to Sage and then say

    "We know that distributing SageMath might be illegal, but it is unlikely
    that somebody will sue. Use at your own risk!"

    I doubt that this is such a good deal.

    David Joyner

    unread,
    Oct 17, 2017, 7:21:52 AM10/17/17
    to sage-devel
    Has anyone emailed them and just tell them the plan is for SageMath
    in November 2017 to include OpenSSL and include a license clarification
    of some type, and ask the OpenSSL people for their reaction?

    According to https://www.openssl.org/community/contacts.html the email
    for licensing issues is in...@opensslfoundation.org, but they have a
    developers' email list at
    https://www.openssl.org/community/mailinglists.html.


    > The vote will take place as answers to this post, and will be open until
    > Monday October 23, 14h UTC.
    >
    > Sincerely yours,
    >
    >
    > Emmanuel Charpentier
    >
    > * Perusing the results of searching Trac and sage-devel Google group is
    > enlightening...
    > --
    > Emmanuel Charpentier
    >

    Emmanuel Charpentier

    unread,
    Oct 17, 2017, 9:09:47 AM10/17/17
    to sage-devel


    Le mardi 17 octobre 2017 13:21:52 UTC+2, David Joyner a écrit :

    [ Snip... ]


    Has anyone emailed them and just tell them the plan is for SageMath
    in November 2017 to include OpenSSL and include a license clarification
    of some type, and ask the OpenSSL people for their reaction?

    Not yet, as far as I know. That would be an interesting first step in implementation  of the inclusion if the current vote ends up in favor of immediate inclusion. In the other case, the point would be moot, since MPL *is* compatible with GPL...

    Emmanuel Charpentier

    unread,
    Oct 17, 2017, 9:34:20 AM10/17/17
    to sage-devel
    We may combine these approaches  by, sequentially,
    1. Depend on a systemwide OpenSSL :
      1. update README, the Installation Guide and possibly the Developer's Guide ;
      2. test for OpenSSL in the configure script, failing if not installed ;
      3. (possibly) fail early at Sage startup if the runtime isn't found (necessary for binary installations, such as Erik's Cygwin 64 port, that can't reliably depend on other packages. Binary packages such as Debian's or Ubuntu's may depend on the relevant runtime library packages) ;
      4. find and remove OpenSSL-specific patches.
    2. (After OpenSSL relicensing) include OpenSSL :
      1. update README, Installation guide and Developer's guide again ;
      2. test for OpenSSL in the configure script, installing it in the Sage tree if necessary.
    That could be discussed after the vote if its result is in favor of immediate inclusion. In the other case, 1.4+2. is a possible approach.

    Further discussion should probably wait for the vote's result...

    HTH,

    --
    Emmanuel Charpentier

    Michael Orlitzky

    unread,
    Oct 17, 2017, 3:49:50 PM10/17/17
    to sage-...@googlegroups.com
    Not to mention that the addition of terms under Section 7 of GPLv3
    amounts to a license change ("shall be treated as though they were
    included in this License"), and possibly requires the consent of all
    SageMath contributors.

    Which, by the way, is why OpenSSL has not actually relicensed. Some of
    the contributors said "no," and that's the last I heard about it.

    Throwing my vote away:

    [X] Require OpenSSL to be installed on the system.

    Emmanuel Charpentier

    unread,
    Oct 17, 2017, 3:56:20 PM10/17/17
    to sage-devel
    Dear Michael,

    That's not one of the proposed options. But it seems to imply that we should wait for OpenSSL relicensing. If that's what you think should happen, could you explicitly vote in that direction ?

    Richard_L

    unread,
    Oct 17, 2017, 3:59:06 PM10/17/17
    to sage-devel

    [X] Require OpenSSL to be installed on the system.
    This sidesteps the whole license issue and, since more bugs are sure to be found, we do not have to keep yet another package up to date.

    Michael Orlitzky

    unread,
    Oct 17, 2017, 4:46:52 PM10/17/17
    to sage-...@googlegroups.com
    On 10/17/2017 03:56 PM, Emmanuel Charpentier wrote:
    >
    > Throwing my vote away:
    >
    > [X] Require OpenSSL to be installed on the system.
    >
    >
    > That's not one of the proposed options.

    That's what I meant by "throwing my vote away" =)


    > But it seems to imply that we should wait for OpenSSL relicensing.

    We don't have to wait if we require it to be installed on the system.

    "Wait for OpenSSL to relicense" is a bad option because they will
    probably never relicense -- even if they ultimately change their LICENSE
    file, the process by which they've done it is extremely dubious, and
    several copyright holders are on record withholding permission to do so.

    "Include it anyway" is a bad option because we've just announced to the
    world that we're going to commit (willful) copyright infringement.

    Nicolas M. Thiery

    unread,
    Oct 17, 2017, 5:52:47 PM10/17/17
    to sage-...@googlegroups.com

    I have no strong opinion on whether to make OpenSSL a hard requirement
    or providing it if it's not there. But *not* having OpenSSL is a
    recurrent pain (e.g. for pip installing package) and it would be
    really helpful to be able to rely on it.

    If we make it a hard requirement, how to install OpenSSL on the
    classical systems should be well documented. I spent too many hours
    helping people on mac/... that were missing it.

    Cheers,
    Nicolas
    --
    Nicolas M. Thiéry "Isil" <nth...@users.sf.net>
    http://Nicolas.Thiery.name/

    Dima Pasechnik

    unread,
    Oct 17, 2017, 6:56:03 PM10/17/17
    to sage-devel


    On Tuesday, October 17, 2017 at 10:52:47 PM UTC+1, Nicolas M. Thiéry wrote:

    I have no strong opinion on whether to make OpenSSL a hard requirement
    or providing it if it's not there. But *not* having OpenSSL is a
    recurrent pain (e.g. for pip installing package) and it would be
    really helpful to be able to rely on it.

    If we make it a hard requirement, how to install OpenSSL on the
    classical systems should be well documented. I spent too many hours
    helping people on mac/... that were missing it.

    Yeah, this brings up the story about OpenSSL on OSX again, as what's provided by the system still a looks bit broken, it seems.

    The problem is that we cannot, as rightfully pointed here by Michael, provide a tarball with OpenSSL source, as this
    would be an outright copyright violation. Thus we ought to rely on the system libraries.

    A couple of years ago that was, however, impossible on OSX, more or less, as their openssl implementation
    was old and lacking features.
    Now it appears that they updated openssl to a pretty new version 1.1.0c.

    There are some weird messages about certificates missing, but this could be my broken setup, or perhaps
    it's something trivial to fix by installing a free certificate bundle.
    Could someone knowledgeable about these things comment?

    Dima

    Dr. David Kirkby (Kirkby Microwave Ltd)

    unread,
    Oct 17, 2017, 7:35:52 PM10/17/17
    to sage-devel


    On 17 Oct 2017 23:56, "Dima Pasechnik" <dim...@gmail.com> wrote:
    >
    > On Tuesday, October 17, 2017 at 10:52:47 PM UTC+1, Nicolas M. Thiéry wrote:
    >>
    >>

    > The problem is that we cannot, as rightfully pointed here by Michael, provide a tarball with OpenSSL source, as this
    > would be an outright copyright violation. Thus we ought to rely on the system libraries.

    There are a lot of number theorists using Sagemath. Could one or more consider implementing the functionality of OpenSSL in a re-write? Maybe a Google Summer of Code project? 

    If a subset of the SSL developers are happy to have the code they wrote GPL, then there would be no need to rewrite those bits, if those individuals would relicense the bits they personally wrote.  That would mean a rewrite of the bit written by by developers who will not relicense what they wrote. 

    Dave.

    William Stein

    unread,
    Oct 17, 2017, 7:39:07 PM10/17/17
    to sage-...@googlegroups.com
    On Tue, Oct 17, 2017 at 4:35 PM Dr. David Kirkby (Kirkby Microwave Ltd) <drki...@kirkbymicrowave.co.uk> wrote:


    On 17 Oct 2017 23:56, "Dima Pasechnik" <dim...@gmail.com> wrote:
    >
    > On Tuesday, October 17, 2017 at 10:52:47 PM UTC+1, Nicolas M. Thiéry wrote:
    >>
    >>

    > The problem is that we cannot, as rightfully pointed here by Michael, provide a tarball with OpenSSL source, as this
    > would be an outright copyright violation. Thus we ought to rely on the system libraries.

    There are a lot of number theorists using Sagemath. Could one or more consider implementing the functionality of OpenSSL in a re-write? Maybe a Google Summer of Code project? 


    Absolutely not.   That's not how security software works (and would be insulting to the OpenSSL developers).   You are **epically** understimating what OpenSSL is and does.

    If a subset of the SSL developers are happy to have the code they wrote GPL, then there would be no need to rewrite those bits, if those individuals would relicense the bits they personally wrote.  That would mean a rewrite of the bit written by by developers who will not relicense what they wrote. 

    Dave.

    --
    You received this message because you are subscribed to the Google Groups "sage-devel" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
    To post to this group, send email to sage-...@googlegroups.com.
    Visit this group at https://groups.google.com/group/sage-devel.
    For more options, visit https://groups.google.com/d/optout.
    --
    -- William Stein

    Maarten Derickx

    unread,
    Oct 17, 2017, 8:42:25 PM10/17/17
    to sage-devel


    On Tuesday, 17 October 2017 22:46:52 UTC+2, Michael Orlitzky wrote:
    On 10/17/2017 03:56 PM, Emmanuel Charpentier wrote:
    >
    >     Throwing my vote away:
    >
    >     [X] Require OpenSSL to be installed on the system.
    >
    >
    > That's not one of the proposed options.

    That's what I meant by "throwing my vote away" =)


    > But it seems to imply that we should wait for OpenSSL relicensing.

    We don't have to wait if we require it to be installed on the system.

    "Wait for OpenSSL to relicense" is a bad option because they will
    probably never relicense -- even if they ultimately change their LICENSE
    file, the process by which they've done it is extremely dubious, and
    several copyright holders are on record withholding permission to do so.


    What makes you think their process is dubious? They are reaching out for consent from all people who have contributed, and they have removed the code from the several people who have objected on record.

    Maarten Derickx

    unread,
    Oct 17, 2017, 9:02:18 PM10/17/17
    to sage-devel
    |x| Yes, we should fully support OpenSSL now, and clarify the licensing issue.

    Where I want to add that my support is conditional on that this can be done in a legal way.

    William Stein

    unread,
    Oct 17, 2017, 9:09:12 PM10/17/17
    to sage-...@googlegroups.com
    The choice for users installing the Sage binary is between: 

     (a) using a broken version of the Python/R/Sage stack that exposes them to installing malware, or 

     (b) using a more robust version of the Python/R/Sage stack, but with the risk that somebodywho owns the copyright of some GPL'd part of the Sage distribution sues the distributor of Sage for violating their GPL license.

    A copyright holder that might have a valid complaint would not be OpenSSL, since it's not the OpenSSL license that is violated; it's the GPL that is violated.   

    Honestly, it's never even been clear to me that the GPL is definitely violated here, since OpenSSL only binary links with Python, and Python is not GPL'd.   The meaning of derived work for software that is combined only at runtime via an interpreter environment is still unclear to me (e.g., see recent inclusion of ZFS in Ubuntu). 

    The organizations doing the distribution of the Sage binaries would probably be taking the biggest risk here.

    William


    --
    -- William Stein

    Michael Orlitzky

    unread,
    Oct 17, 2017, 9:35:15 PM10/17/17
    to sage-...@googlegroups.com
    On 10/17/2017 08:42 PM, Maarten Derickx wrote:
    >
    > What makes you think their process is dubious? They are reaching out for
    > consent from all people who have contributed, and they have removed the
    > code from the several people who have objected on record.
    >

    The mail that they sent to contributors ended with,

    If we do not hear from you, we will assume that you have no objection.

    That's not the way it works, but if they've changed their approach, I'll
    be happy to be wrong in this case.

    William Stein

    unread,
    Oct 17, 2017, 9:38:02 PM10/17/17
    to sage-...@googlegroups.com
    On Tue, Oct 17, 2017 at 6:35 PM Michael Orlitzky <mic...@orlitzky.com> wrote:
    On 10/17/2017 08:42 PM, Maarten Derickx wrote:
    >
    > What makes you think their process is dubious? They are reaching out for
    > consent from all people who have contributed, and they have removed the
    > code from the several people who have objected on record.
    >

    The mail that they sent to contributors ended with,

      If we do not hear from you, we will assume that you have no objection.

    That's not the way it works,

    Says who?   This is all about how things work legally, and the rules there are not so simple.

    but if they've changed their approach, I'll
    be happy to be wrong in this case.




    --
    You received this message because you are subscribed to the Google Groups "sage-devel" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
    To post to this group, send email to sage-...@googlegroups.com.
    Visit this group at https://groups.google.com/group/sage-devel.
    For more options, visit https://groups.google.com/d/optout.
    --
    -- William Stein

    Michael Orlitzky

    unread,
    Oct 17, 2017, 9:41:13 PM10/17/17
    to sage-...@googlegroups.com
    On 10/17/2017 09:37 PM, William Stein wrote:
    >
    > The mail that they sent to contributors ended with,
    >
    >   If we do not hear from you, we will assume that you have no objection.
    >
    > That's not the way it works,
    >
    >
    > Says who?   This is all about how things work legally, and the rules
    > there are not so simple.
    >

    If I email Disney with "I'm going to download Lilo and Stitch, and if
    you don't reply by the end of the week, I'm going to assume you're cool
    with it" -- that's totally legit and legally binding?

    William Stein

    unread,
    Oct 17, 2017, 9:42:55 PM10/17/17
    to sage-...@googlegroups.com
    I am not a lawyer.  And that is not the same thing l, except maybe to a mathematician. 




    --
    You received this message because you are subscribed to the Google Groups "sage-devel" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
    To post to this group, send email to sage-...@googlegroups.com.
    Visit this group at https://groups.google.com/group/sage-devel.
    For more options, visit https://groups.google.com/d/optout.
    --
    -- William Stein

    Ralf Stephan

    unread,
    Oct 18, 2017, 1:31:39 AM10/18/17
    to sage-devel
    First, to voice my opinion:
     [X] Require OpenSSL to be installed on the system. 
    I really think that the Mac folks should resolve this and not require Sage to make awkward choices.

    As to the vote:
    |X| Yes, we should fully support OpenSSL now, and clarify the licensing issue.

    Regards,

    Jeroen Demeyer

    unread,
    Oct 18, 2017, 4:51:21 AM10/18/17
    to sage-...@googlegroups.com
    On 2017-10-18 03:08, William Stein wrote:
    > The choice for users installing the Sage binary is between:

    So you are worried about *binaries*? Are there any distros that we ship
    binaries for that *don't* have a systemwide OpenSSL installed by default?

    Jeroen Demeyer

    unread,
    Oct 18, 2017, 4:58:28 AM10/18/17
    to sage-...@googlegroups.com
    On 2017-10-18 03:08, William Stein wrote:
    > (a) using a broken version of the Python/R/Sage stack that exposes
    > them to installing malware

    Is that really the case? I think pip is actually fail-safe in the sense
    that it simply refuses to download if OpenSSL is not supported. So there
    is no exposure to malware here.

    Does anybody know how this works for R?

    Dima Pasechnik

    unread,
    Oct 18, 2017, 5:05:16 AM10/18/17
    to sage-devel
    there used to be the case with OSX that their systemwide OpenSSL was outdated. This is however no
    longer then case, at least on OSX 10.12. There still could be some headaches with absence of proper certificates
    installed on OSX (something fixable, I guess, but I'm not sure how), at least with my very limited understanding of the situation there.


     

    Jeroen Demeyer

    unread,
    Oct 18, 2017, 5:10:38 AM10/18/17
    to sage-...@googlegroups.com
    First of all, I think that your email is unfair because it presents the
    "Yes" option as something that we could just easily do. However, as
    mentioned in another post in this thread, the "Yes" option might
    actually be illegal.

    So my vote is "No".

    Emmanuel Charpentier

    unread,
    Oct 18, 2017, 5:17:17 AM10/18/17
    to sage-devel

    Cygwin ?

    And, as far as I know, there is no way for a Cygwin binary to *reliably* depend on another Cygwin package...

    --
    Emmanuel Charpentier

    Dima Pasechnik

    unread,
    Oct 18, 2017, 5:20:45 AM10/18/17
    to sage-devel
    I think the elaboration part of the "Yes" option was not very carefully worded, this is what Michael pointed out.
    We cannot HOST OpenSSL source (this is illegal with its present license),
    but nothing prevents us from providing means to install it legally.

    To be on a safe side with binary distribution of Sage, one might like to add the
    exception clause to the license, as I suggested.

    Emmanuel Charpentier

    unread,
    Oct 18, 2017, 5:22:53 AM10/18/17
    to sage-devel
    Le mercredi 18 octobre 2017 10:58:28 UTC+2, Jeroen Demeyer a écrit :
    On 2017-10-18 03:08, William Stein wrote:
    >   (a) using a broken version of the Python/R/Sage stack that exposes
    > them to installing malware

    Is that really the case? I think pip is actually fail-safe in the sense
    that it simply refuses to download if OpenSSL is not supported.

    And *that's* an excruciating pain in the @$$...
     
    So there
    is no exposure to malware here.

    In other words, you're protecting the target system the same way Origen (supposedly)  tried to protect himself from temptation.

    --
    Emmanuel Charpentier

    Emmanuel Charpentier

    unread,
    Oct 18, 2017, 5:40:12 AM10/18/17
    to sage-devel


    Le mercredi 18 octobre 2017 10:58:28 UTC+2, Jeroen Demeyer a écrit :

    1) There are *currently* http-accessible R repositories. The question is "for how long whal these repositories be mantained and curated ?".

    2) The same is true of Bioconductor, R-forge and Omegahat repositories.

    3) I have no extensive knowledge of the 11626 (as of today) available R packages in the CRAN repository aind its mirrors. However, I would be deeply surprised if none of them offered or neeeded access to https-only resources, such as distributed databases.

    4) There are also a $#i+load of non-CRAN repositories offering not-yet-published packages. Similarly, a number of published works (papers, books, etc...) offer access to non-CRAN repositories of data and complementary analyses. There is no guarantee that these resources are http-accessible.

    To be unable to *programatically* access these resources from R is (another) pain in the @$$.

    --
    Emmanuel Charpentier

    Maarten Derickx

    unread,
    Oct 18, 2017, 5:47:09 AM10/18/17
    to sage-devel
    Hmmm that line in the e-mail is very different from what is suggested by their public announcements, like for example https://www.coreinfrastructure.org/news/announcements/2017/03/openssl-re-licensing-apache-license-v-20-encourage-broader-use-other-foss . To quote from that section:

    “This re-licensing activity will make OpenSSL, already the world's most widely-used FOSS encryption software, more convenient to incorporate in the widest possible range of free and open source software," said Mishi Choudhary, Legal Director of Software Freedom Law Center (SFLC) and counsel to OpenSSL. “OpenSSL's team has carefully prepared for this re-licensing, and their process will be an outstanding example of 'how to do it right.’

    This gives me the confidence that they will not take a dubious unlawful approach, even though I find the wording in the e-mail dubious as well.

    To all the people reading this and caring about the OpenSSL license change: please visit https://license.openssl.org/cgi-bin/authors.py . Here you can see a list of contributors who have not yet responded to the license change request, and helping with getting OpenSSL to reach them will surely help with the license change. 

    Dr. David Kirkby (Kirkby Microwave Ltd)

    unread,
    Oct 18, 2017, 5:52:47 AM10/18/17
    to sage-devel

    On 18 Oct 2017 00:39, "William Stein" <wst...@gmail.com> wrote:
    >
    >
    > On Tue, Oct 17, 2017 at 4:35 PM Dr. David Kirkby (Kirkby Microwave Ltd) <drki...@kirkbymicrowave.co.uk> wrote:

    >> There are a lot of number theorists using Sagemath. Could one or more consider implementing the functionality of OpenSSL in a re-write? Maybe a Google Summer of Code project? 
    >
    >
    > Absolutely not.   That's not how security software works (and would be insulting to the OpenSSL developers).   You are **epically** understimating what OpenSSL is and does.

    I don't see how it is insulting to someone to say we like what you have done,  but need a different licence model, so will need to implement the algoithms ourselves.

    How is that materially different to Octave implementing MATLAB functionality but under an open source licence?

    I feel an unacceptable licence and/or a broken implementation on one platform (OSX) are both reasons for a rewrite.  It seems that there are both problems now. 

    What in my opinion is insulting is to

    1) Add the OpenSSL code to Sagemath, knowing full well it is against the licence. How anybody can justify such action is beyond me. 

    2) Email people and say that you assume that they agree unless they say they object.

    Dave

    Emmanuel Charpentier

    unread,
    Oct 18, 2017, 6:12:58 AM10/18/17
    to sage-devel


    Le mercredi 18 octobre 2017 11:52:47 UTC+2, Dr. David Kirkby (Kirkby Microwave Ltd) a écrit :

    On 18 Oct 2017 00:39, "William Stein" <wst...@gmail.com> wrote:
    >
    >
    > On Tue, Oct 17, 2017 at 4:35 PM Dr. David Kirkby (Kirkby Microwave Ltd) <drki...@kirkbymicrowave.co.uk> wrote:

    >> There are a lot of number theorists using Sagemath. Could one or more consider implementing the functionality of OpenSSL in a re-write? Maybe a Google Summer of Code project? 
    >
    >
    > Absolutely not.   That's not how security software works (and would be insulting to the OpenSSL developers).   You are **epically** understimating what OpenSSL is and does.

    I don't see how it is insulting to someone to say we like what you have done,  but need a different licence model, so will need to implement the algoithms ourselves.

     
    Implementing crypto algorithms is (relatively) easy.

    Implementing crypto algorithms *correctly* (i. e. with no failure points) is *incredibly* *hard*. The implementation needs not only implement the algorithm, it has to do so leaving no exploitable traces or exploitable backdoors. As you should know, proving the *absence* of such attack points is really *not* easy.

    The recent history offers a *lot* of security issues attributable to faulty implementations of sound algorithms. The last one has been published just the day before yesterday (Google for "WPA KRACK" if you're not convinced...)

    I don't know how to do that correctly. However, I know that I don't know....
     

    How is that materially different to Octave implementing MATLAB functionality but under an open source licence?


    Try it for yourself ;-).
     

    I feel an unacceptable licence and/or a broken implementation on one platform (OSX) are both reasons for a rewrite.  It seems that there are both problems now. 

    What in my opinion is insulting is to

    1) Add the OpenSSL code to Sagemath, knowing full well it is against the licence.


    Whose license ? The problem is not with OpenSSL licenses (yes, there are two of them...) but with the GPL. Terms that we may amend at will. A suggestion (written and tested by the authors of another Gnu package) has been made, which some find faulty. They are welcome to propose better terms...
     

    How anybody can justify such action is beyond me.


    See above.
     

     

    2) Email people and say that you assume that they agree unless they say they object.


    That's (alas...) common practice, in much more general ways that software licensing. You should try to explain to your tax collector that you didn't consent to pay taxes...

    At least, that's not shrink-wrap license...

    --
    Emmanuel Charpentier

    Jeroen Demeyer

    unread,
    Oct 18, 2017, 6:51:52 AM10/18/17
    to sage-...@googlegroups.com
    On 2017-10-18 01:38, William Stein wrote:
    > Absolutely not. That's not how security software works (and would be
    > insulting to the OpenSSL developers). You are **epically**
    > understimating what OpenSSL is and does.

    +1

    Implementing crypto in practice is very different from implementing a
    toy RSA function in Python.

    Also: if re-implementing OpenSSl really would have been so easy, it
    would already have been done a long time ago.

    Erik Bray

    unread,
    Oct 18, 2017, 9:13:43 AM10/18/17
    to sage-devel
    On Wed, Oct 18, 2017 at 11:52 AM, Dr. David Kirkby (Kirkby Microwave
    Ltd) <drki...@kirkbymicrowave.co.uk> wrote:
    > On 18 Oct 2017 00:39, "William Stein" <wst...@gmail.com> wrote:
    >>
    >>
    >> On Tue, Oct 17, 2017 at 4:35 PM Dr. David Kirkby (Kirkby Microwave Ltd)
    >> <drki...@kirkbymicrowave.co.uk> wrote:
    >
    >>> There are a lot of number theorists using Sagemath. Could one or more
    >>> consider implementing the functionality of OpenSSL in a re-write? Maybe a
    >>> Google Summer of Code project?
    >>
    >>
    >> Absolutely not. That's not how security software works (and would be
    >> insulting to the OpenSSL developers). You are **epically** understimating
    >> what OpenSSL is and does.
    >
    > I don't see how it is insulting to someone to say we like what you have
    > done, but need a different licence model, so will need to implement the
    > algoithms ourselves.
    >
    > How is that materially different to Octave implementing MATLAB functionality
    > but under an open source licence?
    >
    > I feel an unacceptable licence and/or a broken implementation on one
    > platform (OSX) are both reasons for a rewrite. It seems that there are both
    > problems now.
    >
    > What in my opinion is insulting is to
    >
    > 1) Add the OpenSSL code to Sagemath, knowing full well it is against the
    > licence. How anybody can justify such action is beyond me.

    Note: We're not talking about adding *any* OpenSSL code to SageMath.
    Sage would never be distributed with code from OpenSSL. We're only
    talking about providing a means to download and install it from
    source, and about shipping binaries that includes it.

    Dr. David Kirkby (Kirkby Microwave Ltd)

    unread,
    Oct 18, 2017, 9:37:13 AM10/18/17
    to sage-devel
    On 18 October 2017 at 14:13, Erik Bray <erik....@gmail.com> wrote:
    On Wed, Oct 18, 2017 at 11:52 AM, Dr. David Kirkby (Kirkby Microwave
    Note: We're not talking about adding *any* OpenSSL code to SageMath.
    Sage would never be distributed with code from OpenSSL.  We're only
    talking about providing a means to download and install it from
    source, and about shipping binaries that includes it.

     
    How exactly do you intend shipping binaries that include the OpenSSL library, while not including *any*  OpenSSL code?

    Dave

    Emmanuel Charpentier

    unread,
    Oct 18, 2017, 10:57:08 AM10/18/17
    to sage-devel

    Simple : the same way that e. g. the Cygwin port ships code interacting with, say, Windows without *including* *any* Windows code... : by using published APIs to published libraries. We do not have to include OpenSSL *code* to use OpenSSL as long as OpenSSL is installed somewhere reachable by Sage.

    The contentious point is : how do we ensure that OpenSSL *is* installed ? With most packages, we include it, by posting it on the Sage servers and fetching if necessary. This, in some eyes, is "including" code, since we post a copy on our servers.

    To these (respectable) people, this is "legally" dubious (in which legislation(s), BTW ???), until the OpenSSL license changes. We need at least an interm solution.

    • Depending on a systemwide SSL is a possibility.
    • If we retain the option "Include now", other possibilities could be discussed.
    --
    Emmanuel Charpentier

    Dave

    Thierry

    unread,
    Oct 18, 2017, 12:23:53 PM10/18/17
    to sage-...@googlegroups.com
    Hi,

    the dichotomy of the vote is not clear to me.

    I am -1 to make openssl a stantard package (hence shipped with the source
    tarball), not only regarding licensing issues but also for security
    reasons: our "package manager" is such that packages can not be updated
    unless Sage itself is updated (because the package version is hardcoded).
    Hence, when a security issue is found and fixed in openssl, the user who
    installed it from Sage won't get it until the user upgrades Sage (while
    every decent distro will provide a hotfix).

    However, i am +1 that we should do our best to let the user have an
    openssl-enabled version of Sage (for pip, R, some cryptographic hash,...),
    an acceptable workflow could be:

    - check if libssl-dev (or similar) is installed on the OS
    - yes:
    - use it
    - no:
    - strongly complain about it, provide documentation on how to do it
    (possibly provide a doc that depends on the system),
    - propose 3 options:
    - "i will install openssl from the distro, and come back later
    (recommended)"
    - "i want Sage to install openssl optional package, i know that
    there will be security issues"
    - "i do not want openssl support, i know that i will not be able
    to install any R or Python package from the web"

    If the last point (compiling Sage without openssl support) requires a lot
    of work, i am OK to remove it (i am not sure if this is the point of the
    vote).

    Note that that there is no chicken-and-eggs issue since the way our
    "package manager" works allows to install an optional package without
    having to rely on openssl (no https), we only rely on the computation of
    sha1 which python-hashlib offers even if it is build without openssl
    support.

    By the way, Sage is not GPL-3+ but GPL-2+.

    <troll>

    Mac fans claim that paying a computer 1.5 the price of a random PC with
    similar charateristics if justified by the fact that OSX is soooo
    user-friendly, perhaps didn't they find the openssl one-click installer
    right in the middle of the screen yet.

    Proposal: require Apple a grant, corresponding to the huge amount of time
    Sage developpers waste in porting Sage components (not only openssl, just
    have a look at trac, sage-devel and ask timelines) on their broken and
    constantly changing OS. This is not our job to help Apple pretend their
    system is user-friendly, we are losing a lot of energy which could be
    spent in much more interesting parts of Sage (e.g. mathematics).

    </troll>

    Ciao,
    Thierry




    On Mon, Oct 16, 2017 at 03:08:51AM -0700, Emmanuel Charpentier wrote:
    > [ The first post started too fast... Sorry for the interruption ! ]
    >
    > Following numerous discussions on this list and various Trac tickets*, the
    > issue of maintaining Sage-specific patches to various components of Sage
    > emerged again about the proposed upgrade
    > <https://trac.sagemath.org/ticket/24026> of R to 3.4.2 (discussed here
    > <https://groups.google.com/forum/#!topic/sage-devel/rhMrNK_2c24>). William
    > again raises
    > <https://groups.google.com/d/msg/sage-devel/rhMrNK_2c24/WQ5FPmsiAQAJ> the
    > issue of security.
    >
    > Since Trac#22189 <https://trac.sagemath.org/ticket/22189>, installation of
    > a systemwide opennssl is recommended (may be too strongly
    > <https://trac.sagemath.org/ticket/22620>, in the taste of some respectable
    > Sage developers...). The ongoing relicensing of OpenSSL should lift the
    > last barriers to its inclusion in sage. A discussed here
    > <https://groups.google.com/forum/#!topic/sage-devel/rhMrNK_2c24>,, the
    > probability of a legal problem related to the incusion of this library in
    > Sage seems infinitesimal.
    >
    > It has beeen furthermore suggested
    > <https://groups.google.com/d/msg/sage-devel/rhMrNK_2c24/GYHzsSd6BAAJ> to
    > add to our licensing (an adaptatin of) the following language, used in Gnu
    > Wget License (GPL) :
    >
    > "Additional permission under GNU GPL version 3 section 7
    >
    > If you modify this program, or any covered work, by linking or combining it
    > with the OpenSSL project's OpenSSL library (or a modified version of that
    > library), containing parts covered by the terms of the OpenSSL or SSLeay
    > licenses, the Free Software Foundation grants you additional permission to
    > convey the resulting work. Corresponding Source for a non-source form of
    > such a combination shall include the source code for the parts of OpenSSL
    > used as well as that of the covered work."
    >
    >
    > The proposed inclusion would entail :
    >
    > - Deprecation of our OpenSSL-avidance patches
    > - Standardization of SSL communications on OpenSSL
    > - At compilation, research of a systemwide OpenSSL
    > - If found : do nothing
    > - In not found : installation of OpenSSL in the Sage tree from a
    > Sage-specific repository (as for most of our standard and optional
    > packages...).
    > - Licensing clarification
    >
    > In short, we have two options : include OpenSSL now (using language
    > clarification), or wait for the complete OpenSSL relicensing. The exact
    > terms of the vote are therefore :
    >
    > |_| Yes, we should fully support OpenSSL now, and clarify the licensing
    > issue.
    >
    > |_| No, we should wait until OpenSSL finishes fixing their license
    > situation formally.
    >
    > The vote will take place as answers to this post, and will be open until
    > Monday October 23, 14h UTC.
    >
    > Sincerely yours,
    >
    >
    > Emmanuel Charpentier
    >
    > * Perusing the results of searching Trac and sage-devel Google group is
    > enlightening...
    > --
    > Emmanuel Charpentier

    William Stein

    unread,
    Oct 18, 2017, 12:37:53 PM10/18/17
    to sage-...@googlegroups.com
    For what it is worth, I strongly agree with everything you write above.  +1

    William
    --
    -- William Stein

    Emmanuel Charpentier

    unread,
    Oct 18, 2017, 1:02:43 PM10/18/17
    to sage-devel
    Dear Thierry,


    Le mercredi 18 octobre 2017 18:23:53 UTC+2, Thierry (sage-googlesucks@xxx) a écrit :
    Hi,

    the dichotomy of the vote is not clear to me.

    I am -1 to make openssl a stantard package (hence shipped with the source
    tarball), not only regarding licensing issues but also for security
    reasons: our "package manager" is such that packages can not be updated
    unless Sage itself is updated (because the package version is hardcoded).
    Hence, when a security issue is found and fixed in openssl, the user who
    installed it from Sage won't get it until the user upgrades Sage (while
    every decent distro will provide a hotfix).

    That's a good and important point that I (among others) had overlooked. It could even preclude any "licensing issues" argument...

    To be discussed post-vote, with other "implementation" issues ?
     
    However, i am +1 that we should do our best to let the user have an
    openssl-enabled version of Sage (for pip, R, some cryptographic hash,...),
    an acceptable workflow could be:

    - check if libssl-dev (or similar) is installed on the OS
      - yes:
        - use it
      - no:
        - strongly complain about it, provide documentation on how to do it
          (possibly provide a doc that depends on the system),
        - propose 3 options:
            - "i will install openssl from the distro, and come back later
              (recommended)"

    This one is acceptable, and does not raise any pseudo-legalistic question.
     
            - "i want Sage to install openssl optional package, i know that
              there will be security issues"

    We should first upgrade our optional package top post-1.1.0 : the build system has changed *incompatibly !*

    Furthermore, your (important) remark about our unability to "rush" security-related upgrades also apply here...
     
            - "i do not want openssl support, i know that i will not be able
              to install any R or Python package from the web"

    Yikes ! Aaaarghhh ! And all that sort of things...

    This option commits us to maintain (unnecessary and dangerous, IMHO) Sage-specifc SSL patches at least in R, Python and pip, and this until the Last Judgement. Or OpenSSL relicensing (whichever comes first).

    Do we have such an excess of workforce that we can allow to waste on this ?

    Do we want to accept the responsibility of shipping a (voluntarily) crippled tool ?

    If the last point (compiling Sage without openssl support) requires a lot
    of work, i am OK to remove it (i am not sure if this is the point of the
    vote).

    I'd remove it in a cinch...

    Note that that there is no chicken-and-eggs issue since the way our
    "package manager" works allows to install an optional package without
    having to rely on openssl (no https), we only rely on the computation of
    sha1 which python-hashlib offers even if it is build without openssl
    support.

    Indeed, but that's a point we should fix. But that needs to be sure to have an https-enabled download tool ;-)...

    By the way, Sage is not GPL-3+ but GPL-2+.

    <troll>

    Mac fans claim that paying a computer 1.5 the price of a random PC with
    similar charateristics if justified by the fact that OSX is soooo
    user-friendly, perhaps didn't they find the openssl one-click installer
    right in the middle of the screen yet.

    Proposal: require Apple a grant, corresponding to the huge amount of time
    Sage developpers waste in porting Sage components (not only openssl, just
    have a look at trac, sage-devel and ask timelines) on their broken and
    constantly changing OS.

    Seconded ! Now, that's a get-rich-fast scheme...
     
    This is not our job to help Apple pretend their
    system is user-friendly, we are losing a lot of energy which could be
    spent in much more interesting parts of Sage (e.g. mathematics).

    </troll>
    Ciao,
    Thierry

    --
    Emmanuel Charpentier
     

    Erik Bray

    unread,
    Oct 18, 2017, 1:26:12 PM10/18/17
    to sage-devel
    Also +1 with some quibbles about <troll> section (agree with in
    principle, but in tone or nuance).

    But big +1 for the proposed implementation, including the strong and
    informative warning messages :)

    Jeroen Demeyer

    unread,
    Oct 18, 2017, 2:36:47 PM10/18/17
    to sage-...@googlegroups.com
    On 2017-10-18 19:02, Emmanuel Charpentier wrote:
    > This option commits us to maintain (unnecessary and dangerous, IMHO)
    > Sage-specifc SSL patches at least in R, Python and pip

    Really? Which Sage-specific SSL patches does this require in Python and pip?

    It seems to me that R is the only package causing us so much trouble
    with SSL.

    Erik Bray

    unread,
    Oct 19, 2017, 3:19:00 AM10/19/17
    to sage-devel
    There *was* an issue in pip (not Python itself though that I can
    recall--Python works fine without SSL support obviously). However,
    the pip issue has since been fixed upstream (pip can work without SSL
    for installing packages from sources other than PyPI, there was just a
    bug with imports failing if the ssl module was not importable).

    If R still requires such a patch I think we should get really pushy
    with upstream about getting them to accept it. R should at least be
    able to function without it, even if it means disabling package
    downloads from CRAN which is fine.

    Ralf Stephan

    unread,
    Oct 19, 2017, 3:26:46 AM10/19/17
    to sage-devel
    After the previous comments I'd like to change my vote from Yes to

    |X| No 

    Regards,

    kcrisman

    unread,
    Oct 19, 2017, 9:49:17 AM10/19/17
    to sage-devel

    > For what it is worth, I strongly agree with everything you write above.  +1

    Also +1 with some quibbles about <troll> section (agree with in
    principle, but in tone or nuance).


     perhaps didn't they find the openssl one-click installer
    right in the middle of the screen yet. 

    That sounds awesome, but I have no idea what you are talking about - can you point us to it/add to instructions?  This seems pretty reasonable to require users who want this on Mac ... unless I'm totally missing some weird joke.  At any rate, my Mac doesn't have an "install ssl" button on its screen, though I wish it did.  Our binary dmgs don't have such a button, do they?

    Erik Bray

    unread,
    Oct 19, 2017, 10:30:34 AM10/19/17
    to sage-devel
    I have no idea. But also keep in mind that many users have no idea
    what an "OpenSSL" is and aren't going to assume they need it if they
    don't know what it is.

    Emmanuel Charpentier

    unread,
    Oct 19, 2017, 10:58:27 AM10/19/17
    to sage-devel
    Dear Jeroen,

    Unless you correct me, I'll tally your vote as 
    |X| No, we should wait until OpenSSL finishes fixing their license situation formally.

    --
    Emmanuel Charpentier

    Emmanuel Charpentier

    unread,
    Oct 19, 2017, 10:58:35 AM10/19/17
    to sage-devel
    OK. Unless you correct me, I'll tally your vote as :
    |X| No, we should wait until OpenSSL finishes fixing their license situation formally.

    --
    Emmanuel Charpentier

    Emmanuel Charpentier

    unread,
    Oct 19, 2017, 11:19:46 AM10/19/17
    to sage-devel
    Dear Erik


    Le jeudi 19 octobre 2017 09:19:00 UTC+2, Erik Bray a écrit :
    On Wed, Oct 18, 2017 at 8:36 PM, Jeroen Demeyer <jdem...@cage.ugent.be> wrote:
    > On 2017-10-18 19:02, Emmanuel Charpentier wrote:
    >>
    >> This option commits us to maintain (unnecessary and dangerous, IMHO)
    >> Sage-specifc SSL patches at least in R, Python and pip
    >
    >
    > Really? Which Sage-specific SSL patches does this require in Python and pip?
    >
    > It seems to me that R is the only package causing us so much trouble with
    > SSL.

    There *was* an issue in pip (not Python itself though that I can
    recall--Python works fine without SSL support obviously).  However,
    the pip issue has since been fixed upstream (pip can work without SSL
    for installing packages from sources other than PyPI, there was just a
    bug with imports failing if the ssl module was not importable).

    If R still requires such a patch I think we should get really pushy
    with upstream about getting them to accept it.

    No : this policy decision is recent, and the "R Core Team" seems pretty adamant to switch to https repositories for all of CRAN. I seriously doubt that they would reverse this decision on the behalf of a bunch of mathematicians for whom R usage may be deemed as "peripheral"...
     
    R should at least be
    able to function without it, even if it means disabling package
    downloads from CRAN which is fine.

    Again : R is not only a software package but also an ecosystem. The 11638 (as of today) packages available to R users are a large part of R usefulness to its users. So, "disabling downloads from CRAN" is *NOT* fine (to them, at least...). 

    And, BTW : which is your vote ?

    --
    Emmanuel Charpentier

    Emmanuel Charpentier

    unread,
    Oct 19, 2017, 11:21:11 AM10/19/17
    to sage-devel


    Le mercredi 18 octobre 2017 20:36:47 UTC+2, Jeroen Demeyer a écrit :
    On 2017-10-18 19:02, Emmanuel Charpentier wrote:
    > This option commits us to maintain (unnecessary and dangerous, IMHO)
    > Sage-specifc SSL patches at least in R, Python and pip

    Really? Which Sage-specific SSL patches does this require in Python and pip? *

    From the Python license page :

    The modules hashlibposixsslcrypt use the OpenSSL library for added performance if made available by the operating system. Additionally, the Windows and Mac OS X installers for Python may include a copy of the OpenSSL libraries, so we include a copy of the OpenSSL license here: 

    ISTR that last year update of Python (to which you contributed mightily) bumped on SSL issues. But my memories are foggy.

    And, BTW, "our" pip is partially nonfunctional, thanks (among other) to SSL issues. So it *should* be patched.
     
    It seems to me that R is the only package causing us so much trouble
    with SSL.

    No : *I* cause "so much trouble" with SSL issues because they become important to R real-world usage, and I do not think that a non-communicating R is useful in Sage.  Python/pip SSL usage do not cause "so much trouble" with SSL issues because the potential users of SSL functionalities are more patient than I am...

    --
    Emmanuel Charpentier

    William Stein

    unread,
    Oct 19, 2017, 11:24:20 AM10/19/17
    to sage-...@googlegroups.com
    On Thu, Oct 19, 2017 at 8:19 AM Emmanuel Charpentier <emanuel.c...@gmail.com> wrote:
    Dear Erik

    Le jeudi 19 octobre 2017 09:19:00 UTC+2, Erik Bray a écrit :
    On Wed, Oct 18, 2017 at 8:36 PM, Jeroen Demeyer <jdem...@cage.ugent.be> wrote:
    > On 2017-10-18 19:02, Emmanuel Charpentier wrote:
    >>
    >> This option commits us to maintain (unnecessary and dangerous, IMHO)
    >> Sage-specifc SSL patches at least in R, Python and pip
    >
    >
    > Really? Which Sage-specific SSL patches does this require in Python and pip?
    >
    > It seems to me that R is the only package causing us so much trouble with
    > SSL.

    There *was* an issue in pip (not Python itself though that I can
    recall--Python works fine without SSL support obviously).  However,
    the pip issue has since been fixed upstream (pip can work without SSL
    for installing packages from sources other than PyPI, there was just a
    bug with imports failing if the ssl module was not importable).

    If R still requires such a patch I think we should get really pushy
    with upstream about getting them to accept it.

    No : this policy decision is recent, and the "R Core Team" seems pretty adamant to switch to https repositories for all of CRAN. I seriously doubt that they would reverse this decision on the behalf of a bunch of mathematicians for whom R usage may be deemed as "peripheral"...

    Good, as well they should.   Like you, they likely feel a responsibility to their users to do the right thing regarding security.   I really appreciate the "so much trouble" you are "causing" Emmanuel.

     -- William
    --
    -- William Stein

    Luca De Feo

    unread,
    Oct 19, 2017, 2:07:45 PM10/19/17
    to sage-...@googlegroups.com
    |X| Yes, we should fully support OpenSSL now, and clarify the
    licensing issue.

    > the way our
    > "package manager" works allows to install an optional package without
    > having to rely on openssl (no https), we only rely on the computation of
    > sha1

    There you go for something crippled! https://shattered.io/

    SHA1 has been deprecated by any reasonable browser nowaday.

    Luca

    Thierry

    unread,
    Oct 19, 2017, 4:56:21 PM10/19/17
    to sage-...@googlegroups.com
    Hi,

    On Thu, Oct 19, 2017 at 08:07:19PM +0200, Luca De Feo wrote:
    > |X| Yes, we should fully support OpenSSL now, and clarify the
    > licensing issue.
    >
    > > the way our
    > > "package manager" works allows to install an optional package without
    > > having to rely on openssl (no https), we only rely on the computation of
    > > sha1
    >
    > There you go for something crippled! https://shattered.io/

    sha256 is also supported by python-hashlib compiled without openssl
    support, so we could/should easily move to using it in Sage "package
    manager" (build/sage_bootstrap/tarball.py).

    Ciao,
    Thierry


    > SHA1 has been deprecated by any reasonable browser nowaday.
    >
    > Luca
    >

    Dima Pasechnik

    unread,
    Oct 19, 2017, 5:17:10 PM10/19/17
    to sage-devel
    the 1-click openssl install image for OSX is called Xcode, and one can go for a long lunch while waiting for it to finish, even on a fast network...

    Apple should pick up the bill for these lunches, and much more, I fully agree.

    John H Palmieri

    unread,
    Oct 19, 2017, 6:29:46 PM10/19/17
    to sage-devel


    On Thursday, October 19, 2017 at 2:17:10 PM UTC-7, Dima Pasechnik wrote:
    the 1-click openssl install image for OSX is called Xcode, and one can go for a long lunch while waiting for it to finish, even on a fast network...

    Apple should pick up the bill for these lunches, and much more, I fully agree.


    But I don't think that Xcode includes the openssl headers. At least I've never been able to find them. Plus perhaps their installed versions of libssl are outdated? That's what I've heard about their reasons for not including the header files.

    --
    John

    Jeroen Demeyer

    unread,
    Oct 20, 2017, 4:49:40 AM10/20/17
    to sage-...@googlegroups.com
    On 2017-10-19 17:24, William Stein wrote:
    > Good, as well they should. Like you, they likely feel a responsibility
    > to their users to do the right thing regarding security. I really
    > appreciate the "so much trouble" you are "causing" Emmanuel.

    I also agree here. The only options should be "use https" or "don't
    download anything". That's exactly what pip does.

    Jeroen Demeyer

    unread,
    Oct 20, 2017, 4:51:17 AM10/20/17
    to sage-...@googlegroups.com
    On 2017-10-19 17:19, Emmanuel Charpentier wrote:
    > Again : R is not only a software package but also an ecosystem.

    But why? One could say the same for Python, but you can still install
    Python without OpenSSL.

    What if I simply want to use R without any external packages? Or what if
    I want to download/install packages in a different way?

    Dima Pasechnik

    unread,
    Oct 20, 2017, 4:54:39 AM10/20/17
    to sage-devel
    Once upon a time, http was not universally supported, one needed to use ftp instead.
    Nowadays you often need at least http.
    That is to say, this point is moot; if you do not want to use https, you're on your own.

    Jeroen Demeyer

    unread,
    Oct 20, 2017, 4:58:32 AM10/20/17
    to sage-...@googlegroups.com
    On 2017-10-19 20:07, Luca De Feo wrote:
    > There you go for something crippled! https://shattered.io/

    I don't think that this is actually relevant. This attack would only
    work if an attacker is able to provide a specially manufactured source
    tarball and get it accepted by SageMath. At that point, the attacker
    could instead just insert arbitrary code in the source tarball.

    Jeroen Demeyer

    unread,
    Oct 20, 2017, 5:13:54 AM10/20/17
    to sage-...@googlegroups.com
    On 2017-10-20 10:54, Dima Pasechnik wrote:
    > Once upon a time, http was not universally supported, one needed to use
    > ftp instead.

    You misunderstood my point. It is not about http vs. https.

    What bothers me is that "downloading packages from CRAN" is considered
    so important by R that it refuses to build when that functionality is
    not available. Regardless of how important pip is to Python, it is still
    possible to run Python without pip. And it should be like this: the core
    functionality should be separated from the package manager.

    Luca De Feo

    unread,
    Oct 20, 2017, 5:32:35 AM10/20/17
    to sage-...@googlegroups.com
    >> There you go for something crippled! https://shattered.io/
    >
    >
    > I don't think that this is actually relevant. This attack would only work if
    > an attacker is able to provide a specially manufactured source tarball and
    > get it accepted by SageMath. At that point, the attacker could instead just
    > insert arbitrary code in the source tarball.

    So according to your point checking the SHA1 is useless, because
    attackers are not able to get malicious source tarballs accepted by
    SageMath.

    Anyway, we're digressing. The move to SHA256 needs to be addressed in
    another topic, and it is so elementary that we may as well open a
    ticket and continue the discussion there.

    Luca

    Jeroen Demeyer

    unread,
    Oct 20, 2017, 5:44:41 AM10/20/17
    to sage-...@googlegroups.com
    On 2017-10-20 11:32, Luca De Feo wrote:
    > So according to your point checking the SHA1 is useless, because
    > attackers are not able to get malicious source tarballs accepted by
    > SageMath.

    That is totally not what I said. We don't care about collision
    resistance, but we still need preimage resistance. That is still fine
    for SHA1 (even MD5 as far as I know).

    Dima Pasechnik

    unread,
    Oct 20, 2017, 6:04:13 AM10/20/17
    to sage-devel


    On Friday, October 20, 2017 at 10:13:54 AM UTC+1, Jeroen Demeyer wrote:
    On 2017-10-20 10:54, Dima Pasechnik wrote:
    > Once upon a time, http was not universally supported, one needed to use
    > ftp instead.

    You misunderstood my point. It is not about http vs. https.

    What bothers me is that "downloading packages from CRAN" is considered
    so important by R that it refuses to build when that functionality is
    not available.

    It is a good reason for stopping shipping R with  Sage, provided that we can still have the
    needed interfaces  up and  running. Otherwise it seems not too meaningful to complain about R here.

    Luca De Feo

    unread,
    Oct 20, 2017, 7:26:12 AM10/20/17
    to sage-...@googlegroups.com
    > That is totally not what I said. We don't care about collision resistance,
    > but we still need preimage resistance. That is still fine for SHA1 (even MD5
    > as far as I know).

    If that's your point, an attacker can produce two colliding packages:
    a perfectly sound mathematical package and a malicious one. He gets
    the mathematical package accepted by SageMath, then uses the malicious
    one to perform the attack.

    I'm not saying that's easy to do. The perfectly sound mathematical
    package would still have to contain some weird octets, but a package
    that looks like, e.g., a database of polynomials, could conceivably
    evade detection.

    I'm saying all this to satisfy the applied cryptographer in you. There
    are certainly easier ways to insert malicious code into Sage (just
    create a ticket and have a buddy positive review it), but that's not a
    good reason to keep using SHA1.

    Luca

    Maarten Derickx

    unread,
    Oct 20, 2017, 10:54:09 AM10/20/17
    to sage-devel
    +1 for this.
     
    If the last point (compiling Sage without openssl support) requires a lot
    of work, i am OK to remove it (i am not sure if this is the point of the
    vote).

    Note that that there is no chicken-and-eggs issue since the way our
    "package manager" works allows to install an optional package without
    having to rely on openssl (no https), we only rely on the computation of
    sha1 which python-hashlib offers even if it is build without openssl
    support.

    By the way, Sage is not GPL-3+ but GPL-2+.

    <troll>

    Mac fans claim that paying a computer 1.5 the price of a random PC with
    similar charateristics if justified by the fact that OSX is soooo
    user-friendly, perhaps didn't they find the openssl one-click installer
    right in the middle of the screen yet.

    Proposal: require Apple a grant, corresponding to the huge amount of time
    Sage developpers waste in porting Sage components (not only openssl, just
    have a look at trac, sage-devel and ask timelines) on their broken and
    constantly changing OS. This is not our job to help Apple pretend their
    system is user-friendly, we are losing a lot of energy which could be
    spent in much more interesting parts of Sage (e.g. mathematics).

    </troll>

    Ciao,
    Thierry




    > |_| Yes, we should fully support OpenSSL now, and clarify the licensing
    > issue.
    >
    > |_| No, we should wait until OpenSSL finishes fixing their license
    > situation formally.
    >
    > The vote will take place as answers to this post, and will be open until
    > Monday October 23, 14h UTC.
    >
    > Sincerely yours,
    >
    >
    > Emmanuel Charpentier
    >
    > * Perusing the results of searching Trac and sage-devel Google group is
    > enlightening...
    > --
    > Emmanuel Charpentier
    >

    kcrisman

    unread,
    Oct 20, 2017, 2:20:17 PM10/20/17
    to sage-devel


    On Thursday, October 19, 2017 at 6:29:46 PM UTC-4, John H Palmieri wrote:


    On Thursday, October 19, 2017 at 2:17:10 PM UTC-7, Dima Pasechnik wrote:
    the 1-click openssl install image for OSX is called Xcode, and one can go for a long lunch while waiting for it to finish, even on a fast network...

    Oh, but that's not the same thing as "it's sitting on your screen".  Asking people to install all of Xcode does seem a bit excessive, given that we currently don't ask people to even install the developer tools unless they want to do developing (or Cython, probably).
     

    Apple should pick up the bill for these lunches, and much more, I fully agree.


     
    True.
     
    But I don't think that Xcode includes the openssl headers. At least I've never been able to find them. Plus perhaps their installed versions of libssl are outdated? That's what I've heard about their reasons for not including the header files.


     And just having headers isn't the same as "it just works" unless they go in the right place, I suppose.

    Dima Pasechnik

    unread,
    Oct 20, 2017, 2:25:18 PM10/20/17
    to sage-devel
    In fact, John pointed out that I am wrong; while openssl is supported by Xcode binaries, there are no headers available!
    (it used to be the case that they were present in some hidden directories, but this seems to be not true any more)

    Eric Gourgoulhon

    unread,
    Oct 21, 2017, 12:02:33 PM10/21/17
    to sage-devel
    Hi,

    Having read the discussion, I would add a big +1 to what Thierry proposes in
    https://groups.google.com/d/msg/sage-devel/fE45025Wphs/FheYtjBWAAAJ

    So I guess that in terms of vote this means

    |X| Yes, we should fully support OpenSSL now, and clarify the licensing issue.

    BUT following Thierry's procedure:


    - check if libssl-dev (or similar) is installed on the OS
      - yes:
        - use it
      - no:
        - strongly complain about it, provide documentation on how to do it
          (possibly provide a doc that depends on the system),
        - propose 3 options:
            - "i will install openssl from the distro, and come back later
              (recommended)"
            - "i want Sage to install openssl optional package, i know that
              there will be security issues"
            - "i do not want openssl support, i know that i will not be able
              to install any R or Python package from the web"

    Best wishes,

    Eric.

    Emmanuel Charpentier

    unread,
    Oct 21, 2017, 2:37:57 PM10/21/17
    to sage-devel
    And the fly in the ointment is that, for example, "our" pip doesn't give us the choice. It forces us to "don't download anything'...

    I'm afraid that R users will sooner or later be confronted to the same choice. In Hobson's variant if we don't have an https-enabled SSL library. Nowadays, that means OpenSSL...

    --
    Emmanuel Charpentier

    Emmanuel Charpentier

    unread,
    Oct 21, 2017, 2:45:20 PM10/21/17
    to sage-devel


    Le vendredi 20 octobre 2017 10:51:17 UTC+2, Jeroen Demeyer a écrit :
    On 2017-10-19 17:19, Emmanuel Charpentier wrote:
    > Again : R is not only a software package but also an ecosystem.

    But why? One could say the same for Python, but you can still install
    Python without OpenSSL.

    There might be practical use of {Sage|Python} without having to download. But...

    What if I simply want to use R without any external packages? Or what if
    I want to download/install packages in a different way?

    You may accept to hop through flaming circles in effect to accomplish what is specified in the R documentation as a simple operation. Tou may also accept to lose the benefits of the package management system of R (dependencies, version checking and the like...). Mos "normal" R users wont't stand it.

    Furthermore, by refusing to ensure the presence of a library necessary to R, you force the poor sod willing to maintain R (me, currently), to maintaint a patch that serves no useful purpose other that to protect your precious OpenSSL maidenhood. That's a bit rich...

    --
    Emmanuel Charpentier
     

    David Joyner

    unread,
    Oct 21, 2017, 2:53:38 PM10/21/17
    to sage-devel
    I agree with this too.

    On a related matter (discussed in this thread and another one), I would not
    object if we dropped the distribution of R, except maybe for cocalc.com,
    but supporting the interface with the system R.
    As a data point: in my dept lots of people use R and lots use SageMath
    but none use the R in SageMath.

    > Best wishes,
    >
    > Eric.

    Emmanuel Charpentier

    unread,
    Oct 21, 2017, 2:53:58 PM10/21/17
    to sage-devel

    Wrong : a MITM attack could be used to redirect you to a doctored repository. Ditto, BTW, for a DNS attack... HTTPS offers *some* saveguards against that.. 

    --
    Emmanuel Charpentier

    Erik Bray

    unread,
    Oct 23, 2017, 5:16:05 AM10/23/17
    to sage-devel
    On Thu, Oct 19, 2017 at 5:19 PM, Emmanuel Charpentier
    <emanuel.c...@gmail.com> wrote:
    > Again : R is not only a software package but also an ecosystem. The 11638
    > (as of today) packages available to R users are a large part of R usefulness
    > to its users. So, "disabling downloads from CRAN" is *NOT* fine (to them, at
    > least...).

    I'm not saying it shouldn't be possible to install R packages at all,
    any less than it should be possible to install Python packages. My
    point here was that with Python, for example, one can manually
    download a package tarball or wheel from PyPI using, say, curl for
    example (maybe if they running on an air-gapped network this was done
    on a separate machine and sneaker-netted over, etc.). pip can then
    install from the manually downloaded package file.

    I don't know if the same is possible with R but you'd think it should
    be. However R installs packages the sequence still has to be
    something like

    1) Download package from CRAN
    2) Verify that package downloaded successfully (maybe it does this
    maybe it doesn't)
    3) Install the package

    So it should be possible to do steps 1 and 2 manually, and then skip
    straight to 3. Admittedly running R on an air-gapped network is
    probably not a situation the developers have in mind but I have very
    little doubt that the use case exists.

    > And, BTW : which is your vote ?

    My vote now is for a re-vote :)

    Erik Bray

    unread,
    Oct 23, 2017, 5:19:14 AM10/23/17
    to sage-devel
    On Thu, Oct 19, 2017 at 5:21 PM, Emmanuel Charpentier
    <emanuel.c...@gmail.com> wrote:
    >
    >
    > Le mercredi 18 octobre 2017 20:36:47 UTC+2, Jeroen Demeyer a écrit :
    >>
    >> On 2017-10-18 19:02, Emmanuel Charpentier wrote:
    >> > This option commits us to maintain (unnecessary and dangerous, IMHO)
    >> > Sage-specifc SSL patches at least in R, Python and pip
    >>
    >> Really? Which Sage-specific SSL patches does this require in Python and
    >> pip? *
    >
    >
    > From the Python license page :
    >
    > The modules hashlib, posix, ssl, crypt use the OpenSSL library for added
    > performance if made available by the operating system. Additionally, the
    > Windows and Mac OS X installers for Python may include a copy of the OpenSSL
    > libraries, so we include a copy of the OpenSSL license here:

    All of this is talking about optional functionality--either modules
    that are completely optional, or functionality within some modules
    that are optional. Python works just fine without SSL support.

    >> It seems to me that R is the only package causing us so much trouble
    >> with SSL.
    >
    >
    > No : *I* cause "so much trouble" with SSL issues because they become
    > important to R real-world usage, and I do not think that a non-communicating
    > R is useful in Sage. Python/pip SSL usage do not cause "so much trouble"
    > with SSL issues because the potential users of SSL functionalities are more
    > patient than I am...

    Ditto William on thanking you for causing this "trouble". I know this
    is a battle you've been fighting a long time and I really hope we can
    come to a resolution soon. In pointing out that SSL is not an issue
    where Python is concerned I hope it alleviates some of your concerns.
    Hopefully we can browbeat R into making the same acceptance (at least
    insofar as allowing offline package installs).

    Erik Bray

    unread,
    Oct 23, 2017, 5:21:34 AM10/23/17
    to sage-devel
    On Thu, Oct 19, 2017 at 10:56 PM, Thierry
    <sage-goo...@lma.metelu.net> wrote:
    > Hi,
    >
    > On Thu, Oct 19, 2017 at 08:07:19PM +0200, Luca De Feo wrote:
    >> |X| Yes, we should fully support OpenSSL now, and clarify the
    >> licensing issue.
    >>
    >> > the way our
    >> > "package manager" works allows to install an optional package without
    >> > having to rely on openssl (no https), we only rely on the computation of
    >> > sha1
    >>
    >> There you go for something crippled! https://shattered.io/
    >
    > sha256 is also supported by python-hashlib compiled without openssl
    > support, so we could/should easily move to using it in Sage "package
    > manager" (build/sage_bootstrap/tarball.py).

    +1 to switching to and/or adding support for SHA-256 hashes. As
    Thierry noted that are several SHA-256 implementations for Python, in
    fact, that don't rely in any way on OpenSSL or have problematic
    licenses.

    Erik Bray

    unread,
    Oct 23, 2017, 5:23:44 AM10/23/17
    to sage-devel
    That's not true. The whole point is that HTTP is ridiculously easy to
    hijack, so an attacker need not get anything accepted by Sage.

    It's an unlikely attack vector to be sure, but not impossible.

    Jeroen Demeyer

    unread,
    Oct 23, 2017, 5:24:18 AM10/23/17
    to sage-...@googlegroups.com
    On 2017-10-19 17:21, Emmanuel Charpentier wrote:
    > I do not think that a
    > non-communicating R is useful in Sage.

    A non-communicating R in Sage can be very useful if you are not using R
    in Sage at all (which is very likely the vast majority of Sage users).

    Erik Bray

    unread,
    Oct 23, 2017, 5:28:58 AM10/23/17
    to sage-devel
    Or, as you and I have both pointed out, you *are* using R but would
    prefer to use a different download method than the one built into R
    (which may still use HTTPS, especially if it's necessary to access
    CRAN at all, but that doesn't mean R has to be the one doing the
    downloading).

    Erik Bray

    unread,
    Oct 23, 2017, 5:39:29 AM10/23/17
    to sage-devel
    Doing a little more research on this*, it seems that R can in fact
    perfectly well install packages from compressed tarballs, etc.
    (through the UI, importantly). For downloading files it actually
    supports multiple download methods, one of which is to use libcurl
    (which can be built without HTTPS support; I've done so myself). But
    it also has a built-in "internal" method which may be part of the
    problem. If that can't be built without SSL support then that needs
    to be fixed. Finally, it also supports pointing to alternative
    package repositories which may or may not use HTTPS.

    Naturally, for the "average" user we *do* want to support transparent
    package installation from CRAN with HTTPS support which is why the
    OpenSSL issue needs to be resolved. But assuming that R *can* be
    built without SSL, and can work without SSL, that is acceptable too
    especially for users who know what they're doing and don't need the
    SSL support. I suggest that this be allowed to proceed, just with a
    loud warning (as Thierry suggested). I'll have a look at your patch
    to see what it's doing and if such a patch really needs to be
    maintained or not...


    * https://www.rdocumentation.org/packages/utils/versions/3.4.1/topics/INSTALL
    https://www.rdocumentation.org/packages/utils/versions/3.4.1/topics/install.packages
    https://www.rdocumentation.org/link/download.file?package=utils&version=3.4.1

    Emmanuel Charpentier

    unread,
    Oct 23, 2017, 5:57:19 AM10/23/17
    to sage-devel
    Dear Erik,

    Indeed, it *is* possible to install a manually downloaded package (not as straightforward as  the automatic download-and-install default method). But the problem isn't here :

    There are a large number of CRAN packages (11660 as of this morning). More and more of these packages have mutual dependencies, which are easily accounted for bu install.packages() but are a pain to deal with "manually".

    In fact, the problem (roughly) has same magnitude as the maintainance of your operating system : it *is* indeed possible to maintain a Unix/Linux installation using only tarballs conveyed to the system via sneakersnet, but it' certainly a) not fun and b) chronophage to the extreme...

    As it happened with Linux distributions, these intermutual dependencies, which started scarce and lightweight, are getting more and more important. My prediction (or prognostication, or oracle, if you wish...;-) is that they will reach the level of complexity of operating system distributions in an amount of time short enough for this problem to be of interest to all but the oldest (i. e. closest to retirement or death) of R users.

    I'm not old enough to not feel concerned ;-)...

    > And, BTW : which is your vote ?

    My vote now is for a re-vote :)

    Not possible,  given the terms of the vote (I can't pull an after-the-fact rule on the people who have already voted). But whatever the result is, there will remain the question of implementation, which might need a formal vote on sub-issues and methods

    --
    Emmanuel Charpentier

    Erik Bray

    unread,
    Oct 23, 2017, 6:17:06 AM10/23/17
    to sage-devel
    I understand all that, and the same is true of some Python packages as
    well (which is why sometimes Sage winds up having to add quite a few
    spkgs for Python packages--see for example the dependencies of Flask).
    But that's simply a matter that has to be dealt with, and people do
    (*I* do, even, in some cases).

    My point here is not that that that's a nice thing to foist on average
    users (it isn't). Just that it *should* be possible regardless,
    without patching or anything else. My concern is just why are we
    maintaining a patch just to be able to build R without SSL...

    Emmanuel Charpentier

    unread,
    Oct 23, 2017, 6:19:47 AM10/23/17
    to sage-devel
    Dear Jeroen

    ????? You are saying that X is useful if you don't use X at all ?

    Either :
    • I totally miss your point : I can't see how something can be useful if you don't use it at all. Or :
    • either :
      • you are a Zen master using padoxes to enlighten your student (and you failed to do so) ;
      • you are a dandy  enjoying paradoxes for the hell of it ;
      • you are preparing for a career in management or politics.
    Would you care to explain to a dentist of very little mind ?

    as for "which is very likely the vast majority of Sage users" : I do not know. I agree that I seem to be the only such voice on the sage-xxx lists. Which might be a strong argument for splitting R off Sage
     as already discussed on sage devel, and still discussed in this thread. But the last time this was discussed, the agreement seemed to be that R had to stay in Sage.

    And that does *not* solve "our" pip's lack of functionality, nor the (interesting) security problem raised by Thierry.

    --
    Emmanuel Charpentier


     

    Emmanuel Charpentier

    unread,
    Oct 23, 2017, 6:27:53 AM10/23/17
    to sage-devel

    ...which is *exactly* my point !

    Presently, the only reason ever given to this patch (which lifts upstream R's requirement on an https-enabled library) is to be able to build Sage without OpenSSL because it's not, in some eyes, GPL-compatible.
     
    I balk at the idea of maintaining this patch again and again for this issue, which can be solved either by depending on a systemwide openssl (but there goes our "batteries included" philosophy) or including OpenSSL (currently not possible for licensing reasons, but that should change).

    I also balk at the idea of shipping a crippled pip.

    Erik Bray

    unread,
    Oct 23, 2017, 8:32:03 AM10/23/17
    to sage-devel
    On Mon, Oct 23, 2017 at 12:27 PM, Emmanuel Charpentier
    I agree.

    > I also balk at the idea of shipping a crippled pip.

    It's not crippled if you don't need it to install from HTTPS which not
    everyone does.

    Okay. I think I see the problem here. We're talking about multiple
    issues simultaneously and some things are getting confused--some
    streams crossed.

    First: There is the *general* issue of whether or not these packages
    need any kind of SSL support in order to function. This is an issue
    *completely* independent from Sage (except insofar as we may or may
    not need to maintain some patches to ensure that they don't require
    SSL, but that is a separate issue that I will get to next). From the
    general standpoint of pip's design and functionality, it does not in
    any way require SSL to work. There was a time when it did, but only
    due to a bug where some module assumed the `ssl` module would always
    be importable which is not necessarily the case. There are completely
    legitimate reasons to either *explicitly* run pip without SSL support,
    and there are completely legitimate reasons to simply not need it at
    all or not care. Therefore pip should be able to work (and does)
    without SSL support, without patching. The same should be true for R,
    and if this is not the case (and I'm not convinced it isn't), then it
    should be for the same or similar reasons as pip. Again, this is
    purely from an abstract standpoint in how that software should,
    ideally, be designed.

    Second: The question is, should it be possible to build/install Sage
    (specifically the distribution) without an SSL library? There are two
    sub-problems that arise from this question:

    a) If we agree that that should at least be *possible* (I think it
    should be, even if not the default) to build Sage without SSL, then
    all of Sage's dependencies should be able to build without SSL. This
    includes pip and R. If these packages satisfy the first issue then
    there is no problem; nothing to discuss. But your assertion is that R
    can't even be built without SSL support unless it is patched. I would
    consider that an upstream deficiency that should be addressed
    aggressively just in principle if nothing else.

    b) If we agree that it should not be possible to build Sage
    without SSL support (which I think is a poor decision, due to the
    first point) then there is no issue about needing to maintain a patch
    to remove SSL requirement from any package. But there is the separate
    issue of having this added dependency that may be inconvenient for
    some people.

    Third: There's a question of who the audience is. For the "casual"
    user who does not care about building things or compiling things or
    licenses or patches and just wants to run Sage, and be able to install
    packages from pip and CRAN (should the need even arise) this should
    all Just Work. From that point of view, they should be installing
    Sage from pre-built binaries, and those binaries should include the
    necessary SSL support, period, end of story. This requirement is
    independent of how SSL support is achieved, what that means for
    developers or maintainers of binary packages, etc.

    Does that help at all?

    I think we're nearly in agreement, except maybe about the extent to
    which SSL support should be mandated in all cases.

    Best,
    Erik

    Erik Bray

    unread,
    Oct 23, 2017, 8:43:18 AM10/23/17
    to sage-devel
    On Mon, Oct 23, 2017 at 2:31 PM, Erik Bray <erik....@gmail.com> wrote:
    > The same should be true for R,
    > and if this is not the case (and I'm not convinced it isn't)

    This part I take back. I see now that in R's configure it really does
    refuse to proceed if it doesn't find the right libcurl with SSL
    support. It's not possible to disable since it's now the default
    download method. This is silly though, since other download methods
    still work. It should be possible to disable the requirement at
    configure time and fallback to a different default. It's a shame we
    require a patch for this for now but I can help push for an upstream
    fix to this if need be, because I think it's fairly silly.

    Emmanuel Charpentier

    unread,
    Oct 23, 2017, 9:19:51 AM10/23/17
    to sage-devel

    Unless I'm mistaken, pip won't  instamm from Pipy if https isn't enabled. Isn't that an important source of Python modules ?

    Okay.  I think I see the problem here.  We're talking about multiple
    issues simultaneously and some things are getting confused--some
    streams crossed.

    Indeed : SSL is a *general* tool, and its absence affects *many* things.
     
    First: There is the *general* issue of whether or not these packages
    need any kind of SSL support in order to function.  This is an issue
    *completely* independent from Sage (except insofar as we may or may
    not need to maintain some patches to ensure that they don't require
    SSL, but that is a separate issue that I will get to next).  From the
    general standpoint of pip's design and functionality, it does not in
    any way require SSL to work.  There was a time when it did, but only
    due to a bug where some module assumed the `ssl` module would always
    be importable which is not necessarily the case.  There are completely
    legitimate reasons to either *explicitly* run pip without SSL support,
    and there are completely legitimate reasons to simply not need it at
    all or not care.

     Okay : we have another confusion here : you want to *run* pip (or R) without SSL support. The problem (at least with R) is that we shouldn't be able to *build* R without SSL support.

    Therefore pip should be able to work (and does)
    without SSL support, without patching.  The same should be true for R,
    and if this is not the case (and I'm not convinced it isn't),

    It isn't : I've spent CPU *days* (two or three upgrades ago, can't exactly rember when) to prove that neither GnuTLS nor Netscape's library allow to compile an unmodified upstream R. You are, of course, welcome to check.
     
    then it
    should be for the same or similar reasons as pip.  Again, this is
    purely from an abstract standpoint in how that software should,
    ideally, be designed.

    Second: The question is, should it be possible to build/install Sage
    (specifically the distribution) without an SSL library?  There are two
    sub-problems that arise from this question:

        a) If we agree that that should at least be *possible* (I think it
    should be, even if not the default) to build Sage without SSL, then
    all of Sage's dependencies should be able to build without SSL.

    I think that this is where we disagree : I tend to follow upstream's authors : see below.
     
    This
    includes pip and R.  If these packages satisfy the first issue then
    there is no problem; nothing to discuss.  But your assertion is that R
    can't even be built without SSL support unless it is patched.

    See above for the source of this assertion. The R Core Team (roughly the author of upstream R) has decided that R *must* support https-enabled downloads. It has taken steps to ensure that such support *is* available at compile time. Therefore, we have to patch "our" R to lift this enforcement.
     
    I would
    consider that an upstream deficiency that should be addressed
    aggressively just in principle if nothing else.

    Again, you are welcome to try to change R Core Team's mind. I doubt you can manage it : they seem to think that http mirrors are too insecure to be trusted, and decided to enforce https *ability* in recent versions of R. They probably won't be impressed by a couple of mathematicians with pseudo-legal licensing issues.

    And, BTW, R is GPL-licensed... Stop brandishing that strawman. Thank you.

        b) If we agree that it should not be possible to build Sage
    without SSL support (which I think is a poor decision, due to the
    first point) then there is no issue about needing to maintain a patch
    to remove SSL requirement from any package.

    Indeed...
     
     But there is the separate
    issue of having this added dependency that may be inconvenient for
    some people.

    The same problem applies the the same people wanting to use upstream's R.
     
    Third: There's a question of who the audience is.  For the "casual"
    user who does not care about building things or compiling things or
    licenses or patches and just wants to run Sage, and be able to install
    packages from pip and CRAN (should the need even arise) this should
    all Just Work.  From that point of view, they should be installing
    Sage from pre-built binaries, and those binaries should include the
    necessary SSL support, period, end of story.

    Agreed. Fully.

    I'd like to add that the "average user" of R seems to be be much more prone to install external packages than the "average user" of Sage : CRAN is a very large part of the "R ecosystem".

    From a larger point of view : for an applied statistician, as judged by their respective presences in scholarly publications, R is the dominant platform for code supporting the papers, with Matlab being a (distant) second ; Python has yet to make a marked dent ; SAS, which was dominant about 20 years ago, seems to slide in oblivion.

    So, maintaining compatibility with the R-and--CRAN ecosystem is important.

    Being a (very) casual user of Python, I am not aware of the nature and importance of the Python ecosystem. But I'd be surprised if it pip-oriented repositories didn't become as important to "the Python users community" as CRAN is to the "R users community...
     
    This requirement is
    independent of how SSL support is achieved, what that means for
    developers or maintainers of binary packages, etc.

    Indeed. You're welcome to try and find an unencubered SSL library that is swallowed by upstream's R as acceptable for https support, and to propose the patches necessary to its use to the R Core Team. I didn't succeed, but, hey, I'm just a dentist that tries to *use* R...

    Does that help at all?

    Yes.

    Emmanuel Charpentier

    unread,
    Oct 23, 2017, 9:28:11 AM10/23/17
    to sage-devel


    Le lundi 23 octobre 2017 14:43:18 UTC+2, Erik Bray a écrit :
    On Mon, Oct 23, 2017 at 2:31 PM, Erik Bray <erik....@gmail.com> wrote:
    > The same should be true for R,
    > and if this is not the case (and I'm not convinced it isn't)

    This part I take back.  I see now that in R's configure it really does
    refuse to proceed if it doesn't find the right libcurl with SSL
    support.  It's not possible to disable since it's now the default
    download method.  This is silly though, since other download methods
    still work.

    Probably not as silly as it seems : it pushes users towards SSL downloading (for good reasons, I think), while leaving the CRAN mirrors managers some time to adapt (i. e. getting an SSL certificate from an acknowledged CA).

    I think that a second step is the programmed disappearance of the http access to CRAN mirrors. But we're not yet there.
     
    It should be possible to disable the requirement at
    configure time and fallback to a different default.  It's a shame we
    require a patch for this for now but I can help push for an upstream
    fix to this if need be, because I think it's fairly silly.

    Could you explain why ? I think that the move towards authentication of the download sources is a Good Idea (TM), but I may be wrong. In any case, the "silliness" of this is nor obvious to my dentist's eyes...

    --
    Emmanuel Charpentier

    Erik Bray

    unread,
    Oct 23, 2017, 9:39:25 AM10/23/17
    to sage-devel
    On Mon, Oct 23, 2017 at 3:19 PM, Emmanuel Charpentier
    *sigh* I feel like I've been over this but pip works without PyPI and
    this is *by design*.

    >> Okay. I think I see the problem here. We're talking about multiple
    >> issues simultaneously and some things are getting confused--some
    >> streams crossed.
    >
    >
    > Indeed : SSL is a *general* tool, and its absence affects *many* things.
    >
    >>
    >> First: There is the *general* issue of whether or not these packages
    >> need any kind of SSL support in order to function. This is an issue
    >> *completely* independent from Sage (except insofar as we may or may
    >> not need to maintain some patches to ensure that they don't require
    >> SSL, but that is a separate issue that I will get to next). From the
    >> general standpoint of pip's design and functionality, it does not in
    >> any way require SSL to work. There was a time when it did, but only
    >> due to a bug where some module assumed the `ssl` module would always
    >> be importable which is not necessarily the case. There are completely
    >> legitimate reasons to either *explicitly* run pip without SSL support,
    >> and there are completely legitimate reasons to simply not need it at
    >> all or not care.
    >
    >
    > Okay : we have another confusion here : you want to *run* pip (or R)
    > without SSL support. The problem (at least with R) is that we shouldn't be
    > able to *build* R without SSL support.

    Yes, we should be able to. We can't currently (without patch), but we
    should be able to.

    >> Therefore pip should be able to work (and does)
    >> without SSL support, without patching. The same should be true for R,
    >> and if this is not the case (and I'm not convinced it isn't),
    >
    >
    > It isn't : I've spent CPU *days* (two or three upgrades ago, can't exactly
    > rember when) to prove that neither GnuTLS nor Netscape's library allow to
    > compile an unmodified upstream R. You are, of course, welcome to check.

    I don't mean a different SSL library. I mean without *any*.

    >> then it
    >> should be for the same or similar reasons as pip. Again, this is
    >> purely from an abstract standpoint in how that software should,
    >> ideally, be designed.
    >>
    >> Second: The question is, should it be possible to build/install Sage
    >> (specifically the distribution) without an SSL library? There are two
    >> sub-problems that arise from this question:
    >>
    >> a) If we agree that that should at least be *possible* (I think it
    >> should be, even if not the default) to build Sage without SSL, then
    >> all of Sage's dependencies should be able to build without SSL.
    >
    >
    > I think that this is where we disagree : I tend to follow upstream's authors
    > : see below.

    I don't know. Do we really disagree? Or are you just agreeing with
    the upstream authors?

    >> This
    >> includes pip and R. If these packages satisfy the first issue then
    >> there is no problem; nothing to discuss. But your assertion is that R
    >> can't even be built without SSL support unless it is patched.
    >
    >
    > See above for the source of this assertion. The R Core Team (roughly the
    > author of upstream R) has decided that R *must* support https-enabled
    > downloads. It has taken steps to ensure that such support *is* available at
    > compile time. Therefore, we have to patch "our" R to lift this enforcement.

    This is wrong. R works without *any* kind of download capability
    whatsoever. This includes installing R packages (which I agree is
    important functionality). But that doesn't mean R has to be able to
    download them. It's not like starting up R is going to fail if it
    can't download anything (if that were the case you could never use it
    offline). And indeed it isn't the case. They're simply refusing to
    modularize their functionality for some reason.

    >> I would
    >> consider that an upstream deficiency that should be addressed
    >> aggressively just in principle if nothing else.
    >
    >
    > Again, you are welcome to try to change R Core Team's mind. I doubt you can
    > manage it : they seem to think that http mirrors are too insecure to be
    > trusted, and decided to enforce https *ability* in recent versions of R.
    > They probably won't be impressed by a couple of mathematicians with
    > pseudo-legal licensing issues.

    They are welcome to require HTTPS for CRAN and any other public
    mirrors. I would insist on it too. But that doesn't mean there's no
    good reason for plain HTTP networks (e.g. on a closed network). Or,
    as I feel like I've said repeatedly and am perhaps still unclear about
    somehow: It's not necessary for it to have *any* kind of download
    capability much less with/without SSL.

    > And, BTW, R is GPL-licensed... Stop brandishing that strawman. Thank you.

    I'm not sure what strawman you're referring to here but it's probably
    not one I've used. I don't personally care about licensing issues at
    all even if I probably should :)

    >> b) If we agree that it should not be possible to build Sage
    >> without SSL support (which I think is a poor decision, due to the
    >> first point) then there is no issue about needing to maintain a patch
    >> to remove SSL requirement from any package.
    >
    >
    > Indeed...
    >
    >>
    >> But there is the separate
    >> issue of having this added dependency that may be inconvenient for
    >> some people.
    >
    >
    > The same problem applies the the same people wanting to use upstream's R.

    It's not a problem if they're installing from binaries, which most
    are? Same for Sage.

    >> Third: There's a question of who the audience is. For the "casual"
    >> user who does not care about building things or compiling things or
    >> licenses or patches and just wants to run Sage, and be able to install
    >> packages from pip and CRAN (should the need even arise) this should
    >> all Just Work. From that point of view, they should be installing
    >> Sage from pre-built binaries, and those binaries should include the
    >> necessary SSL support, period, end of story.
    >
    >
    > Agreed. Fully.
    >
    > I'd like to add that the "average user" of R seems to be be much more prone
    > to install external packages than the "average user" of Sage : CRAN is a
    > very large part of the "R ecosystem".

    Agreed, but this still goes back to my first point which is the
    "average user" is also not everyone, and there are legitimate reasons
    to not care about the R library itself handling package downloads
    (regardless of how that affects Sage).

    > From a larger point of view : for an applied statistician, as judged by
    > their respective presences in scholarly publications, R is the dominant
    > platform for code supporting the papers, with Matlab being a (distant)
    > second ; Python has yet to make a marked dent ; SAS, which was dominant
    > about 20 years ago, seems to slide in oblivion.
    >
    > So, maintaining compatibility with the R-and--CRAN ecosystem is important.

    This is all well understood as to the need for this functionality in a
    normal install of R, but it's still irrelevant to the general point
    that it's not essential functionality merely for building and
    installing R in all cases.

    > Being a (very) casual user of Python, I am not aware of the nature and
    > importance of the Python ecosystem. But I'd be surprised if it pip-oriented
    > repositories didn't become as important to "the Python users community" as
    > CRAN is to the "R users community...

    They are and were before "pip" even came along. I used to manage an
    internal PyPI instance, for example, for proprietary Python packages.
    This is irrelevant though.

    >> This requirement is
    >> independent of how SSL support is achieved, what that means for
    >> developers or maintainers of binary packages, etc.
    >
    >
    > Indeed. You're welcome to try and find an unencubered SSL library that is
    > swallowed by upstream's R as acceptable for https support, and to propose
    > the patches necessary to its use to the R Core Team. I didn't succeed, but,
    > hey, I'm just a dentist that tries to *use* R...

    I don't care about that, and am not sure how I indicated that I am.
    I'm also not sure who accused you of being a "dentist" (also I know
    some very smart dentists...); I'm sorry if you feel on the defensive
    here because again I think we're almost entirely in agreement. I want
    what you want, for the same reasons, just from a
    different...perspective? I'm not sure what the disconnect is here
    except that you and the R developers seem to believe that it should
    be literally impossible to even build their software unless it has
    even the *capability* of connecting to CRAN, even though that's not
    necessary for any of its core functionality. This is very strange to
    me, even if we can all agree that CRAN is important to its ecosystem
    and typical user-base.

    Best,
    Erik

    Emmanuel Charpentier

    unread,
    Oct 23, 2017, 9:40:46 AM10/23/17
    to sage-devel
    My vote :


    |X| Yes, we should fully support OpenSSL now, and clarify the licensing issue.

    BTW : the vote closes in about 20 minutes. This is your last chance to take back any "too hasty" votes.

    --
    Emmanuel Charpentier

    Erik Bray

    unread,
    Oct 23, 2017, 9:44:09 AM10/23/17
    to sage-devel
    On Mon, Oct 23, 2017 at 3:28 PM, Emmanuel Charpentier
    <emanuel.c...@gmail.com> wrote:
    >> It should be possible to disable the requirement at
    >> configure time and fallback to a different default. It's a shame we
    >> require a patch for this for now but I can help push for an upstream
    >> fix to this if need be, because I think it's fairly silly.
    >
    >
    > Could you explain why ? I think that the move towards authentication of the
    > download sources is a Good Idea (TM), but I may be wrong. In any case, the
    > "silliness" of this is nor obvious to my dentist's eyes...

    Perhaps this should clarify: If the CRAN service is switched to using
    HTTPS, then it can't be accessed without HTTPS. If the user tries to
    access the site with software that doesn't have HTTPS support then
    they are prevented from performing insecure downloads, QED.

    In other words, the security here is being provided by the service.
    The client is free to decide whether or not they wish to implement
    their end in order to be able to access the service.

    Julien Puydt

    unread,
    Oct 23, 2017, 9:50:01 AM10/23/17
    to sage-...@googlegroups.com


    Le 23/10/2017 à 15:40, Emmanuel Charpentier a écrit :
    > BTW : the vote closes in about 20 minutes. This is your last chance to
    > take back any "too hasty" votes.

    My vote: no openSSL now - wait until the license issues are solved

    Snark on #sagemath
    It is loading more messages.
    0 new messages