Sage requires SSL

85 views
Skip to first unread message

Jeroen Demeyer

unread,
Mar 20, 2012, 10:12:47 AM3/20/12
to sage-devel
Sage requires SSL, otherwise Python's hashlib module won't work.

I think this is a non-trivial dependency of Sage. Is it known that this
is a dependency? Should it be documented in the install guide?

leif

unread,
Mar 20, 2012, 12:16:46 PM3/20/12
to sage-devel
If it's not, surely.

It frequently pops up on the mailing lists, e.g. from people who don't
have OpenSSL installed on Linux.


-leif

Jeroen Demeyer

unread,
Mar 20, 2012, 12:27:37 PM3/20/12
to sage-...@googlegroups.com

I'm slightly confused because we had some discussion on this mailing
list about removing GNUTLS and requiring OpenSSL. But if we already
require OpenSSL, there was not much to discuss...

leif

unread,
Mar 20, 2012, 12:39:23 PM3/20/12
to sage-devel
Does *hashlib* really require SSL?

I thought HTTPS (which requires SSL of course) wasn't strictly needed
in Sage, but since it is omnipresent (many Python packages use it), it
doesn't make sense to not have it. I.e., people frequently happen to
notice they don't have _ssl *after* they've built Sage's Python
package without SSL support.

(Does the Python 2.7.x build *fail* without it?)

IIRC there even used to be an optional (most probably meanwhile
outdated) OpenSSL spkg.


I have no idea what the situation is on [Open]Solaris and MacOS X...


-leif

Jason Grout

unread,
Mar 20, 2012, 12:43:01 PM3/20/12
to sage-...@googlegroups.com

I've never had a problem with having hashlib on OSX 10.6.8, and I don't
think I installed any new libraries to get it. I think I concluded a
long time ago that this was a non-issue on OSX (10.6).

Jason

Jeroen Demeyer

unread,
Mar 20, 2012, 12:53:12 PM3/20/12
to sage-...@googlegroups.com
Okay, maybe Python's hashlib builds if you don't have OpenSSL on your
system. However, it *fails* if you do have an OpenSSL library but it
can't be linked.

I consider this to be a bug in Python. Put some random file at
/usr/lib/libssl.so and Python thinks you have OpenSSL installed. This
will cause the build of hashlib to fail.

Context: I'm trying to "cross-compile" a 32-bit Sage on a 64-bit Linux
system. The Python installer looks at the 64-bit libraries to determine
its linker flags.

John Cremona

unread,
Mar 20, 2012, 1:05:41 PM3/20/12
to sage-...@googlegroups.com
On 20 March 2012 16:39, leif <not.r...@online.de> wrote:

>
> I thought HTTPS (which requires SSL of course) wasn't strictly needed
> in Sage, but since it is omnipresent (many Python packages use it), it
> doesn't make sense to not have it.  I.e., people frequently happen to
> notice they don't have _ssl *after* they've built Sage's Python
> package without SSL support.

Indeed: every time I build my working copy of Sage (not counting test
version here) I have to do first sage -i openssl and then sage -f
python to rebuild python. Otherwise some mercurial stuff does not
work (complains of not having _ssl, I think). The sort of thing which
does not work is pull/push requests froma remote repository.

This is a real nuisance, and I have long wanted a way to get this
built when I first build Sage, and not as an afterthought. If the
current discussion will help that, I would be grateful! (Also if
anyone can say what magic would make this work with Sage as currently
organised.)

John

Keshav Kini

unread,
Mar 20, 2012, 1:13:13 PM3/20/12
to sage-...@googlegroups.com
John Cremona <john.c...@gmail.com> writes:
> Indeed: every time I build my working copy of Sage (not counting test
> version here) I have to do first sage -i openssl and then sage -f
> python to rebuild python. Otherwise some mercurial stuff does not
> work (complains of not having _ssl, I think). The sort of thing which
> does not work is pull/push requests froma remote repository.

The git spkg I just made is unable to communicate with repositories over
HTTP (unless you just happen to have libcurl and libexpat installed
systemwide - I didn't want to package them since libcurl's source code
is almost as large again as git's; I didn't look at libexpat).

Would this be a problem for you once we switch to git? What do you use
Sage's Mercurial for which requires pushing and pulling to/from a remote
repository?

-Keshav

----
Join us in #sagemath on irc.freenode.net !

Harald Schilly

unread,
Mar 20, 2012, 1:14:26 PM3/20/12
to sage-...@googlegroups.com


On Tuesday, March 20, 2012 6:05:41 PM UTC+1, John Cremona wrote:
> … openssl …

the basic problem is this clause in their license:
"""
All advertising materials mentioning features or use of this
 *    software must display the following acknowledgment:
"This product includes software developed by the OpenSSL Project
 * for use in the OpenSSL Toolkit. (http://www.openssl.org/)"
""" 

That's the same issue as with gnuplot. Maybe it would be helpful to email those developers, what they say about this and us including openssl in sage.

H

William Stein

unread,
Mar 20, 2012, 1:17:02 PM3/20/12
to sage-...@googlegroups.com

He uses

http://code.google.com/p/lmfdb/issues/list

and

http://code.google.com/p/purplesage/

which both *require* https for pushing (though not for pulling if you
url hack).

William

>
> -Keshav
>
> ----
> Join us in #sagemath on irc.freenode.net !
>

> --
> To post to this group, send an email to sage-...@googlegroups.com
> To unsubscribe from this group, send an email to sage-devel+...@googlegroups.com
> For more options, visit this group at http://groups.google.com/group/sage-devel
> URL: http://www.sagemath.org

--
William Stein
Professor of Mathematics
University of Washington
http://wstein.org

William Stein

unread,
Mar 20, 2012, 1:19:18 PM3/20/12
to sage-...@googlegroups.com
On Tue, Mar 20, 2012 at 1:14 PM, Harald Schilly
<harald....@gmail.com> wrote:
>
>
> On Tuesday, March 20, 2012 6:05:41 PM UTC+1, John Cremona wrote:
>> … openssl …
>
> the basic problem is this clause in their license:
> """
>
> All advertising materials mentioning features or use of this
> * software must display the following acknowledgment:
>
> "This product includes software developed by the OpenSSL Project
>  * for use in the OpenSSL Toolkit. (http://www.openssl.org/)"
> """
>
> That's the same issue as with gnuplot.

? Can you provide a reference? I thought the mainissue with
gnuplot's weird GPL-incomplate license is that only the original
authors are allowed to initiate distribution of modified versions of
the source; anybody else must distribute the unchanged sources +
patches.

> Maybe it would be helpful to email
> those developers, what they say about this and us including openssl in sage.

Maybe we can include openssl in Sage as long as no GPL'd components of
Sage binary link with it... but that is a fine line. Linking against
it as a system library is less of a fine line.

>
> H

Keshav Kini

unread,
Mar 20, 2012, 1:14:56 PM3/20/12
to sage-...@googlegroups.com
Keshav Kini <kesha...@gmail.com> writes:
> Would this be a problem for you once we switch to git? What do you use
> Sage's Mercurial for which requires pushing and pulling to/from a remote
> repository?

Sorry - I mean, *over HTTP(S)*.

John Cremona

unread,
Mar 20, 2012, 1:20:27 PM3/20/12
to sage-...@googlegroups.com
On 20 March 2012 17:17, William Stein <wst...@gmail.com> wrote:
> On Tue, Mar 20, 2012 at 1:13 PM, Keshav Kini <kesha...@gmail.com> wrote:
>> John Cremona <john.c...@gmail.com> writes:
>>> Indeed:  every time I build my working copy of Sage (not counting test
>>> version here) I have to do first sage -i openssl and then sage -f
>>> python to rebuild python.  Otherwise some mercurial stuff does not
>>> work (complains of not having _ssl, I think).  The sort of thing which
>>> does not work is pull/push requests froma remote repository.
>>
>> The git spkg I just made is unable to communicate with repositories over
>> HTTP (unless you just happen to have libcurl and libexpat installed
>> systemwide - I didn't want to package them since libcurl's source code
>> is almost as large again as git's; I didn't look at libexpat).
>>
>> Would this be a problem for you once we switch to git? What do you use
>> Sage's Mercurial for which requires pushing and pulling to/from a remote
>> repository?
>
> He uses
>
>   http://code.google.com/p/lmfdb/issues/list
>
> and
>
>   http://code.google.com/p/purplesage/

Correct: it's to use google code.

I don't mind the openssl not beeing included by default for license
reasons. But it would be good if I could (say by setting an
environment variable or in .sagerc file or similar) record my choice
to include openssl *once and for all*, so that whenever I build Sage
it has it in it.

John

Harald Schilly

unread,
Mar 20, 2012, 1:27:47 PM3/20/12
to sage-...@googlegroups.com
On Tuesday, March 20, 2012 6:19:18 PM UTC+1, William wrote:

?  Can you provide a reference?  


ah, you are right, sorry. i guess i have mixed this up with some other custom license.

Keshav Kini

unread,
Mar 20, 2012, 1:28:52 PM3/20/12
to sage-...@googlegroups.com
William Stein <wst...@gmail.com> writes:
> He uses
>
> http://code.google.com/p/lmfdb/issues/list
>
> and
>
> http://code.google.com/p/purplesage/
>
> which both *require* https for pushing (though not for pulling if you
> url hack).

I guess Purple Sage will have to switch to git when Sage does, since
their code bases are deeply connected. lmfdb, which I hadn't heard of
before, looks to be a separate codebase (like sagenb is). Is it
switching to git too? If not, then do we still need to ship Mercurial in
Sage to allow people to hack on lmfdb, or will they need to start using
systemwide versions of Mercurial?

Harald Schilly

unread,
Mar 20, 2012, 1:35:09 PM3/20/12
to sage-...@googlegroups.com
On Tuesday, March 20, 2012 6:28:52 PM UTC+1, Keshav Kini wrote:

... or will they need to start using
systemwide versions of Mercurial?


systemwide hg is just fine, most of us use it this way anyways. it's just not available on some machines sometimes …

leif

unread,
Mar 20, 2012, 1:50:00 PM3/20/12
to sage-devel
On 20 Mrz., 18:05, John Cremona <john.crem...@gmail.com> wrote:
Well, I wouldn't mind having openssl a /standard/ spkg, which only
gets installed if SSL is missing or the present library isn't
functional (we should test that first), although it is currently 3.1
MB (openssl-1.0.0.p0.spkg), which isn't /that/ small. But perhaps we
could drop the GNUTLS spkg.


In the long run, I think we should change our prerequisites script and
make a couple of currently-standard spkgs optional, and/or offer
different tarballs (e.g. with and without GCC, or Fortran).

I know we don't want [to require] internet connections *during* the
build, but to me it doesn't make much of a difference whether the
prerequisite script tells (e.g. Linux users) "use your package manager
to install package foo, bar and baz first", or says "Please install
the optional packages x and y first, or choose another tarball.", or
queries the user whether he wants to automatically download these
now. Perhaps we should also provide some prerequisites checking
script *separately*, such that users could check *in advance* what
they'd have to download/install or /which/ Sage tarball they should
choose to download.

Since #10492 [1] now *generates* the Makefile (previously spkg/
standard/deps) anyway, we could easily add dependencies (i.e., include
usually "optional" spkgs like openssl, but also iconv etc.) on-the-
fly.


-leif

[1] http://trac.sagemath.org/sage_trac/ticket/10492

Jonathan Bober

unread,
Mar 20, 2012, 1:52:05 PM3/20/12
to sage-...@googlegroups.com
On Tue, Mar 20, 2012 at 10:28 AM, Keshav Kini <kesha...@gmail.com> wrote:
> I guess Purple Sage will have to switch to git when Sage does, since
> their code bases are deeply connected. lmfdb, which I hadn't heard of
> before, looks to be a separate codebase (like sagenb is). Is it
> switching to git too? If not, then do we still need to ship Mercurial in
> Sage to allow people to hack on lmfdb, or will they need to start using
> systemwide versions of Mercurial?
>
> -Keshav

Actually, the code base of psage is not "deeply connected" to sage. It
really is just a python library that more or less depends on "all" of
sage being present. (Or at least a lot of sage.) It could probably
switch to git without much of an issue. As for the LMFDB, I haven't
really thought about it at all. I would like it if it used git, but
I'm not going push for people to switch without much reason.

I don't think Sage _needs_ to ship mercurial if it switches to git,
but it would be nice for me, and probably other, if it did. As I've
noted before, mercurial seems to be a PITA to build from source when I
don't have root access, and the last time I tried to do it I gave up
and decided to use Sage's mercurial. (git was easy to build, though.)

In this (still hypothetical) situation, since most of the work has
already been done, mercurial could probably also just stick around as
an optional package, at least until it breaks from lack of
maintenance.

leif

unread,
Mar 20, 2012, 1:57:59 PM3/20/12
to sage-devel
On 20 Mrz., 18:14, Harald Schilly <harald.schi...@gmail.com> wrote:
> On Tuesday, March 20, 2012 6:05:41 PM UTC+1, John Cremona wrote:
> > … openssl …
>
> the basic problem is this clause in their license:
> """
>
> All advertising materials mentioning features or use of this
>  *    software must display the following acknowledgment:
>
> "This product includes software developed by the OpenSSL Project
>  * for use in the OpenSSL Toolkit. (http://www.openssl.org/)"
> """

Well, unless we /mention/ OpenSSL or its features (in "ads", btw.), we
don't have to include that acknowledgement.

That's what it says.


-leif

Keshav Kini

unread,
Mar 20, 2012, 2:01:55 PM3/20/12
to sage-...@googlegroups.com
Jonathan Bober <jwb...@gmail.com> writes:
> Actually, the code base of psage is not "deeply connected" to sage. It
> really is just a python library that more or less depends on "all" of
> sage being present. (Or at least a lot of sage.) It could probably
> switch to git without much of an issue. As for the LMFDB, I haven't
> really thought about it at all. I would like it if it used git, but
> I'm not going push for people to switch without much reason.

I see. Makes sense.

> I don't think Sage _needs_ to ship mercurial if it switches to git,
> but it would be nice for me, and probably other, if it did. As I've
> noted before, mercurial seems to be a PITA to build from source when I
> don't have root access, and the last time I tried to do it I gave up
> and decided to use Sage's mercurial. (git was easy to build, though.)

By the way, I've noticed many people saying "Sage should provide X as an
SPKG because it is difficult for me to build X otherwise, not that Sage
really has much of a dependency on X." I think this is exactly the
problem that switching to Prefix would solve. For such packages, you
could just grab an ebuild from the Gentoo people and stick it in Sage,
rather than needing a Sage developer to create an SPKG for you.

> In this (still hypothetical) situation, since most of the work has
> already been done, mercurial could probably also just stick around as
> an optional package, at least until it breaks from lack of
> maintenance.

Fair enough.

leif

unread,
Mar 20, 2012, 2:10:58 PM3/20/12
to sage-devel
On 20 Mrz., 18:20, John Cremona <john.crem...@gmail.com> wrote:
> I don't mind the openssl not beeing included by default for license
> reasons.  But it would be good if I could (say by setting an
> environment variable or in .sagerc file or similar) record my choice
> to include openssl *once and for all*, so that whenever I build Sage
> it has it in it.

Well, you could call a trivial shell script (few lines) each time
before you build a Sage version from scratch, which just brings the
openssl package into place and hacks the deps file to make the Python
spkg depend on it.

I'm not sure whether you could right now also just call (presumably an
analogue of) "./sage -i openssl" *before* you run 'make'.


-leif

leif

unread,
Mar 20, 2012, 2:15:51 PM3/20/12
to sage-devel
Or the "default" / available version is 1.4.* ... 8/

abort: requirement 'dotencode' not supported!


-leif

William Stein

unread,
Mar 20, 2012, 2:20:09 PM3/20/12
to sage-...@googlegroups.com
On Tue, Mar 20, 2012 at 2:01 PM, Keshav Kini <kesha...@gmail.com> wrote:
> Jonathan Bober <jwb...@gmail.com> writes:
>> Actually, the code base of psage is not "deeply connected" to sage. It
>> really is just a python library that more or less depends on "all" of
>> sage being present. (Or at least a lot of sage.) It could probably
>> switch to git without much of an issue. As for the LMFDB, I haven't
>> really thought about it at all. I would like it if it used git, but
>> I'm not going push for people to switch without much reason.
>
> I see. Makes sense.
>
>> I don't think Sage _needs_ to ship mercurial if it switches to git,
>> but it would be nice for me, and probably other, if it did. As I've
>> noted before, mercurial seems to be a PITA to build from source when I
>> don't have root access, and the last time I tried to do it I gave up
>> and decided to use Sage's mercurial. (git was easy to build, though.)
>
> By the way, I've noticed many people saying "Sage should provide X as an
> SPKG because it is difficult for me to build X otherwise, not that Sage
> really has much of a dependency on X." I think this is exactly the
> problem that switching to Prefix would solve.

Why don't such people just use Prefix already then? If Prefix solves
this problem why do we have to deal with it? This might be a
rhetorical question....

> For such packages, you
> could just grab an ebuild from the Gentoo people and stick it in Sage,
> rather than needing a Sage developer to create an SPKG for you.
>
>> In this (still hypothetical) situation, since most of the work has
>> already been done, mercurial could probably also just stick around as
>> an optional package, at least until it breaks from lack of
>> maintenance.
>
> Fair enough.
>
> -Keshav
>
> ----
> Join us in #sagemath on irc.freenode.net !
>

Keshav Kini

unread,
Mar 20, 2012, 2:29:08 PM3/20/12
to sage-...@googlegroups.com
William Stein <wst...@gmail.com> writes:
> On Tue, Mar 20, 2012 at 2:01 PM, Keshav Kini <kesha...@gmail.com> wrote:
> Why don't such people just use Prefix already then? If Prefix solves
> this problem why do we have to deal with it? This might be a
> rhetorical question....

I assume you mean, use Prefix independently of Sage, since "official"
Sage doesn't use Prefix yet, and unofficial Prefix-using versions of
Sage (i.e. lmonade) are experimental and don't pass tests.

1. Prefix is difficult to set up and install. There's no easy installer
or single command you can run to do it.
2. Prefix bootstraps almost everything from scratch, which is not worth
it for many people.
3. Maybe people don't know about Prefix, and as Sage developers their
main exposure to a mini-distribution sitting in a folder is Sage
itself. Or maybe they don't want to have *two* mini-distributions
sitting in folders on their computer.

Addressing the first two is, I think, one of Burcin's main goals for
lmonade, and he's made quite some progress on both of them. lmonade
installs with a single command, and uses the system toolchain (afaik).
But in upstream, Sage-unrelated Gentoo Prefix, those two points are
still big problems for the use case I mention.

leif

unread,
Mar 20, 2012, 2:53:43 PM3/20/12
to sage-devel
On 20 Mrz., 19:10, leif <not.rea...@online.de> wrote:
> Well, you could call a trivial shell script (few lines) each time
> before you build a Sage version from scratch, which just brings the
> openssl package into place and hacks the deps file to make the Python
> spkg depend on it.
>
> I'm not sure whether you could right now also just call (presumably an
> analogue of) "./sage -i openssl" *before* you run 'make'.

Looks as if you currently (as of Sage 5.0.beta8) could do

tar xf sage-x,y,z,tar
cd sage-x.y.z
./sage -i /path/to/openssl-u.v.w.spkg
# (but not ./sage -i openssl)
# then run make to build Sage, as usual

I.e., works *for me*; I don't know OTOH whether openssl may need some
other Sage (standard) packages to be installed prior if the system
lacks some libraries.


-leif

Dima Pasechnik

unread,
Apr 14, 2012, 12:46:49 AM4/14/12
to sage-...@googlegroups.com
Has there been any conclusion reached on this?
Should Sage
1) Require openssl?
2) Provide openssl
3) make openssl optional?
In particular this is relevant to sagenb (#11080)

(1 is the easiest, 2 might have licensing issues, 3 need some work...)

Dima

William Stein

unread,
Apr 14, 2012, 1:10:40 AM4/14/12
to sage-...@googlegroups.com
On Fri, Apr 13, 2012 at 9:46 PM, Dima Pasechnik <dim...@gmail.com> wrote:
> Has there been any conclusion reached on this?
> Should Sage
> 1) Require openssl?
> 2) Provide openssl
> 3) make openssl optional?
> In particular this is relevant to sagenb (#11080)
>
> (1 is the easiest, 2 might have licensing issues, 3 need some work...)

I think (1) is the only realistic solution.

(2) has serious license issues.

Regarding (3), there is already an openssl optional package, FWIW.

-- William

Jeroen Demeyer

unread,
Apr 14, 2012, 4:47:15 AM4/14/12
to sage-...@googlegroups.com
On 2012-04-14 07:10, William Stein wrote:
> Regarding (3), there is already an openssl optional package, FWIW.
I think with (3) he meant: make it such that the flask-notebook works
without OpenSSL.

Jason Grout

unread,
Apr 14, 2012, 9:13:34 AM4/14/12
to sage-...@googlegroups.com

I believe the pyOpenSSL and openid flask plugin couldn't be installed
either, and of course, the notebook wouldn't have openid or https
support without openssl.

Of course, it's easier for us to do option (1). OpenSSL (not just the
libs, but the development headers) comes standard on OSX and many other
OSs, but Jeroen's testing has uncovered several systems that don't have
the OpenSSL dev headers installed by default.

I vote for option 1. We already require non-standard packages that
people need to install in order to compile or run Sage---this would just
add one more very standard dependency. If someone had a convincing
argument for how hard it is to install OpenSSL on a system that doesn't
have it installed already, I'm not absolutely fixed on option 1.

Thanks,

Jason

Dima Pasechnik

unread,
Apr 14, 2012, 11:44:07 AM4/14/12
to sage-...@googlegroups.com
indeed, I meant to say that e.g. sagenb on localhost on a single-user system (e.g. laptop) probably won't need openssl.
Although this is a  sizable amount of work for relatively little gain.   
Reply all
Reply to author
Forward
0 new messages