Sage for Windows installer (take 2)

[Erik Bray]

Hi all,

TL;DR: if you have a 64-bit Windows (Windows 7 and up should work)
please demo and give me your thoughts on the new build of Sage for
Windows using the installer at [1].

Some of you may recall that this spring I worked on a Windows
installer for Sage [2] based on Docker. The use of Docker was an
attempt at an end-run around a working Sage for Windows--the installer
was a complicated contraption that installed and configured Docker for
Windows and tried to make Sage work on that in a manner as transparent
as possible to the user (the point here being that while Sage works
well in Docker, configuring and using Docker--especially on
Windows--is complicated enough to present a significant hurdle
especially to novice users who should have a simple "point and click"
experience).

Many of you were very helpful in testing out my builds of that
installer and providing feedback, and I was grateful. However, while
I was able to make this approach work on some level it was still
fragile and at times difficult. I outlined the problems with the
approach at [3].

Since then I have worked to create a build of Sage that works 100%
"natively" on Windows using Cygwin (the quotes indicating that while
it is ultimately, through Cygwin, using native Windows system calls
that fact is significantly obscured by Cygwin, and relatively few
actual source code changes are needed to Sage or its dependencies to
support that :)

Although no official release of Sage yet supports Cygwin fully, I have
a branch of Sage 7.4 with a few additional fixes needed for full
Cygwin support. This is a "fully functional" build insofar as all
`sage -t -a --long` tests pass* (though no optional packages are
included yet). There may also be as of yet unseen issues with
parallel processing--although all the tests do pass, Cygwin's fork()
implementation can be fragile, and I wouldn't count out the
possibility of problems even if I haven't seen them yet. Otherwise
the system is pretty functional, especially for beginners--this
includes running the Jupyter notebook with the Sage kernel.

The only other issue to be aware of is due to the large number of
files installed and heavy compression the installer takes a while to
run (around 15 minutes on my machine) so get it started, then go get a
coffee and chat with some colleagues before coming back to it :)

Thanks,
Erik


* The tests pass in my local build system, but still have some
failures in the "installed" build from this installer--this still has
a few failures mostly due to issues with the installation environment,
but nothing serious AFAICT.

§FAQs
=====

- Why can't Windows just use the Sage VM appliance?

They can, but I hope it shouldn't need to be said that this is hardly
ideal. Windows users, especially students and researchers, should be
able to run Sage on their own computer without the hassles associated
with virtualization.

- Have you tried the Windows Subsystem for Linux (aka "Bash for Windows")?

I've addressed this before, but since it seems to come up every time I
mention Sage on Windows: Yes, this has been considered and has
promise. Early experiments with running the Sage binary build on WSL
showed some success, but with caveats. Bill Hart has a nice writeup
at [4]. I have also heard of issues with building Sage from source on
the WSL.

That said, in its current form the WSL does not address all the goals
of this installer. For one it is not accessible to all Windows
users--it is only available as of the Windows 10 "Anniversary Update",
and not on any earlier Windows versions many of which (especially
Windows 7 and up) are still common in the wild. Second of all, by
Microsoft's own insistance it is designed to be a tool for developers,
so that developers working on Windows can more easily test their
software against and run POSIX-based development tools. It is not
meant for production environments or as a software installation
system. While it certainly *could*, in theory, be used that way it is
not well set up for that. Enabling the feature is a somewhat involved
process (especially for novices) and requires some large downloads
(larger even than the Sage installer :) It is not a particularly
accessible process, and is not easily encapsulated in some kind of
stand-alone installer.

Finally, at least for now, it's still in "beta" and has a number of
known issues anyways. While it may eventually as people learn to
exploit it more effectively, for now Cygwin remains the most portable
and well-tested POSIX emulation layer for Window, even if it has its
own issues.

[1] https://github.com/embray/sage-windows/releases/download/0.1a1-7.4/SageMath-7.4.exe
[2] https://groups.google.com/d/msg/sage-devel/To6wNO7FSxw/7YNmr7WqDQAJ
[3] https://github.com/OpenDreamKit/OpenDreamKit/blob/3ad46237fc71cb6ec2fde568ea1d0f7525e2a8dc/WP3/D3.1/status-report.md#caveats
[4] https://groups.google.com/d/msg/sage-devel/MHyD5tcyO5Y/_guoPVQUBwAJ

[Samuel Lelievre]

Wed 2016-12-07 16:39:25 UTC+1, Erik Bray:

 

TL;DR: if you have a 64-bit Windows (Windows 7 and up should work)
please demo and give me your thoughts on the new build of Sage for
Windows using the installer at [1].

Thanks,
Erik

[1] https://github.com/embray/sage-windows/releases/download/0.1a1-7.4/SageMath-7.4.exe

Hoorray! 

[William]

Worked for me -- I installed Windows into a VM, then ran Erik's installer, and it worked.

[Jean-Pierre Flori]

On Wednesday, December 7, 2016 at 7:34:01 PM UTC+1, William wrote:

Worked for me -- I installed Windows into a VM, then ran Erik's installer, and it worked.

Ahah that's exactly how I used to work on Sage on Cygwin: from a Windows VM on top of Linux!

[Jean-Pierre Flori]

And good job Erik!

[William Stein]

Hi,

I tried some random benchmarks and things look pretty good -- nothing
I tried was disturbingly slow -- even pexpect is reasonable.

Doing plot(sin) in a terminal claims to launch a png viewer, but
doesn't do anything, so maybe you can add that to your todo list.

William

> --
> You received this message because you are subscribed to the Google Groups
> "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sage-devel+...@googlegroups.com.
> To post to this group, send email to sage-...@googlegroups.com.
> Visit this group at https://groups.google.com/group/sage-devel.
> For more options, visit https://groups.google.com/d/optout.



--
William (http://wstein.org)

[William Stein]

Classic sagenb also seems to fully work.

William

--
William (http://wstein.org)

[kcrisman]

Nice!!! 

[Johan S. H. Rosenkilde]

> TL;DR: if you have a 64-bit Windows (Windows 7 and up should work)
> please demo and give me your thoughts on the new build of Sage for
> Windows using the installer at [1].

Great work, Erik!

Best,
Johan

[Peter Luschny]

Hi Erik!

The kernel dies on my system (64-bit Windows 10).

I enclose the trace.

Peter

[Erik Bray]

On Dec 7, 2016 20:18, "William Stein" <wst...@gmail.com> wrote:

Hi,

I tried some random benchmarks and things look pretty good -- nothing
I tried was disturbingly slow -- even pexpect is reasonable.

Thanks, that's good to know. I wouldn't expect anything to be slow computation-wise--I *would* expect starting up subprocesses to be noticeably slower, but at least once they're started communication should be fairly expedient as in pexpect. There shouldn't be too much overhead for most IPC methods. 

I do wish Sage had more benchmark tests since I don't personally have a firm grasp on how fast many things should be. In the past I've recommended Airspeed Velocity for this... 

Doing plot(sin) in a terminal claims to launch a png viewer, but
doesn't do anything, so maybe you can add that to your todo list.

Right, I had noticed that as well and meant to mention it as an open issue. I think it should be fixable...

On Wed, Dec 7, 2016 at 11:09 AM, Jean-Pierre Flori <jpf...@gmail.com> wrote:
>
>
> On Wednesday, December 7, 2016 at 7:34:01 PM UTC+1, William wrote:
>>
>> Worked for me -- I installed Windows into a VM, then ran Erik's installer,
>> and it worked.
>
> Ahah that's exactly how I used to work on Sage on Cygwin: from a Windows VM
> on top of Linux!
>
> --
> You received this message because you are subscribed to the Google Groups
> "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to sage-devel+unsubscribe@googlegroups.com.

> To post to this group, send email to sage-...@googlegroups.com.
> Visit this group at https://groups.google.com/group/sage-devel.
> For more options, visit https://groups.google.com/d/optout.

--
William (http://wstein.org)

--
You received this message because you are subscribed to the Google Groups "sage-devel" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscribe@googlegroups.com.

[Erik Bray]

Thanks for the update! I'm glad it (mostly) worked for you.

On Wed, Dec 7, 2016 at 11:16 PM, Jean THIERY <Jean....@modlibre.info> wrote:
> Hello Erik,

>
> Le 07/12/2016 à 16:39, Erik Bray a écrit :
>> Hi all,
>>
>> TL;DR: if you have a 64-bit Windows (Windows 7 and up should work)
>> please demo and give me your thoughts on the new build of Sage for
>> Windows using the installer at [1].

> Thank you very much for your Windows installer ready in time.
>
> SageMath was easily installed on my computer with Windows 10.
>
> My Jupyter worksheet prepared for the next week Conference
> works well (except for some errors probably due to old conventions).
>
> Some simple examples use the old turtle graphics
> but « from turtle import * » is not accepted !

Aha. Yes, it looks like by default Cygwin's Python does not come with
the Tkinter module, and hence no turtle module either. This would be
worth adding, but it looks like it will be somewhat involved, as it
also requires an X server. Certainly doable, entirely within Cywin,
but will take some additional setup. I'll look into it.

> I hope to have some time tomorrow for other tests.
>
> Latter on, would it be possible to run the interact examples
> which are very interesting ?
> https://wiki.sagemath.org/interact
> They work with the old notebook !

Yep--as William pointed out the classic sagenb seems to be working. I
could either add another shortcut for it (instead of Jupyter), Or you
can run the "SageMath Shell" shortcut to get a shell prompt, and run
`sage --notebook=sagenb`

Thanks for testing it out,

Erik

[Erik Bray]

Thanks--this looks like the exact same problem Sebastien had, including the

sys:1: RuntimeWarning: not adding directory '' to sys.path since it's
writable by an untrusted group.
Untrusted users could put files in this directory which might then be
imported by your Python code. As a general precaution from similar
exploits, you should not execute Python code from this directory

What path did you install into? The default (under Program Files)?

[Peter Luschny]

What path did you install into?  The default (under Program Files)?

The program is installed in C:\Program Files\SageMath

It was a one-click installation in which I did not intervene manually.

Peter

[Peter Luschny]

I can now also add an IPython post-mortem report.

[Nicolas M. Thiery]

On Thu, Dec 08, 2016 at 11:49:03AM +0100, Erik Bray wrote:
> > Some simple examples use the old turtle graphics
> > but « from turtle import * » is not accepted !
>
> Aha. Yes, it looks like by default Cygwin's Python does not come with
> the Tkinter module, and hence no turtle module either. This would be
> worth adding, but it looks like it will be somewhat involved, as it
> also requires an X server. Certainly doable, entirely within Cywin,
> but will take some additional setup. I'll look into it.

It would actually be nice (and plausibly sufficient for our purposes)
if the turtle module would work in the Jupyter notebook. Typically
through an ipywidget backend.

Vaguely related note: for our introductory class "programming, in C++"
here in Orsay, I'd love to have such a turtle in the notebook module
working with the C++ interpreter cling. I discussed this with Jupyter
devs last week in Orsay, and there is some hope!

Cheers,
Nicolas
--
Nicolas M. Thiéry "Isil" <nth...@users.sf.net>
http://Nicolas.Thiery.name/

[Nicolas M. Thiery]

On Wed, Dec 07, 2016 at 11:16:50PM +0100, Jean Thiéry (Papa) wrote:
> Latter on, would it be possible to run the interact examples
> which are very interesting ?
> https://wiki.sagemath.org/interact
> They work with the old notebook !

Sage's interacts should work in the Jupyter notebook soon (in the next
stable release maybe?):

https://trac.sagemath.org/ticket/21267

[Harald Schilly]

Found this old file on my harddisk today ^^

-- h

[William Stein]

Here's a turtle graphics implementation.  It should be to adapt any notebook...

https://cloud.sagemath.com/projects/4a5f0542-5873-4eed-a85c-a18c706e8bcd/files/support/2014-12-12-turtle/Sage%20Turtle%20via%20JSXGraph.sagews

---

https://cloud.sagemath.com/4a5f0542-5873-4eed-a85c-a18c706e8bcd/raw/support/2014-12-12-turtle/Sage%20Turtle%20via%20JSXGraph.sagews.html

William

[Erik Bray]

On Dec 10, 2016 11:58, "Nicolas M. Thiery" <Nicolas...@u-psud.fr> wrote:

On Thu, Dec 08, 2016 at 11:49:03AM +0100, Erik Bray wrote:
> > Some simple examples use the old turtle graphics
> > but « from turtle import * » is not accepted !
>
> Aha.  Yes, it looks like by default Cygwin's Python does not come with
> the Tkinter module, and hence no turtle module either.  This would be
> worth adding, but it looks like it will be somewhat involved, as it
> also requires an X server.  Certainly doable, entirely within Cywin,
> but will take some additional setup.  I'll look into it.

It would actually be nice (and plausibly sufficient for our purposes)
if the turtle module would work in the Jupyter notebook. Typically
through an ipywidget backend.

Ah yeah, I should have thought of that. I tried to implement a turtle graphics for IPython Notebook years ago, but it was before the widget interface was implemented and it proved too tricky without modifying the notebook source itself. 

I've since heard that it's been done though I haven't tried it yet. 

That said it still might be good to support Tkinter. 

[Emmanuel Charpentier]

For the hell of it, I tried this installer on a "spare" Windows VM.

Bravo, bravissimo ! Now, there might be some use to all those Windows boxes...

I tried to install some optional package ; it turns out that the installed setup isindeed  minimal :

(sage-sh) charpent@DESKTOP-P0B7HOE:~$ time ( for i in database_gap dot2tex giac giacpy_sage ; do sage -i $i ; done )
/opt/sagemath-7.4/src/bin/sage: ligne 302: make : commande introuvable
/opt/sagemath-7.4/src/bin/sage: ligne 302: make : commande introuvable
/opt/sagemath-7.4/src/bin/sage: ligne 302: make : commande introuvable
/opt/sagemath-7.4/src/bin/sage: ligne 302: make : commande introuvable

Either that or the necessary utilities are out of Sage-subshell's reach.

A bit of Windows-specific documentation is needed. A (relatively) inexperienced user should not have to struggle to find :

• How to open a cygwin console (as opposed to a Sage subshell) in the setup created by the installer.
• What (and how) to install in order  to install optional packages.
• What (and how) to install in order to recompile (e. g. testing a git branch).
  
• A pointer to Cygwin docs.
• Advice about what should be installed under cygwin and what should be installed outside (e. g. emacs, TeX, various Unix-like tools).
• Advice about how to use Sage-installed programs outside of Sage (e. g. Maxima, lisp, Python, etc...).
  

I'll play a bit more with that setup (and, of course, report errors an quirks). But my first impression is that it *is* a vast improvement over the VM solution.

A big "thank you" on behalf of Windows-stranded users !

--
Emmanuel Charpentier

[Erik Bray]

On Sun, Dec 11, 2016 at 3:27 PM, Emmanuel Charpentier
<emanuel.c...@gmail.com> wrote:
> For the hell of it, I tried this installer on a "spare" Windows VM.
>
> Bravo, bravissimo ! Now, there might be some use to all those Windows
> boxes...

Thanks! :D

> I tried to install some optional package ; it turns out that the installed
> setup isindeed minimal :
>
> (sage-sh) charpent@DESKTOP-P0B7HOE:~$ time ( for i in database_gap dot2tex
> giac giacpy_sage ; do sage -i $i ; done )
> /opt/sagemath-7.4/src/bin/sage: ligne 302: make : commande introuvable
> /opt/sagemath-7.4/src/bin/sage: ligne 302: make : commande introuvable
> /opt/sagemath-7.4/src/bin/sage: ligne 302: make : commande introuvable
> /opt/sagemath-7.4/src/bin/sage: ligne 302: make : commande introuvable
>
> Either that or the necessary utilities are out of Sage-subshell's reach.

Hmm, no, make is just not installed I don't think. I'll have to think
about that... I think it would be better if the installer just came
with all optional packages pre-installed as well. That said, it
already does come with gcc I'm pretty sure, so adding make wouldn't be
a huge addition.

> A bit of Windows-specific documentation is needed. A (relatively)
> inexperienced user should not have to struggle to find :

A few responses to your list, which I've reordered slightly:

> How to open a cygwin console (as opposed to a Sage subshell) in the setup
> created by the installer.

> A pointer to Cygwin docs.
> Advice about what should be installed under cygwin and what should be
> installed outside (e. g. emacs, TeX, various Unix-like tools).

I think you have the wrong idea of the intention behind using Cygwin.
This installer is *not* meant to provide a full working Cygwin
installation. Ideally, for the most part, the user is agnostic to the
existence of Cygwin under the hood. I do see your point about how
they might want to bring other utilities into the environment, but
really the point of this is just to run Sage as a stand-alone
application. The fact that the sources are included is only even
incidental, as there are several things that break if they aren't
(something I'm actively trying to fix, which will make packagers for
Linux happy as well I suspect.

That said, I may also be naive as to how users will want to use this.
But I also don't want to get tied down with supporting basic Cygwin
usage.

> What (and how) to install in order to install optional packages.
> What (and how) to install in order to recompile (e. g. testing a git
> branch).

For optional packages, again, if all it takes for `sage -i` to work is
make I might include that, though I suspect it's more complicated than
that. I haven't gotten all the optional packages working on Cygwin
yet, I don't think. I think easier just have them all pre-installed.
A novice user who might otherwise just use SMC shouldn't have to worry
about "standard" or "optional" packages--they should just get the full
kit.

This is also not meant to "work" how Sage has been traditionally
used--sitting all the time in a source repository checkout and
switching branches and recompiling. It works otherwise like a normal
Windows application--the files are installed once, and then normal
users don't touch them except to uninstall. I would like to work on
documentation for someone who wants to do something more like "sage
development" on Windows, under Cygwin. I'm working on putting that
together now in the process of assembling reproducible build tools for
the installer.

> Advice about how to use Sage-installed programs outside of Sage (e. g.
> Maxima, lisp, Python, etc...).

I think that's mainly what the "Shell" shortcut is for.

> --
> You received this message because you are subscribed to the Google Groups
> "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to sage-devel+...@googlegroups.com.