For the record : the lil' Win10 notebook that (almost) could (barely).

[Emmanuel Charpentier]

Dear list,

This is a followup on the threads dedicated to the alternatives existing to use Sage math on Windows. More specifically, I did an informal comparison of Erik Bray's Cygwin-based installer and using Windows 10 64 bits Windows Subsystem for Linux.

I recently acquired (for other, unrelated, purposes) a small (< 11") and light (<800g) notebook, bound to become (literally) a pocket machine. This machine has an Atom z5 processor (1,4 MHz), 4 Gb memory and a 128 Gb SSD. It starts with Win10 64 bit ("home edition").

It is still too new to be well-supported by common "live" distributions :

• Ubuntu boots its initial screen (rotated pi/2), allowing to start the "live" system, but never displays anything else.
• Debian (which needs to be installed on an UEFI-enabled USB key, see there) starts with displaying its initial screen rotated pi/2, goes a long way in its booting process, starts its graphical mode (still rotated pi/2), but ends up crashing gnome.

So I decided to give Win10 a chance.

Cygwin

Erik Bray's installer for Sage 7.4 works flawlessly, and results in a usable, if slow, Sage notebook. However, I never managed to get a prompt on (native port of)emacs' sage_shell_mode.
Furthermore, as far as I understand, installing an optional package requiring compilation is not obvious...

So I decided to be a bit adventurous, and try :

Windows Subsystem for Linux

To avoid the limitations of the currently released system, I took the following steps (details on demand) :

• Uninstalled Windows' emacs
• Upgraded to the latest Win10 release available via the "Windows Insider" program (that cost me twenty-two fucking gigabytes (!) of "Windows.old" recovery directory...).
• Upgraded the Ubuntu distribution to Xenial
• installed the requirements given in Sage's README
• Followed my own advice and installed a matching set of gcc, gfortran and g++, as well as OpenSSL and its development libraries.
• installed (Ubuntu's) git.
• Cloned the git Sage tree, switched to develop and built Sage 7.6.beta3/
• Built Sage 7.6.beta3 (see below).
  
• Installed the VcXsrv X server for Windows
• Installed (Ubuntu's) emacs, texlive, gnuplot, ghostscript (see below) and imagemagick
  
• added an "alternative" to www-browser opening Windows' Firefox.exe

Results :

• Built is *extremely* slow : building from scratch took about 21 hours (can't be precise, the compilation having been interrupted by network losses and changes of place).

This was partly expected (one thread, low-power processor) ; however, a main cause is that anything doisk-bound is dramatically slow : studying the compilation log showed that the "system time" was about the same as "user time" (in a normal Unix compilation, I usually get a "system time" about 2-5% of "user time").

• Starting Sage from the command line (Windows (via 'bash -c "DISPLAY=:0 sage"' or Linux) :
• Startup is slow
• Sage complains about a nonexistent  /proc/vmstat
• Once started, Sage proceeds as expected on a Unix machine. Plots are displayed by Imagemagick.
• I checked that the system is able to install and compile optional and experimental packages.
  
• Starting the Jupyter notebook :
• Slow startup
• works as expected, including typeset output, 2D and 3D graphics
• Emacs + sage_shell_mode. That doesn't really works :
• Slow startup
• Sage complains about /proc/vmstat
• I never gets prompts
• Typing expression works and returns the expected results (in text form)
• toggling either inline typesetting or inline plots starts an infinite loop.

In order to explore this, I tested the imaxima emacs interface to Maxima, usting what is distributed with Sage (in $SAGE_ROOT/local/share/maxima/emacs/). This works flawlessly : I get correct results, correctly typeset, and both inline (wxplot2d) or stndalone (displayed with gnuplot) graphs. Emacs seems innocent of sage_shell_mode problems.

The resulting system integrates well with the Windows interface (including cut n' paste).

Partial and interim conclusions :

• The Linux subsystem has only, nonblocking, minor incompatibilities with what Sage expects.
• The CPU performance is about what is expected from such a low-power CPU.
• The disk-related performance is currently awful.

This is probably bound to the translation of disk I/O system calls to NTFS-managed actions. To take an example, moving the Sage source tree after "git clone", (i. e. "sudo mv sage /usr/local/sage-7"), which is more or les instantaneous on an ext4 partition, takes about 30 seconds in the Windows Subsystem for Linux.

• The interoperability of the subsystem with the host is acceptable (one can call a Windows application from Linux, and a Linux application from Windows, more or less transparently).
• The emacs problem remains to be understood.

Subjective point of view : I'm now a Unix die-hard. Installing a decent Linux distribution and compiling Sage here seemed (to me) simpler than configuring a set of Windows applications. The results show that this can be done :

• using an unreleased version of Windows
• with a penalty in terms of initial installation complication

(installing a pre-compiled image or ppa would have spared me this, but I wanted to assess the complication of the whole process)

• with a penalty in terms of disk I/O
• with acceptable results
• with the possibility of adding optional packages
  

The trade-offs are not obvious. No solution dominates the other. But both solutions explored here (WSL vs Cygwin) seem to dominate the VM solution.

HTH,

-
Emmanuel Charpentier

[William Stein]

On Wed, Feb 15, 2017 at 11:26 PM, Emmanuel Charpentier
<emanuel.c...@gmail.com> wrote:
> Dear list,
>[...]

> The trade-offs are not obvious. No solution dominates the other. But both
> solutions explored here (WSL vs Cygwin) seem to dominate the VM solution.

Did you try actually using a VM on this computer? My experience with
Windows to date has been that for a technically sophisticated user who
cares about performance (like you), using VirtualBox is vastly
superior to WSL or Cygwin...

William

>
> HTH,
>
> -
> Emmanuel Charpentier
>
> --
> You received this message because you are subscribed to the Google Groups
> "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sage-devel+...@googlegroups.com.
> To post to this group, send email to sage-...@googlegroups.com.
> Visit this group at https://groups.google.com/group/sage-devel.
> For more options, visit https://groups.google.com/d/optout.



--
William (http://wstein.org)

[Emmanuel Charpentier]

Dear William,

Le jeudi 16 février 2017 08:45:36 UTC+1, William a écrit :

On Wed, Feb 15, 2017 at 11:26 PM, Emmanuel Charpentier
<emanuel.c...@gmail.com> wrote:
> Dear list,
>[...]
> The trade-offs are not obvious. No solution dominates the other. But both
> solutions explored here (WSL vs Cygwin) seem to dominate the VM solution.

Did you try actually using a VM on this computer?

Huh ? I have "only" 4 GB of RAM, to share (to *dedicate*) to boith systems. Even if Windows may run on 2 GB RAM, I doubt that VM+Linux+Sage+whatever is needed to serve Jupyter would run on 2 GB RAM. Crawl, maybe..

Using the "old" notebook might be better here (the browser could run on the Windows side. But the main problem is the *dedication* (*fixed* allocation) of resources to each system.

Hell, I don't even know how many execution threads my CPU has...

Less important, but still a hurdle : configuring network-like communications between the two machines. I know how to do this from Liux, but I'm nit so at ease in Windows (I don't even know if Windows Home will let me create the necessary pseudo-adapter, aka tun device in Linux parlance...).

More generally : I'm more or less competent on Linux, but hopeless in Windows. I may find my way around it, but that's not a pretty sight... My attempt is a temporary measure while waiting the availability of the necessary hardware support in Linux : as soon as this is available, I'll probably kiss Windows goodbye.
 

 My experience with
Windows to date has been that for a technically sophisticated user who
cares about performance (like you),

I'm more concerned by competence than by performance : for example, I'm not concerned by a few seconds more for a LaTeX compilation (a few minutes more would be another problem...) ; not having emacs or an useful (synctex-enabled) PDF viewer would be more of a concern, possibly a deal-breaker.
 

using VirtualBox is vastly
superior to WSL or Cygwin...

William

Thanks for your advice. I'll consider it.

--
Emmanuel Charpentier
 

[William Stein]

On Thu, Feb 16, 2017 at 12:20 AM, Emmanuel Charpentier
<emanuel.c...@gmail.com> wrote:
> Huh ? I have "only" 4 GB of RAM, to share (to *dedicate*) to boith systems.

I guess I'm probably a lot older than you.... but 4GB of RAM is plenty
to run VirtualBox + Linux, etc. I've spent years using laptops with
a VM and another OS and that much (maybe less) memory, while working
on Sage.

> Even if Windows may run on 2 GB RAM, I doubt that VM+Linux+Sage+whatever is
> needed to serve Jupyter would run on 2 GB RAM. Crawl, maybe..
>
> Using the "old" notebook might be better here (the browser could run on the
> Windows side. But the main problem is the *dedication* (*fixed* allocation)
> of resources to each system.
>
> Hell, I don't even know how many execution threads my CPU has...

It doesn't really matter much for using VirtualBox.

You should see if you have VT-x instructions -- that's critical for
performance. The Atom Z530 does:
http://ark.intel.com/products/35463/Intel-Atom-Processor-Z530-512K-Cache-1_60-GHz-533-MHz-FSB

> Less important, but still a hurdle : configuring network-like communications
> between the two machines. I know how to do this from Liux, but I'm nit so at
> ease in Windows (I don't even know if Windows Home will let me create the
> necessary pseudo-adapter, aka tun device in Linux parlance...).

This isn't very hard. I've done it a million times on Windows, OS X,
etc. It used to be kind of hard with VMware back in 1998, but now
there is this thing called Google which makes it very easy to find
information about how to do things...

> Thanks for your advice. I'll consider it.

Well give it a shot. I'm a little conflicted, since obviously it is
also very useful to us (the sage community) that you're testing Erik
Bray's cygwin installer, etc...

--
William (http://wstein.org)

[Dima Pasechnik]

On Thursday, February 16, 2017 at 8:20:21 AM UTC, Emmanuel Charpentier wrote:

Dear William,

Le jeudi 16 février 2017 08:45:36 UTC+1, William a écrit :

On Wed, Feb 15, 2017 at 11:26 PM, Emmanuel Charpentier
<emanuel.c...@gmail.com> wrote:
> Dear list,
>[...]
> The trade-offs are not obvious. No solution dominates the other. But both
> solutions explored here (WSL vs Cygwin) seem to dominate the VM solution.

Did you try actually using a VM on this computer?

Huh ? I have "only" 4 GB of RAM, to share (to *dedicate*) to boith systems. Even if Windows may run on 2 GB RAM, I doubt that VM+Linux+Sage+whatever is needed to serve Jupyter would run on 2 GB RAM. Crawl, maybe..

I think the total of 4GB for this config should be more than enough.

But, if I were you, I'd just get the right Linux running on your machine  natively

(perhaps something like https://www.linuxliteos.com/ or gentoo)

[Erik Bray]

To be clear (and I'll respond more on this later) the current version
of my Windows build is aimed toward what I guess you could call
"casual" users and is designed more toward making Sage look like a
real "software product" that one can install and have 'just work'.
There are no optional packages yet just because I haven't tested them
all on Cygwin yet, and if there ever are they'll probably be installed
from binaries, not compiled on the user's local machine.

It's not intended for developers of Sage or "power users" who want to
compile and fine-tune their own packages.

[Erik Bray]

Thanks for the testing. I don't have too much to say about WSL except
what I've said before--which is that I think it (will be) great for
doing development on Windows, but it is *not* a solution for software
distribution to users.

You mentioned a "slow Sage notebook". Was this the Jupyter notebook
that it comes with a shortcut to? Or the sagenb? If the latter, I
know it works but I don't know if it's slow or not. If it was the
former, I've noticed that it is slow on the first couple commands
because the Sage Kernel takes a long time to start up (I wish there
were a clearer indication that it is still loading), but after that it
should be snappy.

Did you also test plotting? It should work, but one or two people
have reported it not working (it should work as long as your Windows
has a default program registered for viewing PNGs or whatever file
type the plots are saved as).

Finally, I don't know anything about emacs except how to get out of
it. If you cold tell me more about this "sage_shell_mode" maybe I can
do something about it, though I suspect it's not really something in
scope of the Windows installer.

[Erik Bray]

On Thu, Feb 16, 2017 at 8:44 AM, William Stein <wst...@gmail.com> wrote:
> On Wed, Feb 15, 2017 at 11:26 PM, Emmanuel Charpentier
> <emanuel.c...@gmail.com> wrote:
>> Dear list,
>>[...]
>> The trade-offs are not obvious. No solution dominates the other. But both
>> solutions explored here (WSL vs Cygwin) seem to dominate the VM solution.
>
> Did you try actually using a VM on this computer? My experience with
> Windows to date has been that for a technically sophisticated user who
> cares about performance (like you), using VirtualBox is vastly
> superior to WSL or Cygwin...

Docker is another option here, lighter-weight than using a full VM
(granted, Docker for Windows does run a lightweight Linux VM). Since
I do so much work on Windows these days I actually do a lot of my Sage
work (on Linux) in a Docker container. Performance is on par with
running on bare metal since, for the most part, you are.

The only problem I have with it right now is that shared directories
between the Windows host and the Docker VM, while functional, have
problems with symlinks. As of a few months ago symlinks *work*, but
it's still buggy. I believe this may actually be due to a bug in cifs
on Linux but I haven't investigated deeply yet. In any case, the
workaround is to put your source code in a Docker volume, in which
case it's actually on an ext4 filesystem inside the Linux VM. Just
make sure to give the VM image a decent amount of disk space.

This has been the best option for me, as a power user (in conjunction
with Cygwin for general shell-based tasks).

[Emmanuel Charpentier]

Dear Erik,

That's the case *now* (esopecially if you consider rrthe need foir an unreleased version of Windows to open gates in the walls of the "walled garden" you described in an earlier thread.

But maybe Microsoft will stay serious this time and offer this as an end-user useable tool. Even maybe with a wider choice of Linux distributions (it *is* at least possible).

You mentioned a "slow Sage notebook".  Was this the Jupyter notebook
that it comes with a shortcut to?

Yes.
 

Or the sagenb?  If the latter, I
know it works but I don't know if it's slow or not.  If it was the
former, I've noticed that it is slow on the first couple commands
because the Sage Kernel takes a long time to start up (I wish there
were a clearer indication that it is still loading), but after that it
should be snappy.

"Snappy is a bit hyperbolic (consider the machine I was running it on...), but its livable...

Did you also test plotting?

In the Jupyter notebook. It worked. I can't recall having tested it in a terminal windows nor in a mintty console, but I would have probably noted if it didn't work...
 

 It should work, but one or two people
have reported it not working (it should work as long as your Windows
has a default program registered for viewing PNGs or whatever file
type the plots are saved as).

I don't know if a "stock" Windows machine even has such a program : .png is quite foreign in the Windows world, where the natives are expected to speak .bpm (yuck !) .wmf (yuckier :-) or .emf (yuckiest...)

Finally, I don't know anything about emacs except how to get out of
it.

:-) It's a start. Don't despair... More seriously, it's the IDE-posing-as-an-editor that sold me (a couple decades back) to Unix. It's a marvelous way to work simultaneously on computations and on their description in plain globish (=what passes for English in the international scientific community) or French (it still exists, you know...).

 If you cold tell me more about this "sage_shell_mode"

Better ask Sho Takemori, the author...
 

maybe I can
do something about it, though I suspect it's not really something in
scope of the Windows installer.

I think that the problem is bound to the prompts : it happens exactly the same way in Sagemath installer + Windows port of Emacs as it happens with WSL + Ubuntu  + my self-compiled Sage + emacs25.

Probably on the emacs-lisp side of the things... I'll open an issue on this.

[Samuel Lelievre]

Thu 2017-02-16 16:33:56 UTC+1, Emmanuel Charpentier:

> Dear Erik,
>
> Le jeudi 16 février 2017 11:09:00 UTC+1, Erik Bray a écrit :
>

>> On Thu, Feb 16, 2017 at 8:26 AM, Emmanuel Charpentier wrote: 
>>
>> > This is a followup on the threads dedicated to the alternatives existing to 
>> > use Sage math on Windows. More specifically, I did an informal comparison of 
>> > Erik Bray's Cygwin-based installer and using Windows 10 64 bits Windows 
>> > Subsystem for Linux. 

[...]

 
>> Did you also test plotting?
>
> In the Jupyter notebook. It worked. I can't recall having tested it in a terminal

> windows nor in a mintty console, but I would have probably noted

> if it didn't work...
>  
>>  It should work, but one or two people 
>> have reported it not working (it should work as long as your Windows 
>> has a default program registered for viewing PNGs or whatever file 
>> type the plots are saved as). 
>
> I don't know if a "stock" Windows machine even has such a program :

> .png is quite foreign in the Windows world, where the natives are

> expected to speak .bpm (yuck !) .wmf (yuckier :-) or .emf (yuckiest...)

A stock windows machine probably has a web browser,

and any modern web browser can view PNGs.

If nothing is registered as the default program for viewing PNGs,

I guess you could set the default to your favourite web browser

by visiting the appropriate preference pane.

[Erik Bray]

Right--the way I've handled plotting (in the command line, that is) is
I set the BROWSER environment variable to cygstart, and Sage uses this
variable, if it's set, in determining how to open image files.
cygstart is a program in Cygwin that just opens a file with the
default handler for that filetype in the registry. I believe if no
default handler is found then the standard Windows pop-up for unknown
file types, allowing you to specify a default program (or "app" as it
now says on Windows 8 and up) to open files of that type. So in other
words, the behavior (and indeed Windows API calls) are the same as
when you double-click a file in Explorer.