Porting SageMath to Windows 64

[Bill Hart]

HI all,

William Stein recently bemoaned the fact that SageMath currently only runs natively on some brands of Linux, and not natively on the latest Windows or OSX (that is to say nothing of BSD). [1]

Until recently, a port of SageMath to Windows has seemed like a pipe dream. However, things have changed a lot, and I think it would now be possible to achieve this (for some definition of the word "native").

I've tried VM's before and it has always ended in disaster. They either screw up my networking, they have performance issues, or don't support my mouse properly, or change my keyboard layout, or it's impossible to get files from my hard drive into the system easily, or they take up too much disk space or need to be constantly downloaded to update them, or some features don't work, or people stop supporting them, etc. The disadvantages of VMs are so numerous I hardly need to enumerate them. And even if it is a solution for users, it's hardly a solution for serious Windows developers.

As some people know, I've been recently working on a Julia based "computer algebra system" for a much more limited subset of "computer algebra" than Sage targets. What people may not know is that that entire technology stack works natively on Windows.

I can't express how fantastic it was to be sitting on a train recently, with no web access whatsoever and to be able to do work on my Windows 10 (64 bit) laptop on the train. And everything ran as fast, or in some cases faster, than my Linux server. That's the first time that has happened since I started doing computational stuff about 10 years ago!

MSYS2 as a solution

================

The new technology that makes all this work is MSYS2. Here are some of its features:

* Installing MSYS2 on Windows couldn't be easier. [2]

* It supports proper Windows exception handling protocols.

* It has an amazing command line package manager called Pacman, which Just Works TM.

* Unlike Cygwin, it's very minimal, and takes little time to install.

* It's fast. Very fast.

* Parallel make works.

* The resulting binaries are fast, sometimes faster on my laptop than on the Linux server I usually use for development.

* MSYS2 provides a posix layer! (Applications can be distributed with an MSYS2 dll for this.)

* Multithreading using pthreads Just Works TM (Applications can be distributed with a winthreads dll for this. I've actually used this, no patching required, so I am not speaking theoretically here.)

* Like native Windows, sizeof(long) = 4 and sizeof(long long) = 8 on MSYS2. This means interop with native Windows applications or assembly objects is a cynch.

* The resulting applications can be run on Windows as essentially native applications. They don't have to be run from within the MSYS2 shell. They can also be distributed as binary packages for those that don't care about the source code. But here's the thing: it's not *necessary* to distribute them as binary packages. It's now quite feasible for developers to actually *build* packages on Windows. And let's face it, this is the type of developer we actually want on the Windows platform. Simply supporting users and not developers is just going to increase the maintenance burden for people who work on other platforms. We want actual Windows developers, not just users!

 

The only bad thing about MSYS2 is that autotools is still incredibly slow (Flint roll-your-own configure takes 5s at most on Windows, whereas autotools can take some minutes to configure a package). Note: make is not slow (at least not on Windows 64 -- it's not as fast on 32 bit Windows, but fortunately almost no one is using Windows 32 any more anyway).

Proposal

=======

What I propose is to begin porting Sage dependencies to Windows (note that by Windows, I mean Windows 64, i.e. what everyone is using). What this would mean is porting them to MSYS2 (at least initially). Note it's not a big step from there to MinGW or MSVC if people want to do that later. Microsoft have made MSVC much, much more friendly to GNU conventions, recently. (For some users, porting to Windows natively is synonymous with supporting MSVC using Visual Solution files. But I'm not suggesting we do that.)

Since we know that many upstream projects simply will not accept thousands of patches to their repositories for supporting Windows 64, much less do they have developer resources to support them on an ongoing basis, here is what I think needs to be done:

1) Automagically patch their source trees at build time so that long becomes long long and unsigned long becomes unsigned long long on Windows.

2) Automagically patch their source trees at build time so that printf/scanf("%ld") becomes printf/scanf("%lld") on Windows.

3) Send a very small number of "Windows" patches upstream for the few instances of nonportable code that probably doesn't work anywhere except Linux, let alone on Windows.

Is this feasible? 

Yes it is. Note that libpari still works today on Windows 64 (actually natively with MinGW, not just with MSYS). On Windows 64 their build automagically patches their source tree. As far as I know, this hasn't been touched in years, and it still works today as well as the day it was written (modulo a couple of trivial bugs in their actual build system)!

[Optionally, we should investigate the feasibility of providing Makefiles for Windows, automatically generated by autotools, but stored on a server so that they do not have to be generated at build time (of course the user still has the option of doing that if they really want). This may not be feasible, but it's worth investigating. Obviously paths would have to be patched on a per user basis. But Windows is a very homogeneous environment, so most of what autotools actually does on Windows is pretty pointless.]

What we've done so far

==================

So far, the following packages (and probably many more) work fine on Windows 64, with the provisions noted:

* MPIR-2.7.0

* MPFR-3.1.3

* Flint-2.4.5

* libpari-2.7.4 -- GP is not patched, and there are some bugs in the Pari build script, but I've been able to hack them to get libpari to build on Windows 64. The %ld -> %lld patching is not done yet.

* Antic

* OpenBLAS (the Julia people already build this on Windows, since they use it heavily)

Python must also work, since I use IJulia on my Windows 64 laptop, which is basically the same thing as the IPython notebook. This I download as part of the Anaconda distribution currently.

You can find Windows 32 and 64 dlls for most of the packages mentioned above, here:

http://nemocas.org/binaries/

My intention is to publish a note every time I update this collection of dlls and to gradually expand them to cover more packages. We'd eventually want to use dlltray, or the OpenSUSE cross platform build service [3] for hosting these (the latter is not trivial at the moment, since their package management system doesn't handle some of the complexities of building packages that Sage would rely on).

It would be fantastic if others were to join me in extending this list over time. I'd be happy to host the dlls there (actually the machine is provided by William Stein), at least until the hosting requirements start to become nontrivial.

The only requirement would be to send dlls for Window 32 and 64 and a tarball of the source used to generate them (or alternatively a git repo I can clone), so that we aren't violating anyone's GPL. We'd also have to agree on what exception handling protocol to use. It would be very convenient for me should that match the one used by the Julia project.

We will shortly be porting Arb to Windows 64. I'll post an update when that is done.

Note that Julia itself works on Windows and they have an interest in porting lots of stuff to work natively on Windows. So it's possible some joint effort could be mutually beneficial.

MPIR vs GMP

==========

One of MPIR's dirty little secrets is we abandon the GMP interface on Windows 64, for good reason. They make all their _ui and _si function take an unsigned long/long instead of an unsigned long long/long long, even though limbs are 64 bits on Windows.

In some timings I did recently, this slows some operations down by a factor of 6, and in Flint we have to supply a GMP compatibility header, so that we can use GMP in the same way we use MPIR on Windows 64 (which is pretty time consuming to write and obviously a performance sink).

As I'm not interested in just having something that works on Windows but doesn't care about performance, I personally believe that such an effort should be based on MPIR not GMP for the time being.

Final note

========

The Julia project targeted Windows very early on, and they have attracted serious, extremely knowledgeable Windows developers (who have been very helpful to me). I think this speaks for itself.

Bill.

[1] https://plus.google.com/115360165819500279592/posts/HiXvz9bjumu

[2] https://msys2.github.io/

[3] https://build.opensuse.org/

[Jeroen Demeyer]

Is there any reason to assume that porting using MSYS2 is easier than
porting using Cygwin? Because the latter is already hard enough.

I can the main "elephant in the room" is the POSIX layer. Many pieces of
Sage assume some kind of POSIX environment.

Jeroen.

[Francesco Biscani]

Some random thoughts:

- I am not so convinced the strategy of automatic long -> long long patching is actually feasible, I think in practice this is gonna be a big can of worms. Pushing upstream to fix their code is a much better long-term solution IMO (and I'd rather have nothing to do with projects which refuse portability and code-quality patches, but that's just me :)

- mingw-w64 is a high quality compiler. I recently had to develop for a while on a Windows 64 machine, and had to recompile a full stacks of dependencies (MPFR, GMP, Boost, etc.). It was not so easy (most issues were related to build systems), but the end result was pretty impressive performance-wise.

- While for C code you can probably interoperate with libraries compiled with MSVC, for C++ you will end up having issues related, e.g., to different implementations of exception handling. My suggestion would be to forget about MSVC and compile the full stack of dependencies exclusively with mingw.

- Python on windows 64 does not play properly with mingw due to some long-standing issues in some header files. If you want to compile Python C/C++ extensions on Win64 you will have to patch slightly the Python headers and re-generate the definitions of the Python library. The issue is explored, e.g., here:

http://ascend4.org/Setting_up_a_MinGW-w64_build_environment#Setup_Python_for_compilation_of_extensions

Generally speaking, mingw-w64 is a really good option for C/C++ development on Windows. The biggest problem is the proliferation of distributions (mingw-w64, TDM mingw, mingw-build, msys2, etc.), but the toolchain is solid. clang is getting there as well, so it's worth keeping it in mind for the future.

Cheers,

  Francesco.

--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To post to this group, send email to sage-...@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

[Jean-Pierre Flori]

As far as I remember, apart from the lack of POSIX compatibility on Windows/MSYS, the main obstacle to "natively" compile Sage on Windows 64 were:
* PARI which assumes that sizeof(long) == sizeof(void*), there is an experimental branch fixing this: http://pari.math.u-bordeaux.fr/archives/pari-dev-1505/msg00021.html
* GAP memory allocation, no sure this is actually an issue, see https://groups.google.com/d/msg/sage-devel/QXy_2KMbP1k/Z3cfpCieYzgJ for "details".

[Bill Hart]

On Wednesday, 7 October 2015 17:15:14 UTC+2, Jeroen Demeyer wrote:

Is there any reason to assume that porting using MSYS2 is easier than
porting using Cygwin? Because the latter is already hard enough.

Cygwin is personally of no use to me (native applications like Julia can't work with it). I don't think I've ever downloaded a Sage Cygwin binary. For one, it's bloated and too slow.

Cygwin in not considered a native environment by serious Windows developers. It's a Linux on Windows, nothing more. It doesn't even try to play nice with native applications or the Windows ABI.

The main reason for the existence of the MSYS2 project in their own words is "better interoperability with native Windows software".

It also has a proper package manager. It's really, really fast. It has a posix layer. It has proper Windows exception handling (including zero overhead exceptions) and supports a couple of different threading models. Basically it's an extremely high quality piece of software engineering. 

I can the main "elephant in the room" is the POSIX layer. Many pieces of
Sage assume some kind of POSIX environment.

Cygwin also provides a POSIX environment, so I'm not sure I understand why this is an "elephant in the room".

Bill.

 

Jeroen.

[Bill Hart]

On Wednesday, 7 October 2015 18:18:02 UTC+2, Jean-Pierre Flori wrote:

As far as I remember, apart from the lack of POSIX compatibility on Windows/MSYS, the main obstacle to "natively" compile Sage on Windows 64 were:

Just to clarify, I'm not talking about MSYS. That's a different thing entirely. MSYS2 has a posix layer.

 

* PARI which assumes that sizeof(long) == sizeof(void*), there is an experimental branch fixing this: http://pari.math.u-bordeaux.fr/archives/pari-dev-1505/msg00021.html

I am using Pari (not GP) today on Windows 64. It was minimal effort on my part to do so. I am not using a special branch.

* GAP memory allocation, no sure this is actually an issue, see https://groups.google.com/d/msg/sage-devel/QXy_2KMbP1k/Z3cfpCieYzgJ for "details".

Gasman might be an issue. My understanding is it tries to allocate all available memory up front. It's also not threadsafe. But this is already an issue for HPC Gap (so they use Boehm instead, as I understand it, albeit at a small performance cost).

Bill. 

[Bill Hart]

Perhaps I should also say that our long term plans for the Julia project I mentioned definitely include Gap and Singular, so we will be investing time and expertise into solving any issues with these, I am sure.

It's a long term strategy for sure, but not one that is going to disappear overnight or bitrot.

Bill.

[Jeroen Demeyer]

On 2015-10-07 18:23, Bill Hart wrote:
> Cygwin also provides a POSIX environment, so I'm not sure I understand
> why this is an "elephant in the room".

It's an elephant in the room because your original post completely seems
to ignore any possible problems with the POSIX layer, while the POSIX
layer is where most of the problems on Cygwin come from.

[Jean-Pierre Flori]

On Wednesday, October 7, 2015 at 6:23:21 PM UTC+2, Bill Hart wrote:

On Wednesday, 7 October 2015 17:15:14 UTC+2, Jeroen Demeyer wrote:

Is there any reason to assume that porting using MSYS2 is easier than
porting using Cygwin? Because the latter is already hard enough.

Cygwin is personally of no use to me (native applications like Julia can't work with it). I don't think I've ever downloaded a Sage Cygwin binary. For one, it's bloated and too slow.

Cygwin in not considered a native environment by serious Windows developers. It's a Linux on Windows, nothing more. It doesn't even try to play nice with native applications or the Windows ABI.

The main reason for the existence of the MSYS2 project in their own words is "better interoperability with native Windows software".

You skipped the "based on modern Cygwin (POSIX compatibility layer) and MinGW-w64" part, that's not very fair :p

[Francesco Biscani]

* PARI which assumes that sizeof(long) == sizeof(void*), there is an experimental branch fixing this: http://pari.math.u-bordeaux.fr/archives/pari-dev-1505/msg00021.html

I am using Pari (not GP) today on Windows 64. It was minimal effort on my part to do so. I am not using a special branch.

This was a problem for the x32 architecture as well. See, e.g., here:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=724320

Glad that the PARI devs fixed this eventually.

[kcrisman]

William Stein recently bemoaned the fact that SageMath currently only runs natively on some brands of Linux, and not natively on the latest Windows or OSX (that is to say nothing of BSD). [1]

 

Sage works on FreeBSD, or did recently, anyway; I haven't heard anything recently.  See http://trac.sagemath.org/query?status=!closed&component=porting%3A+BSD for some tickets, perhaps some outdated.

 

[Jean-Pierre Flori]

It was not fixed by an "actual" PARI dev, but what's important is that some code is now publicly available.

[Bill Hart]

On Wednesday, 7 October 2015 17:38:00 UTC+2, bluescarni wrote:

Some random thoughts:

- I am not so convinced the strategy of automatic long -> long long patching is actually feasible, I think in practice this is gonna be a big can of worms. Pushing upstream to fix their code is a much better long-term solution IMO (and I'd rather have nothing to do with projects which refuse portability and code-quality patches, but that's just me :)

I agree that's better if they will allow it. But I'm not sure some of the things SageMath depends on are even still maintained, let alone do all projects have the resources to keep maintaining such things, which is eventually what they get asked to do. Moreover, not all developers feel equally warm and fuzzy when you mention Windows. Some people are positively against supporting Windows. So what I am suggesting is a pragmatic compromise. And based on what I've seen with Pari, it works quite well. (I am sorry, I just haven't had the chance to look up when this porting work was done and how often it has been maintained, but I think it was something like 2008 or 2010 and with zero maintenance since, and it still works today).

[Bill Hart]

You mean there were bugs in the way posix was implemented in Cygwin?

I would have to see some examples to be able to judge how serious a problem this would be. I suspect Lisp might be problematic. But how many features of Sage that the average user makes use of hit these issues?

Anyway, the alternative is pretty much to do nothing. 

Bill.

[Bill Hart]

I really meant, the main reason given that Cygwin already exists. I think that the posix compatibility layer and MinGW-w64 are both features, but don't explain why it exists in the presence of other such projects.

Bill. 

[Francesco Biscani]

I agree that's better if they will allow it. But I'm not sure some of the things SageMath depends on are even still maintained, let alone do all projects have the resources to keep maintaining such things, which is eventually what they get asked to do. Moreover, not all developers feel equally warm and fuzzy when you mention Windows. Some people are positively against supporting Windows. So what I am suggesting is a pragmatic compromise. And based on what I've seen with Pari, it works quite well. (I am sorry, I just haven't had the chance to look up when this porting work was done and how often it has been maintained, but I think it was something like 2008 or 2010 and with zero maintenance since, and it still works today).

I think we risk of mixing different things up. One thing is actively supporting a Windows port from the point of view of build systems, providing binary packages, and so on (which people might or might not want to do due to personal inclinations, time/effort tradeoffs, etc.), another thing is code quality. The issue above with PARI, for instance, was affecting Linux x32 as well and it is just poor engineering in general.

I think that a high-quality project not providing support for on a OS as a "first-class citizen" is fixable with relatively little effort, even if upstream is not cooperative. To some extent, that is what the packagers of linux distributions do. But it becomes orders of magnitude more difficult if the project relies on platform-dependent system calls, libraries, or if it just happens to work only on specific arch/OS setups.

Personally I don't care about Windows or OSX as a user, but I make sure my software does compile and run properly on those platforms as a matter of code quality. Also, as you pointed out, it helps a lot if one starts doing this early in the development.

[Bill Hart]

I read through that entire Debian issue and still haven't got a clue what the issue was. There seems to have been a lot of arguing there about whose fault this actually was. None of these people is a dummy, so I don't understand why they couldn't figure it out.

Anyway, x32 is not something I'm interested in supporting. It's great that GMP does, and some other software does, and Flint is moving towards supporting it. But it's not a priority.

Bill. 

[Jeroen Demeyer]

On 2015-10-07 18:50, Bill Hart wrote:
>
>
> On Wednesday, 7 October 2015 18:33:01 UTC+2, Jeroen Demeyer wrote:
>
> On 2015-10-07 18:23, Bill Hart wrote:
> > Cygwin also provides a POSIX environment, so I'm not sure I
> understand
> > why this is an "elephant in the room".
>
> It's an elephant in the room because your original post completely
> seems
> to ignore any possible problems with the POSIX layer, while the POSIX
> layer is where most of the problems on Cygwin come from.
>
>
> You mean there were bugs in the way posix was implemented in Cygwin?

Well, it's not really bugs. The problem is that some things just don't
have a Windows equivalent. fork() is a good example, that just doesn't
exist on Windows.

Jeroen.

[Bill Hart]

Those are fair comments. But I'm not talking about the theoretical possibility of maybe thinking about Windows support. I'm actually right now building and using software on Windows.

What I'm trying to get started is a discussion about SageMath actually supporting Windows.

There's no practical way that can happen if we just decide that it's poor software engineering to not support a variety of OS's and that the reason nothing happens on Windows is because Sage picked a whole pile of dependencies that were designed by people who were poor software engineers. That won't lead to a practical solution to the problem. Sage is clearly not going to decide that these projects should no longer be used and kick them out. We are stuck with the choices that were made, for better or for worse (though I have personally learned a lot from the mistakes that have been made!)

As upstream packages are sometimes quite hostile to the idea of supporting Windows the proper way, it's clearly not pragmatic to impose the responsibility of fixing the problem on them. And experience shows they won't even accept the patches if you do it yourself.

The only practical solution in such instances is to do the patching in an automated way. You most certainly do not want to have an ongoing maintenance burden.

Bill.

 

[Francesco Biscani]

PARI requires (required?) sizeof(mp_limb_t)==sizeof(void*), which is not guaranteed and not enforceable by PARI (as it's up to GMP to decide what exactly an mp_limb_t is).

--

[Bill Hart]

Sure, but this doesn't make porting to Windows impossible. MSYS2 of course emulates fork/exec just like Cygwin. And yes, you still have the BLODA (big list of dodgy apps) that can cause fork to fail.

Whilst there are some Windows shells that try to deal with this issue, they aren't gaining much mindshare. 

In many cases, one can find an alternative to fork/exec (which apparently isn't even the most efficient way to do things on Linux much of the time that it is used).

In the end, a truly native Windows app wouldn't use fork. But I'm not advocating going the whole hog right away. There needs to be some intermediate steps between no Windows support and complete native Windows support. MSYS2 is a practical approach which serious Windows developers will take seriously (unlike Cygwin).

Bill.

[Bill Hart]

Actually, now I'm not even sure I understand what x32 is.

I looked it up and found this page and found considerable disagreement on what it is:

http://stackoverflow.com/questions/7635013/difference-between-x86-x32-and-x64-architectures

I think I'll give it a miss for a while.

Bill.

[Dima Pasechnik]

On Wednesday, 7 October 2015 07:35:22 UTC-7, Bill Hart wrote:

HI all,

William Stein recently bemoaned the fact that SageMath currently only runs natively on some brands of Linux, and not natively on the latest Windows or OSX (that is to say nothing of BSD). [1]

Until recently, a port of SageMath to Windows has seemed like a pipe dream. However, things have changed a lot, and I think it would now be possible to achieve this (for some definition of the word "native").

I've tried VM's before and it has always ended in disaster. They either screw up my networking, they have performance issues, or don't support my mouse properly, or change my keyboard layout, or it's impossible to get files from my hard drive into the system easily, or they take up too much disk space or need to be constantly downloaded to update them, or some features don't work, or people stop supporting them, etc. The disadvantages of VMs are so numerous I hardly need to enumerate them. And even if it is a solution for users, it's hardly a solution for serious Windows developers.

As some people know, I've been recently working on a Julia based "computer algebra system" for a much more limited subset of "computer algebra" than Sage targets. What people may not know is that that entire technology stack works natively on Windows.

I can't express how fantastic it was to be sitting on a train recently, with no web access whatsoever and to be able to do work on my Windows 10 (64 bit) laptop on the train. And everything ran as fast, or in some cases faster, than my Linux server. That's the first time that has happened since I started doing computational stuff about 10 years ago!

MSYS2 as a solution

================

The new technology that makes all this work is MSYS2. Here are some of its features:

* Installing MSYS2 on Windows couldn't be easier. [2]

* It supports proper Windows exception handling protocols.

* It has an amazing command line package manager called Pacman, which Just Works TM.

* Unlike Cygwin, it's very minimal, and takes little time to install.

* It's fast. Very fast.

* Parallel make works.

* The resulting binaries are fast, sometimes faster on my laptop than on the Linux server I usually use for development.

* MSYS2 provides a posix layer! (Applications can be distributed with an MSYS2 dll for this.)

* Multithreading using pthreads Just Works TM (Applications can be distributed with a winthreads dll for this. I've actually used this, no patching required, so I am not speaking theoretically here.)

* Like native Windows, sizeof(long) = 4 and sizeof(long long) = 8 on MSYS2. This means interop with native Windows applications or assembly objects is a cynch.

* The resulting applications can be run on Windows as essentially native applications. They don't have to be run from within the MSYS2 shell. They can also be distributed as binary packages for those that don't care about the source code. But here's the thing: it's not *necessary* to distribute them as binary packages. It's now quite feasible for developers to actually *build* packages on Windows. And let's face it, this is the type of developer we actually want on the Windows platform. Simply supporting users and not developers is just going to increase the maintenance burden for people who work on other platforms. We want actual Windows developers, not just users!

 

The only bad thing about MSYS2 is that autotools is still incredibly slow (Flint roll-your-own configure takes 5s at most on Windows, whereas autotools can take some minutes to configure a package). Note: make is not slow (at least not on Windows 64 -- it's not as fast on 32 bit Windows, but fortunately almost no one is using Windows 32 any more anyway).

If  MSYS2 has (Cygwin's?) fork(), it should not differ much from Cygwin, in the sense of porting of Sage.
fork() was the major pain on Win32, as building Python extensions results in huge number of dlls that have hard time
fitting into the (32bit) address space, so (semi)automatic running of rebase during the build was necessary...

(Note that on Cygwin (like elsewhere) Sage does not use Cygwin's python, it uses its own Python, which
seems to be quite different from Cygwin's Python)

OK, let me try to see if one gets anywhere with GAP on MSYS2...

Cheers,
Dima

[Bill Hart]

On Wednesday, 7 October 2015 19:48:54 UTC+2, bluescarni wrote:

PARI requires (required?) sizeof(mp_limb_t)==sizeof(void*), which is not guaranteed and not enforceable by PARI (as it's up to GMP to decide what exactly an mp_limb_t is).

I think I understand now. I was misled by the person claiming Pari required sizeof(long) == sizeof(void*) and Bill Allombert debunking it. Bill was of course correct (he is a Pari dev).

As for assuming sizeof(mp_limb_t) == sizeof(void*) we probably have the same issue in Flint. We will very slowly address this and have been making patches here and there. It's not a priority.

However, that issue is entirely irrelevant to MSYS2. By default on that system both sizeof(mp_limb_t) and sizeof(void*) are 8 on Windows 64.

The x32 issue is only affecting people who are wanting to support this trendy new ABI.

Bill.

[Francesco Biscani]

Practically, it's an architecture that supports "natively" 64 bit ints but the pointers are 32 bits wide. AFAIK, this is supposed to improve performance for pointer-heavy workloads that do not need to allocate much RAM but still benefit from the 64 bit ints.

--

[Francesco Biscani]

I think in FLINT you also have the issue that you are using tagged pointers (last time I checked anyway).

--

[Bill Hart]

I am really, really uninterested in Windows 32, as are, I imagine, most Windows developers these days. Given that Sage is so very far behind the ball game on Windows, I would skip it entirely. You will likely have to visit a museum to get a working Win32 box by the time the port is complete. (This was already the case back when Sage started its Cygwin port).

 

(Note that on Cygwin (like elsewhere) Sage does not use Cygwin's python, it uses its own Python, which
seems to be quite different from Cygwin's Python)

Seriously? So SageMath also risks not even having a supported Python in the future. Wowsers.

 

OK, let me try to see if one gets anywhere with GAP on MSYS2...

I'll give it a go myself, just to see what happens. It is probably completely broken, but only one way to find out.

Bill.

[Bill Hart]

On Wednesday, 7 October 2015 20:05:11 UTC+2, bluescarni wrote:

Practically, it's an architecture that supports "natively" 64 bit ints but the pointers are 32 bits wide. AFAIK, this is supposed to improve performance for pointer-heavy workloads that do not need to allocate much RAM but still benefit from the 64 bit ints.

I could be wrong, but this doesn't sound like it includes SageMath. :-)

[Bill Hart]

On Wednesday, 7 October 2015 20:06:41 UTC+2, bluescarni wrote:

I think in FLINT you also have the issue that you are using tagged pointers (last time I checked anyway).

Yeah, we merged some patches recently to fix some of the issues here. I don't think we got them all yet.

Sometimes compilers actually give warnings about this, but in a few cases I'm pretty convinced the compiler warning is actually incorrect.

Eventually we'll sort it out.

[Francesco Biscani]

On 7 October 2015 at 20:13, Bill Hart <goodwi...@googlemail.com> wrote:

I could be wrong, but this doesn't sound like it includes SageMath. :-)

Probably :)

I am not sure about the allocation limit, as the limit might only apply to large contiguous allocations. Or there might be other memory addressing tricks at play. Not 100% sure.

[Jean-Pierre Flori]

On Wednesday, October 7, 2015 at 8:03:15 PM UTC+2, Bill Hart wrote:

On Wednesday, 7 October 2015 19:48:54 UTC+2, bluescarni wrote:

PARI requires (required?) sizeof(mp_limb_t)==sizeof(void*), which is not guaranteed and not enforceable by PARI (as it's up to GMP to decide what exactly an mp_limb_t is).

I think I understand now. I was misled by the person claiming Pari required sizeof(long) == sizeof(void*) and Bill Allombert debunking it. Bill was of course correct (he is a Pari dev).

Pari also needed till recently sizeof(long) == sizeof(void*)

Look at http://pari.math.u-bordeaux.fr/cgi-bin/gitweb.cgi?p=pari.git;a=blob;f=src/headers/parigen.h;h=b8347dcfedd3e98154238faa4cb6eb04ce80d810;hb=HEAD#l16
This is dark magic to ensure that long get defined as long long on Windows 64.
This is also mentioned at page 5 here: raim2013.lip6.fr/theme/PDF/PARI-GP/[Belabas]PARI_GP.pdf

[Jean-Pierre Flori]

Same issue for Windows64, that's basically the way DLLs work which makes fork unimplementable in a not-broken way on Windows.
And yes, I'm quite sure MSYS2 has the same issue as Cygwin as MSYS2 is a frequently resynced fork of Cygwin.

[Peter Luschny]

 7. Oktober 2015 20:05:11 UTC+2, bluescarni:

Practically, it's an architecture that supports "natively" 64 bit ints but the pointers are 32 bits wide. AFAIK, this is supposed to improve performance for pointer-heavy workloads that do not need to allocate much RAM but still benefit from the 64 bit ints.

Wasn't it initiated by Donald Knuth's flame: "It is absolutely idiotic to have 64-bit pointers 

when I compile a program that uses less than 4 gigabytes of RAM." 

Peter

[Bill Hart]

4GB of Ram should be enough for anyone. :-)

[Bill Hart]

Gap built fine for me under MSYS2. There was one warning. (I used MPIR not GMP.)

However, I see why Gap built fine. Like Cygwin, MSYS2 sets sizeof(long) to 8. 

When I earlier said otherwise, I had in fact set up my MSYS2 to use the mingw-w64, and this builds things without the posix layer. Of course Gap is heavily posix dependent, so requires the MSYS2 gcc. So building with posix layer effectively results in sizeof(long) = 8, same as Linux.

So in that regard I'm actually not sure what the advantages are over Cygwin64 in terms of native interop are. 

Either way, that's one more project to add to the list that seems to work ok on Windows 64.

Bill.

On Wednesday, 7 October 2015 19:54:30 UTC+2, Dima Pasechnik wrote:

[Bill Hart]

OK, after more reading I find that these are the main benefits of MSYS2 over Cygwin:

* Support for interop with mingw-w64 built packages.

* Ability to switch from MSYS to MinGW mode by setting an environment variable (MSYSTEM).

* Automatic conversion on the fly in the msys2.0.dll between different path formats.

* Automatic handling in the msys2.0.dll of the different line endings (posix vs Windows).

* A direct recompilation of the Arch Linux package manager (Pacman).

* Various other more technical things.

The main benefit as explained by an MSYS2 person is that with MSYS2 you'll use a lot more natively compiled binaries, which will be faster.

The main benefits over MinGW are:

* It's actually being developed.

* It is regularly synced with the Cygwin project.

* MSYS2 bash is not inferior to Cygwin bash. 

Bill.

[Jean-Pierre Flori]

On Wednesday, October 7, 2015 at 11:27:25 PM UTC+2, Bill Hart wrote:

OK, after more reading I find that these are the main benefits of MSYS2 over Cygwin:

* Support for interop with mingw-w64 built packages.

* Ability to switch from MSYS to MinGW mode by setting an environment variable (MSYSTEM).

* Automatic conversion on the fly in the msys2.0.dll between different path formats.

* Automatic handling in the msys2.0.dll of the different line endings (posix vs Windows).

* A direct recompilation of the Arch Linux package manager (Pacman).

* Various other more technical things.

That's right. See:
* http://sourceforge.net/p/msys2/wiki/How%20does%20MSYS2%20differ%20from%20Cygwin/

In fact, there would even be a possibility to have a unique project offering both features but communication seems to be complicated between the two parties.
* https://github.com/Alexpux/MSYS2-packages/issues/83
* https://cygwin.com/ml/cygwin/2014-12/msg00143.html
 

The main benefit as explained by an MSYS2 person is that with MSYS2 you'll use a lot more natively compiled binaries, which will be faster.

I'm not sure that's completely true nor what natively really means.
What's sure is that whatever uses the POSIX compatibility layer (whether it is called cygwin1.dll or msys-2.0.dll) will surely sacrifice performance.
So the real performance deal is rather to use the mingw-w64 compiler to compile without POSIX compatibility or the cygwin(32/64)/msys(2) and get POSIX compatibility through an "emulation layer".

The main benefits over MinGW are:

* It's actually being developed.

* It is regularly synced with the Cygwin project.

* MSYS2 bash is not inferior to Cygwin bash. 

Note sure if you mean the old mingw(32?) project (http://www.mingw.org/) or the mingw(-w?)64 (or whatever it's called, http://mingw-w64.org/doku.php).
One of the main goal of MSYS2  is "just" to offer the possibility to run autotools and gcc-*-*-*-*-*-mingw(-w64) from a shell on Windows.
Remark you can also cross compile from Linux using mingw(32/64) and even run tests from Linux using Wine(64)!
Or "cross-"compile from Cygwin using mingw-w64 and run tests form a Cygwin shell :p !

So I guess that in the end, anyone should use the environment it prefers to compile his stuff, but the produced result will mostly depend on the compiler used and need for POSIXness:
* either POSIX compatibility is needed and then one needs cygwin1.dll or msys-2.0.dll (the latter one seemingly interacting better with Windows filesystem naming convention, the former with the POSIX naming convention),
* no POSIX compatibiliy is needed and then the way to go is mingw-w64 (or MSVC!).
The real deal is to make sure that both kind of produced libraries can communicate and that's not trivial when they assume that long's are of different sizes, some evil header magic has surely to be involved.
Fortunately, both kind of produced libraries use the same function call ABI (the (thanks Microsoft to have created your own ABI) Windows 64 one: https://msdn.microsoft.com/en-us/library/ms235286.aspx, different from the more usual system v one, see https://en.wikipedia.org/wiki/X86_calling_conventions#x86-64_calling_conventions and http://stackoverflow.com/questions/4429398/why-does-windows64-use-a-different-calling-convention-from-all-other-oses-on-x86), otherwise interoperability would have been even funnier to achieve.

Best,
JPF

[Simon King]

Hi Bill,

On 2015-10-07, Bill Hart <goodwi...@googlemail.com> wrote:
> 4GB of Ram should be enough for anyone. :-)

I have it on my laptop, and it is hardly enough to reasonably
build the Sage documentation with make -j2...

Cheers,
Simon

[Bill Hart]

On Thursday, 8 October 2015 10:26:06 UTC+2, Jean-Pierre Flori wrote:

On Wednesday, October 7, 2015 at 11:27:25 PM UTC+2, Bill Hart wrote:

OK, after more reading I find that these are the main benefits of MSYS2 over Cygwin:

* Support for interop with mingw-w64 built packages.

* Ability to switch from MSYS to MinGW mode by setting an environment variable (MSYSTEM).

* Automatic conversion on the fly in the msys2.0.dll between different path formats.

* Automatic handling in the msys2.0.dll of the different line endings (posix vs Windows).

* A direct recompilation of the Arch Linux package manager (Pacman).

* Various other more technical things.

That's right. See:
* http://sourceforge.net/p/msys2/wiki/How%20does%20MSYS2%20differ%20from%20Cygwin/

In fact, there would even be a possibility to have a unique project offering both features but communication seems to be complicated between the two parties.
* https://github.com/Alexpux/MSYS2-packages/issues/83
* https://cygwin.com/ml/cygwin/2014-12/msg00143.html
 

The main benefit as explained by an MSYS2 person is that with MSYS2 you'll use a lot more natively compiled binaries, which will be faster.

I'm not sure that's completely true nor what natively really means.
What's sure is that whatever uses the POSIX compatibility layer (whether it is called cygwin1.dll or msys-2.0.dll) will surely sacrifice performance.
So the real performance deal is rather to use the mingw-w64 compiler to compile without POSIX compatibility or the cygwin(32/64)/msys(2) and get POSIX compatibility through an "emulation layer".

JP, you are absolutely correct. And this is what I have overlooked in this whole discussion, and ultimately what kills the idea.

Currently, projects like Gap require far too much posix to make it easy to build them any other way but with the posix layer, and then it is no longer a native application.

It seems that I misunderstood the entire purpose of MSYS2 all along. It's really not much better than Cygwin from a native developer's point of view.

The reasons I got this wrong are complicated. It has to do with the fact that I believed that the native Julia app could make use of dlls built using the posix layer, so long as msys-2.0.dll is also supplied. This turns out to be false. The reason I believed it is still somewhat of a mystery to me, but must have to do with MSYSTEM being set to MINGW64 on my system when I built libflint, or something like that. But Julia most definitely doesn't support non-native dlls, and this won't likely change in a hurry.

One good thing came of all this. I have discovered that it is currently impossible to an build MPIR dll using the MSYS2 (with posix layer) gcc. This is due to configure and libtool not recognising msys as a host. Again, I formerly thought I was building MPIR with the MSYS2 gcc, but apparently was just building it using the mingw-w64 gcc.

Last night I did manage to get it to produce a dll, probably for the first time ever, on MSYS2, but it requires extensive patching. Fortunately GMP also doesn't build a dll there either.

A new version of MPIR will follow when I find time to patch it properly (if this is even possible).

So, my conclusion after all this is that it is currently NOT feasible to produce a native Windows 64 SageMath, without first weaning libraries off posix. For Gap, that would be quite some work.

Bill.

[Jeroen Demeyer]

On 2015-10-08 12:22, Bill Hart wrote:
> Currently, projects like Gap require far too much posix to make it easy
> to build them any other way but with the posix layer, and then it is no
> longer a native application.

There is one thing I don't understand in this whole discussion: what's
the problem really with the POSIX layer? Why do you insist on a *native*
application? Getting Sage to work properly as Cygwin/MSYS2 application
would already be a great achievement, so why not aim for that?

[Bill Hart]

It's very simple. Julia cannot access non-native dlls. And in fact, neither can other native binaries.

The Julia devs explain, "Loading and using 2 different C runtime libraries (the posix cygwin dll, and the non-posix msvcrt used by mingw-w64) from inside the same application is unlikely to work"

Therefore there would in fact be little overlap between SageMath requirements (which include posix) and Julia requirements (which exclude posix).

I suppose there would be some benefit in having libraries fix the long->long long issues. But it's nowhere near all the way to a native application. SageMath would indeed benefit. But that doesn't help me or others working with native apps.

Bill.

[Bill Hart]

Actually, I suppose there is some common overlap, namely in those packages which currently don't build on Windows 64 that don't have a posix layer requirement. Those could be useful ports for both SageMath and for Julia. So there is perhaps still some hope for something.

Bill.

[Bill Hart]

On Wednesday, 7 October 2015 17:38:00 UTC+2, bluescarni wrote:

Some random thoughts:

- I am not so convinced the strategy of automatic long -> long long patching is actually feasible, I think in practice this is gonna be a big can of worms. Pushing upstream to fix their code is a much better long-term solution IMO (and I'd rather have nothing to do with projects which refuse portability and code-quality patches, but that's just me :)

- mingw-w64 is a high quality compiler. I recently had to develop for a while on a Windows 64 machine, and had to recompile a full stacks of dependencies (MPFR, GMP, Boost, etc.). It was not so easy (most issues were related to build systems), but the end result was pretty impressive performance-wise.

- While for C code you can probably interoperate with libraries compiled with MSVC, for C++ you will end up having issues related, e.g., to different implementations of exception handling. My suggestion would be to forget about MSVC and compile the full stack of dependencies exclusively with mingw.

- Python on windows 64 does not play properly with mingw due to some long-standing issues in some header files. If you want to compile Python C/C++ extensions on Win64 you will have to patch slightly the Python headers and re-generate the definitions of the Python library. The issue is explored, e.g., here:

http://ascend4.org/Setting_up_a_MinGW-w64_build_environment#Setup_Python_for_compilation_of_extensions

Someone has asked me off list to point out the following:

"the MSYS2 project have ported CPython and have provided four packages that don't use MSVC: mingw-w64-{i686,x86_64}-ptython{2,3} (as well as ones that link to msys-2.0.dll)."

I see that there is a list of packages here:

http://sourceforge.net/p/msys2/wiki/Packages/

 

Generally speaking, mingw-w64 is a really good option for C/C++ development on Windows. The biggest problem is the proliferation of distributions (mingw-w64, TDM mingw, mingw-build, msys2, etc.), but the toolchain is solid. clang is getting there as well, so it's worth keeping it in mind for the future.

Cheers,

  Francesco.

On 7 October 2015 at 16:35, Bill Hart <goodwi...@googlemail.com> wrote:

HI all,

William Stein recently bemoaned the fact that SageMath currently only runs natively on some brands of Linux, and not natively on the latest Windows or OSX (that is to say nothing of BSD). [1]

Until recently, a port of SageMath to Windows has seemed like a pipe dream. However, things have changed a lot, and I think it would now be possible to achieve this (for some definition of the word "native").

I've tried VM's before and it has always ended in disaster. They either screw up my networking, they have performance issues, or don't support my mouse properly, or change my keyboard layout, or it's impossible to get files from my hard drive into the system easily, or they take up too much disk space or need to be constantly downloaded to update them, or some features don't work, or people stop supporting them, etc. The disadvantages of VMs are so numerous I hardly need to enumerate them. And even if it is a solution for users, it's hardly a solution for serious Windows developers.

As some people know, I've been recently working on a Julia based "computer algebra system" for a much more limited subset of "computer algebra" than Sage targets. What people may not know is that that entire technology stack works natively on Windows.

I can't express how fantastic it was to be sitting on a train recently, with no web access whatsoever and to be able to do work on my Windows 10 (64 bit) laptop on the train. And everything ran as fast, or in some cases faster, than my Linux server. That's the first time that has happened since I started doing computational stuff about 10 years ago!

MSYS2 as a solution

================

The new technology that makes all this work is MSYS2. Here are some of its features:

* Installing MSYS2 on Windows couldn't be easier. [2]

* It supports proper Windows exception handling protocols.

* It has an amazing command line package manager called Pacman, which Just Works TM.

* Unlike Cygwin, it's very minimal, and takes little time to install.

* It's fast. Very fast.

* Parallel make works.

* The resulting binaries are fast, sometimes faster on my laptop than on the Linux server I usually use for development.

* MSYS2 provides a posix layer! (Applications can be distributed with an MSYS2 dll for this.)

* Multithreading using pthreads Just Works TM (Applications can be distributed with a winthreads dll for this. I've actually used this, no patching required, so I am not speaking theoretically here.)

* Like native Windows, sizeof(long) = 4 and sizeof(long long) = 8 on MSYS2. This means interop with native Windows applications or assembly objects is a cynch.

* The resulting applications can be run on Windows as essentially native applications. They don't have to be run from within the MSYS2 shell. They can also be distributed as binary packages for those that don't care about the source code. But here's the thing: it's not *necessary* to distribute them as binary packages. It's now quite feasible for developers to actually *build* packages on Windows. And let's face it, this is the type of developer we actually want on the Windows platform. Simply supporting users and not developers is just going to increase the maintenance burden for people who work on other platforms. We want actual Windows developers, not just users!

 

The only bad thing about MSYS2 is that autotools is still incredibly slow (Flint roll-your-own configure takes 5s at most on Windows, whereas autotools can take some minutes to configure a package). Note: make is not slow (at least not on Windows 64 -- it's not as fast on 32 bit Windows, but fortunately almost no one is using Windows 32 any more anyway).

[3] https://build.opensuse.org/

--

[David Joyner]

Before this thread drops off the radar, I have a question. How hard
would it be to rebuild (not port) Sage starting with windows Python,
then adding windows GAP, windows SIngular, networkxx, and
SymPy+friends, of which GAP+Singular communicate with the Sage
terminal via pexpect? Call it WinSage 1.0:-)

I don't even use windows so really this is a totally naive question.

[William Stein]

On Thursday, October 8, 2015, David Joyner <wdjo...@gmail.com> wrote:

Before this thread drops off the radar, I have a question. How hard
would it be to rebuild (not port) Sage starting with windows Python,
then adding windows GAP, windows SIngular, networkxx, and
SymPy+friends, of which GAP+Singular communicate with the Sage
terminal via pexpect? Call it WinSage 1.0:-)

Just curious - what do you mean by "the sage terminal"? 

Also to stop people jumping on pexpect which doesn't at all make sense or work natively on Windows, replace that by "some sort of IPC".

--
Sent from my massive iPhone 6 plus.

[David Joyner]

On Thu, Oct 8, 2015 at 2:58 PM, William Stein <wst...@gmail.com> wrote:
>
>
> On Thursday, October 8, 2015, David Joyner <wdjo...@gmail.com> wrote:
>>
>> Before this thread drops off the radar, I have a question. How hard
>> would it be to rebuild (not port) Sage starting with windows Python,
>> then adding windows GAP, windows SIngular, networkxx, and
>> SymPy+friends, of which GAP+Singular communicate with the Sage
>> terminal via pexpect? Call it WinSage 1.0:-)
>
>
> Just curious - what do you mean by "the sage terminal"?
>

I was referring to some sort of cygwin command line (since I hardly
ever use anything else), but if people prefer an ipython notebook-like
interface then that's fine too.

[William Stein]

On Thursday, October 8, 2015, David Joyner <wdjo...@gmail.com> wrote:

On Thu, Oct 8, 2015 at 2:58 PM, William Stein <wst...@gmail.com> wrote:
>
>
> On Thursday, October 8, 2015, David Joyner <wdjo...@gmail.com> wrote:
>>
>> Before this thread drops off the radar, I have a question. How hard
>> would it be to rebuild (not port) Sage starting with windows Python,
>> then adding windows GAP, windows SIngular, networkxx, and
>> SymPy+friends, of which GAP+Singular communicate with the Sage
>> terminal via pexpect? Call it WinSage 1.0:-)
>
>
> Just curious - what do you mean by "the sage terminal"?
>

I was referring to some sort of cygwin command line (since I hardly
ever use anything else), but if people prefer an ipython notebook-like
interface then that's fine too.

This whole thread is about creating such a thing - in any way at all that actually *works*  - in the first place.   

There is no usable sage command line for Windows.  Native or not.  

[Bill Hart]

On Thursday, 8 October 2015 20:32:20 UTC+2, David Joyner wrote:

Before this thread drops off the radar, I have a question. How hard
would it be to rebuild (not port) Sage starting with windows Python,
then adding windows GAP, windows SIngular, networkxx, and
SymPy+friends, of which GAP+Singular communicate with the Sage
terminal via pexpect? Call it WinSage 1.0:-)

Assuming you allow posix, probably not hard at all. Python, Gap, Singular all build under MSYS2. SymPy is just pure Python right? (I don't know anything about networkx or pexpect.) You probably need Cython. Don't know what's required there, but it's Python isn't it?

Native would be difficult but not impossible. Gap is probably the most difficult to port to native, since you'd have to remove the posix stuff, which is used for tty and sockets. It also has some dependencies, some of which have some posix. There's probably some posix in Singular too, to communicate with Gap I assume. I didn't check yet.

As someone pointed out to me this morning, there is a big push at the moment for maths libraries on Native Windows from the scientific Python guys, R and Julia. So now is the time to jump on the native bandwagon.

Bill.

[Jean-Pierre Flori]

On Thursday, October 8, 2015 at 9:10:17 PM UTC+2, William wrote:

On Thursday, October 8, 2015, David Joyner <wdjo...@gmail.com> wrote:

On Thu, Oct 8, 2015 at 2:58 PM, William Stein <wst...@gmail.com> wrote:
>
>
> On Thursday, October 8, 2015, David Joyner <wdjo...@gmail.com> wrote:
>>
>> Before this thread drops off the radar, I have a question. How hard
>> would it be to rebuild (not port) Sage starting with windows Python,
>> then adding windows GAP, windows SIngular, networkxx, and
>> SymPy+friends, of which GAP+Singular communicate with the Sage
>> terminal via pexpect? Call it WinSage 1.0:-)
>
>
> Just curious - what do you mean by "the sage terminal"?
>

I was referring to some sort of cygwin command line (since I hardly
ever use anything else), but if people prefer an ipython notebook-like
interface then that's fine too.

This whole thread is about creating such a thing - in any way at all that actually *works*  - in the first place.   

There is no usable sage command line for Windows.  Native or not.  

On cygwin32 the result was decent.
On cygwin64 a little more work was needed.
See http://trac.sagemath.org/wiki/Cygwin64Port

[William Stein]

How many people *use* it?   

 

On cygwin64 a little more work was needed.
See http://trac.sagemath.org/wiki/Cygwin64Port

 

--

You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To post to this group, send email to sage-...@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

[Jean-Pierre Flori]

On Friday, October 9, 2015 at 12:27:05 AM UTC+2, William wrote:

On Thursday, October 8, 2015, Jean-Pierre Flori <jpf...@gmail.com> wrote:

On Thursday, October 8, 2015 at 9:10:17 PM UTC+2, William wrote:

On Thursday, October 8, 2015, David Joyner <wdjo...@gmail.com> wrote:

On Thu, Oct 8, 2015 at 2:58 PM, William Stein <wst...@gmail.com> wrote:
>
>
> On Thursday, October 8, 2015, David Joyner <wdjo...@gmail.com> wrote:
>>
>> Before this thread drops off the radar, I have a question. How hard
>> would it be to rebuild (not port) Sage starting with windows Python,
>> then adding windows GAP, windows SIngular, networkxx, and
>> SymPy+friends, of which GAP+Singular communicate with the Sage
>> terminal via pexpect? Call it WinSage 1.0:-)
>
>
> Just curious - what do you mean by "the sage terminal"?
>

I was referring to some sort of cygwin command line (since I hardly
ever use anything else), but if people prefer an ipython notebook-like
interface then that's fine too.

This whole thread is about creating such a thing - in any way at all that actually *works*  - in the first place.   

There is no usable sage command line for Windows.  Native or not.  

On cygwin32 the result was decent.

How many people *use* it?  

Not even me!
I guess it misses advertising, proper packaging, proper testing, proper continuous integration, ...

[William Stein]

>>> On cygwin32 the result was decent.
>>
>>
>>
>> How many people *use* it?
>
> Not even me!
> I guess it misses advertising, proper packaging, proper testing, proper
> continuous integration, ...

The last few times I spent months on Windows porting, the net result
of using Cygwin was -- in practice -- so much worse than just using a
Linux VM, that nobody would actually use it. I don't want to
minimize the great value of the work you've done, but I was never
convinced that Cygwin sage was actually viable at all.

-- William

--
William (http://wstein.org)

[Bill Hart]

What were the main issues? Were they mainly in the build process or in actually using Sage?

Some ways to possibly improve it might be:

* Have a script that downloads binaries for all the dependencies of Sage (still keep the source code intact of course) so that these don't need to be built to use it.

* Peg to a commit that is known to work (and for which binaries are available), but naturally allow the user to advance to later commits and rebuild anything that needs to be rebuilt.

* Ensure that rebuilding Sage only rebuilds those binary dependencies that have actually been changed when doing a git pull.

* Advertise it.

* Give some idea to users how long it is expected to take on a modern Windows laptop to get up and running from scratch, including an estimate for download of source and binaries on a certain speed internet connection. People want to know if they can fit it in between lectures or if it will tie up their machine for the next week and a half.

* Link to the BLODA.

* MSYS2 might be nicer in the long run. It has a command line package manager, so you don't need to tell users to get this and that dependency even before starting. Admittedly not all the dependencies you ask for are there yet. MSYS2 is also itself quicker and nicer to install in my experience.

* Provide a binary only version for people who want to try it out first, or who really don't want to do any development.

* I still think some more modularisation of Sage would help. Anything that isn't needed to do computations in Sage could be separate. Some particularly expensive parts of sage bloat-wise could also have hooks that just tell you that such and such is missing, and telling you precisely what you have to type to get it, like what happens in the shell in Ubuntu.

People will use it if it is practical to do so, especially if others are using it and word gets about. If it's clear that it won't ever be practical, then maybe another strategy is needed.

I was curious about the comment on the Cygwin64 port page where Pari declared 131 composite. Knowing just how little is involved in that computation, it's hard to understand how that happened on Cygwin 64. Was this Cygwin related?

Bill.