FYI: discussions of linux wheels happening

[Nathaniel Smith]

Hi all,

Since there's probably some overlap between people who care about
trying to distribute windows wheels and people who care about trying
to distribute linux wheels, I figured you might be interested in a
heads-up that some of us are talking about that:

https://github.com/manylinux/manylinux
https://groups.google.com/forum/#!forum/manylinux-discuss

There are definitely some topics in common (e.g. what to do with BLAS
libraries), though in both cases there are some grotty
platform-specific hurdles that need to be overcome first.

Cheers,
-n

--
Nathaniel J. Smith -- http://vorpus.org

[Olivier Grisel]

Thanks for the heads up, this thread is really interested to read, I
learned a lot. I subscribed to the manylinux group.

---
Olivier

[carlkl]

This thread - as well as the corresponding numpy-discussion thread - is indeed a very interesting reading.

Windows is a very different beast. It is like running Linux with multiple GLIBC versions. Some kind of schizophrenia. The reason is, that there is a huge amount of software all around the world depending on all this different msvcr##.dll runtime versions. MS has to support all this old stuff because almost no one is recompiling software on Windows. Self contained binary distributions without source are the gold standard for Windows users. For the good AND the bad.

Linux software is based on source distributions. Package managers like pacman, rpm and also conda are only lifesavers that cuts back the flexibility but reduces the maintainance burden because others has done the job for you.

The reason for the existance of tools like cygwin or msys2 on Windows is not only to support POSIX functionality on Windows to some extend but also to have a software environment similar to Unix: It should allow the usage of make, autotools, libtool in addition to a gcc toolchain and OSS libraries. That is, bring the power of OSS to windows. And the power of UNIX tools for software development. Very different compared to the world of Visual Studio.

My vision is, that mingwpy does not only support the gcc toolchain itself, but also supports OSS libraries deployed as subpackags of mingwpy. To build OSS libraries one depends on systems like msys2 as well as some support code to make a suitable wheel. In the end the user should be able to just pip install mingwpy.openblas to install OpenBLAS with import library and header files into the mingwpy folder structure. Conda instead of pip should work as well.

I impression is, that is the easiest way to support OSS libraries that are needed for compiling python binary extensions on windows. In the end a python programmer just wants to install all software components as easy as possible to be able to produce his own extensions.

Carl

[Nathaniel Smith]

On Fri, Jan 15, 2016 at 12:56 PM, carlkl <cmkle...@gmail.com> wrote:
> My vision is, that mingwpy does not only support the gcc toolchain itself,
> but also supports OSS libraries deployed as subpackags of mingwpy. To build
> OSS libraries one depends on systems like msys2 as well as some support code
> to make a suitable wheel. In the end the user should be able to just pip
> install mingwpy.openblas to install OpenBLAS with import library and header
> files into the mingwpy folder structure. Conda instead of pip should work as
> well.

Generally agree with everything in this email, but I just wanted to
pull this one piece out and respond--

The downside to having mingwpy.openblas is that we also want openblas
wheels on osx and linux :-). And ideally we should not have three
different openblas wheel projects with different names. So what you
say sounds right to me, just we'll probably want to eventually pull it
out from under the mingwpy umbrella to become 'openblaspy' or
'pylibs.openblas' or something, and share the work across operating
systems.

[carlkl]

Am Montag, 18. Januar 2016 01:16:47 UTC+1 schrieb Nathaniel Smith:

I propose to have 3rd party libraries under the hood of mingwpy:

- mingwpy_openblas
- mingwpy_png
- mingwpy_jpg
- mingwpy_fftw
- mingwpy_hdf5
- mingwpy-XXXX

This is a possibility to build up an installation of the toolchain with all required libraries WITHIN a python installation. This would allow creation of binary extensions depending only on pypi wheels. The downside of this is that this names of the packages are tied up to windows installations.

Carl
 

[Nathaniel Smith]

Right, it's that last sentence that I'm worried about :-). How about
clib_openblas, clib_png, etc.? Because the alternative is having
mingwpy_openblas and osx_openblas and linux_openblas, which are all
identical except for what compiler is used.

-n

--
Nathaniel J. Smith -- https://vorpus.org

[carlkl]

Ok, I will use release names like clib_openblas, clib_libjpeg and so on if there is an agreement to use this package name mangling for packages containing library binary and development files.

Carl