Status update?

[Olivier Grisel]

Hi Carl!

Now that phase 1 is validated, can you please give a status update on
the next steps on your side? Do you need help? I can build an uploads
to rackspace two toolchains to get started with the script to generate
the mingwpy wheel packages.

--
Olivier
http://twitter.com/ogrisel - http://github.com/ogrisel

[Carl Kleffner]

Hi Olivier,

after some hassle at beginning of the week I was much more involved in using the toolchain rather than continuing on phase2. Some projects has been waiting for it.

I would like to have stable download points of the standalone toolchains first, because the scripts to build mingwpy wheels depend on these toolchains. This is a not a problem for local development as I do, but I want the scripts to be usable for others as well.

An upload on Rackspace would really be helpful.

Cheers,

Carl

--
You received this message because you are subscribed to the Google Groups "mingwpy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mingwpy+u...@googlegroups.com.
To post to this group, send email to min...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mingwpy/CAFvE7K5doHjjwKG8zM4YFkNJ-1vWi0Kq-Ju7sfNb7xE82SYYhg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Olivier Grisel]

Ok starting a VM to rebuild the toolchain from current trunk now. I
will upload the results in a couple of hours hopefully.

--
Olivier

[Olivier Grisel]

Unfortunately the build is broken again (when using the mingw-w64
trunk), this time it's related to libgfortran:

https://gist.githubusercontent.com/anonymous/d7966605ffdf23e23f1d/raw/f94bdebeafcb8d9581893234f70611ed399a98db/gcc-build-failure.txt

```
libtool: link: ar rc .libs/libcaf_single.a single.o
libtool: link: ranlib .libs/libcaf_single.a
libtool: link: ar rc .libs/libgfortranbegin.a fmain.o
C:\msys64\home\Administrator\mingw-builds\toolchains\mingw64\bin\ranlib.exe:
unable to rename '.libs/libcaf_single.a'; reason: No such file or
directory
libtool: link: ranlib .libs/libgfortranbegin.a
Makefile:1467: recipe for target 'libcaf_single.la' failed
make[3]: *** [libcaf_single.la] Error 1
```

I think we need to make the mingwpy mingw-builds script download a
specific revision of the mingw-w64 trunk that is know to work.

--
Olivier

[Olivier Grisel]

The above error was obtained when building the 64 bit toolchain. I am
now trying to see if the 32 bit toolchain has the same issue.

--
Olivier

[Olivier Grisel]

The build for the 32 bit toolchain was successful, I will upload it now.

--
Olivier

[Carl Kleffner]

Olivier,

you may try to rebuild the 64bit toolchain with a reduced number of jobs. I got a similar error some time ago with 4 jobs on 4 cores and a sucessfull build with 2 jobs.

Thanks for your efforts

Carl

--
Olivier

--
You received this message because you are subscribed to the Google Groups "mingwpy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mingwpy+u...@googlegroups.com.
To post to this group, send email to min...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/mingwpy/CAFvE7K6Ev6J%3DmLx04xKuaCreeoS20R6mBeYSGJ_CiLkcU82xDw%40mail.gmail.com.

[Olivier Grisel]

2016-03-17 22:17 GMT+01:00 Carl Kleffner <cmkle...@gmail.com>:
> Olivier,
>
> you may try to rebuild the 64bit toolchain with a reduced number of jobs. I
> got a similar error some time ago with 4 jobs on 4 cores and a sucessfull
> build with 2 jobs.

I suspected a race condition in the parallel build system re-launched
(still with 32 jobs) to see if this error is deterministic.

--
Olivier

[Carl Kleffner]

BTW,

the next steps will be

(1) creating updated mingwpy wheels

(2) compiling and testing scipy on the top of matthews numpy wheels

(3) redo the numpy PR for mingwpy, redo (2) on the top of mingwpy compiled numpy wheels

(4) all other stuff for Phase2

Carl

--
Olivier

--
You received this message because you are subscribed to the Google Groups "mingwpy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mingwpy+u...@googlegroups.com.
To post to this group, send email to min...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/mingwpy/CAFvE7K5HuyT0wXgVahLq7%2BfUeADrA%3Dvv2LVt%3DdCufOL0qJOOrg%40mail.gmail.com.

[Carl Kleffner]

made with mingwpy:

https://anaconda.org/carlkl/libtfr: wheels for libtfr, see https://github.com/melizalab/libtfr/issues/4, that will be needed for iq_suite (unreleased): https://github.com/xaratustrah/iq_suite

C

[Olivier Grisel]

Nice!

[Olivier Grisel]

Ok I good the x86_64 toolchain to build the second time. Here are the links:

https://8f55531b0432ac914da3-7f407ad05f03da00978ba0b8ec6c202f.ssl.cf2.rackcdn.com/i686-5.3.0-static-release-win32-sjlj-rt_v4-rev20160317.7z
https://8f55531b0432ac914da3-7f407ad05f03da00978ba0b8ec6c202f.ssl.cf2.rackcdn.com/x86_64-5.3.0-static-release-win32-seh-rt_v4-rev20160317.7z

--
Olivier

[Carl Kleffner]

great work!

Carl

--
Olivier

--
You received this message because you are subscribed to the Google Groups "mingwpy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mingwpy+u...@googlegroups.com.
To post to this group, send email to min...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/mingwpy/CAFvE7K6WgG1x_u53Q_ziG2fgA_454%2BBCMiMMea6RoUpSudtRqg%40mail.gmail.com.

[Matthew Brett]

Hi Carl,

Did you succeed in building the scipy wheels with the ATLAS libraries?

I've got a setup I'd like to test them on, if they are available somewhere.

Cheers,

Matthew

[carlkl]

Hi Matthew,

do you have renewed ATLAS builds available with shrunken exports for the DLL (without the gfortran symbols)? With that I would like to build scipy against ATLAS.

Carl

[carlkl]

Hi,

I made an adhoc test of scipy build against numpy-atlas.dll using Matthew Bretts MSVC/ATLAS numpy builds available at https://pypi.python.org/pypi/numpy/1.10.4. I didn't build against python-3.5 as mingwpy still does not support the VS 2015 UCRT runtime.

python-2.7 (64bit) as well as python-3.4 (64bit) do not show any errors:

3.4: Ran 20177 tests in 195.518s
     OK (KNOWNFAIL=98, SKIP=1696)
2.7: Ran 20335 tests in 155.239s
     OK (KNOWNFAIL=97, SKIP=1706)

The 32 bit versions pops up with a lot of failures and errors:

3.4: Ran 20177 tests in 179.717s
     FAILED (KNOWNFAIL=98, SKIP=1696, errors=172, failures=22)
2.7: Ran 20335 tests in 161.703s
     FAILED (KNOWNFAIL=97, SKIP=1706, errors=172, failures=21)

I try to rerun against a drop-in replacement with the latest OpenBLAS version 0.2.16 and give more feedback.

Carl

[carlkl]

I copied the scipy test errors (python-3.4.4 win32) on: https://gist.github.com/carlkl/9e9aa45f49fedb1a1ef7

Carl

[Matthew Brett]

On Sun, Mar 20, 2016 at 1:25 PM, carlkl <cmkle...@gmail.com> wrote:
> I copied the scipy test errors (python-3.4.4 win32) on:
> https://gist.github.com/carlkl/9e9aa45f49fedb1a1ef7
>
> Carl
>
>
> Am Sonntag, 20. März 2016 16:38:02 UTC+1 schrieb carlkl:
>>
>> Hi,
>>
>> I made an adhoc test of scipy build against numpy-atlas.dll using Matthew
>> Bretts MSVC/ATLAS numpy builds available at
>> https://pypi.python.org/pypi/numpy/1.10.4. I didn't build against python-3.5
>> as mingwpy still does not support the VS 2015 UCRT runtime.
>>
>> python-2.7 (64bit) as well as python-3.4 (64bit) do not show any errors:
>>
>> 3.4: Ran 20177 tests in 195.518s
>> OK (KNOWNFAIL=98, SKIP=1696)
>> 2.7: Ran 20335 tests in 155.239s
>> OK (KNOWNFAIL=97, SKIP=1706)
>>
>> The 32 bit versions pops up with a lot of failures and errors:
>>
>> 3.4: Ran 20177 tests in 179.717s
>> FAILED (KNOWNFAIL=98, SKIP=1696, errors=172, failures=22)
>> 2.7: Ran 20335 tests in 161.703s
>> FAILED (KNOWNFAIL=97, SKIP=1706, errors=172, failures=21)
>>
>> I try to rerun against a drop-in replacement with the latest OpenBLAS
>> version 0.2.16 and give more feedback.

Ouch.

I guess this is scipy 0.17? Or trunk?

When was the last time / version that scipy got tested on Windows 32-bit?

Thanks very much for working on this.

Matthew

[carlkl]

This was scipy-0.1.7.0. Most errors are due to Arpack (there was a change to Arpack-NG, recently?). I have to recompile scipy with OpenBLAS to cross check, but possibly not today :(

Carl


Am Sonntag, 20. März 2016 21:31:49 UTC+1 schrieb Matthew Brett:

[anatoly techtonik]

On Friday, March 18, 2016 at 12:30:57 AM UTC+3, carlkl wrote:

BTW,

the next steps will be

(1) creating updated mingwpy wheels

(2) compiling and testing scipy on the top of matthews numpy wheels

(3) redo the numpy PR for mingwpy, redo (2) on the top of mingwpy compiled numpy wheels

 

(4) all other stuff for Phase2

I can do Appveyour and Travis CI scripts as well as build automation

https://mingwpy.github.io/proposal_december2015.html

I am short of money, so maybe there are also funding opportunities

for this task? I'd also like to try fundraising for the cause on

https://gratipay.com/

[Olivier Grisel]

I don't want to speak for the numpy devs but I think the Numfocus /
PSF grants targeted contributions that required specific skills that
are not commonly found among regular open source contributors of the
Python ecosystems, more specifically, inner details of the binary
compatibility of extensions built by custom compiler toolchains. To me
at least this is the priority for funding allocations (stuff I would
be completely incapable of doing myself).

[Ralf Gommers]

Agreed. Also, it's not like we have lots of money freely available. We managed to get a limited amount of funding, after a lot of development work already put in by Carl plus a nontrivial amount of discussions and grant writing, and that funding is already committed.

Ralf

[anatoly techtonik]

I see. So it a lot of hidden work behind the scene. I would be really
interested to know some estimation of the efforts (in hours?) so that I can
reference this thread for people from open source scene who are looking
into applying for a grant too.
https://github.com/ralphtheninja/open-funding

The main questions that people are interested to know if it is worthy to
spend time to apply for a grant, because if the time required to prepare a
call is so great, then all funds raised will just go into compensating the
paperwork efforts. We have some rumors about that, but no real data.

Also, if that data can prove that project is underfunded, maybe it is
possible apply for an additional funding along the way? At least it is
possible to register the project on the http://gratipay.com/ to see if people
find the idea worthy of a weekly tips.

--
anatoly t.