Threshold for test failures, and randstate.pyx

[Ximin Luo]

Hi,

Just wondering what is considered an "acceptable" number of test failures to release in distro-packaged Sage.

François has told us before that Gento has about 30. At the moment on Debian amd64 we have about 40 for Sage 7.4, 13 of which are the following (edited for clarity):

sage -t --long src/sage/misc/randstate.pyx
**********************************************************************
File "src/sage/misc/randstate.pyx", line 57, in sage.misc.randstate
Failed example:
rtest()
Expected:
([..], 1/2*x^2 - 1/95*x - 1/2, (1,3,2), [ 0, 0, 0, 0, 1 ], [..])
Got:
([..], 1/2*x^2 - 1/95*x - 1/2, (1,3), [ 0, 0, 0, 0, 1 ], [..])
**********************************************************************
File "src/sage/misc/randstate.pyx", line 60, in sage.misc.randstate
Failed example:
rtest()
Expected:
([..], -3*x^2 - 1/12, (1,3,2), [ 0, 1, 1, 0, 0 ], [..])
Got:
([..], -3*x^2 - 1/12, (1,2), [ 0, 1, 1, 0, 0 ], [..])
**********************************************************************
File "src/sage/misc/randstate.pyx", line 63, in sage.misc.randstate
Failed example:
rtest()
Expected:
([..], 4*x^2 + 1/2, (1,3,2), [ 0, 0, 1, 0, 1 ], [..])
Got:
([..], 4*x^2 + 1/2, (1,2)(4,5), [ 0, 0, 1, 0, 1 ], [..])

[.. etc all similar to these .. ]

1 item had failures:
13 of 60 in sage.misc.randstate
[144 tests, 13 failures, 12.92 s]

but we have coded our build scripts to pass anything < 80 test failures (and without "special" failures like segfault, timeout, abort, etc). Is this "good enough"? And does anyone have a hint on this randstate.pyx issue?

X

--
GPG: ed25519/56034877E1F87C35
GPG: rsa4096/1318EFAC5FBBDBCE
https://github.com/infinity0/pubkeys.git

[nqn...@gmail.com]

I don't think that the raw number of test failures is an adequate measure for anything. You can't compare a test that gives an incorrect result or a test that segfaults with one that fails because of different output format or different installation path of some libraries.
In Arch we currently have lots of test failures (with 7.5.1) which are due to the changes in output format in the latest versions of maxima and pari. The results are still correct, so I don't consider them a problem for releasing it or bother trying to fix them. One test that segfaults would be a much more serious issue than 100 failures of this type IMO.

[Ximin Luo]

nqn...@gmail.com:

> I don't think that the raw number of test failures is an adequate measure for anything. You can't compare a test that gives an incorrect result or a test that segfaults with one that fails because of different output format or different installation path of some libraries.
> In Arch we currently have lots of test failures (with 7.5.1) which are due to the changes in output format in the latest versions of maxima and pari. The results are still correct, so I don't consider them a problem for releasing it or bother trying to fix them. One test that segfaults would be a much more serious issue than 100 failures of this type IMO.
>

I agree in general and on small scales, but I was hoping that Sage is large enough that the law of large numbers takes over.

I also agree that many test failures are not so important, but the issue with blanket-ignoring them is that this could hide other more serious issues. If today you have 180 test failures and tomorrow you have 195, you might overlook this, but you'd pay more attention if you had 15 one day then 30 the next.

So if we fix the "unimportant" cases, this at least brings down the total number of failures to a small-enough value, that "important failures" are more noticeable.

For example, an "important" problem might cause >20 test failures, but we'd only really notice this if we were disciplined about actually overriding and patching the "unimportant" failures. Plus, I don't know what the correct number is for Sage, is it >20, >100, or >1?

In other words, have you guys ever experienced a "important" (e.g. release-critical) issue that only caused 1 doctest failure? If not 1, what is the minimal number, in your experience?

(After a bit more investigation, it seems the randstate.pyx failures are probably not so serious, since it is "just" about generating random group elements, and the difference is most likely caused by different-versioned-dependencies in Debian. However I would probably try to dig a bit deeper this weekend, just to reduce the failures by 1/3.)

[Jeroen Demeyer]

On 2017-01-19 12:25, Ximin Luo wrote:
> In other words, have you guys ever experienced a "important" (e.g. release-critical) issue that only caused 1 doctest failure? If not 1, what is the minimal number, in your experience?

Sometimes, there are important release-critical issues causing 0 doctest
failures...

So I guess that just 1 failure could certainly happen.

[Jeroen Demeyer]

On 2017-01-19 10:29, Ximin Luo wrote:
> Hi,
>
> Just wondering what is considered an "acceptable" number of test failures to release in distro-packaged Sage.
>
> François has told us before that Gento has about 30. At the moment on Debian amd64 we have about 40 for Sage 7.4, 13 of which are the following (edited for clarity):

I would really aim for 0 failures. It's the only reasonable thing that
you can test. You could artificially reduce the number of failures by
patching the doctest output if needed. That way, there is a patch
showing the doctest "failure" which is very explicit and can be reviewed
more easily than looking at build failures.

[Ximin Luo]

Jeroen Demeyer:

Right, that makes sense. And thanks for the confirmation that upstream do aim for 0 failures.

I'll see if we can push towards that, though it does become a bit of a cost-benefit game here. Also at the moment we have different numbers of test failures on amd64 vs i386, so we'd need to do some sort of conditional patching of the cases. It might be easier to just reduce the "80" threshold when we are able to.

[Jeroen Demeyer]

On 2017-01-19 13:30, Ximin Luo wrote:
> Also at the moment we have different numbers of test failures on amd64 vs i386, so we'd need to do some sort of conditional patching

Sage has many ways to deal with this, see
http://doc.sagemath.org/html/en/developer/coding_basics.html#special-markup-to-influence-tests

[Francois Bissey]

I am lucky enough that I can pin version of some software in Gentoo.
Here are all my doctests failures at the 7.5.rc3 level (I am somewhat
still on holidays for sage)
----------------------------------------------------------------------
sage -t --long /usr/lib64/python2.7/site-packages/sage/tests/cmdline.py # 1 doctest failed
sage -t --long /usr/lib64/python2.7/site-packages/sage/misc/cython.py # 3 doctests failed
sage -t --long /usr/lib64/python2.7/site-packages/sage/rings/integer.pyx # 1 doctest failed
sage -t --long /usr/lib64/python2.7/site-packages/sage/repl/interpreter.py # 2 doctests failed
sage -t --long /usr/lib64/python2.7/site-packages/sage/interfaces/tests.py # 1 doctest failed
sage -t --long /usr/lib64/python2.7/site-packages/sage/libs/libecm.pyx # 1 doctest failed
sage -t --long /usr/lib64/python2.7/site-packages/sage/misc/gperftools.py # 1 doctest failed
sage -t --long /usr/lib64/python2.7/site-packages/sage/interfaces/qepcad.py # 1 doctest failed
sage -t --long /usr/lib64/python2.7/site-packages/sage/libs/cypari2/pari_instance.pyx # 1 doctest failed
sage -t --long /usr/lib64/python2.7/site-packages/sagenb/notebook/misc.py # 2 doctests failed
sage -t --long /usr/lib64/python2.7/site-packages/sage/ext/memory.pyx # 1 doctest failed
———————————————————————————————————
Of these, only one is “numerical", libecm.pyx. The others are what I would
call "system reports”.

François

> --
> You received this message because you are subscribed to the Google Groups "sage-packaging" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-packagin...@googlegroups.com.
> To post to this group, send email to sage-pa...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-packaging/a98c56f4-8a56-039a-c326-278c404b89fa%40debian.org.
> For more options, visit https://groups.google.com/d/optout.

[Dima Pasechnik]

On Thursday, January 19, 2017 at 11:01:05 AM UTC, Ximin Luo wrote:
> Hi,
>
> Just wondering what is considered an "acceptable" number of test failures to release in distro-packaged Sage.
>
> François has told us before that Gento has about 30. At the moment on Debian amd64 we have about 40 for Sage 7.4, 13 of which are the following (edited for clarity):

one should abandon 7.4 and work with 7.5.
Working on packaging 7.4 is about recreating a large part of 7.5 from patches. :)

[Ximin Luo]

Dima Pasechnik:

> On Thursday, January 19, 2017 at 11:01:05 AM UTC, Ximin Luo wrote:
>> Hi,
>>
>> Just wondering what is considered an "acceptable" number of test failures to release in distro-packaged Sage.
>>
>> François has told us before that Gento has about 30. At the moment on Debian amd64 we have about 40 for Sage 7.4, 13 of which are the following (edited for clarity):
>
> one should abandon 7.4 and work with 7.5.
> Working on packaging 7.4 is about recreating a large part of 7.5 from patches. :)

Yes, I know. :) In fact our 7.4 already contains quite a lot of 7.5 patches. And Tobias has already started to package 7.5 for Debian, I am only polishing 7.4 today because the freeze date for "Debian stable" is very soon, I am almost finished..

I looked into this a bit more, I suspect it is to do with the fact we're using Gap 4.8.6 in Debian and Sage is still only on 4.8.3.

By following the source code, I can recover the GAP seeds:

sage: K.<x> = QQ[]
....: rgp = Gp()
....: def rtest_gap_seed(n):
....: set_random_seed(n)
....: current_randstate().set_seed_gp(rgp)
....: ZZ.random_element(1000)
....: RR.random_element()
....: K.random_element()
....: return ZZ.random_element(long(1)<<128)
....:
sage: rtest_gap_seed(0)
33158109024466993361178095302095368758
sage: rtest_gap_seed(1)
2738802622997200540675978798197515372
sage: rtest_gap_seed(2)
114300066641175859146812601978977101594

Then I can reproduce the output I get in the above sage tests, directly in GAP:

$ gap
┌───────┐ GAP 4.8.6, 12-Nov-2016, build of 2016-12-18 09:03:11 (UTC)
│ GAP │ http://www.gap-system.org
└───────┘ Architecture: x86_64-linux-gnu-gcc-default64
Libs used: gmp, readline
Loading the library and packages ...
Packages: AutPGrp 1.5, GAPDoc 1.5.1
Try '??help' for help. See also '?copyright', '?cite' and '?authors'
gap> g := Group((1,2), (1,2,3)(4,5));
Group([ (1,2), (1,2,3)(4,5) ])
gap> Reset (GlobalMersenneTwister, 33158109024466993361178095302095368758);;
gap> Random(g);
(1,3)
gap> Reset (GlobalMersenneTwister, 2738802622997200540675978798197515372);;
gap> Random(g);
(1,2)
gap> Reset (GlobalMersenneTwister, 114300066641175859146812601978977101594);;
gap> Random(g);
(1,2)(4,5)
gap>

Could someone run the above in Sage's GAP 4.8.3 please? The same below, for easy copy-paste:

g := Group((1,2), (1,2,3)(4,5));
Reset (GlobalMersenneTwister, 33158109024466993361178095302095368758);;
Random(g);
Reset (GlobalMersenneTwister, 2738802622997200540675978798197515372);;
Random(g);
Reset (GlobalMersenneTwister, 114300066641175859146812601978977101594);;
Random(g);

According to randstate.pyx you should get:

(1,3,2)
(1,3,2)
(1,3,2)

If this is the case, then I will assume my diagnosis is correct and simply patch these doctests in Debian.

[Dima Pasechnik]

yes, this is what I get on (Sage's or not) GAP 4.8.3.

>
> If this is the case, then I will assume my diagnosis is correct and simply patch these doctests in Debian.

I don't understand - you cannot simply drop in GAP 4.8.6 tarball onto latest Sage and hope it will work. It will not, for the obstacle is libGAP Sage's package, which is basically a fork of GAP (forced upon Sage by GAP's devs, in fact), and it's not too easy to update.

Although there is an open pull request for update of libGAP to 4.8.6, see
https://bitbucket.org/vbraun/libgap/pull-requests/12/update-to-gap-486/diff
and so hopefully it will be fixed real soon.

[Tobias Hansen]

We already updated libgap to 4.8.6 for Debian in November, following the
instructions in libgaps README.

Best,
Tobias

[Dima Pasechnik]

Great, but why have you decided not to tell the upstream?
It is on my overful todo list for many months.
So now arch people did the same, see that pull request...

[Tobias Hansen]

I did this only once for gap 4.8.4. In that case it was simply a matter
of following the steps in the readme. The generated patch had no
conflicts. So I assumed if upstream wanted to update they could easily
do it. All other work on libgap for Debian was done by Jerome.

[Ximin Luo]

Dima Pasechnik:

> Great, but why have you decided not to tell the upstream?
> It is on my overful todo list for many months.
> So now arch people did the same, see that pull request...
>

We had a ton of work to do in December and earlier this month, to make some Debian release deadlines. Sorry for this delay, but we will start submitting our patches upstream to you soon, over the next few months.

[Dima Pasechnik]

Well, my experience with updating to 4.8.3 was quite unpleasant. Indeed there is no telling how hard a particular update is.

Is it a pity that Debian is not notifying upstream(s) of such improvements straight away.

Just telling us that an update was done and the patches can be extacted from such and such place would already be great.

[Ximin Luo]

Dima Pasechnik:

> Well, my experience with updating to 4.8.3 was quite unpleasant. Indeed there is no telling how hard a particular update is.
>
> Is it a pity that Debian is not notifying upstream(s) of such improvements straight away.
>
> Just telling us that an update was done and the patches can be extacted from such and such place would already be great.
>

Yes, yes, I am aware of this especially after I had a big go at some others here a few months ago for not upstreaming Sage's patches. :)

Our patches are here:

https://anonscm.debian.org/cgit/debian-science/packages/sagemath.git/tree/debian/patches/

If you forget this link, I also added it to the sage wiki, see "patches" here:

https://wiki.sagemath.org/Distribution#Debian

I have tried to categorise our patches nicely, you will find an explanation of the u0, d1 etc prefixes here:

https://anonscm.debian.org/cgit/debian-science/packages/sagemath.git/tree/debian/patches/series

Basically, u0 you have already applied, u1 and u2 we will begin upstreaming "soon" but of course feel free to jump in ahead! The df/dt stuff needs a closer look (i.e. by us), most of it is probably not suitable to merge but some of them might help to make Sage build more reliably across more situations. Apologies in advance for any sarcastic comments in the patches, it helps me blow off some steam when debugging.

As far as I remember, pretty much all the patches are necessary to make things work on Debian; we did not change any "behavioural" aspects of Sage in the sense of sagelib's API contracts etc.

[Francois Bissey]

I am starting to look at your patches….
You are standardising to gslcblas for cblas? Why not another
cblas implementation? Or is it because you are hard linking to gslcblas
in gsl?
I see you have done the leg work for the gmp-ecm upgrade that I completely
stalled some months ago. I could steal it for ticket #20385 but I would rather
the credit go to someone in debian, at least for the sage part (I can do
the packaging part).

We should definitely have some proper discussion and coordination about some of that
stuff. Not just on mailing list. I guess the sage days in March will be an occasion
even I have to do it remotely like last year.
Looking at some patches I feel like I may pull some rugs from under you by
sending stuff to various upstream.

François

> --
> You received this message because you are subscribed to the Google Groups "sage-packaging" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-packagin...@googlegroups.com.
> To post to this group, send email to sage-pa...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-packaging/82a4c56a-691f-a1c4-a752-a56bc79cbfe1%40debian.org.

[Dima Pasechnik]

On Sunday, January 22, 2017 at 12:43:51 AM UTC, Ximin Luo wrote:
> Dima Pasechnik:
> > Well, my experience with updating to 4.8.3 was quite unpleasant. Indeed there is no telling how hard a particular update is.
> >
> > Is it a pity that Debian is not notifying upstream(s) of such improvements straight away.
> >
> > Just telling us that an update was done and the patches can be extacted from such and such place would already be great.
> >
>
> Yes, yes, I am aware of this especially after I had a big go at some others here a few months ago for not upstreaming Sage's patches. :)

mind you, the relevant Sage ticket is more that 6 months old:
https://trac.sagemath.org/ticket/20914
I posted the relevant comment there.

[Jeroen Demeyer]

On 2017-01-22 01:43, Ximin Luo wrote:
> Yes, yes, I am aware of this especially after I had a big go at some others here a few months ago for not upstreaming Sage's patches. :)

Even worse, people from Debian were complaining that Sage had patches in
the first place (even if they were upstreamed).

[Antonio Rojas]

El domingo, 22 de enero de 2017 1:42:20 (CET) Francois Bissey escribió:
>
> We should definitely have some proper discussion and coordination about some of that
> stuff. Not just on mailing list. I guess the sage days in March will be an occasion
> even I have to do it remotely like last year.
> Looking at some patches I feel like I may pull some rugs from under you by
> sending stuff to various upstream.
>
> François

It would be great if we could have a common place to post all our patches (as a previous step to upstreaming them), so we don't have to end up duplicating work or go rummaging each other's SCM

Perhaps a github clone of sagemath/sage where we'd have a branch for each repo with our patches

[Jeroen Demeyer]

On 2017-01-22 10:41, Antonio Rojas wrote:
> It would be great if we could have a common place to post all our patches (as a previous step to upstreaming them), so we don't have to end up duplicating work or go rummaging each other's SCM

Ideally, things would be fixed in upstream Sage. In that case the Sage
Trac would be the proper place.

Although certainly not all patches would make sense to become
upstreamed, especially those patches which are workarounds for package
bugs where the distro doesn't want to patch the package.

[Tobias Hansen]

So far I was fine with rummaging your SCM. There are links to the
locations of distro patches for Arch, Fedora, Gentoo and Debian on our
Wiki page:
https://wiki.debian.org/DebianScience/Sage#Links

[Ximin Luo]

Dima Pasechnik:

> On Sunday, January 22, 2017 at 12:43:51 AM UTC, Ximin Luo wrote:
>> Dima Pasechnik:
>>> Well, my experience with updating to 4.8.3 was quite unpleasant. Indeed there is no telling how hard a particular update is.
>>>
>>> Is it a pity that Debian is not notifying upstream(s) of such improvements straight away.
>>>
>>> Just telling us that an update was done and the patches can be extacted from such and such place would already be great.
>>>
>>
>> Yes, yes, I am aware of this especially after I had a big go at some others here a few months ago for not upstreaming Sage's patches. :)
>
> mind you, the relevant Sage ticket is more that 6 months old:
> https://trac.sagemath.org/ticket/20914
> I posted the relevant comment there.
>

And our libGAP source is here:

https://anonscm.debian.org/cgit/debian-science/packages/libgap-sage.git/tree/

I'm less familiar with this but it looks like Jerome Benoit the maintainer does not use libGAP upstream releases directly, but instead builds them by combining GAP and libGAP git via a script:

https://anonscm.debian.org/cgit/debian-science/packages/libgap-sage.git/tree/debian/get-orig-source.sh

Possibly this makes things easier to upgrade, at least evidently he himself prefers this approach.

>>
>> Our patches are here:
>>
>> https://anonscm.debian.org/cgit/debian-science/packages/sagemath.git/tree/debian/patches/
>>
>> If you forget this link, I also added it to the sage wiki, see "patches" here:
>>
>> https://wiki.sagemath.org/Distribution#Debian
>>
>> I have tried to categorise our patches nicely, you will find an explanation of the u0, d1 etc prefixes here:
>>
>> https://anonscm.debian.org/cgit/debian-science/packages/sagemath.git/tree/debian/patches/series
>>
>> Basically, u0 you have already applied, u1 and u2 we will begin upstreaming "soon" but of course feel free to jump in ahead! The df/dt stuff needs a closer look (i.e. by us), most of it is probably not suitable to merge but some of them might help to make Sage build more reliably across more situations. Apologies in advance for any sarcastic comments in the patches, it helps me blow off some steam when debugging.
>>
>> As far as I remember, pretty much all the patches are necessary to make things work on Debian; we did not change any "behavioural" aspects of Sage in the sense of sagelib's API contracts etc.
>>

[Tobias Hansen]

On 01/22/2017 10:21 AM, Ximin Luo wrote:
> Dima Pasechnik:
>> On Sunday, January 22, 2017 at 12:43:51 AM UTC, Ximin Luo wrote:
>>> Dima Pasechnik:
>>>> Well, my experience with updating to 4.8.3 was quite unpleasant. Indeed there is no telling how hard a particular update is.
>>>>
>>>> Is it a pity that Debian is not notifying upstream(s) of such improvements straight away.
>>>>
>>>> Just telling us that an update was done and the patches can be extacted from such and such place would already be great.
>>>>
>>>
>>> Yes, yes, I am aware of this especially after I had a big go at some others here a few months ago for not upstreaming Sage's patches. :)
>>
>> mind you, the relevant Sage ticket is more that 6 months old:
>> https://trac.sagemath.org/ticket/20914
>> I posted the relevant comment there.
>>
>
> And our libGAP source is here:
>
> https://anonscm.debian.org/cgit/debian-science/packages/libgap-sage.git/tree/
>
> I'm less familiar with this but it looks like Jerome Benoit the maintainer does not use libGAP upstream releases directly, but instead builds them by combining GAP and libGAP git via a script:
>
> https://anonscm.debian.org/cgit/debian-science/packages/libgap-sage.git/tree/debian/get-orig-source.sh
>
> Possibly this makes things easier to upgrade, at least evidently he himself prefers this approach.

If I understand Jerome correctly he first tried to use the normal libgap
tarball, but that was rejected from the Debian NEW queue because libgap
modifies gap files but not their copyright headers. That's why he does
it in this way now. I don't know the details of this rejection though.
That's what he said:

Jerome Benoit:
Actually I did prefer the original tarball: the package was reject for
copyright reasons: the issue is that the source files are modified but
not their copyright header. It is why I play with the git material
which contains only non-modified material.

[Dima Pasechnik]

well, on top of this we have our own little war with GAP people who force us to maintain a fork :-)

[Dima Pasechnik]

I think it's just plain wrong attitude on Debian side here.

Debian should have tried offering their patches (or help) to upstream first, and only then resorted to do patching libGAP on their own, and not the other way around.

I understand that in situations where upstream is basically dead this mihgt have been OK, but this is emphatically not the case here.

On Sunday, January 22, 2017 at 10:31:45 AM UTC, Tobias Hansen wrote:

[Ximin Luo]

Dima Pasechnik:

> I think it's just plain wrong attitude on Debian side here.
>
> Debian should have tried offering their patches (or help) to upstream first, and only then resorted to do patching libGAP on their own, and not the other way around.
>
> I understand that in situations where upstream is basically dead this mihgt have been OK, but this is emphatically not the case here.
>

I'm not sure on the exact details on the situation with libGAP, but it is different from most other software projects because it contains a good portion of GAP.

I (and Debian mostly) take the view that a software package is a "unit of maintenance". The libGAP authors do not maintain the GAP source code so I don't consider GAP as part of its source code. In other words, I consider it to be just the patches.

I would *guess* that Jerome has not made any significant changes to the patches, since if he had, he should have submitted them upstream. Or, perhaps the situation with GAP makes it a bit awkward to do this.

However if by "[our] libGAP patches" you mean the simple act of importing the latest GAP version into libGAP then I would *guess* that this was trivial to do (as Tobias also guessed), because otherwise Jerome would have already submitted them upstream. Really, he is very good with these sorts of things, he has been submitting patches to cysignals, as Jeroen knows, and various other projects.

If he did not do this, I would *guess* it is because Sage is still using 4.8.3 and it is unlikely libGAP would have merged his conflict resolutions until a much later date - and we had some Debian deadlines to meet in December.

FWIW, I'm also pretty sure we do not have any major patches for GAP 4.8.6 in Sage itself in Debian - at least going through the patches I linked before. There is only the ~15 test failures I mentioned already - 13 from randstate.pyx and a couple more in other files.

Just to make clear, we're (of course) not against Sage patches, it is when they change the external behaviour of 3rd-party software (which we in Debian have to ensure still work with other non-Sage software), that we get annoyed. I'm not sure who it was that Jeroen mentioned "complaining that Sage had patches in the first place", it probably was not me and I am guessing there was some miswording and they meant something slightly different.

X

[Ximin Luo]

Francois Bissey:

> I am starting to look at your patches….
> You are standardising to gslcblas for cblas? Why not another
> cblas implementation? Or is it because you are hard linking to gslcblas
> in gsl?

https://anonscm.debian.org/cgit/debian-science/packages/sagemath.git/tree/debian/patches/d0-gsl-cblas.patch

I did this back when I was still a Sage noob (last July) and I can't remember the exact reason now. It's possible that I simply assumed we had to use gslcblas because we were already using gsl. Or that it was the easiest to figure out the correct compiler flags for.

Now I see that actually the other implementations do provide a .pc file so perhaps we can just revert that patch.

$ ls -la /usr/lib/pkgconfig/blas* /etc/alternatives/blas.pc
lrwxrwxrwx 1 root root 35 Jan 21 16:09 /etc/alternatives/blas.pc -> /usr/lib/pkgconfig/blas-openblas.pc
-rw-r--r-- 1 root root 302 Aug 6 20:27 /usr/lib/pkgconfig/blas-atlas.pc
-rw-r--r-- 1 root root 218 Dec 29 17:26 /usr/lib/pkgconfig/blas-netlib.pc
-rw-r--r-- 1 root root 340 Sep 6 20:24 /usr/lib/pkgconfig/blas-openblas.pc
lrwxrwxrwx 1 root root 25 Jan 21 16:09 /usr/lib/pkgconfig/blas.pc -> /etc/alternatives/blas.pc

The actual object files are stored in different subdirs in /usr/lib at runtime so we'd probably also have to add -Wl,-rpath options when compiling. Perhaps that's why I chose gslcblas instead. AFAICT Debian today does not allow the user to select a default libblas.so.X at runtime.

Do you have a preference?

> I see you have done the leg work for the gmp-ecm upgrade that I completely
> stalled some months ago. I could steal it for ticket #20385 but I would rather
> the credit go to someone in debian, at least for the sage part (I can do
> the packaging part).
>

https://anonscm.debian.org/cgit/debian-science/packages/sagemath.git/tree/debian/patches/u1-version-ecm-7.patch

This was actually pulled from Arch so credit goes to them :) Not sure if Jerome also did extra work on the actual gmp-ecm package though.

> We should definitely have some proper discussion and coordination about some of that
> stuff. Not just on mailing list. I guess the sage days in March will be an occasion
> even I have to do it remotely like last year.
> Looking at some patches I feel like I may pull some rugs from under you by
> sending stuff to various upstream.
>

OK, would be good if you could notify us when do you that!

I'd be up for meeting up too, but my own March is very full already. I'll keep an eye out for other events though.

[Dima Pasechnik]

On Sunday, January 22, 2017 at 12:17:36 PM UTC, Ximin Luo wrote:
> Dima Pasechnik:
> > I think it's just plain wrong attitude on Debian side here.
> >
> > Debian should have tried offering their patches (or help) to upstream first, and only then resorted to do patching libGAP on their own, and not the other way around.
> >
> > I understand that in situations where upstream is basically dead this mihgt have been OK, but this is emphatically not the case here.
> >
>
> I'm not sure on the exact details on the situation with libGAP, but it is different from most other software projects because it contains a good portion of GAP.

Good or bad, GAP people maintain that libGAP is a fork of GAP.
And the upstream of libGAP is here:
https://bitbucket.org/vbraun/libgap

>
> I (and Debian mostly) take the view that a software package is a "unit of maintenance". The libGAP authors do not maintain the GAP source code so I don't consider GAP as part of its source code. In other words, I consider it to be just the patches.

We do maintain the libGAP, whether it is a set of patches or not is irrelevant.

And for you guys libgap *is* a package:
https://anonscm.debian.org/cgit/debian-science/packages/libgap-sage.git/

>
> I would *guess* that Jerome has not made any significant changes to the patches, since if he had, he should have submitted them upstream. Or, perhaps the situation with GAP makes it a bit awkward to do this.
>
> However if by "[our] libGAP patches" you mean the simple act of importing the latest GAP version into libGAP then I would *guess* that this was trivial to do (as Tobias also guessed), because otherwise Jerome would have already submitted them upstream. Really, he is very good with these sorts of things, he has been submitting patches to cysignals, as Jeroen knows, and various other projects.

>
> If he did not do this, I would *guess* it is because Sage is still using 4.8.3 and it is unlikely libGAP would have merged his conflict resolutions until a much later date - and we had some Debian deadlines to meet in December.

I am sure that if in November 2016 work has been done to update libGAP in Sage, for sure 7.5 would have contained these updates.

[Dima Pasechnik]

On Sunday, January 22, 2017 at 1:42:35 PM UTC, Dima Pasechnik wrote:
> On Sunday, January 22, 2017 at 12:17:36 PM UTC, Ximin Luo wrote:
> > Dima Pasechnik:
> > > I think it's just plain wrong attitude on Debian side here.
> > >
> > > Debian should have tried offering their patches (or help) to upstream first, and only then resorted to do patching libGAP on their own, and not the other way around.
> > >
> > > I understand that in situations where upstream is basically dead this mihgt have been OK, but this is emphatically not the case here.
> > >
> >
> > I'm not sure on the exact details on the situation with libGAP, but it is different from most other software projects because it contains a good portion of GAP.
>
> Good or bad, GAP people maintain that libGAP is a fork of GAP.
> And the upstream of libGAP is here:
> https://bitbucket.org/vbraun/libgap
>
>
> >
> > I (and Debian mostly) take the view that a software package is a "unit of maintenance". The libGAP authors do not maintain the GAP source code so I don't consider GAP as part of its source code. In other words, I consider it to be just the patches.

Mind you, it is 15K of very tricky C code, its own (nontrivial too - you need to build a dynamical library) build system, tests, and machinery to patch the pristine GAP source.
AFAICS, Debian does quite a bit of changes to the build system and tests at least.