Upgrade PARI to git master

[Jeroen Demeyer]

Hello sage-devel and sage-packaging,

I propose to upgrade the PARI package to the git master version instead
of the current released version.

The main motivation for me is to use the new plotting engine for the
PARI Jupyter kernel. The plotting engine has changed a lot since the
stable release. This is split into many commits and it affects even the
build system, so I think it would be too much work to do this with a
patch. Currently, I do have plotting support in my PARI Jupyter kernel,
but it requires patches which were written by me but substantially
changed by upstream PARI. So it doesn't work with any upstream version
of PARI and I want to fix this. In the current stable release, plotting
is implemented only in the GP command line program, not in the PARI
library. So a stable release cannot work for me.

A second motivation is that stable releases of PARI are very slow. The
most recent stable PARI releases (excluding bugfix releases) were in
november 2016, march 2014 and june 2011. Upgrading now to git master
gives the advantage that we can profit from improvements in PARI. It
will also make the transition to the next stable release easier: the
large time between stable releases implies many changes. Since many
components of Sage use PARI, there are a substantial number of changes
needed in Sage with every PARI upgrade. By doing this upgrade in steps,
this becomes more manageable.

Many versions of Sage have used git versions of PARI, for the second
reason I mentioned above. So there is certainly a precedent for doing this.

I do know that this request is controversial because distributions are
not likely to accept a "git master" version. However, I feel that I
cannot make progress with the PARI Jupyter kernel unless this is
resolved. So I would like distributions to accept that Sage is moving
again to a git master version of PARI. If it worked in the past with
older Sage and PARI versions, it should also work in the future.

I didn't make a Trac ticket for this yet precisely because I know it is
controversial.


Comments? Suggestions?
Jeroen.

[William Stein]

+1 -- there were a lot of people at Sage Days last week asking about feasibility of making a Pari-master package,

since pari stable is so old, and they really needed things in the newers version of Pari.

William

--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To post to this group, send email to sage-...@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

--

-- William Stein

[Ximin Luo]

Jeroen Demeyer:

Hey, thanks for notifying. I can understand your frustration that PARI only releases every few years and the differences are huge in between.

In Debian we don't have the PARI jupyter kernel, since it's an "optional" package and nobody has had the time to package those yet. Do you think it would be feasible for Sage "standard" to maintain compatibility with both PARI stable and the development version? (Except for the PARI jupyter kernel, which would require a development version of PARI.)

X

--
GPG: ed25519/56034877E1F87C35
GPG: rsa4096/1318EFAC5FBBDBCE
https://github.com/infinity0/pubkeys.git

[Jeroen Demeyer]

On 2017-07-25 22:34, Ximin Luo wrote:
> Do you think it would be feasible for Sage "standard" to maintain compatibility with both PARI stable and the development version?

I have not actually tried, so it is hard to tell right now.

Given the past history of the PARI package inside Sage, I expect that we
will regularly upgrade PARI in Sage to future git master versions. Every
time, the difference with the stable version increases and the chance of
compatibility decreases. So even if we are compatible now, there is no
guarantee that this will remain the case in the future. Moreover, we do
not have machinery in place to check this compatibility: I can do it for
this upgrade, but it might easily be forgotten the next time.

Everything also depends on how you define "compatibility". There might
be a large difference between "Sage builds and works" and "all doctests
pass". The first one is much more likely than the second one.


Jeroen.

[François Bissey]

I know you already signalled in the post but my only answer is

AGAIN!!??

Nevertheless I don’t see that as a biggy anymore because most of the
dependencies of pari (in Gentoo or sage-on-gentoo) happen to be sage
dependencies as well - apart from 2 that are desperately stuck at pari 2.3,
there is no helping those.
Since all the important pari dependencies will be moving along with it,
I have no issues with moving to git master again.
I also have much better tools to deal with this than when were at pari-2.4.

François

> --
> You received this message because you are subscribed to the Google Groups "sage-packaging" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-packagin...@googlegroups.com.
> To post to this group, send email to sage-pa...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-packaging/5977A70E.6010905%40cage.ugent.be.

[Dr. David Kirkby (Kirkby Microwave Ltd)]

On 25 Jul 2017 21:16, "Jeroen Demeyer" <jdem...@cage.ugent.be> wrote:
>
> Hello sage-devel and sage-packaging,
>
> I propose to upgrade the PARI package to the git master version instead of the current released version.
>
>

> A second motivation is that stable releases of PARI are very slow. The most recent stable PARI releases (excluding bugfix releases) were in november 2016, march 2014 and june 2011. Upgrading now to git master gives the advantage that we can profit from improvements in PARI. It will also make the transition to the next stable release easier: the large time between stable releases implies many changes. Since many components of Sage use PARI, there are a substantial number of changes needed in Sage with every PARI upgrade. By doing this upgrade in steps, this becomes more manageable.
>

> I do know that this request is controversial because distributions are not likely to accept a "git master" version. However, I feel that I cannot make progress with the PARI Jupyter kernel unless this is resolved. So I would like distributions to accept that Sage is moving again to a git master version of PARI.

> I didn't make a Trac ticket for this yet precisely because I know it is controversial.
>
> Comments? Suggestions?
> Jeroen.

Would it be worth creating a fork of PARI, where the fork just puts out a stable release when needed? Call is Sage-PARI or something,  then you would avoid the problems of a distribution not accepting a git-master copy.

Dave

[Ximin Luo]

Dr. David Kirkby (Kirkby Microwave Ltd):
> [..]

>
> Would it be worth creating a fork of PARI, where the fork just puts out a
> stable release when needed? Call is Sage-PARI or something, then you would
> avoid the problems of a distribution not accepting a git-master copy.
>

The ideal solution would be to ask upstream to do an "officially-endorsed" snapshot version, which perhaps we should do at some point if it's the case that "there were a lot of people [..] asking about [it]".

Sage doing a fork would also be OK, the main thing for distros is to have it be co-installable with the pari stable version (i.e. using different paths and so on) . I think, if I remember right, this was partly why PARI upstream did not do that yet, and also a barrier to why we didn't create a "pari-sage" package in Debian yet (besides the fact we don't really want to). I'd be interested in how Gentoo solves this, could you go into some detail François?

(At the moment the only Sage-specific fork we have in Debian is maxima-sage, and that is because the maintainer has been a bit stubborn, IMO, in refusing to use ECL with maxima - he is part of the GCL team, maintains it in Debian, and argues that a ECL version would be "hard-to-maintain duplication" or something along those lines.)

[François Bissey]

> On 26/07/2017, at 23:44, Ximin Luo <infi...@debian.org> wrote:
>
> Sage doing a fork would also be OK, the main thing for distros is to have it be co-installable with the pari stable version (i.e. using different paths and so on) . I think, if I remember right, this was partly why PARI upstream did not do that yet, and also a barrier to why we didn't create a "pari-sage" package in Debian yet (besides the fact we don't really want to). I'd be interested in how Gentoo solves this, could you go into some detail François?
>

I stopped worrying too much about it when sage was using pari-2.8. There are
a couple of packages in gentoo that still have a dependency to pari 2.3 -
a perl interface package and clisp. Since the current version of pari is 2.9.x
we really need to consider a special pari-2.3 for those.

When sage was using pari-2.4 I made 2.4 installable alongside 2.3 but
everything was suffixed with 2.4. I had to patch all of sage’s dependencies,
and sage itself, accordingly. Lots of work. As it happens all the consumers
of pari apart from the two aforementioned packages are dependencies of sage
and sage itself (if you have other packages consuming pari in debian, I’d like
to know about them). So I took the view that it is OK to just use “unstable”
pari. It is stable enough to be used by sage after all. Now pari could make things
easier on their side to use/consume snapshots but they obviously don’t have too.

> (At the moment the only Sage-specific fork we have in Debian is maxima-sage, and that is because the maintainer has been a bit stubborn, IMO, in refusing to use ECL with maxima - he is part of the GCL team, maintains it in Debian, and argues that a ECL version would be "hard-to-maintain duplication" or something along those lines.)

I call rubbish on that one. Sure there is a bit more work in the set up.
But maxima’s build system allow you to build and install maxima for multiple
lisp implementations in one go. And there is a good system to select the
desired lisp, with a default picking order in the absence of option to select
a lisp. Which is why my sage’s pexpect interface is patched to explicitly
call `maxima -l ecl`.

François

[Tobias Hansen]

On 07/26/2017 01:09 PM, François Bissey wrote:
>> On 26/07/2017, at 23:44, Ximin Luo <infi...@debian.org> wrote:
>>
>> Sage doing a fork would also be OK, the main thing for distros is to have it be co-installable with the pari stable version (i.e. using different paths and so on) . I think, if I remember right, this was partly why PARI upstream did not do that yet, and also a barrier to why we didn't create a "pari-sage" package in Debian yet (besides the fact we don't really want to). I'd be interested in how Gentoo solves this, could you go into some detail François?
>>
> I stopped worrying too much about it when sage was using pari-2.8. There are
> a couple of packages in gentoo that still have a dependency to pari 2.3 -
> a perl interface package and clisp. Since the current version of pari is 2.9.x
> we really need to consider a special pari-2.3 for those.
>
> When sage was using pari-2.4 I made 2.4 installable alongside 2.3 but
> everything was suffixed with 2.4. I had to patch all of sage’s dependencies,
> and sage itself, accordingly. Lots of work. As it happens all the consumers
> of pari apart from the two aforementioned packages are dependencies of sage
> and sage itself (if you have other packages consuming pari in debian, I’d like
> to know about them). So I took the view that it is OK to just use “unstable”
> pari. It is stable enough to be used by sage after all. Now pari could make things
> easier on their side to use/consume snapshots but they obviously don’t have too.
>

Having only unstable pari in Debian is out of the question, because the
maintainer of the pari package also has a say in that. It would be
really nice if coinstallation of the two versions could be made possible
upstream.

Best,
Tobias

[Ximin Luo]

Tobias Hansen:

Right, upstream PARI (Bill Allobmert) also maintains it in Debian and I don't think he would allow this.

But even if this wasn't the case I'd hesitate, because I can believe that at least *some users* would want stable versions rather than git snapshots, even if no other packages depend specifically on the stable version.

$ echo $(aptitude search -F '%p' --disable-columns '~Dlibpari-gmp-tls5 ~rnative')
eclib-tools lcalc libec3 libgiac0 liblfunction0 libpari-dev libpari-gmp-tls5-dbgsym python-cypari2 python-cysignals-pari sagemath xcas

So in fact everything that depends on PARI is indeed already a dependency of sagemath, but I'd still hesitate to try to replace pari-stable with pari-unstable in Debian.

Now I've realised a further problem still exists, even if co-installation was possible: in Debian we'd presumably still have most of the above packages link against pari-stable, but if sage itself links against pari-unstable that would bring in both versions into the same process memory space with conflicting symbols. We had this issue with ntl a while back but it was easy to solve just by recompiling the older packages against the newer ntl. But this wouldn't be possible here. Maybe it would be fixable by setting LD_LIBRARY_PATH in (our version of) sage-env though... that assumes the other packages work with both pari-stable and pari-unstable, which I'd hope remains the case.

[Tobias Hansen]

I think the only clean solution would be to use pari-unstable for all
these packages. That would make the whole thing even more controversial.
And would require a transition for every pari update. Do the pari
unstable snapshots have proper sonames?

[Ximin Luo]

Jeroen Demeyer:

> [..]

What's the overall context to this? PARI-jupyter is only an "optional" package in Sage at the moment. Can this work not be delayed until the next PARI stable version? (Or a short while before it, like how we were testing 7.4 in Debian with Sage's pari git snapshots for several months, then they finally released it as stable.)

If someone is sponsoring you to work on PARI-jupyter, perhaps they are friendly enough with upstream to convince them to shorten their release cycles a bit?

[Jeroen Demeyer]

On 2017-07-26 00:46, Dr. David Kirkby (Kirkby Microwave Ltd) wrote:
> Would it be worth creating a fork of PARI

What *exactly* do you mean with that?

I feel like "fork" is just a word and you can already consider the
PARI-in-Sage to be a fork of PARI.

> you would avoid the problems of a distribution not accepting a git-master copy.

I'm not convinced that distributions would accept a fork either (again,
this may depend on the precise meaning of "fork"). For example, distros
generally use GMP instead of the fork MPIR that Sage uses.

[François Bissey]

> On 27/07/2017, at 21:01, Jeroen Demeyer <jdem...@cage.ugent.be> wrote:
>
>
>> you would avoid the problems of a distribution not accepting a git-master copy.
>
> I'm not convinced that distributions would accept a fork either (again, this may depend on the precise meaning of "fork"). For example, distros generally use GMP instead of the fork MPIR that Sage uses.

<rant>

MPIR is victim of some of its own compatibility layer.
1) GMP is used by the gcc toolchain, you have to convince the toolchain people
to switch. They are a very conservative bunch.
2) MPIR enables a fake libgmp{,xx} library. Which means virtually no one is
interested in linking to libmpir instead of libgmp. If packages, including
sage had an option to link against libmpir instead of libgmp, toolchain
wouldn’t be an obstacle as you wouldn’t have to install MPIR’s libgmp.

The only packages that looks for libmpir (on top of my head) are flint and
arb. The fact that mpir and flint and arb share developers is likely to have
something to do with it.

Because of the number of packages with interdependencies that link to libgmp
we cannot just switch a few as they are ready. That way leads to insanity.

</rant>
This is a rant but I guess it can provide ammunitions against forking.

François

[Jeroen Demeyer]

On 2017-07-26 14:57, Ximin Luo wrote:
> Can this work not be delayed until the next PARI stable version?

Well, I currently have a version of pari_jupyter where plotting works
only with patches which are only in Sage. This is bad. I want to make it
work with PARI master, which would be an improvement (PARI stable is out
of the question unfortunately, since it requires changes to the plotting
engine which are not in the stable release).

Besides this, there might be other reasons to use a master version of
PARI in Sage, see the reply of William Stein for example.

> If someone is sponsoring you to work on PARI-jupyter, perhaps they are friendly enough with upstream to convince them to shorten their release cycles a bit?

It is actually better than that. I am being payed by the *same* project
(OpenDreamKit) that also (partially) pays the PARI developers. Still, I
don't think this will help.

[François Bissey]

The requirement of having access to the stable pari at least is the biggest
argument for being able to co-install.

Some details that are useful for the work:
soname:
stable version of pari have soname of the form
libpari-{$threadmodel}-{$gmp}.so. $N
with N=5 for pari-2.9, 4, for 2.7 etc.
Unless they have changed things this time unstable pari always
have N=0.
I would much prefer N+1 even if that means the next stable is N+2.

The executable gp is actually a link to gp-$version where version
is the full version 2.9.2, 2.9.3, at least for stable version.

So both gp and libpari could be made alternatives in the debian
sense - relatively easily.

What is harder is co-installation of the header and documentation.
Furthermore gphelp is not a link to a full versioned executable
and the location of the documentation is hardcoded in the executable
is my memory serves me right.

By the way a few sage dependencies (sympow for example) and sage
itself rely directly on the gp executable. sage itself relies on
gphelp to access pari’s documentation. So some care may be required
to make sure the right gp/gphelp is used in those programs.

If the way you use gp is stable you may be able to get away with
only installing the stable one. The consequence could be broken
tests in sage’s pexpect interface if some numerical results change.

François

> --
> You received this message because you are subscribed to the Google Groups "sage-packaging" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-packagin...@googlegroups.com.
> To post to this group, send email to sage-pa...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-packaging/34b1aee2-04fe-e62c-5105-8fad2ed42624%40debian.org.

[Dr. David Kirkby (Kirkby Microwave Ltd)]

On 27 July 2017 at 10:01, Jeroen Demeyer <jdem...@cage.ugent.be> wrote:

On 2017-07-26 00:46, Dr. David Kirkby (Kirkby Microwave Ltd) wrote:

Would it be worth creating a fork of PARI

What *exactly* do you mean with that?

I feel like "fork" is just a word and you can already consider the PARI-in-Sage to be a fork of PARI.

Just periodically release a latest github version of Pair under another name, with a "stable" version number - having test it well of course.

 

you would avoid the problems of a distribution not accepting a git-master copy.

I'm not convinced that distributions would accept a fork either (again, this may depend on the precise meaning of "fork"). For example, distros generally use GMP instead of the fork MPIR that Sage uses.

In that case, perhaps my "solution" would not work, although perhaps distributions would consider that development of PARI is ongoing, but new releases are 2 years apart. That is pretty unusual.

An ideal solution would be to try to encourage the PARI developers to release a new version more regularly, but you can't force anyone to do that.

Dave

[Jeroen Demeyer]

On 2017-07-25 22:44, Jeroen Demeyer wrote:
> Everything also depends on how you define "compatibility". There might
> be a large difference between "Sage builds and works" and "all doctests
> pass". The first one is much more likely than the second one.

I think the above is an important point that I didn't receive comments
on. What do the packagers expect?

I'm rebooting this discussion because we have fixed cypari2 (the package
providing the PARI <-> Python interface which used to be a part of Sage)
to support multiple PARI versions. So at least having Sage build and
work with PARI master seems realistic.

[François Bissey]

This is indeed a potential issue. How does cypari2 support multiple pari?
sage will certainly build. Building the documentation or doctesting may
fail if some features are not found in the installed cypari2.

I imagine binary distro could tie cypari2 to a snapshot version of the pari
library, while keeping the normal pari stable.

I note there are 12 .pyx in sage needing libpari. They may also need to be
linked to the snapshot, I don’t know for sure.

The gp binary can probably be the stable one as it is only needed for the pexpect
interface. You may not need to test new pari feature in that interface. Of course
some results could change…

François

[Jeroen Demeyer]

On 2017-09-06 12:39, François Bissey wrote:
> This is indeed a potential issue.

What is a potential issue?

> How does cypari2 support multiple pari?

Most of the PARI <-> Python interface consists of auto-generated code.
For example, for every GP function such as nfinit(), a Python method
nfinit() is generated in the appropriate class and a C declaration is
added to an auto-generated .pxd file.

> sage will certainly build. Building the documentation or doctesting may
> fail if some features are not found in the installed cypari2.

I don't expect documentation problems (except for producing plots, the
docbuilder imports but doesn't really execute any code. And currently,
the plots do not require state-of-the-art PARI features). Doctests will
certainly fail: PARI has a lot of functions where the answer is not
unique. Within one version, the answer the deterministic but it
typically changes with every PARI version.

> I imagine binary distro could tie cypari2 to a snapshot version of the pari
> library, while keeping the normal pari stable.

I don't quite get what you mean with this... For most distros (Gentoo is
the exception here), the version of a package is fixed. You cannot have
multiple versions of PARI on the same system. In any case, within Sage,
all components must use the same PARI library.

> I note there are 12 .pyx in sage needing libpari.
> They may also need to be
> linked to the snapshot, I don’t know for sure.

Most likely, it doesn't matter. But again, it is important that those
are linked to the *same* version of PARI as cypari2.

> The gp binary can probably be the stable one as it is only needed for the pexpect
> interface.

Most likely yes. Anyway, the gp pexpect interface is not used a lot in Sage.

[Jeroen Demeyer]

On 2017-07-25 22:16, Jeroen Demeyer wrote:
> Hello sage-devel and sage-packaging,
>
> I propose to upgrade the PARI package to the git master version instead
> of the current released version.

Now that the cypari2 upgrade has been merged, I am working on this.
Thanks to the new cypari2, *building* Sage with PARI master works out of
the box already.

There are quite a bit of doctest failures though (for now, I only tested
src/sage/rings and src/sage/schemes/ because that is where most
non-trivial uses of PARI are):

----------------------------------------------------------------------
sage -t src/sage/schemes/elliptic_curves/ell_number_field.py # 2
doctests failed
sage -t src/sage/schemes/projective/projective_morphism.py # 1 doctest
failed
sage -t src/sage/rings/number_field/number_field.py # 3 doctests failed
sage -t src/sage/schemes/elliptic_curves/ell_rational_field.py # 11
doctests failed
sage -t src/sage/schemes/elliptic_curves/heegner.py # 37 doctests failed
sage -t src/sage/rings/qqbar.py # 2 doctests failed
sage -t src/sage/rings/number_field/number_field_ideal.py # 1 doctest
failed
sage -t src/sage/rings/number_field/number_field_rel.py # 2 doctests failed
sage -t src/sage/schemes/elliptic_curves/period_lattice.py # 13
doctests failed
sage -t src/sage/rings/padics/padic_capped_absolute_element.pyx # 1
doctest failed
sage -t src/sage/rings/padics/padic_capped_relative_element.pyx # 1
doctest failed
sage -t src/sage/rings/padics/padic_fixed_mod_element.pyx # 1 doctest
failed
sage -t src/sage/schemes/elliptic_curves/modular_parametrization.py # 8
doctests failed
----------------------------------------------------------------------

Usually, this means that the output is still correct, but different. So
most of these will be fixed by simply changing the expected output.

[François Bissey]

> On 6/09/2017, at 23:41, Jeroen Demeyer <jdem...@cage.ugent.be> wrote:
>
>> I imagine binary distro could tie cypari2 to a snapshot version of the pari
>> library, while keeping the normal pari stable.
>
> I don't quite get what you mean with this... For most distros (Gentoo is the exception here), the version of a package is fixed. You cannot have multiple versions of PARI on the same system. In any case, within Sage, all components must use the same PARI library.

Incorrect. At runtime you only need the library with the right soname.
You can have several versions of the same library installed so long
as they have a different soname.
Of course only one of them can be used for linking when you install
dev package on a binary distro.
In the case of pari we have
libpari-gmp.so.5 for pari-2.9.x
libpari-gmp.so.0 for pari-2.{6,8,10}

So you can provide a binary package that only include
libpari-gmp.so.0 for linking specific packages at runtime.
At the same time you can have a full pari-2.9 package installed.
But packages like libpari-dev-2.9 and libpari-dev-2.10 which
provides libpari.so and the headers cannot be installed at the same
time.

François

[Jeroen Demeyer]

On 2017-09-06 13:53, François Bissey wrote:
>
>> On 6/09/2017, at 23:41, Jeroen Demeyer <jdem...@cage.ugent.be> wrote:
>>
>> For most distros (Gentoo is the exception here), the version of a package is fixed. You cannot have multiple versions of PARI on the same system. In any case, within Sage, all components must use the same PARI library.
>
> Incorrect.

I meant what is typical for distros. I don't think that Debian would
agree to have several stable packages linking against different versions
of PARI (even if that is technically possible). Please correct me if I'm
wrong...

[John Cremona]

Indeed. Let me know if you want help with judging this. I can't tell
why heegner should be so much worse....

>
>
> --
> You received this message because you are subscribed to the Google Groups

> "sage-devel" group.

> To unsubscribe from this group and stop receiving emails from it, send an

> email to sage-devel+...@googlegroups.com.

> To post to this group, send email to sage-...@googlegroups.com.
> Visit this group at https://groups.google.com/group/sage-devel.

[Ximin Luo]

Jeroen Demeyer:

I think there was some slight misunderstanding of word usage here. It's certainly possible to *install* multiple versions of one library.

However, it is usually *not* possible to *link in* multiple versions of one library in the same *runtime process* due to conflicting symbol names. e.g. when François says: "12 .pyx in sage needing libpari [..] may also need to be linked to the snapshot" I'd guess the answer is "yes". Or as Jeroen said earlier, "all components must use the same PARI library".

For this reason, Debian tries to avoid having multiple versions of a library *available for installation* (in the online repos) and we developers don't aim towards situations where some binaries use one version and other binaries use other versions. To ease transitions however, it is possible for user systems to have multiple versions of a library installed. The idea is that old ones are (a) not available in the online repos, but are only installed locally and (b) will be automatically removed when no other installed packages need it.

If SageMath is linked against PARI-snapshot then it would force Debian to also link these packages against PARI-snapshot:

- eclib-tools
- lcalc
- libec3
- libgiac0
- liblfunction0
- python-cypari2
- python-cysignals-pari
- xcas

I'm not sure I'm comfortable with that. There is also the heavy work of making both PARI-stable and PARI-snapshot co-installable as François described earlier.

OTOH if SageMath retains compatibility with PARI stable then there is no problem. It seems like this should be possible, if cypari2 already supports it? Then there is the case of failing doctests, which I'll reply to shortly elsewhere in the thread.

[Ximin Luo]

(I'm not on sage-devel@ due to volume, please keep me CC'd in follow-ups)

John Cremona:

> On 6 September 2017 at 12:48, Jeroen Demeyer <jdem...@cage.ugent.be> wrote:
>> On 2017-07-25 22:16, Jeroen Demeyer wrote:
>>>
>>> Hello sage-devel and sage-packaging,
>>>
>>> I propose to upgrade the PARI package to the git master version instead
>>> of the current released version.
>>
>>
>> Now that the cypari2 upgrade has been merged, I am working on this. Thanks
>> to the new cypari2, *building* Sage with PARI master works out of the box
>> already.
>>
>> There are quite a bit of doctest failures though (for now, I only tested
>> src/sage/rings and src/sage/schemes/ because that is where most non-trivial
>> uses of PARI are):
>>

>> [..]

>>
>> Usually, this means that the output is still correct, but different. So most
>> of these will be fixed by simply changing the expected output.
>
> Indeed. Let me know if you want help with judging this. I can't tell
> why heegner should be so much worse....
>

Perhaps it's time to think about enhancing the doctest framework so you can specify greater tolerances in the test results? For example:

- a "recursive-rtol" annotation to apply rtol recursively to a whole data structure.
- a "in-any-order" annotation to allow the test case to pass even if the list/tuple/set is output in a different order
- a "+/-" annotation to allow some elements to be specified with an opposite sign

I've seen numerous cases where Sage has to change the expected test output simply because a dependency was upgraded. There has to be a more sustainable way of achieving this...

[Jeroen Demeyer]

On 2017-09-06 14:53, Ximin Luo wrote:
> I've seen numerous cases where Sage has to change the expected test output simply because a dependency was upgraded. There has to be a more sustainable way of achieving this...

Suggestions welcome...

Note that none of your above proposals is relevant to the typical PARI
failures. We would need support in the doctest framework for stuff like
"# generates same group" or "# same element of H^1(...)" or whatever. I
don't think this can be fixed in the doctest framework.

[Jeroen Demeyer]

On 2017-09-06 14:53, Ximin Luo wrote:

> - a "recursive-rtol" annotation to apply rtol recursively to a whole data structure.

"# tol" works on the level of strings, so it automatically applies to
everything.

[Ximin Luo]

Jeroen Demeyer:

OK. Another suggestion would be to wrap many of the test cases in normalising functions like round() or whatever is appropriate (for PARI it would be group theory related stuff). For example:

>>> calculate_many_floats()
((0.100, 25.000), (3.555, 123.000))
- becomes
>>> recursive_round(calculate_many_floats(), 4)
((0.100, 25.000), (3.555, 123.000))

and

>>> generate_group()
# whatever
- becomes
>>> normalise(generate_group())
# whatever

If there is no "normalise" function for that context, perhaps there is at least a method to check that two objects represent the same mathematical group, so:

>>> generate_group()
[some representation]
- becomes
>>> is_same_group(generate_group(), [other representation])
True

Hopefully this is less brittle than what happens currently.

[Vincent Delecroix]

On 06/09/2017 10:28, Ximin Luo wrote:
> Jeroen Demeyer:
>> On 2017-09-06 14:53, Ximin Luo wrote:
>>> I've seen numerous cases where Sage has to change the expected test output simply because a dependency was upgraded. There has to be a more sustainable way of achieving this...
>>
>> Suggestions welcome...
>>
>> Note that none of your above proposals is relevant to the typical PARI failures. We would need support in the doctest framework for stuff like "# generates same group" or "# same element of H^1(...)" or whatever. I don't think this can be fixed in the doctest framework.
>>
>
> OK. Another suggestion would be to wrap many of the test cases in normalising functions like round() or whatever is appropriate (for PARI it would be group theory related stuff). For example:

+1 It would be nice to have the testsuite pass on the different PARI
versions.

[Antonio Rojas]

El martes, 25 de julio de 2017, 22:44:41 (UTC+2), Jeroen Demeyer escribió:
> Everything also depends on how you define "compatibility". There might
> be a large difference between "Sage builds and works" and "all doctests
> pass". The first one is much more likely than the second one.

As of 8.2.beta7, Sage now effectively hard-depends on a devel version of PARI and doesn't build with 2.9.x anymore, due to c8015d7a71f6b14b0e42b94011eca5b88cdb2889. Any chance the hyperellratpoints usage could be made conditional so Sage builds with 2.9.x until 2.11 is released?

[Julian Rüth]

* Antonio Rojas <nqn...@gmail.com> [2018-03-03 04:11:02 -0800]:

If it makes a packager's life easier, I suppose that we could make the
descent_two_isogeny use conditional compilation and just throw a
NotImplementedError when it needs hyperellratpoints and it has not been built
with the right version of Pari. Would that help?

julian

[Julian Rüth]

* Julian Rüth <julian...@fsfe.org> [2018-03-04 14:52:02 +0100]:

Probably this could look something like this:
https://gitlab.com/saraedum/sage/snippets/1701998

Is there an easy way to find out about PARI's version?

julian

[Antonio Rojas]

El domingo, 4 de marzo de 2018 15:38:35 (CET) Julian Rüth escribió:
> Probably this could look something like this:
> https://gitlab.com/saraedum/sage/snippets/1701998
>
> Is there an easy way to find out about PARI's version?
>
> julian
>

There's PARI_VERSION and PARI_VERSION_CODE in paricfg.h, although they don't seem to be exported by cypari.

[Julian Rüth]

* Antonio Rojas <nqn...@gmail.com> [2018-03-05 14:35:44 +0100]:

> El domingo, 4 de marzo de 2018 15:38:35 (CET) Julian Rüth escribió:
> > Probably this could look something like this:
> > https://gitlab.com/saraedum/sage/snippets/1701998
> >
> > Is there an easy way to find out about PARI's version?

> There's PARI_VERSION and PARI_VERSION_CODE in paricfg.h, although they don't seem to be exported by cypari.

sage: import cypari2
sage: cypari2.Pari().version()
[2, 10, 0]

This could work I think.

julian