Proposal : Add pplpy and gmpy2 as standard packages.

[Vincent Klein]

Hi!

As stated in sage ticket #23024 we propose to replace `sage.libs.ppl` by the standalone pplpy.

This require to have pplpy as a standard package. As pplpy use gmpy2 the latter should be stantard too.

Do we agree to do this ?

Regards

Vincent K.

[Jeroen Demeyer]

On 2018-02-05 09:11, Vincent Klein wrote:
> This require to have pplpy as a standard package. As pplpy use gmpy2 the
> latter should be stantard too.
>
> Do we agree to do this ?

Fine for me.

[Volker Braun]

Isn't having yet another integer type in Sage a huge liability without advantage? Anybody care to explain why we'd want two different gmp/mpir wrappers in Sage? Also, confusion factor for users...

This introduces various breaking changes, for example right now I can do 

sage: from sage.libs.ppl import Variable, Linear_Expression

sage: Linear_Expression(10).inhomogeneous_term().factor()

2 * 5

[Vincent Delecroix]

On 05/02/2018 15:33, Volker Braun wrote:
> Isn't having yet another integer type in Sage a huge liability without
> advantage? Anybody care to explain why we'd want two different gmp/mpir
> wrappers in Sage? Also, confusion factor for users...

- because gmpy2 is de facto a Python standard (used in mpmath and sympy
for example)

- no Standard Sage user is asked to know or use gmpy2 or pplpy.
These are respectively used as gmp data transition and backend
for polyhedra

And without having gmpy2 it is hard to set pplpy a standalone
package. And I see several point in making it happen

- a potentially much larger user base

- a potentially much larger developer base

- pplpy is Python2/Python3 compatible and tested on both

- pplpy currently includes more ppl features than sage.libs.ppl

> This introduces various breaking changes, for example right now I can do
>
> sage: from sage.libs.ppl import Variable, Linear_Expression
> sage: Linear_Expression(10).inhomogeneous_term().factor()
> 2 * 5

Indeed, this is broken but

- As already said sage.libs.ppl is not imported on startup and only
used as a backed in the polyhedron code. People who do use it can
easily handle the transition

- factorizing 10 this way is a bit strange :-)

- factor(Linear_Expression(10).inhomogeneous_term()) should work (if it
does not already)

[Dima Pasechnik]

On Monday, February 5, 2018 at 7:14:12 PM UTC, vdelecroix wrote:

On 05/02/2018 15:33, Volker Braun wrote:
> Isn't having yet another integer type in Sage a huge liability without
> advantage? Anybody care to explain why we'd want two different gmp/mpir
> wrappers in Sage? Also, confusion factor for users...

- because gmpy2 is de facto a Python standard (used in mpmath and sympy
   for example)

- no Standard Sage user is asked to know or use gmpy2 or pplpy.
   These are respectively used as gmp data transition and backend
   for polyhedra

And without having gmpy2 it is hard to set pplpy a standalone
package. And I see several point in making it happen

- a potentially much larger user base

- a potentially much larger developer base

- pplpy is Python2/Python3 compatible and tested on both

- pplpy currently includes more ppl features than sage.libs.ppl

> This introduces various breaking changes, for example right now I can do
>
> sage: from sage.libs.ppl import Variable, Linear_Expression
> sage: Linear_Expression(10).inhomogeneous_term().factor()
> 2 * 5

Indeed, this is broken but

- As already said sage.libs.ppl is not imported on startup and only
   used as a backed in the polyhedron code.

ppl is also a backend to linear optimisation, see

src/sage/numerical/backends/ppl_backend.pyx

[Volker Braun]

On Monday, February 5, 2018 at 8:14:12 PM UTC+1, vdelecroix wrote:

And without having gmpy2 it is hard to set pplpy a standalone
package.

Obviously thats the only real argument here; The question is, whats in it for Sage users? None of your arguments are particularly convincing. 

And I see several point in making it happen
- a potentially much larger user base

In other words, by making ppl complicated to use from Sage we can get rid of all users on that side?

A better implementation would be to make the integer implementation(s) a build-time option; pplpy could internally use a fused type for all available implementations.

[Vincent Delecroix]

On 05/02/2018 22:47, Volker Braun wrote:
> On Monday, February 5, 2018 at 8:14:12 PM UTC+1, vdelecroix wrote:
>>
>> And without having gmpy2 it is hard to set pplpy a standalone
>> package.
>
>
>
> Obviously thats the only real argument here; The question is, whats in it
> for Sage users? None of your arguments are particularly convincing. >> And I see several point in making it happen
>> - a potentially much larger user base
>>
>
>
> In other words, by making ppl complicated to use from Sage we can get rid
> of all users on that side?

How complicated? It would help with another example that is not about
factoring 10. You again ignore the fact that most Sage users (not you)
are using the PPL interface through the Polyhedron/MILP classes and not
directly. I don't see how moving to pplpy would prevent any Sage
user from using it.

As a general remark, more integration with standard standalone Python
libraries such as gmpy2, numpy, scipy, sympy is a good move not a
negative one. Even from a Sage user point of view.

Note also that even though pplpy return gmpy2 mpz/mpq, Sage
Integer/Rational are accepted as inputs in pplpy classes thanks
to #29928 and the efforts in upstream gmpy2.

> A better implementation would be to make the integer implementation(s) a
> build-time option; pplpy could internally use a fused type for all
> available implementations.

This is indeed a good point that is very close to a discussion we had
with M. Köppe and J. Demeyer in #21725. Jeroen pushed hard to have it
the way it is with gmpy2 while I was more toward a fused type for return
values as you suggested. However, I now think that having a Python
module whose API is different with and without Sage is a bad thing (=
more confusing). The gmpy2 way is a simple way of having a Sage
complient and Python independent module.

Vincent

[Volker Braun]

On Monday, February 5, 2018 at 11:58:27 PM UTC+1, vdelecroix wrote:

How complicated? It would help with another example that is not about
factoring 10.

Its a simplified example but not all that synthetic; The output of integral polyhedral computations is often about counting something, and looking at prime factorization is very natural.

As a general remark, more integration with standard standalone Python
libraries such as gmpy2, numpy, scipy, sympy is a good move not a
negative one.

Integers are now printed as mpz(10) and your existing Sage notebooks broke, but we are certain that you will enjoy the better integration outside of Sage.

values as you suggested. However, I now think that having a Python
module whose API is different with and without Sage is a bad thing

Its a Cython module, not pure Python. And I disagree, pretty much every C library has optional features as well. This is hardly a new concept.

[Erik Bray]

On Mon, Feb 5, 2018 at 10:47 PM, Volker Braun <vbrau...@gmail.com> wrote:
> On Monday, February 5, 2018 at 8:14:12 PM UTC+1, vdelecroix wrote:
>>
>> And without having gmpy2 it is hard to set pplpy a standalone
>> package.
>
>
>
> Obviously thats the only real argument here; The question is, whats in it
> for Sage users? None of your arguments are particularly convincing.

Obviously it's not the only real argument here.

"What's in it for Sage users?" -- Eventually, anyways, being able to
drop code from Sage in favor of other, better-supported code that
provides the same thing, so that Sage developers can work on other
things.

[Volker Braun]

On Tuesday, February 6, 2018 at 2:25:58 PM UTC+1, Erik Bray wrote:

"What's in it for Sage users?" -- Eventually, anyways, being able to
drop code from Sage in favor of other, better-supported code that
provides the same thing, so that Sage developers can work on other
things.

Well if we are speaking about stategic "Eventual" goals, what is the effing plan for integers? Replace Sage ZZ with gmpy2? I don't think thats desirable. If that is not the goal, then this ticket is going full steam in the wrong direction. If that is indeed the goal on the other hand, I want to first hear a plan how sage integers are going to be replaced without breaking everything. 

Just "in a couple of years there is some benefit" is not an argument in favor of this ticket; That may very well be the case but we don't have to change anythnig in Sage until that is the case.

[Jeroen Demeyer]

On 2018-02-05 22:47, Volker Braun wrote:
> A better implementation would be to make the integer implementation(s) a
> build-time option; pplpy could internally use a fused type for all
> available implementations.

I think that's a lot easier said than done. Also, a Python class cannot
really contain a fused type: it must contain a particular type.

[Volker Braun]

But implementation details can surely be worked out. A totally basic idea: put the specifics of the integer type and integer wrap/unwrapping in an cython include file. At build time, copy the include file with the desired implementation.

[Vincent Delecroix]

I agree with Volker that having configurable output type (at build time)
is doable and not so complicated technically. Though it is not
desirable. As I already said, having a library with different behavior
with/without Sage will be a nightmare for users (different behaviors)
and for integration in distributions (two incompatible libraries).

An other alternative that we also discussed in #21725 is to have a
configurable output at run time. That is to say, one would have "Cython
plugins" for the various return type that would be configurable at run
time via something ljike

sage: import ppl
sage: ppl.set_return_type("sage")

This is also doable. But it makes it impossible to have Python libraries
to communicate (what if library A needs library B with return type "X"
but the user wants it with return type "Y"). See also the discussion at
#21725.

The gmpy2 way is not ideal (one more integer/rational/floating point
type) but it is
- convenient and error safe with respect to the points mentioned above
- has a working implementation
The only negative point with respect to the previous implementation is
that you have to perform an explicit conversion to a Sage type. To my
mind the inconvenient is worth the benefits.

Cheers
Vincent

PS: Note that it is also possible to integrate better gmpy2 numbers in
Sage ecosystem, see #23052 and #24677.

PPS: To answer to some previous thread, I am not aiming at replacing
Sage integers with gmpy2 integers. Sage integers are far better. I am
just seeing gmpy2 as a convenient way to pass mpz/mpq/mpfr pointers around.

[Volker Braun]

On Wednesday, February 7, 2018 at 9:49:29 AM UTC+1, vdelecroix wrote:

what if library A needs library B with return type "X"
but the user wants it with return type "Y"

So we solve the problem by making it impossible for the user to get the return type he wants (namely Sage intergers by your own admission). Nice!

 

[Jeroen Demeyer]

On 2018-02-07 00:35, Volker Braun wrote:
> But implementation details can surely be worked out. A totally basic
> idea: put the specifics of the integer type and integer wrap/unwrapping
> in an cython include file. At build time, copy the include file with the
> desired implementation.

But what is the "desired implementation"? How can the pplpy package know
which kind of integers the user wants? What if there are multiple users
with different preferences?

I think that this is going to cause more problems than it solves.

[Vincent Klein]

Le mercredi 7 février 2018 09:49:29 UTC+1, vdelecroix a écrit :

The gmpy2 way is not ideal (one more integer/rational/floating point
type) but it is
  - convenient and error safe with respect to the points mentioned above
  - has a working implementation

And a logical design choice for pplpy given ppl return gmp numbers.

[Volker Braun]

On Wednesday, February 7, 2018 at 10:31:43 AM UTC+1, Jeroen Demeyer wrote:

But what is the "desired implementation"? 

Again, surely its "Sage integers" for users of Sage. Anything else is just dumping a giant turd on your users just because its the lazy thing to do. 

There are surely ways this can work. E.g. importing from pplpy.gmp gives you stuff that uses gmp ints, and importing from pplpy.sage gives you stuff that uses Sage integers. Or pplpy vs sage_pplpy, etc. Its not impossible to make it work, it just requires a bit more than the minimal developer effort to set it up. 

[Vincent Delecroix]

On 07/02/2018 12:17, Volker Braun wrote:
> On Wednesday, February 7, 2018 at 10:31:43 AM UTC+1, Jeroen Demeyer wrote:
>>
>> But what is the "desired implementation"?
>>
>
> Again, surely its "Sage integers" for users of Sage. Anything else is just
> dumping a giant turd on your users just because its the lazy thing to do.

Certainly not. The lazy way is to use Python integers and rationals. It
was this way at the begining and I changed it partly because of Sage as
I wanted to avoid useless traductions (at C level) and better C level
integration. This was a non-trivial amount of work in upstream
gmpy2.

> There are surely ways this can work. E.g. importing from pplpy.gmp gives
> you stuff that uses gmp ints, and importing from pplpy.sage gives you stuff
> that uses Sage integers. Or pplpy vs sage_pplpy, etc. Its not impossible to
> make it work, it just requires a bit more than the minimal developer effort
> to set it up.

I never claimed that it was technically impossible. I said that it was
not desirable to have two non-compatible API's. Having two libraries as
you proposed above is one solution. Though I don't like it to have two
copies of the same code. Another option is that I provide "universal
wrappers" that makes the necessary translations to make you happy.
Another option is to go with this small annoyance of dealing with
explicit conversions.

Vincent

[jplab]

Hi all,

Allow me to jump in the discussion as a daily "Sage user" of code using the ppl backend in the Polyhedron class.
I put the "Sage user" in quotes, because it seems that it was used for different types of users.

What's in for the normal strictly routine not-developing user?

Well, IMHO more transparency about libraries used. Basically, I just figured out where the wrapper for ppl was(!).
Ok, perhaps I'm ignorant or stupid; but to me, it seems like making pplpy and gmpy2 standard packages
is not a revolutionary idea that is going to break everything, or is it? Further,

- no Standard Sage user is asked to know or use gmpy2 or pplpy.
   These are respectively used as gmp data transition and backend
   for polyhedra

Further, I believe that the strictly routine not-developing user will not notice the difference as the vast majority
(of the users of sage that I know) are not even aware that there is such a thing as a "python integer" and a
"Sage integer" and they keep on using Sage despite some of its incongruities. So if we speak of those users,
I can not really see how any smelly unpleasantness would be laid down on their heads by making ppl available
through a standard library as opposed to being Cython wrapped.

I am aware that I do not have a full overview of the changes implied by making the transition smooth, but since I felt
concerned as an indirect daily user of ppl, I wanted to express my opinion on the matter:

Propose to replace `sage.libs.ppl` by the standalone pplpy. This require to have pplpy as a standard package. 

As pplpy use gmpy2 the latter should be stantard too.

+1

As Vincent pointed out:

- pplpy currently includes more ppl features than sage.libs.ppl

and could always be improved by a larger user/developer base being a python2&3 package.

Jean-Philippe

[Erik Bray]

+1 the vast majority of routine users do not care about such
implementation details. It is a minority of users who do, and if we
retain backwards compatibility at first, and transition instructions,
those few users who care (and are generally more advanced) are able to
adapt. In the long run it's win-win for everyone.

[Vincent Klein]

In ticket #23024 we made pplpy and independant package with gmpy2 as return types.

As opinion for having pplpy returning Sage's type has also been expressed, the goal of this message is to check 

if we globally agree to pass pplpy and gmpy2 as standard packages.

Vincent K.

[Dima Pasechnik]

+1

--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To post to this group, send email to sage-...@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.