Support for gmpy2

[casevh]

Hello,

I maintain GMPY and GMPY2 and I'd like to provide a patch to resolve compatibility issues with GMPY2.

The most significant difference is that support for mpq.numerator and mpq.denominator was added in GMPY 1.13 and is available in GMPY2 (and in Python's standard Fraction type). The code in polys uses the old mpq.numer() and mpq.denom() functions that were removed in GMPY2. Would it be acceptable to change the minimum version of GMPY to 1.13 from 1.03? Then the mpq.numer() calls could be changed to mpq.numerator, etc. 

I've resolved most of the other remaining issues but there are a couple that I'm still tracking down. I just want to verify that upgrading the minimum version to 1.13 is acceptable.

Regards,

casevh

[Aaron Meurer]

On Jan 28, 2013, at 8:54 PM, casevh <cas...@gmail.com> wrote:

Hello,

I maintain GMPY and GMPY2 and I'd like to provide a patch to resolve compatibility issues with GMPY2.

The most significant difference is that support for mpq.numerator and mpq.denominator was added in GMPY 1.13 and is available in GMPY2 (and in Python's standard Fraction type). The code in polys uses the old mpq.numer() and mpq.denom() functions that were removed in GMPY2. Would it be acceptable to change the minimum version of GMPY to 1.13 from 1.03? Then the mpq.numer() calls could be changed to mpq.numerator, etc. 

Absolutely. Gmpy is an optional dependency, so supporting only the latest version is fine. And unlike supporting old versions of Python or IPython, old versions of gmpy are likely to lead to wrong result errors (assumedly). 

What is the difference between numerator and numer?

By the way, I was planning in applying your patch fixing the version check, but I haven't gotten around to it. So if you want to do that as well, go ahead. 

Aaron Meurer

I've resolved most of the other remaining issues but there are a couple that I'm still tracking down. I just want to verify that upgrading the minimum version to 1.13 is acceptable.

Regards,

casevh

--
You received this message because you are subscribed to the Google Groups "sympy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sympy+un...@googlegroups.com.
To post to this group, send email to sy...@googlegroups.com.
Visit this group at http://groups.google.com/group/sympy?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Case Van Horsen]

On Mon, Jan 28, 2013 at 8:45 PM, Aaron Meurer <asme...@gmail.com> wrote:
> On Jan 28, 2013, at 8:54 PM, casevh <cas...@gmail.com> wrote:
>
> Hello,
>
> I maintain GMPY and GMPY2 and I'd like to provide a patch to resolve
> compatibility issues with GMPY2.
>
> The most significant difference is that support for mpq.numerator and
> mpq.denominator was added in GMPY 1.13 and is available in GMPY2 (and in
> Python's standard Fraction type). The code in polys uses the old mpq.numer()
> and mpq.denom() functions that were removed in GMPY2. Would it be acceptable
> to change the minimum version of GMPY to 1.13 from 1.03? Then the
> mpq.numer() calls could be changed to mpq.numerator, etc.
>
>
> Absolutely. Gmpy is an optional dependency, so supporting only the latest
> version is fine. And unlike supporting old versions of Python or IPython,
> old versions of gmpy are likely to lead to wrong result errors (assumedly).

GMPY 1.13 was released in 2010 and was one of the first versions that
supported Python 3.x well. There have only been a few bug fixes since
then to the 1.x series.

>
> What is the difference between numerator and numer?

numer() is a function that lives in both the "gmpy" namespace and is
also available as a method of an mpq instance. numerator was
introduced as an attribute of Python's Fraction and integer types in
2.5 (IIRC) as part of the abc numerical tower. I added
numerator/denominater as mpq attributes in 1.13. In the 2.x series, I
removed numer() and denom() as methods on an mpq instance and made
sure the attribute lookups were present. The numer() and denom()
functions are still present. The attribute lookups are slightly
faster, too.

>
> By the way, I was planning in applying your patch fixing the version check,
> but I haven't gotten around to it. So if you want to do that as well, go
> ahead.

The patch is correct for mpmath, but it breaks code in the polys
module. I'll work on a complete patch.

[Aaron Meurer]

I believe it was actually 2.6. But don't worry if gmpy can't work in
Python 2.5. Neither does IPython, and we will probably drop 2.5
support either in the next release or the one after it (depending on
how soon we release again).

>
>>
>> By the way, I was planning in applying your patch fixing the version check,
>> but I haven't gotten around to it. So if you want to do that as well, go
>> ahead.
>
> The patch is correct for mpmath, but it breaks code in the polys
> module. I'll work on a complete patch.

Great. By the way, what is the status of gmpy2? Should be looking to
support that as well (or instead of gmpy1)?

Aaron Meurer

[Case Van Horsen]

For integer and rational support, gmpy2 is stable. gmpy2 also wraps
the MPFR and MPC libraries but the API may change. Since sympy uses
mpmath for real and complex arithmetic, I don't sympy needs gmpy2's
new features. I'm confident I can make any version >= 1.13 work.

Here is an update on my progress.

I found gmpy imported twice in sympy code and either gmpy2 or gmpy
imported in mpmath. As an experiment, I imported gmpy2 or gmpy in
sympy.core.compatibility and then tweaked the other import locations.
This did work, but I think I'll follow a different approach (see
below).

The majority of the remaining changes were (1) replacing mpq.numer() &
mpq.denom() with mpq.numerator and mpq.denominator and (2) replacing
changing the qdiv() method calls to gmpy.qdiv() function calls. (gmpy
duplicated almost all function calls with method calls; gmpy2 removes
most of duplicate method calls).

There are still a few subtle issues. 'bitcount' is directly imported
from mpmath. The pure Python version of bitcount works with a sympy
Integer, but the gmpy version does not. I created a simple wrapper
function that returns mpmath_bitcount(int(a)). Is that the recommended
way to force an Integer to an int or is there a different strategy I
should follow?

I've also found code in scaled_zero that does "type(mag) is int". This
fails when gmpy is used since the type is actually an mpz. This could
also fail if mag is a long on Python 2.x. mpmath uses a tuple call
int_types that contains all the low-level integer types - the
appropriate combination of int, long, and mpz. I just imported
int_types from mpmath and changed the code to "type(mag) in int_types"
in the various places I found an "is int:" fragment. I made this
change aggressively and didn't check that is caused an issue with gmpy
since it could also trigger a bug with Python 2.x.

core/numbers.py also uses isinstance(n, (int, long, Rational)) which I changed.

I'm down to one failed test to resolve.

Given the dependencies on mpmath's int_types, I think I'll change my
approach to use mpmath's version of gmpy throughout. This will require
that (1) mpmath gets imported before anything else that uses gmpy (I
think that is true) and (2) the minimum version check in mpmath needs
to be changed.

Regards,
casevh

[Aaron Meurer]

That should work. If you know it's always an Integer, a.p is probably
a little faster.

>
> I've also found code in scaled_zero that does "type(mag) is int". This
> fails when gmpy is used since the type is actually an mpz. This could
> also fail if mag is a long on Python 2.x. mpmath uses a tuple call
> int_types that contains all the low-level integer types - the
> appropriate combination of int, long, and mpz. I just imported
> int_types from mpmath and changed the code to "type(mag) in int_types"
> in the various places I found an "is int:" fragment. I made this
> change aggressively and didn't check that is caused an issue with gmpy
> since it could also trigger a bug with Python 2.x.

These "type is" calls should actually use isinstance. And we've been
hit before with bugs from not checking for long.

>
> core/numbers.py also uses isinstance(n, (int, long, Rational)) which I changed.
>
> I'm down to one failed test to resolve.
>
> Given the dependencies on mpmath's int_types, I think I'll change my
> approach to use mpmath's version of gmpy throughout. This will require
> that (1) mpmath gets imported before anything else that uses gmpy (I
> think that is true) and (2) the minimum version check in mpmath needs
> to be changed.

Ok, but make sure that setting the ground types to Python still
disables gmpy in the polys, and conversely, disabling gmpy in mpmath
(I think there's some environment variable) doesn't disable it in the
polys.

Aaron Meurer

[Case Van Horsen]

I'll change the ones encounter to use isinstance.

>
>>
>> core/numbers.py also uses isinstance(n, (int, long, Rational)) which I changed.
>>
>> I'm down to one failed test to resolve.
>>
>> Given the dependencies on mpmath's int_types, I think I'll change my
>> approach to use mpmath's version of gmpy throughout. This will require
>> that (1) mpmath gets imported before anything else that uses gmpy (I
>> think that is true) and (2) the minimum version check in mpmath needs
>> to be changed.
>
> Ok, but make sure that setting the ground types to Python still
> disables gmpy in the polys, and conversely, disabling gmpy in mpmath
> (I think there's some environment variable) doesn't disable it in the
> polys.

Just curious: why wouldn't you use a common base integer type in both
polys and mpmath? I would be concerned with accidentally leaking base
types between polys and mpmath. Do you consistently convert to a
Python type when interfacing between the two (or does interfacing the
base types rarely happen)?

casevh

[Aaron Meurer]

I believe code that calls mpmath is very careful to convert everything
to mpf first. And the polys are also careful to always use the correct
type, as long as you are at a high enough level (low level code
doesn't do error checking for performance reasons).

Aaron Meurer

[Ondřej Čertík]

Hi Case!

Thanks for working on this. I have finally found time and added gmpy
support into our Travis-CI testing:

https://github.com/sympy/sympy/pull/1756

the tests discovered a few minor gmpy failures today:

http://code.google.com/p/sympy/issues/detail?id=3616

Btw, I also noticed, that gmpy2 doesn't have the sqrt method, required
by mpmath.

Ondrej

[Aaron Meurer]

From the previous response: "gmpy duplicated almost all function calls
with method calls; gmpy2 removes most of duplicate method calls". So I
guess sqrt is only a function in gmpy2.

Aaron Meurer

>
> Ondrej

[Case Van Horsen]

On Tue, Feb 5, 2013 at 5:42 PM, Aaron Meurer <asme...@gmail.com> wrote:
> On Tue, Feb 5, 2013 at 5:46 PM, Ondřej Čertík <ondrej...@gmail.com> wrote:
>> Hi Case!
>>
>> Thanks for working on this. I have finally found time and added gmpy
>> support into our Travis-CI testing:
>>
>> https://github.com/sympy/sympy/pull/1756
>>
>> the tests discovered a few minor gmpy failures today:
>>
>> http://code.google.com/p/sympy/issues/detail?id=3616
>>
>> Btw, I also noticed, that gmpy2 doesn't have the sqrt method, required
>> by mpmath.

In gmpy2, I added support for MPFR and MPC and one of the goals was to
make gmpy2 follow existing Python conventions as much as possible.
gmpy2.sqrt now returns an mpfr type and gmpy2.isqrt returns the
integer square root.

The mpmath repository includes a couple fixes for gmpy2. For example,
one of fixes in mpmath is

if BACKEND == 'gmpy':
if gmpy.version() >= "2":
isqrt_small = isqrt_fast = isqrt = gmpy.isqrt
sqrtrem = gmpy.isqrt_rem
else:
isqrt_small = isqrt_fast = isqrt = gmpy.sqrt
sqrtrem = gmpy.sqrtrem

I've follwed a similar approach in sympy.

casevh

[Case Van Horsen]

Hi,

A question on the approach you'd like me take.

I'm setting a value (HAS_GMPY) in sympy.core.compatibility to reflect
if a valid version of gmpy/gmpy2 is available and the version number.
groundtypes.py checks that value to determine which version to import.
polys.domains.__init__.py uses HAS_GMPY and the SYMPY_GROUND_TYPES env
variable to determine which integer type to support. But
groundtypes.py doesn't take SYMPY_GROUND_TYPES into account and will
always import gmpy/gmpy2 if HAS_GMPY is set. It's not a performance
issue since gmpy/gmpy2 has already been imported.

I'd also like to define couple of constants that contain the valid
integer and rational base types. For example,

if sys.version_info[0] == '2':
SYMPY_INTS = (int, long)
else:
SYMPY_INTS = (int,)

if HAS_GMPY and GROUND_TYPES == 'gmpy'
SYMPY_INTS += (type(gmpy.mpz(0)),)

I'd like to define SYMPY_INTS in sympy.core.compatibility along with
HAS_GMPY. The list of valid base integer types should reflect the
GROUND_TYPES selection in addition to the Python base type(s). Would
it be okay to move the GROUND_TYPES logic to sympy.core.compatibility
and also to reflect the choice for GROUND_TYPES in the setting for
HAS_GMPY? (i.e. if GROUND_TYPES equals 'python', just set HAS_GMPY to
0, etc.) polys.domains.__init__.py would just import GROUND_TYPES.

casevh

[Aaron Meurer]

This all sounds good to me.

Aaron Meurer

[Ondřej Čertík]

I see. Thanks a lot for fixing it.

Ondrej

[Case Van Horsen]

I created issue 3622 and uploaded a patch. All the tests pass on my
machine with either gmpy or gmpy2. The sympy test uncovered a bug in
gmpy2 so I'll release a new beta version in a few days.

I haven't quite figured git yet so I thought I'd get the patch out there.

casevh

[Ondřej Čertík]

Awesome, thanks a lot! Here is the PR with your patch:

https://github.com/sympy/sympy/pull/1762

you can watch the Travis tests over there.

> I haven't quite figured git yet so I thought I'd get the patch out there.

That was very helpful. In case you are curious what I did with your patch:

git co -b gmpy
git apply ~/Downloads/0001-Changes-to-support-gmpy-1.13-or-later-or-gmpy2-2.0.0.patch
git ci -a --author="Case Van Horsen <cas...@gmail.com>"
# copy & pasted your git log, formatted it to nicely fit the first line
git push ondrej gmpy

went to github, clicked send pull request and that's it.

Ondrej

[Case Van Horsen]

There is no exact answer, but based on some tests, I would guess the running time will be weeks, possibly months.

casevh

On Tue, Mar 29, 2016 at 9:48 AM, <gkat...@gmail.com> wrote:

Test Linux 64 4 GB ram......To be continued
gmpy2.next_prime(1,041,393 Digits)

Htop


end forecast?

--
You received this message because you are subscribed to a topic in the Google Groups "sympy" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/sympy/_XTdBQOR-Jk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to sympy+un...@googlegroups.com.

To post to this group, send email to sy...@googlegroups.com.

Visit this group at https://groups.google.com/group/sympy.
To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/c52e1893-ab3f-427c-bb7a-1492a6564909%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.