Where sympy processes not-evaluated powers?

44 views
Skip to first unread message

Michał Pawłowski

unread,
Jun 14, 2021, 3:28:04 AM6/14/21
to sympy
Hello.

I'm trying to modify sympy to avoid of hanging, when numer which is gonna to be computed is tu large. I know how to raise exception, but In which file should i look for power computing without eval?

In core/numbers.py there are mothods for evaluation of numbers. But where should I look for methods for not evaluated powers?

Thanks
Mike

Chris Smith

unread,
Jun 16, 2021, 7:52:42 AM6/16/21
to sympy
There are `_eval_power` routines for many objects. So you have to find the object that you don't want to evaluate when the exponent is too large. If that abject doesn't have a routine then you can give it one and handle this case of interest.

/c

Aaron Meurer

unread,
Jun 18, 2021, 3:39:23 PM6/18/21
to sympy
Most likely the method you are looking for is in
sympy/core/numbers.py. I would recommend taking a version of the
expression that has small numbers and using a debugger to figure out
what methods are called.

You should be aware also that if you just type something like 2**100
directly into Python, that doesn't involve SymPy at all and gets
evaluated by Python, which we cannot prevent. SymPy only comes into
play if one of the numbers is a SymPy type.

Aaron Meurer
> --
> You received this message because you are subscribed to the Google Groups "sympy" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sympy+un...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/c091e73e-782a-4b83-9b16-23797465fb66n%40googlegroups.com.

Michał Pawłowski

unread,
Jun 19, 2021, 8:28:24 AM6/19/21
to sympy
Hey, thank you. I figured out how to block integer power of integer, if result is tu high (over 10**4 decimal digits). I also figured out how to block integer and float multiplication when result is too high

But I highly recomend, to put it in official release of new version of SymPy. My code can be broken. :(

from mpmath import log10 as _log10
from mpmath import power as _pow
from mpmath import fabs as _fabs

def checkIf_Pow_MaxValReached(b, e):
    b = _fabs(str(b))
    e = _fabs(str(e))
    
    if b == 0:
        b = 1
    
    b, e = str(b), str(e)
    
    if _pow(e, 1) * _fabs(_log10(b)) > 10000:   #DO POPRRAWY
        raise ValueError("tooBig")

I call this function in power.py file in __new__ method al the beginning

        if evaluate:
            if b.is_integer and e.is_integer:
                checkIf_Pow_MaxValReached(b, e)

To block multiply i needed to edit numbers.py file inside of Integer class in __mul__ method. 

Please add this functionality into official sympy :(

Oscar Benjamin

unread,
Jun 19, 2021, 9:03:44 AM6/19/21
to sympy
I agree that this should be changed in sympy itself. It is just a case that someone needs to do the work for this. It's not just about disabling the evaluation because then there are potentially other parts of the codebase that might subtly break if evaluation does not happen. I would investigate this by setting the limit to be very low and then running the test suite to see what fails and whether the code that expects evaluation can be improved.

Your suggestion above is to raise ValueError but I think that having an evaluated power is probably better in many (but not all) situations.

There are also some points to consider such as how a user could choose what the limits should be in case they do want to work with extremely large integers.

Also I find that most often this issue arises as part of evalf. Changing the Float._eval_power has no effect on the internal code of evalf. Basically someone needs to go through all of the evalf routines and consider where absurdly large numbers might appear and then abort the operation if some limit would be exceeded. Probably this should be a controllable option for evalf as well which has a maximum option for precision but not for the size of the exponent.

Oscar

David Bailey

unread,
Jun 20, 2021, 4:05:25 PM6/20/21
to sy...@googlegroups.com
Dear group,

I recently spent some time debugging something which was producing an
expression involving k*zoo.

Eventually I used help on zoo and discovered it meant complex infinity!

k/0

gives

k*zoo

Wouldn't it be more helpful to spell this out as k*ComplexInfinity?
Maybe oo could be spelled out as Infinity as well?

David

Aaron Meurer

unread,
Jun 21, 2021, 6:14:05 PM6/21/21
to sympy
It's worth noting that if you used a Jupyter notebook with LaTeX
output, zoo prints in a nice way (\tilde{\infty}), that makes it
easier to tell what it is.

The problem with printing oo and zoo in longer form is that it makes
things more verbose, especially for oo, which might appear many times
in an expression, if it is in the limits of integration for instance.
I do think you should be able to enter them by typing out Infinity and
ComplexInfinity. Presently, you can do this if you add an S, like
S.Infinity, but only the shorter names are in the top-level namespace.

I do agree that the name "zoo" is a little opaque (I guess it means
z-oo, where oo is infinity and z means "the complex variable z"). Even
oo is confusing until you realize what it is. Personally I would have
just called it inf.

Aaron Meurer
> --
> You received this message because you are subscribed to the Google Groups "sympy" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sympy+un...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/ac945d13-2eb6-f073-1186-b16358e6ac4e%40dbailey.co.uk.

David Bailey

unread,
Jun 22, 2021, 6:06:29 AM6/22/21
to sy...@googlegroups.com
Thanks for that explanation. Clearly oo is something that is routinely
input by users - as an integration limit, for example - but to my
knowledge complex infinity is far less useful. I do wonder if rather
than generating expressions involving zoo, it might be better to simply
raise an exception?


David


Aaron Meurer

unread,
Jun 22, 2021, 6:18:34 PM6/22/21
to sympy
On Sat, Jun 19, 2021 at 7:03 AM Oscar Benjamin
<oscar.j....@gmail.com> wrote:
>
> I agree that this should be changed in sympy itself. It is just a case that someone needs to do the work for this. It's not just about disabling the evaluation because then there are potentially other parts of the codebase that might subtly break if evaluation does not happen. I would investigate this by setting the limit to be very low and then running the test suite to see what fails and whether the code that expects evaluation can be improved.

This is the tracking issue for this https://github.com/sympy/sympy/issues/6835

>
> Your suggestion above is to raise ValueError but I think that having an evaluated power is probably better in many (but not all) situations.
>
> There are also some points to consider such as how a user could choose what the limits should be in case they do want to work with extremely large integers.
>
> Also I find that most often this issue arises as part of evalf. Changing the Float._eval_power has no effect on the internal code of evalf. Basically someone needs to go through all of the evalf routines and consider where absurdly large numbers might appear and then abort the operation if some limit would be exceeded. Probably this should be a controllable option for evalf as well which has a maximum option for precision but not for the size of the exponent.

This seems like something that would need to be implemented in mpmath,
not evalf. Fortunately, there it could be implemented in just a single
place (on the mpf type), and it wouldn't require doing bounds
estimates on every possible mpmath function. Furthermore, it is quite
rare to have mpfs with very large exponents, so it's unlikely this
would break much if we set the limit to some reasonably large number.
mpmath is actually relatively unique in that it does allow arbitrarily
large exponents. Most other arbitrary precision libraries that I've
seen allow arbitrarily large precision, but limit the exponent to a
machine word.

Aaron Meurer
> To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/CAHVvXxS9rKkWB4-N5ebPG%2BY%2BTuuzZmeXr_7Q9Xau%3D2aYP4PvzA%40mail.gmail.com.

Aaron Meurer

unread,
Jun 23, 2021, 5:49:24 PM6/23/21
to sympy
Complex infinity is a mathematically meaningful object, and SymPy
generally prefers to give mathematically meaningful results when it
can, even if they are unevaluated. You're right that if you get zoo
out of something it usually means you made a mistake somewhere, but it
is also possible to use it intentionally.

Although this argument could be made for something like nan, which
isn't as mathematically meaningful. It's really just a way to say
"undefined" and exists primarily so that we can avoid exceptions.

Aaron Meurer
> To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/981f28be-8571-a036-318e-a5671a18cbb6%40dbailey.co.uk.

Oscar Benjamin

unread,
Jun 23, 2021, 6:15:59 PM6/23/21
to sympy
There are some useful things that you can do with zoo like:

In [1]: 1/zoo
Out[1]: 0

If it weren't for things like this then I think exceptions would
always be better.


--
Oscar
> To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/CAKgW%3D6LxVWYjX%3DFmW%3DbfCUtaRkWqsGPKQnTTAo4igHrh6tfd0w%40mail.gmail.com.

David Bailey

unread,
Jun 24, 2021, 10:40:28 AM6/24/21
to sy...@googlegroups.com
On 23/06/2021 23:15, Oscar Benjamin wrote:
> There are some useful things that you can do with zoo like:
>
> In [1]: 1/zoo
> Out[1]: 0
>
I suppose I might question the word 'useful' in your example! I mean oo
is clearly extremely useful for limits, integrals, etc, but zoo isn't,
because its phase is indeterminate.

Aaron also mentioned nan's, and indeed zoo/zoo returns nan! He justifies
these objects because they avoid raising exceptions - but exceptions
were invented because they are useful.

In a string of calculations, zoo's and nan's will just end up being
passed to other processes that will ultimately raise exceptions that are
harder to interpret. For example:

integrate(exp(-x),(x,0,zoo))

David


Aaron Meurer

unread,
Jun 24, 2021, 4:45:27 PM6/24/21
to sympy
On Thu, Jun 24, 2021 at 8:40 AM David Bailey <da...@dbailey.co.uk> wrote:
>
> On 23/06/2021 23:15, Oscar Benjamin wrote:
> > There are some useful things that you can do with zoo like:
> >
> > In [1]: 1/zoo
> > Out[1]: 0
> >
> I suppose I might question the word 'useful' in your example! I mean oo
> is clearly extremely useful for limits, integrals, etc, but zoo isn't,
> because its phase is indeterminate.

Anyone who's studied complex numbers knows that complex infinity (the
north pole of the Riemann sphere) is a very useful concept. The
usefulness in SymPy is limited by what it is able to compute. For
example, limits currently don't work with zoo. The main benefit for
SymPy is that zoo is a mathematically consistent answer for 1/0, which
helps avoid wrong results.

>
> Aaron also mentioned nan's, and indeed zoo/zoo returns nan! He justifies
> these objects because they avoid raising exceptions - but exceptions
> were invented because they are useful.

I do agree with you here regarding nan, but not zoo. Although I
haven't thought too hard about what the consequences might be for
making code give exceptions where it would currently give nan.

>
> In a string of calculations, zoo's and nan's will just end up being
> passed to other processes that will ultimately raise exceptions that are
> harder to interpret. For example:
>
> integrate(exp(-x),(x,0,zoo))

In this case, it's garbage in, garbage out (the integral is
mathematically meaningless). I'm not sure I see how an indirect
computation could result in this particular example.

In general, if you make a mistake in a way that's still mathematically
valid, you aren't going to get an error at the site of the mistake.
For nan, the point is made that the expression stops being
mathematically valid as soon as it appears, but for zoo, it is valid
in complex numbers. That just might not be what you originally
intended.

By the way, there is some work to make the assumptions on SymPy
symbols finite by default, which can help with some of these concerns.
Right now a Symbol('x') with no given assumptions isn't assumed to be
finite, so it could be zoo as far as the assumptions are concerned.

Aaron Meurer

>
> David
>
>
> --
> You received this message because you are subscribed to the Google Groups "sympy" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sympy+un...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/e19b91f1-3b6c-8656-ab23-ac62b16496dc%40dbailey.co.uk.
Reply all
Reply to author
Forward
0 new messages