TypeError using sympy.stats.P

96 views
Skip to first unread message

Sebastian Koslowski

unread,
Mar 19, 2015, 10:24:18 AM3/19/15
to sy...@googlegroups.com
Hey,

I am trying to use the stats module to calculate probabilities for a normal distributed RV.
Problem is, I sometimes get "TypeError: cannot determine truth value of -oo < -inf". 
Weird thing is, the error is not always reproducible between (ipython) sessions (see below)

Can somebody explain this behavior? Or is this a bug?

Thanks,
Sebastian


First run:

Python 3.3.2 (default, Dec  4 2014, 12:49:00) 
Type "copyright", "credits" or "license" for more information.

IPython 2.3.1 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: import sympy; sympy.__version__
Out[1]: '0.7.6'

In [2]: from sympy.stats import Normal, P

In [3]: N0 = Normal('N0', 0, 1)

In [4]: P(N0 < 1.1)
Out[4]: erf(0.55*sqrt(2))/2 + erf(+inf*sqrt(2))/2

In [5]: P(N0 < 1.1001)
Out[5]: erf(0.55005*sqrt(2))/2 + erf(+inf*sqrt(2))/2

In [6]: P(N0 <= 1.1001)
Out[6]: erf(0.55005*sqrt(2))/2 + erf(+inf*sqrt(2))/2


Second run:

Python 3.3.2 (default, Dec  4 2014, 12:49:00) 
Type "copyright", "credits" or "license" for more information.

IPython 2.3.1 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: import sympy; sympy.__version__
Out[1]: '0.7.6'

In [2]: from sympy.stats import Normal, P

In [3]: N0 = Normal('N0', 0, 1)

In [4]: P(N0 < 1.1001)
Out[4]: erf(0.55005*sqrt(2))/2 + erf(+inf*sqrt(2))/2

In [5]: P(N0 <= 1.1001)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-5-b60ca102def7> in <module>()
----> 1 P(N0 <= 1.1001)

/usr/lib/python3.3/site-packages/sympy/stats/rv.py in probability(condition, given_condition, numsamples, evaluate, **kwargs)
    615 
    616     # Otherwise pass work off to the ProbabilitySpace
--> 617     result = pspace(condition).probability(condition, **kwargs)
    618     if evaluate and hasattr(result, 'doit'):
    619         return result.doit()

/usr/lib/python3.3/site-packages/sympy/stats/crv.py in probability(self, condition, **kwargs)
    294         # Univariate case can be handled by where
    295         try:
--> 296             domain = self.where(condition)
    297             rv = [rv for rv in self.values if rv.symbol == domain.symbol][0]
    298             # Integrate out all other random variables

/usr/lib/python3.3/site-packages/sympy/stats/crv.py in where(self, condition)
    320         rv = tuple(rvs)[0]
    321         interval = reduce_rational_inequalities_wrap(condition, rv)
--> 322         interval = interval.intersect(self.domain.set)
    323         return SingleContinuousDomain(rv.symbol, interval)
    324 

/usr/lib/python3.3/site-packages/sympy/sets/sets.py in intersect(self, other)
     93 
     94         """
---> 95         return Intersection(self, other)
     96 
     97     def intersection(self, other):

/usr/lib/python3.3/site-packages/sympy/sets/sets.py in __new__(cls, *args, **kwargs)
   1285         # Reduce sets using known rules
   1286         if evaluate:
-> 1287             return Intersection.reduce(args)
   1288 
   1289         args = list(ordered(args, Set._infimum_key))

/usr/lib/python3.3/site-packages/sympy/sets/sets.py in reduce(args)
   1370                 new_args = False
   1371                 for t in args - set((s,)):
-> 1372                     new_set = s._intersect(t)
   1373                     # This returns None if s does not know how to intersect
   1374                     # with t. Returns the newly intersected set otherwise

/usr/lib/python3.3/site-packages/sympy/sets/sets.py in _intersect(self, other)
    818         if self.start <= other.end and other.start <= self.end:
    819             # Get topology right.
--> 820             if self.start < other.start:
    821                 start = other.start
    822                 left_open = other.left_open

/usr/lib/python3.3/site-packages/sympy/core/relational.py in __nonzero__(self)
    101 
    102     def __nonzero__(self):
--> 103         raise TypeError("cannot determine truth value of\n%s" % self)
    104 
    105     __bool__ = __nonzero__

TypeError: cannot determine truth value of
-oo < -inf

Joachim Durchholz

unread,
Mar 19, 2015, 1:02:46 PM3/19/15
to sy...@googlegroups.com
Am 19.03.2015 um 11:38 schrieb Sebastian Koslowski:
> Hey,
>
> I am trying to use the stats module to calculate probabilities for a normal
> distributed RV.
> Problem is, I sometimes get "TypeError: cannot determine truth value of -oo
> < -inf".
> Weird thing is, the error is not always reproducible between (ipython)
> sessions (see below)
>
> Can somebody explain this behavior?

One cause of random behaviour is built right into Python.
Set the environment variable PYTHONHASHSEED to an integer when starting
SymPy, and this source of randomness is eliminated and you'll get
repeatable behaviour (different behaviour with different hash seeds).

I don't know whether the stats module uses some kind of Monte-Carlo
algorithm when working with probability distributions. If it does, this
might be some intermediate result causing crashes.

> Or is this a bug?

I'm not sure; it's likely, but it could be a side effect.

Sebastian Koslowski

unread,
Mar 24, 2015, 6:18:14 AM3/24/15
to sy...@googlegroups.com


On Thursday, March 19, 2015 at 6:02:46 PM UTC+1, Joachim Durchholz wrote:

One cause of random behaviour is built right into Python.
Set the environment variable PYTHONHASHSEED to an integer when starting
SymPy, and this source of randomness is eliminated and you'll get
repeatable behaviour (different behaviour with different hash seeds).

Thanks, that seems to remove the randomness of the error. 
 
I don't know whether the stats module uses some kind of Monte-Carlo
algorithm when working with probability distributions. If it does, this
might be some intermediate result causing crashes.

Looks like it fails determining the domain from the condition. 
Before the actual calculation.
 

 > Or is this a bug?

I'm not sure; it's likely, but it could be a side effect.

Judging from the error message and traceback I looks like a bug to me.


Joachim Durchholz

unread,
Mar 24, 2015, 7:33:01 AM3/24/15
to sy...@googlegroups.com
Am 24.03.2015 um 11:18 schrieb Sebastian Koslowski:
>
> Judging from the error message and traceback I looks like a bug to me.

Can you post an example?
I.e. full session, from PYTHONHASHSEED setting to traceback?
Then we have a chance to repeat the issue.

Sebastian Koslowski

unread,
Mar 24, 2015, 8:10:25 AM3/24/15
to sy...@googlegroups.com
Sure:

[koslowski@host ~]$ PYTHONHASHSEED=123 ipython3
Python 3.3.2 (default, Dec  4 2014, 12:49:00) 
Type "copyright", "credits" or "license" for more information.

IPython 2.3.1 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: from sympy.stats import Normal, P

In [2]: N0 = Normal('N0', 0, 1)

In [3]: P(N0 <= 1.1)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-3-38fe444a5c96> in <module>()
----> 1 P(N0 <= 1.1)
TypeError: cannot determine truth value of
-oo < -inf

In [4]: P(N0 < 1.1)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-4-6d8c34e18be2> in <module>()
----> 1 P(N0 < 1.1)
TypeError: cannot determine truth value of
-oo < -inf


--
You received this message because you are subscribed to a topic in the Google Groups "sympy" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/sympy/X5-mfRAzQI4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to sympy+unsubscribe@googlegroups.com.
To post to this group, send email to sy...@googlegroups.com.
Visit this group at http://groups.google.com/group/sympy.
 To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/55114B65.9040500%40durchholz.org.

For more options, visit https://groups.google.com/d/optout.

Francesco Bonazzi

unread,
Mar 24, 2015, 8:13:30 AM3/24/15
to sy...@googlegroups.com

Can you try to see whether you get this bug by calling P( ... condition ... , evaluate=False) ?

I think this is a bug in the integration algorithm, no in the stats module.

Try this:

res = P(N0 <= 1.1001, evaluate=Fales)
res.doit()

and see whether the exception is raised by the first or the second step.

Sebastian Koslowski

unread,
Mar 24, 2015, 8:17:32 AM3/24/15
to sy...@googlegroups.com
On Tue, Mar 24, 2015 at 1:13 PM, Francesco Bonazzi <franz....@gmail.com> wrote:

Try this:

res = P(N0 <= 1.1001, evaluate=Fales)
res.doit()

and see whether the exception is raised by the first or the second step.

It is raised by the first step. 

Francesco Bonazzi

unread,
Mar 24, 2015, 9:26:40 AM3/24/15
to sy...@googlegroups.com


On Tuesday, March 24, 2015 at 1:10:25 PM UTC+1, Sebastian Koslowski wrote:
Sure:

[koslowski@host ~]$ PYTHONHASHSEED=123 ipython3

I've been able to reproduce this bug. Apparently it's related to Python 3.3 version. I don't get it on versions 2.7 and 3.4 of Python.

Francesco Bonazzi

unread,
Mar 24, 2015, 9:41:57 AM3/24/15
to sy...@googlegroups.com
Better reproduction of this bug:

In [1]: from sympy import *

In [2]: inf = Float('inf')

In [3]: oo < inf
Out[3]: False

In [4]: -oo < -inf
Out[4]: -oo < -inf

In [5]: bool(-oo < -inf)

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-5-a2165769cd19> in <module>()
----> 1 bool(-oo < -inf)

/usr/local/lib/python2.7/dist-packages/sympy/core/relational.pyc in __nonzero__(self)

    101
    102     def __nonzero__(self):
--> 103         raise TypeError("cannot determine truth value of\n%s" % self)
    104
    105     __bool__ = __nonzero__

TypeError: cannot determine truth value of
-oo < -inf


Now I get it on Python 2.7 as well, without that hashseed.

Sebastian Koslowski

unread,
Mar 25, 2015, 6:58:30 AM3/25/15
to sy...@googlegroups.com
Reply all
Reply to author
Forward
0 new messages