Pandas error - invalid value encountered

[Ben Gorman]

Installed a fresh copy of Anaconda, launched Spyder, then ran the following

import pandas as pd
import numpy as np

train = pd.read_csv('/Users/Ben/Documents/Kaggle/Titanic/train.csv')
train

which gave me the errors

//anaconda/lib/python3.4/site-packages/pandas/core/format.py:1969: RuntimeWarning: invalid value encountered in greater
  has_large_values = (abs_vals > 1e8).any()
//anaconda/lib/python3.4/site-packages/pandas/core/format.py:1970: RuntimeWarning: invalid value encountered in less
  has_small_values = ((abs_vals < 10 ** (-self.digits)) &
//anaconda/lib/python3.4/site-packages/pandas/core/format.py:1971: RuntimeWarning: invalid value encountered in greater
  (abs_vals > 0)).any()
(although the dataframe did print after these errors).  Why am I seeing this and what should I do?  I do not get these errors when I run the same code in an Ipython Notebook.

Also posted this on StackOverlow.

Thanks!

[Gonzalo Peña-Castellanos]

Hi Ben,

Is this happening only when ran from within spyder?, Have you tried running the same code in a bare python console?

Cheers

[Adrian Klaver]

On 05/28/2015 08:03 PM, Ben Gorman wrote:
> Installed a fresh copy of Anaconda, launched Spyder, then ran the following
>
> |import pandasas pd
> import numpyas np
>

> train= pd.read_csv('/Users/Ben/Documents/Kaggle/Titanic/train.csv')

> train|
>
> which gave me the errors
>
> |//anaconda/lib/python3.4/site-packages/pandas/core/format.py:1969: RuntimeWarning: invalid value encounteredin greater

> has_large_values= (abs_vals> 1e8).any()

> //anaconda/lib/python3.4/site-packages/pandas/core/format.py:1970: RuntimeWarning: invalid value encounteredin less

> has_small_values= ((abs_vals< 10 ** (-self.digits)) &
> //anaconda/lib/python3.4/site-packages/pandas/core/format.py:1971: RuntimeWarning: invalid value encounteredin greater

> (abs_vals> 0)).any()|
>
> |(although the dataframe did print after these errors). Why am I seeing
> this and what should I do? I do not get these errors when I run the same
> code in an Ipython Notebook.|

Are you using the same version of Pandas/numpy in the Spyder case as in
the IPython Notebook case.

> |
> |
> |Also posted this on StackOverlow
> <http://stackoverflow.com/questions/30519487/pandas-error-invalid-value-encountered>.|
> |
> |
> Thanks!
>
> --
> You received this message because you are subscribed to the Google
> Groups "spyder" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to spyderlib+...@googlegroups.com
> <mailto:spyderlib+...@googlegroups.com>.
> To post to this group, send email to spyd...@googlegroups.com
> <mailto:spyd...@googlegroups.com>.
> Visit this group at http://groups.google.com/group/spyderlib.
> For more options, visit https://groups.google.com/d/optout.


--
Adrian Klaver
adrian...@aklaver.com

[Ben Gorman]

Hey, thanks for y'all's help.  To answer your questions

1. I tested the same exact code in an IpythonNotebook as well as from the Terminal and I ONLY get the warnings when running it in Spyder (via the internal Ipython Console or the Python Console).

2. pd.__version__ gives me '0.15.2' and np.__version__ gives me '1.9.2' in all locations (terminal, ipython notebook, spyder)

I uninstalled my Anaconda package using sudo rm -rf ~/anaconda, resinstalled a fresh copy and I still get the warnings.

[Ben Gorman]

UPDATE after playing around with read_csv, it appears as though a NaN value for a float64 column is somehow triggering the warnings.  Would really appreciate an explanation of why and what (if anything) I should do.  Thanks

On Friday, May 29, 2015 at 10:35:02 AM UTC-5, Ben Gorman wrote:

[Adrian Klaver]

On 05/30/2015 09:16 AM, Ben Gorman wrote:
> Hey, thanks for y'all's help. To answer your questions
>
> 1. I tested the same exact code in an IpythonNotebook as well as from
> the Terminal and I ONLY get the warnings when running it in Spyder (via
> the internal Ipython Console or the Python Console).
>
> 2. pd.__version__ gives me '0.15.2' and np.__version__ gives me '1.9.2'
> in all locations (terminal, ipython notebook, spyder)
>
>
> I uninstalled my Anaconda package using sudo rm -rf ~/anaconda,
> resinstalled a fresh copy and I still get the warnings.

I tried to replicate on my machine using the dataset from your SO
question. I am on openSUSE 13.1 and using the following:

* Spyder Version: 2.3.3
* Python Version: 2.7.6
* Qt Version : 4.8.5, PyQt4 (API v2) 4.10.3 on Linux

## Optional dependencies

pyflakes >=0.5.0: 0.7.3 (OK)
pep8 >=0.6 : 1.5.7 (OK)
IPython >=1.0 : 2.3.0 (OK)
zmq >=2.1.11 : 14.5.0 (OK)
pygments >=1.6 : 1.6 (OK)
pandas >=0.13.1 : 0.15.2 (OK)
sphinx >=0.6.6 : 1.2.3 (OK)
psutil >=0.3 : 2.2.1 (OK)
jedi >=0.8.1 : 0.8.1-final0 (OK)
rope >=0.9.2 : 0.10.2 (OK)
matplotlib >=1.0: 1.3.1 (OK)
sympy >=0.7.0 : None (NOK)
pylint >=0.25 : 1.4.3 (OK)

np.__version__
Out[2]: '1.9.2'

I could not get the same error as you. All I can point you at is the
following:

Similar problem to yours-
https://github.com/pydata/pandas/issues/9950

The pandas bug fix-
https://github.com/pydata/pandas/issues/9764

The fix applied in pandas 0.16.1-
http://pandas.pydata.org/pandas-docs/stable/whatsnew.html#bug-fixes


If it is possible you may try upgrading to 0.16.1.

>
>
> On Friday, May 29, 2015 at 10:35:02 AM UTC-5, Ben Gorman wrote:
>
> Installed a fresh copy of Anaconda, launched Spyder, then ran the
> following
>

> |import pandasas pd
> import numpyas np
>

> train= pd.read_csv('/Users/Ben/Documents/Kaggle/Titanic/train.csv')

> train|
>
> which gave me the errors
>
> |//anaconda/lib/python3.4/site-packages/pandas/core/format.py:1969: RuntimeWarning: invalid value encounteredin greater

> has_large_values= (abs_vals> 1e8).any()

> //anaconda/lib/python3.4/site-packages/pandas/core/format.py:1970: RuntimeWarning: invalid value encounteredin less

> has_small_values= ((abs_vals< 10 ** (-self.digits)) &
> //anaconda/lib/python3.4/site-packages/pandas/core/format.py:1971: RuntimeWarning: invalid value encounteredin greater

> (abs_vals> 0)).any()|
>
> |(although the dataframe did print after these errors). Why am I
> seeing this and what should I do? I do not get these errors when I
> run the same code in an Ipython Notebook.|
> |
> |

[Ben Gorman]

Appreciate the help.  I updated Pandas to 0.16.1 but am still getting the warning.  Fortunately it hasn't caused any real issues for me thus far - it's just been an annoyance.

--
You received this message because you are subscribed to a topic in the Google Groups "spyder" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/spyderlib/wGak1ocs5eU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to spyderlib+...@googlegroups.com.
To post to this group, send email to spyd...@googlegroups.com.

[Adrian Klaver]

On 05/30/2015 04:05 PM, Ben Gorman wrote:
> Appreciate the help. I updated Pandas to 0.16.1 but am still getting
> the warning. Fortunately it hasn't caused any real issues for me thus
> far - it's just been an annoyance.
>

Since this seems to be a float issue, you might want to play around with
the float_precision argument to read_csv():

http://pandas.pydata.org/pandas-docs/stable/io.html#io-read-csv-table

float_precision : string, default None. Specifies which converter the C
engine should use for floating-point values. The options are None for
the ordinary converter, ‘high’ for the high-precision converter, and
‘round_trip’ for the round-trip converter.


--
Adrian Klaver
adrian...@aklaver.com