TypeError: ufunc 'isnan' not supported ... and so on with pandas

12,392 views
Skip to first unread message

leitn...@googlemail.com

unread,
Mar 23, 2013, 11:23:50 AM3/23/13
to pyqt...@googlegroups.com
Hi together,

recently i started with pyqtgraph (great work! i tried some plotters to find a convient one).
The data i want to plot is stored in a pandas dataframe, from which i grab a column as series and add it as curve, here:

curve.setData( x=data.index,  y=data.values )

data is the series, where the index is the time in seconds and the values are - well - the corresponding values.
They look like this through a print:

The values: (--> y)

[ 1.04719755  1.37789151  1.48188333  1.86999563  1.82650736  2.24399475
  2.12269774  2.61799388  2.45436926  2.90888209  2.80499344  3.02076217
  2.53354246  2.61799388  2.3099946   2.61799388  2.24399475  2.45436926
  2.24399475  2.45436926  2.18166156  2.37999443  2.06683727  2.3099946
  2.12269774  2.18166156  2.06683727  2.18166156  1.96349541  2.12269774
  1.91560528  2.06683727  1.86999563  2.06683727  1.78499583  1.96349541
  1.74532925  1.91560528  1.70738731  1.78499583  1.63624617  1.74532925
  1.57079633  1.63624617  1.51038108  1.60285339  1.42799666  1.51038108
  1.37789151  1.42799666  1.28753797  1.33118333  1.18999722  1.22718463
  1.0758879   1.12199738  0.9817477   1.00692072  0.88246985  0.89249791
  0.72722052  0.70756591  0.54165391  0.47599889  0.0273087   1.45444104
  1.40249672  1.57079633  1.63624617  2.37999443  2.24399475  2.90888209
  2.70826953  3.27249235  3.14159265  3.56999165  3.14159265  3.02076217
  2.24399475  2.3099946   2.18166156  2.3099946   2.12269774  2.3099946
  2.06683727  2.24399475  2.01384144  2.24399475  1.96349541  2.12269774
  1.91560528  2.06683727  1.86999563  2.01384144  1.78499583  1.86999563
  1.70738731  1.86999563  1.63624617  1.78499583  1.60285339  1.74532925
  1.48188333  1.67105992  1.45444104  1.57079633  1.35413476  1.45444104
  1.28753797  1.37789151  1.18999722  1.28753797  1.12199738  1.20830487
  1.01999761  1.09083078  0.91325368  0.94626285  0.8014267   0.79333148
  0.67128048  0.63853509  0.48183936  0.39269908  0.07465762  1.24666375
  1.18999722  1.20830487  1.1061946   1.12199738  0.99417489  1.03341864
  0.87266463  0.89249791  0.72054877  0.72054877  0.59052493  0.5530973
  0.38689565  0.21877386  0.06184238  3.56999165  3.27249235  3.73999125
  3.14159265  3.56999165  3.14159265  3.92699082  3.14159265  3.27249235
  2.90888209  3.27249235  2.90888209  3.41477462  2.70826953  3.27249235
  2.80499344  3.14159265  2.80499344  3.14159265  2.80499344  3.02076217
  2.70826953  3.02076217  2.80499344  2.90888209  2.70826953  2.90888209
  2.61799388  2.80499344  2.61799388  2.80499344  2.53354246  2.80499344
  2.45436926  2.70826953  2.37999443  2.70826953  2.37999443  2.53354246
  2.37999443  2.53354246  2.24399475  2.53354246  2.24399475  2.45436926
  2.18166156  2.37999443  2.18166156  2.3099946   2.06683727  2.3099946
  1.96349541  2.3099946   1.91560528  2.12269774  1.91560528  2.06683727
  1.86999563  2.01384144  1.78499583  1.96349541  1.74532925  1.91560528
  1.70738731  1.78499583  1.60285339  1.74532925  1.5399964   1.67105992
  1.45444104  1.57079633  1.40249672  1.48188333  1.30899694  1.40249672
  1.22718463  1.30899694  1.13825821  1.18999722  1.03341864  1.0758879
  0.89249791  0.96962736  0.8014267   0.8014267   0.63338562  0.47890132
  0.25835466  0.14075236  2.45436926  3.14159265  2.70826953  3.27249235
  2.80499344  3.27249235  3.41477462  3.92699082  3.14159265  3.56999165
  3.14159265  3.27249235  3.02076217  3.41477462  3.02076217  3.27249235
  2.90888209  3.27249235  3.02076217  3.14159265  2.80499344  3.27249235
  2.80499344  3.14159265  2.80499344  3.14159265  2.70826953  3.02076217
  2.80499344  2.90888209  2.70826953  3.02076217  2.53354246  2.90888209
  2.61799388  2.80499344  2.53354246  2.80499344  2.45436926  2.80499344
  2.37999443  2.70826953  2.37999443  2.70826953  2.3099946   2.61799388
  2.3099946   2.53354246  2.24399475  2.45436926  2.24399475  2.37999443
  2.18166156  2.37999443  2.06683727  2.3099946   2.01384144  2.24399475
  1.96349541  2.18166156  1.91560528  2.12269774  1.86999563  2.06683727
  1.82650736  1.96349541  1.74532925  1.86999563  1.70738731  1.82650736
  1.60285339  1.74532925  1.48188333  1.70738731  1.45444104  1.57079633
  1.40249672  1.51038108  1.33118333  1.42799666  1.20830487  1.28753797
  1.1061946   1.22718463  1.01999761  1.09083078  0.94626285  0.9817477
  0.82673491  0.81812309  0.67128048  0.63853509  0.51333213  0.44123492
  0.34599038  3.02076217  3.27249235  3.73999125  3.14159265  3.56999165
  3.14159265  3.56999165  3.14159265  3.73999125  3.14159265  3.56999165
  3.14159265  3.27249235  3.02076217  3.41477462  3.02076217  3.27249235
  2.90888209  3.41477462  2.90888209  3.14159265  2.90888209  3.27249235
  2.80499344  3.14159265  2.80499344  3.02076217  2.80499344  3.02076217
  2.70826953  3.02076217  2.70826953  2.90888209  2.70826953  2.80499344
  2.61799388  2.80499344  2.61799388  2.80499344  2.53354246  2.70826953
  2.45436926  2.70826953  2.45436926  2.61799388  2.37999443  2.53354246
  2.37999443  2.53354246  2.24399475  2.53354246  2.24399475  2.45436926
  2.18166156  2.37999443  2.18166156  2.3099946   2.06683727  2.3099946
  2.01384144  2.18166156  1.96349541  2.12269774  1.86999563  2.12269774
  1.82650736  1.96349541  1.78499583  1.96349541  1.70738731  1.86999563
  1.63624617  1.78499583  1.57079633  1.67105992  1.51038108  1.60285339
  1.42799666  1.51038108  1.35413476  1.42799666  1.24666375  1.35413476
  1.17223606  1.24666375  1.09083078  1.13825821  0.99417489  1.04719755
  0.89249791  0.90275651  0.75519054  0.74094166  0.59499861  0.53067443
  0.37578859]


The index (--> x)

array([0.7317, 0.7374, 0.7427, 0.7469, 0.7512, 0.7547, 0.7584, 0.7614,
       0.7646, 0.7673, 0.7701, 0.7727, 0.7758, 0.7788, 0.7822, 0.7852,
       0.7887, 0.7919, 0.7954, 0.7986, 0.8022, 0.8055, 0.8093, 0.8127,
       0.8164, 0.82, 0.8238, 0.8274, 0.8314, 0.8351, 0.8392, 0.843, 0.8472,
       0.851, 0.8554, 0.8594, 0.8639, 0.868, 0.8726, 0.877, 0.8818, 0.8863,
       0.8913, 0.8961, 0.9013, 0.9062, 0.9117, 0.9169, 0.9226, 0.9281,
       0.9342, 0.9401, 0.9467, 0.9531, 0.9604, 0.9674, 0.9754, 0.9832,
       0.9921, 1.0009, 1.0117, 1.0228, 1.0373, 1.0538, 1.3414, 1.3468,
       1.3524, 1.3574, 1.3622, 1.3655, 1.369, 1.3717, 1.3746, 1.377,
       1.3795, 1.3817, 1.3842, 1.3868, 1.3903, 1.3937, 1.3973, 1.4007,
       1.4044, 1.4078, 1.4116, 1.4151, 1.419, 1.4225, 1.4265, 1.4302,
       1.4343, 1.4381, 1.4423, 1.4462, 1.4506, 1.4548, 1.4594, 1.4636,
       1.4684, 1.4728, 1.4777, 1.4822, 1.4875, 1.4922, 1.4976, 1.5026,
       1.5084, 1.5138, 1.5199, 1.5256, 1.5322, 1.5383, 1.5453, 1.5518,
       1.5595, 1.5667, 1.5753, 1.5836, 1.5934, 1.6033, 1.615, 1.6273,
       1.6436, 1.6636, 1.7688, 1.7751, 1.7817, 1.7882, 1.7953, 1.8023,
       1.8102, 1.8178, 1.8268, 1.8356, 1.8465, 1.8574, 1.8707, 1.8849,
       1.9052, 1.9411, 2.0681, 2.0703, 2.0727, 2.0748, 2.0773, 2.0795,
       2.082, 2.084, 2.0865, 2.0889, 2.0916, 2.094, 2.0967, 2.099, 2.1019,
       2.1043, 2.1071, 2.1096, 2.1124, 2.1149, 2.1177, 2.1203, 2.1232,
       2.1258, 2.1286, 2.1313, 2.1342, 2.1369, 2.1399, 2.1427, 2.1457,
       2.1485, 2.1516, 2.1544, 2.1576, 2.1605, 2.1638, 2.1667, 2.17,
       2.1731, 2.1764, 2.1795, 2.183, 2.1861, 2.1896, 2.1928, 2.1964,
       2.1997, 2.2033, 2.2067, 2.2105, 2.2139, 2.2179, 2.2213, 2.2254,
       2.2291, 2.2332, 2.237, 2.2412, 2.2451, 2.2495, 2.2535, 2.258,
       2.2621, 2.2667, 2.2711, 2.276, 2.2805, 2.2856, 2.2903, 2.2957,
       2.3007, 2.3063, 2.3116, 2.3176, 2.3232, 2.3296, 2.3356, 2.3425,
       2.3491, 2.3567, 2.364, 2.3728, 2.3809, 2.3907, 2.4005, 2.4129,
       2.4293, 2.4597, 2.5155, 2.5187, 2.5212, 2.5241, 2.5265, 2.5293,
       2.5317, 2.534, 2.536, 2.5385, 2.5407, 2.5432, 2.5456, 2.5482,
       2.5505, 2.5531, 2.5555, 2.5582, 2.5606, 2.5632, 2.5657, 2.5685,
       2.5709, 2.5737, 2.5762, 2.579, 2.5815, 2.5844, 2.587, 2.5898,
       2.5925, 2.5954, 2.598, 2.6011, 2.6038, 2.6068, 2.6096, 2.6127,
       2.6155, 2.6187, 2.6215, 2.6248, 2.6277, 2.631, 2.6339, 2.6373,
       2.6403, 2.6437, 2.6468, 2.6503, 2.6535, 2.657, 2.6603, 2.6639,
       2.6672, 2.671, 2.6744, 2.6783, 2.6818, 2.6858, 2.6894, 2.6935,
       2.6972, 2.7014, 2.7052, 2.7095, 2.7135, 2.718, 2.7222, 2.7268,
       2.7311, 2.736, 2.7405, 2.7458, 2.7504, 2.7558, 2.7608, 2.7664,
       2.7716, 2.7775, 2.783, 2.7895, 2.7956, 2.8027, 2.8091, 2.8168,
       2.824, 2.8323, 2.8403, 2.8498, 2.8594, 2.8711, 2.8834, 2.8987,
       2.9165, 2.9392, 2.9418, 2.9442, 2.9463, 2.9488, 2.951, 2.9535,
       2.9557, 2.9582, 2.9603, 2.9628, 2.965, 2.9675, 2.9699, 2.9725,
       2.9748, 2.9774, 2.9798, 2.9825, 2.9848, 2.9875, 2.99, 2.9927,
       2.9951, 2.9979, 3.0004, 3.0032, 3.0058, 3.0086, 3.0112, 3.0141,
       3.0167, 3.0196, 3.0223, 3.0252, 3.028, 3.031, 3.0338, 3.0368,
       3.0396, 3.0427, 3.0456, 3.0488, 3.0517, 3.0549, 3.0579, 3.0612,
       3.0643, 3.0676, 3.0707, 3.0742, 3.0773, 3.0808, 3.084, 3.0876,
       3.0909, 3.0945, 3.0979, 3.1017, 3.1051, 3.109, 3.1126, 3.1166,
       3.1203, 3.1245, 3.1282, 3.1325, 3.1365, 3.1409, 3.1449, 3.1495,
       3.1537, 3.1585, 3.1629, 3.1679, 3.1726, 3.1778, 3.1827, 3.1882,
       3.1934, 3.1992, 3.2047, 3.211, 3.2168, 3.2235, 3.2298, 3.237,
       3.2439, 3.2518, 3.2593, 3.2681, 3.2768, 3.2872, 3.2978, 3.311,
       3.3258, 3.3467], dtype=object)

Just passing the values does a fine (but yet not timescaled) plot. Passing also the index as x does this:

Traceback (most recent call last):
  File "Test-qtpygraph.py", line 98, in <module>
    plt = Plotter(mer.outm)
  File "Test-qtpygraph.py", line 42, in __init__
    self.addSinglePlot( port )
  File "Test-qtpygraph.py", line 56, in addSinglePlot
    self.addCurve(plotWid, self.mdata[port] )
  File "Test-qtpygraph.py", line 70, in addCurve
    curve.setData( x=np.asarray(data.index),  y=data.values )
  File "/home/ulrich/Dokumente/Code/duckDAQ/pyqtgraph/graphicsItems/PlotDataItem.py", line 372, in setData
    self.updateItems()
  File "/home/ulrich/Dokumente/Code/duckDAQ/pyqtgraph/graphicsItems/PlotDataItem.py", line 396, in updateItems
    x,y = self.getData()
  File "/home/ulrich/Dokumente/Code/duckDAQ/pyqtgraph/graphicsItems/PlotDataItem.py", line 416, in getData
    nanMask = np.isnan(self.xData) | np.isnan(self.yData) | np.isinf(self.xData) | np.isinf(self.yData)
TypeError: ufunc 'isnan' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule 'safe'


But as shown above, there is no NaN in the array. I dont understand that error, can someboy help me out?
Thank you,

Ulrich


Sourcecode is below:


# -*- coding: utf-8 -*-

from PySide import QtGui, QtCore
from sys import argv
import numpy as np
import pyqtgraph as pg
from Measurement import *
from Filter import *
import pandas as pd
from Device import *


class Plotter(QtGui.QMainWindow):
    """
    def update(self):
        curve.setData(data[ptr%10])
       
        if ptr == 0:
            plot.enableAutoRange('xy', False)  ## stop auto-scaling after the first data set is plotted
        ptr = ptr+1
    """
   
    def __init__(self, inm):
        QtGui.QMainWindow.__init__(self)
       
        self.inMeas = inm
       
        tmpmdata = self.inMeas.data_ndarray()
        tmpmdataT = tmpmdata.transpose()
        idx=tmpmdataT[0].transpose()
        data=tmpmdataT[1:].transpose()

        self.mdata = pd.DataFrame( data, index=idx, columns=self.inMeas.ports)
       
        self.resize(1000,600)
        self.setWindowTitle('Plotter')
        self.tabWid = QtGui.QTabWidget(parent=self)
        self.tabWid.setTabsClosable(True)
        self.setCentralWidget(self.tabWid)
      
        for port in self.mdata.columns:
            self.addSinglePlot( port )
       
        self.show()

        """
        timer = QtCore.QTimer()
        timer.timeout.connect(update)
        timer.start(500)
        """

    def addSinglePlot(self, port):
        plotWid = pg.PlotWidget(parent=self.tabWid, name=port)
        self.tabWid.addTab(plotWid, port)

        self.addCurve(plotWid, self.mdata[port] )

    def addCurve(self, plotWid, data, pen="r"):
        curve = plotWid.plot(pen=pen)
       
        """
        pd.set_printoptions(max_rows='nan')
        dd = np.vstack( (data.values, data.index) )
        print dd.transpose()
        curve.setData( dd.transpose() )
        """
        print data.values
        print data.index
       
        curve.setData( x=data.index,  y=data.values )
       
        return curve
       

## Start Qt event loop unless running in interactive mode or using pyside.
if __name__ == '__main__':
    import sys
   
    m = Measurement()
    m.data_csv_read("dwheel10000.csv")
 
    cs = Channel_Selector(m, "AIN1")
    cs.start_block()

    sw = SpikeWheel(cs.outm)
    sw.start_block()

    sp = ChannelSplitter(sw.outm)
    sp.start_block()

    mer = ChannelMerger( [sp.outm[1], sp.outm[0], sp.outm[2] ] )
    mer.start_block()
    print mer.outm.queue.get()
    print mer.outm.queue.get()

    app = QtGui.QApplication(argv)

    plt = Plotter(mer.outm)
   
    exit( app.exec_() )






Luke Campagnola

unread,
Mar 23, 2013, 11:47:17 AM3/23/13
to pyqt...@googlegroups.com
On Sat, Mar 23, 2013 at 11:23 AM, <leitn...@googlemail.com> wrote:
Hi together,

recently i started with pyqtgraph (great work! i tried some plotters to find a convient one).
The data i want to plot is stored in a pandas dataframe, from which i grab a column as series and add it as curve, here:

curve.setData( x=data.index,  y=data.values )

data is the series, where the index is the time in seconds and the values are - well - the corresponding values.
[ snip ] 
Just passing the values does a fine (but yet not timescaled) plot. Passing also the index as x does this:

Traceback (most recent call last):
  File "Test-qtpygraph.py", line 98, in <module>
    plt = Plotter(mer.outm)
  File "Test-qtpygraph.py", line 42, in __init__
    self.addSinglePlot( port )
  File "Test-qtpygraph.py", line 56, in addSinglePlot
    self.addCurve(plotWid, self.mdata[port] )
  File "Test-qtpygraph.py", line 70, in addCurve
    curve.setData( x=np.asarray(data.index),  y=data.values )
  File "/home/ulrich/Dokumente/Code/duckDAQ/pyqtgraph/graphicsItems/PlotDataItem.py", line 372, in setData
    self.updateItems()
  File "/home/ulrich/Dokumente/Code/duckDAQ/pyqtgraph/graphicsItems/PlotDataItem.py", line 396, in updateItems
    x,y = self.getData()
  File "/home/ulrich/Dokumente/Code/duckDAQ/pyqtgraph/graphicsItems/PlotDataItem.py", line 416, in getData
    nanMask = np.isnan(self.xData) | np.isnan(self.yData) | np.isinf(self.xData) | np.isinf(self.yData)
TypeError: ufunc 'isnan' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule 'safe'


But as shown above, there is no NaN in the array. I dont understand that error, can someboy help me out?

Hi Ulrich,
Pyqtgraph plot curves check all data for NaNs before plotting to avoid generating errors later on. They use numpy.isnan to perform this check, but that function only works with arrays of specific dtype. In your case, the index array has dtype=object, which is causing the isnan check to fail. I would recommend converting this to a float array before plotting.

Luke

leitn...@googlemail.com

unread,
Mar 23, 2013, 1:45:29 PM3/23/13
to pyqt...@googlegroups.com
Thank you very much, this was quick und your hint did the trick. Pandas seems to store the index generally as object.
Interestingly, a coversion by mdata.index.astype(np.float64) does not work, it has to be

xdata = np.array( mdata.indey, dtype=float64)

if someone should read this in future ...

Ciao!

Dave McDaniel

unread,
Aug 7, 2013, 3:43:53 PM8/7/13
to pyqt...@googlegroups.com
Thanks, glad to find this.  I just ran into this issue. 

I agree its a little odd that:
mdata.index.astype(float)

does not work, but accessing the values property does work:

mdata.index.values.astype(float)
Message has been deleted

Jaco du Toit

unread,
Apr 2, 2015, 6:26:16 PM4/2/15
to pyqt...@googlegroups.com
Hi Dave,

Can you please share some code showing how to plot a pandas dataframe with pyqtgraph using datetime in the x axis? I am finding it impossible to do...

Kind regards,

Jaco
Reply all
Reply to author
Forward
0 new messages