datetime.datetime to numpy.datetime64 automatic conversion in Pandas 0.10.1

[Chris Diehl]

Hi All, 

I just upgraded from Pandas 0.9.1 to 0.10.1 and had some code break. It looks like the issue is the following. I have a DataFrame with one column of datetime.datetime objects. 

I have one line of code that subsets the DataFrame to a smaller number of columns. Something of the form: df = df[[<col1>,<col2>,<col3>]]. That operation results in the column of datetime objects getting automatically converted to numpy.datetime64 objects. This causes code that follows to break since it's assuming datetime objects are available. Is there a way for me to avoid this automatic conversion? Under way conditions will this happen?

I appreciate any pointers you can offer. 

Thanks! Chris

[Goyo]

El jueves, 7 de febrero de 2013 02:39:44 UTC+1, Chris Diehl escribió:

Hi All, 

I just upgraded from Pandas 0.9.1 to 0.10.1 and had some code break. It looks like the issue is the following. I have a DataFrame with one column of datetime.datetime objects. 

I have one line of code that subsets the DataFrame to a smaller number of columns. Something of the form: df = df[[<col1>,<col2>,<col3>]]. That operation results in the column of datetime objects getting automatically converted to numpy.datetime64 objects. This causes code that follows to break since it's assuming datetime objects are available. Is there a way for me to avoid this automatic conversion?

AFAICT there's no way to avoid this. Try df['col'].astype(object) if you want a Series of datetime objects.

Goyo

[Jeff Reback]

FYI

https://github.com/pydata/pandas/pull/2752

upcoming in 0.10.2

I can be reached on my cell 917-971-6387

--
You received this message because you are subscribed to the Google Groups "PyData" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pydata+un...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Chris Diehl]

Roger that. Think I'll stick with 0.9.1 for the moment until 0.10.2 rolls out. Chris

[Jeff Reback]

as a workaround

df.convert_objects()

or explicitly casting will work as well

eg

Series(aseries,dtype='M8[ns]')

of course if you really want explict datetimes

just loop over the series

the reasons for these changes are to provide 'hopefully' more transparency in dealing with missing values (NaT)

I can be reached on my cell 917-971-6387

[Isaac Xin Pei]

Dear All,

I am searching for pandas datetime conversion error solutions but could not find a good solution. The issue I have is that when reading from Excel, or even CSV files, sometimes it seems that pandas will determine the type arbitarily?

I don't know why, but come of my dates columns are read as text (which is ok, that can be converted), while some others were read as datetime64 - that's real problematic, as I am stuck there - not able to convert it to some date type, or float ... easily ...

any suggestions?

Thanks,

Isaac