about concat in pandas : using row data to create new columns

[Abhishek Pratap]

Hey Guys

I want to do a custom concat i.e using the rows in a group by object
to create new cols.

Here is an contrived example:

#Input data frame
name age
foo 12
bar 14


df = pandas.DataFrame({ 'name':['foo','bar'],'age': [12,14] })


#expected output, a pandas data frame with four cols
foo 12 bar 14

PS: I am looking for an efficient solution as this would be applied to
a grouped pandas object containing 800k odd groupings.

Thanks!
-Abhi

[Wouter Overmeire]

2013/5/15 Abhishek Pratap <apr...@lbl.gov>

--
You received this message because you are subscribed to the Google Groups "PyData" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pydata+un...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

In [43]: pandas.DataFrame(df[['name', 'age']].values.reshape(1, 4))

Out[43]:

     0   1    2   3

0  foo  12  bar  14

 

[Abhishek Pratap]

Hi Wouter

I think I am seeing performance issues using reshape...see below

When I use the same methodology and apply to a grouped data frame the
performance takes a hit of about 3x.

The grouped data frame has 500K groups, each of which processed by a
sorting function and the final steps is to take first 3 rows and
convert them to a single line N col data frame which is integrated
with grouped by data frame.

The reshape and conversion back to a pandas data frame is taking 3x
more time..any way to do this faster ?

Sorry for no examples in this case.

[Wouter Overmeire]

2013/5/21 Abhishek Pratap <apr...@lbl.gov>

Sorry, but i can`t help i`m afraid. I don`t see to what you are comparing or how you process the GroupBy object.