Date: Fri, 16 Nov 2012 16:13:19 -0800 (PST)
From: Gagi <dragol...@gmail.com>
To: pydata@googlegroups.com
Message-Id: <1c994509-39f8-4bc7-9531-8dc9baff5595@googlegroups.com>
In-Reply-To: <92fa933a-eb48-441d-b3c2-ed1cbb3df996@googlegroups.com>
References: <92fa933a-eb48-441d-b3c2-ed1cbb3df996@googlegroups.com>
Subject: Re: pandas.pivot_table indexing problem/bug:
MIME-Version: 1.0
Content-Type: multipart/mixed; 
	boundary="----=_Part_1019_27434916.1353111199105"

------=_Part_1019_27434916.1353111199105
Content-Type: multipart/alternative; 
	boundary="----=_Part_1020_27198020.1353111199105"

------=_Part_1020_27198020.1353111199105
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

And Here is the code that fails. Note that in the code below I am pivoting 
on combinations of 4 Columns A, B, C, D, and this fails for only 1000 rows, 
but the above code works on 1M rows.


import pandas as pd
import numpy as np

# Generate Long File & Test Pivot
NUM_ROWS = 1000
df = pd.DataFrame({'A' : np.random.randint(100, size=NUM_ROWS),
                                'B' : np.random.randint(300, 
size=NUM_ROWS), 
                                'C' : np.random.randint(-7, 7, 
size=NUM_ROWS), 
                                'D' : np.random.randint(-19,19, 
size=NUM_ROWS),
                                'E' : np.random.randint(3000, 
size=NUM_ROWS),
                                'F' : np.random.randn(NUM_ROWS)})

df_pivoted = df.pivot_table(rows=['A', 'B', 'C', 'D'], cols='E', values='F')
df_pivoted

------=_Part_1020_27198020.1353111199105
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: quoted-printable

And Here is the code that fails. Note that in the code below I am=20
pivoting on combinations of 4 Columns A, B, C, D, and this fails for=20
only 1000 rows, but the above code works on 1M rows.<div class=3D"im"><br><=
br>import pandas as pd<br>import numpy as np<br><br># Generate Long File &a=
mp; Test Pivot<br>NUM_ROWS =3D 1000<br></div><div class=3D"im">df =3D pd.Da=
taFrame({'A' : np.random.randint(100, size=3DNUM_ROWS),<br>&nbsp;&nbsp;&nbs=
p;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&=
nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbs=
p;&nbsp;&nbsp;<wbr>&nbsp; 'B' : np.random.randint(300, size=3DNUM_ROWS), <b=
r></div><div><div class=3D"im">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&n=
bsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp=
;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<wbr>&nbsp; 'C=
' : np.random.randint(-7, 7, size=3DNUM_ROWS), <br>&nbsp;&nbsp;&nbsp;&nbsp;=
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nb=
sp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;=
&nbsp;<wbr>&nbsp; 'D' : np.random.randint(-19,19, size=3DNUM_ROWS),<br>&nbs=
p;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&=
nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbs=
p;&nbsp;&nbsp;&nbsp;&nbsp;<wbr>&nbsp; 'E' : np.random.randint(3000, size=3D=
NUM_ROWS),<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&=
nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbs=
p;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<wbr>&nbsp; 'F' : np.random.ran=
dn(NUM_ROWS)})<br><br></div>df_pivoted =3D df.pivot_table(<span style=3D"ba=
ckground-color:rgb(255,255,0)">rows=3D['A', 'B', 'C', 'D'</span>], cols=3D'=
E', values=3D'F')<br>df_pivoted</div>
------=_Part_1020_27198020.1353111199105--

------=_Part_1019_27434916.1353111199105--