MultiComparison w/ Pandas Series

10 views
Skip to first unread message

Javier Burroni

unread,
May 9, 2013, 1:51:27 PM5/9/13
to pystat...@googlegroups.com
hi, 
I wanted to perform MultiComparison using pandas series but it fails.
mod = MultiComparison(x, groups)
this statement never ends (or it doesn't in few minutes).
But, if I do this 
mod = MultiComparison(x.values, group.values)
it works perfectly.

Apparently the issue is with unique, inv = np.unique(groups, return_inverse=True) which gives a different value for unique as using unique = np.unique(groups).

I have no created an issue as I couldn't make a synthetic example. Also, I don't know to whom report. Most probably this is not the place and it should be reported to pandas, but as this affects pystatsmodels behavior, I'm writing this email

results:

In [113]: np.unique(groups.values, return_inverse=True)

Out[113]: A

(array([u'A', u'B', u'C'], dtype=object),

 array([2, 0, 0, ..., 2, 2, 1]))


In [114]: np.unique(role, return_inverse=True)

Out[114]: 

(role

3641           NaN

59108          NaN

43653          NaN

1581           NaN

75610     C

55740          NaN

16712        A

75614          NaN

102468         NaN

55742          NaN

102470         NaN

30488          NaN

102472    C

30489          NaN

4962           NaN

...

38778         NaN

38779       A

38781         NaN

38782    C

38784         NaN

38785         NaN

38786       A

38787         NaN

38789         NaN

38790         NaN

38791         NaN

38792         NaN

38793         NaN

38794    B

38795         NaN

Name: role, Length: 105590,

 array([68079, 21442,  5820, ..., 77651, 77652, 24166]))



regards
jb
  


--
" To be is to do " ( Socrates )
" To be or not to be " ( Shakespeare )
" To do is to be " ( Sartre )
" Do be do be do " ( Sinatra )

josef...@gmail.com

unread,
May 9, 2013, 2:18:49 PM5/9/13
to pystat...@googlegroups.com
On Thu, May 9, 2013 at 1:51 PM, Javier Burroni <javier....@gmail.com> wrote:
> hi,
> I wanted to perform MultiComparison using pandas series but it fails.
> mod = MultiComparison(x, groups)
> this statement never ends (or it doesn't in few minutes).
> But, if I do this
> mod = MultiComparison(x.values, group.values)
> it works perfectly.
>
> Apparently the issue is with unique, inv = np.unique(groups,
> return_inverse=True) which gives a different value for unique as using
> unique = np.unique(groups).
>
> I have no created an issue as I couldn't make a synthetic example. Also, I
> don't know to whom report. Most probably this is not the place and it should
> be reported to pandas, but as this affects pystatsmodels behavior, I'm
> writing this email

It's the right place to report, you could open an issue and just copy
your message in the description.

The class doesn't convert the data to arrays, that's a bug in MultiComparison.

We might want to use more information that we can get from a pandas
data structure, but this was written before pandas or without pandas
in mind. (If groups are a pandas factor, then we don't need to find
the levels again.)

related for weightstats https://github.com/statsmodels/statsmodels/issues/771

(A warning: except for tukeyhsd and it's supporting code, there is not
much test coverage for that module. 23% only)

Thanks for reporting

Josef

Javier Burroni

unread,
May 9, 2013, 2:28:21 PM5/9/13
to pystat...@googlegroups.com
issue submitted. Thank you for the (fast) feedback
Reply all
Reply to author
Forward
0 new messages