Warning: scipy.stats.distributions rvs broadcasting

12 views
Skip to first unread message

josef...@gmail.com

unread,
Nov 7, 2011, 11:50:23 AM11/7/11
to pystat...@googlegroups.com
Where's my noise?

I just got bitten again by http://projects.scipy.org/scipy/ticket/1544

scale is scalar, mu_true is array of loc

good:
y_obs = self.rvs(mu_true, scale=scale, size=nobs)

bad:
y_obs = self.rvs(mu_true, scale=scale)
this has no noise, one random draw gets shifted by the loc array

If the array is for the shape parameter and loc and scale are scalars,
then it's fine.

Josef

Wes McKinney

unread,
Nov 7, 2011, 11:52:21 AM11/7/11
to pystat...@googlegroups.com

Major pet peeve of mine. I've been bitten by this before too-- is it
an easy fix?

josef...@gmail.com

unread,
Nov 7, 2011, 12:17:44 PM11/7/11
to pystat...@googlegroups.com

You didn't open a ticket. Major pet peeves are supposed to be
reported, maybe they are bugs. :)

Not in 5 minutes, but a warning would be good.

The general solution might require quite a bit of shape checking to
see that things broadcast or don't broadcast correctly. But I haven't
looked at the details yet.
I don't know if anyone intentionally uses the current behavior.

Josef

Warren Weckesser

unread,
Nov 7, 2011, 12:27:33 PM11/7/11
to pystat...@googlegroups.com


I was just talking to Travis about this.  He suspects is was just an oversight.  We can fix this (I'll do it if no one beats me to it), but statsmodels will still have to work around the bug in the current releases of scipy.

Warren


 
Josef

josef...@gmail.com

unread,
Nov 7, 2011, 1:08:42 PM11/7/11
to pystat...@googlegroups.com

Thanks Warren. I'm glad if I don't have to beat you to it. I'm still
writing tests (with size=...) and bug hunting in GAM, and then I would
like to hunt for my own bugs (or write new buggy code so I can look
for them:).

Josef

>
> Warren
>
>
>
>>
>> Josef
>
>

josef...@gmail.com

unread,
Nov 23, 2011, 9:54:14 PM11/23/11
to pystat...@googlegroups.com

I found by chance how to create truncated normal random variables in
Greene, only to realize it is just the standard ppf transformation as
it is also used in scipy.stats.

After reinventing the wheel, I thought I also reinvent broadcasting
(so my time is not completely wasted).

http://groups.google.com/group/scipy-user/browse_thread/thread/f778f5ff4660e451?hl=en

(aside: this should be useful for simulating Probit models.)

Josef

>
> Josef
>
>>
>> Warren
>>
>>
>>
>>>
>>> Josef
>>
>>
>

Reply all
Reply to author
Forward
0 new messages