> Erik, I agree that some of the methods I mention for "Poisson count statistics" at
> might not be general enough for the astropy core (like Feldman&Cousins), but as Tom pointed out, some (probably are.
> I think it will be easier to discuss what belongs in the core after I implement these functions,
> they will all be standalone and easy to put in astropy.stats or an affiliated package.
I see your point that we don't want to be copying a lot of
functionality that we can find just as well elsewhere. (thanks for
pointing out the PCA stuff in scikits-learn... I hadn't seen that
That said, I guess my attitude for astropy.stats is not just that the
*functionality* be astronomy-specific, but also the language and
conventions. That is, if I, as an average astronomer, look at a
dataset and say "I want to do PCA on this", I shouldn't necessarily
have to undertand how to install and use scikits-learn. But I admit
that argument could be made on just about anything, so maybe you're
right this shouldn't be included unless/until it's needed in other
parts of astropy.
Anyone else have thoughts on this? Is PCA familiar to enough
astronomers that it should be included here?
> Question to non-photon-counting (e.g. radio, optical … i.e. not Poisson stats as for X-ray or gamma-ray) astronomers:
> Are there some general methods you use to compute errors, significance, sensitivity, upper limits?
> Can you implement them in one line using numpy and scipy.stats or would it be worth coding them up in astropy.stats?
Arguably optical/NIR is still in the photon-counting regime, even if
we're sometimes too lazy to do it properly :)
There are some standard sensitivity/SNR calculations relevant for CCDs
that I imagine would be useful both in imaging and spectroscopic
situations. But it's not clear what goes in astropy.stats vs
photutils (or similar).
Also, there's completeness calculations pertaining to redshift
surveys. But a lot of that will probably use similar methods as the
X-ray/gamma Poisson stuff.
>> It's worth noting that the initial functions that Christoph proposed
>> are really just source significance and sensitivity in the Poisson (as
>> opposed to Gaussian) regime. They are not highly specialized, and
>> have been around for 30 years, so there is not a big worry about the
>> release cycle here. Certainly the language used in the initial API
>> does make them look like gamma-ray routines, but Christoph already
>> agreed that this would be changed. Poisson and low-count statistics
>> are obviously useful throughout astronomy, particularly in analyzing
>> sample results, so we should not consider them as niche-specific.
Fair enough - it may be fine once the code is all laid out. I just
wanted to point out that astropy.stats is an area where I could easily
see out-of-control feature creep if we aren't a bit cautious.
>> - Tom
>>> I'd still love to see this wiki page get used to "brainstorm," but
>>> just keep in mind that we might not want it all inthe core when it
>>> actually starts getting coded up.
>>> On Tue, Jan 8, 2013 at 2:43 AM, Christoph Deil
>>>> Dear all,
>>>> the astropy.stats package is planned to hold statistical functions or algorithms used in astronomy and astropy.
>>>> At the moment it only contains one function: sigmaclip.
>>>> I would like to add some commonly used methods in X- and gamma-ray astronomy for Poisson count statistics.
>>>> Steve Crawford wants to add methods for robust statistics.
>>>> What other common astronomy statistical methods do we want (that are not already available in numpy, scipy, or maybe scikit-learn, scikit-image, statsmodels)?
>>>> Please add to the whishlist on this wiki page:
>>>> Or simply reply here if you prefer.
>>>> Fell free to propose methods even if you don't have time to implement them yourself.
>>>> Once we know what we want, the next step will be an API proposal.
>>>> Best wishes,