The standard errors example

[Tyler Rinker]

Originally I sent this as an email (containing 2 questions) directly to Hadley who requested I send them to the ggplot help list as separate questions (this is question 1):


I am asked to do bar plots for people pretty often using count data (my field is literacy so counting is a fairly common augment to qualitative analysis).  I feel funny (as in slightly unethical) about just providing count data with no error bars but saw here http://had.co.nz/ggplot2/docs/geom_errorbar.html you use error bars on what appears to be count data (very first example; 2nd figure).  That inspired me to learn how to calculate standard errors on count data which some kind folks at talkstats.com told me I could do (doesn't necessarily make sense to do so) so using the Poisson distribution.  So I created a function to calculate SE for count data that also gives me confidence intervals (also I know people use +/- 1  SE for the error bars as well but that's pretty easy to calculate as you did in your example).  Here's that function in a .txt script from my drop box (
http://dl.dropbox.com/u/61803503/SEerror.txt ).


When I try it on your data set, given in the link to the ggplot2 site above, I get very different standard errors (
http://dl.dropbox.com/u/61803503/SEerror.txt ).  Can you tell me if you just randomly made up the standard errors on your data set for demonstration purposes or if they are authentic?  If they're authentic may I ask how you calculated them?

Thanks 
Tyler Rinker

[Ben Bolker]

On 12-04-30 04:37 PM, Tyler Rinker wrote:
>
> Originally I sent this as an email (containing 2 questions)
> directly to Hadley who requested I send them to the ggplot help
> list as separate questions (this is question 1):
>
>
> I am asked to do bar plots for people pretty often using count
> data (my field is literacy so counting is a fairly common augment
> to qualitative analysis). I feel funny (as in slightly unethical)
> about just providing count data with no error bars but saw here
> http://had.co.nz/ggplot2/docs/geom_errorbar.html you use error
> bars on what appears to be count data (very first example; 2nd
> figure). That inspired me to learn how to calculate standard errors
> on count data which some kind folks at talkstats.com told me I
> could do (doesn't necessarily make sense to do so) so using the
> Poisson distribution. So I created a function to calculate SE for
> count data that also gives me confidence intervals (also I know
> people use +/- 1 SE for the error bars as well but that's pretty
> easy to calculate as you did in your example). Here's that
> function in a .txt script from my drop box (
> http://dl.dropbox.com/u/61803503/SEerror.txt ).

It occurs to me to wonder offhand if calculating means and +/-1.96
standard errors on the square-root scale and then back-transforming
might give similar (or even exact??) answers as fitting a Poisson
model for a single category and finding (back-transformed) CI. (The
guess is based on the fact that the Poisson family uses var=mean as a
variance function ...)
For binomial models there are some fairly detailed developments of
exact small-sample confidence intervals (see e.g Agresti _Categorical
Data Analysis_ pp. 18-21); http://www.jstor.org/stable/2685469 .
epitools:::binom.exact() implements a variety of these recipes.

For Poisson, I would proceed based on the information here:
http://www.math.mcmaster.ca/peter/s743/poissonalpha.html
http://www.jstor.org/stable/10.2307/3314762
Googling "poisson confidence intervals" gives lots of good stuff ...

These are all for the case where the data are known to be
Poisson-distributed. In fact, if you have multiple counts per
category then you don't have to assume var=mean, you can allow for
overdispersion ...

[Ben Bolker]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 12-04-30 04:37 PM, Tyler Rinker wrote:
>

PS

http://biostats.bepress.com/ucbbiostat/paper242/
http://www.tandfonline.com/doi/abs/10.1080/00949650701727352

based on googling "negative binomial confidence intervals" ...

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJPnv3fAAoJED2whTVMEyK9YDoIAKoflwW5f6SmaQoSeCjCFY/j
nzmUih5A/m8tmKCjFSQKA91j4k5AGCOwzHiUpLKjqhcfS3Gx8nLo3IRpyESy352F
fVF83PF48GAWwy+g/VLZSYQBR065kDPIe8xm5pN3ZCYUI6V3og+H1S9E7w/bLG49
soSYqFg+Bid7q0lg40GSynI/8cFDKM+7F7L4aSLIXiBw7toS3I51OIFfs4NYDhtp
jqBBz76Nlr6KWLg3mkLqxiHX/hV6dltxM7+BFSovhymBkpn0rCHvM0yrW0Pp83nz
81cOVwV0u8glz5co8PYZIfXdJlyHLj3oducz8AbNclGF/xLo/Y61ggW9mVMAthY=
=s+Xz
-----END PGP SIGNATURE-----