ggplot2 feature request: simple stats

221 views
Skip to first unread message

Alex@775

unread,
Mar 16, 2012, 2:20:47 AM3/16/12
to ggplot2
Hi,

stat_summary allows stats per x
stat_quantile allows stats per graph with some x correlation

I'd like to see a stat_group or similar that allows simple stat
calculation per group.

ggplot() + stat_group(aes(group=cyl,x=mean(mpg),y=max(..y..)))

Alternativelty, I'd like to see function calculations in aes be
applied per group:

for instance, I want to be able to plot a mean hline and vline on a
scatter plot:

> mc=mutate(mtcars,cyl=factor(cyl)

> ggplot(data=mc,x=cyl,y=mpg) + geom_point() +
> geom_hline(aes(y=mean(mpg),group=cyl))

another for instance, if I have grouped data, I might want to
summarise and add a text label for each, in an appropriate location:

> qplot(data=data.frame(x=rnorm(100),cat=sample(letters[1:4],
100,T)),
x=x)+geom_text(aes(group=cat,y=10,label=10,x=mean(x,y=max(x))))

Where x=mean is calculated per group. This does not currently
operate like this. It's odd that I can put a trendline or smoothed
kernel line but not a mean line.

-Alex

Jean-Olivier Irisson

unread,
Mar 16, 2012, 8:41:16 AM3/16/12
to Alex@775, ggplot2
On 2012-Mar-16, at 07:20 , Alex@775 wrote:
>
> stat_summary allows stats per x
> stat_quantile allows stats per graph with some x correlation
>
> I'd like to see a stat_group or similar that allows simple stat
> calculation per group.
>
> ggplot() + stat_group(aes(group=cyl,x=mean(mpg),y=max(..y..)))

I thought this was already the case: if you use stat_summary and specify a colour or a group to aes, it should compute the summary per group (stat_qq does at least).

> Alternativelty, I'd like to see function calculations in aes be
> applied per group:
>
> for instance, I want to be able to plot a mean hline and vline on a
> scatter plot:
>
>> mc=mutate(mtcars,cyl=factor(cyl)
>
>> ggplot(data=mc,x=cyl,y=mpg) + geom_point() +
>> geom_hline(aes(y=mean(mpg),group=cyl))
>
> another for instance, if I have grouped data, I might want to
> summarise and add a text label for each, in an appropriate location:
>
>> qplot(data=data.frame(x=rnorm(100),cat=sample(letters[1:4],
> 100,T)),
> x=x)+geom_text(aes(group=cat,y=10,label=10,x=mean(x,y=max(x))))
>
> Where x=mean is calculated per group.

The answer is probably: compute your summaries first and plot afterwards. plyr is there to help you compute such per-group summaries in one line. ggplot can't do everything.

> This does not currently
> operate like this. It's odd that I can put a trendline or smoothed
> kernel line but not a mean line.

You can in those instances because the data is fed through a statistic which computes things per group (hence my remark above). In the syntax you present here, the data is fed to stat_identify which does not compute anything (and actually, the data fed to stat_identify is probably the result of the computation specified in the aes call already).

Jean-Olivier Irisson
---
Observatoire Océanologique
Station Zoologique, B.P. 28, Chemin du Lazaret
06230 Villefranche-sur-Mer
Tel: +33 04 93 76 38 04
Mob: +33 06 21 05 19 90
http://jo.irisson.com/

Alex Brown

unread,
Mar 16, 2012, 10:20:29 AM3/16/12
to Jean-Olivier Irisson, ggplot2

-Alex Brown

On Mar 16, 2012, at 5:41 AM, Jean-Olivier Irisson <iri...@normalesup.org> wrote:

> On 2012-Mar-16, at 07:20 , Alex@775 wrote:
>>
>>
>> I'd like to see a stat_group or similar that allows simple stat
>> calculation per group.
>

> I thought this was already the case: if you use stat_summary and specify a colour or a group to aes, it should compute the summary per group (stat_qq does at least).

Stat summary computes stats per group per unique x. I want just per group.


>
> The answer is probably: compute your summaries first and plot afterwards. plyr is there to help you compute such per-group summaries in one line. ggplot can't do everything.
>

That's what I do today. However, I say again, it's odd ggplot is happy to compute smoothing kernels and per x stats but not something so clean amd simple as mean per group.

I want convenience methods sometimes. And it would be so easy.

Also there's a problem in .89 at least with pre-computed stats, if nrow subset data for one facet is 0, ggplot fails.

-Alex

Kohske Takahashi

unread,
Mar 16, 2012, 10:25:09 AM3/16/12
to Alex Brown, Jean-Olivier Irisson, ggplot2
Is this what you want?

ggplot(mtcars, aes(x = cyl, y=mpg)) + geom_point() +
geom_hline(aes(yintercept=mean(mpg)))


2012年3月16日23:20 Alex Brown <alexb...@gmail.com>:

> --
> You received this message because you are subscribed to the ggplot2 mailing list.
> Please provide a reproducible example: http://gist.github.com/270442
>
> To post: email ggp...@googlegroups.com
> To unsubscribe: email ggplot2+u...@googlegroups.com
> More options: http://groups.google.com/group/ggplot2

--
--
Kohske Takahashi <takahash...@gmail.com>

Research Center for Advanced Science and Technology,
The University of Tokyo, Japan.
http://www.fennel.rcast.u-tokyo.ac.jp/profilee_ktakahashi.html

Jean-Olivier Irisson

unread,
Mar 16, 2012, 12:21:45 PM3/16/12
to Alex Brown, ggplot2
On 2012-Mar-16, at 15:20 , Alex Brown wrote:
>
>>>
>>> I'd like to see a stat_group or similar that allows simple stat
>>> calculation per group.
>>
>> I thought this was already the case: if you use stat_summary and specify a colour or a group to aes, it should compute the summary per group (stat_qq does at least).
>
> Stat summary computes stats per group per unique x. I want just per group.
>>
>> The answer is probably: compute your summaries first and plot afterwards. plyr is there to help you compute such per-group summaries in one line. ggplot can't do everything.
>
> That's what I do today. However, I say again, it's odd ggplot is happy to compute smoothing kernels and per x stats but not something so clean amd simple as mean per group.

That made me think that this:

library("ggplot2")
d <- data.frame(x=runif(100), y=c(runif(50), 1.5*runif(50)), a=rep(c("a", "b"), each=50))
ggplot(d, aes(x=x, y=y, colour=a)) + geom_point() + geom_smooth(se=FALSE, method="lm", formula=y~1)

does what you want (one mean per group)… but that's probably cheating ;)
NB: lm~1 is a linear model without slope.

Then I understood your point when trying with a continuous x. stat_summary is obviously meant for discrete x and I don't know how to overcome that, sorry.

Alex@775

unread,
Mar 16, 2012, 1:06:43 PM3/16/12
to ggplot2

On Mar 16, 7:25 am, Kohske Takahashi <takahashi.koh...@gmail.com>
wrote:
> Is this what you want?
>
> ggplot(mtcars, aes(x = cyl, y=mpg)) + geom_point() +
> geom_hline(aes(yintercept=mean(mpg)))
>

No, this is what I want:

NOTE: I have simplified this case to the point where stat_summary
would work. see end for why that's not suitable.

> mtcars=mutate(mtcars,cyl=factor(cyl))
> q = ggplot(mtcars, aes(x = cyl, y=mpg)) + geom_point()
> q + geom_hline(aes(yintercept=mean(mpg),group=cyl))

but it doesn't work - the mean is taken for all data points, not per
group. Here follows a discussion of my attempts to do it inside
ggplot (without externally generating stats)

I can (almost) simulate what I want this using the degenerate case of
stat_smooth, where I simply use lm to calculate group means:

First, let's check degenerate lm works. I have to mess with the
output coefficients a little:

> mn=coef(lm(mpg~cyl,mtcars));c(mn[1],mn[-1]+mn[1])
(Intercept) cyl6 cyl8
26.66364 19.74286 15.10000
> ddply(mc,.(cyl),summarize,m=mean(mpg))
cyl m
1 4 26.66364
2 6 19.74286
3 8 15.10000

That all seems sane

> q + stat_smooth(method=lm, fun=mpg~cyl , aes(group=1), geom="point", size=5)

But it gets the point positions wrong for some subtle reason I'm
missing. If I check the values:

> q + stat_smooth(method=lm, fun=mpg~cyl , aes(group=1,label=sprintf("%.2f",..y..)), geom="text", size=5, level=1)

26.38, 20.63, 14.88... huh.

If I specify to apply the smooth per group, it gives an error:

> q + stat_smooth(method=lm, fun=mpg~cyl , aes(group=cyl), geom="point", size=5)
geom_smooth: Only one unique x value each group.Maybe you want
aes(group = 1)?

It should look like this looks:

> q + geom_point(data=ddply(mtcars,.(cyl),summarize,mpg=mean(mpg)),size=5)

You might reasonably point out that I should use stat_summary in this
case, since x==group. However, that's not the usual data I deal with
- I use x for something else. I'm just simplifying for this example.
For example, this should ideally work too:

> q = ggplot(mtcars, aes(x = wt, y=mpg)) + geom_point()
> q + geom_point(aes(mpg=mean(mpg), wt=mean(wt), group=cyl),size=5)

and should be equivalent to

> q = ggplot(mtcars, aes(x = wt, y=mpg)) + geom_point()
> q + geom_point(data=ddply(mtcars,.(cyl),summarize,mpg=mean(mpg), wt=mean(wt)),size=5)

One problem I have with the ddply approach is that I need to re-supply
the data into ggplot.
This can lead to bugs, for instance if I had already filtered the
data, I might forget to filter it again.
if I could replace that with a reference to the current data set, I
would be happier.

This works, but fails badly and unpredictably if I combine the ggplot
and geom into one expression

> q + geom_point(data=ddply(q$data,.(cyl),summarize,mpg=mean(mpg), wt=mean(wt)),size=5)

-Alex

Alex@775

unread,
Mar 16, 2012, 1:16:05 PM3/16/12
to ggplot2
That is pretty awesome, thanks. I shall probably use this on
occasion.

> q = ggplot(mtcars, aes(x = wt, y=mpg, group=cyl)) + geom_point() + geom_smooth(se=F, method="lm", formula=y~1)
> q

It does feel a bit cheaty.

It goes a bit wrong if I want to annotate with text - it turns out
smooth generates multiple points.

> q = ggplot(mtcars, aes(x = wt, y=mpg, group=cyl)) + geom_point() + stat_smooth(se=F, method="lm", formula=y~1, aes(label=sprintf("%.2f",..y..)), geom=c("text","hline"))
> q

It also only deals with one dimension - I may want to position x and y
using simple stats line mean, max, min.

On Mar 16, 9:21 am, Jean-Olivier Irisson <iris...@normalesup.org>
wrote:
> Tel:+33 04 93 76 38 04begin_of_the_skype_highlighting            +33 04 93 76 38 04      end_of_the_skype_highlighting
> Mob:+33 06 21 05 19 90begin_of_the_skype_highlighting            +33 06 21 05 19 90      end_of_the_skype_highlightinghttp://jo.irisson.com/

Alex@775

unread,
Mar 16, 2012, 5:12:40 PM3/16/12
to ggplot2
I've coded up a stat_summaryGroup which behaves as I want - is it
appropriate to post it here? It's based upon stat_summary code.

used like this:

> qplot(data=diamonds,carat,price,label=color,group=color)+geom_text(color="red",stat="summaryGroup",fun.x=mean,fun.y=mean)


-Alex
> > Tel:+33 04 93 76 38 04begin_of_the_skype_highlighting            +33 04 93 76 38 04      end_of_the_skype_highlightingbegin_of_the_skype_highlighting            +33 04 93 76 38 04begin_of_the_skype_highlighting            +33 04 93 76 38 04      end_of_the_skype_highlighting      end_of_the_skype_highlighting
> > Mob:+33 06 21 05 19 90begin_of_the_skype_highlighting            +33 06 21 05 19 90      end_of_the_skype_highlightingbegin_of_the_skype_highlighting            +33 06 21 05 19 90begin_of_the_skype_highlighting            +33 06 21 05 19 90      end_of_the_skype_highlighting      end_of_the_skype_highlightinghttp://jo.irisson.com/- Hide quoted text -
>
> - Show quoted text -

Willem

unread,
Mar 17, 2012, 1:15:59 PM3/17/12
to ggplot2
Hi,
If I am not mistaken, you want something like this:
http://stackoverflow.com/questions/4244157/ggplot2-add-line-for-average-per-group
That is my question and I have been able to do it with the code
described there.

Kind regards,

Willem

On Mar 16, 10:12 pm, "Alex@775" <alexbbr...@gmail.com> wrote:
> I've coded up a stat_summaryGroup which behaves as I want - is it
> appropriate to post it here?  It's based upon stat_summary code.
>
> used like this:
>
> > qplot(data=diamonds,carat,price,label=color,group=color)+geom_text(color="r ed",stat="summaryGroup",fun.x=mean,fun.y=mean)
> > > Mob:+33 06 21 05 19 90begin_of_the_skype_highlighting            +33 06 21 05 19 90      end_of_the_skype_highlightingbegin_of_the_skype_highlighting            +33 06 21 05 19 90begin_of_the_skype_highlighting            +33 06 21 05 19 90      end_of_the_skype_highlighting      end_of_the_skype_highlightinghttp://jo.irisson.com/-Hide quoted text -

Alex@775

unread,
Mar 17, 2012, 3:04:21 PM3/17/12
to ggplot2
Here's an example implementation where I extend stat_summary into
stat_summaryGroup, and require x transformation to be specified as
well (via fun.x, fun.xmin etc.)

Just execute the code in your r terminal, then run the sample

Source code to stat_summaryGroup: http://pastebin.com/NbeGmAds

Example

> qplot(data=diamonds,carat,price,label=color,group=color)+geom_text(color="r­ed",stat="summaryGroup",fun.x=mean,fun.y=mean)

This is not production ready, since

* it doesn't match my favourite implmentation, where no 'stat' is
required, just aes with summary functions
* it should really be implemented by generalising stat_summary to
allow summarised groups to be specified, for example x, y, z, group.

-Alex


On Mar 16, 2:12 pm, "Alex@775" <alexbbr...@gmail.com> wrote:
> I've coded up a stat_summaryGroup which behaves as I want - is it
> appropriate to post it here?  It's based upon stat_summary code.
>
> used like this:
>
> > qplot(data=diamonds,carat,price,label=color,group=color)+geom_text(color="r­ed",stat="summaryGroup",fun.x=mean,fun.y=mean)
> > > Tel:+33 04 93 76 38 04begin_of_the_skype_highlighting            +33 04 93 76 38 04      end_of_the_skype_highlightingbegin_of_the_skype_highlighting            +33 04 93 76 38 04begin_of_the_skype_highlighting            +33 04 93 76 38 04      end_of_the_skype_highlighting      end_of_the_skype_highlightingbegin_of_the_skype_highlighting            +33 04 93 76 38 04begin_of_the_skype_highlighting            +33 04 93 76 38 04      end_of_the_skype_highlightingbegin_of_the_skype_highlighting            +33 04 93 76 38 04      end_of_the_skype_highlighting      end_of_the_skype_highlighting
> > > Mob:+33 06 21 05 19 90begin_of_the_skype_highlighting            +33 06 21 05 19 90      end_of_the_skype_highlightingbegin_of_the_skype_highlighting            +33 06 21 05 19 90begin_of_the_skype_highlighting            +33 06 21 05 19 90      end_of_the_skype_highlighting      end_of_the_skype_highlightingbegin_of_the_skype_highlighting            +33 06 21 05 19 90begin_of_the_skype_highlighting            +33 06 21 05 19 90      end_of_the_skype_highlightingbegin_of_the_skype_highlighting            +33 06 21 05 19 90      end_of_the_skype_highlighting      end_of_the_skype_highlightinghttp://jo.irisson.com/-Hide quoted text -
>
> > - Show quoted text -- Hide quoted text -

Alex@775

unread,
Mar 17, 2012, 3:05:46 PM3/17/12
to ggplot2


On Mar 17, 10:15 am, Willem <wligtenb...@gmail.com> wrote:
> Hi,
> If I am not mistaken, you want something like this:http://stackoverflow.com/questions/4244157/ggplot2-add-line-for-avera...
> That is my question and I have been able to do it with the code
> described there.

Hi, I want the effect, not the labour. This should IMHO be integral.
The fact that this question appears on stackoverflow reflects the
nonobviousness of the operation in existing ggplot mechanisms.

>
> Kind regards,

Thanks,

-Alex
<clip>

Alex@775

unread,
Mar 17, 2012, 3:06:56 PM3/17/12
to ggplot2

On Mar 17, 10:15 am, Willem <wligtenb...@gmail.com> wrote:
> Hi,
> If I am not mistaken, you want something like this:http://stackoverflow.com/questions/4244157/ggplot2-add-line-for-avera...
> That is my question and I have been able to do it with the code
> described there.

Oh, and that's my answer, right there at #1 with a big green tick and
the working code example.

-Alex
Reply all
Reply to author
Forward
0 new messages