ggplot2 performance
74 views
Skip to first unread message
jacktanner
Aug 30, 2010, 1:52:38 PM8/30/10
to ggplot2
I know everyone here's aware of ggplot2 performance issues... I seem
to recall Hadley saying something like there was work going on to
improve speed. Any updates?
In case it'd be helpful, I came across an easy-to-reproduce example
that's particularly slow, IMHO.
x=expand.grid(list(id=1:30,f2=1:5,f3=1:5)) # data frame for 750 rows
x$id = factor(x$id)
x=cbind(x, y=sample(10, 750, replace=TRUE))
ptm <- proc.time()
ggplot(x) + geom_line(aes(x=f2, y=y, group=id, colour=id)) +
facet_grid(id ~ f3)
proc.time() - ptm
# user system elapsed
# 105.75 1.01 132.81
to recall Hadley saying something like there was work going on to
improve speed. Any updates?
In case it'd be helpful, I came across an easy-to-reproduce example
that's particularly slow, IMHO.
x=expand.grid(list(id=1:30,f2=1:5,f3=1:5)) # data frame for 750 rows
x$id = factor(x$id)
x=cbind(x, y=sample(10, 750, replace=TRUE))
ptm <- proc.time()
ggplot(x) + geom_line(aes(x=f2, y=y, group=id, colour=id)) +
facet_grid(id ~ f3)
proc.time() - ptm
# user system elapsed
# 105.75 1.01 132.81
Brandon Hurr
Aug 30, 2010, 3:35:00 PM8/30/10
to jacktanner, ggplot2
user system elapsed
31.572 0.244 31.629
Not nearly as bad for me (late 2007 MacbookPro). Admittedly, when I graph on my netbook the wait time can be brutal. Minutes for a relatively simple faceted plot. I usually get a coffee or something.
R version 2.11.1 (2010-05-31)
x86_64-apple-darwin9.8.0
locale:
[1] en_US.UTF-8/en_US.UTF-8/C/C/en_US.UTF-8/en_US.UTF-8
attached base packages:
[1] grid stats graphics grDevices utils datasets methods base
other attached packages:
[1] ggplot2_0.8.8 reshape_0.8.3 plyr_1.1 proto_0.3-8
loaded via a namespace (and not attached):
[1] digest_0.4.2
--
You received this message because you are subscribed to the ggplot2 mailing list.
Please provide a reproducible example: http://gist.github.com/270442
To post: email ggp...@googlegroups.com
To unsubscribe: email ggplot2+u...@googlegroups.com
More options: http://groups.google.com/group/ggplot2
Dennis Murphy
Aug 30, 2010, 6:45:29 PM8/30/10
to jacktanner, ggplot2
Hi:
Here's my take (Win 7, 64-bit R w/ 8Gb memory):
> system.time(print({ggplot(x) + geom_line(aes(x=f2, y=y, group=id, colour=id)) +
+ facet_grid(id ~ f3)}))
user system elapsed
26.40 0.13 26.63
A more or less equivalent version in lattice is given below.
x$id <- factor(x$id)
library(lattice)
library(latticeExtra) # needed for the 2D labels
> system.time(print({mykey <- list(space = 'right',
+ title = 'ID',
+ cex.title = 1,
+ text = list(levels(x$id), cex = 0.7),
+ lines = list(lty = 1, col = levels(x$id)) )
+ myxy <- xyplot(y ~ f2 | f3 * id, data = x, type = 'l', groups = id,
+ key = mykey,
+ par.strip.text = list(cex = 0.7),
+ scales = list(cex = 0.7))
+ useOuterStrips(myxy)}) )
user system elapsed
7.28 0.05 7.33
So lattice is a little less than four times as fast as the ggplot version. Which do you think took less time to code? Although there isn't a lot of lattice code here, it's reasonably sophisticated, especially in relation to the corresponding ggplot2 code. The time it takes to program the plot you want needs to be factored into the discussion, too. That being said, both plots look nice, but I would have messed around with the legend a bit in the ggplot2 graph (e.g., add a title, shrink the size of the labels) and fussed with the y-axis labels...such fix-ups would even out the overall timings a bit, I reckon...
HTH,
Dennis
Here's my take (Win 7, 64-bit R w/ 8Gb memory):
> system.time(print({ggplot(x) + geom_line(aes(x=f2, y=y, group=id, colour=id)) +
+ facet_grid(id ~ f3)}))
user system elapsed
26.40 0.13 26.63
A more or less equivalent version in lattice is given below.
x$id <- factor(x$id)
library(lattice)
library(latticeExtra) # needed for the 2D labels
> system.time(print({mykey <- list(space = 'right',
+ title = 'ID',
+ cex.title = 1,
+ text = list(levels(x$id), cex = 0.7),
+ lines = list(lty = 1, col = levels(x$id)) )
+ myxy <- xyplot(y ~ f2 | f3 * id, data = x, type = 'l', groups = id,
+ key = mykey,
+ par.strip.text = list(cex = 0.7),
+ scales = list(cex = 0.7))
+ useOuterStrips(myxy)}) )
user system elapsed
7.28 0.05 7.33
So lattice is a little less than four times as fast as the ggplot version. Which do you think took less time to code? Although there isn't a lot of lattice code here, it's reasonably sophisticated, especially in relation to the corresponding ggplot2 code. The time it takes to program the plot you want needs to be factored into the discussion, too. That being said, both plots look nice, but I would have messed around with the legend a bit in the ggplot2 graph (e.g., add a title, shrink the size of the labels) and fussed with the y-axis labels...such fix-ups would even out the overall timings a bit, I reckon...
HTH,
Dennis
Brandon Hurr
Aug 31, 2010, 5:38:59 AM8/31/10
to Dennis Murphy, jacktanner, ggplot2
user system elapsed
158.075 2.717 162.974
R version 2.11.1 (2010-05-31)
i386-apple-darwin9.8.0
1.6 GHz Atom processor w/ 2GB RAM (Dell Mini10v)
It took a while, but really wasn't too bad considering how much data you're looking at. It would be painful if you were tweaking it though.
Brandon
Mike Lawrence
Aug 31, 2010, 8:21:48 AM8/31/10
to ggplot2
MacBook5,1
OS X 10.6.4
2.4GHz Core 2 Duo
4MB 1067MHz DDR3
OS X 10.6.4
2.4GHz Core 2 Duo
4MB 1067MHz DDR3
user system elapsed
30.770 0.160 31.279
> sessionInfo()
R version 2.11.1 (2010-05-31)
x86_64-apple-darwin9.8.0
locale:
[1] en_CA.UTF-8/en_CA.UTF-8/C/C/en_CA.UTF-8/en_CA.UTF-8
attached base packages:
[1] grid stats graphics grDevices utils datasets methods
[8] base
other attached packages:
[1] ggplot2_0.8.8 proto_0.3-8 reshape_0.8.3 plyr_1.1
loaded via a namespace (and not attached):
[1] digest_0.4.2 tools_2.11.1
Stephen O'Connell
Aug 31, 2010, 10:07:59 AM8/31/10
to ggplot2
Another datapoint on performance:
VMware based VM running Windows XP SP3, (1) Vcpu with 2GB, running on a 4X3Ghz, 32GB HP blade, with 12 other active VMs.
user system elapsed
32.10 2.50 38.47
32.10 2.50 38.47
> sessionInfo()
R version 2.11.1 (2010-05-31)
R version 2.11.1 (2010-05-31)
i386-pc-mingw32
locale:
[1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252
[3] LC_MONETARY=English_United States.1252 LC_NUMERIC=C
[5] LC_TIME=English_United States.1252
[1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252
[3] LC_MONETARY=English_United States.1252 LC_NUMERIC=C
[5] LC_TIME=English_United States.1252
attached base packages:
[1] grid stats graphics grDevices utils datasets methods base
[1] grid stats graphics grDevices utils datasets methods base
other attached packages:
[1] ggplot2_0.8.8 proto_0.3-8 reshape_0.8.3 plyr_1.0.3
[1] ggplot2_0.8.8 proto_0.3-8 reshape_0.8.3 plyr_1.0.3
loaded via a namespace (and not attached):
[1] digest_0.4.2
[1] digest_0.4.2
language R
version.string R version 2.11.1 (2010-05-31)
version.string R version 2.11.1 (2010-05-31)
--
thomas
Aug 31, 2010, 11:07:32 AM8/31/10
to ggplot2
just out of curiosity, what hardware are you using to achieve
105seconds?
Another datapoint on performance:
MacPro
OS X 10.6.4
2x2.26GHz Intel Xeon Quad
9GB 1067MHz DDR3
user system elapsed
28.502 0.181 29.683
105seconds?
Another datapoint on performance:
MacPro
OS X 10.6.4
2x2.26GHz Intel Xeon Quad
9GB 1067MHz DDR3
user system elapsed
28.502 0.181 29.683
Jonathan Christensen
Aug 31, 2010, 8:00:24 PM8/31/10
to thomas, ggplot2
Windows 7 running R 2.10.1
2.4GHz Intel Core Duo
2 GB RAM
user system elapsed
45.31 0.39 46.11
OS X running R 2.11.1
2 GHz Intel Core 2 Duo
2 GB Ram
2.4GHz Intel Core Duo
2 GB RAM
user system elapsed
45.31 0.39 46.11
OS X running R 2.11.1
2 GHz Intel Core 2 Duo
2 GB Ram
user system elapsed
39.666 0.204 39.745
Jonathan
--
Joshua Wiley
Aug 31, 2010, 9:10:37 PM8/31/10
to ggplot2
At this point, I believe we may reasonably conclude the following:
1) The code takes *some* time to run
2) The code takes a disparate number of seconds to run depending on
who sent in the timings
> ggplot.times <- c(132.81, 31.629, 26.63, 162.974, 31.279,
+ 38.47, 29.683, 46.11, 39.745)
> chisq.test(x = ggplot.times, simulate.p.value = TRUE, B = 10^5)
data: ggplot.times
X-squared = 344.3292, df = NA, p-value = 1e-05
2.a) We may cautiously posit this is due to different hardware and/or software
3) The users of ggplot2 have lots*** ofcombinations of CPUs, OSes,
RAM, R, and ggplot2
On Mon, Aug 30, 2010 at 3:45 PM, Dennis Murphy <djm...@gmail.com> wrote:
> That being said, both plots look nice, but I would have messed around with
> the legend a bit in the ggplot2 graph (e.g., add a title, shrink the size of
> the labels) and fussed with the y-axis labels...such fix-ups would even out
> the overall timings a bit, I reckon...
I don't know...I kind of liked it. It could be called "Legend Exceeds Plot:
exploring information overload in the digital age"
>
> HTH,
> Dennis
Joshua
*** This in the sense of fortune("lots of R users")
Brandon Hurr
Sep 1, 2010, 8:31:14 AM9/1/10
to Joshua Wiley, ggplot2
I dunno, seems like there may be a strong effect of CPU speed in the lower end, but it's hard to draw conclusions from such a small sample size with missing factors.
structure(list(user = c(105.75, 31.572, 26.4, 158.075, 30.77,
32.1, 28.502, 45.31, 39.666), system = c(1.01, 0.244, 0.13, 2.717,
0.16, 2.5, 0.181, 0.39, 0.204), elapsed = c(132.81, 31.629, 26.63,
162.97, 31.279, 38.47, 29.683, 46.11, 39.745), Platform = structure(c(2L,
1L, 3L, 1L, 1L, 3L, 1L, 3L, 1L), .Label = c("Mac", "Unknown",
"Windows"), class = "factor"), GHz = c(NA, 2.4, NA, 1.6, 2.4,
3, 2.26, 2.4, 2), X32.64 = c(NA, 64L, 64L, 32L, 64L, 32L, 32L,
32L, 32L), RAM = c(NA, 3L, 8L, 2L, 4L, 2L, 9L, 2L, 2L), UseR = structure(c(3L,
1L, 2L, 1L, 5L, 6L, 7L, 4L, 4L), .Label = c("Brandon", "Dennis",
"jacktanner", "Jonathan", "Mike", "Stephen", "Thomas"), class = "factor")), .Names = c("user",
"system", "elapsed", "Platform", "GHz", "32.64", "RAM", "UseR"
), class = "data.frame", row.names = c(NA, -9L))
Mark Connolly
Sep 1, 2010, 8:47:24 AM9/1/10
to Brandon Hurr, Joshua Wiley, ggplot2
I like where you are going with this. Come up with specifications for
data collected, number of trials, factors controlled, and have everyone
submit their observations. Then we can infer the relationship between
platform characteristics and this specific plot and make ggplot faster.
data collected, number of trials, factors controlled, and have everyone
submit their observations. Then we can infer the relationship between
platform characteristics and this specific plot and make ggplot faster.
:^
Andrea Nixon
Sep 1, 2010, 10:38:22 AM9/1/10
to ggplot2
32-bit version of R:
ggplot2 0.8.7 took 99
ggplot2 0.8.8 took 102 seconds
64-bit verion of R:
ggplot2 0.8.8 took 96 seconds
Hardware (used in all three instances):
MacBook Pro
Intel Core 2 Duo
2.4 GHz
Bus speed 800 MHz
--
Andrea Nixon, Ph.D.
Director of Curricular and Research Support
Dean of the College Office
Carleton College
<ggplot.speed.png>
Roman Luštrik
Sep 1, 2010, 11:46:21 AM9/1/10
to ggplot2
user system elapsed
29.594 1.308 38.355
on
AMD Athlon X2 245 2.9 GHz (dual core)
2 GB RAM
mother board ASUS M4A78T-E
on Ubuntu 10.04
29.594 1.308 38.355
on
AMD Athlon X2 245 2.9 GHz (dual core)
2 GB RAM
mother board ASUS M4A78T-E
on Ubuntu 10.04
Ben Bolker
Sep 1, 2010, 11:56:28 AM9/1/10
to Roman Luštrik, ggplot2
user system elapsed
47.391 1.356 55.109
47.391 1.356 55.109
Ubuntu 8.04
R 2.11.1, ggplot2 0.8.8
cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 15
model name : Intel(R) Core(TM)2 CPU T5500 @ 1.66GHz
stepping : 2
cpu MHz : 1000.000
cache size : 2048 KB
cat /proc/meminfo
MemTotal: 2063584 kB
MemFree: 79188 kB
--
Ben Bolker
bbo...@gmail.com , bol...@mcmaster.ca
http://www.math.mcmaster.ca/~bolker
GPG key: http://www.math.mcmaster.ca/~bolker/benbolker-publickey.asc
jacktanner
Sep 1, 2010, 1:45:20 PM9/1/10
to ggplot2
My original timings are on a Pentium M 1.87GHz w/ 2GB RAM, Windows 7
32 bit, R 2.11.1. Time for an upgrade? :)
Still, even 30 sec seems way too high to produce that plot. It's only
750 data points. There's no statistical manipulation, just dots and
lines and rectangles to draw.
32 bit, R 2.11.1. Time for an upgrade? :)
Still, even 30 sec seems way too high to produce that plot. It's only
750 data points. There's no statistical manipulation, just dots and
lines and rectangles to draw.
Matthieu Dubois
Sep 1, 2010, 2:01:31 PM9/1/10
to ggplot2
Using 32-bits R
user system elapsed
30.221 0.162 31.209
Using 64-bits R
user system elapsed
27.972 0.183 28.017
Both measurments made on a MacBook Pro
2.8 GHz Intel Core 2 Duo
4Go RAM
running Mac OS 10.6.4
R version 2.11.1 (2010-05-31) with ggplot2 v0.8.7
Best,
Matthieu
user system elapsed
30.221 0.162 31.209
Using 64-bits R
user system elapsed
27.972 0.183 28.017
Both measurments made on a MacBook Pro
2.8 GHz Intel Core 2 Duo
4Go RAM
running Mac OS 10.6.4
R version 2.11.1 (2010-05-31) with ggplot2 v0.8.7
Best,
Matthieu
Hadley Wickham
Sep 1, 2010, 2:03:58 PM9/1/10
to jacktanner, ggplot2
Hi Jack,
Thanks for the benchmark - it's a good example of something that
should be much much faster. It's so slow because there are so many
facets - there's a lot of overhead in facetting that I need to
improve. I have some ideas on how to improve it, and I'm currently in
the middle of a rewrite of how the layout system (including scales,
facets and coordinate systems) work, so I'm hopeful that I should be
able to make some big improvements.
Hadley
> --
> You received this message because you are subscribed to the ggplot2 mailing list.
> Please provide a reproducible example: http://gist.github.com/270442
>
> To post: email ggp...@googlegroups.com
> To unsubscribe: email ggplot2+u...@googlegroups.com
> More options: http://groups.google.com/group/ggplot2
>
--
Assistant Professor / Dobelman Family Junior Chair
Department of Statistics / Rice University
http://had.co.nz/
jacktanner
Sep 1, 2010, 10:27:24 PM9/1/10
to ggplot2
Thanks for all your hard work, Hadley. Looking forward to the
improvements.
improvements.
John
Sep 3, 2010, 6:03:25 PM9/3/10
to ggplot2
Another data point:
user system elapsed
55.692 0.617 61.150
> sessionInfo()
Model Identifier: MacBookAir2,1
Processor Name: Intel Core 2 Duo
Processor Speed: 2.13 GHz
Number Of Processors: 1
Total Number Of Cores: 2
L2 Cache: 6 MB
Memory: 2 GB
Bus Speed: 1.07 GHz
On Sep 1, 8:31 am, Brandon Hurr <bhiv...@gmail.com> wrote:
> I dunno, seems like there may be a strong effect of CPU speed in the lower
> end, but it's hard to draw conclusions from such a small sample size with
> missing factors.
>
> structure(list(user = c(105.75, 31.572, 26.4, 158.075, 30.77,
> 32.1, 28.502, 45.31, 39.666), system = c(1.01, 0.244, 0.13, 2.717,
> 0.16, 2.5, 0.181, 0.39, 0.204), elapsed = c(132.81, 31.629, 26.63,
> 162.97, 31.279, 38.47, 29.683, 46.11, 39.745), Platform = structure(c(2L,
> 1L, 3L, 1L, 1L, 3L, 1L, 3L, 1L), .Label = c("Mac", "Unknown",
> "Windows"), class = "factor"), GHz = c(NA, 2.4, NA, 1.6, 2.4,
> 3, 2.26, 2.4, 2), X32.64 = c(NA, 64L, 64L, 32L, 64L, 32L, 32L,
> 32L, 32L), RAM = c(NA, 3L, 8L, 2L, 4L, 2L, 9L, 2L, 2L), UseR =
> structure(c(3L,
> 1L, 2L, 1L, 5L, 6L, 7L, 4L, 4L), .Label = c("Brandon", "Dennis",
> "jacktanner", "Jonathan", "Mike", "Stephen", "Thomas"), class = "factor")),
> .Names = c("user",
> "system", "elapsed", "Platform", "GHz", "32.64", "RAM", "UseR"
> ), class = "data.frame", row.names = c(NA, -9L))
>
> 156KViewDownload
user system elapsed
55.692 0.617 61.150
> sessionInfo()
R version 2.11.1 (2010-05-31)
i386-apple-darwin9.8.0
Model Name: MacBook Air
i386-apple-darwin9.8.0
Model Identifier: MacBookAir2,1
Processor Name: Intel Core 2 Duo
Processor Speed: 2.13 GHz
Number Of Processors: 1
Total Number Of Cores: 2
L2 Cache: 6 MB
Memory: 2 GB
Bus Speed: 1.07 GHz
On Sep 1, 8:31 am, Brandon Hurr <bhiv...@gmail.com> wrote:
> I dunno, seems like there may be a strong effect of CPU speed in the lower
> end, but it's hard to draw conclusions from such a small sample size with
> missing factors.
>
> structure(list(user = c(105.75, 31.572, 26.4, 158.075, 30.77,
> 32.1, 28.502, 45.31, 39.666), system = c(1.01, 0.244, 0.13, 2.717,
> 0.16, 2.5, 0.181, 0.39, 0.204), elapsed = c(132.81, 31.629, 26.63,
> 162.97, 31.279, 38.47, 29.683, 46.11, 39.745), Platform = structure(c(2L,
> 1L, 3L, 1L, 1L, 3L, 1L, 3L, 1L), .Label = c("Mac", "Unknown",
> "Windows"), class = "factor"), GHz = c(NA, 2.4, NA, 1.6, 2.4,
> 3, 2.26, 2.4, 2), X32.64 = c(NA, 64L, 64L, 32L, 64L, 32L, 32L,
> 32L, 32L), RAM = c(NA, 3L, 8L, 2L, 4L, 2L, 9L, 2L, 2L), UseR =
> structure(c(3L,
> 1L, 2L, 1L, 5L, 6L, 7L, 4L, 4L), .Label = c("Brandon", "Dennis",
> "jacktanner", "Jonathan", "Mike", "Stephen", "Thomas"), class = "factor")),
> .Names = c("user",
> "system", "elapsed", "Platform", "GHz", "32.64", "RAM", "UseR"
> ), class = "data.frame", row.names = c(NA, -9L))
>
> On Wed, Sep 1, 2010 at 02:10, Joshua Wiley <jwiley.ps...@gmail.com> wrote:
> > At this point, I believe we may reasonably conclude the following:
>
> > 1) The code takes *some* time to run
> > 2) The code takes a disparate number of seconds to run depending on
> > who sent in the timings
>
> > > ggplot.times <- c(132.81, 31.629, 26.63, 162.974, 31.279,
> > + 38.47, 29.683, 46.11, 39.745)
> > > chisq.test(x = ggplot.times, simulate.p.value = TRUE, B = 10^5)
>
> > data: ggplot.times
> > X-squared = 344.3292, df = NA, p-value = 1e-05
>
> > 2.a) We may cautiously posit this is due to different hardware and/or
> > software
> > 3) The users of ggplot2 have lots*** ofcombinations of CPUs, OSes,
> > RAM, R, and ggplot2
>
> > At this point, I believe we may reasonably conclude the following:
>
> > 1) The code takes *some* time to run
> > 2) The code takes a disparate number of seconds to run depending on
> > who sent in the timings
>
> > > ggplot.times <- c(132.81, 31.629, 26.63, 162.974, 31.279,
> > + 38.47, 29.683, 46.11, 39.745)
> > > chisq.test(x = ggplot.times, simulate.p.value = TRUE, B = 10^5)
>
> > data: ggplot.times
> > X-squared = 344.3292, df = NA, p-value = 1e-05
>
> > 2.a) We may cautiously posit this is due to different hardware and/or
> > software
> > 3) The users of ggplot2 have lots*** ofcombinations of CPUs, OSes,
> > RAM, R, and ggplot2
>
> > On Mon, Aug 30, 2010 at 3:45 PM, Dennis Murphy <djmu...@gmail.com> wrote:
>
> > > That being said, both plots look nice, but I would have messed around
> > with
> > > the legend a bit in the ggplot2 graph (e.g., add a title, shrink the size
> > of
> > > the labels) and fussed with the y-axis labels...such fix-ups would even
> > out
> > > the overall timings a bit, I reckon...
>
> > I don't know...I kind of liked it. It could be called "Legend Exceeds
> > Plot:
> > exploring information overload in the digital age"
>
> > > HTH,
> > > Dennis
>
> > Joshua
>
> > *** This in the sense of fortune("lots of R users")
>
> > --
> > You received this message because you are subscribed to the ggplot2 mailing
> > list.
> > Please provide a reproducible example:http://gist.github.com/270442
>
> > To post: email ggp...@googlegroups.com
> > To unsubscribe: email ggplot2+u...@googlegroups.com<ggplot2%2Bunsu...@googlegroups.com>
>
> > > That being said, both plots look nice, but I would have messed around
> > with
> > > the legend a bit in the ggplot2 graph (e.g., add a title, shrink the size
> > of
> > > the labels) and fussed with the y-axis labels...such fix-ups would even
> > out
> > > the overall timings a bit, I reckon...
>
> > I don't know...I kind of liked it. It could be called "Legend Exceeds
> > Plot:
> > exploring information overload in the digital age"
>
> > > HTH,
> > > Dennis
>
> > Joshua
>
> > *** This in the sense of fortune("lots of R users")
>
> > --
> > You received this message because you are subscribed to the ggplot2 mailing
> > list.
> > Please provide a reproducible example:http://gist.github.com/270442
>
> > To post: email ggp...@googlegroups.com
> 156KViewDownload
jacktanner
Sep 3, 2010, 11:25:41 PM9/3/10
to ggplot2
Hadley, would you find it helpful if there were test cases of slow
performance related to non-faceting plots? Not saying that I have any,
but I'd be happy to keep track.
performance related to non-faceting plots? Not saying that I have any,
but I'd be happy to keep track.
James Howison
Sep 4, 2010, 9:13:19 AM9/4/10
to ggplot2
Does anyone have a downloadable script that users could run and report timings? Might be a great addition to ggExtras or even ggplot2. Then devs could add performance tests and we could periodically re-run it (could even report results directly to a server ...)
--J
Hadley Wickham
Sep 4, 2010, 1:59:07 PM9/4/10
to jacktanner, ggplot2
Yes, definitely!
Hadley
Hadley
Hadley Wickham
Sep 4, 2010, 2:01:12 PM9/4/10
to James Howison, ggplot2
I have a half written benchmarking package that I've been using for
plyr - http://github.com/hadley/bench
plyr - http://github.com/hadley/bench
Hadley
Greg Hunt
Sep 4, 2010, 3:57:12 PM9/4/10
to ggplot2
You are unlikely to get usable data just looking at Intel clock speeds. Memory access speeds can vary hugely between processor models that have similar clock speeds and depending on the mix of on-chip and off-chip processing, if for example the R code has poor locality of reference or the amount of memory access for data is large, then memory access speeds will be the dominant performance factor. It would be useful to add in processor age and model to the data collected.
Joshua Wiley
Sep 4, 2010, 5:11:39 PM9/4/10
to ggplot2
I wonder what is the real goal or question being asked? Are we trying
to get some 'true' estimate of the code's speed, partialing out
'error' due to hardware? Are we hoping to be able to make
recommendations as to what the 'ideal' hardware is or find the
relative contribution of processor frequency vs. memory access speeds,
vs. quantity of memory vs. ...? Will knowing the time it takes to run
code on various hardware configurations help to write faster code? It
seems to me that we have jumped straight to the data collection phase,
without considering the goals and design of our study.
to get some 'true' estimate of the code's speed, partialing out
'error' due to hardware? Are we hoping to be able to make
recommendations as to what the 'ideal' hardware is or find the
relative contribution of processor frequency vs. memory access speeds,
vs. quantity of memory vs. ...? Will knowing the time it takes to run
code on various hardware configurations help to write faster code? It
seems to me that we have jumped straight to the data collection phase,
without considering the goals and design of our study.
Josh
<snip>
jacktanner
Sep 5, 2010, 8:39:34 PM9/5/10
to ggplot2
On Sep 4, 5:11 pm, Joshua Wiley <jwiley.ps...@gmail.com> wrote:
> I wonder what is the real goal or question being asked?
I think the question ought to be regression testing (computer science
> I wonder what is the real goal or question being asked?
regression, not stats regression).
a) what is the performance of ggplot2 on some reproducible tests?
b) when developing new versions of ggplot2, how can we make sure that
performance improves, or at least doesn't regress?
Leonardo Salas
Sep 6, 2010, 12:21:47 PM9/6/10
to ggplot2
It seems what you need is the difference in time to run the same code in two different versions of ggplot2 - or more appropriately (because the different versions may run on different versions of R or require different versions of dependencies) code that calculates the difference between a ggplot2 plot and the same output using graphics::plot (an intrinsic benchmark) for both versions of ggplot2. So, the difference between differences, right?
I'd be happy to run any such code for any two versions of ggplot2. Does anyone have an example?
Leo
--
Reply all
Reply to author
Forward
0 new messages