Jaeggi 2008: means and standard deviations
101 views
Skip to first unread message
Gwern Branwen
May 19, 2012, 11:51:10 PM5/19/12
to N-back
Hi everyone. I've been working on a little meta-analysis of the
various n-back studies, and I've been unable to incorporate 5 of the
studies because I want the means and standard deviations for the
post-intervention IQ test for the experimental & control groups, but
that's not always reported (or it's reported in a form which I am too
ignorant to extract). Can anyone help with the following?
1. Jaeggi 2008
2. Qiu 2009
3. Seidler 2010
4. Chooi 2011
5. Kundu et al 2012
In particular, I *think* the information should be extractable from
Jaeggi 2008 & Chooi 2011. I just can't do it.
I think I'm going to have to email Qiu, Seidler, and Kundu.
(If anyone is curious, with 10 studies so far, my R code is reporting
that under the fixed effect model, the SMD is 0.6423 and the 95% CI is
[0.4620; 0.8227].)
--
gwern
http://www.gwern.net/DNB%20FAQ
various n-back studies, and I've been unable to incorporate 5 of the
studies because I want the means and standard deviations for the
post-intervention IQ test for the experimental & control groups, but
that's not always reported (or it's reported in a form which I am too
ignorant to extract). Can anyone help with the following?
1. Jaeggi 2008
2. Qiu 2009
3. Seidler 2010
4. Chooi 2011
5. Kundu et al 2012
In particular, I *think* the information should be extractable from
Jaeggi 2008 & Chooi 2011. I just can't do it.
I think I'm going to have to email Qiu, Seidler, and Kundu.
(If anyone is curious, with 10 studies so far, my R code is reporting
that under the fixed effect model, the SMD is 0.6423 and the 95% CI is
[0.4620; 0.8227].)
--
gwern
http://www.gwern.net/DNB%20FAQ
Jonathan Toomim
May 19, 2012, 11:59:13 PM5/19/12
to brain-t...@googlegroups.com
Can't you just reverse-engineer the graphs? The error bars in Fig. 3 of
Jaeggi (2008) denote standard error, so multiply by sqrt(n) and you
should have stdev.
http://www.datathief.org/
On 5/19/2012 8:51 PM, Gwern Branwen wrote:
> Hi everyone. I've been working on a little meta-analysis of the
> various n-back studies, and I've been unable to incorporate 5 of the
> studies because I want the means and standard deviations for the
> post-intervention IQ test for the experimental& control groups, but
Jaeggi (2008) denote standard error, so multiply by sqrt(n) and you
should have stdev.
http://www.datathief.org/
On 5/19/2012 8:51 PM, Gwern Branwen wrote:
> Hi everyone. I've been working on a little meta-analysis of the
> various n-back studies, and I've been unable to incorporate 5 of the
> studies because I want the means and standard deviations for the
> that's not always reported (or it's reported in a form which I am too
> ignorant to extract). Can anyone help with the following?
>
> 1. Jaeggi 2008
> 2. Qiu 2009
> 3. Seidler 2010
> 4. Chooi 2011
> 5. Kundu et al 2012
>
> In particular, I *think* the information should be extractable from
> Jaeggi 2008& Chooi 2011. I just can't do it.
> ignorant to extract). Can anyone help with the following?
>
> 1. Jaeggi 2008
> 2. Qiu 2009
> 3. Seidler 2010
> 4. Chooi 2011
> 5. Kundu et al 2012
>
> In particular, I *think* the information should be extractable from
Gwern Branwen
May 20, 2012, 12:09:01 AM5/20/12
to brain-t...@googlegroups.com
On Sat, May 19, 2012 at 11:59 PM, Jonathan Toomim <jto...@jtoomim.org> wrote:
> Can't you just reverse-engineer the graphs? The error bars in Fig. 3 of
> Jaeggi (2008) denote standard error, so multiply by sqrt(n) and you should
> have stdev.
>
> http://www.datathief.org/
I did eyeball it to get means of 11.8 & 10.5, but that's more
> Can't you just reverse-engineer the graphs? The error bars in Fig. 3 of
> Jaeggi (2008) denote standard error, so multiply by sqrt(n) and you should
> have stdev.
>
> http://www.datathief.org/
imprecise than I'd like...
Also, I'm not sure I follow. The bar for experimental looks like it's
1.2 long, so halve that and times by the sample n is 34 * sqrt 0.6 =
26.3?
--
gwern
http://www.gwern.net
Jonathan Toomim
May 20, 2012, 12:39:55 AM5/20/12
to brain-t...@googlegroups.com
I counted pixels, and I got this:
Means:
Control
9.64 pre
10.52 post
Training
9.58 pre
11.81 post
Error bars:
Control
1.09 pre
1.26 post
Training
1.18 pre
1.18 post
Sample size is 34 (training) or 35 (control), so stdevs should be:
Control
(1.09)/2*sqrt(35) = 3.22 pre
3.73 post
Training
3.45 pre
3.42 post
(For the stdev calculations, I only rounded at the end.)
On 5/19/2012 9:09 PM, Gwern Branwen wrote:
> I did eyeball it to get means of 11.8 & 10.5, but that's more
> imprecise than I'd like... Also, I'm not sure I follow. The bar for
> experimental looks like it's 1.2 long, so halve that and times by the
> sample n is 34 * sqrt 0.6 = 26.3?
SE = stdev/(sqrt(n)), so stdev = SE*sqrt(n), not stdev = n*sqrt(SE).
Means:
Control
9.64 pre
10.52 post
Training
9.58 pre
11.81 post
Error bars:
Control
1.09 pre
1.26 post
Training
1.18 pre
1.18 post
Sample size is 34 (training) or 35 (control), so stdevs should be:
Control
(1.09)/2*sqrt(35) = 3.22 pre
3.73 post
Training
3.45 pre
3.42 post
(For the stdev calculations, I only rounded at the end.)
On 5/19/2012 9:09 PM, Gwern Branwen wrote:
> I did eyeball it to get means of 11.8 & 10.5, but that's more
> imprecise than I'd like... Also, I'm not sure I follow. The bar for
> experimental looks like it's 1.2 long, so halve that and times by the
> sample n is 34 * sqrt 0.6 = 26.3?
Gwern Branwen
May 21, 2012, 7:21:24 PM5/21/12
to N-back
So an update. My results as of yesterday:
http://www.gwern.net/DNB%20FAQ#meta-analysis
Further work:
- Chooi sent me a table with the data I needed, so that's taken care of.
- Jaeggi has sent me a table for Jaeggi 2008, so that's also done
- Jaeggi sent me a copy of the poster for her upcoming study, which
turns out to be the UMinn big distance study we were discussing here;
it has a nice _n_=78, but the difference in final RAPM scores is very
small
- Kane's talk turned out to be about Redick et al's big study
- Redick just sent me a copy of it, and it is *very* interesting;
besides discussing a meta-analysis of WM & IQ which I had never heard
of, they found no transfer in their sample of 73, and in particular
they surveyed participants and many of the n-backers were convinced it
had affected their lives... despite no transfer effects.
- Hadwin (of Roughan & Hadwin 2011) did not seem to take my email
kindly, so I don't expect any literature pointers there
- I'm still waiting on sample-size/means/deviations from Qiu, Kundu,
and Seidler; Stephenson hasn't replied yet
- With Jaeggi, Chooi, and Redick incorporated, my new result is:
SMD 95%-CI z p.value
Fixed effect model 0.3516 [0.2033; 0.4998] 4.6489 < 0.0001
Random effects model 0.4710 [0.0901; 0.8519] 2.4238 0.0154
That is, the best guess effect size is somewhere in the
neighborhood of 0.35 or 0.47, although this may be skewed by the
extreme value of Stephenson 2010 (it really jumps out on the funnel
plot)
- Unfortunately I included some studies which used Cogmed as their WM
task, and Jaeggi tells me Cogmed is *not* dual n-back, so some studies
will have to be removed from that... testing it out, it doesn't seem
to make that much of a difference so far.
--
gwern
http://www.gwern.net/DNB%20FAQ#meta-analysis
Further work:
- Chooi sent me a table with the data I needed, so that's taken care of.
- Jaeggi has sent me a table for Jaeggi 2008, so that's also done
- Jaeggi sent me a copy of the poster for her upcoming study, which
turns out to be the UMinn big distance study we were discussing here;
it has a nice _n_=78, but the difference in final RAPM scores is very
small
- Kane's talk turned out to be about Redick et al's big study
- Redick just sent me a copy of it, and it is *very* interesting;
besides discussing a meta-analysis of WM & IQ which I had never heard
of, they found no transfer in their sample of 73, and in particular
they surveyed participants and many of the n-backers were convinced it
had affected their lives... despite no transfer effects.
- Hadwin (of Roughan & Hadwin 2011) did not seem to take my email
kindly, so I don't expect any literature pointers there
- I'm still waiting on sample-size/means/deviations from Qiu, Kundu,
and Seidler; Stephenson hasn't replied yet
- With Jaeggi, Chooi, and Redick incorporated, my new result is:
SMD 95%-CI z p.value
Fixed effect model 0.3516 [0.2033; 0.4998] 4.6489 < 0.0001
Random effects model 0.4710 [0.0901; 0.8519] 2.4238 0.0154
That is, the best guess effect size is somewhere in the
neighborhood of 0.35 or 0.47, although this may be skewed by the
extreme value of Stephenson 2010 (it really jumps out on the funnel
plot)
- Unfortunately I included some studies which used Cogmed as their WM
task, and Jaeggi tells me Cogmed is *not* dual n-back, so some studies
will have to be removed from that... testing it out, it doesn't seem
to make that much of a difference so far.
--
gwern
Jonathan Toomim
May 21, 2012, 7:49:36 PM5/21/12
to brain-t...@googlegroups.com
Nice work. Are you hoping/planning to publish this anywhere besides your
website?
Also, a http://en.wikipedia.org/wiki/Forest_plot would be nice.
On 5/21/2012 4:21 PM, Gwern Branwen wrote:
> So an update. My results as of yesterday:
> http://www.gwern.net/DNB%20FAQ#meta-analysis
>
> Further work:
>
> - Chooi sent me a table with the data I needed, so that's taken care of.
> - Jaeggi has sent me a table for Jaeggi 2008, so that's also done
> - Jaeggi sent me a copy of the poster for her upcoming study, which
> turns out to be the UMinn big distance study we were discussing here;
> it has a nice _n_=78, but the difference in final RAPM scores is very
> small
> - Kane's talk turned out to be about Redick et al's big study
> - Redick just sent me a copy of it, and it is *very* interesting;
> besides discussing a meta-analysis of WM& IQ which I had never heard
website?
Also, a http://en.wikipedia.org/wiki/Forest_plot would be nice.
On 5/21/2012 4:21 PM, Gwern Branwen wrote:
> So an update. My results as of yesterday:
> http://www.gwern.net/DNB%20FAQ#meta-analysis
>
> Further work:
>
> - Chooi sent me a table with the data I needed, so that's taken care of.
> - Jaeggi has sent me a table for Jaeggi 2008, so that's also done
> - Jaeggi sent me a copy of the poster for her upcoming study, which
> turns out to be the UMinn big distance study we were discussing here;
> it has a nice _n_=78, but the difference in final RAPM scores is very
> small
> - Kane's talk turned out to be about Redick et al's big study
> - Redick just sent me a copy of it, and it is *very* interesting;
> of, they found no transfer in their sample of 73, and in particular
> they surveyed participants and many of the n-backers were convinced it
> had affected their lives... despite no transfer effects.
> - Hadwin (of Roughan& Hadwin 2011) did not seem to take my email
> they surveyed participants and many of the n-backers were convinced it
> had affected their lives... despite no transfer effects.
Jonathan Toomim
May 21, 2012, 7:55:56 PM5/21/12
to brain-t...@googlegroups.com
By the way, I've been having second thoughts about the divide-by-two
operation for the error bars to get the standard error. You said Dr.
Jaeggi sent you data from their 2008 study? Were the SEs around 1.2 or
0.6? (SDs around 3 or 6?)
On 5/21/2012 4:21 PM, Gwern Branwen wrote:
operation for the error bars to get the standard error. You said Dr.
Jaeggi sent you data from their 2008 study? Were the SEs around 1.2 or
0.6? (SDs around 3 or 6?)
On 5/21/2012 4:21 PM, Gwern Branwen wrote:
> So an update. My results as of yesterday:
> http://www.gwern.net/DNB%20FAQ#meta-analysis
>
> Further work:
>
> - Chooi sent me a table with the data I needed, so that's taken care of.
> - Jaeggi has sent me a table for Jaeggi 2008, so that's also done
> - Jaeggi sent me a copy of the poster for her upcoming study, which
> turns out to be the UMinn big distance study we were discussing here;
> it has a nice _n_=78, but the difference in final RAPM scores is very
> small
> - Kane's talk turned out to be about Redick et al's big study
> - Redick just sent me a copy of it, and it is *very* interesting;
> besides discussing a meta-analysis of WM& IQ which I had never heard
> http://www.gwern.net/DNB%20FAQ#meta-analysis
>
> Further work:
>
> - Chooi sent me a table with the data I needed, so that's taken care of.
> - Jaeggi has sent me a table for Jaeggi 2008, so that's also done
> - Jaeggi sent me a copy of the poster for her upcoming study, which
> turns out to be the UMinn big distance study we were discussing here;
> it has a nice _n_=78, but the difference in final RAPM scores is very
> small
> - Kane's talk turned out to be about Redick et al's big study
> - Redick just sent me a copy of it, and it is *very* interesting;
> of, they found no transfer in their sample of 73, and in particular
> they surveyed participants and many of the n-backers were convinced it
> had affected their lives... despite no transfer effects.
> - Hadwin (of Roughan& Hadwin 2011) did not seem to take my email
> they surveyed participants and many of the n-backers were convinced it
> had affected their lives... despite no transfer effects.
Gwern Branwen
May 21, 2012, 8:06:07 PM5/21/12
to brain-t...@googlegroups.com
On Mon, May 21, 2012 at 7:49 PM, Jonathan Toomim <jto...@jtoomim.org> wrote:
> Nice work. Are you hoping/planning to publish this anywhere besides your
> website?
Not really; it'd be a lot more work and I think a lot of the academic
> Nice work. Are you hoping/planning to publish this anywhere besides your
> website?
interest is stolen by
- Melby-Lervåg, M., & Hulme, C. (in press). Is working memory training
effective? A meta-analytic review. Developmental Psychology
An IQ-only meta-analysis is of great interest to us, however.
Fortunately for you, `meta` supports forest plots as well as funnels:
1. forest: http://i.imgur.com/9THL1h.png
2. funnel: http://i.imgur.com/JcMwUh.png
On Mon, May 21, 2012 at 7:55 PM, Jonathan Toomim <jto...@jtoomim.org> wrote:
> By the way, I've been having second thoughts about the divide-by-two
> operation for the error bars to get the standard error. You said Dr. Jaeggi
> sent you data from their 2008 study? Were the SEs around 1.2 or 0.6? (SDs
> around 3 or 6?)
"study" "year" "n.e" "mean.e" "sd.e" "n.c" "mean.c" "sd.c"
-"Jaeggi1" 2008 34 11.81 3.42 35 10.52 3.73
+"Jaeggi1" 2008 34 11.82 3.398 35 10.51 3.721
--
gwern
http://www.gwern.net
chesterbenninchanged
May 23, 2012, 4:59:04 AM5/23/12
to brain-t...@googlegroups.com
On the subject, what is the mean for dual n-back, i read somewhere its something like 7 after practice. What is the standard deviation (after practice) for the n level for dual n-back, is the s.d ~ 1 level (where 1 level includes both 1 extra audio and position)?
Reply all
Reply to author
Forward
0 new messages