Melby-Lervåg & Hulme 2012: "Is Working Memory Training Effective? A Meta-Analytic Review"

[Gwern Branwen]

> It has been suggested that working memory training programs are effective both as treatments for attention-deficit/hyperactivity disorder (ADHD) and other cognitive disorders in children and as a tool to improve cognitive ability and scholastic attainment in typically developing children and adults. However, effects across studies appear to be variable, and a systematic meta-analytic review was undertaken. To be included in the review, studies had to be randomized controlled trials or quasi-experiments without randomization, have a treatment, and have either a treated group or an untreated control group.
>
> 23 studies with 30 group comparisons met the criteria for inclusion. The studies included involved clinical samples and samples of typically developing children and adults. Meta-analyses indicated that the programs produced reliable short-term improvements in working memory skills. For verbal working memory, these near-transfer effects were not sustained at follow-up, whereas for visuospatial working memory, limited evidence suggested that such effects might be maintained. More importantly, there was no convincing evidence of the generalization of working memory training to other skills (nonverbal and verbal ability, inhibitory processes in attention, word decoding, and arithmetic). The authors conclude that memory training programs appear to produce short-term, specific training effects that do not generalize. Possible limitations of the review (including age differences in the samples and the variety of different clinical conditions included) are noted. However, current findings cast doubt on both the clinical relevance of working memory training programs and their utility as methods of enhancing cognitive functioning in typically developing children and healthy adults.

They do not seem to have broken out IQ specifically (and so my
meta-analysis is not *completely* redundant), but they may have done
much the same thing on pg12/21 where they have a "Forest plot for
immediate training effects on nonverbal ability", with a small overall
_d_=0.19. They did some more work, which I ought to figure out how to
do for my own meta-analysis:

> The heterogeneity between studies was significant, Q(21) = 39.17, p Ͻ .01, I^2 = 46.38%. The funnel plot indicated a publication bias to the right of the mean (i.e., studies with a higher effect size than the mean appeared to be missing), and in a trim and fill analysis, the adjusted effect size after imputation of five studies was d = 0.34, 95% CI [0.17, 0.52]. A sensitivity analysis showed that after removing outliers, the overall effect size ranged from d = 0.16, 95% CI [0.00, 0.32], to d = 0.23, 95% CI [0.06, 0.39].

Particularly important:

> Moderators of immediate transfer effects of working memory training to measures of nonverbal ability are shown in Table 2. There was a significant difference in outcome between studies with treated controls and studies with only untreated controls. In fact, the studies with treated control groups had a mean effect size close to zero. More specifically, several of the research groups demonstrated significant transfer effects to nonverbal ability when they used untreated control groups but did not replicate such effects when a treated control group was used (e.g., Jaeggi, Buschkuehl, Jonides, & Shah, 2011; Nutley, Söderqvist, Bryde, Thorell, Humphreys, & Klingberg, 2011). Similarly, the difference in outcome between randomized and nonrandomized studies was close to significance (p = .06), with the randomized studies giving a mean effect size that was close to zero. Notably, all the studies with untreated control groups are also nonrandomized; it is apparent from these analyses that the use of randomized designs with an alternative treatment control group are essential to give unambiguous evidence for training effects in this field.

The Shipstead, Redick, & Engle criticisms of WM studies seem to be
borne out by this meta-analysis...

I'm looking forward to reading Stephenson & Halpern 2012?, which Chooi
tells me used an audio-only n-back to observe lack of transfer - just
as I've been hoping to get all these years.

Between Redick et al, Melby-Lervåg & Hulme, Stephenson & Halpern, and
other forebears, it's looking like we may finally have a complete
explanation of dual n-back and IQ: there were hollow gains on visual
IQ tests from visual WM training exacerbated by lack of randomization
and comparable controls, with anecdotal evidence fed by placebo
effects.

This certainly has been an educational experience.

--
gwern
http://www.gwern.net

[jttoto2]

Interesting study.

Before I reach any conclusions, is there a copy of the full paper
available. I also have some questions about how the researchers
categorized their data. For example, what constitutes a treated
control group vs. an untreated one? Also, I seem to recall that the
big criticism of WM training research was that so few studies used a
wide variety of cognitive tests to measure far-transfer (many only
using the RAPM as a proxy), so it would make sense that there would be
no convincing evidence that WM training transfers to other skills, by
the fact that there is a lack of data to begin with.

[XFMQ902SF]

Jausovec 2012 has shown that wm training transfers to a verbal
analogies set and a paper folding test. Also in an online interview
Priti Shah has hinted to an upcoming study which shows transfer from
wm training to fluid intelligence to measures of math and reading
skill.

[jttoto2]

Yes, that was precisely my point. Lack of evidence is not evidence of
no evidence. Some researchers have found transfers to multiple
domains, so what qualifies as "not convincing" seems to be up for
interpretation. Also, the authors point out an important limitation,
the variety of conditions (which could be differences in training
method, length of training, etc.) Wouldn't it make more sense to
compare studies with identical methods, conditions, and age groups,
instead of lumping them together?

[polar]

Thanks gwern, interesting study. But may I ask, just out of curiosity, why are you so glad when studies show no transfer? Why do you look forward to it? Does it bother you when there's an open problem?

Let me remind you, that the intellectual gains after using n-back, according to both authors and participants of many studies, were definitely not "hollow" (not mentioning benefits in areas like adhd or shizophrenia, or physiological effects on dopamine). Labelling iq tests as "visual" does not make them ani less valid too. And while it is perfectly reasonable that audio training does not transfer to visual tests (audio is much less taxing), I would say opposite way would work (what a pity we dont have audio iq tests - maybe reading comprehension?). 

Regarding randomization - after Stueder-Luethi (2012) found strong correlations of iq gains and version of n-back, there's a serious probability, that different people benefit from different methods. Logically, when you randomize n-back version assignment (or you use just one version), you lose the effect. This explanation is just as valid candidate for "complete explanation of dual n-back and IQ" as it is a plain "no transfer" hypothesis.

[Gwern Branwen]

On Wed, May 23, 2012 at 7:24 AM, polar <pol...@gmail.com> wrote:
> Thanks gwern, interesting study. But may I ask, just out of curiosity, why
> are you so glad when studies show no transfer? Why do you look forward to
> it? Does it bother you when there's an open problem?

As I've pointed out before, increases in IQ are both personally
valuable and have considerable global consequences; and on the
down-side, N-back is *expensive* (at a quick estimate, my 2200
sessions mean I have sunk in >74 hours of high-quality thinking time
that could've been spent on things like programming or reading
philosophy papers over the years). The value of information that would
settle the n-back question is huge. Hence, I welcome studies that test
various criticisms and suggestions since they reduce this expensive
uncertainty.

> Let me remind you, that the intellectual gains after using n-back, according
> to both authors and participants of many studies, were definitely not
> "hollow" (not mentioning benefits in areas like adhd or shizophrenia, or
> physiological effects on dopamine).

Ooh, "many studies". How about 23 studies used in the meta-analysis,
is that "many"? Lack of hollowness needs to be proven, especially
given the reasonable doubt created by all the nulls etc...

> Labelling iq tests as "visual" does not
> make them ani less valid too.

Of course it can make them less valid, just like teaching kids how
matrix tests are constructed may reduce their validity or drilling
them on grammar can reduce the g-loading of the vocab subtests. How
many times must I point this out...

> And while it is perfectly reasonable that
> audio training does not transfer to visual tests (audio is much less
> taxing), I would say opposite way would work (what a pity we dont have audio
> iq tests - maybe reading comprehension?).

If the audio training does transfer to IQ, then it most certainly
should transfer to visual IQ tests! That's what IQ is, a latent factor
showing increased performance over many disparate domains such as
audio and visual tasks.

> Regarding randomization - after Stueder-Luethi (2012) found strong
> correlations of iq gains and version of n-back, there's a serious
> probability, that different people benefit from different methods.
> Logically, when you randomize n-back version assignment (or you use just one
> version), you lose the effect. This explanation is just as valid candidate
> for "complete explanation of dual n-back and IQ" as it is a plain "no
> transfer" hypothesis.

Note that that was just re-analyzing a n-back study which purported to
find average gains in IQ to begin with... If some people intrinsically
do not benefit, that simply drags down the averages, it doesn't make
the effect disappear completely.

BTW, one part of Redick et al 2012 I haven't mentioned was that they
tried Jaeggi's post hoc analysis of the kids; you'll remember I was
highly skeptical of it:

> A recent study with children that trained on an adaptive single n-back task identified the amount of n-back improvement as a critical variable determining whether or not transfer to intelligence was observed (Jaeggi, Buschkuehl, Jonides, & Shah, 2011). When Jaeggi et al. (2011) halved the n-back training group based on the amount of improvement observed, the children with the biggest gain showed transfer relative to an active-control group, whereas the children with smaller gains did not. We therefore attempted a similar analysis, by dividing our dual n-back subjects into high and low improvement groups, using a median split on the difference score of mean dual n- back level in sessions 19-20 versus sessions 1-2.
>
> This post-hoc analysis is limited by sample size (only 12 and 12 subjects in the high- and low-improvement groups, respectively), but with that caveat in mind, no significant Group (high dual n-back improvement, low dual n-back improvement, no-contact control) effects were obtained for the fluid intelligence, multitasking, and WM composite standardized gains (Table 7). A similar median-split analysis for the visual search group (15 and 14 subjects in the high- and low-improvement groups, respectively) also produced no significant Group effects on the composite standardized gains (Table 7).

--
gwern
http://www.gwern.net

[polar]

I understand and agree with your statements that iq (gains) are very valuable, and rigorousness in evaluating different methods is vital (be it nootropics, tcds, n-back, image streaming...). But that actually doesn't answer my question (not that you have to) - why are you negatively biased against n-back (in my opinion - feel free to deny it)? Reasons for this being, you look forward to studies with no transfer, and your FAQ is advocating null hypothesis from its beginning (for years now, since there were only few studies).

And you're right, that if there's personality factor that modulates gains from different cognitive training methods, indiscriminate meta-analysis just "drags down the averages." But remember we are mostly talking less then 10 hours of training here (!), so this can shift the effect outside the 95% confidentiality interval with ease. 

You 're right even in that teaching about how the visual test are made (or training them) definitely makes them less valid (less so with grammar and vocabulary tests, as these are usually not about grammar, but about knowing the meaning of the word, sometimes retention or semantics). Nevertheless, n-back is essentially different from any matrix iq test - there is no pattern recognition, no problem solving, no teaching of principles. It's rude drill of simple squares... to me, that is hell of a difference from matrix iq test.

And to add some more context, intelligence is not the sole thing you can gain from things like n-back - significantly better attention and memory are pretty valuable things too. And btw, our group survey has now about 180 entries, and in half of the cases people claim their intelligence improved (subjective claim of course - mostly), nearly everyone claims his attention and memory improved, and just a few people report "no real difference". In addition to former research (last one was jausovec 2012 - do you remember?), this is a very good reason for me to look forward to further n-back research. And definitely not only for null hypothesis results.

Dne středa, 23. května 2012 17:34:38 UTC+2 gwern napsal(a):

[☉]

On May 23, 2:50 pm, polar <pol...@gmail.com> wrote:
> why are you negatively biased
> against n-back (in my opinion - feel free to deny it)? Reasons for this
> being, you look forward to studies with no transfer, and your FAQ is
> advocating null hypothesis from its beginning (for years now, since there
> were only few studies).
>

I suppose we could ask the same question of those positively biased
towards n-back. Then again, I've often seen people apparently become
quite confused when someone says, "Well, guys, I just don't think the
evidence is conclusive", since the responses tend to be "What?! You
MUST BE BIASEDDDD!!!!!11111"

Looking forward to more information is never a bad thing. Doubt is
never a bad thing, unless you're one of those "gotta have faith"
types... which is a logical fallacy (i.e., appeal to faith).

>
> You 're right even in that teaching about how the visual test are made (or
> training them) definitely makes them less valid (less so with grammar and
> vocabulary tests, as these are usually not about grammar, but about knowing
> the meaning of the word, sometimes retention or semantics). Nevertheless,
> n-back is essentially different from any matrix iq test - there is no
> pattern recognition, no problem solving, no teaching of principles. It's
> rude drill of simple squares... to me, that is hell of a difference from
> matrix iq test.
>

No, n-back teaches one how to keep track of previously learned
information and to match the learned pattern with a newly presented
pattern. I'm surprised I would have to even point this out. And there
appear to be differences in aptitude for this, even though as I
suspect the variance is large enough to hide the more subtle
differences in ability to get better at the task.

>
> And to add some more context, intelligence is not the sole thing you can
> gain from things like n-back - significantly better attention and memory
> are pretty valuable things too.
>

"Can" does not imply necessity, which is one of the recurring themes
in discussions like this. People seem to attempt to construct
arguments on the matter when there is no argument to be made. Heck, I
could say that staring at a red dot on a wall for 30 minutes a day for
30 days can make you a genius, but there haven't been any studies
done. See how specious that line of argument is? Just because the
potential effects of an intervention are significant does not
therewith indicate that there will be a significant number of persons
who experience those significant effects. Oh yeah, and staring at that
dot will definitely train your attention, even if you don't become a
genius afterwards.

>
> And btw, our group survey has now about 180
> entries, and in half of the cases people claim their intelligence improved
> (subjective claim of course - mostly), nearly everyone claims his attention
> and memory improved, and just a few people report "no real difference".
>

I know that many people have come back from near-death experiences who
believe they've seen the godhead. But is that really how you want to
bias yourself towards a false positive?

I certainly would not argue against some of the immediate, perceived
effects of NB, like the fact that (including other tasks out there) it
can improve one's ability to attend to dynamic information and train
one's attention thereto, but it is a slightly different matter when we
get into I.Q. The early studies showed promise, as they were more or
less pilot studies meant to motivate further research. We're now 4
years into this game and a lot of fanfare has been made of it;
however, not very many solid results have been shown. And we shouldn't
be surprised by this, because it would be more reasonable to
anticipate null results on the basis of the huge mountain of evidence
that exists on the general topic of brain training vis-à-vis
intelligence.

argumzio

[FerrousFerriss]

> Can" does not imply necessity, which is one of the recurring themes
> in discussions like this. People seem to attempt to construct
> arguments on the matter when there is no argument to be made. Heck, I
> could say that staring at a red dot on a wall for 30 minutes a day for
> 30 days can make you a genius, but there haven't been any studies
> done. See how specious that line of argument is? Just because the
> potential effects of an intervention are significant does not
> therewith indicate that there will be a significant number of persons
> who experience those significant effects. Oh yeah, and staring at that
> dot will definitely train your attention, even if you don't become a
> genius afterwards.

Research into training using n-back has privileged further hypotheses about benefits that are independent of I.Q. -- reading comprehension, executive control for those lacking in it such as ADHD children or drug addicts. I.Q. gains seem to be treading water if not dead in it, but this isn't necessarily the end of n-back especially for certain populations. The space of possibles for n-back isn't infinitely large due to these cases. I'll take this as the notion of "can" oft confused with "may" and hope you won't consider it specious.

[polar]

Research and evidence is important? Faith is a logical fallacy? Attention

and intelligence are "slightly different matters"? You dont say, arguemzio!

Your explanation how early studies on n-back "were meant", and comparing

statements of others in this group to a person having NDE really leaves me

in awe. I look forward to your next lectures.

Dne čtvrtek, 24. května 2012 3:21:30 UTC+2 ☉ napsal(a):

[☉]

On May 23, 11:42 pm, FerrousFerriss <kenneth.bruskiew...@gmail.com>
wrote:

> Research into training using n-back has privileged further hypotheses about
> benefits that are independent of I.Q. -- reading comprehension, executive
> control for those lacking in it such as ADHD children or drug addicts. I.Q.
> gains seem to be treading water if not dead in it, but this isn't
> necessarily the end of n-back especially for certain populations. The space
> of possibles for n-back isn't infinitely large due to these cases. I'll
> take this as the notion of "can" oft confused with "may" and hope you won't
> consider it specious.
>

For the relationship between I.Q. and n-back, things are looking
rather dubious. Due to such inconclusive data, n-back is more and more
being pushed into the same crowd as all of the other "brain training"
tasks that can be found on the market; and the supposed gains to I.Q.
were (are?) the main selling point to the importance of n-back in the
first place. People seem to bewail the denial of accrued I.Q. points
as a dismissal of the task outright, but this isn't the case, and such
a reaction stems from a confusion of the issues being discussed in
threads like this.

Maybe each of us can conduct our own meta-analyses and let the great
Thinkatorium decide with an impersonal, impartial answer regarding the
observed significance of the data, which is never exhausted as we
continue to delve deeper into the wellsprings of experience. To me,
this is immensely preferable to having to deal with persons of rather
unstable constitution argue, decry, and desperately mock those who
have only made a straightforward, analogically illustrated case of the
ridiculousness of their harangues.

argumzio

[☉]

On May 24, 4:44 am, polar <pol...@gmail.com> wrote:
> You dont say, arguemzio!

All in a day's work.

argumzio

[whoisbambam]

i suspected dnb does not improve iq.

i have also suspected that nothing really seems to improve iq
(practice effect aside)

it may help for some other things.

[Gwern Branwen]

On Wed, May 23, 2012 at 3:50 PM, polar <pol...@gmail.com> wrote:
> I understand and agree with your statements that iq (gains) are very
> valuable, and rigorousness in evaluating different methods is vital (be it
> nootropics, tcds, n-back, image streaming...). But that actually doesn't
> answer my question (not that you have to) - why are you negatively biased
> against n-back (in my opinion - feel free to deny it)? Reasons for this
> being, you look forward to studies with no transfer, and your FAQ is
> advocating null hypothesis from its beginning (for years now, since there
> were only few studies).

I started the FAQ because conflicting evidence was starting to show
up, and I didn't want to lose track of it.

As I've pointed out many times before, people have always been trying
to improve IQ in normal subjects - and always failing. At this point,
just by Laplace's Law of Succession, we're at a prior that any new
claim of IQ improvements has something like a <5% chance of being
correct, no matter how well done it looks, because dozens of previous
interventions failed abysmally. And the DNB studies were pretty
quickly pointed out to have methodological problems.

If I'm a critic, perhaps it's just because so many of the supporters
are being mindless and uncritical about it. It's hard not to be a
little disgusted by some of the discussions here - "hey, have some
shrimp antioxidants!" "n-back improved my juggling! Thanks, n-back!"
"n-back is making me impotent!" "This Chinese paper with no data and a
control group that got 10 points stupider doesn't strike me as
suspicious at all!" (All real examples.)

Look at Pontus's latest bit of bile about my 'dogma'. Dogma? Is that
why the last couple studies have gone against DNB & IQ, and provided
explanations for previous positive results? If this be dogma, make the
most of it.

> And you're right, that if there's personality factor that modulates gains
> from different cognitive training methods, indiscriminate meta-analysis just
> "drags down the averages." But remember we are mostly talking less then 10
> hours of training here (!), so this can shift the effect outside the 95%
> confidentiality interval with ease.

The 95% CIs in both my meta-analysis and this one aren't that broad.

> You 're right even in that teaching about how the visual test are made (or
> training them) definitely makes them less valid (less so with grammar and
> vocabulary tests, as these are usually not about grammar, but about knowing
> the meaning of the word, sometimes retention or semantics). Nevertheless,
> n-back is essentially different from any matrix iq test - there is no
> pattern recognition, no problem solving, no teaching of principles. It's
> rude drill of simple squares... to me, that is hell of a difference from
> matrix iq test.

It's still teaching facility for manipulating sequences of blocks. On
this ML, people have talked of how they remember the pattern as a
moving "snake" - perfectly useful for trying out various movements of
dots and blocks inside a matrix test...

? And btw, our group survey has now about 180

> entries, and in half of the cases people claim their intelligence improved
> (subjective claim of course - mostly), nearly everyone claims his attention
> and memory improved, and just a few people report "no real difference".

BS anecdotes. I've pointed out well before Redick et al 2012 how
anecdotes are close to worthless, and that study is just the icing on
the cake. A thousand anecdotes are worth less than one study.

> In
> addition to former research (last one was jausovec 2012 - do you remember?),

Of course I remember http://www.gwern.net/DNB%20FAQ#jau%C5%A1ovec-2012
- I especially remember its small sample and bizarre IQ testing
procedure, as well as a questionable "active" control group. To quote
Melby:

> Ideally, an alternative active training procedure, delivered in exactly the same way, should be compared with the working memory training procedure. This controls for apparently irrelevant aspects of the training that might nevertheless affect performance. In a review of educational research Clark and Sugrue (1991) estimated that such Hawthorne or expectancy effects account for up to 0.3 standard deviations improvement in many studies. Studies that only compare working memory training with an untreated control group therefore run the risk that positive results may simply reflect expectancy effects. Although negative results from such trials would suggest that training is not effective, the reasons for such null results may be hard to interpret.

A communications course is not remotely similar.

> And to add some more context, intelligence is not the sole thing you can
> gain from things like n-back - significantly better attention and memory are

> pretty valuable things too....

> this is a very good reason for me to look forward to further n-back
> research. And definitely not only for null hypothesis results.

You do, huh? Let me just requote from my original email and add emphasis:

> For verbal working memory, these near-transfer effects were not sustained at follow-up, whereas for visuospatial working memory, limited evidence suggested that such effects might be maintained. More importantly, *there was no convincing evidence of the generalization of working memory training to other skills (nonverbal and verbal ability, inhibitory processes in attention, word decoding, and arithmetic)*.

--
gwern
http://www.gwern.net

[Gwern Branwen]

On Wed, May 23, 2012 at 3:50 PM, polar <pol...@gmail.com> wrote:

> Reasons for this being, you look forward to studies with no transfer, and
> your FAQ is advocating null hypothesis from its beginning (for years now,
> since there were only few studies).

And incidentally, this is not true. The very first version of the FAQ
didn't even mention Moody:

Wed Mar 25 18:57:26 EDT 2009 gwe...@gmail.com
* +N-back FAQ
addfile ./N-back\32\FAQ.page
hunk ./N-back\32\FAQ.page 1
-
+# B-back training
+
+## Should I do multiple daily sessions, or just one?
+
+[Most users](http://groups.google.com/group/brain-training/browse_thread/thread/5e989058544ad122)
seem to go for one long N-back session, citing the idea that this
forces long-term concentration. Others do one session in the morning
and one in the evening so they can focus better on each one.
+
+More than that and you're probably wasting time due to overhead,
and may not be getting enough exercise in each session.
+
+## When should I do my session?
+
+There is [some research
support](http://groups.google.com/group/brain-training/browse_thread/thread/43b64f72bc6dabf0)
that doing N-back before bed is better than most other times; ["Sleep
Accelerates the Improvement in Working Memory
Performance"](http://www.jneurosci.org/cgi/content/abstract/28/40/10145)
([PDF](http://brainworkshop.sourceforge.net/10145.pdf):
+
+> "Working memory (WM) performance, which is an important factor
for determining problem-solving and reasoning ability, has been firmly
believed to be constant. However, recent findings have demonstrated
that WM performance has the potential to be improved by repetitive
training. Although various skills are reported to be improved by
sleep, the beneficial effect of sleep on WM performance has not been
clarified. Here, we show that improvement in WM performance is
facilitated by posttraining naturalistic sleep. A spatial variant of
the n-back WM task was performed by 29 healthy young adults who were
assigned randomly to three different experimental groups that had
different time schedules of repetitive n-back WM task sessions, with
or without intervening sleep. Intergroup and intersession comparisons
of WM performance (accuracy and response time) profiles showed that
n-back accuracy after posttraining sleep was significantly improved
compared with that after the same period of wakefulness, independent
of sleep timing, subject's vigilance level, or circadian influences.
On the other hand, response time was not influenced by sleep or
repetitive training schedules. The present study indicates that
improvement in n-back accuracy, which could reflect WM capacity,
essentially benefits from posttraining sleep."
+
+## Are strategies good or bad?
+
+People [frequently](http://groups.google.com/group/brain-training/browse_thread/thread/fd08b93399939fb9)
[ask](http://groups.google.com/group/brain-training/browse_thread/thread/d779edc2b922db95)
[and](http://groups.google.com/group/brain-training/browse_thread/thread/676676a8e34f56d3)
[discuss](http://groups.google.com/group/brain-training/browse_thread/thread/c4eaff3059759531)
whether they should use some sort of strategy, and if so, what.
+
+Many N-backers adopt an 'intuition' strategy. Rather than
explicitly rehearsing sequences of letters ('f-up, h-middle; f-up,
h-middle; g-down, f-up...'), they simply think very hard and wait for
a feeling that they should press 'a' (audio match), or 'l' (location
match). Some, like SwedishChef can be quite vociferous [about
it](http://groups.google.com/group/brain-training/browse_thread/thread/d779edc2b922db95):
+
+
+> "The challenges are in helping people understand that dual-n-back is
+NOT about remembering n number of visual and auditory stimuli. It's about
+developing a new mental process that intuitively recognizes when
it has seen
+or heard a stimuli n times ago."<br>
+> "Initially, most students of dual n-back want to remember n
items as fast as they can so they can conquer the dual-n-back hill.
They use their own already developed techniques to help them remember.
They may try to hold the images in their head mentally and review
them every time a new image is added and say the sounds out loud and
review the sounds everytime a new sound is added. This is NOT what we
want. We want the brain to learn a new process that intuitively
recognizes if an item and sound was shown 3 back or 4 back. It's sort
of like playing a new type of musical instrument.<br>
+> I've helped some students on the site try to understand this.
It's not about how much you can remember, it's about learning a new
process. In theory, this new process translates into a better working
memory, which helps you make connections better and faster."
+
+Jaeggi herself is [much more
moderate](http://groups.google.com/group/brain-training/browse_thread/thread/08bc4ee2ccd0df80)[](http://groups.google.com/group/brain-training/browse_thread/thread/955524caaf2e9001):
+
+> "I would NOT recommend you [train the visual and auditory task
separately]
+if you want to train the dual-task (the one we used in our study). The
+reason is that the combination of both modalities is an entirely different
+task than doing both separately! If you do the task separately, I
assume you
+use some "rehearsal strategies", e.g. you repeat the letters or positions
+for yourself. In the dual-task version however, these strategies might be
+more difficult to apply (since you have to do 2 things simultaneously...),
+and that is exactly what we want... We don't want to train strategies, we
+want to train processes. Processes that then might help you in the
+performance of other, non-trained tasks (and that is our ultimate
goal). So,
+it is not important to reach a 7- or 8-back... It is important to fully
+focus your attention on the task as well as possible.<br>
+> "I can assure you, it is a very tough training regimen.... You
can't divert
+your attention even 1 second (I'm sure you have noticed...). But
eventually,
+you will see that you get better at it and maybe you notice that you are
+better able to concentrate on certain things, to remember things more
+easily, etc. (hopefully)."
+
+> "this is a question i am being asked a lot and unfortunately, i
don't really know whether i can help with that. i can only tell you
what we tell (or rather not tell) our participants and what they tell
us. so, first of all, we don't tell people at all what strategy to use
- it is up to them. thing is, there are some people that tell us what
you describe above, i.e. some of them tell us that it works best if
they don't use a strategy at all and just "let the squares/letters
flow by". but of course, many participants also use more conscious
strategies like rehearsing or grouping items together. but again - we
let people chose their strategies themselves!"
+
+But it may make no difference. Even if you are engaged in a
complex mnemonic-based strategy, you're still working your memory.
Strategies may not even work; quoting from Jaeggi's 2008 paper:
+
+> "By this account, one reason for having obtained transfer
between working memory and measures of Gf is that our training
procedure may have facilitated the ability to control attention. This
ability would come about because the constant updating of memory
representations with the presentation of each new stimulus requires
the engagement of mechanisms to shift attention. Also, our training
task discourages the development of simple task-specific strategies
that can proceed in the absence of controlled allocation of
attention."
+
+Hopefully even if a trick lets you jump from 3-back to 5-back,
Brain Workshop will just keep escalating the difficulty until you are
challenged again. It's not the level you reach, but the work you do.
+
+# And the flashing right/wrong feedback?
+
+A matter of preference, although those in favor of disabling the
visual feedback (`SHOW_FEEDBACK = False`) seem to be
[slightly](http://groups.google.com/group/brain-training/browse_thread/thread/775fba46e8c163f1)
[more](http://groups.google.com/group/brain-training/browse_thread/thread/420f332fcd4317c9)
vocal or numerous. Brain Twister
[apparently](http://groups.google.com/group/brain-training/browse_thread/thread/a8934cc1e04075f9)
doesn't give feedback.[Jaeggi
says](http://groups.google.com/group/brain-training/browse_thread/thread/955524caaf2e9001):
+
+> "the gaming literature also disagrees on this issue - there are
different ways to think about this: whereas feedback after each trial
gives you immediate feedback whether you did right or wrong, it can
also be distracting as you are constantly monitoring (and evaluating)
your performance. we decided that we wanted people to fully and
maximally concentrate on the task itself and thus chose the approach
to only give feedback at the end of the run. however, we have newer
versions of the task for kids in which we give some sort of feedback
(points) for each trial. thus - i can't tell you what the optimal way
is - i guess there are interindividual differences and preferences as
well."
+
+
+## How can I do better on N-back?
+
+Focus harder. Play more. Sleep well, and eat healthily. The less
stressful you are, the better you can do.
+
+## Am I wasting time if I can't get past 3-back?
+
+Not at all! The crucial thing about N-back is just that you are
stressing your working memory, that's all. The actual level doesn't
matter very much, just whether you can barely manage it; it is
somewhat like lifting weights, in that regard. From Jaeggi 2008:
+
+> "The finding that the transfer to Gf remained even after taking
the specific training effect into account seems to be
counterintuitive, especially because the specific training effect is
also related to training time. The reason for this capacity might be
that participants with a very high level of n at the end of the
training period may have developed very task specific strategies,
which obviously boosts n-back performance, but may prevent transfer
because these strategies remain too task-specific (5, 20). The
averaged n-back level in the last session is therefore not critical to
predicting a gain in Gf; rather, it seems that working at the capacity
limit promotes transfer to Gf."
+
+Mailing list members [report
benefits](http://groups.google.com/group/brain-training/browse_thread/thread/eacf724158e04506)
even if they have plateaued at 3 or 4-back.
+
+## I heard 12-back is possible
+
+Some users have reported being able to go all the way up to 13-back.
+
+Ashirgo [offers
up](http://groups.google.com/group/brain-training/browse_thread/thread/72aa2de2fdf75edf/6c777b2646227c38?q)
her 8-point scheme as to how to accomplish such feats:
+
+> 1. Be focused at all cost. The fluid intelligence itself is sometimes
+called "the strength of focus".
+> 2. You had better not rehearse the last position/sound . It will
+eventually decrease your performance! I mean the rehearsal "step by
+step": it will slow you down and distract. The only rehearsal allowed
+should be nearly unconscious and "effortless" (you will soon realize
+its meaning :)
+> 3. Both points 1 & 2 thus imply that you must be focused on the most
+current stimulus as strongly as you can. Nevertheless, you cannot
+forget about the previous stimuli. How to do that? You should hold the
+image of them (image, picture, drawing, whatever you like) in your
+mind. Notice that you still do not rehearse anything that way.
+> 4. Consider dividing the stream of data (n) on smaller parts. 6-back
+will be then two 3-back, for instance.
+> 5. Follow square with your eyes as it changes its position.
+> 6. Just turn on the Jaeggi mode with all the options to ensure your
+task is closest to the original version.
+> 7. Consider doing more than 20 trials. I am on my way to do no less
+than 30 today. It may also help.
+> 8. You may lower the difficulty by reducing the fall-back and advance
+levels from >75 and =<90 to 70 and 85 respectively (for instance).
+
+## Does it really work?
+
+### N-back improves working memory
+
+There are quite a few studies showing significant increases in
working memory. WM is something that can be trained.
+
+http://lib.bioinfo.pl/pmid:17597168
+http://www.nature.com/neuro/journal/v7/n1/abs/nn1165.html
+
+### Measuring with IQ tests
+
+Because N-back is supposed to improve your pure '[fluid
intelligence](!Wikipedia)', and not, say, your English vocabulary, the
most accurate tests are going to be ones that avoid vocabulary or
literature or tests of subject area knowledge. That is,
'culture-neutral' IQ tests. (A non-neutral test is likely to overstate
your true IQ, and minimize any gains or losses in your IQ due to
N-back training, so scores on them are not very informative.) As one
ML member writes:
+
+> "The WAIS test involves crystallized intelligence and is
unsuitable for judging fluid intelligence. High working memory will
not spawn the ability to solve complex mathematical and verbal
problems on its own, you have to put your extended capacity to
learning. All very-high-level IQ tests are largely crystallized IQ
tests, therefore working memory gains will not be immediately apparent
by their measure."[](http://groups.google.com/group/brain-training/browse_thread/thread/8af44f3b20df9904)
+
+The gold-standard of culture-neutral IQ tests is [Raven's
progressive matrices](!Wikipedia). Unfortunately, Raven's is not
available for free online, but there are a number of clones one can
use - bearing in mind their likely inaccuracy, and remembering that if
you are testing at the beginning and end of your training there is
probably going to be a [practice effect](!Wikipedia). They are:
+
+- [iqtest.dk](http://iqtest.dk/main.swf)
+- [A Spanish site](http://www.clipsite.com.ar/HOME/Salud/Test/Raven/)
+- [Mensa Norway](http://mensa.no/olavtesten/#)
+- [Quasi-ravens (unnormed?)](http://www.knowl.demon.co.uk/page111.html)
+- [Queendom.com's "Culture Fair IQ
Test"](http://queendom.com/tests/access_page/index.htm?idRegTest=1112)
+- <http://www.clipsite.com.ar/HOME/Salud/Test/Raven/Principal.asp>
+- <http://www.iqtest.com/> (not free)
+- [High IQ Society Online Test](http://www.highiqsociety.org/iq_tests/)
+
+If Raven-style tests bore you or you've gone through the previous
ones, there are a wealth of difficult tests at Miyaguchi's
["Uncommonly Difficult IQ Tests"](http://www.eskimo.com/~miyaguch/).
+
+### IQ test results
+
+
+Reports of IQ tests have been mixed. Some results have been stunning:
+
+> "LSaul [posted
about](http://groups.google.ca/group/dualnback/browse_thread/thread/97b2340497476ecc/9959b6da18f8fbea)
his apparent rise in IQ back in October. From what I remember, he had
recently failed to qualify for MENSA, which requires a score of about
131 (98th percentile). He then got a 151 (99.97th percentile) on a
professionally administered IQ test (WAIS) three months later, after 2
months of regular dual-n-back use."
[MR](http://groups.google.com/group/brain-training/browse_thread/thread/8af44f3b20df9904)
+
+Some have not:
+
+> "I took the Online Denmark IQ test again [after N-back
training] and I got 140 (the same
+result)<br>
+> I took a standardized (and charged) online IQ test from www.iqtest.com
+and I got 134 (though it may be a bit higher because English is not my
+mother tongue)"
+
+Tofu [writes](http://groups.google.com/group/brain-training/browse_thread/thread/d1e53e8c69c95c3a):
+
+> "I've purposely not been doing anything to practice for the
tests or anything else I thought could increase my score so I wouldn't
have to factor other things into an improvement in iq, which makes
improvements more likely attributable to dual n-back. Before I took
the test I scored at 117, a score about 1 in about 8 people can get
(7.78 to be exact), and yesterday I scored at 127 (a score that 1 in
28 people would get). Its a pretty big difference I would say."
+
+Keep in mind, that if IQ is improved, that doesn't necessarily
mean anything unless one employs it to some end. It would be a shame
to boost one's IQ through N-back, but never use it because one was too
busy playing!
+
+### Benefits
+
+[Jack Nguyen
asked](http://groups.google.com/group/brain-training/browse_thread/thread/14c843162167c376)
what benefits people saw from N-back use. The responses (and quotes
from other threads) are, of course, entirely anecdotal, so take them
as you will.
+
+- Ashirgo: "To be honest, I do not feel any obvious difference.
There are moments in which I perceive a significant improvement,
though, as well as particulars task which are much easier now." "I
have also experienced better dream recalling, with all these reveries
and other hallucinations included ;) I am more happier now than ever.
I did doubt it would be ever possible! I am also more prone to get
excited...Now people in my motherland are just boring to listen to.
They speak too slow and seem as though it took them pains to express
anything. I did not notice that after I had done my first ninety days
of n-back, but now (after 2.5 months) it is just
conspicuous."[](http://groups.google.com/group/brain-training/browse_thread/thread/1c44c7570cdb4a35)
+
+- chinmi04: "For me, it definitely has taught me how to focus.
But I'm still not sure whether that has something to do with merely
coming to realize the importance of focusing, or whether the program
has really physically rewired my brain to focus better. In any case,
it appears that I'm now faster at mental reasoning, creative thinking
and speaking fluency. But again, the effects are not so clear as to
completely eliminate any doubt regarding the connection with the
n-back program." "I have been maintaining a personal blog on wordpress
since 3 years ago. Average post per month : a little over 1. Then I
started with dual-n-back at the end of november... number of posts in
January : 7! (none are about n-back)"
+- Confuzedd: "[asked if felt 'sharper']: Nothing."
+- ArseneLupin: "Not much, yet, but I feel that I can easier get a
hold of a discussion. The feeling is the same as when I am mastering a
certain n-back in the game (a bit hard to explain)."
+- John: "I feel much sharper since I started in the middle of
last November...My productivity is much higher these days. I'm a
non-fiction writer, so having a higher working memory and fluid
intelligence directly leads to better (and faster) performance. It's
amazing to see the stuff I produce today and compare it to before I
began the Dual N-Back training. Also, I am simultaneously learning
German, French and Spanish, and I'm certain this is helping me learn
those languages faster."
+- Ginkgo: "DN-Back has probably helped me with one of my hobbies."
+- BamaDoc: "I note a subjective difference in recall. There
might be some increase in attention, but I certainly do notice a
difference in recall. It might be placebo, but I am convinced enough
that I continue to find time to use the
program."[](http://groups.google.com/group/brain-training/browse_thread/thread/f11ff07eb9eba1a4)
+- karnautrahl: "Since November however, I began to read the
Neuroscience book in more detail. I mentioned late December I think
that I was finding I could understand more stuff. I've spent about
[_<U+00C2>_][_<U+00A3>_]1000 on books since November. The large
majority are books on the brain, source from Amazon reviews, reading
lists and out of my own pirate list when I liked a book. I stopped
Dual n Back in December, early. The benefits have stayed however. I
tested this the other day, very easily going to 3 n back, which was
mostly where I was before. I guess in a way I'm trying to say that for
me, whilst the focus may have been on G increase and IQ etc, now the
focus is on--what's *really* happened and what can I do with it :).
What I can do with it is choose to concentrate long enough to
genuinely understand fairly technical in depth chapters on subjects
often new to me."[](http://groups.google.com/group/brain-training/browse_thread/thread/7a674cf0305a6f5c)
"After not using this since around December last year, I still
attribute my vastly improved concentration to DNB training...Some are
degree textbooks or for med school students. I'm having no real
trouble working through any of these....This isn't a case of how smart
I became because I definitly have no comparison for this as I'm not on
a course nor am I eligable for any degree placements at this time.
What it is though, is my own personal testimony as to how much greater
concentration I have than I ever
had."[](http://groups.google.com/group/brain-training/browse_thread/thread/a9e8c326d95f0da6)
+- negatron: "One perhaps coincidental thing I noticed is that
dream recollection went up substantially. A good while after I stopped
I developed an odd curiosity for what I previously considered
unpleasant material, such as advanced mathematics. Never imagined I'd
consider the thought of advanced calculus exciting. I began reading up
on such subjects far more frequently than I used to. This was well
after I've long forgotten about dual n-back so I find it hard to
attribute it to a placebo effect, believing that I'm more adapted to
this material. On the other hand I don't recall reading anything about
motivational benefits to dual n-back training so I still consider this
conjecture and perhaps an eventful coincidence just the
same."[](http://groups.google.com/group/brain-training/browse_thread/thread/7a674cf0305a6f5c)
+- Chris: "One thing I have noticed is the recollection of a
number of very unpleasant images in dreams. Specifically, images of
bodily disease, mutilation, injury and post-mortem decomposition. I
find it difficult to believe it's just a coincidence, because I can't
remember when I last had such a dream, and I've had maybe half a dozen
since I started dual n-back. But perhaps it's simply owing to better
recall."[](http://groups.google.com/group/brain-training/browse_thread/thread/7a674cf0305a6f5c)
+- sutur: "i didn't really notice any concrete changes in my
thinking process, which probably, if existent, are rather hard to
detect reliably anyway. one thing i did notice however is an increased
sense of calmness. i used to move my legs around an awful lot while
sitting which i now don't feel the urge to anymore. but of course this
could be placebo or something else entirely. i also seem to be able to
read text (in books or on screen) more fluently now with less danger
of distraction. however, personally i am quite skeptic when people
describe the changes they notice. changes in cognitive capacity are
probably quite subtle, build up slowly and are hard to notice through
introspection."[](http://groups.google.com/group/brain-training/browse_thread/thread/1c44c7570cdb4a35)
+- astriaos: "By 'robust', I mean practically everything I do is
qualitatively different from how I did things 30 days previous to the
dual n-back training. For instance, in physics class I went from
vaguely understanding most of the concepts covered in class to a
mastery thorough enough that now my questions usually transcend the
scope of the in-class and textbook material, routinely stupefying my
physics teacher into longer-than-average pauses. It's the same
experience for all of my classes. Somehow, I've learned more-than-I
usually learn of physics/government/ etc. (all of my classes, and any
topic in general) information from sources outside of class, and
without what I consider significant effort. I feel like my learning
speed has gone up by some factor greater than 1; I can follow longer
arguments with greater precision; my vocabulary has improved; I can
pay attention longer; my problem solving skills are significantly
better... Really, it's amazing how much cognition depends on
attention!"[](http://groups.google.com/group/brain-training/browse_thread/thread/1c44c7570cdb4a35)
+- flashquartermaster reports N-back cured his [chronic fatigue
syndrome](http://groups.google.com/group/brain-training/browse_thread/thread/84d227fee313b60a)?
+
+# What's some relevant research?
+
+The main Jaeggi studies (2003, and 2008) are available in the
Group's [Files](http://groups.google.com/group/brain-training/files)
folder, as is the McNab 2009 study showing physical changes to
dopamine neurochemistry after N-back training.
+
+- ["Training and transfer effects of executive functions in preschool
+children"](http://www.klingberglab.se/pub/Thorell2008.pdf):
+
+> "Executive functions, including working memory and inhibition,
are of central importance to much of human behavior. Interventions
intended to improve executive functions might therefore serve an
important purpose. Previous studies show that working memory can be
improved by training, but it is unknown if this also holds for
inhibition, and whether it is possible to train executive functions in
preschoolers. In the present study, preschool children received
computerized training of either visuo-spatial working memory or
inhibition for 5 weeks. An active control group played commercially
available computer games, and a passive control group took part in
only pre- and posttesting. Children trained on working memory improved
significantly on trained tasks; they showed training effects on
non-trained tests of spatial and verbal working memory, as well as
transfer effects to attention. Children trained on inhibition showed a
significant improvement over time on two out of three trained task
paradigms, but no significant improvements relative to the control
groups on tasks measuring working memory or attention. In neither of
the two interventions were there effects on non-trained inhibitory
tasks. The results suggest that working memory training can have
significant effects also among preschool children. The finding that
inhibition could not be improved by either one of the two training
programs might be due to the particular training program used in the
present study or possibly indicate that executive functions differ in
how easily they can be improved by training, which in turn might
relate to differences in their underlying psychological and neural
processes."
+
+- ["Common and unique components of inhibition and working
memory: An fMRI, within-subjects
investigation"](http://www.klingberglab.se/pub/McNab2008.pdf)
+
+> "Behavioural findings indicate that the core executive
functions of inhibition and working memory are closely linked, and
neuroimaging studies indicate overlap between their neural correlates.
There has not, however, been a comprehensive study, including several
inhibition tasks and several working memory tasks, performed by the
same subjects. In the present study, 11 healthy adult subjects
completed separate blocks of 3 inhibition tasks (a stop task, a
go/no-go task and a flanker task), and 2 working memory tasks (one
spatial and one verbal). Activation common to all 5 tasks was
identified in the right inferior frontal
+gyrus, and, at a lower threshold, also the right middle frontal
gyrus and right parietal regions (BA 40 and BA 7). Left inferior
frontal regions of interest (ROIs) showed a significant conjunction
between all tasks except the flanker task. The present study could not
pinpoint the specific function of each common region, but the parietal
region identified here has previously been consistently related to
working memory storage and the right inferior frontal gyrus has been
associated with inhibition in both lesion and imaging studies. These
results support the notion that inhibitory and working memory tasks
involve common neural components, which may provide a neural basis for
the interrelationship between the two systems."
+
+http://www.ncbi.nlm.nih.gov/pubmed/18686052
+
+- Huijbers et al. ["When Learning and Remembering Compete: A
Functional MRI Study"](http://dx.doi.org/10.1371/journal.pbio.1000011).
PLoS Biology, 2009; 7 (1):
+
+> "Recent functional neuroimaging evidence suggests a bottleneck
between learning new information and remembering old information. In
two behavioral experiments and one functional MRI (fMRI) experiment,
we tested the hypothesis that learning and remembering compete when
both processes happen within a brief period of time. In the first
behavioral experiment, participants intentionally remembered old words
displayed in the foreground, while incidentally learning new scenes
displayed in the background. In line with a memory competition, we
found that remembering old information was associated with impaired
learning of new information. We replicated this finding in a
subsequent fMRI experiment, which showed that this behavioral effect
was coupled with a suppression of learning-related activity in visual
and medial temporal areas. Moreover, the fMRI experiment provided
evidence that left mid-ventrolateral prefrontal cortex is involved in
resolving the memory competition, possibly by facilitating rapid
switching between learning and remembering. Critically, a follow-up
behavioral experiment in which the background scenes were replaced
with a visual target detection task provided indications that the
competition between learning and remembering was not merely due to
attention. This study not only provides novel insight into our
capacity to learn and remember, but also clarifies the neural
mechanisms underlying flexible behavior."
+
+- Psychol Sci. 2008 Sep;19(9):881-8.
+["Gaining control: training executive function and far transfer
of the ability to resolve
interference"](http://www2.psychology.su.se/staff/jpn/papers/PerssonJ_PsychologicalScience_2008.pdf).
Persson J, Reuter-Lorenz PA:
+
+> "Functional brain-imaging data document overlapping sites of
activation in prefrontal cortex across memory tasks, suggesting that
these tasks may share common executive components. We leveraged this
evidence to develop a training regimen and a set of transfer tasks to
examine the trainability of a putative executive-control process:
interference resolution. Eight days of training on high-interference
versions of three different working memory tasks increased the
efficiency with which proactive interference was resolved on those
particular tasks. Moreover, an improved ability to resolve
interference was also transferred to different working memory,
semantic memory, and episodic memory tasks, a demonstration of
far-transfer effects from process-specific training. Participants
trained with noninterference versions of the tasks did not exhibit
transfer. We infer that the transfer we demonstrated resulted from
increased efficiency of the interference-resolution process.
Therefore, this aspect of executive control is plastic and adaptive,
and can be improved by training."
+
+(There's also a worthwhile blog article on this one: ["Training
The Mind: Transfer Across Tasks Requiring Interference
Resolution"](http://scienceblogs.com/developingintelligence/2008/10/training_the_mind_transfer_acr.php)
+
+["How distractible are you? The answer may lie in your working
memory capacity"](http://scienceblogs.com/cognitivedaily/2009/03/how_distractible_are_you_the_a.php)
+
+- Jennifer C. McVay, Michael J. Kane (2009). "Conducting the
train of thought: Working memory capacity, goal neglect, and mind
wandering in an executive-control task". Journal of Experimental
Psychology: Learning, Memory, and Cognition, 35 (1), 196-204 DOI:
[10.1037/a0014104](http://dx.doi.org/10.1037/a0014104):
+
+> "On the basis of the executive-attention theory of working
memory capacity (WMC; e.g., M. J. Kane, A. R. A. Conway, D. Z.
Hambrick, & R. W. Engle, 2007), the authors tested the relations among
WMC, mind wandering, and goal neglect in a sustained attention to
response task (SART; a go/no-go task). In 3 SART versions, making
conceptual versus perceptual processing demands, subjects periodically
indicated their thought content when probed following rare no-go
targets. SART processing demands did not affect mind-wandering rates,
but mind-wandering rates varied with WMC and predicted goal-neglect
errors in the task; furthermore, mind-wandering rates partially
mediated the WMC-SART relation, indicating that WMC-related
differences in goal neglect were due, in part, to variation in the
control of conscious thought."
+
+The relation of caffeine to learning & memory is complicated; for
now, see [the thread on
it](http://groups.google.com/group/brain-training/browse_thread/thread/74991713608a29a5).
+
+# Software
+
+## Brain Workshop
+
+### Configuring
+
+#### Fullscreen
+
+Edit data/config.ini as usual; edit it to read `USE_FULLSCREEN = True`.
+
+## 'Official' N-back
+
+The program used in the Jaeggi studies, [Brain
Twister](http://www.apn.psy.unibe.ch/content/application/braintwister),
is available to the public. You can get something very similar to the
commercial game by enabling 'Jaeggi mode' in the data/config.ini file,
by editing the `JAEGGI_MODE = False` field.
+
+Other commercial programs include 'Soak Your Head' or 'Brain
Fitness Pro'. Given prices like
[_<U+00E2>_][_<U+0082>_][_<U+00AC>_]40, though, they'd have to be
awfully good to beat BW's free.
+
+## N-back online
+
+There are many implementations in Flash etc. online; one is
<http://dual-n-back.com/>. <http://cognitivefun.net/test/22>
+
+## N-back on the iPhone
+
+See <http://tnxbai.com/> and <http://neurosnack.com/>.
+
+## Offline N-back
+
+You can play N-back in the real world, without a computer, if you
like. See the ML thread ["Non-electronic game version of N-back
task"](http://groups.google.com/group/brain-training/browse_thread/thread/e85b55de47df536d)
and the [SnapBack
rules](http://www.toothycat.net/wiki/wiki.pl?DouglasReay/SnapBackGameRules).
+
+# What else can I do?
+
+Forum members have recommended a number of other things for
general mental fitness.
+
+- Buddhist-style meditation has been recommended (there is a good
Vipassana textbook available online; see ["Mindfulness in Plain
English"](http://www.urbandharma.org/udharma4/mpe.html), and the
<http://openfocus.com/> website has been mentioned).
+- [Spaced repetition](!Wikipedia) programs such as
[Mnemosyne](!Wikipedia "Mnemosyne (software)") are very useful for
memorizing & remembering things
+- Crypto [recommends](http://groups.google.com/group/brain-training/browse_thread/thread/8af44f3b20df9904)
[image streaming](!Wikipedia) as another mental exercise
+- Exercise is right up there with nutrition and sleep!

Moody didn't appear until Sun May 31 22:50:59 EDT 2009.

--
gwern
http://www.gwern.net

[hallu]

On Sunday, May 27, 2012 5:35:12 PM UTC+2, gwern wrote:

It's still teaching facility for manipulating sequences of blocks. On
this ML, people have talked of how they remember the pattern as a
moving "snake" - perfectly useful for trying out various movements of
dots and blocks inside a matrix test...

This is not entirely correct, because the sequencing in n-back is too primitive to bear an entire Gf test on its own (and skimming iqtest.dk reveals only ~10 sequencing question out of 39). Besides that, given 6 being the average position length, it's child's play compared to the sequencing complexity of 'Go'.

[hallu]

Patterns are made from data residing inside matrix frames, not from sequencing the order of those frames. It is possible, that some people are not aware about certain problems having the frames being randomly distributed across the entire matrix. In order to solve such question, proper sequencing has to be established first. It won't be possible to find a pattern without searching for and recognizing frame pairs first. Position n-back can solve this visual deficiency by inducing involuntary search for random frame pairs due to the position randomly jumping across the grid. So after playing position n-back the subject may, instead of stubbornly pairing the first frame with the second frame or neglecting frames from other rows (first and second frames have no connections with each other, but the test taker is not aware of it, he thinks he can't find the pattern only), engage himself into non-linear frame pairing and thus start scoring higher on Gf tests. 

[☉]

On May 28, 9:37 am, hallu <halluc...@gmail.com> wrote:
> On Monday, May 28, 2012 1:35:34 PM UTC+2, hallu wrote:
> > This is not entirely correct, because the sequencing in n-back is too

> > primitive to bear an entire Gf test on its own (and skimming iqtest.dkreveals only ~10 sequencing question out of 39). Besides that, given 6

> > being the average position length, it's child's play compared to the
> > sequencing complexity of 'Go'.
>

I don't think one needs a one-to-one correspondence between the tasks
to realize that there isn't very much "far-transfer" to speak of
between them. And even if there were a rigorously defined and
worthwhile meaning to the term being used in the research, not much of
it has demonstrated such an effect.

> It is possible, that some people are
> not aware about certain problems having the frames being randomly
> distributed across the entire matrix. In order to solve such question,
> proper sequencing has to be established first.

Yes, and one can sequence the ordered movements of an n-length n-back
pattern as a "randomly distributed" pattern across certain parts of
the matrix. [Identifiable pattern removed.] Certain patterns are not
uncommon and are not that hard to recognize, as they refer to the
global pattern of the matrix itself, which still remains not a random
pattern, but a highly organized one. In other words, for all one knows
any given matrix is "random", but eventually the intelligence, or
ability, one has will detect the order thereof and correctly select
the appropriate item for the missing part of the matrix. That is,
there is no such thing as a "random matrix" in an intelligence test.
If there were, it wouldn't be a test of intelligence, because "random"
suggests that there is no optimal answer.

> It won't be possible to find
> a pattern without searching for and recognizing frame pairs first.
> Position n-back can solve this visual deficiency by inducing involuntary
> search for random frame pairs due to the position randomly jumping across
> the grid. So after playing position n-back the subject may, instead of
> stubbornly pairing the first frame with the second frame or neglecting
> frames from other rows (first and second frames have no connections with
> each other, but the test taker is not aware of it, he thinks he can't find
> the pattern only), engage himself into non-linear frame pairing and thus
> start scoring higher on Gf tests.

Unfortunately, openly discussing how certain items are designed on
such tests helps those with higher verbal ability artificially score
higher on such tests - inducing the Flynn effect, far more than has
arguably already been done by tasks like n-back. In any case, it
sounds like you understand that n-back isn't that different from
matrix-style tests in how it might suggest to a player who otherwise
would not have realized the connection: to think of the matrix in more
than one way (even if they are equivalent) so as to better understand
it and choose the appropriate response.

Anyway, I think the connection is too obvious for us to take seriously
the idea that there is any far-transfer going on. Whatever is going
on, it's near-transfer at the most.

argumzio

[Gwern Branwen]

Full text: http://www.apa.org/pubs/journals/releases/dev-ofp-melby-lervag.pdf

I've noticed apa.org fulltexts sometimes seem to disappear after a few
months; if you're interested, download. (As usual, it's been entered
into my archive queue so it *should* be available in future years in
the Internet Archive or WebCite, but no guarantees.)

--
gwern
http://www.gwern.net

[Green]

   I read the study. I didn't think it was that damning.

1.) The authors did not find any evidence of publication bias. Although I don't understand the technical statistical tool they used to make this determination (e.g. funnel plotting), I had always feared that positive results in these cognitive training studies were happening due to type I errors getting published and legitimate null results being buried. The authors indicate that this is not the case. Hurray!

2.) As the authors point out - all the studies that were accepted were very short-term (usually less than 12 hours total of training),   (I'm not sure there was any at all). I think you have to accept that if WM training has real benefits for cognition, those benefits are not going to emerge with <=12 hours of training, unless 
In real life, developing a skill requires more practice than this. I think about how quickly you forgot the math you learned in high school or college, or how quickly you become 'rusty' if you don't practice the guitar, or piano, or singing, or whatever skill-hobby you work with. The commercial training protocols that this meta-analysis examined are being marketed to parents who have children with ADHD, and so they promise unrealistically quick results because more realistic projections would be unmarketable.

3.) The same point as 2.), but applied to practice for skill-retention: if you don't practice a skill after you've acquired it, you will lose your gains. This could explain why they found no evidence that the gains remained 9 months after training. But, again, common sense suggests this.

But the study has some genuinely promising remarks:


" Meta-analyses indicated that the programs produced reliable short-term improvements in working memory skills."   ....And they found similar, albeit smaller gains in visual-spatial WM.

And also this:

"For attention (inhibition in the Stroop task), there is a small to moderate effect immediately after training..."

These three facts are promising: evidence that training can actually improve some aspects of cognition.

In sum, the meta-analysis is looking at two types of data-points: results from the use commercially marketed training protocols and experiments designed to prove theoretical points about cognition. I submit that neither form of data is appropriate to decide whether WM training can effect real-world performance. In light of this, it should be considered promising that the meta-analysis found any positive results at all.

-Green

P.S.

Full disclosure: my personal theory is that WM is just one aspect of an executive-attention system that includes at least two other processes: inhibition and task-switching. Almost all real-world tasks involve two or more of these processes. (see Miyake et al. 2001) Therefore, WM trainers may not get IQ or other far-transfer effects because of a bottleneck effect. I am agnostic about whether Dual N-back can improve IQ, but I am sympathetic to the idea that there exists a set of training tasks that, done together over long periods of time, would improve real-world performance at a variety of G-loaded tasks.

[polar]

Nice that you report they found no publication bias, its never granted these days (or rather it never was nor can be). And I totally agree with the notion "shortterm wm gains, longterm gf/g gains." Regarding executive functions, I'm fond of the trio "updating, shifting, inhibition" ( as elaborated e.g. in http://pss.sagepub.com/content/17/2/172.short :)

Dne pátek, 15. června 2012 4:06:20 UTC+2 Green napsal(a):

[Gwern Branwen]

On Thu, Jun 14, 2012 at 10:06 PM, Green <dmuc...@gmail.com> wrote:
> 1.) The authors did not find any evidence of publication bias. Although I
> don't understand the technical statistical tool they used to make this
> determination (e.g. funnel plotting), I had always feared that positive
> results in these cognitive training studies were happening due to type I
> errors getting published and legitimate null results being buried. The
> authors indicate that this is not the case. Hurray!

Neither did I in my little meta-analysis; but while we're
congratulating everyone, let's remember the limits: this merely shows
that a thorough and complete search of databases for even the
obscurest study finds that there is no clear signal of publication
bias. We can still have a de facto sort of publication bias if
everyone pays attention only to the 'hits'. For example, if one were
to try to do a mass media check of publication bias, one would find it
rampant - everyone discussing Jaeggi 2008 (yesterday Google Alerts
turned up a 2012 Swiss newspaper article on it with no reference to
other studies!) and no one discussing the others.

> 2.) As the authors point out - all the studies that were accepted were very
> short-term (usually less than 12 hours total of training),   (I'm not sure
> there was any at all). I think you have to accept that if WM training has
> real benefits for cognition, those benefits are not going to emerge with
> <=12 hours of training, unless
> In real life, developing a skill requires more practice than this.

Even after 12 hours of dedicated training, an effect should be
noticeable, especially with >20 studies to draw on. If you had a bunch
of kids who all knew the rules of chess and had played one or two
games, and you gave 12 hours of chess instruction and problem solving
and match playing to one group, would you expect to see a difference
in ELOs between them? Especially if you did this with 20 studies
covering hundreds of kids? I would.

> I think
> about how quickly you forgot the math you learned in high school or college,
> or how quickly you become 'rusty' if you don't practice the guitar, or
> piano, or singing, or whatever skill-hobby you work with.

But when they resume the activity, their skill levels quickly ramp up
and they are easily distinguished from people who never learned it in
the first place.

> 3.) The same point as 2.), but applied to practice for skill-retention: if
> you don't practice a skill after you've acquired it, you will lose your
> gains. This could explain why they found no evidence that the gains remained
> 9 months after training. But, again, common sense suggests this.

I didn't look at the longevity part... I can't imagine very many
studies were doing 9 month followups, so I don't know how strong this
result would be. It may be present only in kids, due to their peers
catching up in development.

--
gwern
http://www.gwern.net

[Green]

On Friday, June 15, 2012 3:25:11 AM UTC-5, polar wrote:

  Regarding executive functions, I'm fond of the trio "updating, shifting, inhibition" ( as elaborated e.g. in http://pss.sagepub.com/content/17/2/172.short :)

 Good point. Miyake calls it "shifting" not "task-switching," and updating is more or less the same thing as working memory.

[Green]

 

Even after 12 hours of dedicated training, an effect should be
noticeable, especially with >20 studies to draw on. If you had a bunch
of kids who all knew the rules of chess and had played one or two
games, and you gave 12 hours of chess instruction and problem solving
and match playing to one group, would you expect to see a difference
in ELOs between them? Especially if you did this with 20 studies
covering hundreds of kids? I would.

   Yes, there should be some observable effects for 12 hours of training. But the study supported those findings. For example:

"In summary, working memory training produces large immediate
gains on measures of verbal working memory.."
 
and...

"working memory training produces moderately
sized immediate gains on measures of visuospatial working memory"




> I think

> about how quickly you forgot the math you learned in high school or college,
> or how quickly you become 'rusty' if you don't practice the guitar, or
> piano, or singing, or whatever skill-hobby you work with.

But when they resume the activity, their skill levels quickly ramp up
and they are easily distinguished from people who never learned it in
the first place.

 Right, but tests don't give you a chance to 'ramp up'.You might take the test cold after a nine month delay, and perform poorly.

 


 

--
gwern
http://www.gwern.net

[Gwern Branwen]

On Fri, Jun 15, 2012 at 2:15 PM, Green <dmuc...@gmail.com> wrote:
>    Yes, there should be some observable effects for 12 hours of training.
> But the study supported those findings. For example:
>
> "In summary, working memory training produces large immediate
> gains on measures of verbal working memory.."
>
> and...
>
> "working memory training produces moderately
> sized immediate gains on measures of visuospatial working memory"

You're trying to have your cake and eat it too. If 12 hours training
on average is enough to find 'large' and 'moderately sized' gains on
metrics A and B, how are you going to argue that C is suddenly
unreliable and gosh 12 hours just isn't enough to see any gains on it?

--
gwern
http://www.gwern.net

[Green]

    
But, I think the metrics may measure underlying processes that are differentially sensitive to the training.  To make an analogy, if a scientist was measuring the effects of tobacco exposure, and she gave 40 people 4 weeks of exposure to tobacco, and then looked at three things: A = blood pressure at 4 weeks, B = blood pressure at 6 weeks C = death rate at 6 weeks, she would find that A changed a great deal relative to her control group, B did a little, and C did not change at all. If she knew nothing else about tobacco, she might then type up a report that concludes "while tobacco exposure changes health biomarkers in the short-term, we found little evidence that it actually impacts health in a long-term, meaningful way." But this would be because of two things: 1) some of the harmful effects of tobacco fade after cessation of use, and 2) the exposure wasn't long enough to alter the death rate in a detectable way.

[Thales]

Another possible issue with this study is how they divided up post-training results as either measures of near-transfer or far-transfer. For instance, in this study they found that "working memory training produces large immediate gains on measures of verbal working memory." However, these gains were only considered examples of "near-transfer" and thus were not considered evidence of useful cognitive improvement.

I'm fairly sure though in most other studies the results from post-training measures of working memory always were taken as evidence of a significant kind of transfer as long as they were not identical the task being trained. So my concern then is that they seem to be employing a stricter concept of "transfer" than other leading researchers, and that as a result, certain post-training gains that other researches would consider more than near transfer are being clumped into the near-transfer (useless) category.

[Green]

 

So my concern then is that they seem to be employing a stricter concept of "transfer" than other leading researchers, and that as a result, certain post-training gains that other researches would consider more than near transfer are being clumped into the near-transfer (useless) category.

   I agree with you. The authors themselves made an effort to point out that 'near-transfer' is still, legitimately, a form of transfer. But they take issue with the near-transfer results as well, pointing out that there are two serious shortcomings to the near-transfer results: "(a) studies that test near transfer
using STM tasks and (b) near transfer tasks that closely resemble the method of training." They seem to say that the studies overwhelmingly suffered from these defects. Only a few studies used very different, untrained WM tasks to test for near-transfer, and those generally did not find positive results.

   Rereading the article again, I see much support for Gwern's original pessimism - the results, even for near-transfer are really sporadic and don't fit into any kind of meaningful trend. The authors did a good job of pointing out that even positive results often don't make any sense within the context of the theory - for example, when a study finds far-transfer to attention or Gf, but NO near-transfer to WM, how can you take that as good news? The theory says that near-transfer to WM is the force that drives far-transfer to attention and Gf, so mixed results only disconfirm the theory.  I would be inclined to agree that future research should look for a new theoretical model to explain transfer - and such a model is likely to be one in which far-transfer occurs in more narrowly defined ways. An obvious downer for people who want to train at one specific kind of WM task in order to boost global measures of intellectual functioning like IQ.

     Cogmed has a response the study posted on their website.

    http://www.cogmed.com/working-memory-training-effective-comments-media-headlines

    It includes this remark:

    "[Cogmed] studies show strong results...But that cannot offset the results from the other studies that have used other kinds of working memory training, on other groups and in other ways. This is why the authors sweepingly conclude that working memory training does not work: that when all programs, protocols and populations are looked at together, the results are not strong. This is a somewhat obvious finding, and again, does not say much about how effective Cogmed training is for those people it is designed for.

     It is a reasonable point - their program, which uses a wider variety of training tasks, does have better results than the other studies. But I'm not sure how independent Cogmed studies are. 

[Alex C]

Cogmed response also talks about how Cogmed works for ppl who are "constrained by their working memory capacity". So how do I find if am constrained by WM? Are Cogmed type evaluation exams available on net? 

 

Eight of them are referred to as “CogMed”, and only four of those studies refer to relevant populations (individuals constrained by their working memory capacity). In other words, only four of the 30 studies in the entire meta-analysis are relevant for how Cogmed is used in real life, in schools and in health care practices around the world.

[hallu]

On Friday, June 15, 2012 7:58:23 PM UTC+2, Green wrote:

  Regarding executive functions, I'm fond of the trio "updating, shifting, inhibition" ( as elaborated e.g. in http://pss.sagepub.com/content/17/2/172.short :)

 Good point. Miyake calls it "shifting" not "task-switching," and updating is more or less the same thing as working memory.

Miyake (2000*) separates shifting from task-switching and finds them not being related to each other in any way. Friedman (2006*) shows that shifting an inhibition have meager connections with intelligence.

* The unity and diversity of executive functions and their contributions to complex "Frontal Lobe" tasks: a latent variable analysis

* Not All Executive Functions Are Related to Intelligence

[Green]

  
  Friedman (2006*) shows that shifting an inhibition have meager connections with intelligence.

   That illustrates the gap between instruments that measures Gf and overall real-world ability. A person can have normal Gf but still be incompetent at carrying out activities of daily living. We see this in people who have certain kinds of frontal-lobe brain damage. Pen-and-paper tests of intelligence do not necessarily measure the kind of abilities most people would hope to improve through cognitive training protocols.

 

[hallu]

The current instruments presumably predict real-world ability. Similar results were obtained in Duan 2010[1].

Duan 2010[1]  

Further analysis indicated that these three EFs were correlated to intelligence in 11-

to 12-year-old children differently, with the updating most closely correlated to intelli-

gence. SEM revealed that when inter-EFs correlations were considered, the correlations

between updating and intelligence measures were undiminished, but the correlations

between inhibiting and intelligence and between shifting and intelligence were no longer

significant.

Brydges 2012[2]

Duan et al. (2010) reported that inhibition and updating both

significantly predicted gF in children

Miyake & Friedman 2012[3]

(...) new way of examining individual differences in EFs (...) after accounting for the unity

(common EF), there is no unique variance left for inhibition (...) Stated differently, the inhibition

factor happens to correlate virtually perfectly with common EF, leaving no inhibition-specific variance.

The first two quotes are meant to evoke a kind of LOL response. Third hints inhibition is the least important EF to train. Last one about relationship between intelligence and rational thinking.

Toplak 2010[4]

Overall, only a small proportion of the studies reported a

statistically significant relationship between IGT performance and these cognitive abilities. The majority of

studies reported a non-significant relationship. Of the minority of studies that reported statistically significant

effects, effect sizes were, at best, small to modest, and confidence intervals were large, indicating that

considerable variability in performance on the IGT is not captured by current measures of executive function and

intelligence. These findings highlight the separability between decision-making on the IGT and cognitive abilities,

which is consistent with recent conceptualizations that differentiate rationality from intelligence.

(...)

Conceptual separation aside, there is ample empirical evidence

that measures of intelligence and measures of rationality show

considerable dissociation. Some measures of rational thought show

modest correlations with cognitive ability (in the range of .20 to .35),

but many rationality tasks show no association with cognitive ability

[1] The relationship between executive functions and intelligence on 11- to 12-year-old children

[2] A unitary executive function predicts intelligence in children

[3] The Nature and Organization of Individual Differences in Executive Functions Four General Conclusions

[4] Decision-making and cognitive abilities: A review of associations between Iowa Gambling Task performance, executive functions, and intelligence

[Green]

 

   These are some interesting studies you've found. You've done some great research.

    I'm not claiming that measures of intelligence do not correlate with real-world ability. Under normal circumstances, a person who scores higher than average on an IQ test will also score above average on a test of executive function, and will be generally more competent at most real-world tasks.  Because nature generally packages the two together – just like hearts generally come packaged with kidneys.  But what improves the function of the heart does not necessarily improve the function of the kidney.   

     The research you've cited supports the idea that EF and IQ can come apart. A person can have high IQ, but be impulsive. A person can have high IQ but be unable to easily shift gears from one task to another. Such persons may be smart but largely dysfunctional. Although he uses novel constructs, this is what Toplak 2012 seems to be saying:

Conceptual separation aside, there is ample empirical evidence

that measures of intelligence and measures of rationality show

considerable dissociation. Some measures of rational thought show

modest correlations with cognitive ability (in the range of .20 to .35),

but many rationality tasks show no association with cognitive ability

We've all met people for whom intelligence and rationality have come apart. Think of the guy who majored in math in college but believes that 9/11 was an inside job, and makes bad decisions with his personal or financial life. This guy is smart, but not rational.  The further the two come apart the less good either does.

Now, look at these other quotes:

Brydges 2012[2]

Duan et al. (2010) reported that inhibition and updating both

significantly predicted gF in children

EFs will predict intelligence. Because nature tends to (but doesn't always) package EFs and intelligence together.  

Duan 2010[1]  

Further analysis indicated that these three EFs were correlated to intelligence in 11-

to 12-year-old children differently, with the updating most closely correlated to intelli-

gence. SEM revealed that when inter-EFs correlations were considered, the correlations

between updating and intelligence measures were undiminished, but the correlations

between inhibiting and intelligence and between shifting and intelligence were no longer

significant.

When they adjusted for the fact that every EF correlates with every EF to some extent (package deal again), they found that inhibition and shifting no longer correlated with intelligence.

Miyake 2012 reported something more nuanced:

"The path coefficient between updating and intelligence is significant,
indicating that they share about 35% variances, p < .01; that between inhibition and
intelligence is marginally significant, indicating that they share about 19% variances, p <
.1, and that between shifting and intelligence is not significant, indicating that they only
share about 7% variances"

 Only 35 % of the variance between updating and intelligence is shared - meaning there is a lot of potential for someone's Z-score on one test to be higher than the other. It's worse for the other EFs - inhibition and intelligence only share 19 % variance, and the link between shifting and intelligence is only 7 %.

   With regards to the importance of training inhibition, the research is unclear whether inhibition is distinct from the other EFs. Some studies suggest it is, some suggest it is not. But if inhibition is not distinct because it underlies the other EFs, then training it independently would still potentially be a good investment.  In that case, training inhibition would potentially act as a 'two-fer' - boosting shifting and updating skills.  

    As a caveat, I'm not a psychologist and I'm not as briefed on the literature as I'd like to be. I haven’t always been as clear as I could be, and so I appreciate what you’ve done to raise the quality of this discussion by pointing out what’s unclear or mistaken. I hope we can continue to have a conversation and develop some of these ideas further together.

[jttoto2]

Assuming intelligence and EF are mutually exclusive (I'm not saying it is), which is more important?  EF or intelligence?  Those with higher EF tend to make better grades, more than IQ, and the impact of intelligence on GPA is more equivocal (The SAT is a different story, with intelligence having a clear benefit)  Higher EF means less prone to depression, OCD, or psychosis as well.  http://www.sciencedirect.com/science/article/pii/S0885200609000386

On Tuesday, May 22, 2012 2:12:04 PM UTC-4, gwern wrote:

> It has been suggested that working memory training programs are effective both as treatments for attention-deficit/hyperactivity disorder (ADHD) and other cognitive disorders in children and as a tool to improve cognitive ability and scholastic attainment in typically developing children and adults. However, effects across studies appear to be variable, and a systematic meta-analytic review was undertaken. To be included in the review, studies had to be randomized controlled trials or quasi-experiments without randomization, have a treatment, and have either a treated group or an untreated control group.
>
> 23 studies with 30 group comparisons met the criteria for inclusion. The studies included involved clinical samples and samples of typically developing children and adults. Meta-analyses indicated that the programs produced reliable short-term improvements in working memory skills. For verbal working memory, these near-transfer effects were not sustained at follow-up, whereas for visuospatial working memory, limited evidence suggested that such effects might be maintained. More importantly, there was no convincing evidence of the generalization of working memory training to other skills (nonverbal and verbal ability, inhibitory processes in attention, word decoding, and arithmetic). The authors conclude that memory training programs appear to produce short-term, specific training effects that do not generalize. Possible limitations of the review (including age differences in the samples and the variety of different clinical conditions included) are noted. However, current findings cast doubt on both the clinical relevance of working memory training programs and their utility as methods of enhancing cognitive functioning in typically developing children and healthy adults.

They do not seem to have broken out IQ specifically (and so my
meta-analysis is not *completely* redundant), but they may have done
much the same thing on pg12/21 where they have a "Forest plot for
immediate training effects on nonverbal ability", with a small overall
_d_=0.19. They did some more work, which I ought to figure out how to
do for my own meta-analysis:

> The heterogeneity between studies was significant, Q(21) = 39.17, p Ͻ .01, I^2 = 46.38%. The funnel plot indicated a publication bias to the right of the mean (i.e., studies with a higher effect size than the mean appeared to be missing), and in a trim and fill analysis, the adjusted effect size after imputation of five studies was d = 0.34, 95% CI [0.17, 0.52]. A sensitivity analysis showed that after removing outliers, the overall effect size ranged from d = 0.16, 95% CI [0.00, 0.32], to d = 0.23, 95% CI [0.06, 0.39].

Particularly important:

> Moderators of immediate transfer effects of working memory training to measures of nonverbal ability are shown in Table 2. There was a significant difference in outcome between studies with treated controls and studies with only untreated controls. In fact, the studies with treated control groups had a mean effect size close to zero. More specifically, several of the research groups demonstrated significant transfer effects to nonverbal ability when they used untreated control groups but did not replicate such effects when a treated control group was used (e.g., Jaeggi, Buschkuehl, Jonides, & Shah, 2011; Nutley, Söderqvist, Bryde, Thorell, Humphreys, & Klingberg, 2011). Similarly, the difference in outcome between randomized and nonrandomized studies was close to significance (p = .06), with the randomized studies giving a mean effect size that was close to zero. Notably, all the studies with untreated control groups are also nonrandomized; it is apparent from these analyses that the use of randomized designs with an alternative treatment control group are essential to give unambiguous evidence for training effects in this field.

The Shipstead, Redick, & Engle criticisms of WM studies seem to be
borne out by this meta-analysis...

I'm looking forward to reading Stephenson & Halpern 2012?, which Chooi
tells me used an audio-only n-back to observe lack of transfer - just
as I've been hoping to get all these years.

Between Redick et al, Melby-Lervåg & Hulme, Stephenson & Halpern, and
other forebears, it's looking like we may finally have a complete
explanation of dual n-back and IQ: there were hollow gains on visual
IQ tests from visual WM training exacerbated by lack of randomization
and comparable controls, with anecdotal evidence fed by placebo
effects.

This certainly has been an educational experience.

--
gwern
http://www.gwern.net

[hallu]

Duckworth, A. L., & Seligman, M. E. P. (2005). Self-discipline outdoes IQ in predicting academic performance of adolescents. Psychological Science, 16(12), 939–944. doi:10.1111/j.1467-9280.2005.01641.x

When it comes to predicting student achievement, does self-
discipline outdo IQ? In Study 2, we found that correlation co-
efficients between self-discipline and most achievement indi-
cators were significantly higher than and at least twice the size of
correlations between IQ and the same outcomes. Also, the
standardized regression coefficient of self-discipline was more

than twice that of IQ in a simultaneous multiple regression

predicting final GPA. These results suggest that, indeed, self-

discipline has a bigger effect on academic performance than

does intellectual talent.

Moffitt, T. E., Arseneault, L., Belsky, D., Dickson, N., Hancox, R. J., Harrington, H., Houts, R., et al. (2011). A gradient of childhood self-control predicts health, wealth, and public safety. Proceedings of the National Academy of Sciences of the United States of America, 108(7), 2693–2698. doi:10.1073/pnas.1010076108

Following a cohort of 1,000

children from birth to the age of 32 y, we show that childhood self-

control predicts physical health, substance dependence, personal

finances, and criminal offending outcomes, following a gradient

of self-control. Effects of children’s self-control could be disen-

tangled from their intelligence and social class as well as from

mistakes they made as adolescents. In another cohort of 500

sibling-pairs, the sibling with lower self-control had poorer out-

comes, despite shared family background.

Miyake, A., & Friedman, N. P. (2012). The Nature and Organization of Individual Differences in Executive Functions Four General Conclusions. Current Directions in Psychological Science, 21(1), 8–14. doi:10.1177/0963721411429458

The better-self-restraint

group demonstrated significantly better common EF (by .60

standard-deviation, SD, units), virtually no difference in the

updating-specific ability (.05 SDs better), and significantly

worse shifting-specific ability (.42 SDs worse) than did the

worse-self-restraint group. (....)

The evidence of worse shifting-specific ability associated

with better childhood self-restraint may be counterintuitive,

but it illustrates something we alluded to earlier: The two com-

ponents of shifting ability—common EF and shifting-specific

abilities—sometimes show opposing patterns of correlations

with other constructs. As Goschke (2000) suggested, the abil-

ity to actively maintain a single task goal may indeed be a

force that makes it difficult for individuals to flexibly switch

to a different goal.