Review of dual n-back for IQ and working memory

[Mark A. Smith]

Here is a useful review of the latest meta-analyses looking at the working memory (e.g. dual n-back) and executive control training.

http://www.hrplab.org/brief-review-of-2014-2015-studies-on-working-memory-training-for-iq-and-working-memory/

This kind of training has had an unjustifiably bad press in recent years.

[T. Lavon Lawrence]

Thanks!

--
You received this message because you are subscribed to the Google Groups "Dual N-Back, Brain Training & Intelligence" group.
To unsubscribe from this group and stop receiving emails from it, send an email to brain-trainin...@googlegroups.com.
To post to this group, send email to brain-t...@googlegroups.com.
Visit this group at http://groups.google.com/group/brain-training.
For more options, visit https://groups.google.com/d/optout.

--

T. Lavon Lawrence
Author, INDISTRACTIBLE!© Brain Fitness System
Dynamic Mental Fitness Training

[Brain Train]

thank you, Mark, for bringing something inspiring, encouraging!

it helps us in continuing our brain development journey through various brain-training exercises.

[polar]

Thank you Dr. Smith, for your informed meta-analysis - it echoes my opinion on the matter (after few years of scientific inquiry). Yet, I have two questions:

1) The point of meta-analysis is to include any available and relevant studies on the subject. Actually, manipulating criteria of study inclusion is always suspect, often even a loophole in methodology. Why did you choose time (years 2014+2015) as a criterion for your analysis? It excludes tens of previous studies, thus becoming a selective summary rather than meta-analysis. 

2) As I said, I definitely believe that cognitive training (and n-back in particular) holds much promise in improving general cognitive abilities. But I also KNOW, that transfer effects are far from being a rock-solid universal truth. Now, when you as a scientist use the word "guaranteed" on your web (by which I believe you mean financial guarantees, not factual) its borderline deceptive advertising, borderline unethical. You must know that big _negative_ statements about efficacy are an easy target, you label them "sensationalist" and I agree. So why do you think that "guarantee of 10-20 IQ point increase in three weeks" is not sensationalist? Put like this, its light years away from scientific consensus, actually again borderline unethical AND an easy target for skeptics ("a pessimist is a person who has had to listen to too many optimists"). So how do you justify this bold claim to yourself?

[Mark]

The choice of 2014-15 was simply because the past two or three years have seen a lot of new studies which these meta-analyses include in their analysis. The more recent the better provided it is comprehensive. Also the more recent meta-reviews incorporate all the 'best practices' for meta-analysis. And do you know of any other meta-analyses for working memory training for IQ?

The review article is a scientific one that should enable readers to judge for themselves. There is an appendix added for basic experimental design concepts. If you've followed the links through in the 'which apps' section to our own software that's fine, but it shouldn't detract from the purpose of this article which is to do justice to the current literature - something that is not done in the popular scientific press. The default view seems to be (unjustifiably) that 'brain training doesn't work'. We have developed software based on the science which offers credible training.  For our own software, the guarantee is is similar to 'guaranteed satisfaction or your money back'  Since the 'raw' IQ gain is around 7 points with traditional n-back training (no one disagrees with this result in the meta-analyses), with additional executive control and interference training (which is built into the IQ Mindware apps), this is a perfectly reasonable aspiration for most people, and a reasonable promise.

To repeat the point, the popular press is now generally anti-brain training. One or two studies have swayed the debate in a way that is not justified. Brain training has considerable potential, but programs need to be more rigorous than you get with e.g. Lumosity or Elevate. Then you can expect some real benefits. Here are a couple of other reviews - one for brain training for over-50s. The other for brain training for depression and anxiety.

http://www.hrplab.org/review-brain-training-depression-anxiety/

http://www.hrplab.org/brain-training-for-over-50s/

Cheers,

Mark

[polar]

So your intention was to emphasize research done in specific time period, because it yields results you like. I like those results too, but it's misleading to call it "meta-analysis", if you omit tens of studies with not-so-desirable outcomes. To amend your argument, when you need comprehensiveness (all studies), then each subset is equally essential (be it "most recent" or "least recent").

Also, if you target layman readers, it's unfair to expect they can judge your scientific article "for themselves" - that's the one thing they can't do. And man, is there plenty of disagreement on effectiveness of cognitive training - even in meta-analyses and especially on the amount. Au et al. states Gf effect size 0.24, Schwaighofer 0.14 (that's 42% difference!), which translates to 3.6 or 2.1 points of IQ respectively - so what stable 7 points are you talking about? Even not taking into account other factors (disputed Au, not all tests standardized for sd=15, or that further meta-analyses are underway - I myself was requested by 2 other scientists to share my data in last year) - to advertise a precise number of IQ points improvement is borderline ridiculous.

So, we probably see this differently, but you unfortunately confirmed my doubts about the amount of science in your claims (be it those "2014-2015 meta-analysis" or marketing texts). Therefore I don't consider productive to discuss this topic further with you. Nevertheless, thanks for an interesting summary, and especially for the "anxiety improvement" link (I'm also a proponent of CT influencing personality - and even vice versa, here's my dissertation if you're interested http://goo.gl/jDUl0O ).

Cordially,

VM

[Mark]

No - there is a difference of course between 'studies' and 'meta-analyses'. The meta-analyses I used are the only ones I know of (they happen to have been published between 2013-2015), and they review ALL the published studies on the topic going back a decade or more. There is no cherry picking going on.

So your intention was to emphasize research done in specific time period, because it yields results you like. I like those results too, but it's misleading to call it "meta-analysis", if you omit tens of studies with not-so-desirable outcomes. To amend your argument, when you need comprehensiveness (all studies), then each subset is equally essential (be it "most recent" or "least recent").

 There is no disagreement in meta-analyses about the effectiveness of dual n-back training for working memory for instance, whether visuo-spatial or verbal. And WM is a core component of IQ. The IQ estimates you mention are both significant afterall. The 7 points is simply the difference between post-test results and pre-test IQ (fluid intelligence) tests, and no one is doubting this for dual n-back training. This can't be all 'placebo' as Au et al and active control studies (the ones you quote) show, and as you can see from the latest Lumosity study on their own training programs - see 'progressive matrices'. 

The green is the total gain from post-test to pre-test after about 10 hours of Lumosity training, comparable to the time period of dual n-back training. The absolute gains are more or less half that of dual n-back, but you have the same placebo problem. So there is nothing suspect about advertising a 10 point IQ gain when you're also integrating interference control which is known to have a strong link to IQ. What are we supposed to do? Not try to promote this kind of intervention (compared to e.g. Lumosity approaches), when we can see it has a bunch of advantages for people?  I've chosen to spend my entire professional career developing this kind of intervention based on all the science we have - by choice - and I take issue that I'm engaged in some some cheap commercial 'false advertising'..Jaeggi and her lab have done something similar.

 

Also, if you target layman readers, it's unfair to expect they can judge your scientific article "for themselves" - that's the one thing they can't do. And man, is there plenty of disagreement on effectiveness of cognitive training - even in meta-analyses and especially on the amount. Au et al. states Gf effect size 0.24, Schwaighofer 0.14 (that's 42% difference!), which translates to 3.6 or 2.1 points of IQ respectively - so what stable 7 points are you talking about? Even not taking into account other factors (disputed Au, not all tests standardized for sd=15, or that further meta-analyses are underway - I myself was requested by 2 other scientists to share my data in last year) - to advertise a precise number of IQ points improvement is borderline ridiculous.

 

Mark

[Mark]

You're welcome:)

[Mark A. Smith]

I will have a look at your manuscript thanks.

[Mark]

So to summarize:

We can conclude based on the meta-reviews of all the published experiments that IQ gains from training are real – for both younger and older adults.

• Estimates of the overall training benefit for IQ (both verbal and non-verbal) range between 2.0 – 5.5 standardized points (1,2,3 )
• Estimates of the overall training benefit for IQ when we only look at experiments in which the comparison/control group is ‘passive’ and does no computer activity is ~7 IQ points (1)

Most criticism of the training-for-IQ literature hinges on the active vs passive control issue – and whether gains on IQ measures  (which there certainly are) are due to a placebo effect or not. Counterarguments to this criticism include the following:

• Au et al (2015)  have a strong case that there is no evidence that the type of control groupper se moderates the effects of working memory training on measures of fluid intelligence.
• Other meta-reviews that conclude there is a real IQ gain explicitly address the active vs passive control issue (1). For example –

om Karbach & Verhaeghen (2014)

• There is little controversy in the literature that working memory training results in gains in attention control and working memory. Even the IQ-gain skeptics Melby-Lervag and Hulme in their 2013 meta-analysis concluded that working memory training resulted in visuo-spatial and verbal working memory gains. Given the functional overlap of these cognitive abilities and IQ it would be surprising if there was no IQ gain.
• Studies that look a cortical network and synaptic neuroplasticity effects from WM and CC training are consistent with wide transfer interpretations of the behavioral data.
• There is evidence that in the US particularly  over the last 25 years the difference in effectiveness between real drugs and placebo ones has narrowed considerably, suggesting that Americans are particularly susceptible to the placebo effect (5, 6). Geographical location is confounded with active controls in the meta-analyses. This could mask a true effect on Gf.
• Typical active control tasks such as perceptual discrimination or attention control may themselves involve activation of shared neural networks with working memory training (2,3, 4), resulting in far transfer IQ gains.
• Active control tasks may result in a hormesis response  – a response that is known to benefit brain function (see work by Mark Mattson) and may improve Gf  in a way that is not additive.
• I have not here addressed the Bayesian analysis critique by Dougherty and colleagues (2015) here, but this only concludes that in passive control conditions there is a clear IQ increase effect, and in active control conditions we don’t know if there is an effect or not.

What is needed before placebo criticisms can be regarded as a serious challenge is evidence for a placebo effect in the form of experiments where expectations are systematically varied, or adding a third group to the controlled trial set-up, which takes an existing intervention that is known to work – if both that group and the group given the effective intervention fail to beat the placebo, researchers know that their trial design is flawed. It’s important to do studies that directly look at neuroplasticity effects in brain mechanisms too.

Based on the 2014-2015 meta-analyses and commentaries we can conclude that there are not sufficient grounds to discredit the claim that working memory training is an effective and efficient strategy for improving IQ.

---

Whether you choose Brain Workshop, or CogMed or IQ Mindware software, there is a point to training with the dual n-back for IQ increases, despite the bad press. And as the review explains, there are substantial gains with working memory that last for several months after training due to neurplasticity change in the fronto-parietal network.

[polar]

I didn't want to respond, but I have to apologize, I thought you were doing your own meta-analysis, while you only reported those which I cite too.

Nevertheless, you follow with a flood of copy/pasted texts from your webpage, 90% of that convincing me that cognitive training really works (or rather everybody else, because I agree with that). The problem is, that you "guarantee" 10-20 points of IQ improvement on your webpage, based on 1) the most optimistic number out of four meta-analyses 2) added speculations ("interference should add at least three points") 3) and multiplying this wild fantasy of 10 points by factor of 1.5 (to get 10-20).

This indeed is "cheap commercial 'false advertising'", regardless of the fact that "Jaeggi and her lab have done something similar".

It's great that you try to improve human cognition, many of us do. But as a scientist, please don't lie to yourself and the public, that your software can improve 15 point of IQ in few weeks.

[jotaro]

why wouldnt he lie?

the point to make those claims, is to make his product more attractive

so he can sell more.

now, it is based on lies but nowdays everyone fucking lies.

--

[Mark]

Can you see from the subsequent response what your kind of criticism results in? Good quality scientific critique get's instantly labelled as 'commercially motivated' when it couldn't be further from the truth. You yourself, with some scientific training, didn't even read the article. If you had, you'd have seen instantly that it was based on the latest meta-analyses. The graphs themselves show this in an instant. So what happens is that a few seconds are devoted to something that takes weeks of work, and are instantly classified as commercial lies and deception. That's a great service to the debate! My software is nothing to do with this article in practice. I keep track of sales inn my company that has been running for four years now. Over the week that this has been released I have had precisely ONE sale. That's how commercially angled this article is. It's simply pointless engaging in forums such as this one when this is the kind of response to what is in fact an accomplished contribution to the debate. Look at the science and critique - not my software! If you want something free for IQ, go to Brain Workshop - that's great. But as a science minded person don't get deflected from the main arguments, or you simply open the floodgates for characters like 'JokyBoy' and so on - which presumably is not what you want. I've got better things to do with my time than engage in this kind of forum! If I wanted to 'be commercial' it's certainly the least likely route for me to do it! Believe me, this kind of article leaves the general public dry. It doesn't motivate people to buy software, and it's not intended to. 

[Mark]

And if you want to step in the commercial world - the real one, not a student's or scientist's version of one - then OF COURSE there is going to be some marketing hype and bias on the actual website that sells the product. Try to find one commercial brain training company that does not do this - does not promise something that sounds appealing, does not use their best recommendations, does not try to sell the product! That's the point of marketing. But one advantage businesses have over academics in the commercial world when it comes to things like cognitive or health interventions is that they are focused on 'delivery' which academics aren't - i.e.methods for allowing information or training methods to be 'consumed' in a way that is effective in a broad market. 90% of people simply wouldn't have the motivation to go through 20 sessions of standard dual n-back - I know because I keep track of user stats. That's why there are efforts to gamify interventions.  That's also why you see simplified infographics and catchy explanations in commercial settings and not detailed arguments about active vs passive controls and detailed graphs and so on. It just draws a blank.

On Monday, November 23, 2015 at 12:37:20 PM UTC, polar wrote: