"Working Memory Training Does Not Improve Performance on Measures of Intelligence", Melby-Lervag et al 2016

[Gwern Branwen]

"Working Memory Training Does Not Improve Performance on Measures of
Intelligence or Other Measures of 'Far Transfer': Evidence From a
Meta-Analytic Review" http://pps.sagepub.com/content/11/4/512.full
http://pps.sagepub.com/content/suppl/2016/07/28/11.4.512.DC1/RedickPPSsupplemental.pdf
, Melby-Lervag et al 2016:

> It has been claimed that working memory training programs produce diverse beneficial effects. This article presents a meta-analysis of working memory training studies (with a pretest-posttest design and a control group) that have examined transfer to other measures (nonverbal ability, verbal ability, word decoding, reading comprehension, or arithmetic; 87 publications with 145 experimental comparisons). Immediately following training there were reliable improvements on measures of intermediate transfer (verbal and visuospatial working memory). For measures of far transfer (nonverbal ability, verbal ability, word decoding, reading comprehension, arithmetic) there was no convincing evidence of any reliable improvements when working memory training was compared with a treated control condition. Furthermore, mediation analyses indicated that across studies, the degree of improvement on working memory measures was not related to the magnitude of far-transfer effects found. Finally, analysis of publication bias shows that there is no evidential value from the studies of working memory training using treated controls. The authors conclude that working memory training programs appear to produce short-term, specific training effects that do not generalize to measures of “real-world” cognitive skills. These results seriously question the practical and theoretical importance of current computerized working memory programs as methods of training working memory skills.

--
gwern
http://www.gwern.net

[Nathan]

This might be the final nail in the coffin for me; I think I will focus solely on typing training from now on. I highly agree with this statement from you:

To those whose time is limited: you may wish to stop reading here. If you seek to improve your life, and want the greatest ‘bang for the buck’, you are well-advised to look elsewhere. Meditation, for example, is easier, faster, and ultra-portable. Typing training will directly improve your facility with a computer, a valuable skill for this modern world. Spaced repetition memorization techniques offer unparalleled advantages to students. Nootropics are the epitome of ease (just swallow!), and their effects are much more easily assessed - one can even run double-blind experiments on oneself, impossible with dual N-back. Other supplements like melatonin can deliver benefits incommensurable with DNB - what is the cognitive value of another number in working memory thanks to DNB compared to a good night’s sleep thanks to melatonin? Modest changes to one’s diet and environs can fundamentally improve one’s well-being. Even basic training in reading, with the crudest tachistoscope techniques, can pay large dividends if one is below a basic level of reading like 200WPM & still subvocalizing. And all of these can start paying off immediately.

It wonder if there has been a study on the combined effect of real world training and DNB training. Then you would potentially train the real world skills to utilize a growing WM. Just like how an athlete doesn't simply lift weights but also does a host of other activities which vary in their relatedness to his profession, all of which synergize together.

My conclusion on DNB is similar to yours Gwern in that I believe DNB fills the same role as Go, Chess, Juggling, or Painting. The one benefit I think it does provide is satisfaction of the visual-spatial/creative/exploratory need that many people have. Clearly many of us are in that boat.

[Fourier]

Melby-Lervag et al 2016 is mainly on certain forms of cognitive training other than DNB.

There is, however, a paragraph (p. 520) on related effects, which states a significant effect on nonverbal ability (g=0.15). The meta-study changes the value (g=0.10) by removing one study (Schweizer, Hampshire & Dalgleish, 2011). After having given three criteria for such removal (small sample size, only one post-test measuring nonverbal ability, unexplained pretest-to-posttest decreases for the control group) Schweizer, Hampshire & Dalgleish 2011 is called "the most problematic study", while, in fact, it does not meet any of the criteria given. 

Regarding the matter of possible effects of DNB training, this meta-study is not only partial, but not meeting its own design on the aspect described.

Schweizer, Hampshire & Dalgleish, 2011:

http://www.ncbi.nlm.nih.gov/pmc/articles/pmid/21949712

[James Austin]

It seems that there are just as many researchers that find DNB generates "benefit" as there are those that find it does not.

On a personal level, DNB when I work at it, I find my mind becomes clearer and sharper. Which in turn has flow on effects to a host of other activities.

As such, it may not be making me smarter, but it optimizes what I've already got.

[Simon Bevan]

I agree completely that my mind personally becomes clearer and sharper too. Also, having lived with ADHD for a while, and having a real struggle with tolerance for boredom and 'dry' tasks, I find that using dual n-back very noticeably improves my ability to stay focused in a disciplined way; this is the case in proportion with how many sessions and how regularly I use it. 

I also meditate daily and proactively develop my cognitive abilities in various other areas too, such as typing skills as referred to in the previous post earlier; in addition, I've done extensive practice with a variety of speed reading techniques, each of which has offered something valuable and have complemented each other. 

For those reasons, I think it can be too easy for some people to get distracted by the seeming lack of 'repeatability' of the dual n-back studies, which although sensible to scrutinize academic research studies, can ultimately be self-defeating since science itself as a field is historically characterised by uncertainty and by constantly updated views on what is factually accurate. 

Having studied secondary education, including educational psychology, one thing I learned was that there are very few if any cognitive skills programs which have 100% transferability or are universally generalisable; yet from my own experience as well as reading various other people's anecdotal reports of experiences benefiting from dual n-back, this appears to be a lot more generalisable than almost every other type of brain training ever developed - many of the games developed by Lumosity for example are very specific in the cognitive skills they develop; dual n-back focuses on working memory. 

Having said that, the more authoritative source of dual n-back I have come across, Dr Mark Ashton Smith who has developed i9 Mindware, has since continued to develop more wholistic and integrated 'brain x-training' programs for consolidating the gains made using dual n-back. Adaptive movement patterns are a key example with activities like wrestling (or other forms of grappling), agility training involving direction changing and rock climbing. 

What I do think is a huge fallacy is the idea that nootropics, on their own, without any cognitve training, will improve cognition significantly in the long term. 

[diff...@yahoo.com]

My personal experience is very similar. Brain training doesn't make me feel much different, except in a subtle way.  It's very much like weight lifting or calisthenics.  Over time your mental physique is simply more robust, but like exercise, you have to do enough to get any effect.  One, two or three sessions doesn't work.  The effect is evident in retrospect when I can observe that I stayed focused on a task that required sustained concentration until it was complete without brain fog setting in. That isn't something I noticed at the time but in the results. 

Moreover I have an anecdotal report from someone with cognitive deficits who I personally helped by administering a brain training regime.  The self-reported effects from n-back were impressive and in agreement with a published study of n-back training with subjects having the same physiologically diagnosable condition.   That is the source of my desire to defend this line of scientific inquiry.  In my opinion it is more valuable for providing a "leg up" for someone who is having difficulty keeping up with the pack than as some kind of brain supercharger for someone who is already way ahead. 

[Nathan]

The main reason I continue to do brain training (and really, more mind sports like chess or go these days), in addition to the purported direct effect on WM, is that I believe these activities train some aspect of the mind that is not readily measurable. Call it self-organization, will-power, planning, I don't know. People say that these games train only your performance of the games themselves, and that the evidence for it is that cognitive performance on certain tests is not reliably improved. I would counter by saying that it's possible that these games train your "holistic" performance and self-organization. Kind of like how weightlifting does not actually make your muscle fibers stronger, but does make you stronger overall by increasing the numbers of fibers and your neuromuscular connections. I also think that many games like chess, go, or poker do have clear real world parallels - especially if you are playing them in person with other people. That's not to say they will be a replacement for actual practice of real world skills, but they're much better than nothing. In other words, it would be interesting to see the effect of games on social or financial performance, rather than isolated cognitive tests. These games may not be actually increasing the size of your working memory, but they may be improving the utilization of that working memory. For example, after doing n-back, you maybe are able to chunk things more readily. WM has not increased, but the effective, real world WM has increased.

[Latitude]

Personally I feel sharper, I remember more things I read or watch and I am actually wittier after 3 weeks straight of N-Backing 20-30 minute actual time spent playing so with taking like 1 to 2 minute breaks in between 4 minute reps its like an hour or less. Then I maintain like every other day or so and then in another 6 months you gotta do another 3 weeks straight.  The power N-Backing gives you is not permanent in my experience it goes away if you don't do it for like 3 weeks I would say. But if you do it, i believe it is definitely doing something! 

Where is Susanne M. Jaeggi on all this?!?!?! why is she not commenting on all the debunking that is being placed upon her pride and joy discovery! Anyone know of her speaking out on this, saying anything new about her thoughts on NBACKing?

[Latitude]

[robert chalean]

I train with my program http://competicionmental.appspot.com/tts3dnback# It say words not letters and you can use imagination and visualisation . I train 5-back level since 3 weeks and I am programmer I havee considerable less errors while programming. I think 3D N-back is not more difficult that 2D. About a week i train with triple 3-back with emotion images. I memorize the word images with a emotional situation. Here my system

http://vernetit.blogspot.com.ar/2016/10/eo-and-peo-memory-system-memorize-with.html

[Rotem Segev]

Robert do you see any different benefits training with regular n-back versus 3D n-back?

[robert chalean]

I think 3D n-back is more easy since there is not much position repetitions (outside those configured by porcent number input). But i do not see Much difference when you are accustomed. I think may be is more fun 3D. I think the difference comes from using words instead of letters of piano sounds and and also using emotion expressions and combine words with mnemonics. This training Add emotional faces and emotional situations to my dreams. The people in my dreams smile, feel disgust, anger. In multiple situations in a single dream. 

My program http://competicionmental.appspot.com/tts3dnback#

[Learning]

robert chalean,

Would you agree that 3D 4X4 random rotation is more effective? To me this would seem obvious. I train on 3D 4X4 random rotation combination n-back and have found it more difficult than any other mode (even though I would ideally like to have something that's a lot more complicated to figure out than the combination n-back mode while not compromising the n-back mode). 

Otherwise to include the difficulty of having less squares and thus more interference you could just increase the interference on the 3D version, this would actually improve the enhancement of the difficulty of 3D (as the increase in size is no longer self negating to the agenda of improving difficulty), something which I recommend if you haven't already thought about getting to including this addition. 

Thank you for the creations by the way, very much appreciated :) .

[robert chalean]

 I don't know if camera rotation with Three combination is very difficult to me (graphically fuzzy). I use http://competicionmental.appspot.com/tts3dnback# not xByNback . I think that 4x4x4 n-back is good but fuzzy may be with practice is more easy. To new modes creation I think in 4D n-back . And word text to speech three-combination n-back mode. But i don't know very good how add the words text to squares. I don't understand the concept of interference . the posible match are configured in the %: [20] text input box in the right top corner of screen. I think that is linked to interference concept. Do you know that configuration input (i think it can augment exercise difficulty)?

I want to include my Emotion expression memory system that it can by apply to n-back training http://vernetit.blogspot.com.ar/2016/10/eo-and-peo-memory-system-memorize-with.html

Bye!

[Learning]

"e . the posible match are configured in the %.... Do you know that configuration input (i think it can augment exercise difficulty)?"


Yeah, I've tested this out and it seems to be chance of guaranteed match not interference. Unfortunately the best advice I have so far is to checkout how its been implemented in Brain Workshop, along with perhaps reading previous discussions on the subject found in this groups history.

Interference is basically something that creates a false positive in our mind, so as a separate example if our job is to match the colour bright yellow over n-2 instead of presenting the colour green as false match it would instead present a very slightly brighter yellow. This example works analogously with the other domains like position, letters, sounds, etc with the exception of the interference instead being based around very close variation in n-back number not very close variation in stimulus. For example, in playing 3 position traditional n-back where we have the first position being in the top right hand corner, the second being in the bottom left hand corner and the third being in the top left hand corner, in representing a high interference pattern of say 0.70 (rough estimate) on the next cycle of stimuli we might have the first being the top left hand corner, the second being in the top right hand corner and the third being in the bottom left hand corner.

Given the extra mental resources that are required to solve this task it might just be the most important extra setting to ever include in any n-back cycle, especially considering it enhances not only task complexity but it reinforces the positives of any other task in the activity and overall, the goal of training on any one cognitive domain (meaning if we're training on spatial variables we're likely going to improve faster and to more in depth levels here; and the same with letters, sounds, etc).

4D n-back sounds real interesting as well, I don't have any suggestions in relation to that at this point however. Even though I have a preference for it I agree that 4X4 3D rotation is a little fuzzy though and thus I'm not sure how you'd be able to overcome 4D problems at this point. Like you've probably already been doing I recommend just continuing to experiment with trial and error simulations in relation to the mathematical inputs, rotational or otherwise, you'll may be able to eventually arrive at something that just scrapes through, and just incrementally improve it from there.