reply from Torkel Klinberg re: Is Working Memory Training Effective? A Meta-Analytic Review
243 views
Skip to first unread message
Rustam
May 27, 2012, 7:33:11 PM5/27/12
to Dual N-Back, Brain Training & Intelligence
Hi group, I am new to the group and quite intriqued with your
discussions. I have corresponded with the Cogmed listserve and asked
for Dr Klinberg to clarify his view. Please see below comment from Dr.
Klingberg on the recent meta-analysis: “Is Working Memory Training
Effective? A Meta-Analytic Review”
"The main problem with this meta-analysis is the heterogeneity of both
participants and methods. The analysis includes both typically
developing children, adolescents and young adults, children with ADHD,
children with dyslexia, children with low working memory and children
with low IQ (55-85). It is unlikely to expect similar outcome in such
a diverse sample of participants. The training methods used in the
different article also vary widely. They include subjects training on
updating (e.g. Dahlin et al.), spatial-span like task (Klingberg et
al.) or listening to stories (St Clair-Thompson). Furthermore, the
measure of "attention" is performance on the Stroop task, which is
generally considered a measure of inhibition, not attention. Therefore
this meta-analysis cannot give any information of whether Cogmed
Working Memory Training improves working memory and attention in
children with ADHD or working memory problems." – Torkel Klingberg MD,
PhD
Rustam
discussions. I have corresponded with the Cogmed listserve and asked
for Dr Klinberg to clarify his view. Please see below comment from Dr.
Klingberg on the recent meta-analysis: “Is Working Memory Training
Effective? A Meta-Analytic Review”
"The main problem with this meta-analysis is the heterogeneity of both
participants and methods. The analysis includes both typically
developing children, adolescents and young adults, children with ADHD,
children with dyslexia, children with low working memory and children
with low IQ (55-85). It is unlikely to expect similar outcome in such
a diverse sample of participants. The training methods used in the
different article also vary widely. They include subjects training on
updating (e.g. Dahlin et al.), spatial-span like task (Klingberg et
al.) or listening to stories (St Clair-Thompson). Furthermore, the
measure of "attention" is performance on the Stroop task, which is
generally considered a measure of inhibition, not attention. Therefore
this meta-analysis cannot give any information of whether Cogmed
Working Memory Training improves working memory and attention in
children with ADHD or working memory problems." – Torkel Klingberg MD,
PhD
Rustam
☉
May 28, 2012, 1:16:17 AM5/28/12
to Dual N-Back, Brain Training & Intelligence
Yes, now we have to deal with poor meta-analyses; but of course
Klingberg will want to emphasise that *his business* is no way put
into question by this. The fun just never ends.
I think we should keep in mind that due to the diversity of studies
done, a meta-analysis cannot be conducted willy-nilly. Studies of
similar design should be used. However, considering that the studies,
insofar as they are similar in design and the subjects used, may
themselves be poor, it will be hard to derive meaningful results from
them, which should lead us back to the fact that bad studies are just
bad. In other words, studies which are bad that are later used in a
meta-analysis will lead to a bad meta-analysis.
argumzio
Klingberg will want to emphasise that *his business* is no way put
into question by this. The fun just never ends.
I think we should keep in mind that due to the diversity of studies
done, a meta-analysis cannot be conducted willy-nilly. Studies of
similar design should be used. However, considering that the studies,
insofar as they are similar in design and the subjects used, may
themselves be poor, it will be hard to derive meaningful results from
them, which should lead us back to the fact that bad studies are just
bad. In other words, studies which are bad that are later used in a
meta-analysis will lead to a bad meta-analysis.
argumzio
polar
May 28, 2012, 4:31:42 AM5/28/12
to brain-t...@googlegroups.com
Thanks Rustam. I still did not read that analysis - some diversity is always necessary, but there's a difference if you just describe things, or you put them into statistical melting pot. For a statistical meta-analysis the sample really sounds much too diverse.
Dne pondělí, 28. května 2012 1:33:11 UTC+2 Rustam napsal(a):
polar
May 28, 2012, 5:33:00 AM5/28/12
to brain-t...@googlegroups.com
I think the fun just begins with you ironizing one of the greatest experts on WM, just because he says some meta-analysis has its methodological flaws (or that he mentions cogmed in cogmed mailinglist). Fyi, cogmed is used in lot of hospitals (even at least in two from five in my city) - and there are people (say oncology patients), which desperately need cognitive training. I think its great they have cogmed, and dont have to use pen+paper, or group with one instructor.
And its really hard to overlook, that so diverse population AND training methods probably shouldnt be evaluated together statistically. And btw, there's a difference between "good studies/metaanalyses, but with negative results" and "improperly performed ("bad") studies/metaanalyses".
☉
May 28, 2012, 1:35:00 PM5/28/12
to Dual N-Back, Brain Training & Intelligence
On May 28, 4:33 am, polar <pol...@gmail.com> wrote:
> just because he says some meta-analysis has its methodological flaws
> (or that he mentions cogmed in cogmed mailinglist).
>
No, I pointed out he is running a business so that it is reasonable
> just because he says some meta-analysis has its methodological flaws
> (or that he mentions cogmed in cogmed mailinglist).
>
for us to be skeptical of his position. Just like pharmaceutical
companies run their businesses and recommend to MD's that their stuff
should be prescribed more than others, we wouldn't want to swallow
everything they tell us, now would we? I think you're being far too
confrontational on this list than is appropriate.
> And btw,
> there's a difference between "good studies/metaanalyses, but with negative
> results" and "improperly performed ("bad") studies/metaanalyses".
>
Good Positive Studies & M-A's
Bad Positive Studies & M-A's
Good Negative Studies & M-A's
Bad Negative Studies & M-A's
So far, most of the positive studies (for "far-transfer") are quite
bad. And this isn't the first time that someone on this ML has pointed
out the need for really good studies to settle these questions to
which some feel so emotionally indebted that they want to defend it
heroically at all costs, even personal dignity. It is interesting,
however, that of the good studies, most of them are negative (can
anyone categorize the studies for us perhaps?), which leads one to
think that if one chooses to conduct a good study, it is less likely
it will turn out positive. So too bad turning out positive. I recently
shared an article on the ML about "questionable research practices"
being the norm in psychology research these days. But I guess if one
relaxes one's ethics to guarantee positive results, we can all
celebrate the fame and glory of the researchers who give us those
results...
And can you put away the neurotic tic, please? It has been quite
insipid since a while ago.
You would think that if it were in a school text book that it *had* to
be right. Why else would anyone have wasted the $200.00 being sent to
the publisher, right? But that it is just so wrongheaded it hardly
bears mentioning.
argumzio
whoisbambam
May 29, 2012, 1:42:51 AM5/29/12
to Dual N-Back, Brain Training & Intelligence
too confrontational?
i am not sure i have seen that.
i am the one who is too confrontational.
when it comes down to it......when confronted by a man......there
probably isnt anything i wouldnt do.....of course, he has to start it
and escalate it........when it comes to THIS issue, I am a disgrace to
'humanity'.
furthermore, i dont think i have ever come across a situation you CANT
handle, argumzio (in written format).
i am not sure i have seen that.
i am the one who is too confrontational.
when it comes down to it......when confronted by a man......there
probably isnt anything i wouldnt do.....of course, he has to start it
and escalate it........when it comes to THIS issue, I am a disgrace to
'humanity'.
furthermore, i dont think i have ever come across a situation you CANT
handle, argumzio (in written format).
polar
May 29, 2012, 6:01:48 AM5/29/12
to brain-t...@googlegroups.com
:) no, I'm not too confrontational in this list - I'm confrontational with you, because
you are terribly narcissistic, and lecture me on the most basic things, even on the
things I point out first. It must be very hard for you to imagine people are not stupid.
So - I KNOW that textbooks can be wrong. And I WAS aware of different types
of studies, because I pointed the differences to YOU. And, please stop with things
like "if one relaxes one's ethics to guarantee positive results, we can all celebrate
the fame and glory of the researchers who give us those results..." - it can simply
be true other way around too.
☉
May 29, 2012, 9:43:34 AM5/29/12
to Dual N-Back, Brain Training & Intelligence
On May 29, 5:01 am, polar <pol...@gmail.com> wrote:
> :) no, I'm not too confrontational in this list - I'm confrontational with
> you, because
> you are terribly narcissistic, and lecture me on the most basic things,
You call me "narcissistic" because of your own errors in writing and
> :) no, I'm not too confrontational in this list - I'm confrontational with
> you, because
> you are terribly narcissistic, and lecture me on the most basic things,
reasoning? How more confrontational, and irresponsible, can you be,
polar? Do you honestly think of yourself as so terribly brilliant that
your errors simply don't exist? Have you ever had a reasoned debate
with someone without bursting a blood vessel at the slightest hint of
their thinking that you weren't all you thought yourself to be?
> even on the
> things I point out first.
> It must be very hard for you to imagine people
> are not stupid.
absurd statement.
> So - I KNOW that textbooks can be wrong. And I WAS aware of different types
> of studies, because I pointed the differences to YOU. And, please stop with
symmetric differences between studies.
> things
> like "if one relaxes one's ethics to guarantee positive results, we can all
> celebrate
> the fame and glory of the researchers who give us those results..." - it
> can simply
> be true other way around too.
relax one's ethics to show that something doesn't happen? You do
realize that, since you graciously leave this "other way around" vague
and ambiguous, I can only suppose that you mean that some might try to
reverse the burden of proof - but this is fundamentally unscientific a
gesture, and thoroughly difficult to do within science itself; whereas
my point is w.r.t. the more subtle, more intractable aspects of
scientific praxis, and didn't mention it by pulling something out of
my hat but by mentioning a recent publication. So what do you mean
this time, polar? Do you suppose there's some hidden conspiracy to
undermine the beauty and awesome power of DNB or something? Please
elaborate and stop wasting our (mainly my) time with your petty
squabbles which stem from your hatred of my not acknowledging your
self-professed perfection in this forum.
argumzio, fallibilist par excellence
☉
May 29, 2012, 9:47:00 AM5/29/12
to Dual N-Back, Brain Training & Intelligence
On May 29, 12:42 am, whoisbambam <a...@horseracingfirm.com> wrote:
> too confrontational?
>
Yes, I suppose I was a bit hyperbolic. Just remove the "too". I'll
handle the rest. But of taste there is no dispute: I don't make it a
highlight of my day to deal with persons who can't handle matters on a
non-emotional basis. Perhaps that's *my* problem.
argumzio
> too confrontational?
>
Yes, I suppose I was a bit hyperbolic. Just remove the "too". I'll
handle the rest. But of taste there is no dispute: I don't make it a
highlight of my day to deal with persons who can't handle matters on a
non-emotional basis. Perhaps that's *my* problem.
argumzio
Gwern Branwen
May 29, 2012, 10:22:38 AM5/29/12
to brain-t...@googlegroups.com
On Sun, May 27, 2012 at 7:33 PM, Rustam <ryu...@gmail.com> wrote:
> "The main problem with this meta-analysis is the heterogeneity of both
> participants and methods. The analysis includes both typically
> developing children, adolescents and young adults, children with ADHD,
> children with dyslexia, children with low working memory and children
> with low IQ (55-85). It is unlikely to expect similar outcome in such
> a diverse sample of participants. The training methods used in the
> different article also vary widely. They include subjects training on
> updating (e.g. Dahlin et al.), spatial-span like task (Klingberg et
> al.) or listening to stories (St Clair-Thompson). Furthermore, the
> measure of "attention" is performance on the Stroop task, which is
> generally considered a measure of inhibition, not attention. Therefore
> this meta-analysis cannot give any information of whether Cogmed
> Working Memory Training improves working memory and attention in
> children with ADHD or working memory problems." – Torkel Klingberg MD,
> PhD
One wonders what he would have said if the meta-analysis had been
> "The main problem with this meta-analysis is the heterogeneity of both
> participants and methods. The analysis includes both typically
> developing children, adolescents and young adults, children with ADHD,
> children with dyslexia, children with low working memory and children
> with low IQ (55-85). It is unlikely to expect similar outcome in such
> a diverse sample of participants. The training methods used in the
> different article also vary widely. They include subjects training on
> updating (e.g. Dahlin et al.), spatial-span like task (Klingberg et
> al.) or listening to stories (St Clair-Thompson). Furthermore, the
> measure of "attention" is performance on the Stroop task, which is
> generally considered a measure of inhibition, not attention. Therefore
> this meta-analysis cannot give any information of whether Cogmed
> Working Memory Training improves working memory and attention in
> children with ADHD or working memory problems." – Torkel Klingberg MD,
> PhD
narrower - say, just 9 studies dealing with DNB and IQ. "This
meta-analysis uses too few studies to produce any reliable synthesis"?
And what sort of dramatic effect like WM training is supposed to be
operates *only* in young children, or young adults, or old adults? Is
there any principled a priori prediction from a theory that WM would
succeed only in the particular demographic group Klingberg favors?
I wonder too if he's actually read the meta-analysis, because the
authors did carefully record various modifiers like the demographics.
If one goes to page 7, for example, one can easily examine a breakdown
of effect sizes on transfer to WM performance by multiple groups:
"Young children", "children", "Young adults", and "Older adults".
"children", BTW, do the worst - worse than either "Young children" or
"Older adults". His criticism of Stroop makes it sound as if it's
being cunningly folded into all the other data in order to water down
some true effect, but the Stroop analysis is broken out as a separate
table as well, on page 8.
And he does something interesting in the final line - he demands
evidence broken out solely for young children using Cogmed on WM or
attention; apparently just the meta-analysis for young children on WM
(_d_=1.41/0.46) or Cogmed on WM (_d_=1.18/0.86) is not sufficient.
Given at least 5 coding variables with ~14 subdivisions, well, let's
just say breaking out all possible combinations of 2 values would lead
to a rather lengthy paper...
--
gwern
http://www.gwern.net
polar
May 30, 2012, 7:10:30 AM5/30/12
to brain-t...@googlegroups.com
Well, argumzio, "projection" is what comes to my mind when I read
your last post (much more offensive that I ever was btw). And let me
remind you, I'm not reponsible if you waste your time on internet
(and you do seem to have an awful lot of time).
Dne úterý, 29. května 2012 15:43:34 UTC+2 ☉ napsal(a):
☉
May 30, 2012, 5:29:01 PM5/30/12
to Dual N-Back, Brain Training & Intelligence
On May 30, 6:10 am, polar <pol...@gmail.com> wrote:
> Well, argumzio, "projection" is what comes to my mind when I read
> your last post (much more offensive that I ever was btw). And let me
> remind you, I'm not reponsible if you waste your time on internet
> (and you do seem to have an awful lot of time).
Yeah, I think I can safely conclude you're a troll who doesn't deserve
> Well, argumzio, "projection" is what comes to my mind when I read
> your last post (much more offensive that I ever was btw). And let me
> remind you, I'm not reponsible if you waste your time on internet
> (and you do seem to have an awful lot of time).
further serious responses. Thanks.
argumzio
whoisbambam
May 31, 2012, 1:29:29 AM5/31/12
to Dual N-Back, Brain Training & Intelligence
i dont know about being a troll
but i do agree it doesnt warrant much of a response.
:(
too bad.
I appreciate most of the POVs here in this forum, pros and cons--I
wouldnt have it any other way (esp. as it relates to possible
shortcomings--critical analyses)
the facts are clear. There is not enough research on dnb. There needs
to be several very large studies with as many controls as possible
before we can draw sound conclusions.
It should be clear that we can NOT draw a conclusion based on the
current research that DNB improves IQ.
It is also possible that DNB does not improve IQ nor fluid
intelligence, or anything else for that matter that may be explained
by practice/re-test effects, etc.
This should be *obvious* by now.
we can share our opinions.
:)
it is my opinion that dnb probably does little to nothing to improve
IQ, fluid intelligence, or any 'intelligence' benefit at all (that is
practicable or transfers)
it is my opinion that it *might* help with focus and willpower and
possibly improve working memory in those of us that have a poor
working memory--but this may be attributed to an increase in focus
i did not do controls on myself.....it is difficult to say with
certainty that i may have benefited from dnb training as i also did
tdcs and various other actions (exercise, diet, supplements,
chemicals, etc)
but i do agree it doesnt warrant much of a response.
:(
too bad.
I appreciate most of the POVs here in this forum, pros and cons--I
wouldnt have it any other way (esp. as it relates to possible
shortcomings--critical analyses)
the facts are clear. There is not enough research on dnb. There needs
to be several very large studies with as many controls as possible
before we can draw sound conclusions.
It should be clear that we can NOT draw a conclusion based on the
current research that DNB improves IQ.
It is also possible that DNB does not improve IQ nor fluid
intelligence, or anything else for that matter that may be explained
by practice/re-test effects, etc.
This should be *obvious* by now.
we can share our opinions.
:)
it is my opinion that dnb probably does little to nothing to improve
IQ, fluid intelligence, or any 'intelligence' benefit at all (that is
practicable or transfers)
it is my opinion that it *might* help with focus and willpower and
possibly improve working memory in those of us that have a poor
working memory--but this may be attributed to an increase in focus
i did not do controls on myself.....it is difficult to say with
certainty that i may have benefited from dnb training as i also did
tdcs and various other actions (exercise, diet, supplements,
chemicals, etc)
Reply all
Reply to author
Forward
0 new messages