Reinterpretation of Jaeggi et al.
554 views
Skip to first unread message
Ari
May 24, 2009, 1:42:54 PM5/24/09
to Dual N-Back, Brain Training & Intelligence
Published in the journal Intelligence by David E. Moody, apparently a
math tutor for high school students. What do you make of this?
http://dx.doi.org/10.1016/j.intell.2009.04.005
The May 13, 2008 issue of Proceedings of the National Academy of
Sciences featured a cover article that purported to demonstrate
increases in fluid intelligence following training on a task of
working memory (Jaeggi, Buschkuehl, Jonides, & Perrig, 2008). The
authors described their own findings as a “landmark result”. Their
study was the subject of an introductory comment by Robert Sternberg
(2008), as well as articles in the mainstream media, including a
lengthy column in a recent edition of The New York Times (Wang &
Aamodt, 2009).
In view of the potential significance of the study and the quantity of
attention it has received, the results have been subjected to
remarkably little critical analysis. A close examination of the
evidence reported by Jaeggi et al. shows that it is not in fact
sufficient to support the authors' conclusion of any increase in their
subjects' fluid intelligence.
What Jaeggi et al. reported were modest increases in performance on a
test of fluid intelligence following several days of training on a
task of working memory. The reported increases in performance are not
in question here. But the manner in which the test was administered
severely undermines the authors' interpretation that their subjects'
intelligence itself was increased.
The subjects were divided into four groups, differing in the number of
days of training they received on the task of working memory. The
group that received the least training (8 days) was tested on Raven's
Advanced Progressive Matrices (Raven, 1990), a widely used and well-
established test of fluid intelligence. This group, however,
demonstrated negligible improvement between pre- and post-test
performance.
The other three groups were not tested using Raven's Matrices, but
rather on an alternative test of much more recent origin. The Bochumer
Matrices Test (BOMAT) (Hossiep, Turck, & Hasella, 1999) is similar to
Raven's in that it consists of visual analogies. In both tests, a
series of geometric and other figures is presented in a matrix format
and the subject is required to infer a pattern in order to predict the
next figure in the series. The authors provide no reason for switching
from Raven's to the BOMAT.
The BOMAT differs from Raven's in some important respects, but is
similar in one crucial attribute: both tests are progressive in
nature, which means that test items are sequentially arranged in order
of increasing difficulty. A high score on the test, therefore, is
predicated on subjects' ability to solve the more difficult items.
However, this progressive feature of the test was effectively
eliminated by the manner in which Jaeggi et al. adminstered it. The
BOMAT is a 29-item test which subjects are supposed to be allowed 45
min to complete. Remarkably, however, Jaeggi et al. reduced the
allotted time from 45 min to 10. The effect of this restriction was to
make it impossible for subjects to proceed to the more difficult items
on the test. The large majority of the subjects—regardless of the
number of days of training they received—answered less than 14 test
items correctly.
By virtue of the manner in which they administered the BOMAT, Jaeggi
et al. transformed it from a test of fluid intelligence into a speed
test of ability to solve the easier visual analogies.
The time restriction not only made it impossible for subjects to
proceed to the more difficult items, it also limited the opportunity
to learn about the test—and so improve performance—in the process of
taking it. This factor cannot be neglected because test performance
does improve with practice, as demonstrated by the control groups in
the Jaeggi study, whose improvement from pre- to post-test was about
half that of the experimental groups. The same learning process that
occurs from one administration of the test to the next may also
operate within a given administration of the test—provided subjects
are allowed sufficient time to complete it.
Since the whole weight of their conclusion rests upon the validity of
their measure of fluid intelligence, one might assume the authors
would present a careful defense of the manner in which they
administered the BOMAT. Instead they do not even mention that subjects
are normally allowed 45 min to complete the test. Nor do they mention
that the test has 29 items, of which most of their subjects completed
less than half.
The authors' entire rationale for reducing the allotted time to 10 min
is confined to a footnote. That footnote reads as follows:
Although this procedure differs from the standardized procedure,
there is evidence that this timed procedure has little influence on
relative standing in these tests, in that the correlation of speeded
and non-speeded versions is very high (r = 0.95; ref. 37).
The reference given in the footnote is to a 1988 study (Frearson &
Eysenck, 1986) that is not in fact designed to support the conclusion
stated by Jaeggi et al. The 1988 study merely contains a footnote of
its own, which refers in turn to unpublished research conducted forty
years earlier. That research involved Raven's matrices, not the BOMAT,
and entailed a reduction in time of at most 50%, not more than 75%, as
in the Jaeggi study.
So instead of offering a reasoned defense of their procedure, Jaeggi
et al. provide merely a footnote which refers in turn to a footnote in
another study. The second footnote describes unpublished results,
evidently recalled by memory over a span of 40 years, involving a
different test and a much less severe reduction in time.
In this context it bears repeating that the group that was tested on
Raven's matrices (with presumably the same time restriction) showed
virtually no improvement in test performance, in spite of eight days'
training on working memory. Performance gains only appeared for the
groups administered the BOMAT. But the BOMAT differs in one important
respect from Raven's. Raven's matrices are presented in a 3 × 3
format, whereas the BOMAT consists of a 5 × 3 matrix configuration.
With 15 visual figures to keep track of in each test item instead of
9, the BOMAT puts added emphasis on subjects' ability to hold details
of the figures in working memory, especially under the condition of a
severe time constraint. Therefore it is not surprising that extensive
training on a task of working memory would facilitate performance on
the early and easiest BOMAT test items—those that present less of a
challenge to fluid intelligence.
This interpretation acquires added plausibility from the nature of one
of the two working-memory tasks administered to the experimental
groups. The authors maintain that those tasks were “entirely
different” from the test of fluid intelligence. One of the tasks
merits that description: it was a sequence of letters presented
auditorily through headphones.
But the other working-memory task involved recall of the location of a
small square in one of several positions in a visual matrix pattern.
It represents in simplified form precisely the kind of detail required
to solve visual analogies. Rather than being “entirely different” from
the test items on the BOMAT, this task seems well-designed to
facilitate performance on that test.
More generally, the foregoing considerations suggest a deeper problem
with the conclusions presented by Jaeggi et al.: To what extent does
improvement on any test of fluid intelligence reflect an increase in
actual intelligence rather than merely an increase in test-taking
skills? A full analysis of this issue is beyond the scope of the
present review, but the methodological challenges involved are
formidable and deserve further discussion.
Whatever the meaning of the modest gains in performance on the BOMAT,
the evidence produced by Jaeggi et al. does not support the conclusion
of an increase in their subjects' intelligence. Their research may be
sufficient to encourage further investigation, but any larger
inferences are unwarranted.
math tutor for high school students. What do you make of this?
http://dx.doi.org/10.1016/j.intell.2009.04.005
The May 13, 2008 issue of Proceedings of the National Academy of
Sciences featured a cover article that purported to demonstrate
increases in fluid intelligence following training on a task of
working memory (Jaeggi, Buschkuehl, Jonides, & Perrig, 2008). The
authors described their own findings as a “landmark result”. Their
study was the subject of an introductory comment by Robert Sternberg
(2008), as well as articles in the mainstream media, including a
lengthy column in a recent edition of The New York Times (Wang &
Aamodt, 2009).
In view of the potential significance of the study and the quantity of
attention it has received, the results have been subjected to
remarkably little critical analysis. A close examination of the
evidence reported by Jaeggi et al. shows that it is not in fact
sufficient to support the authors' conclusion of any increase in their
subjects' fluid intelligence.
What Jaeggi et al. reported were modest increases in performance on a
test of fluid intelligence following several days of training on a
task of working memory. The reported increases in performance are not
in question here. But the manner in which the test was administered
severely undermines the authors' interpretation that their subjects'
intelligence itself was increased.
The subjects were divided into four groups, differing in the number of
days of training they received on the task of working memory. The
group that received the least training (8 days) was tested on Raven's
Advanced Progressive Matrices (Raven, 1990), a widely used and well-
established test of fluid intelligence. This group, however,
demonstrated negligible improvement between pre- and post-test
performance.
The other three groups were not tested using Raven's Matrices, but
rather on an alternative test of much more recent origin. The Bochumer
Matrices Test (BOMAT) (Hossiep, Turck, & Hasella, 1999) is similar to
Raven's in that it consists of visual analogies. In both tests, a
series of geometric and other figures is presented in a matrix format
and the subject is required to infer a pattern in order to predict the
next figure in the series. The authors provide no reason for switching
from Raven's to the BOMAT.
The BOMAT differs from Raven's in some important respects, but is
similar in one crucial attribute: both tests are progressive in
nature, which means that test items are sequentially arranged in order
of increasing difficulty. A high score on the test, therefore, is
predicated on subjects' ability to solve the more difficult items.
However, this progressive feature of the test was effectively
eliminated by the manner in which Jaeggi et al. adminstered it. The
BOMAT is a 29-item test which subjects are supposed to be allowed 45
min to complete. Remarkably, however, Jaeggi et al. reduced the
allotted time from 45 min to 10. The effect of this restriction was to
make it impossible for subjects to proceed to the more difficult items
on the test. The large majority of the subjects—regardless of the
number of days of training they received—answered less than 14 test
items correctly.
By virtue of the manner in which they administered the BOMAT, Jaeggi
et al. transformed it from a test of fluid intelligence into a speed
test of ability to solve the easier visual analogies.
The time restriction not only made it impossible for subjects to
proceed to the more difficult items, it also limited the opportunity
to learn about the test—and so improve performance—in the process of
taking it. This factor cannot be neglected because test performance
does improve with practice, as demonstrated by the control groups in
the Jaeggi study, whose improvement from pre- to post-test was about
half that of the experimental groups. The same learning process that
occurs from one administration of the test to the next may also
operate within a given administration of the test—provided subjects
are allowed sufficient time to complete it.
Since the whole weight of their conclusion rests upon the validity of
their measure of fluid intelligence, one might assume the authors
would present a careful defense of the manner in which they
administered the BOMAT. Instead they do not even mention that subjects
are normally allowed 45 min to complete the test. Nor do they mention
that the test has 29 items, of which most of their subjects completed
less than half.
The authors' entire rationale for reducing the allotted time to 10 min
is confined to a footnote. That footnote reads as follows:
Although this procedure differs from the standardized procedure,
there is evidence that this timed procedure has little influence on
relative standing in these tests, in that the correlation of speeded
and non-speeded versions is very high (r = 0.95; ref. 37).
The reference given in the footnote is to a 1988 study (Frearson &
Eysenck, 1986) that is not in fact designed to support the conclusion
stated by Jaeggi et al. The 1988 study merely contains a footnote of
its own, which refers in turn to unpublished research conducted forty
years earlier. That research involved Raven's matrices, not the BOMAT,
and entailed a reduction in time of at most 50%, not more than 75%, as
in the Jaeggi study.
So instead of offering a reasoned defense of their procedure, Jaeggi
et al. provide merely a footnote which refers in turn to a footnote in
another study. The second footnote describes unpublished results,
evidently recalled by memory over a span of 40 years, involving a
different test and a much less severe reduction in time.
In this context it bears repeating that the group that was tested on
Raven's matrices (with presumably the same time restriction) showed
virtually no improvement in test performance, in spite of eight days'
training on working memory. Performance gains only appeared for the
groups administered the BOMAT. But the BOMAT differs in one important
respect from Raven's. Raven's matrices are presented in a 3 × 3
format, whereas the BOMAT consists of a 5 × 3 matrix configuration.
With 15 visual figures to keep track of in each test item instead of
9, the BOMAT puts added emphasis on subjects' ability to hold details
of the figures in working memory, especially under the condition of a
severe time constraint. Therefore it is not surprising that extensive
training on a task of working memory would facilitate performance on
the early and easiest BOMAT test items—those that present less of a
challenge to fluid intelligence.
This interpretation acquires added plausibility from the nature of one
of the two working-memory tasks administered to the experimental
groups. The authors maintain that those tasks were “entirely
different” from the test of fluid intelligence. One of the tasks
merits that description: it was a sequence of letters presented
auditorily through headphones.
But the other working-memory task involved recall of the location of a
small square in one of several positions in a visual matrix pattern.
It represents in simplified form precisely the kind of detail required
to solve visual analogies. Rather than being “entirely different” from
the test items on the BOMAT, this task seems well-designed to
facilitate performance on that test.
More generally, the foregoing considerations suggest a deeper problem
with the conclusions presented by Jaeggi et al.: To what extent does
improvement on any test of fluid intelligence reflect an increase in
actual intelligence rather than merely an increase in test-taking
skills? A full analysis of this issue is beyond the scope of the
present review, but the methodological challenges involved are
formidable and deserve further discussion.
Whatever the meaning of the modest gains in performance on the BOMAT,
the evidence produced by Jaeggi et al. does not support the conclusion
of an increase in their subjects' intelligence. Their research may be
sufficient to encourage further investigation, but any larger
inferences are unwarranted.
Iron
May 24, 2009, 1:58:45 PM5/24/09
to Dual N-Back, Brain Training & Intelligence
An interesting point I would like to bring into this thread is the
mention earlier of people with autism performing higher than average
on the raven's with a lower average working memory. That suggests to
me that the raven's as a test may not require a large working memory
capacity. It looks to me like there may be some other bottleneck for
this test.
It is rather curious that they administered the tests in that fashion
and that they changed the tests mid-experiment. I hope that in the
future a more extensive test is administered after an even longer
period of DNB training. If all this is good for is increasing our
speed at solving problems we could already solve, I still feel that it
is an incredibly valuable exercise.
There are other studies indicating significant improvements in ADHD
symptoms and unrelated skills, so I still maintain the belief that DNB
is good for us and we are improving something vitally important by
training.
mention earlier of people with autism performing higher than average
on the raven's with a lower average working memory. That suggests to
me that the raven's as a test may not require a large working memory
capacity. It looks to me like there may be some other bottleneck for
this test.
It is rather curious that they administered the tests in that fashion
and that they changed the tests mid-experiment. I hope that in the
future a more extensive test is administered after an even longer
period of DNB training. If all this is good for is increasing our
speed at solving problems we could already solve, I still feel that it
is an incredibly valuable exercise.
There are other studies indicating significant improvements in ADHD
symptoms and unrelated skills, so I still maintain the belief that DNB
is good for us and we are improving something vitally important by
training.
Pontus Granström
May 24, 2009, 2:03:57 PM5/24/09
to brain-t...@googlegroups.com
Actually there is much proof of that working memory and fluid intelligence shares the same neural network. http://www.eric.ed.gov/ERICWebPortal/custom/portlets/recordDetails/detailmini.jsp?_nfpb=true&_&ERICExtSearch_SearchValue_0=EJ724232&ERICExtSearch_SearchType_0=no&accno=EJ724232
Pontus Granström
May 24, 2009, 2:20:29 PM5/24/09
to brain-t...@googlegroups.com
Here's another report supporting the claim. "General intelligence (g) is highly correlated with
working-memory capacity (WMC). It has been argued that these central
psychological constructs should share common neural systems. The
present study examines this hypothesis using structural magnetic
resonance imaging to determine any overlap in brain areas where
regional grey matter volumes are correlated to measures of general
intelligence and to memory span. In normal volunteers (N = 48) the results (p
< .05, corrected for multiple comparisons) indicate that a common
anatomic framework for these constructs implicates mainly frontal grey
matter regions belonging to Brodmann area (BA) 10 (right superior
frontal gyrus and left middle frontal gyrus) and, to a lesser degree,
the right inferior parietal lobule (BA 40). These findings support the
nuclear role of a discrete parieto-frontal network.
"
http://www.informaworld.com/smpp/content~content=a788318424~db=all
http://www.informaworld.com/smpp/content~content=a788318424~db=all
jttoto
May 24, 2009, 2:31:01 PM5/24/09
to Dual N-Back, Brain Training & Intelligence
Moody brings up an excellent point, that perhaps DNB really does train
for the easier questions on the BOMAT, considering the larger working
memory load of the BOMAT. While he does highlight flaws in the Jaeggi
study, there are somethings I would like to point out in Moody's
critique.
He admits that improvement on the BOMAT happens because of
improvements in spatial working memory, which DNB trains. He seems to
imply that there is no proof that DNB improves one's analytical
ability (which may be true), which is crucial for solving the more
complex problems on the Raven's. Instead, he says the improvements
are shown because improved working memory alone can help on the easier
portions of the BOMAT.
This may be true, but there are plenty of studies that suggest a
correlation with working memory and Fluid intelligence, so it would
seem reasonable that improving one will improve the other.
Also, isn't it still significant that DNB improves working memory?
Even if that is all it does, I'd say it is still pretty helpful in the
real world.
Ari
May 24, 2009, 2:33:45 PM5/24/09
to Dual N-Back, Brain Training & Intelligence
Yes. While Raven's seems to be more of a test of ability to deal with
complexity (intelligence?), the earlier levels of BOMAT test mostly
working memory. Working memory is indisputably related to
intelligence, but it may be the case that DNB training improves only
speed or capacity, not intelligence however constituted. As you point
out, though, this is a useful skill regardless.
complexity (intelligence?), the earlier levels of BOMAT test mostly
working memory. Working memory is indisputably related to
intelligence, but it may be the case that DNB training improves only
speed or capacity, not intelligence however constituted. As you point
out, though, this is a useful skill regardless.
Pontus Granström
May 24, 2009, 2:41:13 PM5/24/09
to brain-t...@googlegroups.com
But you forget that they saw improvements even among those with a high capacity working memory.
"However, our additional analyses show that there is more to
transfer than mere improvement in working memory capacity in
that the increase in Gf was not directly related to either
preexisting individual differences in working memory capacity
or to the gain in working memory capacity as measured by simple
or complex spans, or even, by the specific training effect itself."
"However, our additional analyses show that there is more to
transfer than mere improvement in working memory capacity in
that the increase in Gf was not directly related to either
preexisting individual differences in working memory capacity
or to the gain in working memory capacity as measured by simple
or complex spans, or even, by the specific training effect itself."
Mike L.
May 24, 2009, 2:54:01 PM5/24/09
to Dual N-Back, Brain Training & Intelligence
But wait, isn't the speed or even the capacity with which one is
endowed to accomplish whatever task they might have at hand, also
strongly correlated with intelligence?
That would be like saying, a faster and more fuel efficient car does
not have a better engine than that of a slower, less fuel efficient
car.
To use a more concrete example, on the WAIS-IV IQ test, there is
indeed a section on processing information and spatial reasoning, as
well as digit-span memorization. These categories, it can be said,
will indeed rise as due to DNB training, thus, increase overall IQ.
Therefore, though, according to the hypothesis moody devises of DNB
not improving analytical processes, training with DNB will still
result in a higher IQ.
endowed to accomplish whatever task they might have at hand, also
strongly correlated with intelligence?
That would be like saying, a faster and more fuel efficient car does
not have a better engine than that of a slower, less fuel efficient
car.
To use a more concrete example, on the WAIS-IV IQ test, there is
indeed a section on processing information and spatial reasoning, as
well as digit-span memorization. These categories, it can be said,
will indeed rise as due to DNB training, thus, increase overall IQ.
Therefore, though, according to the hypothesis moody devises of DNB
not improving analytical processes, training with DNB will still
result in a higher IQ.
Mike L.
May 24, 2009, 3:52:29 PM5/24/09
to Dual N-Back, Brain Training & Intelligence
I wanted to add something:
Moody himself states that the processing of the "easier" tasks became
quicker in those people who trained with DNB. Now, using logic: When
one does something quicker, or finds that they can perform something
quicker, that task has then become easier to do for them. For this
reason, to say that performing something quicker but not finding it
easier to do, is almost illogical, and therefore, to say that harder
tasks remain just as hard for the person who has found it easier to
perform the easy tasks just doesn't make sense.
Taking this into consideration, it is more sensible to believe that
the easy tasks have become easier but the harder tasks have TOO become
easier.
Moody himself states that the processing of the "easier" tasks became
quicker in those people who trained with DNB. Now, using logic: When
one does something quicker, or finds that they can perform something
quicker, that task has then become easier to do for them. For this
reason, to say that performing something quicker but not finding it
easier to do, is almost illogical, and therefore, to say that harder
tasks remain just as hard for the person who has found it easier to
perform the easy tasks just doesn't make sense.
Taking this into consideration, it is more sensible to believe that
the easy tasks have become easier but the harder tasks have TOO become
easier.
Pontus Granström
May 24, 2009, 4:27:15 PM5/24/09
to brain-t...@googlegroups.com
I agree, what they are claiming is that the first questions of raven's and bomat is just questions that takes working memory into account, but then after a few questions it measures analytical intelligence?
Ron Williams
May 24, 2009, 11:39:10 PM5/24/09
to brain-t...@googlegroups.com
You are assuming that the harder tasks are possible at all for a given person. This may not be so. If you are required to keep ten things in mind all at once and do a manipulation of them to achieve the result, it may never be possible, if you are only capable of holding seven things in mind.
The harder tasks of this kind will remain just has impossible as before, and will result in a big X on the score sheet. On the other hand, the easier tasks might well be done much more quickly than before if there's a speed improvement. I.e. speeding up attempts on the impossible will not give a better result!
The harder tasks of this kind will remain just has impossible as before, and will result in a big X on the score sheet. On the other hand, the easier tasks might well be done much more quickly than before if there's a speed improvement. I.e. speeding up attempts on the impossible will not give a better result!
Mike L.
May 25, 2009, 1:07:42 AM5/25/09
to Dual N-Back, Brain Training & Intelligence
No but getting faster at an easier task, at least in terms of
intelligence, and for the most part, is indicative of the possibility
that one has opened their capabilities and as a result is much less
prone to struggle on that easier task and much more likely to succeed
at a harder one.
I mean, you simply cannot rule that out.
On May 24, 11:39 pm, Ron Williams <rhwil...@gmail.com> wrote:
> You are assuming that the harder tasks are possible at all for a given
> person. This may not be so. If you are required to keep ten things in mind
> all at once and do a manipulation of them to achieve the result, it may
> never be possible, if you are only capable of holding seven things in mind.
>
> The harder tasks of this kind will remain just has impossible as before, and
> will result in a big X on the score sheet. On the other hand, the easier
> tasks might well be done much more quickly than before if there's a speed
> improvement. I.e. speeding up attempts on the impossible will not give a
> better result!
>
intelligence, and for the most part, is indicative of the possibility
that one has opened their capabilities and as a result is much less
prone to struggle on that easier task and much more likely to succeed
at a harder one.
I mean, you simply cannot rule that out.
On May 24, 11:39 pm, Ron Williams <rhwil...@gmail.com> wrote:
> You are assuming that the harder tasks are possible at all for a given
> person. This may not be so. If you are required to keep ten things in mind
> all at once and do a manipulation of them to achieve the result, it may
> never be possible, if you are only capable of holding seven things in mind.
>
> The harder tasks of this kind will remain just has impossible as before, and
> will result in a big X on the score sheet. On the other hand, the easier
> tasks might well be done much more quickly than before if there's a speed
> improvement. I.e. speeding up attempts on the impossible will not give a
> better result!
>
Pontus Granström
May 25, 2009, 5:16:36 AM5/25/09
to brain-t...@googlegroups.com
Hi, read this article http://www.psych.rutgers.edu/~jose/courses/578/Conway_etal_2003.pdf especially the "Neuroimaging studies of WMC and g" it shows that there is high correlation between n-back "lure trials" and ravens (r=0.54), so it's the lure trials (which takes alot of executive control) that correlates with intelligence. It also shows that n-back and Ravens engages the same areas of the brain (DLPFC and ACC) and that the correlation depends 92% on lure trials and activity in the DLPFC area, which means that DLPFC is the mediator between G and WMC. Very interesting! Dual-n-backs lure trials engages DLPFC and perhaps strengthen it with we fail? Just like a muscle that fails sends signals to build more muscle tissue? Or just the fact the we activate the area strengthens DLPFC and therefore IQ-score?
Ashirgo
May 25, 2009, 5:22:17 AM5/25/09
to Dual N-Back, Brain Training & Intelligence
Unfortunately, I cannot read this article: "404 Page Not Found: The
Internet Just Broke" . Moreover, could you present your interpretation
of "lure trials"? I have a single conception, but... ;)
Internet Just Broke" . Moreover, could you present your interpretation
of "lure trials"? I have a single conception, but... ;)
Pontus Granström
May 25, 2009, 5:38:07 AM5/25/09
to brain-t...@googlegroups.com
Hi, I've uploaded it in the files section. A "lure trial" is basiclly B-R-B-A in a 3-back session. Here the second b "lures the mind" since it's almost 3-back and therefore takes alot of executive control not to press the L button. Since ravens correlates with the "lure trials" we might want to add a feature to the the Brainworkshop program that display the number of time we didnt fall for a "lure trials"? So after every session it would display
Visual: 80% Audio: 80% and Lure Trials: 75%?
Visual: 80% Audio: 80% and Lure Trials: 75%?
Ashirgo
May 25, 2009, 6:06:02 AM5/25/09
to Dual N-Back, Brain Training & Intelligence
Thank you. One can also consider an option of manipulating maths a
little so as to increase the chance of "lure trials" :) Regrettably, I
cannot see how it could work at higher n-levels. That is to say, I
have never perceived it as a problem/nuisance/etc.
little so as to increase the chance of "lure trials" :) Regrettably, I
cannot see how it could work at higher n-levels. That is to say, I
have never perceived it as a problem/nuisance/etc.
Pontus Granström
May 25, 2009, 6:34:33 AM5/25/09
to brain-t...@googlegroups.com
I think that there are som cases where you are really "lured", not that it might be hard but still relative harder then just a "K-L-R-[K]" type of sequence. For 4 and 5-back there are some really "luring" stuff.
Wade
May 25, 2009, 10:35:06 AM5/25/09
to Dual N-Back, Brain Training & Intelligence
Ron,
Your argument makes a hidden assumption that working memory and "speed
of processing" are fully separable components of mental performance.
They apparently are not. Studies have consistently found that working
memory and "processing speed" covary:
From "Variation in Working Memory", page 61:
"In many studies, WMC (Working memory capacity) was found to be
strongly correlated to measures of processing speed [they give several
citations]. ...one obvious explanation is that working memory tasks
are complex span tasks which require a processing component
themselves, and the speed of performing the component is one source of
variance in complex span tasks. However, this cannot be the whole
story, because not all WMC tasks are complex span tasks, and speed
measures are also correlated to working memory tasks lacking a
processing component. A clear example of this is in the study by
Conway et al, (2002), who found a substantial correlation of speed
with a factor that extracted the common variance of short-term and
working memory tasks, but none found a second factor capturing the
specific variance of working memory. This is the opposite of what one
would expect if speed is related to complex span through the
processing component of the latter."
A question that needs answering is whether the "analytical" abilities
that are supposedly required of the more difficult BOMAT questions
covary with WM and processing speed too. Although it is tempting to
think that they do not, I would not jump to this conclusion either.
On May 24, 10:39 pm, Ron Williams <rhwil...@gmail.com> wrote:
> You are assuming that the harder tasks are possible at all for a given
> person. This may not be so. If you are required to keep ten things in mind
> all at once and do a manipulation of them to achieve the result, it may
> never be possible, if you are only capable of holding seven things in mind.
>
> The harder tasks of this kind will remain just has impossible as before, and
> will result in a big X on the score sheet. On the other hand, the easier
> tasks might well be done much more quickly than before if there's a speed
> improvement. I.e. speeding up attempts on the impossible will not give a
> better result!
>
Your argument makes a hidden assumption that working memory and "speed
of processing" are fully separable components of mental performance.
They apparently are not. Studies have consistently found that working
memory and "processing speed" covary:
From "Variation in Working Memory", page 61:
"In many studies, WMC (Working memory capacity) was found to be
strongly correlated to measures of processing speed [they give several
citations]. ...one obvious explanation is that working memory tasks
are complex span tasks which require a processing component
themselves, and the speed of performing the component is one source of
variance in complex span tasks. However, this cannot be the whole
story, because not all WMC tasks are complex span tasks, and speed
measures are also correlated to working memory tasks lacking a
processing component. A clear example of this is in the study by
Conway et al, (2002), who found a substantial correlation of speed
with a factor that extracted the common variance of short-term and
working memory tasks, but none found a second factor capturing the
specific variance of working memory. This is the opposite of what one
would expect if speed is related to complex span through the
processing component of the latter."
A question that needs answering is whether the "analytical" abilities
that are supposedly required of the more difficult BOMAT questions
covary with WM and processing speed too. Although it is tempting to
think that they do not, I would not jump to this conclusion either.
On May 24, 10:39 pm, Ron Williams <rhwil...@gmail.com> wrote:
> You are assuming that the harder tasks are possible at all for a given
> person. This may not be so. If you are required to keep ten things in mind
> all at once and do a manipulation of them to achieve the result, it may
> never be possible, if you are only capable of holding seven things in mind.
>
> The harder tasks of this kind will remain just has impossible as before, and
> will result in a big X on the score sheet. On the other hand, the easier
> tasks might well be done much more quickly than before if there's a speed
> improvement. I.e. speeding up attempts on the impossible will not give a
> better result!
>
Wade
May 25, 2009, 10:36:30 AM5/25/09
to Dual N-Back, Brain Training & Intelligence
What are "lure" trials?
On May 25, 4:16 am, Pontus Granström <lepon...@gmail.com> wrote:
> Hi, read this articlehttp://www.psych.rutgers.edu/~jose/courses/578/Conway_etal_2003.pdfes...
On May 25, 4:16 am, Pontus Granström <lepon...@gmail.com> wrote:
> Hi, read this articlehttp://www.psych.rutgers.edu/~jose/courses/578/Conway_etal_2003.pdfes...
> the "Neuroimaging studies of WMC and g" it shows that there is
> high correlation between n-back "lure trials" and ravens (r=0.54), so it's
> the lure trials (which takes alot of executive control) that correlates with
> intelligence. It also shows that n-back and Ravens engages the same areas of
> the brain (DLPFC and ACC) and that the correlation depends 92% on lure
> trials and activity in the DLPFC area, which means that DLPFC is the
> mediator between G and WMC. Very interesting! Dual-n-backs lure trials
> engages DLPFC and perhaps strengthen it with we fail? Just like a muscle
> that fails sends signals to build more muscle tissue? Or just the fact the
> we activate the area strengthens DLPFC and therefore IQ-score?
>
> high correlation between n-back "lure trials" and ravens (r=0.54), so it's
> the lure trials (which takes alot of executive control) that correlates with
> intelligence. It also shows that n-back and Ravens engages the same areas of
> the brain (DLPFC and ACC) and that the correlation depends 92% on lure
> trials and activity in the DLPFC area, which means that DLPFC is the
> mediator between G and WMC. Very interesting! Dual-n-backs lure trials
> engages DLPFC and perhaps strengthen it with we fail? Just like a muscle
> that fails sends signals to build more muscle tissue? Or just the fact the
> we activate the area strengthens DLPFC and therefore IQ-score?
>
Wade
May 25, 2009, 10:41:03 AM5/25/09
to Dual N-Back, Brain Training & Intelligence
Cool idea.
On May 25, 4:38 am, Pontus Granström <lepon...@gmail.com> wrote:
> Hi, I've uploaded it in the files section. A "lure trial" is basiclly
> B-R-B-A in a 3-back session. Here the second b "lures the mind" since it's
> almost 3-back and therefore takes alot of executive control not to press the
> L button. Since ravens correlates with the "lure trials" we might want to
> add a feature to the the Brainworkshop program that display the number of
> time we didnt fall for a "lure trials"? So after every session it would
> display
> Visual: 80% Audio: 80% and Lure Trials: 75%?
>
On May 25, 4:38 am, Pontus Granström <lepon...@gmail.com> wrote:
> Hi, I've uploaded it in the files section. A "lure trial" is basiclly
> B-R-B-A in a 3-back session. Here the second b "lures the mind" since it's
> almost 3-back and therefore takes alot of executive control not to press the
> L button. Since ravens correlates with the "lure trials" we might want to
> add a feature to the the Brainworkshop program that display the number of
> time we didnt fall for a "lure trials"? So after every session it would
> display
> Visual: 80% Audio: 80% and Lure Trials: 75%?
>
Vlad
May 25, 2009, 11:54:29 AM5/25/09
to Dual N-Back, Brain Training & Intelligence
Nice article, but for me there was hardly something new (I didn't
catch that change to ONLY bomat in jaeggi). Nevertheless, in my study
(people are flaking the final tests :(, lets hope at least 15 more
will take it) I use only raven, we'll see. And I think that other
theoretical questions appearing in this article (effectivity VS new
quality of intellect) were the subject of discussion here already.
catch that change to ONLY bomat in jaeggi). Nevertheless, in my study
(people are flaking the final tests :(, lets hope at least 15 more
will take it) I use only raven, we'll see. And I think that other
theoretical questions appearing in this article (effectivity VS new
quality of intellect) were the subject of discussion here already.
Mike L.
May 25, 2009, 2:36:33 PM5/25/09
to Dual N-Back, Brain Training & Intelligence
hmm, someone else posted on here that you had already finished your
studies. Guess it was a misinformation.
In any case, interesting stuff; keep us posted, it'd be interesting to
see what other results have come from secondary studies [even if they
are on a more minimal scale].
studies. Guess it was a misinformation.
In any case, interesting stuff; keep us posted, it'd be interesting to
see what other results have come from secondary studies [even if they
are on a more minimal scale].
Ari
May 25, 2009, 2:54:58 PM5/25/09
to Dual N-Back, Brain Training & Intelligence
"For this reason, to say that performing something quicker but not
finding it easier to do, is almost illogical"
Why? I'll give you an example: caffeine vs. amphetamines. The first
finding it easier to do, is almost illogical"
improves speed in many people; you'll work faster but not necessary
better (and possibly worse). There will be no increase in the
complexity of your thoughts.
Amphetamine, by contrast, improves not only speed but also maximum
thought complexity. (At least this has been my experience!)
It's like the distinction between processor speed (GHz) and processor
architecture. The former is a measure of cycles per second, the latter
is design. A well-designed, more efficient processor can outsmart a
poorly-designed one that works faster.
Ari
May 25, 2009, 3:12:53 PM5/25/09
to Dual N-Back, Brain Training & Intelligence
Also, I've uploaded the PDF version.
Message has been deleted
Ron Williams
May 25, 2009, 5:53:53 PM5/25/09
to brain-t...@googlegroups.com
My point is that this is _not_ a continuum, and so one can achievce a
speed increase that will make the easier tasks easier, but leave the
difficult tasks still impossible.
speed increase that will make the easier tasks easier, but leave the
difficult tasks still impossible.
Pontus Granström
May 25, 2009, 5:55:14 PM5/25/09
to brain-t...@googlegroups.com
But then you are saying that these questions doesnt take any "analytical intelligence" into account just speed and memory?
Ron Williams
May 25, 2009, 5:56:49 PM5/25/09
to brain-t...@googlegroups.com
I meant to say 'speed/working memory increase'
Mike L.
May 25, 2009, 6:36:49 PM5/25/09
to Dual N-Back, Brain Training & Intelligence
The examples you give, though, do not relate with the study that was
done; The people in the study became quicker not after having
ingested something alien but rather, after having trained.
The example I pose, and one which i think bears more fruit in
something like the study that was done is the following:
A man who can max at 10 repetitions (in the span of 30 seconds) daily
of a weight of 200 pounds will eventually build muscle and become
more
efficient in his reps for that weight as well as become capable of
lifting more weight. This same man, then, after 19 days of training
(familiar?) would have (theoretically) gained in muscle and can lift
now 250 pounds and do a max of 8 reps (same span of 30 seconds). Now,
if this same man were to revisit the earlier weight of 200 pounds, he
would find that his ability to lift that weight would be much greater
and thus, a weight which he once could max at 10 reps for he can now
max at 15 reps (span of 30 seconds) thus, not only becoming more
efficient by virtue of his ability to lift more weight, but quicker
in
his ability to do more reps of the same weight (200 lb) in the same
time.
done; The people in the study became quicker not after having
ingested something alien but rather, after having trained.
The example I pose, and one which i think bears more fruit in
something like the study that was done is the following:
A man who can max at 10 repetitions (in the span of 30 seconds) daily
of a weight of 200 pounds will eventually build muscle and become
more
efficient in his reps for that weight as well as become capable of
lifting more weight. This same man, then, after 19 days of training
(familiar?) would have (theoretically) gained in muscle and can lift
now 250 pounds and do a max of 8 reps (same span of 30 seconds). Now,
if this same man were to revisit the earlier weight of 200 pounds, he
would find that his ability to lift that weight would be much greater
and thus, a weight which he once could max at 10 reps for he can now
max at 15 reps (span of 30 seconds) thus, not only becoming more
efficient by virtue of his ability to lift more weight, but quicker
in
his ability to do more reps of the same weight (200 lb) in the same
time.
Toto
May 26, 2009, 5:23:18 AM5/26/09
to Dual N-Back, Brain Training & Intelligence
to say that performing something quicker but not finding it
> easier to do, is almost illogical, and therefore, to say that harder
> tasks remain just as hard for the person who has found it easier to
> perform the easy tasks just doesn't make sense.
Of course, the easy tasks became easier to do, but that doesnot yet
> easier to do, is almost illogical, and therefore, to say that harder
> tasks remain just as hard for the person who has found it easier to
> perform the easy tasks just doesn't make sense.
mean that the harder became easier too. You probably have seen a
culture-free test. Once you understand the logic behind a problem, you
have to remember what elements the correct answer should include. You
have to form an image of it, if you can, or check the elements one by
one. If you don't understand the logic, WM is of no use.
> But wait, isn't the speed or even the capacity with which one is
> strongly correlated with intelligence?
It is strongly correlated if it's not trained. If there is correlation
between two things, that doesn't necessarily mean that one causes the
other. There is said to be some correlation between looks and
intelligence. Does this mean that if you have a plastic operation it
will make you smarter?
I admit that by training WM it may be possible to increase IQ SCORE
signifficantly if the test is an easy one or if the initial score was
low , but then intelligence and score will probably be two different
things.
"hmm, someone else posted on here that you had already finished your
studies. Guess it was a misinformation".
the control group yet. It seems he still hasn't. So, it was not a
misinformation, it was a misunderstanding.
Pontus Granström
May 26, 2009, 5:33:47 AM5/26/09
to brain-t...@googlegroups.com
But havent you missed the whole point here Toto? What you are claiming is that the easier questions are just a matter of working memory? How can you then call it a test of analytical intelligence (formally you would have to split the test)?
Who decides what's easy and not? To me the interesting thing that's also mentioned is that it's the shared neural networks that causes the increase in IQ (read the article i posted) and that there is a high correlation between "lure trials" and RAPM score (based on the neurological structure and plasticity). The increased score could not be accounted for by increase in working memory (since all people increased their IQ scores).
Who decides what's easy and not? To me the interesting thing that's also mentioned is that it's the shared neural networks that causes the increase in IQ (read the article i posted) and that there is a high correlation between "lure trials" and RAPM score (based on the neurological structure and plasticity). The increased score could not be accounted for by increase in working memory (since all people increased their IQ scores).
Pontus Granström
May 26, 2009, 5:39:26 AM5/26/09
to brain-t...@googlegroups.com
All questions are supposed to be "harder" and as a such they cannot be equally or all depend on the same thing because if they did they would lose power of dicriminate.
Toto
May 26, 2009, 6:21:01 AM5/26/09
to Dual N-Back, Brain Training & Intelligence
I haven't missed anything. I am not saying that the easier questions
are just a matter of working memory. I'm just saying they are within
the reach of most people. To solve a problem you need to understand
its logic. Then its all about memory.
Easy is what most people can do.
>To me the interesting thing that's also
> mentioned is that it's the shared neural networks that causes the increase
> in IQ .
That must be a hypothesis. We still don't even know if there is an
increase in IQ.
>The increased score could not be accounted for by increase
> in working memory (since all people increased their IQ scores).
All people gave more correct answers. Those with lowest initial scores
showed most improvement. There was increase in the scores as long as
there was increase in n-level (look at the graphics in the study
paper). We don't know what happens afterwards.
The reason I think that IQ score cannot be increased if the test is a
difficult one is my own experience. I took a high-ceiling timed test
before and after training and there was no difference. I could solve
the easier items before DNB, and the difficult ... well, they were
still difficult for me :). I admit that at first I thought you were
all suffering from a mass psychosis :) I hope there really is some
improvement on easier tasks, though I'm sure many people just take a
test too many times, less than six moths after the first time (some
tests should not be taken a second time at all) , or refuse to
understand that scores on different tests could be different too.
On May 26, 12:33 pm, Pontus Granström <lepon...@gmail.com> wrote:
> But havent you missed the whole point here Toto? What you are claiming is
> that the easier questions are just a matter of working memory? How can you
> then call it a test of analytical intelligence (formally you would have to
> split the test)?
> Who decides what's easy and not? To me the interesting thing that's also
> mentioned is that it's the shared neural networks that causes the increase
> in IQ (read the article i posted) and that there is a high correlation
> between "lure trials" and RAPM score (based on the neurological structure
> and plasticity). The increased score could not be accounted for by increase
> in working memory (since all people increased their IQ scores).
>
are just a matter of working memory. I'm just saying they are within
the reach of most people. To solve a problem you need to understand
its logic. Then its all about memory.
Easy is what most people can do.
>To me the interesting thing that's also
> mentioned is that it's the shared neural networks that causes the increase
That must be a hypothesis. We still don't even know if there is an
increase in IQ.
>The increased score could not be accounted for by increase
> in working memory (since all people increased their IQ scores).
showed most improvement. There was increase in the scores as long as
there was increase in n-level (look at the graphics in the study
paper). We don't know what happens afterwards.
The reason I think that IQ score cannot be increased if the test is a
difficult one is my own experience. I took a high-ceiling timed test
before and after training and there was no difference. I could solve
the easier items before DNB, and the difficult ... well, they were
still difficult for me :). I admit that at first I thought you were
all suffering from a mass psychosis :) I hope there really is some
improvement on easier tasks, though I'm sure many people just take a
test too many times, less than six moths after the first time (some
tests should not be taken a second time at all) , or refuse to
understand that scores on different tests could be different too.
On May 26, 12:33 pm, Pontus Granström <lepon...@gmail.com> wrote:
> But havent you missed the whole point here Toto? What you are claiming is
> that the easier questions are just a matter of working memory? How can you
> then call it a test of analytical intelligence (formally you would have to
> split the test)?
> Who decides what's easy and not? To me the interesting thing that's also
> mentioned is that it's the shared neural networks that causes the increase
> in IQ (read the article i posted) and that there is a high correlation
> between "lure trials" and RAPM score (based on the neurological structure
> and plasticity). The increased score could not be accounted for by increase
> in working memory (since all people increased their IQ scores).
>
Pontus Granström
May 26, 2009, 6:36:11 AM5/26/09
to brain-t...@googlegroups.com
So you mean that the IQ-score isnt gaussian distributed, that it's more "flat" and then raises (doesnt make any mathematical sense)? What you are saying is that those who score low doesnt have a low IQ just not a good enough working memory . If it were like that a person could score low on the easy part (due to a lack of WM) but complete the more difficult items?. I still doesnt think your reasoning is consistent/logical.
Toto
May 26, 2009, 7:26:22 AM5/26/09
to Dual N-Back, Brain Training & Intelligence
"So you mean that the IQ-score isnt gaussian distributed, that it's
more
"flat" and then raises (doesnt make any mathematical sense)?"
I have no idea what you are talking about :)
more
"flat" and then raises (doesnt make any mathematical sense)?"
" What you are
saying is that those who score low doesnt have a low IQ just not a
good
enough working memory".
in most cases those with bad WM will have low intelligence too.
"If it were like that a person could score low on the
easy part (due to a lack of WM) but complete the more difficult
items?"
have certain amount of time for the whole test, and the score does not
depend on the difficulty of the problems solved, it depends on their
number. If WM capacity is lower, even if you are very intelligent, it
probably will take you more time to solve a problem. The result will
be less problems solved and, consequently, lower score. If the
problems in a test are more difficult, they are supposed to be solved
in more time and the stress is not so much on WM, it is on some
reasoning ability.
On May 26, 1:36 pm, Pontus Granström <lepon...@gmail.com> wrote:
> So you mean that the IQ-score isnt gaussian distributed, that it's more
> "flat" and then raises (doesnt make any mathematical sense)? What you are
> saying is that those who score low doesnt have a low IQ just not a good
> enough working memory . If it were like that a person could score low on the
> easy part (due to a lack of WM) but complete the more difficult items?. I
> still doesnt think your reasoning is consistent/logical.
>
Pontus Granström
May 26, 2009, 9:56:38 AM5/26/09
to brain-t...@googlegroups.com
Well have you ever heard about the normal distrubtion, IQ is supposed to be distrubuted like that. The problems on ravens etc takes you more to the right on the scale for every problem you solve. So if you have 4 more right questions your IQ will probaly go up with something like 10-15 points. But this shouldnt then be considerad a gain in IQ just in "working memory" according to you, so say someone scores 10 correct answers then 14 he should still not be consider to have a analytical intelligence just a good working memory? At which type of questions does not working memory play a role?
Pontus Granström
May 26, 2009, 10:01:18 AM5/26/09
to brain-t...@googlegroups.com
Is it really fair to assume that all "easy questions" depends to 100% on working memory and all hard doesnt?
Wade
May 26, 2009, 10:35:29 AM5/26/09
to Dual N-Back, Brain Training & Intelligence
Vlad,
So people are participating in your study and they aren't even showing
up to take the test? I wonder why. Maybe they feel they didn't get too
far on dual-n-back so don't feel motivated to retake?
When you say they are scoring 2 or 3 points higher, do you mean their
IQ scores? Or you mean they are successfully answering 2 or 3 more
questions? Also, when they retake, is it the exact same questions or
is their an alternate form to the Raven's?
So people are participating in your study and they aren't even showing
up to take the test? I wonder why. Maybe they feel they didn't get too
far on dual-n-back so don't feel motivated to retake?
When you say they are scoring 2 or 3 points higher, do you mean their
IQ scores? Or you mean they are successfully answering 2 or 3 more
questions? Also, when they retake, is it the exact same questions or
is their an alternate form to the Raven's?
Wade
May 26, 2009, 10:52:38 AM5/26/09
to Dual N-Back, Brain Training & Intelligence
Ron,
Well I guess that testing intelligence cannot be a continuum from
easier questions to harder ones since the the questions provide
discrete feedback on a person's performance. However, this doesn't
mean that the brain circuitry that is responsible for varying levels
of performance on these tests, don't vary according to a continuum
from slow (with poor analytical ability) to fast (with good analytical
ability) --necessarily.
My point is that, to my knowledge at least, it is not established that
the differences in the neural circuitry that are responsible for
faster speed of processing in one person and slower processing in
another person, and better wm in one person and poorer wm in another,
are not the exact same circuitry. In fact, according to many
neuroscientists, it's tempting to think they are since they covary. In
other words the phenomena of "fast thinking" and "working memory" may
only be epiphenomena of the same physical circuitry.
Also, while we do understand "analytical ability" as separate
phenomena, it is still a human construct that we don't yet know just
how it relates to other human constructs like wm and speed. Increases
in wm may tend to improve speed and may tend to improve working
memory, might also improve analytical ability. Maybe analytical
ability improves with improved processing speed (not CAUSED by).
IQ tests, which are discrete, may not be sensitive enough to catch,
what are very likely, subtle changes in these things after only a
month of training.
Well I guess that testing intelligence cannot be a continuum from
easier questions to harder ones since the the questions provide
discrete feedback on a person's performance. However, this doesn't
mean that the brain circuitry that is responsible for varying levels
of performance on these tests, don't vary according to a continuum
from slow (with poor analytical ability) to fast (with good analytical
ability) --necessarily.
My point is that, to my knowledge at least, it is not established that
the differences in the neural circuitry that are responsible for
faster speed of processing in one person and slower processing in
another person, and better wm in one person and poorer wm in another,
are not the exact same circuitry. In fact, according to many
neuroscientists, it's tempting to think they are since they covary. In
other words the phenomena of "fast thinking" and "working memory" may
only be epiphenomena of the same physical circuitry.
Also, while we do understand "analytical ability" as separate
phenomena, it is still a human construct that we don't yet know just
how it relates to other human constructs like wm and speed. Increases
in wm may tend to improve speed and may tend to improve working
memory, might also improve analytical ability. Maybe analytical
ability improves with improved processing speed (not CAUSED by).
IQ tests, which are discrete, may not be sensitive enough to catch,
what are very likely, subtle changes in these things after only a
month of training.
Wade
May 26, 2009, 10:58:34 AM5/26/09