The May 13, 2008 issue of Proceedings of the National Academy of
Sciences featured a cover article that purported to demonstrate
increases in fluid intelligence following training on a task of
working memory (Jaeggi, Buschkuehl, Jonides, & Perrig, 2008). The
authors described their own findings as a “landmark result”. Their
study was the subject of an introductory comment by Robert Sternberg
(2008), as well as articles in the mainstream media, including a
lengthy column in a recent edition of The New York Times (Wang &
Aamodt, 2009).
In view of the potential significance of the study and the quantity of
attention it has received, the results have been subjected to
remarkably little critical analysis. A close examination of the
evidence reported by Jaeggi et al. shows that it is not in fact
sufficient to support the authors' conclusion of any increase in their
subjects' fluid intelligence.
What Jaeggi et al. reported were modest increases in performance on a
test of fluid intelligence following several days of training on a
task of working memory. The reported increases in performance are not
in question here. But the manner in which the test was administered
severely undermines the authors' interpretation that their subjects'
intelligence itself was increased.
The subjects were divided into four groups, differing in the number of
days of training they received on the task of working memory. The
group that received the least training (8 days) was tested on Raven's
Advanced Progressive Matrices (Raven, 1990), a widely used and well-
established test of fluid intelligence. This group, however,
demonstrated negligible improvement between pre- and post-test
performance.
The other three groups were not tested using Raven's Matrices, but
rather on an alternative test of much more recent origin. The Bochumer
Matrices Test (BOMAT) (Hossiep, Turck, & Hasella, 1999) is similar to
Raven's in that it consists of visual analogies. In both tests, a
series of geometric and other figures is presented in a matrix format
and the subject is required to infer a pattern in order to predict the
next figure in the series. The authors provide no reason for switching
from Raven's to the BOMAT.
The BOMAT differs from Raven's in some important respects, but is
similar in one crucial attribute: both tests are progressive in
nature, which means that test items are sequentially arranged in order
of increasing difficulty. A high score on the test, therefore, is
predicated on subjects' ability to solve the more difficult items.
However, this progressive feature of the test was effectively
eliminated by the manner in which Jaeggi et al. adminstered it. The
BOMAT is a 29-item test which subjects are supposed to be allowed 45
min to complete. Remarkably, however, Jaeggi et al. reduced the
allotted time from 45 min to 10. The effect of this restriction was to
make it impossible for subjects to proceed to the more difficult items
on the test. The large majority of the subjects—regardless of the
number of days of training they received—answered less than 14 test
items correctly.
By virtue of the manner in which they administered the BOMAT, Jaeggi
et al. transformed it from a test of fluid intelligence into a speed
test of ability to solve the easier visual analogies.
The time restriction not only made it impossible for subjects to
proceed to the more difficult items, it also limited the opportunity
to learn about the test—and so improve performance—in the process of
taking it. This factor cannot be neglected because test performance
does improve with practice, as demonstrated by the control groups in
the Jaeggi study, whose improvement from pre- to post-test was about
half that of the experimental groups. The same learning process that
occurs from one administration of the test to the next may also
operate within a given administration of the test—provided subjects
are allowed sufficient time to complete it.
Since the whole weight of their conclusion rests upon the validity of
their measure of fluid intelligence, one might assume the authors
would present a careful defense of the manner in which they
administered the BOMAT. Instead they do not even mention that subjects
are normally allowed 45 min to complete the test. Nor do they mention
that the test has 29 items, of which most of their subjects completed
less than half.
The authors' entire rationale for reducing the allotted time to 10 min
is confined to a footnote. That footnote reads as follows:
Although this procedure differs from the standardized procedure,
there is evidence that this timed procedure has little influence on
relative standing in these tests, in that the correlation of speeded
and non-speeded versions is very high (r = 0.95; ref. 37).
The reference given in the footnote is to a 1988 study (Frearson &
Eysenck, 1986) that is not in fact designed to support the conclusion
stated by Jaeggi et al. The 1988 study merely contains a footnote of
its own, which refers in turn to unpublished research conducted forty
years earlier. That research involved Raven's matrices, not the BOMAT,
and entailed a reduction in time of at most 50%, not more than 75%, as
in the Jaeggi study.
So instead of offering a reasoned defense of their procedure, Jaeggi
et al. provide merely a footnote which refers in turn to a footnote in
another study. The second footnote describes unpublished results,
evidently recalled by memory over a span of 40 years, involving a
different test and a much less severe reduction in time.
In this context it bears repeating that the group that was tested on
Raven's matrices (with presumably the same time restriction) showed
virtually no improvement in test performance, in spite of eight days'
training on working memory. Performance gains only appeared for the
groups administered the BOMAT. But the BOMAT differs in one important
respect from Raven's. Raven's matrices are presented in a 3 × 3
format, whereas the BOMAT consists of a 5 × 3 matrix configuration.
With 15 visual figures to keep track of in each test item instead of
9, the BOMAT puts added emphasis on subjects' ability to hold details
of the figures in working memory, especially under the condition of a
severe time constraint. Therefore it is not surprising that extensive
training on a task of working memory would facilitate performance on
the early and easiest BOMAT test items—those that present less of a
challenge to fluid intelligence.
This interpretation acquires added plausibility from the nature of one
of the two working-memory tasks administered to the experimental
groups. The authors maintain that those tasks were “entirely
different” from the test of fluid intelligence. One of the tasks
merits that description: it was a sequence of letters presented
auditorily through headphones.
But the other working-memory task involved recall of the location of a
small square in one of several positions in a visual matrix pattern.
It represents in simplified form precisely the kind of detail required
to solve visual analogies. Rather than being “entirely different” from
the test items on the BOMAT, this task seems well-designed to
facilitate performance on that test.
More generally, the foregoing considerations suggest a deeper problem
with the conclusions presented by Jaeggi et al.: To what extent does
improvement on any test of fluid intelligence reflect an increase in
actual intelligence rather than merely an increase in test-taking
skills? A full analysis of this issue is beyond the scope of the
present review, but the methodological challenges involved are
formidable and deserve further discussion.
Whatever the meaning of the modest gains in performance on the BOMAT,
the evidence produced by Jaeggi et al. does not support the conclusion
of an increase in their subjects' intelligence. Their research may be
sufficient to encourage further investigation, but any larger
inferences are unwarranted.
An interesting point I would like to bring into this thread is the
mention earlier of people with autism performing higher than average
on the raven's with a lower average working memory. That suggests to
me that the raven's as a test may not require a large working memory
capacity. It looks to me like there may be some other bottleneck for
this test.
It is rather curious that they administered the tests in that fashion
and that they changed the tests mid-experiment. I hope that in the
future a more extensive test is administered after an even longer
period of DNB training. If all this is good for is increasing our
speed at solving problems we could already solve, I still feel that it
is an incredibly valuable exercise.
There are other studies indicating significant improvements in ADHD
symptoms and unrelated skills, so I still maintain the belief that DNB
is good for us and we are improving something vitally important by
training.
> The May 13, 2008 issue of Proceedings of the National Academy of
> Sciences featured a cover article that purported to demonstrate
> increases in fluid intelligence following training on a task of
> working memory (Jaeggi, Buschkuehl, Jonides, & Perrig, 2008). The
> authors described their own findings as a “landmark result”. Their
> study was the subject of an introductory comment by Robert Sternberg
> (2008), as well as articles in the mainstream media, including a
> lengthy column in a recent edition of The New York Times (Wang &
> Aamodt, 2009).
> In view of the potential significance of the study and the quantity of
> attention it has received, the results have been subjected to
> remarkably little critical analysis. A close examination of the
> evidence reported by Jaeggi et al. shows that it is not in fact
> sufficient to support the authors' conclusion of any increase in their
> subjects' fluid intelligence.
> What Jaeggi et al. reported were modest increases in performance on a
> test of fluid intelligence following several days of training on a
> task of working memory. The reported increases in performance are not
> in question here. But the manner in which the test was administered
> severely undermines the authors' interpretation that their subjects'
> intelligence itself was increased.
> The subjects were divided into four groups, differing in the number of
> days of training they received on the task of working memory. The
> group that received the least training (8 days) was tested on Raven's
> Advanced Progressive Matrices (Raven, 1990), a widely used and well-
> established test of fluid intelligence. This group, however,
> demonstrated negligible improvement between pre- and post-test
> performance.
> The other three groups were not tested using Raven's Matrices, but
> rather on an alternative test of much more recent origin. The Bochumer
> Matrices Test (BOMAT) (Hossiep, Turck, & Hasella, 1999) is similar to
> Raven's in that it consists of visual analogies. In both tests, a
> series of geometric and other figures is presented in a matrix format
> and the subject is required to infer a pattern in order to predict the
> next figure in the series. The authors provide no reason for switching
> from Raven's to the BOMAT.
> The BOMAT differs from Raven's in some important respects, but is
> similar in one crucial attribute: both tests are progressive in
> nature, which means that test items are sequentially arranged in order
> of increasing difficulty. A high score on the test, therefore, is
> predicated on subjects' ability to solve the more difficult items.
> However, this progressive feature of the test was effectively
> eliminated by the manner in which Jaeggi et al. adminstered it. The
> BOMAT is a 29-item test which subjects are supposed to be allowed 45
> min to complete. Remarkably, however, Jaeggi et al. reduced the
> allotted time from 45 min to 10. The effect of this restriction was to
> make it impossible for subjects to proceed to the more difficult items
> on the test. The large majority of the subjects—regardless of the
> number of days of training they received—answered less than 14 test
> items correctly.
> By virtue of the manner in which they administered the BOMAT, Jaeggi
> et al. transformed it from a test of fluid intelligence into a speed
> test of ability to solve the easier visual analogies.
> The time restriction not only made it impossible for subjects to
> proceed to the more difficult items, it also limited the opportunity
> to learn about the test—and so improve performance—in the process of
> taking it. This factor cannot be neglected because test performance
> does improve with practice, as demonstrated by the control groups in
> the Jaeggi study, whose improvement from pre- to post-test was about
> half that of the experimental groups. The same learning process that
> occurs from one administration of the test to the next may also
> operate within a given administration of the test—provided subjects
> are allowed sufficient time to complete it.
> Since the whole weight of their conclusion rests upon the validity of
> their measure of fluid intelligence, one might assume the authors
> would present a careful defense of the manner in which they
> administered the BOMAT. Instead they do not even mention that subjects
> are normally allowed 45 min to complete the test. Nor do they mention
> that the test has 29 items, of which most of their subjects completed
> less than half.
> The authors' entire rationale for reducing the allotted time to 10 min
> is confined to a footnote. That footnote reads as follows:
> Although this procedure differs from the standardized procedure,
> there is evidence that this timed procedure has little influence on
> relative standing in these tests, in that the correlation of speeded
> and non-speeded versions is very high (r = 0.95; ref. 37).
> The reference given in the footnote is to a 1988 study (Frearson &
> Eysenck, 1986) that is not in fact designed to support the conclusion
> stated by Jaeggi et al. The 1988 study merely contains a footnote of
> its own, which refers in turn to unpublished research conducted forty
> years earlier. That research involved Raven's matrices, not the BOMAT,
> and entailed a reduction in time of at most 50%, not more than 75%, as
> in the Jaeggi study.
> So instead of offering a reasoned defense of their procedure, Jaeggi
> et al. provide merely a footnote which refers in turn to a footnote in
> another study. The second footnote describes unpublished results,
> evidently recalled by memory over a span of 40 years, involving a
> different test and a much less severe reduction in time.
> In this context it bears repeating that the group that was tested on
> Raven's matrices (with presumably the same time restriction) showed
> virtually no improvement in test performance, in spite of eight days'
> training on working memory. Performance gains only appeared for the
> groups administered the BOMAT. But the BOMAT differs in one important
> respect from Raven's. Raven's matrices are presented in a 3 × 3
> format, whereas the BOMAT consists of a 5 × 3 matrix configuration.
> With 15 visual figures to keep track of in each test item instead of
> 9, the BOMAT puts added emphasis on subjects' ability to hold details
> of the figures in working memory, especially under the condition of a
> severe time constraint. Therefore it is not surprising that extensive
> training on a task of working memory would facilitate performance on
> the early and easiest BOMAT test items—those that present less of a
> challenge to fluid intelligence.
> This interpretation acquires added plausibility from the nature of one
> of the two working-memory tasks administered to the experimental
> groups. The authors maintain that those tasks were “entirely
> different” from the test of fluid intelligence. One of the tasks
> merits that description: it was a sequence of letters presented
> auditorily through headphones.
> But the other working-memory task involved recall of the location of a
> small square in one of several positions in a visual matrix pattern.
> It represents in simplified form precisely the kind of detail required
> to solve visual analogies. Rather than being “entirely different” from
> the test items on the BOMAT, this task seems well-designed to
> facilitate performance on that test.
> More generally, the foregoing considerations suggest a deeper problem
> with the conclusions presented by Jaeggi et al.: To what extent does
> improvement on any test of fluid intelligence reflect an increase in
> actual intelligence rather than merely an increase in test-taking
> skills? A full analysis of this issue is beyond the scope of the
> present review, but the methodological challenges involved are
> formidable and deserve further discussion.
> Whatever the meaning of the modest gains in performance on the BOMAT,
> the evidence produced by Jaeggi et al. does not support the conclusion
> of an increase in their subjects' intelligence. Their research may be
> sufficient to encourage further investigation, but any larger
> inferences are unwarranted.
Here's another report supporting the claim. "General intelligence (*g*) is
highly correlated with working-memory capacity (WMC). It has been argued
that these central psychological constructs should share common neural
systems. The present study examines this hypothesis using structural
magnetic resonance imaging to determine any overlap in brain areas where
regional grey matter volumes are correlated to measures of general
intelligence and to memory span. In normal volunteers (*N* = 48) the results
(*p* < .05, corrected for multiple comparisons) indicate that a common
anatomic framework for these constructs implicates mainly frontal grey
matter regions belonging to Brodmann area (BA) 10 (right superior frontal
gyrus and left middle frontal gyrus) and, to a lesser degree, the right
inferior parietal lobule (BA 40). These findings support the nuclear role of
a discrete parieto-frontal network. "
>> The May 13, 2008 issue of Proceedings of the National Academy of
>> Sciences featured a cover article that purported to demonstrate
>> increases in fluid intelligence following training on a task of
>> working memory (Jaeggi, Buschkuehl, Jonides, & Perrig, 2008). The
>> authors described their own findings as a “landmark result”. Their
>> study was the subject of an introductory comment by Robert Sternberg
>> (2008), as well as articles in the mainstream media, including a
>> lengthy column in a recent edition of The New York Times (Wang &
>> Aamodt, 2009).
>> In view of the potential significance of the study and the quantity of
>> attention it has received, the results have been subjected to
>> remarkably little critical analysis. A close examination of the
>> evidence reported by Jaeggi et al. shows that it is not in fact
>> sufficient to support the authors' conclusion of any increase in their
>> subjects' fluid intelligence.
>> What Jaeggi et al. reported were modest increases in performance on a
>> test of fluid intelligence following several days of training on a
>> task of working memory. The reported increases in performance are not
>> in question here. But the manner in which the test was administered
>> severely undermines the authors' interpretation that their subjects'
>> intelligence itself was increased.
>> The subjects were divided into four groups, differing in the number of
>> days of training they received on the task of working memory. The
>> group that received the least training (8 days) was tested on Raven's
>> Advanced Progressive Matrices (Raven, 1990), a widely used and well-
>> established test of fluid intelligence. This group, however,
>> demonstrated negligible improvement between pre- and post-test
>> performance.
>> The other three groups were not tested using Raven's Matrices, but
>> rather on an alternative test of much more recent origin. The Bochumer
>> Matrices Test (BOMAT) (Hossiep, Turck, & Hasella, 1999) is similar to
>> Raven's in that it consists of visual analogies. In both tests, a
>> series of geometric and other figures is presented in a matrix format
>> and the subject is required to infer a pattern in order to predict the
>> next figure in the series. The authors provide no reason for switching
>> from Raven's to the BOMAT.
>> The BOMAT differs from Raven's in some important respects, but is
>> similar in one crucial attribute: both tests are progressive in
>> nature, which means that test items are sequentially arranged in order
>> of increasing difficulty. A high score on the test, therefore, is
>> predicated on subjects' ability to solve the more difficult items.
>> However, this progressive feature of the test was effectively
>> eliminated by the manner in which Jaeggi et al. adminstered it. The
>> BOMAT is a 29-item test which subjects are supposed to be allowed 45
>> min to complete. Remarkably, however, Jaeggi et al. reduced the
>> allotted time from 45 min to 10. The effect of this restriction was to
>> make it impossible for subjects to proceed to the more difficult items
>> on the test. The large majority of the subjects—regardless of the
>> number of days of training they received—answered less than 14 test
>> items correctly.
>> By virtue of the manner in which they administered the BOMAT, Jaeggi
>> et al. transformed it from a test of fluid intelligence into a speed
>> test of ability to solve the easier visual analogies.
>> The time restriction not only made it impossible for subjects to
>> proceed to the more difficult items, it also limited the opportunity
>> to learn about the test—and so improve performance—in the process of
>> taking it. This factor cannot be neglected because test performance
>> does improve with practice, as demonstrated by the control groups in
>> the Jaeggi study, whose improvement from pre- to post-test was about
>> half that of the experimental groups. The same learning process that
>> occurs from one administration of the test to the next may also
>> operate within a given administration of the test—provided subjects
>> are allowed sufficient time to complete it.
>> Since the whole weight of their conclusion rests upon the validity of
>> their measure of fluid intelligence, one might assume the authors
>> would present a careful defense of the manner in which they
>> administered the BOMAT. Instead they do not even mention that subjects
>> are normally allowed 45 min to complete the test. Nor do they mention
>> that the test has 29 items, of which most of their subjects completed
>> less than half.
>> The authors' entire rationale for reducing the allotted time to 10 min
>> is confined to a footnote. That footnote reads as follows:
>> Although this procedure differs from the standardized procedure,
>> there is evidence that this timed procedure has little influence on
>> relative standing in these tests, in that the correlation of speeded
>> and non-speeded versions is very high (r = 0.95; ref. 37).
>> The reference given in the footnote is to a 1988 study (Frearson &
>> Eysenck, 1986) that is not in fact designed to support the conclusion
>> stated by Jaeggi et al. The 1988 study merely contains a footnote of
>> its own, which refers in turn to unpublished research conducted forty
>> years earlier. That research involved Raven's matrices, not the BOMAT,
>> and entailed a reduction in time of at most 50%, not more than 75%, as
>> in the Jaeggi study.
>> So instead of offering a reasoned defense of their procedure, Jaeggi
>> et al. provide merely a footnote which refers in turn to a footnote in
>> another study. The second footnote describes unpublished results,
>> evidently recalled by memory over a span of 40 years, involving a
>> different test and a much less severe reduction in time.
>> In this context it bears repeating that the group that was tested on
>> Raven's matrices (with presumably the same time restriction) showed
>> virtually no improvement in test performance, in spite of eight days'
>> training on working memory. Performance gains only appeared for the
>> groups administered the BOMAT. But the BOMAT differs in one important
>> respect from Raven's. Raven's matrices are presented in a 3 × 3
>> format, whereas the BOMAT consists of a 5 × 3 matrix configuration.
>> With 15 visual figures to keep track of in each test item instead of
>> 9, the BOMAT puts added emphasis on subjects' ability to hold details
>> of the figures in working memory, especially under the condition of a
>> severe time constraint. Therefore it is not surprising that extensive
>> training on a task of working memory would facilitate performance on
>> the early and easiest BOMAT test items—those that present less of a
>> challenge to fluid intelligence.
>> This interpretation acquires added plausibility from the nature of one
>> of the two working-memory tasks administered to the experimental
>> groups. The authors maintain that those tasks were “entirely
>> different” from the test of fluid intelligence. One of the tasks
>> merits that description: it was a sequence of letters presented
>> auditorily through headphones.
>> But the other working-memory task involved recall of the location of a
>> small square in one of several positions in a visual matrix pattern.
>> It represents in simplified form precisely the kind of detail required
>> to solve visual analogies. Rather than being “entirely different” from
>> the test items on the BOMAT, this task seems well-designed to
>> facilitate performance on that test.
>> More generally, the foregoing considerations suggest a deeper problem
>> with the conclusions presented by Jaeggi et al.: To what extent does
>> improvement on any test of fluid intelligence reflect an increase in
>> actual intelligence rather than merely an increase in test-taking
>> skills? A full analysis of this issue is beyond the scope of the
>> present review, but the methodological challenges involved are
>> formidable and deserve further discussion.
>> Whatever the meaning of the modest gains in performance on the BOMAT,
>> the evidence produced by Jaeggi et al. does
Moody brings up an excellent point, that perhaps DNB really does train
for the easier questions on the BOMAT, considering the larger working
memory load of the BOMAT. While he does highlight flaws in the Jaeggi
study, there are somethings I would like to point out in Moody's
critique.
He admits that improvement on the BOMAT happens because of
improvements in spatial working memory, which DNB trains. He seems to
imply that there is no proof that DNB improves one's analytical
ability (which may be true), which is crucial for solving the more
complex problems on the Raven's. Instead, he says the improvements
are shown because improved working memory alone can help on the easier
portions of the BOMAT.
This may be true, but there are plenty of studies that suggest a
correlation with working memory and Fluid intelligence, so it would
seem reasonable that improving one will improve the other.
Also, isn't it still significant that DNB improves working memory?
Even if that is all it does, I'd say it is still pretty helpful in the
real world.
Yes. While Raven's seems to be more of a test of ability to deal with
complexity (intelligence?), the earlier levels of BOMAT test mostly
working memory. Working memory is indisputably related to
intelligence, but it may be the case that DNB training improves only
speed or capacity, not intelligence however constituted. As you point
out, though, this is a useful skill regardless.
On May 24, 12:58 pm, Iron <IronPanc...@gmail.com> wrote:
> An interesting point I would like to bring into this thread is the
> mention earlier of people with autism performing higher than average
> on the raven's with a lower average working memory. That suggests to
> me that the raven's as a test may not require a large working memory
> capacity. It looks to me like there may be some other bottleneck for
> this test.
> It is rather curious that they administered the tests in that fashion
> and that they changed the tests mid-experiment. I hope that in the
> future a more extensive test is administered after an even longer
> period of DNB training. If all this is good for is increasing our
> speed at solving problems we could already solve, I still feel that it
> is an incredibly valuable exercise.
> There are other studies indicating significant improvements in ADHD
> symptoms and unrelated skills, so I still maintain the belief that DNB
> is good for us and we are improving something vitally important by
> training.
But you forget that they saw improvements even among those with a high
capacity working memory.
"However, our additional analyses show that there is more to
transfer than mere improvement in working memory capacity in
that the increase in Gf was not directly related to either
preexisting individual differences in working memory capacity
or to the gain in working memory capacity as measured by simple
or complex spans, or even, by the specific training effect itself."
On Sun, May 24, 2009 at 8:33 PM, Ari <arizz...@gmail.com> wrote:
> Yes. While Raven's seems to be more of a test of ability to deal with
> complexity (intelligence?), the earlier levels of BOMAT test mostly
> working memory. Working memory is indisputably related to
> intelligence, but it may be the case that DNB training improves only
> speed or capacity, not intelligence however constituted. As you point
> out, though, this is a useful skill regardless.
> On May 24, 12:58 pm, Iron <IronPanc...@gmail.com> wrote:
> > An interesting point I would like to bring into this thread is the
> > mention earlier of people with autism performing higher than average
> > on the raven's with a lower average working memory. That suggests to
> > me that the raven's as a test may not require a large working memory
> > capacity. It looks to me like there may be some other bottleneck for
> > this test.
> > It is rather curious that they administered the tests in that fashion
> > and that they changed the tests mid-experiment. I hope that in the
> > future a more extensive test is administered after an even longer
> > period of DNB training. If all this is good for is increasing our
> > speed at solving problems we could already solve, I still feel that it
> > is an incredibly valuable exercise.
> > There are other studies indicating significant improvements in ADHD
> > symptoms and unrelated skills, so I still maintain the belief that DNB
> > is good for us and we are improving something vitally important by
> > training.
But wait, isn't the speed or even the capacity with which one is
endowed to accomplish whatever task they might have at hand, also
strongly correlated with intelligence?
That would be like saying, a faster and more fuel efficient car does
not have a better engine than that of a slower, less fuel efficient
car.
To use a more concrete example, on the WAIS-IV IQ test, there is
indeed a section on processing information and spatial reasoning, as
well as digit-span memorization. These categories, it can be said,
will indeed rise as due to DNB training, thus, increase overall IQ.
Therefore, though, according to the hypothesis moody devises of DNB
not improving analytical processes, training with DNB will still
result in a higher IQ.
On May 24, 2:33 pm, Ari <arizz...@gmail.com> wrote:
> Yes. While Raven's seems to be more of a test of ability to deal with
> complexity (intelligence?), the earlier levels of BOMAT test mostly
> working memory. Working memory is indisputably related to
> intelligence, but it may be the case that DNB training improves only
> speed or capacity, not intelligence however constituted. As you point
> out, though, this is a useful skill regardless.
> On May 24, 12:58 pm, Iron <IronPanc...@gmail.com> wrote:
> > An interesting point I would like to bring into this thread is the
> > mention earlier of people with autism performing higher than average
> > on the raven's with a lower average working memory. That suggests to
> > me that the raven's as a test may not require a large working memory
> > capacity. It looks to me like there may be some other bottleneck for
> > this test.
> > It is rather curious that they administered the tests in that fashion
> > and that they changed the tests mid-experiment. I hope that in the
> > future a more extensive test is administered after an even longer
> > period of DNB training. If all this is good for is increasing our
> > speed at solving problems we could already solve, I still feel that it
> > is an incredibly valuable exercise.
> > There are other studies indicating significant improvements in ADHD
> > symptoms and unrelated skills, so I still maintain the belief that DNB
> > is good for us and we are improving something vitally important by
> > training.
Moody himself states that the processing of the "easier" tasks became
quicker in those people who trained with DNB. Now, using logic: When
one does something quicker, or finds that they can perform something
quicker, that task has then become easier to do for them. For this
reason, to say that performing something quicker but not finding it
easier to do, is almost illogical, and therefore, to say that harder
tasks remain just as hard for the person who has found it easier to
perform the easy tasks just doesn't make sense.
Taking this into consideration, it is more sensible to believe that
the easy tasks have become easier but the harder tasks have TOO become
easier.
On May 24, 2:54 pm, "Mike L." <cool2bwi...@gmail.com> wrote:
> But wait, isn't the speed or even the capacity with which one is
> endowed to accomplish whatever task they might have at hand, also
> strongly correlated with intelligence?
> That would be like saying, a faster and more fuel efficient car does
> not have a better engine than that of a slower, less fuel efficient
> car.
> To use a more concrete example, on the WAIS-IV IQ test, there is
> indeed a section on processing information and spatial reasoning, as
> well as digit-span memorization. These categories, it can be said,
> will indeed rise as due to DNB training, thus, increase overall IQ.
> Therefore, though, according to the hypothesis moody devises of DNB
> not improving analytical processes, training with DNB will still
> result in a higher IQ.
> On May 24, 2:33 pm, Ari <arizz...@gmail.com> wrote:
> > Yes. While Raven's seems to be more of a test of ability to deal with
> > complexity (intelligence?), the earlier levels of BOMAT test mostly
> > working memory. Working memory is indisputably related to
> > intelligence, but it may be the case that DNB training improves only
> > speed or capacity, not intelligence however constituted. As you point
> > out, though, this is a useful skill regardless.
> > On May 24, 12:58 pm, Iron <IronPanc...@gmail.com> wrote:
> > > An interesting point I would like to bring into this thread is the
> > > mention earlier of people with autism performing higher than average
> > > on the raven's with a lower average working memory. That suggests to
> > > me that the raven's as a test may not require a large working memory
> > > capacity. It looks to me like there may be some other bottleneck for
> > > this test.
> > > It is rather curious that they administered the tests in that fashion
> > > and that they changed the tests mid-experiment. I hope that in the
> > > future a more extensive test is administered after an even longer
> > > period of DNB training. If all this is good for is increasing our
> > > speed at solving problems we could already solve, I still feel that it
> > > is an incredibly valuable exercise.
> > > There are other studies indicating significant improvements in ADHD
> > > symptoms and unrelated skills, so I still maintain the belief that DNB
> > > is good for us and we are improving something vitally important by
> > > training.
I agree, what they are claiming is that the first questions of raven's and
bomat is just questions that takes working memory into account, but then
after a few questions it measures analytical intelligence?
On Sun, May 24, 2009 at 9:52 PM, Mike L. <cool2bwi...@gmail.com> wrote:
> I wanted to add something:
> Moody himself states that the processing of the "easier" tasks became
> quicker in those people who trained with DNB. Now, using logic: When
> one does something quicker, or finds that they can perform something
> quicker, that task has then become easier to do for them. For this
> reason, to say that performing something quicker but not finding it
> easier to do, is almost illogical, and therefore, to say that harder
> tasks remain just as hard for the person who has found it easier to
> perform the easy tasks just doesn't make sense.
> Taking this into consideration, it is more sensible to believe that
> the easy tasks have become easier but the harder tasks have TOO become
> easier.
> On May 24, 2:54 pm, "Mike L." <cool2bwi...@gmail.com> wrote:
> > But wait, isn't the speed or even the capacity with which one is
> > endowed to accomplish whatever task they might have at hand, also
> > strongly correlated with intelligence?
> > That would be like saying, a faster and more fuel efficient car does
> > not have a better engine than that of a slower, less fuel efficient
> > car.
> > To use a more concrete example, on the WAIS-IV IQ test, there is
> > indeed a section on processing information and spatial reasoning, as
> > well as digit-span memorization. These categories, it can be said,
> > will indeed rise as due to DNB training, thus, increase overall IQ.
> > Therefore, though, according to the hypothesis moody devises of DNB
> > not improving analytical processes, training with DNB will still
> > result in a higher IQ.
> > On May 24, 2:33 pm, Ari <arizz...@gmail.com> wrote:
> > > Yes. While Raven's seems to be more of a test of ability to deal with
> > > complexity (intelligence?), the earlier levels of BOMAT test mostly
> > > working memory. Working memory is indisputably related to
> > > intelligence, but it may be the case that DNB training improves only
> > > speed or capacity, not intelligence however constituted. As you point
> > > out, though, this is a useful skill regardless.
> > > On May 24, 12:58 pm, Iron <IronPanc...@gmail.com> wrote:
> > > > An interesting point I would like to bring into this thread is the
> > > > mention earlier of people with autism performing higher than average
> > > > on the raven's with a lower average working memory. That suggests to
> > > > me that the raven's as a test may not require a large working memory
> > > > capacity. It looks to me like there may be some other bottleneck for
> > > > this test.
> > > > It is rather curious that they administered the tests in that fashion
> > > > and that they changed the tests mid-experiment. I hope that in the
> > > > future a more extensive test is administered after an even longer
> > > > period of DNB training. If all this is good for is increasing our
> > > > speed at solving problems we could already solve, I still feel that
> it
> > > > is an incredibly valuable exercise.
> > > > There are other studies indicating significant improvements in ADHD
> > > > symptoms and unrelated skills, so I still maintain the belief that
> DNB
> > > > is good for us and we are improving something vitally important by
> > > > training.
You are assuming that the harder tasks are possible at all for a given person. This may not be so. If you are required to keep ten things in mind all at once and do a manipulation of them to achieve the result, it may never be possible, if you are only capable of holding seven things in mind.
The harder tasks of this kind will remain just has impossible as before, and will result in a big X on the score sheet. On the other hand, the easier tasks might well be done much more quickly than before if there's a speed improvement. I.e. speeding up attempts on the impossible will not give a better result!
On Mon, May 25, 2009 at 5:52 AM, Mike L. <cool2bwi...@gmail.com> wrote:
> I wanted to add something:
> Moody himself states that the processing of the "easier" tasks became > quicker in those people who trained with DNB. Now, using logic: When > one does something quicker, or finds that they can perform something > quicker, that task has then become easier to do for them. For this > reason, to say that performing something quicker but not finding it > easier to do, is almost illogical, and therefore, to say that harder > tasks remain just as hard for the person who has found it easier to > perform the easy tasks just doesn't make sense.
No but getting faster at an easier task, at least in terms of
intelligence, and for the most part, is indicative of the possibility
that one has opened their capabilities and as a result is much less
prone to struggle on that easier task and much more likely to succeed
at a harder one.
I mean, you simply cannot rule that out.
On May 24, 11:39 pm, Ron Williams <rhwil...@gmail.com> wrote:
> You are assuming that the harder tasks are possible at all for a given
> person. This may not be so. If you are required to keep ten things in mind
> all at once and do a manipulation of them to achieve the result, it may
> never be possible, if you are only capable of holding seven things in mind.
> The harder tasks of this kind will remain just has impossible as before, and
> will result in a big X on the score sheet. On the other hand, the easier
> tasks might well be done much more quickly than before if there's a speed
> improvement. I.e. speeding up attempts on the impossible will not give a
> better result!
> On Mon, May 25, 2009 at 5:52 AM, Mike L. <cool2bwi...@gmail.com> wrote:
> > I wanted to add something:
> > Moody himself states that the processing of the "easier" tasks became
> > quicker in those people who trained with DNB. Now, using logic: When
> > one does something quicker, or finds that they can perform something
> > quicker, that task has then become easier to do for them. For this
> > reason, to say that performing something quicker but not finding it
> > easier to do, is almost illogical, and therefore, to say that harder
> > tasks remain just as hard for the person who has found it easier to
> > perform the easy tasks just doesn't make sense.
Hi, read this article
http://www.psych.rutgers.edu/~jose/courses/578/Conway_etal_2003.pdfes... the "Neuroimaging studies of WMC and g" it shows that there is
high correlation between n-back "lure trials" and ravens (r=0.54), so it's
the lure trials (which takes alot of executive control) that correlates with
intelligence. It also shows that n-back and Ravens engages the same areas of
the brain (DLPFC and ACC) and that the correlation depends 92% on lure
trials and activity in the DLPFC area, which means that DLPFC is the
mediator between G and WMC. Very interesting! Dual-n-backs lure trials
engages DLPFC and perhaps strengthen it with we fail? Just like a muscle
that fails sends signals to build more muscle tissue? Or just the fact the
we activate the area strengthens DLPFC and therefore IQ-score?
On Mon, May 25, 2009 at 7:07 AM, Mike L. <cool2bwi...@gmail.com> wrote:
> No but getting faster at an easier task, at least in terms of
> intelligence, and for the most part, is indicative of the possibility
> that one has opened their capabilities and as a result is much less
> prone to struggle on that easier task and much more likely to succeed
> at a harder one.
> I mean, you simply cannot rule that out.
> On May 24, 11:39 pm, Ron Williams <rhwil...@gmail.com> wrote:
> > You are assuming that the harder tasks are possible at all for a given
> > person. This may not be so. If you are required to keep ten things in
> mind
> > all at once and do a manipulation of them to achieve the result, it may
> > never be possible, if you are only capable of holding seven things in
> mind.
> > The harder tasks of this kind will remain just has impossible as before,
> and
> > will result in a big X on the score sheet. On the other hand, the easier
> > tasks might well be done much more quickly than before if there's a speed
> > improvement. I.e. speeding up attempts on the impossible will not give a
> > better result!
> > On Mon, May 25, 2009 at 5:52 AM, Mike L. <cool2bwi...@gmail.com> wrote:
> > > I wanted to add something:
> > > Moody himself states that the processing of the "easier" tasks became
> > > quicker in those people who trained with DNB. Now, using logic: When
> > > one does something quicker, or finds that they can perform something
> > > quicker, that task has then become easier to do for them. For this
> > > reason, to say that performing something quicker but not finding it
> > > easier to do, is almost illogical, and therefore, to say that harder
> > > tasks remain just as hard for the person who has found it easier to
> > > perform the easy tasks just doesn't make sense.
Unfortunately, I cannot read this article: "404 Page Not Found: The
Internet Just Broke" . Moreover, could you present your interpretation
of "lure trials"? I have a single conception, but... ;)
Hi, I've uploaded it in the files section. A "lure trial" is basiclly
B-R-B-A in a 3-back session. Here the second b "lures the mind" since it's
almost 3-back and therefore takes alot of executive control not to press the
L button. Since ravens correlates with the "lure trials" we might want to
add a feature to the the Brainworkshop program that display the number of
time we didnt fall for a "lure trials"? So after every session it would
display
Visual: 80% Audio: 80% and Lure Trials: 75%?
On Mon, May 25, 2009 at 11:22 AM, Ashirgo <wieloslo...@gmail.com> wrote:
> Unfortunately, I cannot read this article: "404 Page Not Found: The
> Internet Just Broke" . Moreover, could you present your interpretation
> of "lure trials"? I have a single conception, but... ;)
Thank you. One can also consider an option of manipulating maths a
little so as to increase the chance of "lure trials" :) Regrettably, I
cannot see how it could work at higher n-levels. That is to say, I
have never perceived it as a problem/nuisance/etc.
I think that there are som cases where you are really "lured", not that it
might be hard but still relative harder then just a "K-L-R-[K]" type of
sequence. For 4 and 5-back there are some really "luring" stuff.
On Mon, May 25, 2009 at 12:06 PM, Ashirgo <wieloslo...@gmail.com> wrote:
> Thank you. One can also consider an option of manipulating maths a
> little so as to increase the chance of "lure trials" :) Regrettably, I
> cannot see how it could work at higher n-levels. That is to say, I
> have never perceived it as a problem/nuisance/etc.
Your argument makes a hidden assumption that working memory and "speed
of processing" are fully separable components of mental performance.
They apparently are not. Studies have consistently found that working
memory and "processing speed" covary:
From "Variation in Working Memory", page 61:
"In many studies, WMC (Working memory capacity) was found to be
strongly correlated to measures of processing speed [they give several
citations]. ...one obvious explanation is that working memory tasks
are complex span tasks which require a processing component
themselves, and the speed of performing the component is one source of
variance in complex span tasks. However, this cannot be the whole
story, because not all WMC tasks are complex span tasks, and speed
measures are also correlated to working memory tasks lacking a
processing component. A clear example of this is in the study by
Conway et al, (2002), who found a substantial correlation of speed
with a factor that extracted the common variance of short-term and
working memory tasks, but none found a second factor capturing the
specific variance of working memory. This is the opposite of what one
would expect if speed is related to complex span through the
processing component of the latter."
A question that needs answering is whether the "analytical" abilities
that are supposedly required of the more difficult BOMAT questions
covary with WM and processing speed too. Although it is tempting to
think that they do not, I would not jump to this conclusion either.
On May 24, 10:39 pm, Ron Williams <rhwil...@gmail.com> wrote:
> You are assuming that the harder tasks are possible at all for a given
> person. This may not be so. If you are required to keep ten things in mind
> all at once and do a manipulation of them to achieve the result, it may
> never be possible, if you are only capable of holding seven things in mind.
> The harder tasks of this kind will remain just has impossible as before, and
> will result in a big X on the score sheet. On the other hand, the easier
> tasks might well be done much more quickly than before if there's a speed
> improvement. I.e. speeding up attempts on the impossible will not give a
> better result!
> On Mon, May 25, 2009 at 5:52 AM, Mike L. <cool2bwi...@gmail.com> wrote:
> > I wanted to add something:
> > Moody himself states that the processing of the "easier" tasks became
> > quicker in those people who trained with DNB. Now, using logic: When
> > one does something quicker, or finds that they can perform something
> > quicker, that task has then become easier to do for them. For this
> > reason, to say that performing something quicker but not finding it
> > easier to do, is almost illogical, and therefore, to say that harder
> > tasks remain just as hard for the person who has found it easier to
> > perform the easy tasks just doesn't make sense.
> Hi, read this articlehttp://www.psych.rutgers.edu/~jose/courses/578/Conway_etal_2003.pdfes...
> the "Neuroimaging studies of WMC and g" it shows that there is
> high correlation between n-back "lure trials" and ravens (r=0.54), so it's
> the lure trials (which takes alot of executive control) that correlates with
> intelligence. It also shows that n-back and Ravens engages the same areas of
> the brain (DLPFC and ACC) and that the correlation depends 92% on lure
> trials and activity in the DLPFC area, which means that DLPFC is the
> mediator between G and WMC. Very interesting! Dual-n-backs lure trials
> engages DLPFC and perhaps strengthen it with we fail? Just like a muscle
> that fails sends signals to build more muscle tissue? Or just the fact the
> we activate the area strengthens DLPFC and therefore IQ-score?
> On Mon, May 25, 2009 at 7:07 AM, Mike L. <cool2bwi...@gmail.com> wrote:
> > No but getting faster at an easier task, at least in terms of
> > intelligence, and for the most part, is indicative of the possibility
> > that one has opened their capabilities and as a result is much less
> > prone to struggle on that easier task and much more likely to succeed
> > at a harder one.
> > I mean, you simply cannot rule that out.
> > On May 24, 11:39 pm, Ron Williams <rhwil...@gmail.com> wrote:
> > > You are assuming that the harder tasks are possible at all for a given
> > > person. This may not be so. If you are required to keep ten things in
> > mind
> > > all at once and do a manipulation of them to achieve the result, it may
> > > never be possible, if you are only capable of holding seven things in
> > mind.
> > > The harder tasks of this kind will remain just has impossible as before,
> > and
> > > will result in a big X on the score sheet. On the other hand, the easier
> > > tasks might well be done much more quickly than before if there's a speed
> > > improvement. I.e. speeding up attempts on the impossible will not give a
> > > better result!
> > > On Mon, May 25, 2009 at 5:52 AM, Mike L. <cool2bwi...@gmail.com> wrote:
> > > > I wanted to add something:
> > > > Moody himself states that the processing of the "easier" tasks became
> > > > quicker in those people who trained with DNB. Now, using logic: When
> > > > one does something quicker, or finds that they can perform something
> > > > quicker, that task has then become easier to do for them. For this
> > > > reason, to say that performing something quicker but not finding it
> > > > easier to do, is almost illogical, and therefore, to say that harder
> > > > tasks remain just as hard for the person who has found it easier to
> > > > perform the easy tasks just doesn't make sense.
> Hi, I've uploaded it in the files section. A "lure trial" is basiclly
> B-R-B-A in a 3-back session. Here the second b "lures the mind" since it's
> almost 3-back and therefore takes alot of executive control not to press the
> L button. Since ravens correlates with the "lure trials" we might want to
> add a feature to the the Brainworkshop program that display the number of
> time we didnt fall for a "lure trials"? So after every session it would
> display
> Visual: 80% Audio: 80% and Lure Trials: 75%?
> On Mon, May 25, 2009 at 11:22 AM, Ashirgo <wieloslo...@gmail.com> wrote:
> > Unfortunately, I cannot read this article: "404 Page Not Found: The
> > Internet Just Broke" . Moreover, could you present your interpretation
> > of "lure trials"? I have a single conception, but... ;)
Nice article, but for me there was hardly something new (I didn't
catch that change to ONLY bomat in jaeggi). Nevertheless, in my study
(people are flaking the final tests :(, lets hope at least 15 more
will take it) I use only raven, we'll see. And I think that other
theoretical questions appearing in this article (effectivity VS new
quality of intellect) were the subject of discussion here already.
On May 24, 7:42 pm, Ari <arizz...@gmail.com> wrote:
> The May 13, 2008 issue of Proceedings of the National Academy of
> Sciences featured a cover article that purported to demonstrate
> increases in fluid intelligence following training on a task of
> working memory (Jaeggi, Buschkuehl, Jonides, & Perrig, 2008). The
> authors described their own findings as a “landmark result”. Their
> study was the subject of an introductory comment by Robert Sternberg
> (2008), as well as articles in the mainstream media, including a
> lengthy column in a recent edition of The New York Times (Wang &
> Aamodt, 2009).
> In view of the potential significance of the study and the quantity of
> attention it has received, the results have been subjected to
> remarkably little critical analysis. A close examination of the
> evidence reported by Jaeggi et al. shows that it is not in fact
> sufficient to support the authors' conclusion of any increase in their
> subjects' fluid intelligence.
> What Jaeggi et al. reported were modest increases in performance on a
> test of fluid intelligence following several days of training on a
> task of working memory. The reported increases in performance are not
> in question here. But the manner in which the test was administered
> severely undermines the authors' interpretation that their subjects'
> intelligence itself was increased.
> The subjects were divided into four groups, differing in the number of
> days of training they received on the task of working memory. The
> group that received the least training (8 days) was tested on Raven's
> Advanced Progressive Matrices (Raven, 1990), a widely used and well-
> established test of fluid intelligence. This group, however,
> demonstrated negligible improvement between pre- and post-test
> performance.
> The other three groups were not tested using Raven's Matrices, but
> rather on an alternative test of much more recent origin. The Bochumer
> Matrices Test (BOMAT) (Hossiep, Turck, & Hasella, 1999) is similar to
> Raven's in that it consists of visual analogies. In both tests, a
> series of geometric and other figures is presented in a matrix format
> and the subject is required to infer a pattern in order to predict the
> next figure in the series. The authors provide no reason for switching
> from Raven's to the BOMAT.
> The BOMAT differs from Raven's in some important respects, but is
> similar in one crucial attribute: both tests are progressive in
> nature, which means that test items are sequentially arranged in order
> of increasing difficulty. A high score on the test, therefore, is
> predicated on subjects' ability to solve the more difficult items.
> However, this progressive feature of the test was effectively
> eliminated by the manner in which Jaeggi et al. adminstered it. The
> BOMAT is a 29-item test which subjects are supposed to be allowed 45
> min to complete. Remarkably, however, Jaeggi et al. reduced the
> allotted time from 45 min to 10. The effect of this restriction was to
> make it impossible for subjects to proceed to the more difficult items
> on the test. The large majority of the subjects—regardless of the
> number of days of training they received—answered less than 14 test
> items correctly.
> By virtue of the manner in which they administered the BOMAT, Jaeggi
> et al. transformed it from a test of fluid intelligence into a speed
> test of ability to solve the easier visual analogies.
> The time restriction not only made it impossible for subjects to
> proceed to the more difficult items, it also limited the opportunity
> to learn about the test—and so improve performance—in the process of
> taking it. This factor cannot be neglected because test performance
> does improve with practice, as demonstrated by the control groups in
> the Jaeggi study, whose improvement from pre- to post-test was about
> half that of the experimental groups. The same learning process that
> occurs from one administration of the test to the next may also
> operate within a given administration of the test—provided subjects
> are allowed sufficient time to complete it.
> Since the whole weight of their conclusion rests upon the validity of
> their measure of fluid intelligence, one might assume the authors
> would present a careful defense of the manner in which they
> administered the BOMAT. Instead they do not even mention that subjects
> are normally allowed 45 min to complete the test. Nor do they mention
> that the test has 29 items, of which most of their subjects completed
> less than half.
> The authors' entire rationale for reducing the allotted time to 10 min
> is confined to a footnote. That footnote reads as follows:
> Although this procedure differs from the standardized procedure,
> there is evidence that this timed procedure has little influence on
> relative standing in these tests, in that the correlation of speeded
> and non-speeded versions is very high (r = 0.95; ref. 37).
> The reference given in the footnote is to a 1988 study (Frearson &
> Eysenck, 1986) that is not in fact designed to support the conclusion
> stated by Jaeggi et al. The 1988 study merely contains a footnote of
> its own, which refers in turn to unpublished research conducted forty
> years earlier. That research involved Raven's matrices, not the BOMAT,
> and entailed a reduction in time of at most 50%, not more than 75%, as
> in the Jaeggi study.
> So instead of offering a reasoned defense of their procedure, Jaeggi
> et al. provide merely a footnote which refers in turn to a footnote in
> another study. The second footnote describes unpublished results,
> evidently recalled by memory over a span of 40 years, involving a
> different test and a much less severe reduction in time.
> In this context it bears repeating that the group that was tested on
> Raven's matrices (with presumably the same time restriction) showed
> virtually no improvement in test performance, in spite of eight days'
> training on working memory. Performance gains only appeared for the
> groups administered the BOMAT. But the BOMAT differs in one important
> respect from Raven's. Raven's matrices are presented in a 3 × 3
> format, whereas the BOMAT consists of a 5 × 3 matrix configuration.
> With 15 visual figures to keep track of in each test item instead of
> 9, the BOMAT puts added emphasis on subjects' ability to hold details
> of the figures in working memory, especially under the condition of a
> severe time constraint. Therefore it is not surprising that extensive
> training on a task of working memory would facilitate performance on
> the early and easiest BOMAT test items—those that present less of a
> challenge to fluid intelligence.
> This interpretation acquires added plausibility from the nature of one
> of the two working-memory tasks administered to the experimental
> groups. The authors maintain that those tasks were “entirely
> different” from the test of fluid intelligence. One of the tasks
> merits that description: it was a sequence of letters presented
> auditorily through headphones.
> But the other working-memory task involved recall of the location of a
> small square in one of several positions in a visual matrix pattern.
> It represents in simplified form precisely the kind of detail required
> to solve visual analogies. Rather than being “entirely different” from
> the test items on the BOMAT, this task seems well-designed to
> facilitate performance on that test.
> More generally, the foregoing considerations suggest a deeper problem
> with the conclusions presented by Jaeggi et al.: To what extent does
> improvement on any test of fluid intelligence reflect an increase in
> actual intelligence rather than merely an increase in test-taking
> skills? A full analysis of this issue is beyond the scope of the
> present review, but the methodological challenges involved are
> formidable and deserve further discussion.
> Whatever the meaning of the modest gains in performance on the BOMAT,
> the evidence produced by Jaeggi et al. does not support the conclusion
> of an increase in their subjects' intelligence. Their research may be
> sufficient to encourage further investigation, but any larger
> inferences are unwarranted.
hmm, someone else posted on here that you had already finished your
studies. Guess it was a misinformation.
In any case, interesting stuff; keep us posted, it'd be interesting to
see what other results have come from secondary studies [even if they
are on a more minimal scale].
On May 25, 11:54 am, Vlad <pol...@gmail.com> wrote:
> Nice article, but for me there was hardly something new (I didn't
> catch that change to ONLY bomat in jaeggi). Nevertheless, in my study
> (people are flaking the final tests :(, lets hope at least 15 more
> will take it) I use only raven, we'll see. And I think that other
> theoretical questions appearing in this article (effectivity VS new
> quality of intellect) were the subject of discussion here already.
> On May 24, 7:42 pm, Ari <arizz...@gmail.com> wrote:
> > Published in the journal Intelligence by David E. Moody, apparently a
> > math tutor for high school students. What do you make of this?
> > The May 13, 2008 issue of Proceedings of the National Academy of
> > Sciences featured a cover article that purported to demonstrate
> > increases in fluid intelligence following training on a task of
> > working memory (Jaeggi, Buschkuehl, Jonides, & Perrig, 2008). The
> > authors described their own findings as a “landmark result”. Their
> > study was the subject of an introductory comment by Robert Sternberg
> > (2008), as well as articles in the mainstream media, including a
> > lengthy column in a recent edition of The New York Times (Wang &
> > Aamodt, 2009).
> > In view of the potential significance of the study and the quantity of
> > attention it has received, the results have been subjected to
> > remarkably little critical analysis. A close examination of the
> > evidence reported by Jaeggi et al. shows that it is not in fact
> > sufficient to support the authors' conclusion of any increase in their
> > subjects' fluid intelligence.
> > What Jaeggi et al. reported were modest increases in performance on a
> > test of fluid intelligence following several days of training on a
> > task of working memory. The reported increases in performance are not
> > in question here. But the manner in which the test was administered
> > severely undermines the authors' interpretation that their subjects'
> > intelligence itself was increased.
> > The subjects were divided into four groups, differing in the number of
> > days of training they received on the task of working memory. The
> > group that received the least training (8 days) was tested on Raven's
> > Advanced Progressive Matrices (Raven, 1990), a widely used and well-
> > established test of fluid intelligence. This group, however,
> > demonstrated negligible improvement between pre- and post-test
> > performance.
> > The other three groups were not tested using Raven's Matrices, but
> > rather on an alternative test of much more recent origin. The Bochumer
> > Matrices Test (BOMAT) (Hossiep, Turck, & Hasella, 1999) is similar to
> > Raven's in that it consists of visual analogies. In both tests, a
> > series of geometric and other figures is presented in a matrix format
> > and the subject is required to infer a pattern in order to predict the
> > next figure in the series. The authors provide no reason for switching
> > from Raven's to the BOMAT.
> > The BOMAT differs from Raven's in some important respects, but is
> > similar in one crucial attribute: both tests are progressive in
> > nature, which means that test items are sequentially arranged in order
> > of increasing difficulty. A high score on the test, therefore, is
> > predicated on subjects' ability to solve the more difficult items.
> > However, this progressive feature of the test was effectively
> > eliminated by the manner in which Jaeggi et al. adminstered it. The
> > BOMAT is a 29-item test which subjects are supposed to be allowed 45
> > min to complete. Remarkably, however, Jaeggi et al. reduced the
> > allotted time from 45 min to 10. The effect of this restriction was to
> > make it impossible for subjects to proceed to the more difficult items
> > on the test. The large majority of the subjects—regardless of the
> > number of days of training they received—answered less than 14 test
> > items correctly.
> > By virtue of the manner in which they administered the BOMAT, Jaeggi
> > et al. transformed it from a test of fluid intelligence into a speed
> > test of ability to solve the easier visual analogies.
> > The time restriction not only made it impossible for subjects to
> > proceed to the more difficult items, it also limited the opportunity
> > to learn about the test—and so improve performance—in the process of
> > taking it. This factor cannot be neglected because test performance
> > does improve with practice, as demonstrated by the control groups in
> > the Jaeggi study, whose improvement from pre- to post-test was about
> > half that of the experimental groups. The same learning process that
> > occurs from one administration of the test to the next may also
> > operate within a given administration of the test—provided subjects
> > are allowed sufficient time to complete it.
> > Since the whole weight of their conclusion rests upon the validity of
> > their measure of fluid intelligence, one might assume the authors
> > would present a careful defense of the manner in which they
> > administered the BOMAT. Instead they do not even mention that subjects
> > are normally allowed 45 min to complete the test. Nor do they mention
> > that the test has 29 items, of which most of their subjects completed
> > less than half.
> > The authors' entire rationale for reducing the allotted time to 10 min
> > is confined to a footnote. That footnote reads as follows:
> > Although this procedure differs from the standardized procedure,
> > there is evidence that this timed procedure has little influence on
> > relative standing in these tests, in that the correlation of speeded
> > and non-speeded versions is very high (r = 0.95; ref. 37).
> > The reference given in the footnote is to a 1988 study (Frearson &
> > Eysenck, 1986) that is not in fact designed to support the conclusion
> > stated by Jaeggi et al. The 1988 study merely contains a footnote of
> > its own, which refers in turn to unpublished research conducted forty
> > years earlier. That research involved Raven's matrices, not the BOMAT,
> > and entailed a reduction in time of at most 50%, not more than 75%, as
> > in the Jaeggi study.
> > So instead of offering a reasoned defense of their procedure, Jaeggi
> > et al. provide merely a footnote which refers in turn to a footnote in
> > another study. The second footnote describes unpublished results,
> > evidently recalled by memory over a span of 40 years, involving a
> > different test and a much less severe reduction in time.
> > In this context it bears repeating that the group that was tested on
> > Raven's matrices (with presumably the same time restriction) showed
> > virtually no improvement in test performance, in spite of eight days'
> > training on working memory. Performance gains only appeared for the
> > groups administered the BOMAT. But the BOMAT differs in one important
> > respect from Raven's. Raven's matrices are presented in a 3 × 3
> > format, whereas the BOMAT consists of a 5 × 3 matrix configuration.
> > With 15 visual figures to keep track of in each test item instead of
> > 9, the BOMAT puts added emphasis on subjects' ability to hold details
> > of the figures in working memory, especially under the condition of a
> > severe time constraint. Therefore it is not surprising that extensive
> > training on a task of working memory would facilitate performance on
> > the early and easiest BOMAT test items—those that present less of a
> > challenge to fluid intelligence.
> > This interpretation acquires added plausibility from the nature of one
> > of the two working-memory tasks administered to the experimental
> > groups. The authors maintain that those tasks were “entirely
> > different” from the test of fluid intelligence. One of the tasks
> > merits that description: it was a sequence of letters presented
> > auditorily through headphones.
> > But the other working-memory task involved recall of the location of a
> > small square in one of several positions in a visual matrix pattern.
> > It represents in simplified form precisely the kind of detail required
> > to solve visual analogies. Rather than being “entirely different” from
> > the test items on the BOMAT, this task seems well-designed to
> > facilitate performance on that test.
> > More generally, the foregoing considerations suggest a deeper problem
> > with the conclusions presented by Jaeggi et al.: To what extent does
> > improvement on any test of fluid intelligence reflect an increase in
> > actual intelligence rather than merely an increase in test-taking
> > skills? A full analysis of this issue is beyond the scope of the
> > present review, but the methodological challenges involved are
> > formidable and deserve further discussion.
> > Whatever the meaning of the modest gains in performance on the BOMAT,
> > the evidence produced by Jaeggi et al. does not support the conclusion
> > of an increase in their subjects' intelligence. Their research may be
> > sufficient to encourage further investigation, but any larger
> > inferences are unwarranted.
"For this reason, to say that performing something quicker but not
finding it easier to do, is almost illogical"
Why? I'll give you an example: caffeine vs. amphetamines. The first
improves speed in many people; you'll work faster but not necessary
better (and possibly worse). There will be no increase in the
complexity of your thoughts.
Amphetamine, by contrast, improves not only speed but also maximum
thought complexity. (At least this has been my experience!)
It's like the distinction between processor speed (GHz) and processor
architecture. The former is a measure of cycles per second, the latter
is design. A well-designed, more efficient processor can outsmart a
poorly-designed one that works faster.
On May 24, 2:52 pm, "Mike L." <cool2bwi...@gmail.com> wrote:
> Moody himself states that the processing of the "easier" tasks became
> quicker in those people who trained with DNB. Now, using logic: When
> one does something quicker, or finds that they can perform something
> quicker, that task has then become easier to do for them. For this
> reason, to say that performing something quicker but not finding it
> easier to do, is almost illogical, and therefore, to say that harder
> tasks remain just as hard for the person who has found it easier to
> perform the easy tasks just doesn't make sense.
> Taking this into consideration, it is more sensible to believe that
> the easy tasks have become easier but the harder tasks have TOO become
> easier.
> On May 24, 2:54 pm, "Mike L." <cool2bwi...@gmail.com> wrote:
> > But wait, isn't the speed or even the capacity with which one is
> > endowed to accomplish whatever task they might have at hand, also
> > strongly correlated with intelligence?
> > That would be like saying, a faster and more fuel efficient car does
> > not have a better engine than that of a slower, less fuel efficient
> > car.
> > To use a more concrete example, on the WAIS-IV IQ test, there is
> > indeed a section on processing information and spatial reasoning, as
> > well as digit-span memorization. These categories, it can be said,
> > will indeed rise as due to DNB training, thus, increase overall IQ.
> > Therefore, though, according to the hypothesis moody devises of DNB
> > not improving analytical processes, training with DNB will still
> > result in a higher IQ.
> > On May 24, 2:33 pm, Ari <arizz...@gmail.com> wrote:
> > > Yes. While Raven's seems to be more of a test of ability to deal with
> > > complexity (intelligence?), the earlier levels of BOMAT test mostly
> > > working memory. Working memory is indisputably related to
> > > intelligence, but it may be the case that DNB training improves only
> > > speed or capacity, not intelligence however constituted. As you point
> > > out, though, this is a useful skill regardless.
> > > On May 24, 12:58 pm, Iron <IronPanc...@gmail.com> wrote:
> > > > An interesting point I would like to bring into this thread is the
> > > > mention earlier of people with autism performing higher than average
> > > > on the raven's with a lower average working memory. That suggests to
> > > > me that the raven's as a test may not require a large working memory
> > > > capacity. It looks to me like there may be some other bottleneck for
> > > > this test.
> > > > It is rather curious that they administered the tests in that fashion
> > > > and that they changed the tests mid-experiment. I hope that in the
> > > > future a more extensive test is administered after an even longer
> > > > period of DNB training. If all this is good for is increasing our
> > > > speed at solving problems we could already solve, I still feel that it
> > > > is an incredibly valuable exercise.
> > > > There are other studies indicating significant improvements in ADHD
> > > > symptoms and unrelated skills, so I still maintain the belief that DNB
> > > > is good for us and we are improving something vitally important by
> > > > training.
> The May 13, 2008 issue of Proceedings of the National Academy of
> Sciences featured a cover article that purported to demonstrate
> increases in fluid intelligence following training on a task of
> working memory (Jaeggi, Buschkuehl, Jonides, & Perrig, 2008). The
> authors described their own findings as a “landmark result”. Their
> study was the subject of an introductory comment by Robert Sternberg
> (2008), as well as articles in the mainstream media, including a
> lengthy column in a recent edition of The New York Times (Wang &
> Aamodt, 2009).
> In view of the potential significance of the study and the quantity of
> attention it has received, the results have been subjected to
> remarkably little critical analysis. A close examination of the
> evidence reported by Jaeggi et al. shows that it is not in fact
> sufficient to support the authors' conclusion of any increase in their
> subjects' fluid intelligence.
> What Jaeggi et al. reported were modest increases in performance on a
> test of fluid intelligence following several days of training on a
> task of working memory. The reported increases in performance are not
> in question here. But the manner in which the test was administered
> severely undermines the authors' interpretation that their subjects'
> intelligence itself was increased.
> The subjects were divided into four groups, differing in the number of
> days of training they received on the task of working memory. The
> group that received the least training (8 days) was tested on Raven's
> Advanced Progressive Matrices (Raven, 1990), a widely used and well-
> established test of fluid intelligence. This group, however,
> demonstrated negligible improvement between pre- and post-test
> performance.
> The other three groups were not tested using Raven's Matrices, but
> rather on an alternative test of much more recent origin. The Bochumer
> Matrices Test (BOMAT) (Hossiep, Turck, & Hasella, 1999) is similar to
> Raven's in that it consists of visual analogies. In both tests, a
> series of geometric and other figures is presented in a matrix format
> and the subject is required to infer a pattern in order to predict the
> next figure in the series. The authors provide no reason for switching
> from Raven's to the BOMAT.
> The BOMAT differs from Raven's in some important respects, but is
> similar in one crucial attribute: both tests are progressive in
> nature, which means that test items are sequentially arranged in order
> of increasing difficulty. A high score on the test, therefore, is
> predicated on subjects' ability to solve the more difficult items.
> However, this progressive feature of the test was effectively
> eliminated by the manner in which Jaeggi et al. adminstered it. The
> BOMAT is a 29-item test which subjects are supposed to be allowed 45
> min to complete. Remarkably, however, Jaeggi et al. reduced the
> allotted time from 45 min to 10. The effect of this restriction was to
> make it impossible for subjects to proceed to the more difficult items
> on the test. The large majority of the subjects—regardless of the
> number of days of training they received—answered less than 14 test
> items correctly.
> By virtue of the manner in which they administered the BOMAT, Jaeggi
> et al. transformed it from a test of fluid intelligence into a speed
> test of ability to solve the easier visual analogies.
> The time restriction not only made it impossible for subjects to
> proceed to the more difficult items, it also limited the opportunity
> to learn about the test—and so improve performance—in the process of
> taking it. This factor cannot be neglected because test performance
> does improve with practice, as demonstrated by the control groups in
> the Jaeggi study, whose improvement from pre- to post-test was about
> half that of the experimental groups. The same learning process that
> occurs from one administration of the test to the next may also
> operate within a given administration of the test—provided subjects
> are allowed sufficient time to complete it.
> Since the whole weight of their conclusion rests upon the validity of
> their measure of fluid intelligence, one might assume the authors
> would present a careful defense of the manner in which they
> administered the BOMAT. Instead they do not even mention that subjects
> are normally allowed 45 min to complete the test. Nor do they mention
> that the test has 29 items, of which most of their subjects completed
> less than half.
> The authors' entire rationale for reducing the allotted time to 10 min
> is confined to a footnote. That footnote reads as follows:
> Although this procedure differs from the standardized procedure,
> there is evidence that this timed procedure has little influence on
> relative standing in these tests, in that the correlation of speeded
> and non-speeded versions is very high (r = 0.95; ref. 37).
> The reference given in the footnote is to a 1988 study (Frearson &
> Eysenck, 1986) that is not in fact designed to support the conclusion
> stated by Jaeggi et al. The 1988 study merely contains a footnote of
> its own, which refers in turn to unpublished research conducted forty
> years earlier. That research involved Raven's matrices, not the BOMAT,
> and entailed a reduction in time of at most 50%, not more than 75%, as
> in the Jaeggi study.
> So instead of offering a reasoned defense of their procedure, Jaeggi
> et al. provide merely a footnote which refers in turn to a footnote in
> another study. The second footnote describes unpublished results,
> evidently recalled by memory over a span of 40 years, involving a
> different test and a much less severe reduction in time.
> In this context it bears repeating that the group that was tested on
> Raven's matrices (with presumably the same time restriction) showed
> virtually no improvement in test performance, in spite of eight days'
> training on working memory. Performance gains only appeared for the
> groups administered the BOMAT. But the BOMAT differs in one important
> respect from Raven's. Raven's matrices are presented in a 3 × 3
> format, whereas the BOMAT consists of a 5 × 3 matrix configuration.
> With 15 visual figures to keep track of in each test item instead of
> 9, the BOMAT puts added emphasis on subjects' ability to hold details
> of the figures in working memory, especially under the condition of a
> severe time constraint. Therefore it is not surprising that extensive
> training on a task of working memory would facilitate performance on
> the early and easiest BOMAT test items—those that present less of a
> challenge to fluid intelligence.
> This interpretation acquires added plausibility from the nature of one
> of the two working-memory tasks administered to the experimental
> groups. The authors maintain that those tasks were “entirely
> different” from the test of fluid intelligence. One of the tasks
> merits that description: it was a sequence of letters presented
> auditorily through headphones.
> But the other working-memory task involved recall of the location of a
> small square in one of several positions in a visual matrix pattern.
> It represents in simplified form precisely the kind of detail required
> to solve visual analogies. Rather than being “entirely different” from
> the test items on the BOMAT, this task seems well-designed to
> facilitate performance on that test.
> More generally, the foregoing considerations suggest a deeper problem
> with the conclusions presented by Jaeggi et al.: To what extent does
> improvement on any test of fluid intelligence reflect an increase in
> actual intelligence rather than merely an increase in test-taking
> skills? A full analysis of this issue is beyond the scope of the
> present review, but the methodological challenges involved are
> formidable and deserve further discussion.
> Whatever the meaning of the modest gains in performance on the BOMAT,
> the evidence produced by Jaeggi et al. does not support the conclusion
> of an increase in their subjects' intelligence. Their research may be
> sufficient to encourage further investigation, but any larger
> inferences are unwarranted.
My point is that this is _not_ a continuum, and so one can achievce a speed increase that will make the easier tasks easier, but leave the difficult tasks still impossible.
On 5/26/09, Wade <Sean.Wade.Tho...@gmail.com> wrote:
> Your argument makes a hidden assumption that working memory and "speed > of processing" are fully separable components of mental performance. > They apparently are not. Studies have consistently found that working > memory and "processing speed" covary:
