If we're using computer metaphors, I think the right one for this
discussion is Amdah's Law
https://secure.wikimedia.org/wikipedia/en/wiki/Amdahl's_law
The relevant bit being:
"or example, if a program needs 20 hours using a single processor
core, and a particular portion of 1 hour cannot be parallelized, while
the remaining promising portion of 19 hours (95%) can be parallelized,
then regardless of how many processors we devote to a parallelized
execution of this program, the minimal execution time cannot be less
than that critical 1 hour."
The application here is, I think, fairly clear. Moody's 2 essential points are:
1) the 8-dayers showed no improvement on Raven's, which tests WM & Gf
2) there was BOMAT improvement, but the data is only on the easiest questions
2.1) these easy questions only meaningfully test WM, and not Gf
∴ the simplest explanation is that N-back improves WM but not Gf
So even if one 'parallelizes' the WM bits, the analysis still
bottlenecks the solution.
I don't know if Moody is right, since the picture is incomplete: we
don't know what the full BOMAT would've shown, and the 8-dayers aren't
a definitive datapoint against N-back (in the previous Jaeggi studies,
the clearest results were for people who were doing N-back noticeably
longer than 8-days, and it seems plausible to me that it could take 2
weeks or so for neurological changes - muscles adapt on that
timescale, after all).
(But I'm a little embarrassed personally. I had noticed and read
quizzically that footnote, but I did not look at the old study; and I
misunderstood the time limit comment as indicating that subjects had
10 minutes for each question - which I considered eminently reasonable
- and not 10 minutes for the *entire* test! ~_~)
(Or if you don't like Amdahl's Law, then one could point that that if
your program is IO-bound, using less CPU time won't make it go
faster.)
--
gwern
"A subsequent analysis of the gain scores (posttest minus pretest) as a function
of training time (F(3,30) ϭ 9.25;
P Ͻ 0.001; 2 ϭ 0.48; Fig. 3b). Analyses of covariance (AN-
COVA) with the factor group (trained vs. control), the posttest
scores as the dependent variable, and the pretest scores as the
covariate revealed a trend for group differences after 12 days
(F(1,19) ϭ 1.93; P ϭ 0.09; 2 ϭ 0.09), and statistically significant
group differences after 17 (F(1,13) ϭ 4.65; P Ͻ 0.05; 2 ϭ 0.26),
and 19 training days (F(1,12) ϭ 4.53; P Ͻ 0.05; 2 ϭ 0.27). Post
hoc analyses (Gabriel’s procedure; two-tailed) for the training
group revealed significant differences between the following
groups: 8 vs. 17 days (P Ͻ 0.01); 8 vs. 19 days (P Ͻ 0.001); and
12 vs. 19 days (P Ͻ 0.01). There was a trend for a difference
between 12 and 17 days (P ϭ 0.06). "
Yes, the 8-day group fit the dosage-dependent graph, inasmuch as if
you set them to 0, then their near-0 improvement fits perfectly.
I still think there are some key issues that you miss
> here, WM and Gf might share common capacity constraints and activates same
> areas of the brain while some of you see WM and Gf as totally independent of
> each other which they only are from "services point of view" but not from a
> neurological point of view. The research has never stated that working
> memory and Gf is the same, it's stated that by training the working memory
> in a certain way (updating often, pararell executive functions etc) would
> strengthen shared neural pathways leading to an improvement.
Yes, I suppose that's possible. We do have all those other results
linking the two. But as I said before, my takeaway from Moody is that
Jaeggi 2008 doesn't prove it by boosting WM and then Gf.
> Assuming that
> the first 14 questions of 29 (?) only should tax working memory seems highly
> unlikely.
It's true that assuming any random # of questions will only tax
working memory, but we might expect some number of early questions to
test working memory more. The diagrams can only have so many objects
in them, while the relationships can be made ever more
subtle/arbitrary.
In this way it's perfectly possible that the early questions owe more
of their difficulty to WM than to pattern. But per above, with each
question the pattern becomes more difficult, while the WM difficulty,
even if it increases, is quickly bounded.
Or to put it another way, any individual should be surprised to win
the lottery; but we shouldn't be surprised that an individual won the
lottery. It's a little surprising 14 won the lottery, but not
surprising that some number did.
--
gwern
A worse measure, certainly. They may be fine for 'stupid' people,
though. One doesn't use the same questions or tests for geniuses as
for morons. But anyway, given the strong correlation between Gf and
WM, they may be fine questions for the purpose, until one is
interested in WM and Gf as distinct from each other.
> However let's
> say you can solve 40% more questions in a 10 minute interval then you would
> have more time completing the more difficult items which for many might make
> a diffrence.
All that much of one?
Suppose I'm solving 9 out of the 29 questions before training. (The
'vast majority' are solving <14 questions, so this doesn't seem too
bad.)
ow N-back improves me by an amazing 40% - I can do those 9 problems in
just 6 minutes instead of 10. In other words, the second time around,
I now have a luxurious 4 minutes to do 20 harder questions.
Supposing that I solve these harder questions at the same rate I did
the easier ones before training, that means I'm answering less than a
question per minute. So I'll answer another 3 questions, bumping me
all the way up to 12 questions out of 29.
On a 10 minute test, you have so little time that even dramatic
improvements don't buy you much. Amdahl's law again.
--
gwern