The relationship between n-back performance and matrix reasoning — implications for training and transfer
zzzz
http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6W4M-514Y84N-1&_user=10&_coverDate=10%2F02%2F2010&_rdoc=1&_fmt=high&_orig=search&_origin=search&_sort=d&_docanchor=&view=c&_acct=C000050221&_version=1&_urlVersion=0&_userid=10&md5=0ac23a5f4959f76d88cbf4f97228d31e&searchtype=a
Pontus Granström
On Thu, Oct 7, 2010 at 3:07 PM, zzzz <filip...@gmail.com> wrote:
> Don't know whether it's been brought up before:
>
> http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6W4M-514Y84N-1&_user=10&_coverDate=10%2F02%2F2010&_rdoc=1&_fmt=high&_orig=search&_origin=search&_sort=d&_docanchor=&view=c&_acct=C000050221&_version=1&_urlVersion=0&_userid=10&md5=0ac23a5f4959f76d88cbf4f97228d31e&searchtype=a
>
> You received this message because you are subscribed to the Google Groups "Dual N-Back, Brain Training & Intelligence" group.
> To post to this group, send email to brain-t...@googlegroups.com.
> To unsubscribe from this group, send email to brain-trainin...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/brain-training?hl=en.
>
>
Pontus Granström
I will be able to do so, might settle this debate once and for all!
Oye
This perhaps the second study which support the correlation between n-
backtraining and Gf-gains.
Im becoming just more and more convinced that this really works.
On Oct 7, 3:07 pm, zzzz <filippaw...@gmail.com> wrote:
> Don't know whether it's been brought up before:
>
Reece
@Pontus: likeprestige was kind enough to upload a copy in the DNB file
section for everyone.
I found this particularly enlightening:
"As predicted, we found no transfer effects to a measure of working
memory capacity for the single n-back task group. We also found no
transfer for the dual n-back task group even though Study 1 showed
that dual n-back performance was partially predicted by working memory
capacity. We note, however, that the correlation of OSPAN with dual-
task performance was considerably smaller than the correlation of this
task with both matrices tasks.
This finding might be surprising given that both the OSPAN and the n-
back tasks are considered WM tasks. However, previous research has
shown that these tasks do not share considerable common variance,
although they both seem to predict variance in Gf tasks (e.g. [Jaeggi
et al., 2010] and [Kane et al., 2007]). The lack of correlation
between the two WM tasks most likely results from the fact that there
are different processes involved in the two tasks: whereas the main
processes that drive performance in the n-back tasks are familiarity-
and recognition-based discrimination processes ([Oberauer, 2005] and
[Smith and Jonides, 1998]), complex WM span tasks, such as the OSPAN,
require active recall processes rather than recognition. This pattern
is also consistent with our prior findings of no effect on complex
span (Jaeggi et al., 2008). Indeed, Li et al. (2008) have even
reported a significant performance decrease after single n-back
training in a related complex span measure (rotation span). Thus,
although there was no significant performance difference between pre-
and post-test in either of the groups in the present study, one could
speculate that n-back training somehow interferes with performance in
complex span measures as participants might rely more on recognition
instead of recall processes at post-test which might prevent any
performance gain. Since both WM tasks seem to be related via their
relationship to Gf measures, though, one might argue that one could
also train on complex span measures in order to get transfer to Gf.
However, Chein and Morrison (2010) trained their subjects on complex-
span measures, but they did not find any transfer to matrix
reasoning."
I'm quite impressed with the Raven's improvement from single n-back
training. Also, that recognition-based discrimination processes were
the main thing improved might explain why I've been finding multiple
choice tests a lot easier this year :-)
Reece
favoured by many members -- they certainly require a more active
recall process, rather than merely recognition.
Pontus Granström
Thanks likeprestige.
Pontus Granström
WM-tasks are equal, al though I was called dogmatic for saying so.
On Thu, Oct 7, 2010 at 5:25 PM, Reece <rock...@hotmail.com> wrote:
A.Four.Sigma
>
> - Show quoted text -
A.Four.Sigma
>
> > - Show quoted text -- Hide quoted text -
A.Four.Sigma
subjects react to pentuple n backing instead of just single and dual;
maybe jaeggi et al havent heard of brain workshop!
>
> > - Show quoted text -- Hide quoted text -
Pontus Granström
holds for 1 and 2 back, by induction it will hold for 3-4-5 as well.
David
cognitive process; as Reece mentioned they likely require more active
recalling than mere recognition. It would be interesting if they were
to investigate the effects of training quad/pentuple on WM capacity
since these tasks require employment of active recall.
On Oct 7, 9:54 pm, Pontus Granström <lepon...@gmail.com> wrote:
> The point is that task switching does not play a great role, if it
> holds for 1 and 2 back, by induction it will hold for 3-4-5 as well.
>
A.Four.Sigma
crab modes? And do these special modes tax the brain similarly to
OSPAN? From the tone of the article, it seems that OSPAN may be
better.
> >> > - Show quoted text -- Hide quoted text -
>
> >> - Show quoted text -
>
> > --
> > You received this message because you are subscribed to the Google Groups "Dual N-Back, Brain Training & Intelligence" group.
> > To post to this group, send email to brain-t...@googlegroups.com.
> > To unsubscribe from this group, send email to brain-trainin...@googlegroups.com.
A.Four.Sigma
did not prepare me whatever for doing combination variable n-back. We
are left to wonder if more complex span tasks like crab, variable, and
combination are better for improving WM (as measured by other complex
span tasks), and also better at improving Gf.
On Oct 7, 7:38 pm, milestones <wgweathe...@gmail.com> wrote:
> > This might also explain why Variable, Crab, and Combo modes are
> > favoured by many members -- they certainly require a more active
> > recall process, rather than merely recognition.
>
>
> We do have to be careful with "merely recognition" since evidence
> points to recognition as being (at at least one of) the transfer
> mechanisms that increases Gf -- whereas complex span measures do not
> have this transfer effect demonstrated to the same degree. It seems
> to me that the protocol now for people new to BW would be to train
> Dual if their goal is to increase Gf. Also, for future versions, it
> might be a good idea to implement single back version for older and
> very young people who are just getting started given the mounting
> evidence to show transfer to Gf from single N back. While dual is old
> hat to most of us, for the person new to it it is very daunting.
> I have tried to get relatives (now in their golden years) started on
> it and have gotten "thanks but no thanks." If
> dual N back is not altogether as superior as a training tool of Gf
> than single back is (and is clearly less scary to some of the
> population), it might be a good idea to consider it. Right now, the
> only place to train single N back is at cognitivefun and
> mybraintrainer (the former being better imo - in addition to being
> free).
>
> The lack of transfer from both single and dual n back to the complex
> WM span task indicates that it is fair to say
> that N back is not a WM training task but rather a Gf training task.
> For advanced users (many of whom gravitate to a place like this), who
> have maxed out their N back<>fluid G mechanisms by dint of training,
> what's left to do is to incorporate other executive tasks into the N
> back training and that's what we've got with BW 4.8. I think for most
> people the transfer to Gf is not likely going to increase (much) with
> crab, combination, multi stim, etc but it seems pure WM on its own
> will likely improve by training active recall processes (in our
> terminology breadth and
> manipulation). The upshot of balancing "depth" (maintaining a fairly
> high N level) with other modes that tax purer
> WM processes is not clear (and is entirely speculative) but it
> obviates fear of GF transfer atrophy irregardless of whether or not
> that fear is misplaced. Most likely, though, such a fear is misplaced,
> and over-training on N back with the hope of gaining more and more Gf,
> is probably a futile effort after a certain point. However, adding on
> of tasks that up the ante in regard to updating/active recall of
> executive functions will likely offer real benefits for those who want
> to max out both Gf and the apparently unrelated WM mechanisms at the
> same time. There may also come a time when one wants to focus almost
> exclusively on intensive WM aspects without stressing N too much (as
> likeprestige and A.Four.Sigma have described doing). What the benefits
> are of concomitant training of N back with other modalities is
> fascinating to ponder but unknown at this point.
>
> The path is not uniform for everyone but this recent study is another
> helpful navigation tool. Thanks to zzz and likeprestige for raising
> the issue and posting the actual study.
Gwern Branwen
> Don't know whether it's been brought up before:
>
> http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6W4M-514Y84N-1&_user=10&_coverDate=10%2F02%2F2010&_rdoc=1&_fmt=high&_orig=search&_origin=search&_sort=d&_docanchor=&view=c&_acct=C000050221&_version=1&_urlVersion=0&_userid=10&md5=0ac23a5f4959f76d88cbf4f97228d31e&searchtype=a
The first study covered in the paper seems to've been the same study
covered by the _Psychonomics_ poster & cited in Jaeggi 2009
(http://www.gwern.net/N-back%20FAQ.html#jaeggi-2009)
This one included many details missing from the poster, including the
crucial details about how the matrix tests were administered.
They were administered speeded *again* (10min). But at least this time
there's a better attempt at justification for speeding the APM, though
the BOMAT comes with the same lame time excuse.
The second study is better in this respect - 16 minutes for BOMAT
compared to 10.
But I wonder if I'm interpreting the charts on page 7 correctly? It
seems that in study 2, each section of the APM had 18 questions and no
one answered more than 14 or 15 questions at the start, but after
n-back training both n-back groups had at least one person answer all
18 questions. Given that the score increase for the APM was around 1
or 2 questions... This seems to be also true of the BOMAT, but it had
more questions so no one ever answered them all. Does the data
distinguish between the subjects becoming more accurate in the
questions they answer, or just whether they answered more questions
with a similar level of accuracy?
So an interesting paper as usual, but I am very disappointed that the
speed question seems to remain unanswered.
--
gwern
http://www.gwern.net/
milestones
> favoured by many members -- they certainly require a more active
> recall process, rather than merely recognition.
We do have to be careful with "merely recognition" since evidence
points to recognition as being (at at least one of) the transfer
mechanisms that increases Gf -- whereas complex span measures do not
have this transfer effect demonstrated to the same degree. It seems
to me that the protocol now for people new to BW would be to train
train single N back
are at cognitivefun and mybraintrainer (the former being better imo -
free).
The lack of transfer from both single and dual n back to the complex
Gf training task specifically.
For advanced users (many of whom gravitate to a place like this), who
have maxed out their N back<>fluid G mechanisms by dint of training,
what's left to do is to incorporate other executive tasks into the N
back training and that's what we've got with BW 4.8. I think for most
people the transfer to Gf is not likely going to increase (much) with
crab, combination, multi stim, etc but it seems pure WM on its own
breadth and manipulation).
It is my hope that working out both N back/gf and WM updating
mechanisms at the same time might have a synergistic effect -- or, at
the worst, continually improve WM and updating when Gf training has
reached a maximum threshold -- N back serving at this point then as
primarily maintenance.
The path is not uniform for everyone but this recent study is another
helpful navigation tool.
cognitive training for both Gf and WM.
That said, I think the core benefit is training single and dual n back
and gaining in Gf. There is no indication that BW is better at
training executive functions than other places on the internet like
Lumosity and Happy Neuron.
Yet I think the complexity of N back with other modalities that BW
offers will likely be much, much more effective for those who have
reached high (even stratospheric) N levels and Gf....the ceiling for
improvement with BW is truly exospheric. You can't say this about most
brain training games on the net or even other versions of DNB.
Arkanj3l
Gf trainers would also go for tri, quad, and pent? If so what effects
do you think they'd have vs single and dual?
The reason why I ask is due to concern in my lack of progress on D4B.
I've actually been getting inaccurate but equivocal averages of
stimuli recall in both D4B and D6B, which I find rather odd.
Pontus Granström
Reece
Ravens and the BOMAT taking too long to administer (I don't understand
her latter point -- after you've convinced people to do a month of n-
backing, what's the hurry?)
It's too bad -- I have a feeling n-back might show even greater
benefits on a lengthy test due to the attentional improvements many
members here have anecdotally reported.
On Oct 8, 1:41 am, Pontus Granström <lepon...@gmail.com> wrote:
> In what sense is the speed question unanswered?
>
>
>
> On Fri, Oct 8, 2010 at 4:46 AM, Gwern Branwen <gwe...@gmail.com> wrote:
> >> Don't know whether it's been brought up before:
>
polar
interesting questions, will discuss it with my friends.
On 7. Okt, 17:11 h., likeprestige <plastic...@live.com.au> wrote:
> Just uploaded it!
>
> Check it out fellas!!
>
> likeprestige
> On Oct 8, 1:11 am, Pontus Granström <lepon...@gmail.com> wrote:
>
>
>
> > Since it's in press it will take a while before I can download it, but
> > I will be able to do so, might settle this debate once and for all!
>
> > > Anyone who got the whole story?
> > >> Don't know whether it's been brought up before:
>
Gwern Branwen
> In what sense is the speed question unanswered?
I don't know how you can ask this given how many times we've argued
over this and my careful explanation of my interpretation of the data.
Study 1 has the IQ tests speeded and provides little details about
before-after scores, so I don't criticize that.
Study 2 has the IQ tests still speeded, but provides some more
details. Before training, the fastest person could not finish the APM
questions, and so the # of correct answers is necessarily low. After
training, the fastest person finished all the APM questions, and the #
of correct answers went up. Similarly for the BOMAT.
Did that person(s) get smarter - be more likely to answer correctly on
the very hardest questions - or did they just get faster, and would
have had nigh-identical scores before-after if they had had time to
answer/guess at all questions provided? As presented, we don't know.
I don't think speed on an IQ test is the same thing as intelligence; I
think speed and accuracy are two different things (although closely
related), and it is possible to intervene to improve speed but not
accuracy. Yes, they mentioned a study that speeded APM was comparable
to unspeeded APM - but that was not in a context of a task which may
be improving speed! To reuse my analogy, it's like saying that the
verbal subtest of Wechsler is well-correlated with the rest of it
(true), therefore we can save time by only administering that subtest
and then we prove that reading the dictionary vastly increases IQ
(false).
The authors did not cite Moody, though they apparently are aware of
the issue; but the best approach would have been to simply not speeded
the tests! This is getting very frustrating.
--
gwern
http://www.gwern.net/
Pontus Granström
review? Have you ever considered that? Is this part of the conspiracy
perhaps?
Gwern Branwen
> If it were like you are claiming, why do you think it passes the peer
> review? Have you ever considered that? Is this part of the conspiracy
> perhaps?
I've never suggested a conspiracy; I don't know why you bring the idea
up. I don't speculate about the inner life of Jaeggi and her coworkers
- I just note that the data do not say to me what they say to her.
Even if I thought this persistent use of speeding were non-kosher, I
still don't need to postulate a conspiracy or some bizarre misfiring
of the peer review process. Peer review is not perfect; frequent
failure is baked into it just by significance alone (5% = at least 5
out of 100 studies are completely wrong, no?), peer review is weak
against collusion/dishonesty (http://arxiv.org/abs/1008.4324v1), and
in sum, *most* findings may be wrong as
http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0020124
provocatively concludes. I take to heart the reminder of my professor
that when 2 psych studies agree, one is wrong.
--
gwern
http://www.gwern.net/
Pontus Granström
Gwern Branwen
> In your view how much of the variation in timed rapm is explained by speed?
I assume you mean 'how much of the additional ~2 correct answers is
explained by speed improvements'. Anywhere from 0 to ~2...
--
gwern
http://www.gwern.net/
Pontus Granström
that is explained by speed. I guess this has to be the case if I can
improve my score by training on speed?!
Gwern Branwen
> No I mean how much of the variation in a timed performance of RAPM
> that is explained by speed. I guess this has to be the case if I can
> improve my score by training on speed?!
So you mean variation from person to person? I have no idea how much
is due to speed, and I don't think it matters very much. The question
is how much of the post-n-backing increase is due to speed increase.
--
gwern
http://www.gwern.net/
Pontus Granström
a much higher correlation. Meaning that if I improve my speed with a
factor 3 I would be able to increase my score by 30% that was the
average improvement. Given the large amount of people taking the test
wouldn't there be some people who has optimal speed before training
and hence don't show any improvement? Why do we not see these cases?
Reece
through the full BOMAT... I'm sure plenty of the college students who
took part in DNB training would be willing to "suffer through the
BOMAT" for the $14/hour they were paying them to play DNB
On Oct 8, 11:16 am, Gwern Branwen <gwe...@gmail.com> wrote:
Gwern Branwen
> Speed has a 0.40 correlation with unspeeded reasoning. Timed RAPM has
> a much higher correlation.
And the verbal subtest has a 0.x correlation with the rest of the
Wechsler test...
> Meaning that if I improve my speed with a
> factor 3 I would be able to increase my score by 30% that was the
> average improvement.
Really, you're inferring such a causation from your correlations?
> Given the large amount of people taking the test
> wouldn't there be some people who has optimal speed before training
> and hence don't show any improvement? Why do we not see these cases?
Why do none of them seem to finish all the questions in the pre-test?
After all, there should be some people who have optimal speed before
as well as after, if n-backing is improving IQ and not speed.
--
gwern
http://www.gwern.net/
Pontus Granström
speeded reasoning. You can't just "pour speed" onto something and all
of a sudden it goes faster which you suggest. What's your evidence
that n-back is a speed task? In what way does remembering positions of
squares for a couple of seconds even remotely come close to speed
tasks?
This is what you are claiming: 10-min rapm correlates with IQ only if
you haven't trained on speed. Then you suggest that speed training
increases the score but the test variance doesn't depend on speed.
Isn't this a contradiction. The raw performance can't both depend on
speed and not. The conclusion is that speed rapm doesn't depend on
speed but rather other functions such as attentional control, updating
executive function and so on.
Gwern Branwen
> I still don't think you understand, speed is not the same thing as
> speeded reasoning. You can't just "pour speed" onto something and all
> of a sudden it goes faster which you suggest. What's your evidence
> that n-back is a speed task? In what way does remembering positions of
> squares for a couple of seconds even remotely come close to speed
> tasks?
One is increasing speed all over the place - speed of switching one's
attention around, of refocusing on a new round, of fixating on the
squares, and so on.
> This is what you are claiming: 10-min rapm correlates with IQ only if
> you haven't trained on speed. Then you suggest that speed training
> increases the score
I followed you up to here, but past this I don't know what you're talking about.
> but the test variance doesn't depend on speed.
> Isn't this a contradiction. The raw performance can't both depend on
> speed and not. The conclusion is that speed rapm doesn't depend on
> speed but rather other functions such as attentional control, updating
> executive function and so on
--
gwern
http://www.gwern.net/
Pontus Granström
control which is highly linked to performance on IQ-tests. My other
question was this, if I can improve performance A by training on sub
performance B then B must be a part of A. Meaning that even if I
haven't trained my performance B it would still be a part of A. You
claim that speed RAPM does not depend on speed, because you *think*
that speed is not a part of intelligence, but if you train your speed
all of a sudden it does.
Pontus Granström
studying vocabulary. Mental speed is a underlying cognitive function
for almost all tasks and depend on the the thickness of axons as well
as their myeliniation. Far from something you just can "turn up" my
will. Since n-back overlaps with g and causes white matter increases
it might very well be a very real and fundamental increase in IQ
rather than a isolated change to processes that only are statiscally
linked to IQ. I still think your arguments needs some data to support
them, until then read this article.
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2630965/
Gwern Branwen
> I also think it's funny that you compare mental speed training with
> studying vocabulary. Mental speed is a underlying cognitive function
> for almost all tasks and depend on the the thickness of axons as well
> as their myeliniation. Far from something you just can "turn up" my
> will. Since n-back overlaps with g and causes white matter increases
> it might very well be a very real and fundamental increase in IQ
> rather than a isolated change to processes that only are statiscally
> linked to IQ. I still think your arguments needs some data to support
> them, until then read this article.
>
> http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2630965/
From a quick look, this is again a correlation. Again, my vocab
example is very simple and very real.
DNB research ought to be held to the highest standards. We all know
how very important IQ is, how it affects every part of life. If DNB
genuinely could boost IQ by ~10 points reliably in all sorts of
people, this would be one of the most important discoveries ever in
psychology. This would be the sort of thing you change society for,
and with: it would be educational malpractice not to have an hour or
two a week (at least!) of n-backing. Not to do so would be the
equivalent of not having fluoride in the drinking water, iodide in the
salt, or vitamin D in the milk. And so on.
Sloppy practices like speeding make me very angry: if DNB is for real,
it delays any general adoption, it delays any sensible cost-benefit
calculation, and it wastes our time discussing whether the speeding is
a major issue or not. And obviously if DNB is not for real, it delays
discussion and assessment of what DNB *is* good for. (I suspect that a
good fraction of n-backers would not be n-backing if all it did was
increase WM.)
--
gwern
http://www.gwern.net/
Pontus Granström
RAPM you believe RAPM is a test where 100% manages questions 1 95%
question two and so on, this is not the case. The hardest problem has
a 5% completion rate. There are more correct answers on problem 31 and
32 than on problem 28. I think it's because you got the wrong
assumptions about how the test is constructed you feel angry by
applying time limits. Problem 13 has less correct answers than many of
the later problems. So it's the sum over all problems that give your
score and no problem has completion rate lower than 5%. It might
explain why time limits have little influence on the test because all
problems are very similar in difficulty (as noticed by myself).
I suggest you study the frequency for completing RAPM problems in the
article I will upload in a second, "working memory capacity and fluid
intelligence abilities."
Steve
serious issue that cannot be satisfactorily explained away by relying
on suppositions and correlations. The way to resolve it is to remove
the time limit. The fact that Jaeggi used it the first time was highly
suspicious - there are a set of conditions that are supposed to be
used whenever an intelligence test is administered. The fact that
Jaeggi used it again rather than responding to the criticisms makes it
even more suspicious. The claim that the paper is peer-reviewed is
irrelevant. Peer review is concerned with whether the analyses were
done correctly and the study does show something new. Using the time
limit does not mean that the data are invalid and should not be
published. It means that the data do not provide a definitive answer
and further research is needed. Researchers (including the reviewers)
would be aware of the short-comings such as the time limit, and would
expect further research before this issue is resolved. Gwern has a
right to be angry because, rather than resolving a major issue, Jaeggi
have been duplicitous and not addressed it. Unfortunately, there are
some like yourself that have been conned by this. Rather than
acknowledging a short-fall in the study, you are doing everything
possible to defend it.
On Oct 9, 7:30 am, Pontus Granström <lepon...@gmail.com> wrote:
> But gwern n-back does not transfer to WMC. Second when it comes to
> RAPM you believe RAPM is a test where 100% manages questions 1 95%
> question two and so on, this is not the case. The hardest problem has
> a 5% completion rate. There are more correct answers on problem 31 and
> 32 than on problem 28. I think it's because you got the wrong
> assumptions about how the test is constructed you feel angry by
> applying time limits. Problem 13 has less correct answers than many of
> the later problems. So it's the sum over all problems that give your
> score and no problem has completion rate lower than 5%. It might
> explain why time limits have little influence on the test because all
> problems are very similar in difficulty (as noticed by myself).
>
> I suggest you study the frequency for completing RAPM problems in the
> article I will upload in a second, "working memory capacity and fluid
> intelligence abilities."
>
> On Sat, Oct 9, 2010 at 4:18 PM, Gwern Branwen <gwe...@gmail.com> wrote:
Pontus Granström
explanations to why speed rapm might be valid as a test. In fact it
shows higher correlation with intelligence then picking 12 questions.
Here are a few key facts:
The correlation between gf and speed is around 0.2
The correlation between rapm and speed rapm is a lot higher. Meaning
that there has to be something else to the business.
The difficulty of the problems are not linearly increasing but rather
are quite equal in load except for the last one which shows a 5%
completion rate. That they were more difficult were the whole
rationale for not accepting a speed rapm. I agree if you give someone
"easy questions" they would not be able to reach their "true level"
and therefore the test would correlate with speed instead of
IQ, but since the correlation is so much higher for speed rapm than
for speed this cannot be the case.
That's all I am saying.
All this taken together, RAPM might very well be suited for training
since it's a test that depends on accuracy over several equally
difficult problems (more or less). Attentional control might very well
make people more accurate and hence be able to solve more problems
correctly and thereby increasing the score. I believe this is the
case, because the speed theory doesn't hold.
milestones
complete lack of information what are the IQ's of the group that their
testing? The most recent study claims ceiling effects as the reason
for opting to go short versus normal time...leaving us to guess what
the IQ's of the group she's testing and the ceiling of the tests being
administered? University of Michigan is very selective so it seems
likely students would probably have IQ's in the mid 120's (only
slightly less than Harvard kids).
(I have no idea about the subjects at the Taiwanese university used in
study 2).
Very high IQ, but I think you can reliably measure this population
with fluid intelligence tests such as the Raven's without having to
rely on speeded tests. In fact, the RAPM has a long history of being
administered to gifted populations (usually along with a crystalized
measure like the Concept Mastery Test) such as those in John's Hopkins
gifted group. So I don't think ceiling effects of either the BOMAT or
the APM for those in the study are strong enough reasons to shorten
the test.
The last sentence of the abstract indicates zeal for replication,
which is probably why Jaeggi et al., didn't change horses in midstream
either in addressing Moody's criticism or taking on another more
random sample (i.e. not comprised of university students at a public
ivy). If she did change to timed, then there would be no true
replication of previous findings but rather she would have to
replicate these results with yet another study. In any event, I think
what's needed now is to do a study comprised of a heterogenous
population with a normally timed test (such as the standard APM) or
take a somewhat homogenous group of non LD's with Wechsler IQ's
between 100-120 where ceiling effects should be a non issue for normal
administration. If they can replicate this, it would seal it.
Unfortunately, as impressive as the recent study is, holes that were
pointed out before, remain.
Although obviously scientific, her writing tone indicates Jaeggi has a
little bit of the attorney or perhaps burgeoning business woman in
her...I think she might want to be careful of seeming too agenda
driven, as agenda's seem to work better when launching into big theory-
mode rather than conducting scientific studies. I wondered too about
how she used "our hypothesis...to be correct," as much as she did.
Even if true, it reveals something about how high the stakes of the
hypothesis are to her and her team. A little more poker face might
help there, imo.
Pontus Granström
support him nor his assumptions on why she uses speed tests. It's not
like RAPM comes without speed limits. You are given 40 minutes
usually. As we seen in the articles I've uploaded, speed doesn't
contribute to the 10-min version and the correlation is a lot higher
for speed rapm than for speed. This doesn't seem to be a problem for
the critics though. Moody is wrong on point after point.
I think the whole problem is that some have a very emotional loaded
view on intelligence, it's a matter where we project a lot of our own
believes and values into. While Gf-tests usually deals with low level
working memory operations and rules applied to visual figures. It's
not everything that makes us good humans but rather a small fragment
that in many cases doesn't play that great role compared to other
human qualities.
Think about it, the tests were developed in the 50's from intiution.
It only the last years people have tried to understand intelligence as
"biological computer" while the rest still holds abstract
philosophical discussions of it's nature and correlations. Naturally
people like Mensa would not welcome and increase of IQ of 10-20 points
it would ruin the whole society. It's almost like we should be
forbidden to understand intelligence and what IQ-tests measure.
polar
factor in IQ. But nobody knows exactly how important. But as I
believe, that "more quantity after some point becomes new
quality" (whteher we are talking rocks or neurons), I think speed is
quite important factor. Imagine somebody who makes full score in 60
minutes, while another person makes it in 30 minutes. I think they
have substiantally different iq. Altough speed is definitely not the
only one factor, its definitely very important and I dont have too
much of a problem with speeded raven (altough I gave people full
raven). Raven itself is timed too, and that's the point. So its very
appropriate that in the short version, no person made all the tasks at
first.
next thing, I pretty much believe that dnb CAN "genuinely boost IQ by
~10 points reliably in all sorts of people", and in my opinion this IS
"one of the most important discoveries ever in psychology." Because of
that I changed my dissertation topic in 3 days and was willing to
subscribe to it for 4 years. BUT, I would be careful when it comes to
"life changing effects of improved iq". Yeah, you feel the difference
(like winning a lot of money), and you get better at A general skill.
But again, IQ does not correlate with happiness, social skills, or
ethics. And btw, I think that mensa would actually welcome if iq could
be improved. They are iq fans, not elitarians (and I believe at least
half of them is true to ). And dont forget that iq is a statistical
construct, so you can abstract from WM factor in some professional iq
tests (like woodcock johnson).
Pontus Granström
can find it in their official database.
Pontus Granström
Gwern Branwen
Another interesting article that will, perhaps, strike a little closer
to home because it discusses problems in psychology:
"The Truth Wears Off: Is there something wrong with the scientific method?"
http://www.newyorker.com/reporting/2010/12/13/101213fa_fact_lehrer?currentPage=all
> But the data presented at the Brussels meeting made it clear that something strange was happening: the therapeutic power of the drugs appeared to be steadily waning. A recent study showed an effect that was less than half of that documented in the first trials, in the early nineteen-nineties. Many researchers began to argue that the expensive pharmaceuticals weren’t any better than first-generation antipsychotics, which have been in use since the fifties. “In fact, sometimes they now look even worse,” John Davis, a professor of psychiatry at the University of Illinois at Chicago, told me.
...
> [Psychology professor] Schooler tried to put the problem out of his mind; his colleagues assured him that such things happened all the time. Over the next few years, he found new research questions, got married and had kids. But his replication problem kept on getting worse. His first attempt at replicating the 1990 study, in 1995, resulted in an effect that was thirty per cent smaller. The next year, the size of the effect shrank another thirty per cent. When other labs repeated Schooler’s experiments, they got a similar spread of data, with a distinct downward trend. “This was profoundly frustrating,” he says. “It was as if nature gave me this great result and then tried to take it back.” In private, Schooler began referring to the problem as “cosmic habituation,” by analogy to the decrease in response that occurs when individuals habituate to particular stimuli. “Habituation is why you don’t notice the stuff that’s always there,” Schooler says. “It’s an inevitable process of adjustment, a ratcheting down of excitement. I started joking that it was like the cosmos was habituating to my ideas. I took it very personally.”
...
> The craziness of the hypothesis was the point: Schooler knows that precognition lacks a scientific explanation. But he wasn’t testing extrasensory powers; he was testing the decline effect. “At first, the data looked amazing, just as we’d expected,” Schooler says. “I couldn’t believe the amount of precognition we were finding. But then, as we kept on running subjects, the effect size”—a standard statistical measure—“kept on getting smaller and smaller.” The scientists eventually tested more than two thousand undergraduates. “In the end, our results looked just like Rhine’s,” Schooler said. “We found this strong paranormal effect, but it disappeared on us.”
...
> Then the theory started to fall apart. In 1994, there were fourteen published tests of symmetry and sexual selection, and only eight found a correlation. In 1995, there were eight papers on the subject, and only four got a positive result. By 1998, when there were twelve additional investigations of fluctuating asymmetry, only a third of them confirmed the theory. Worse still, even the studies that yielded some positive result showed a steadily declining effect size. Between 1992 and 1997, the average effect size shrank by eighty per cent.
>
> And it’s not just fluctuating asymmetry. In 2001, Michael Jennions, a biologist at the Australian National University, set out to analyze “temporal trends” across a wide range of subjects in ecology and evolutionary biology. He looked at hundreds of papers and forty-four meta-analyses (that is, statistical syntheses of related studies), and discovered a consistent decline effect over time, as many of the theories seemed to fade into irrelevance. In fact, even when numerous variables were controlled for—Jennions knew, for instance, that the same author might publish several critical papers, which could distort his analysis—there was still a significant decrease in the validity of the hypothesis, often within a year of publication. Jennions admits that his findings are troubling, but expresses a reluctance to talk about them publicly.
...
> In recent years, publication bias has mostly been seen as a problem for clinical trials, since pharmaceutical companies are less interested in publishing results that aren’t favorable. But it’s becoming increasingly clear that publication bias also produces major distortions in fields without large corporate incentives, such as *psychology* and ecology.
...
> Between 1966 and 1995, there were forty-seven studies of acupuncture in China, Taiwan, and Japan, and every single trial concluded that acupuncture was an effective treatment. During the same period, there were ninety-four clinical trials of acupuncture in the United States, Sweden, and the U.K., and only fifty-six per cent of these studies found any therapeutic benefits.
(Emphasis added.)
--
gwern
http://www.gwern.net