Re: Jaeggi 2011

[Pontus Granström]

Gwern she doesn't avoid it, she doesn't have to deal with his questions, because it's something that comes from a layman that has his own neuropsychological constructs.

On Wed, Jun 15, 2011 at 9:35 PM, Gwern Branwen <gwe...@gmail.com> wrote:

### Jaeggi 2011

Jaeggi's work with University of Michigan is available as a preprint:
Jaeggi, Buschkuehl, Jonides & Shah. 2011. ["Short- and long-term
benefits of cognitive
training"](http://www.pnas.org/content/early/2011/06/03/1103228108.abstract)
([PDF](http://www.pnas.org/content/early/2011/06/03/1103228108.full.pdf))

> "We trained elementary and middle school children by means of a videogame-like working memory task. We found that only children who considerably improved on the training task showed a performance increase on untrained fluid intelligence tasks. This improvement was larger than the improvement of a control group who trained on a knowledge-based task that did not engage working memory; further, this differential pattern remained intact even after a 3-mo hiatus from training. We conclude that cognitive training can be effective and long-lasting, but that there are limiting factors that must be considered to evaluate the effects of this training, one of which is individual differences in training performance. We propose that future research should not investigate whether cognitive training works, but rather should determine what training regimens and what training conditions result in the best transfer effects, investigate the underlying neural and cognitive mechanisms, and finally, investigate for whom cognitive training is most useful."

It is worth noting that the study used Single N-back (visual). Unlike
Jaeggi 2008, "despite the experimental group’s clear training effect,
we observed no significant group × test session interaction on
transfer to the measures of Gf" (so perhaps the training was long
enough for subjects to hit their ceilings). The group which did n-back
could be split, based on final IQ & n-back scores, into 2 groups;
interestingly "Inspection of n-back training performance revealed that
there were no group differences in the first 3 wk of training; thus,
it seems that group differences emerge more clearly over time [first 3
wk: t(30) < 1; P = ns; last week: t(16) = 3.00; P < 0.01] (Fig. 3)." 3
weeks is ~21 days, or >19 days (the longest period in Jaeggi 2008).
It's also worth noting that Jaeggi 2011 seems to avoid Moody's most
cogent criticism, the speeding of the IQ tests; from the paper's
'Material and Methods' section;

> "We assessed matrix reasoning with two different tasks, the Test of Nonverbal Intelligence (TONI) (23) and Raven’s Standard Progressive Matrices (SPM) (24). Parallel versions were used for the pre, post-, and follow-up test sessions in counterbalanced order. For the TONI, we used the standard procedure (45 items, five practice items; untimed), whereas for the SPM, we used a shortened version (split into odd and even items; 29 items per version; two practice items; timed to 10 min after completion of the practice items. Note that virtually all of the children completed this task within the given timeframe)."

The IQ results were, specifically, the control group averaged
15.33/16.20 (before/after) correct answers on the SPM and 20.87/22.50
on the TONI; the n-back group averaged 15.44/16.94 SPM and 20.41/22.03
TONI. 1.5 more right questions rather than ~1 may not seem like much,
but the split groups look quite different - the 'small training gain'
n-backing group actually fell on its second SPM and improved by <0.2
questions on the TONI, while the 'large training gain' increased >3
questions on the SPM and TONI. The difference is not so dramatic in
the followup 3 months later: the small group is now 17.43/23.43
(SPM/TONI), and the large group 15.67/24.67. Strangely in the
followup, the control group has a higher SPM than the large group (but
not the small group), and a higher TONI than either group. (The
control group has higher IQ scores on both TONI & SPM in the followup
than the aggregate n-back group.)

Jaeggi 2011 has been discussed in mainstream media. From the _Wall
Street Journal_'s ["Boot Camp for Boosting
IQ"](http://online.wsj.com/article/SB10001424052702304432304576371462612272884.html):

> "...when several dozen elementary- and middle-school kids from the Detroit area used this exercise for 15 minutes a day, many showed significant gains on a widely used intelligence test. Most impressive, perhaps, is that these gains persisted for three months, even though the children had stopped training...these schoolchildren showed gains in fluid intelligence roughly equal to five IQ points after one month of training...There are two important caveats to this research. The first is that not every kid showed such dramatic improvements after training. Initial evidence suggests that children who failed to increase their fluid intelligence found the exercise too difficult or boring and thus didn't fully engage with the training."

From _Discover_'s blogs, ["Can intelligence be boosted by a simple
task? For some…"](http://blogs.discovermagazine.com/notrocketscience/2011/06/13/can-intelligence-be-boosted-by-a-simple-task-for-some/),
come additional details:

> She [Jaeggi] recruited 62 children, aged between seven and ten. While half of them simply learned some basic general knowledge questions, the other half trained with a cheerful computerised n-back task. They saw a stream of images where a target object appeared in one of six locations – say, a frog in a lily pond. They had to press a button if the frog was in the same place as it was two images ago, forcing them to store a continuously updated stream of images in their minds. If the children got better at the task, this gap increased so they had to keep more images in their heads. If they struggled, the gap was shortened.
>
> Before and after the training sessions, all the children did two reasoning tests designed to measure their fluid intelligence. At first, the results looked disappointing. On average, the n-back children didn’t become any better at these tests than their peers who studied the knowledge questions. But according to Jaeggi, that’s because some of them didn’t take to the training. When she divided the children according to how much they improved at the n-back task, she saw that those who showed the most progress also improved in fluid intelligence. The others did not. Best of all, these benefits lasted for 3 months after the training. That’s a first for this type of study, although Jaeggi herself says that the effect is “not robust.” Over this time period, all the children showed improvements in their fluid intelligence, “probably [as] a result of the natural course of development”.
>
> ...Philip Ackerman, who studies learning and brain training at the University of Illinois, says, “I am concerned about the small sample, especially after splitting the groups on the basis of their performance improvements.” He has a point – the group that showed big improvements in the n-back training only included 18 children....Why did some of the children benefit from the training while others did not? Perhaps they were simply uninterested in the task, no matter how colourfully it was dressed up with storks and vampires. In Jaeggi’s earlier study with adults, every volunteer signed up themselves and were “intrinsically motivated to participate and train.” By contrast, the kids in this latest study were signed up by their parents and teachers, and some might only have continued because they were told to do so.
>
> It’s also possible that the changing difficulty of the game was frustrating for some of the children. Jaeggi says, “The children who did not benefit from the training found the working memory intervention too effortful and difficult, were easily frustrated, and became disengaged. This makes sense when you think of physical training – if you don’t try and really run and just walk instead, you won’t improve your cardiovascular fitness.” Indeed, a recent study on IQ testing which found that [they reflect motivation as well as intelligence](http://blogs.discovermagazine.com/notrocketscience/2011/04/26/iq-scores-reflect-motivation-as-well-as-intelligence/).

--
gwern
http://www.gwern.net

--
You received this message because you are subscribed to the Google Groups "Dual N-Back, Brain Training & Intelligence" group.
To post to this group, send email to brain-t...@googlegroups.com.
To unsubscribe from this group, send email to brain-trainin...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/brain-training?hl=en.

[Gwern Branwen]

[Windt]

"When she divided the children according to how much they improved at
the n-back task, she saw that those who showed the most progress also
improved in fluid intelligence. The others did not."

It certainly sounds like correlation rather than causation. The kids
who did best at an intelligence-related task also did best at some
other intelligence-related task later on (what exactly IS a test of
fluid intelligence? I thought quantifying intelligence was
problematic).

> IQ"](http://online.wsj.com/article/SB1000142405270230443230457637146261227...

>
> > "...when several dozen elementary- and middle-school kids from the Detroit area used this exercise for 15 minutes a day, many showed significant gains on a widely used intelligence test. Most impressive, perhaps, is that these gains persisted for three months, even though the children had stopped training...these schoolchildren showed gains in fluid intelligence roughly equal to five IQ points after one month of training...There are two important caveats to this research. The first is that not every kid showed such dramatic improvements after training. Initial evidence suggests that children who failed to increase their fluid intelligence found the exercise too difficult or boring and thus didn't fully engage with the training."
>
> From _Discover_'s blogs, ["Can intelligence be boosted by a simple

> task? For some…"](http://blogs.discovermagazine.com/notrocketscience/2011/06/13/can-int...),

> come additional details:
>
> > She [Jaeggi] recruited 62 children, aged between seven and ten. While half of them simply learned some basic general knowledge questions, the other half trained with a cheerful computerised n-back task. They saw a stream of images where a target object appeared in one of six locations – say, a frog in a lily pond. They had to press a button if the frog was in the same place as it was two images ago, forcing them to store a continuously updated stream of images in their minds. If the children got better at the task, this gap increased so they had to keep more images in their heads. If they struggled, the gap was shortened.
>
> > Before and after the training sessions, all the children did two reasoning tests designed to measure their fluid intelligence. At first, the results looked disappointing. On average, the n-back children didn’t become any better at these tests than their peers who studied the knowledge questions. But according to Jaeggi, that’s because some of them didn’t take to the training. When she divided the children according to how much they improved at the n-back task, she saw that those who showed the most progress also improved in fluid intelligence. The others did not. Best of all, these benefits lasted for 3 months after the training. That’s a first for this type of study, although Jaeggi herself says that the effect is “not robust.” Over this time period, all the children showed improvements in their fluid intelligence, “probably [as] a result of the natural course of development”.
>
> > ...Philip Ackerman, who studies learning and brain training at the University of Illinois, says, “I am concerned about the small sample, especially after splitting the groups on the basis of their performance improvements.” He has a point – the group that showed big improvements in the n-back training only included 18 children....Why did some of the children benefit from the training while others did not? Perhaps they were simply uninterested in the task, no matter how colourfully it was dressed up with storks and vampires. In Jaeggi’s earlier study with adults, every volunteer signed up themselves and were “intrinsically motivated to participate and train.” By contrast, the kids in this latest study were signed up by their parents and teachers, and some might only have continued because they were told to do so.
>

> > It’s also possible that the changing difficulty of the game was frustrating for some of the children. Jaeggi says, “The children who did not benefit from the training found the working memory intervention too effortful and difficult, were easily frustrated, and became disengaged. This makes sense when you think of physical training – if you don’t try and really run and just walk instead, you won’t improve your cardiovascular fitness.” Indeed, a recent study on IQ testing which found that [they reflect motivation as well as intelligence](http://blogs.discovermagazine.com/notrocketscience/2011/04/26/iq-scor...).
>
> --
> gwernhttp://www.gwern.net

[Pontus Granström]

When they design IQ-tests they group them accordingly to first and second order factors. For example, matrix reasoning, defined as adding or removing figures, rules for their movement and so on correlates highly with for example finding a pattern in a number series, hence they are placed in the same "bucket". Word knowledge has a high inter correlation, those are placed in the same bucket. Remembering
a bunch of visual objects correlates with rotation of pictures, so they are placed in the "bucket" visual spatial test.

Then they name the buckets after their main "apparent feature". Adding pictures is considered reasoning, explaining words verbal ability and so on. After this has been done they calculate their predictive power, for example Gf tests are more predictive of overall performance than word knowledge so they are highly g-loaded, this usually reveals a hierarchic structure. Almost all predictive power comes
from the G-load. Pure Gv-gc only explains one percent of the variation of the actual performance (what one tries to predict) and their only value is usually that they are statistically linked to G. That is,
the faster your brain is, the better your genetic "make up", hence you learn more per unit of time, this only holds if you don't study words, since it will expand the time spent and thus lowering
the bit/s.

This is how they do it.

[Gwern Branwen]

I'm surprised and a little disappointed to see so little discussion of
Jaeggi 2011, as compared to a random slacker kid or any of the other
random topics that seem to obsess this ML these days.

I expected as much, though, so after pondering the strange IQ test
scores and followups and still not finding any convincing explanation,
I asked another group of people to look at it: LessWrong. I posted
links as a Discussion article:
http://lesswrong.com/lw/68k/nback_news_jaeggi_2011_or_is_there_a/
(Notice I didn't describe what misgivings I had and I specifically
asked people to read the paper *first*.)

Of the 18 comments (many more than here), none seemed to regard it as
even weak evidence, which is interesting. I'll quote some of the most
relevant comments since I know otherwise a lot of people won't bother
reading the link.

[Jonathan Graehl](http://lesswrong.com/lw/68k/nback_news_jaeggi_2011_or_is_there_a/4d34)
(who, incidentally, has expertise in probability & statistics;
http://www.isi.edu/~graehl/publications.html &
http://www.isi.edu/~graehl/CV.html) writes:

> My primary objection is: perhaps some of the students in both groups got smarter (these are 8-9 year olds and still developing) for reasons independent of the interventions, which caused them to improve on the n-back training task AND on the other intelligence tests (fluid intelligence, Gf). If you separated the "active control" group into high and low improvers post-hoc just like was done for the n-back group, you might see that the active control "high improvers" are even smarter than the n-back "high improvers". We should expect some 8-9 year olds to improve in intelligence or motivation over the course of a month or two, without any intervention.
>
> Basically, this result sucks, because of the artificial post-hoc division into high- and low- responders to n-back training, needed to show a strong "effect". I'm not certain that the effect is artificial; I'd have to spend a lot of time doing some kind of sampling to show how well the data is explained by my alternative hypothesis.
>
> It's definitely legitimate to look at the whole n-back group vs. the whole active control group. Those results there aren't impressive at all. I just can't give any credit for the post-hoc division because I don't know how to properly penalize it and it's clearly self-serving for Jaeggi. It's borderline deceptive that the graphs don't show the unsplit n-back population.
>
> It's unsurprising (probably offering no evidence against my explanation) that the initial average n-back score for the low improvers is higher than the initial average for the high improvers; this is what you'd expect if you split a set of paired samples drawn from the same distribution with no change at all, for example.

[Douglas Knight](http://lesswrong.com/lw/68k/nback_news_jaeggi_2011_or_is_there_a/4d3s)
in replying to Jonathan notices the same problem I did in the IQ score
section:

> When you say that the aggregate results "aren't impressive," you imply that they are positive, but if I read table 1 correctly, the aggregate results are often negative.

[Unnamed](http://lesswrong.com/lw/68k/nback_news_jaeggi_2011_or_is_there_a/4d3h)
offers what seems like a pretty good summary:

> The result looks pretty weak. They had 62 kids. First, they gave all the kids a fluid intelligence test to measure their baseline fluid intelligence. Then half the kids (32) were given a month of n-back training (which the authors expect to increase their fluid intelligence) while the other half (30) did a control training which was not supposed to influence fluid intelligence. At the end of the month's training all of the kids took another fluid intelligence test to see if they'd improved, and 3 months later they all took a fluid intelligence test once more to see if they'd retained any improvement.
>
> The result that you'd look for with this design, if n-back training improves fluid intelligence, is that the group that did n-back training would show a larger increase in fluid intelligence scores from the baseline test to the test after training. They looked and did not find that result - in fact, it was not even close to significant (F < 1). That's the effect that the study was designed to find, and it wasn't there. So that's not a good sign.
>
> The kids who did n-back training did improve at the n-back task, so the authors decided to look at the data in another way - they divided the 32 kids in that group in half based on how much they had improved on the n-back task, and looked separately at the 16 who improved the most and the 16 who improved the least. The group of 16 high-improvers did improve on the fluid intelligence test, significantly more than the control group, and they retained that improvement on the follow-up test of fluid intelligence. That is the main result that the paper reports, which they interpret as a causal effect of n-back training. The 16 low-improvers did not have a statistically significant difference from the control group on the fluid intelligence test.
>
> But this just isn't that convincing a result, as the study no longer has an experimental design when you're using n-back performance to divide up the kids. If you give kids 2 intelligence tests (one the n-back task, one the fluid intelligence test), and a month later you give them both intelligence tests again, then it's not surprising that the kids who improved the most on one test would tend to also improve the most on the other test. And that's basically all that they found. Their study design involved training the kids on one of those two tests (n-back) during the month-long gap, but there's no particular reason to think that this had a causal effect on their improvement on the other test. There are plenty of variables that could affect intelligence test performance which would affect performance on both tests similarly (amount of neural development, being sick, learning disability, etc.).
>
> If there is a causal benefit of n-back, then it should show up in the effect that they were originally looking for (more fluid intelligence improvement in the group that did n-back training than the control group). Perhaps they'd need a larger sample size (200 kids instead of 62?) to find it if the benefit only happens to some of the kids (as they claim), but if some kids benefit from the training while others get no effect from it then the net effect should be a measurable benefit. I'd want to see that result before I'm persuaded.

--
gwern
http://www.gwern.net

[ao]

Preferably adults in their 30s onward, i.e., when the brain has
reached a semblance of genetic maturity and before performance
declination.

argumzio


On Jun 28, 8:29 pm, Zaraki <zaraki...@gmail.com> wrote:
> Well, personally I don't put much stock in this study. I got
> disappointed the moment I heard that kids were used, even before I
> read it. The problem with them still developing and beeing poorly
> motivated is not something that should be involved in this mix. Folly
> to choose children from the get-go in other words. Why Jaeggi decided
> to use them I cannot fathom. Did she hope for greater results in
> children, or simply want schools to start implementing the n-back
> training? More proof is needed with adults first.

[Zaraki]

woopsy.. deleted my own post. Had just planned to edit it. Anyways, I
basically said that this study is weak and that older testees should
have been used, not developing children who don't even understand the
value of getting better at this task.

[ao]

"Children who don't even understand the value of getting better at
this task" is irrelevant as other factors can be used as motivators.

"Now, listen, Jimmy boy, if you improve your n-level by this much,
I'll give you something really, really nice, okay?" "Okay! I just love
toys/money/someglucsucfructosepackedthing."

How do you think scientists are able to train animals of sub-human
intelligence to do seemingly complicated tasks? Conditioning. No need
to understand anything besides "I want more of something good and less
of something bad". There's plenty of "value" in that.

The real issue is children are well known not to have terribly
reliable scores on psychometric variables like I.Q. tests, because
their brains are under siege by constantly changing gene expression.
I.Q. test scores from the period of birth to 16 years are remarkably
unreliable measures of the final adult-age I.Q. level. On those
grounds, if there were any kind of gain on account of training, it
could easily be masked by the developmental trajectory of the children
– and it is not all too likely that 62 of them qualify as a random
sample at the end of the day. Speaking generally, of course.

argumzio

[Zaraki]

well, we don't know if they used motivators or just told them to do
it. I expect it to be the latter, as Jaeggi used that reason to divide
the experimental group later on.

[ao]

Actually, in the penultimate page, they write:

"After the posttest, we assessed the children’s engagement
and motivation for training with a self-report questionnaire
consisting of 10 questions in which they rated the training or control
task on
dimensions such as how much they liked it, how difficult it was, and
whether
they felt that they became better at it."

Turns out that .67 of the total variance [for completing the task, if
not task-related gains, one assumes] can be explained by these
intrinsic motivators. Not surprising. Who is motivated to do a task
and get better at it usually does. But who is motivated ("I'm smart, I
can do it", "hey, this problem is fun") also tends to be more
intelligent, particularly among children. Nothing new.

argumzio

[ao]

Ah, it's so easy to ignore null or negative results but so hard in the
case of positive ones.

Unfortunately, scientific practice doesn't work that way.

The result of this study isn't the first of its kind, and it isn't
going to be the last.

argumzio


On Jun 28, 9:36 pm, ailambris <ailamb...@gmail.com> wrote:
> I don't think they were ignorant to the fact that there would be some
> confounding motivational issues. Maybe they believed it would actually
> play in their favor, who knows? Every study is a product to be sold to
> the medical community. Sometimes you gamble, and sometimes you do
> really well for the circumstances. The opposite may be true here. It
> could very well be that they were targeting youth because, well, maybe
> somehow the application may be marketable, is there a patent pending
> or something? If I were a parent, I would be willing to drop heavy
> sums of money on an application if it had been confirmed to produce
> the kind of improvements that n-back is purported to offer.
>
> With a rudimentary understanding of statistics, I know just as well
> that even if your molecule, your device, whatever, actually *does*
> what it is supposed to do, there is still a probability that your
> study is going to be a complete failure. That's why we reproduce them.
> That's why Jaeggi 2011 doesn't bother me.

> ...
>
> read more »

[whoisbambam]

So what is everybody implying here?

just that the study sucks?

or you all believe that dnb does not help improve working memory in
adults who already dont have a good working memory (demonstrated by
the fact that they suck and dnb)?

or what?

> ...
>
> read more »

[The.Fourth.Deviation.]

With the more studies done, and the results being replicated, it gets
more difficult to suggest that results are invalid. People in the
thread keep saying "this study only has N=8, and this one, N=25, or N
= 62. Is it not completely obvious that as you add all of those
together, the actual sample that has been trained with generally the
same tasks, and same improvements, has approached one thousand or more
people, in several different locations and under several different
conditions. Yet the results are the same, positive, time and again.
Retorts?

Also, the people who supposedly don't benefit may actually be
benefitting in some way that is not being tested . E.G. After taking a
few weeks off of training or more, DNB immediately improves my short
term memory whenever I go back to using it. This occurs without fail.
Yet this will not show up in RAPM, because that has to do with mental
rotations etc. In other words, I suggest that these studies may
understate the benefits of DNB, instead of overstate, as some here
imply.

[ao]

Children aren't adults. That's what's being said here, not implied.
Children don't prove squat for a number of obvious, already-stated
reasons.

I regard the study as unconvincing.

argumzio

> ...
>
> read more »

[ailambris]

It'd be nice if you could add the numbers that way, but you can't.

On Jun 28, 9:45 pm, "The.Fourth.Deviation." <davidsky...@gmail.com>
wrote:

[The.Fourth.Deviation.]

I think that's irrelevant. The gestalt is the more important picture.
If DNB causes particular effects in adults, it should also cause the
same effects in children. Not only this, but it should probably cause
stronger adaptations, because training done in youth is quickly
adapted to and causes greater overall benefit. I.e. language training,
musical training before age 12, etc.

The study is also useful, because it can simply be compared to a
longitudinal study which measures the changes in fluid IQ of a normal
group of aging children. If this group appears to show more rapid
increases over such a short time frame, it does indeed give us some
useful information. And the same point I made above, about DNB in
adults vs children, is especially potent. There is no question that
adaptations are being made in all subjects put to training. The
question is how and where those adaptations occur. I.E. all subjects
should show increased cortical activity. It is just the task of the
scientists to show what that means in the case of each individual,
especially since individuals may have difficulty pointing out what has
changed. The battery of tests should be more robust.

> ...
>
> read more »- Hide quoted text -
>
> - Show quoted text -

[The.Fourth.Deviation.]

I don't see any convincing reason that you should not add them for a
rough estimate of overall results.

> > imply.- Hide quoted text -

[genvirO]

Regardless of how unconvinced you are, the results of the study still
remain positive. Regardless of how positive the results of the study
are, we still need more studies but with more substance (brain scans &
other). For instance, we could hypothetically have 20 more studies
that come out with exactly the same thing by the end of the year, yet
we would still have the same unanswered questions.

My belief is that in general n-back does "work" (its obvious that
motivation is a key variable in deciding how much it does "works"
though), however, there are still a lot of unanswered questions.

> ...
>
> read more »

[ao]

I'm guessing you actually read the data table, correct? Are you so
sure such results are "positive"? If you look at the standard
deviation and the mean of both groups for their post results on SPM,
2SD above the mean is equivalent for both groups. (Active: mean=16.2,
SD=5.1; NB grp: mean=16.94, SD=4.75 .) In effect, we're seeing no
change here except in the mean and variance.

By the way, I wrote a response to David earlier, but it hasn't posted
yet, but when it does, if it does, you'll get a better flavor of where
I'm coming from on the "gestalt" of the issue as well.

As regards the asinine idea of adding together different samples from
different studies, such is an erroneous means of deriving explanations
or observations, because the studies were conducted under different
conditions, with different procedures, different individuals, and
different levels of experimental validity. Painting in broad strokes
might work with literature, but in statistics, it isn't so.

A note for others to ruminate on. Let's consider the validity of
separating between a large-training gain group and a small-training
gain group: would not that be equivalent to separating the wheat from
the chaff, i.e., sorting out who learns from a task more quickly and
hence would almost by default do better on a test the second time.

Hmm... if only I could say, "yes! yes! it's all so positive!"

argumzio

> ...
>
> read more »

[Pontus Granström]

Gwern, why I "accuse" you of being biased against n-backing is because I think that you do not give "the other side a chance". For example, you post a full length article that Moody
has written but Jaeggis reply is given as a little link and you have to browse through the whole thread to see her "reply". Same goes with the research, that you first said was "fraud", then
when more recent research was posted you ignored it. There are articles that can give explanations to what might not seem so intuitive obvious. This is the case in many areas. We often
tend to overlook small details etc just when programming and that's also why we need much research.

> ...
>
> read more »

[Pontus Granström]

The study clearly showed that does who advanced in n-backing improved on Gf the whole rationale for transfer. Since they were dealing with kids some of them were unmotivated or found the task
to difficult hence they didn't improve on Gf. Of course to satisfy critics mere looking at the test would improve to Gf to make it "valid". Since they saw improvements in untimed tests in the group that
did improve on n-back it's very likely that this was due to the training, either that, or that it was just a bunch of "lucky guesses", which of course isn't the case. Same goes if you want to improve
your long distance running with intervals, if you do not engage in the intervals at max effort there will be no transfer. If I ask a bunch of kids to do this, some of them would most certainly not complete them, this does not prove that intervals are useless. That's my analysis.

[Pontus Granström]

It might also confirm one of my worries that n-backing will be utilized the most by people who are already above average and got a strong motivation, which will lead to even larger gaps
between different groups.

[genvirO]

"I'm guessing you actually read the data table, correct? Are you so
sure such results are "positive"?"

I don't think the gains are significant, but I also don't think
they're "positively negligible". I was only making reference to the
group that saw the largest gains. The study doesn't enable anyone to
celebrate the glorification of 'intelligence training' (which is, to
the discredit of the "Jaeggi team", what they attempted [being nice]"
but it does bring one closer to understanding what may or may not work
in relation to task parameters and the motivation of subjects in
respect to potential training gains.

On Jun 29, 5:57 pm, Pontus Granström <lepon...@gmail.com> wrote:
> The study clearly showed that does who advanced in n-backing improved on Gf
> the whole rationale for transfer. Since they were dealing with kids some of
> them were unmotivated or found the task
> to difficult hence they didn't improve on Gf. Of course to satisfy critics
> mere looking at the test would improve to Gf to make it "valid". Since they
> saw improvements in untimed tests in the group that
> did improve on n-back it's very likely that this was due to the training,
> either that, or that it was just a bunch of "lucky guesses", which of course
> isn't the case. Same goes if you want to improve
> your long distance running with intervals, if you do not engage in the
> intervals at max effort there will be no transfer. If I ask a bunch of kids
> to do this, some of them would most certainly not complete them, this does
> not prove that intervals are useless. That's my analysis.
>

> On Wed, Jun 29, 2011 at 8:24 AM, Pontus Granström <lepon...@gmail.com>wrote:
>
>
>
> > Gwern, why I "accuse" you of being biased against n-backing is because I
> > think that you do not give "the other side a chance". For example, you post
> > a full length article that Moody
> > has written but Jaeggis reply is given as a little link and you have to
> > browse through the whole thread to see her "reply". Same goes with the
> > research, that you first said was "fraud", then
> > when more recent research was posted you ignored it. There are articles
> > that can give explanations to what might not seem so intuitive obvious. This
> > is the case in many areas. We often
> > tend to overlook small details etc just when programming and that's also
> > why we need much research.
>

> ...
>
> read more »

[Pontus Granström]

From what I could see there was a clear gain, I do not have the exact number infront of me now.

> ...
>
> read more »

[ao]

I would suggest that you read more of the literature that discusses
this with greater claims to expertise than yourself, but then again,
you presumably have already done so, which is why I wouldn't have to
bring up something like this: http://tinyurl.com/26vu6z2 .

Interesting how someone comes away from the literature thinking "the
results are the same, positive, time and again". What a strange state
of affairs. Or perhaps not for a group that does not, almost by
definition, preclude group-think.

Instead of vaguely gesturing at a sample of "one thousand or more
people" why not compile the data and refer to it clearly for us all to
scrutinize without vagueness or ambiguity?

argumzio


On Jun 28, 11:45 pm, "The.Fourth.Deviation." <davidsky...@gmail.com>
wrote:

[Pontus Granström]

We have study after study showing the same thing, still people ask this question: does n-backing work? I am not sure what you would like to see? The exclusion of every probable explanation? That god himself descends and tell us that n-backing works? If the training gain was proportional to the training time does this mean that n-backing was behind this, or was it a coincident? We don't know, that's why we need more research. That's the problem with statistical analysis, it might be due to chance, we don't know why this happened but we know it happened in a group that engaged in n-backing. What else can we say?

[ao]

The results are negligible because what we see in the group with the
largest gains (the point I was trying to make) is that they're able to
learn more with the same amount of training, i.e., are more able,
i.e., are more intelligent, i.e., are more motivated to learn because
they know they're good at it. It isn't as though the training had a
greater effect on them, it is more likely the case that they are more
able, which shows up in both their training gains and their retest
results. One could easily have made a group of higher-retest scores
and lower-retest scores and seen the same general thing.

Anyway, I am seriously not considering this study as a valid indicator
of anything beyond: smart people excel more than not-as-smart people.
Old news in new, seductive clothing. More intelligent people have
always been more motivated, and there's no vague question regarding

"understanding what may or may not work in relation to task parameters
and the motivation of subjects in respect to potential training gains"

as you like to state, which is a red herring if there ever was one.
Funnily enough, though, Jaeggi et al. say, "Our findings show that
transfer to Gf is critically dependent on the amount of the
participants’ improvement on the WM task." That is what I call
marketing, or at a bare minimum _overstating results_.

argumzio

> ...
>
> read more »

[Pontus Granström]

To flip your argument it would mean that not engaging in the task would yield the same results as actively trying to get better? I say this, more research is welcomed. Nothing is more probable then the other, except perhaps that n-backing does work.

> ...
>
> read more »

[ao]

That doesn't flip my argument, and that "n-backing does work" is not
more probable.

argumzio

> ...
>
> read more »

[ao]

Yes, I know it's hard when someone doesn't agree with something you
find so agreeable.

argumzio

[Pontus Granström]

Well maybe not strictly speaking it's a flip I agree (I am not really aware of the definition). But as a consequence of your reasoning, it suggests that the quality of the training is unimportant for the outcome?

> ...
>
> read more »

[Zaraki]

I agree with Jonathan Graehl. No large training gain group was created
for the control group, thereby failing to eliminate 'natural'
improvement as a cause for the gains in fluid intelligence.

Also, referencing to Jaeggi's previous studies in defence of this one
does not improve anything since children weren't used in those.

This is the third time I am trying to send this message, what is going
on?

On Jun 29, 3:17 pm, Pontus Granström <lepon...@gmail.com> wrote:
> We have study after study showing the same thing, still people ask this
> question: does n-backing work? I am not sure what you would like to see? The
> exclusion of every probable explanation? That god himself descends and tell
> us that n-backing works? If the training gain was proportional to the
> training time does this mean that n-backing was behind this, or was it a
> coincident? We don't know, that's why we need more research. That's the
> problem with statistical analysis, it might be due to chance, we don't know
> why this happened but we know it happened in a group that engaged in
> n-backing. What else can we say?
>

[ao]

Define "quality of the training". The only difference indicated by the
study is that highly motivated subjects _tend to do much better_ than
less motivated subjects and hence are more intelligent.

In fact, your "flip" of my argument is a mischaracterization that
grossly misses the point.

argumzio

> ...
>
> read more »

[ao]

Zaraki, Google Groups servers seem to be on the fritz. Just now one of
my messages showed up even though I submitted it 4 hours ago:
http://groups.google.com/group/brain-training/msg/82011d7f25dc2cc8

Jonathan is right. A passive control group was definitely needed in
order to show that such gains really are positive and not a
statistical fluke of highly able children (my premiss).

argumzio

[whoisbambam]

it would be nice if they would study a large, adult group.

perhaps ppl in their 20s who dropped out of high school.

or perhaps blue-collar workers who graduated from high school, perhaps
went to a non-collegiate technical school (non-degree bearing).

i think that would be more objective.


i must admit that i am feeling a little stressed thinking about all
the hours i may have wasted doing dual nback.

:(

:(

Between Gwern, ao, zaraki and others i am beginning to doubt

[ao]

whoisbambam, if it causes you stress, then don't bother. For now, if
you've seen gains (of whatever kind you prefer), then don't stop.

It is simply that at this stage of the research, things are still far
from clear, let alone decisive.

argumzio

[whoisbambam]

argumzio,

well, being a victim of placebo (dnb) sucks.

and remaining in ignorance sucks even more.

so now i find myself indecisive and therefore anxious.

based on the way i feel and the way my ability to debate and think
seems to have improved, it seems very parodoxical to me.

I am familiar with placebo.

Unfortunately, I have had to administer placebo to a retired police
officer with hypertrophic cardiomyopathy--he had come in with heart
attack symptoms, i had to call oregon health sciences univesity (ohsu--
i worked there with some heart-lung transplant patients,
intraventricular-monitored neuro patients, etc) only to be told not to
worry about 'missing' a dose of the medication, as it was a placebo.

And, of couse, i could not tell the patient, family, etc.

I am not so sure i could be a good research scientist.

It was not so easy for me to keep my mouth shut and 'allow' this guy
to die due to sugar-pill adminstration.

[ao]

One's ability to debate improves with... debating more. So that is a
false positive.

I think the use of placebos in the manner you've described is
unethical and applaud you for not keeping your mouth shut.

Anyway, no one blames you if you feel like taking a break with DNB.
Personally, I found more in QNB, so you might want to explore other
modes.

argumzio

[eurekasabiduria]

Dnb ¿working or not working?

[The.Fourth.Deviation.]

The very article posted by argumzio, which supposedly says that WM
training does not work, says in the summary that WM is improved with
training, and that it can impact fluid intelligence. I rest my case.

As for the other individual requesting brain scans, I posted a
powerpoint elsewhere that had about a dozen different brain scan
studies all showing adaptations in response to WM training.

Unfortunately for critics, brain scans are substantial enough to prove
that WM training causes adaptations. Not only this, but also studies
that show cortical efficiency with WM tasks improves with WM training.
You can't "fake" increased cortical activity. It's simple fact.

In addition, WM training can cause a cornucopia of other benefits,
such as improving short term memory, with applications in the real
world that might not be measured by these tests. E.G. today at my job,
after training N back for about 2 weeks straight, I was able to recall
various instructions with 100% accuracy due to improved recall from
DNB training. Hence I would say I am "more" intelligent. Therefore DNB
has helped me perform unrelated tasks.

The critics present multitudinous arguments. Unfortunately, none of
them are convincing, especially in the face of the research and
anecdotes. I can't wait to see how the funny skeptics try to explain
away increased cortical efficiency shown in brain scans (last time I
posted a powerpoint of several studies on this, Argumzio simply
avoided responding by saying he didn't have time to look over the
result of so many studies, while complaining that "the brain does not
exist". Nor was anyone else able to explain away physical changes in
the brain in his stead)

> > > > > >http://groups.google.com/group/brain-training?hl=en.- Hide quoted text -

[odranoel]

whoisbambam,

You are not necessarily a "victim of placebo" if the effects you seek
are in fact achieved through the placebo.

Consider studies that show that antidepressants and placebos perform
about as well in alleviating depression.
If you were depressed, and you took any sort of pill to relieve your
depression, and it worked for you, would you really be the victim of
the placebo? Of course not; your intended target was reduction in
depression. If the means are psychological rather than neurochemical,
it should not matter; you only sought to alter neurochemistry in
service of relieving depression. I realize, by the way, that the
issues are more complicated with antidepressants; they are just
analogous and therefore easy.

As argumzio suggests, continue doing what is working for you,
especially if it is not burdensome to do so.

[ao]

You rest your "case" based on a laughable, absolutely hilarious
misreading of the first page? Are you aware that that was a quote of a
claim by cogmed? Have you even closely read the paper itself?

The abstract of the linked article ( http://tinyurl.com/26vu6z2 ):

"Recently, attempts have been made to alter the capacity of working
memory
(WMC) through extensive practice on adaptive working memory tasks that
adjust difficulty in response to user performance. We discuss the
design
criteria required to claim validity as well as generalizability and
how recent
studies do or do not satisfy those criteria. >>>>>>It is concluded
that, as of yet, the
results are inconsistent and this is likely driven by inadequate
controls and
ineffective measurement of the cognitive abilities of
interest.<<<<<<" (my emphasis "><")

Tisk tisk. Sadly, I don't expect better from you, especially with your
track record. I'll just chalk it up to your idiosyncrasies...

Anecdotes of the sort you share prove close to nothing, but I know, it
must be reassuring to convince oneself through repeated exclamations:
"yes, it works, it really, really works! I can feel it! Just look at
what I can memorize now (even though I could have memorized it before
with a little mental effort)!!!!! IT'S WORKING HOLY MOTHER OF!!!" ...
No, it doesn't matter if _you_ feel anything in terms of the objective
evidence available.

When you have time, actually read, with your eyes (and preferably a
functioning brain), the published paper I linked. I'd be interested to
read the "objections" you have regarding it. (By the way, what is it
about that powerpoint I never said I didn't have time to read? Now
you're a liar, and such can be proved. Furthermore, I never said
brains don't exist, I said (in effect) the notion of "the brain" is
misleading because any given set of brains by default are non-
equivalent. Any further straw clutching you'd like to demonstrate?
http://groups.google.com/group/brain-training/msg/da98e3cc2bd72e8b :
"There is no such thing as "the brain" but many brains.") No, a set of
slides is not a hard and fast paper proving anything. Give me a peer-
reviewed article demonstrating whatever it is you think is the case
with links. Maybe in that case I'd be bothered to read it; this is a
fair request.

Anyone care to take up David's so-called claims to truth and accuracy
regarding these matters without low blows or needless polarization and
emotional pellets, all the while meeting a minimum of college-level
reading comprehension? (One would think if DNB training had led to any
gains, it would have showed up there, but I suppose that is expecting
too much.)

argumzio, in pursuit of truth, not fantasies


On Jun 29, 7:59 pm, "The.Fourth.Deviation." <davidsky...@gmail.com>
wrote:

> > > > > > >http://groups.google.com/group/brain-training?hl=en.-Hide quoted text -

[Pontus Granström]

Critics of dual-n-backing have a tremendous ability to turn even the most positive results into something negative. Based on nothing else than their own views. The evidence demands
for saying something negative is a lot lighter than for those positive, for positive results not even ordinary statistics will do it. While for negative results it's only required that it might sound
reasonable no matter the consequences of such claims in a less "face value way". It seemed like the high gainers had lower scores on the IQ-tests compared to the low gainers, still it's assumed
that the high gainers were more intelligent?

[eurekasabiduria]

is how Newton's laws do not work because it does not win Russian
Roulette

[Pontus Granström]

What do you mean by that?

On Thu, Jun 30, 2011 at 2:40 PM, eurekasabiduria <eurekas...@gmail.com> wrote:

is how Newton's laws do not work because it does not win Russian
Roulette

--

[The.Fourth.Deviation.]

Pg 267/8:

task that resembles combination n back is more effective than a
regular n back task.
Skeptics inadvertently provide more data that DNB is effective.

Laughing.

My conclusion after reading this document? All of the users raving
about the benefits of combination nback are vindicated.
My other conclusion: the findings of certain studies are
"inconclusive". However, paired with MRI studies showing that the
brain indeed does change, conclusiveness,
and anecdotal reports, are vindicated.

as for the MRI powerpoint - skeptics: read it and weep. a dozen
different studies all showing brain adaptations from training.
That just so happens to be the result we all seek.
You cant fake an MRI.

spare us the verbose drivel

> ...
>
> read more »- Hide quoted text -

[The.Fourth.Deviation.]

*ignoring the chaff*
interestingly, pg 268 describes a variation of the combination nback
task, and discusses how it essentially provides more benefit than a
standard N back routine. This is a boon for those of us who have
started using combination n back. The statistically significant
improvement/transfer noted in that group mirrors my own noticeable
improvement after doing CNB, rendering argumzio's exaggarated diatribe
about anecdotes rather jokable.

If anything, the study mentioned shows that individuals doing DNB
should try variable combination N back mode, which contains a large
amount of interference and task switching, since this will likely
provide greater benefit than pure DNB (in fact, this has been noted
anecdotally by the members in the combination n back thread i recently
started).

Also, argumzio's verbosity does not make the fore-mentioned, and
posted, powerpoint and its several *referenced & summarized* studies
disappear. This can be found in the thread linked in the above post,
and it shows post wm training MRIs from several studies.

more to come later when i can give a more in depth reading of the
paper (which may as well be called the paper which vindicates
combination N back users)

On Jun 30, 12:36 am, ao <argum...@gmail.com> wrote:

> ...
>
> read more »- Hide quoted text -

[ao]

It isn't "criticism" of DNB, it's criticism of research. You aren't
even paying attention to the situation but erecting straw men and
ignoring the basic elements of discussion here. You have done this
many times before when others attempted to bring in their views
questioning the efficacy of DNB, which any sane, rational person would
do, and attempt to lambaste them with ad hominem generalizations.

If one takes a group of people, tests them one time, and then tests
them at another, slightly later time, one would easily find among
them, if one so desired to divide them according to retest scores, a
group of higher retest scores (equivalent to the largest-gains group)
and a group of lower retest scores (equivalent to the small-gains
group). It must be hard to acknowledge this reality, but that doesn't
make it any less apparent.

argumzio

> ...
>
> read more »

[ao]

David, all you've done is divert attention away from your obvious
stupidity (I was always told a real man is able to own up to his
mistakes) and attempt to refashion the paper after your liking. I'm
afraid that anyone with a college-level reading comprehension will not
walk away from the paper in such hastily generalized terms as a "the
benefits of combination nback are vindicated", not even counting the
fact that no such task is even mentioned in the paper.

The powerpoint shows nothing of value unless you can give me a papers
that specifically make those claims you think are of such high
quality. I know it's like homework, but please do it, because I'm not
going to do it for you.

argumzio


On Jun 30, 1:59 am, "The.Fourth.Deviation." <davidsky...@gmail.com>
wrote:

> ...
>
> read more »

[Pontus Granström]

Your quote:"

Define "quality of the training". The only difference indicated by the
study is that highly motivated subjects _tend to do much better_ than
less motivated subjects and hence are more intelligent."

But it was the opposite, those who gained the most had the lowest scores. How can this be? Are not SPM and TONI a measure of intelligence?

--

[Pontus Granström]

She also comments on Moody (indirectly) in her discussion part, might be worth to read.

[ao]

You're incorrect, but this further reveals you can't follow how the
statistics have been manipulated. The scores given in the table are
this:

SPM Follow-UP
Active: 16.59
N-back grp: 16.52
LG grp: 15.67
SG grp: 17.43

If one pays attention here, even my original claim is a very generous
reading as it only largely pertains to the pre-test and post-test
results, not the follow-up results, which are even more alarming.

SG grp did better in FU than any other time, LG grp did hardly any
better in FU, N-back group as a whole did worse at FU, and Active did
its best at FU.

If PNB was to have any lasting effect, one would expect more
discrepant scores for N-back grp (as a whole) compared to the Active
control at the follow-up test. Unsurprisingly, Jaeggi et al. say close
to nil about this - just letting us know in the Table 1 comment that
FU = follow-up test.

argumzio

On Jun 30, 8:25 am, Pontus Granström <lepon...@gmail.com> wrote:
> Your quote:"
> Define "quality of the training". The only difference indicated by the
> study is that highly motivated subjects _tend to do much better_ than

> less motivated subjects and hence are* more intelligent.*"

> ...
>
> read more »

[ao]

Provide quotes, we can't read your mind, Pontus.

argumzio

On Jun 30, 8:41 am, Pontus Granström <lepon...@gmail.com> wrote:
> She also comments on Moody (indirectly) in her discussion part, might be
> worth to read.
>

> On Thu, Jun 30, 2011 at 3:25 PM, Pontus Granström <lepon...@gmail.com>wrote:
>
>
>
>
>
>
>
> > Your quote:"
>
> > Define "quality of the training". The only difference indicated by the
> > study is that highly motivated subjects _tend to do much better_ than

> > less motivated subjects and hence are* more intelligent.*"

>
> > But it was the opposite, those who gained the most had the lowest scores.
> > How can this be? Are not SPM and TONI a measure of intelligence?
>

> ...
>
> read more »

[Pontus Granström]

Large training gain Toni (pre): 18.5
Small Training gain Toni (pre): 22.3

> ...
>
> read more »

[Pontus Granström]

In sum, there is no indication that the dif-
ferential transfer effects were driven by improvements in
processing speed.
The current study has several strengths compared with pre-
vious training research and follows the recommendations of re-
cent critiques of this body of work (6, 36, 37): Specifically, in
contrast to many other studies, we used only one well-specified
training task; thus, the transfer effects are clearly attributable
to training on this particular task. Second, unlike many previous
training studies, we used an active control intervention that
was as engaging to participants as the experimental intervention
and designed to be, on the surface, a plausible cognitive training
task. Finally, we report long-term effects of training, something
rarely included in previous work. Although not robust, there was
a strong trend for long-term effects, which, considering the
complete absence of any continued cognitive training between
posttest and follow-up, is remarkable. However, to achieve stron-
ger long-term effects, it might be that as in physical exercise,
behavior therapy, or learning processes in general, occasional
practice or booster sessions are necessary to maximize retention
(33, 38–41).

> ...
>
> read more »

[ao]

It's the same story as my previous message with TONI. Active did
better at FU than every other group.

argumzio

> ...
>
> read more »

[Pontus Granström]

Of course because they gained Intelligence lol, but at the beginning they were less intelligent.

> ...
>
> read more »

[ao]

Okay, great. But that doesn't make the other issues go away. In fact,
I would say the situation hasn't changed much. Gains cannot be
attributed to training.

argumzio

On Jun 30, 8:46 am, Pontus Granström <lepon...@gmail.com> wrote:
> In sum, there is no indication that the dif-
> ferential transfer effects were driven by improvements in
> processing speed.
> The current study has several strengths compared with pre-
> vious training research and follows the recommendations of re-

> cent *critiques of this body of work (6, 36, 37)*: Specifically, in

> ...
>
> read more »

[ao]

You don't seem to follow the point - unless you're joking. Active did
better at the end of the study. This means, one doesn't need PNB to
increase test scores, but only that they should keep taking tests,
i.e., the retest effect.

argumzio

> ...
>
> read more »

[Pontus Granström]

Does it really mean that? That less intelligent people gain more intelligence to the point that they exceed the levels of a more intelligent group a few months earlier? Hmmm...

> ...
>
> read more »

[Gwern Branwen]

On Thu, Jun 30, 2011 at 9:46 AM, Pontus Granström <lepo...@gmail.com> wrote:
> In sum, there is no indication that the dif-
> ferential transfer effects were driven by improvements in
> processing speed.
> The current study has several strengths compared with pre-
> vious training research and follows the recommendations of re-
> cent critiques of this body of work (6, 36, 37): Specifically, in
> contrast to many other studies, we used only one well-specified
> training task; thus, the transfer effects are clearly attributable
> to training on this particular task. Second, unlike many previous
> training studies, we used an active control intervention that
> was as engaging to participants as the experimental intervention
> and designed to be, on the surface, a plausible cognitive training
> task. Finally, we report long-term effects of training, something
> rarely included in previous work. Although not robust, there was
> a strong trend for long-term effects, which, considering the
> complete absence of any continued cognitive training between
> posttest and follow-up, is remarkable. However, to achieve stron-
> ger long-term effects, it might be that as in physical exercise,
> behavior therapy, or learning processes in general, occasional
> practice or booster sessions are necessary to maximize retention
> (33, 38–41).

That would be interesting. If there had been transfer worth a damn
such that the unspeeded IQ tests showed a rise and refuted Moody's
claim that the previous rises were more processing related than IQ
related.

As it is, the paper is just a Texas sharpshooter fallacy. No doubt if
the random variation had come out the other way, so that the kids with
lower initial TONI scores and lower n-back levels had the higher
post-training scores, we would see claims like 'the harder the task
for the subject, the more transfer we saw and it disproportionately
benefitted those with deficits'. No matter how one slices it, one can
justify it post hoc...

--
gwern
http://www.gwern.net

[Pontus Granström]

You don't think 1SD is worth a damn?

--
gwern
http://www.gwern.net

--

[ao]

Now we're talking about the FU results, not a comparison of pretest
and post-test results.

They obviously weren't less intelligent. From what I see there was no
fundamental difference between the Active control and the N-back group
as a whole.

SPM (pre / post / FU)
Active: 15.33 / 16.2 / 16.59
N-back: 15.44 / 16.94 / 16.52

TONI (pre / post / FU)
Active: 20.87 / 22.5 / 24.81
N-back: 20.41 / 22.03 / 24.07

Yes, I really mean "Active did better at the end of the study [FU]
than any other group", which the data clearly indicates; and the only
exception to that is SG grp SPM, because of mere manipulation of the
data. No, I do not mean what you've misconstrued my words to mean.

argumzio

> ...
>
> read more »

[ao]

Post hoc ergo propter hoc, indeed.

argumzio


On Jun 30, 8:53 am, Gwern Branwen <gwe...@gmail.com> wrote:

[genvirO]

The golden ticket is really only found in the comparison between the
active group and the n-back subjects that demonstrated the most gains.
However, its sad to say that this golden ticket loses its liberty when
one realizes that they had to manipulate the data to obtain such a
glimmer.

SPM SPM SPM TONI TONI TONI
pre post FU pre post FU

14.06 17.19 15.67 18.50 21.56 24.67
15.33 16.20 16.59 20.87 22.50 24.81


The first set of stats are for the n-back group that demonstrated the
most gains.
The second set of stats are for the active group.

[ao]

The LG group seems to indicate nothing as their FU results strongly
indicate regression to the mean.

In Figure 4 on page 4, Jaeggi et al. state that they took the post-
test minus pretest and divided this by the pretest SD to obtain the
standardized gain. This motion is invalid. Imagine what would happen
if they did by using the FU results in the place of the posttest
results. Things are looking grim. (If anyone doesn't believe me,
calculate it themselves. If anyone argues the point, I'll just share
what I've calculated.)

For more information on how to understand pretest and posttest
analysis, I recommend this paper: http://cehd.gmu.edu/assets/docs/faculty_publications/dimitrov/file5.pdf

Each step of the way, one is less and less pollyannaish about the
study.

argumzio

[bofu]

There's truth to differences in volunteers compared to those being
forced into it. Neurogenesis is dependent on voluntary activities.
Read the conclusion of this study. http://www.ncbi.nlm.nih.gov/pubmed/21347437

"Voluntary exercise is the most effective intervention in upregulating
the hippocampal BDNF level, and facilitating motor recovery. Rats that
exercised voluntarily also showed less corticosterone stress response
than other groups. The results also suggested that the forced exercise
group was the least preferred intervention with high stress, low brain
BDNF levels and less motor recovery."

On Jun 30, 9:33 am, genvirO <plastic...@live.com.au> wrote:

[genvirO]

It will be interesting to see if this study receives a negative kick-
back from the scientific community, from people like Moody for
instance. Especially considering this study is making a lot of
promises about the performance of the car without actually giving a
description of what is under the hood. I mean, there's barely even a
disclaimer of some sort! Now that's just asking for a lawsuit from an
unhappy driver/customer! Whether this will be framed in the context of
the academic arena in the form of it tainting someone's reputation is
really one ponder that's an exercise in futility, but nevertheless it
will be interesting to see what does unravel.

For me personally, I don't have a problem with the results, mainly
because it involves children and children are certainly hard to
measure concerning these things, let alone test to see if they improve
on these measures. As well as the fact that I'm not a child,
therefore, tisk-tisk-tisk. My main problem is with the interpretation
of the findings on the researchers part. What happened to ever being
cautious and tentative about these things? Now they just sound like
the guy from the "price is right" where he says "It's a brand new
car", I mean come on! Anyway, I wont rant, just disappointed with the
way they conducted themselves, and would send them to the corner where
they sit on the naughty chair if had the opportunity. Tabooshka
Wamooshka, over and out...

[Pontus Granström]

Exactly, those who gained the most had the lowest IQ scores, contrary to what we have been told by Ao.  Those who had highest average scores gained the least. If I asked kids to do physical training that is dependent on
heart rate and they do not raise it to that level of course you will smaller gains in fitness. This does not prove that calling something training causes increases in stamina, rather that you have to engage to in order to benefit from it.
We have to wait for more studies. It would have been better if they have recruited adult volunteers. If you read her analysis, she said that those who didn't gain any n-levels thought the game was hard and frustrating, while those who improved (lowest IQ) thought the task was challenging. In sum, IQ-level does not benefit your ability to complete n-backing. Haven't you said that max n-level is independent of IQ?

[Green]

Pontus,

You have no idea how many DNB experiments may have failed. When
someone does an experiment and it doesn't work out, it usually doesn't
get printed up in a journal. The positive publication bias can create
the illusion of a consistently positive effect when none exists. This
is part of the reason why you have to be skeptical.

> With the more studies done, and the results being replicated, it gets
> more difficult to suggest that results are invalid. People in the
> thread keep saying "this study only has N=8, and this one, N=25, or N
> = 62. Is it not completely obvious that as you add all of those
> together, the actual sample that has been trained with generally the

> same tasks, and same improvements, has approached one thousand or more
> people, in several different locations and under several different

> conditions. Yet the results are the same, positive, time and again.
> Retorts?
>

[genvirO]

TO NOTE: I have confidence in n-back training, however I think there
are limitations to whom it can be adapted for and the conditions that
suit. I also generally believe that other n-back modes offer greater
benefits for the likes of those that are considered above average, but
this is just an opinion.

I offer no new cars (above post), just a key to lock that you choose
to open (this makes absolute no sense, entertainment purposes only -
ignore it if you wish to save yourself the confusion).

[Green]

I am really having trouble understanding why anyone thinks this was
a positive study.

Below is from the first part of the results section. Unless I'm
reading it completely incorrectly, it says "there was no significant
effect of training on Gf."

However, despite the experimental group’s clear training effect, we
observed no significant group × test session interaction on transfer
to the measures of Gf [group × session (post vs. pre): F(1, 59) < 1;
P = not significant (ns);


Post-hoc analysis is a different story. You can't split the kids
into two groups after the experiment is done and then look for an
effect. Post-hoc analysis is just speculation that raises questions
for future researchers to investigate. It doesn't tell us anything.

[Pontus Granström]

That's just a formality. It would be easy to destroy all research aimed to improve intelligence if all you had to do is to take a group of people who do not engage in the task thus lowering the average and put equal sign between those and those
actively are trying to get better.

[Pontus Granström]

We are talking about training not magical stuff. I appreciate the critical mind though, unfortunately you never apply the same for negative results no matter the shortcomings of those studies, clearly revealing you as anti n-back.

[Gwern Branwen]

On Thu, Jun 30, 2011 at 12:04 PM, Pontus Granström <lepo...@gmail.com> wrote:
> We are talking about training not magical stuff. I appreciate the critical
> mind though, unfortunately you never apply the same for negative results no
> matter the shortcomings of those studies, clearly revealing you as anti
> n-back.

Which negative study does ad hoc splitting of groups in order to
eliminate an increase?

--
gwern
http://www.gwern.net

[Pontus Granström]

I am talking about the training methods, the training time, the difference in task etc. Ad hoc analysis is nothing strange, it's scientifically sound in this case. Are people not allowed to analyze data with respect to certain aspects? Let's just take
a group to interval training, they do not engage in intervals, hence interval training is useless?

--
gwern
http://www.gwern.net

--

[whoisbambam]

I think we can boil everything down to this:

1. we dont know if dnb training improves IQ or fluid intelligence

2. We need more studies, in particular studies with adults. Probably 3
groups:
group1: IQ 90-100, mixed genders/race, either high school dropouts,
high school graduates, or non-collgegiate trade school
group2. IQ 100-110, mixed genders/race, official college degree (can
be trade) associates to bachelors
group3: IQ 110-125, mixed genders/race, bachelors (vigrous degree like
organic chem or something) to PH.D level


Each group should consist of 100ppl in at least 2 groups each.


This study should then be replicated by another group, preferrably
skeptics.


Is this what you all are saying? That the existing evidence is non-
substantive, and that we need better studies?


But that dnb *might* work, but we dont know yet?

[Pontus Granström]

The key point is that cognitive training is a huge program that needs to be "debugged". It will take many many years before we "know anything", and even then we might just realize that we know nothing at all, as Socrates would have put it.
Perhaps I should start to use the philosophical thinking like Ao.

--

[Gwern Branwen]

On Thu, Jun 30, 2011 at 1:29 PM, whoisbambam <a2...@horseracingfirm.com> wrote:
> I think we can boil everything down to this:
>
> 1. we dont know if dnb training improves IQ or fluid intelligence
>
> 2. We need more studies, in particular studies with adults. Probably 3
> groups:
> group1: IQ 90-100, mixed genders/race, either high school dropouts,
> high school graduates, or non-collgegiate trade school
> group2. IQ 100-110, mixed genders/race, official college degree (can
> be trade) associates to bachelors
> group3: IQ 110-125, mixed genders/race, bachelors (vigrous degree like
> organic chem or something) to PH.D level
>
>
> Each group should consist of 100ppl in at least 2 groups each.
>

I don't know if that design would be good enough to advertise
n-backing to the entire world as an investment on par with, say,
iodizing one's salt. Many medical studies which were much larger than
600 subjects have been overturned by later studies. (Seriously, does
no one read my footnotes? My skepticism is on much solider grounds
than anyone here's enthusiasm: http://www.gwern.net/DNB%20FAQ#fn43 )

--
gwern
http://www.gwern.net

[Pontus Granström]

And may I ask what's your reason for believing that IQ-tests measures intelligence?

gwern
http://www.gwern.net

--

[whoisbambam]

Gwern,

Yes.

That would be ONE study that would indeed need to be replicated
repeatedly considering the complexity of the situation.

Making a sweeping recommendation based on 200ppl and 2 studies is not
recommended.


What I am trying to say is that if we DID have TWO independent studies
of groups like i mentioned, that it might provide better insight to
this situation than we currently have?


I guess i am trying to get opinions as to what kind of study group may
validate a given condition or effect or what have you, as the BARE
minimum that may be able to draw preliminary conclusions from.

As it stands now, it looks like what has been done is a freakin' mess
and we shouldnt be drawing any preliminary conclusions whatsoever
other than the studies are currently substantially ineffective and
many fallacies have been brought to light.


True scientific minds should always be very sketpical, as they should
also be 'openminded' or they may overlook something.

In this case, it seems the studies are not up to par even remotely.


i personally dont know enough to draw these conclusions, and instead
am trying to follow the arguments of others here.










On Jun 30, 12:54 pm, Gwern Branwen <gwe...@gmail.com> wrote:

[Gwern Branwen]

On Thu, Jun 30, 2011 at 2:25 PM, whoisbambam <a2...@horseracingfirm.com> wrote:
>
> That would be ONE study that would indeed need to be replicated
> repeatedly considering the complexity of the situation.
>
> Making a sweeping recommendation based on 200ppl and 2 studies is not
> recommended.
>
>
> What I am trying to say is that if we DID have TWO independent studies
> of groups like i mentioned, that it might provide better insight to
> this situation than we currently have?

Well, more studies is usually better (unless there's stuff like
publication bias involved, in which case you can be lead arbitrarily
far away from the truth - imagine a drug company running thousands of
studies and only publishing the p=0.001 hits. With every study you
become more confident that the drug works...)

Meta-analysis is very hard though, so I think it's generally better to
have one large study than a bunch of smaller perhaps incomparable
ones.

--
gwern
http://www.gwern.net

[ao]

People are "allowed to analyze data with respect to certain aspects",
but they aren't allowed to generalize from the analysis and make
invalid claims not leveraged by the full evidence given. Anyone with a
modicum of intellectual integrity understands that much.

argumzio

On Jun 30, 11:22 am, Pontus Granström <lepon...@gmail.com> wrote:
> I am talking about the training methods, the training time, the difference
> in task etc. Ad hoc analysis is nothing strange, it's scientifically sound
> in this case. Are people not allowed to analyze data with respect to certain
> aspects? Let's just take
> a group to interval training, they do not engage in intervals, hence
> interval training is useless?
>
>
>
>
>
>
>
> On Thu, Jun 30, 2011 at 6:18 PM, Gwern Branwen <gwe...@gmail.com> wrote:

> > On Thu, Jun 30, 2011 at 12:04 PM, Pontus Granström <lepon...@gmail.com>

[ao]

1. In terms of the evidence objectively published: correct.

2. No, we need BETTER studies, that is, those that aren't undermined
by threats to internal validity, bias from well-known publishers of
previously criticized results, or misuse of statistical data.

argumzio

[The.Fourth.Deviation.]

A hasty reading barely counts as stupidity; you must be daft to even
suggest such a thing. The paper presents various questions about
research, but in my eye, those are easily answered by my powerpoint.
The paper says results are "inconclusive". But brain scans are
conclusive. Hence, positive, yet inconclusive results, back by brain
scans which show positive results, obviously tip the scale in favor
of
DNB proponents.

If you were not so bull-headed, which surely detracts from your
intellect as it prevents you from presenting data that have been
shoved in your face several times, you would see that the powerpoint
is a summary of several different papers. Initially you say that the
number of subjects was too small. When it was shown that there were
further studies in the powerpoint, with larger sample sizes, you
quickly attempted to divert scrutiny to another topic, ostensibly to
spare your legendary, and undeserved, ego from wounds.


And of course combination N back is not mentioned in the paper. The
name was created for brain workshop, why would it be used by
professional researchers? You must be daft. Half a brain suffices to
make the connection between the combination task and the one
mentioned
in the paper. In fact, the combination task is more difficult because
it includes more interference.


Next, a few more brain cells would have told you that if an N back
task with interference shows evidence of significant transfer, an n
back task with more complex interference should as well. Not only
this, but another study, which I posted in the combination N back
thread, shows that this task can improve processing speed, which is a
major factor in intellect. Hence there is a growing mound of evidence
that combination mode should replace standard DNB as a staple of
training due to its increased complexity.


In sum we have: studies inconclusive, studies positive, and MRI's -
positive. The critique paper posted by argumzio is rather blunted by
brain scans of trained subjects. In other words, there are *some*
questions, but they do not pose any substantial threat to whether DNB
causes neurological adaptations.


On a side note, on the way out of the gym today, I saw Argumzio
outside warning people to be "skeptical of strenuous exercise because
it remains uncertain whether statistically significant gains in
cardiovascular and muscular strength noted by trainees were a result
of opening the gym's front door and operating its water fountain."

> > > > > As for the other individual requesting brain scans, I posted a
> > > > > powerpoint elsewhere that had about a dozen different brain scan
> > > > > studies all showing adaptations in response to WM training.
>
> > > > > Unfortunately for critics, brain scans are substantial enough to prove
> > > > > that WM training causes adaptations. Not only this, but also studies
> > > > > that show cortical efficiency with WM tasks improves with WM training.
> > > > > You can't "fake" increased cortical activity. It's simple fact.
>
> > > > > In addition, WM training can cause a cornucopia of other benefits,
> > > > > such as improving short term memory, with applications in the real
> > > > > world that might not be measured by these tests. E.G. today at my job,
> > > > > after training N back for about 2 weeks straight, I was able to recall
> > > > > various instructions with 100% accuracy due to improved recall from
> > > > > DNB training. Hence I would say I am "more" intelligent. Therefore DNB
> > > > > has helped me perform unrelated tasks.
>
> > > > > The critics present multitudinous arguments. Unfortunately, none of
> > > > > them are convincing, especially in the face of the research and
> > > > > anecdotes. I can't wait to see how the funny skeptics try to explain
> > > > > away increased cortical efficiency shown in brain scans (last time I
> > > > > posted a powerpoint of several studies on this, Argumzio simply
> > > > > avoided responding by saying he didn't have time to look over the
> > > > > result of so many studies, while complaining that "the brain does not
> > > > > exist". Nor was anyone else able to explain away physical changes in
> > > > > the brain in his stead)
>
> > > > > On Jun 29, 7:07 pm, ao <argum...@gmail.com> wrote:
>
> > > > > > whoisbambam, if it
>
> ...
>
> read more »- Hide quoted text -
>
> - Show quoted text -

[exigentsky]

Just a quick point but I talked to Toomin about integrating some sort
of Stroop interference task to BW and he dismissed it saying that
interference tasks don't appear to improve intellect.

Here's the crux of my proposal: "One DNB mode that could also help
with this would include distractors - a little like the Stroop task.
One implementation could have 5-7 distractors per session and it could
be represented as a differently colored square lighting up. Both the
auditory and visual stimuli should be ignored when this is present."

[T. Lavon Lawrence]

Are you guys not going to be satisfied until you're walking around with these great big huge noggins that you need a cart to haul for assistance?

Jeez!

Personally, I say do it while juggling with increasing numbers of objects varying from small round balls and bowling pins to active chain saws.

Top THAT, mister!

--
You received this message because you are subscribed to the Google Groups "Dual N-Back, Brain Training & Intelligence" group.
To post to this group, send email to brain-t...@googlegroups.com.
To unsubscribe from this group, send email to brain-trainin...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/brain-training?hl=en.

--
T. Lavon Lawrence
Author, NEURO-SCULPTING!© Brain Fitness System
Mental Fitness Trainer & Training Author
tra...@neuro-sculpting.com
www.neuro-sculpting.com

[D. K. Ohms]

When will this moron shut up already? All he's done is attack
contributors here when it suits his interest. 100 PIQ goes in circles.

Ohms

[brain train]

Cool down guys!

 

you may better spend this time on some other more useful activity-- may be DNB or something else, depending on your opinion/choice.

 

we need critics as well as supporters.. to have debate to arrive at high quality conclusions.

but don't burn your emotional energy- preserve it for better use elsewhere.. in more personal settings.

 

(Personally, i value the opinions of both of you guys... you bring so much valuable info to the platform, for all to see)

 

braintrain

[Pontus Granström]

Some people here like to think of themselves as critical and objective, while I argue that they are biased. Being critical is a good thing, it is of course needed. I wished however that one when
being critical also like a "true man" recognizes what we do not know. We do not know much about what underlies performance on matrix tests, we haven't conducted research on all the subtle
aspects of matrix reasoning. For example the article I posted that showed that RAPM was spatial (which probably explain why I score lower on that compared to pure reasoning) and required things like rapid eye movement (n-backing) and other criticism against the test it is ignored. I have posted other articles the tends to show that verbalizing RAPM seem to improve the performance. It might be that
Raven never though about the rather subtle cognitive processes required to solve his puzzles when he created them 1938. This is rarely discussed, because "RAPM score" can't be improved.

 For example, studies negative to brain training are often accepted without any delay. Because that's the way it should be or rather the way "we" want it to.
Although I would claim that their methods are not comparable with the n-backing regime. Such vital aspects of cognitive training such as duration, task and frequency seems for some reason to be unimportant. As you know from physical training, it is the objective load rather than the "perceived" impression of training that is important.

Another point, if I ask kids to do something, many of them might not complete the task. In this case a demanding working memory game. This does not mean that n-backing when done "properly" does not improve your matrix reasoning ability. Let's say that she had a group that consisted of all motivated
people, then you would argue that it was just luck that she had a group that develops so quickly because they are intelligent or whatever and not due to n-backing. Data is always open to analysis,
we can't really say anything except that those who progressed in n-backing seemed to improve their IQ, it's just as valid and statistically more significant than any other explanation, of course
not 100% certain.

I have also posted other studies done on children that seem to improve their IQ, they have also been ignored. What to make of it? That some here are in search of an objective truth and that
they are not emotionally biased against n-backing for whatever reason? Hardly not.

[brain train]

let us test it ourselves.

do the following simple exercise to quickly benchmark your attention levels/brain performance:

1. count from 1 to 120 aloud as fast as you can and note down the time in seconds.

2. do the stroop test and note the time taken.

 

3. Repeat it after 3 weeks training on dnb (or whatever form of n back training you are doing).

compare the results and see for yourself.

4. share the results with the group (if you are comfortable with that) or results may be collected anonymously.

 

it may serve as direct , first hand proof of effectiveness if the results show improvements.

 

Morevover, moderator may request the new joinees of the group, at the time of including them in the group, to benchmark themselves with these and/or other exercises (like some free or paid IQ test).

 

braintrain

[Pontus Granström]

My "dream" is that they did BIP-testing. It's free, it's easy and it's fast.

[Pontus Granström]

Functional correlations between the BOLD signal obtained from the working memory task (n-back) and intelligence factors were mostly detected in the right prefrontal and bilateral parietal cortices. All these correlations were negative indicating that subjects with high intelligence factor scores had less activation during the n-back memory task, in support of the efficiency model of brain function ([Haier, 1993], [Haier et al., 1988] and [Neubauer and Fink, 2009]). Gray et al (2003) reported that, on more demanding n-back conditions, participants with higher intelligence scores were more accurate and showed greater activity in several frontal and parietal regions. It should be noted that the focus of their analysis was based on an event related design but their report also included the results using a block design which showed a trend of lower activity with higher scores on a single test of fluid intelligence. Waiter et al. (2009) did not find significant correlations between individual differences in brain activity during an n-back task and intelligence scores. Activation levels are known to fluctuate across working memory loads in an inverted-U shape response (Callicott et al., 1999). The position of the inverted-U can shift depending on the working memory capacity of the group or individual (Callicott et al., 2003). Waiter et al. (2009) used a simple version of the n-back task with only 0- and 2-back levels . The limited levels and range of task difficulties in their study may have created confounds related to shifts in the inverted-U curve. In addition, their study focused on elderly (mid to late 60-years-old) subjects, a population that shows a decrease in working memory capacity and related neurophysiology (Mattay et al., 2006) as well as a wide variability among individuals in the extent, rate and pattern of age-related changes that are exhibited at both neural and behavioral levels (Hedden & Gabrieli, 2004). For the current study, a greater range of working memory task was used on a younger population, which could help avoid inconsistencies due to shifts of the inverted-U. This method does not completely avoid the potential anomalies as equal sampling of both sides of the inverted-U is not guaranteed but is less risky than sampling only one level of working memory difficulty. The optimal method would include enough levels and range in the task difficulty to determine the point of peak activation for each subject.

Our n-back results are in agreement with the single published study aimed at quantifying the neuro-anatomical overlap between the general factor of intelligence (g) and working memory capacity (Colom, Jung, & Haier, 2007). That study showed that a common neuro-anatomic framework for these constructs implicates mainly frontal gray matter regions belonging to the right superior frontal gyrus, the left middle frontal gyrus, and the right inferior parietal lobule. These findings (a) were thought to support the role of a discrete parieto-frontal network, as proposed by the P-FIT model, and (b) were consistent with Cowan's (2005) theory which distinguished a capacity limit (related to parietal regions) and the control of attention (related to frontal areas). It was suggested that capacity limits and attention control relate to the commonality between intelligence and working memory. We also note that we found no correlations between our memory factor and any fMRI activations, possibly because the factor was derived as a broader assessment of memory than the more focused processes required for the n-back task, although this is not determined.

[Green]

I have read that training-related improvement at the Stroop task
doesn't transfer to other Stroop-like tasks.

But, there is some evidence that certain kinds of interference
training can lead to transferable gains. http://www.ncbi.nlm.nih.gov/pubmed/18947353

[ao]

Has it ever occurred to you that "negative" studies are generally
accepted because of their merits, not because they're simply negative?

Besides, there have been many studies involving different
interventions that showed increased scores on various tests, perhaps
significant to some degree, e.g., take an intervention like education,
which has been shown to lead to increased scores. There is nothing
remarkable about this. However, they have _nothing_ to do with the
research involving n-back, so they qualify as a false analogy on your
part.

None of this amounts to an "emotional bias", which is a nonsensical
straw man argument due to your inability to comprehend others'
statements here. In fact, the people in this thread who show the most
emotion in their posts (and an obvious lack of ability or reading
comprehension as shown by their false claims about what papers
actually say) are those who are so bent on twisting data or evidence
to bolster a case in favor of n-back that is currently not so strong
as they make it out to be. (Even fMRI studies cannot be used as a
serious source of generalization regarding cognitive enhancement
allegedly afforded by n-back training, even if there is a supposed
overlap with g-related brain areas, because any number of
interventions signal changes therein but not lead to significantly
altered scores. But these crazed proponents with their severe
retardation and unfamiliarity with scientific praxis or progress on
the empirical front seem to think otherwise.) One is equally amused
and annoyed by their gesticulations.

The situation is more suggestive than definitive apropos the current
research. Who denies this automatically brings their own integrity
into doubt.

You seem to acknowledge the weakness of the research when you say, "We
do not know much about what underlies performance on matrix tests...",
but fail to make this connection to the results on studies involving n-
back. Who is the "true man" when your left hand doesn't know what your
right is doing?

argumzio

> ...
>
> read more »