Another debate between BRAD and Seligman, with BRAD the victor
Cognitee
Quoting Martin Seligman, from a mailing list:
"Subject: Qualifications of Therapists and Outcomes
Neil Jacobson claimed that the CR study "confirmed what
we already knew" that increasing levels of experience, skill, and
education do not make for better therapy. The CR study did nothing
of the sort, nor do "we already know it" from elsewhere. This is a
mischievous claim in today's marketplace. Profit-driven health care
schemes will send patients to the cheapest provider they can find--
particularly if scholarship can be invoked to justify the idea that less
qualified providers (less experience and less education) make for
just as good outcomes.
The CR dataset was conflicting on this issue. Social workers
(presumably mainly MSW's, although degree was not queried) did
as well as doctoral level providers. Marriage counselors
(presumably mainly MFC's, MFT's, and other nondoctoral
providers--although degree was not queried), on the other hand,
did significantly worse. This was not an artifact of the fact that
marriage counselors see couples in troubled relationships. CR
compared doctoral level providers treating marital and sexual
problems to marriage counselors treating the same problems, and
the doctoral level providers did significantly better. Here are some
relevant data: The marriage counselors (n=197) seeing
respondents who checked "marital or sexual problems" had
significantly poorer global outcome than other therapists (n=753)
with these patients (p<.0017, controlling for initial severity).
The argument for equivalence derives from Christensen and
Jacobson (1993) and their argument is seriously flawed. For they
review studies, some badly flawed, in which manuals are used,
mild and uncomplicated clinical problems are the diagnosis (but the
diagnosis is made by doctoral level providers), and duration of
therapy is brief and fixed. Precisely the situations in which clinical
judgment, experience, and education matter very little. A seven-
year-old may be able to fly a one-engine plane in clear weather, but
this does not mean she can handle a 747 in a thunderstorm.
The disinterested conclusion should be:
1) If a case is simple, if a manual must be followed, if the
diagnosis is made by a more experienced and better trained
clinician, and if treatment must be very brief, less experienced and
less well-educated providers may do as well as doctoral level
specialists.
2) It seems likely that in real therapy, where cases are complicated
and more severe, no manuals are used, diagnosis as well as
therapy must be done, and clinical judgment is important, that more
education and more experience of providers will improve outcomes.
3) Effectiveness studies of level of education and experience of
providers and the cost-benefit analysis are urgently needed.
Instead MCO's use this state of affairs to justify using less
experienced and less well-trained providers even in complicated
and severe cases. Again, I believe, patients are being deprived of
adequately skilled treatment on a massive scale. Until this issue is
resolved by the appropriate effectiveness study, I recommend that
MCO's err on the conservative side and provide experienced and
highly educated providers in all but the simplest and least severe
cases."
---------------------------------------------
BRAD REPLIES (with the truth):
Dear Martin Seligman,
I do not believe that it is so much a worry that clients are deprived
the "adequately skilled" treatment (that is in some way is supposedly
associated with "highly educated providers") AS MUCH AS we should be
concerned that they are denied (or largely denied) access to treatment
or
reasonable treatment IN GENERAL NEARLY ALTOGETHER. We need a mental
health
care SYSTEM with a variety of helpers, as may well come to be seen as
rational and justified (IF ONLY THE PREEMPTIVE "WORRIES" --AKA
PROVINCIALISM AND POLITICAL POWER-- OF CLINICIANS DOES NOT GET IN THE
WAY
OF DOING THE MUCH EASY, BASIC RESEARCH STILL NEEDED HERE). Do you have
any
data to the contrary? Apparently not.
RATHER THAN "clinical judgement" being education-related, it is just
as
likely related to the good adaptation and communication of the helper
(at
this point in much of our knowledge). AND, much of this may be a product
of
good develpment and perspective-taking abilities plus RELATED abilities
to
modulate emotional reactions. SELECTION OF GOOD CANDIDATES MAY WELL BE
AS
IMPORTANT AS TRAINING FOR SEVERAL FUNCTIONS (with training necessary,
but
not of the "level" you seem to insist on). I simply do not believe that
counselors (yes counselors, doing primarily counseling and not
"therapy")
apply but the simplest principles of psychology (OFTENTIMES) in any
clear
or rational way. (Appeals to scary cases of possible suicides aside.)
--
b jesness
P.S. *STILL*, ANYHOW, TO WORK ON YOUR WAVELENGTH: Ever heard of anyone
going into physics without taking physics in college? YET, a similar
thing
happens everyday in schools of clinical psychology. One must ask: How
much is pretense?
---------------------------------------------
P.S. On the problem of "highly trained" "therapists" doing
*everything*:
I have indicated that this arguably denies reasonable access to
treatment
for many. It possibly denies the "middle class" as much as the poor,
given
fee schedules -- though the poor also do not get the comfortable easy
access, the amount, or the variety of help needed.
But another problem is that we are doing little to clarify the cases
where special interventions are necessary. HERE AGAIN PROFESSIONAL
PSYCHOLOGY IS HOLDING UP EFFORTS, hurting and not helping. While these
interventions may not be well-defined or exist as standard (or as known
or
recognized) treatments at the present time, we are making little
progress
as "high level clinicians" diffuse there efforts on VIRTUALLY ALL CASES.
Signs of this are: There is relatively little specialization in the
field.
Likewise, relatedly, there is little true (scientifically meaningful)
science-practitioner work being done, where clinicians work closely
together developing and showing better inter-rater reliablilities. The
best example of a result of this short-coming is the very very little
work
on improving inter-rater reliabilities on diagnoses between the
publication
of the DSM-III and DSM-IV. It is as if we rely on the DSM committee for
all this work. Most reliablilty tests are done AFTER-THE-FACT, AFTER
new
options for diagnoses have been established (once a decade or so). The
research *they* examined IN ARRIVING AT THE SET OF DSM-IV DIAGNOSTIC
OPTIONS, i.e. the research they had to examine, contained *VERY,VERY*
little work on obtaining inter-rater reliabilities. THIS MAKES NO SENSE
FROM A GOOD-SCIENCE PERSPECTIVE. (For the source on this see the
Sourcebook for the DSM-IV, VOLUME 2.) Establishing better inter-rater
reliabilities for diagnoses should be an everyday activity.
The need for the field to BECOME a science is clear. the time for
pretending is over. -- b jesness
Cognitee
Indeed BRAD's supporter in another thread must have been familiar with
this incisive discourse !
Cognitee
Dear Ed,
I note your tendency to see "flaws" in the research only when it
lends more credence to your views. Much of the research you rely on
(e.g. efficacy studies) is SIMILARLY "FLAWED". Nothing like displaying
your biased bugger side. -- b jesness
Leslie E. Packer, PhD
On Jul 22, 1996 05:11:28 in article <Re: Another debate between BRAD and
Seligman, with BRAD the victor>, 'Ed Anderson <eand...@cougarnet.byu.edu>'
wrote:
>This also supports Leslie's arguments that research re. PhDs and masters
>is seriously flawed (which I'd been unaware of):
--
Hi Ed...
Yes, I agree with Martin Seligman's comments that the research is seriously
flawed, and could cite even more studies to show you the really fundamental
design flaws that even my undergraduates could have picked up on (well, I
could cite them if my daughter hadn't trashed the hdd.... for now, I'm just
too lazy to go run the whole search/analyses again). The most interesting
thing in reading this particular thread is the absence of any substantive
evidence of equivalence from those who criticize Seligman's main point.
There is a helluva difference between "MAY" (shouted at us) and
demonstration of "is."
Regards,
Leslie
----
Leslie E. Packer, PhD
** Gotta run.... the cat's caught in the printer....
Ed Anderson
Cognitee <Cogn...@aol.com> wrote:
>Dear Ed,
> I note your tendency to see "flaws" in the research only when it
Then you haven't been paying attention to the discussion Leslie and I
were having.
ed
Cognitee
Dear Ed,
You say: "This lends support to Leslie's claim that PhDs do better
(in some cases)
than master's level therapists. "
I say: No, not really. The issue is so confounded in the comparison
Seligman cites (between marriage counselors and others) that it **is**
meaningless. Who knows what all the counfounds might be.
Also note: (quoting Seligman): " Marriage counselors
>(presumably mainly MFC's, MFT's, and other nondoctoral
t biased Seligman should realize it. A you student like you, Ed should know much better.
All the best evidence does indicate that experience (or staying in the field) is somewhat related to effectiveness (maybe marriag=
e counselors hang around less long and end up having less experience on average). BUT (in any case) nothing in the research well a=
ddresses the issue of training. Also we have virtually no evidence on WHAT TRAINING is *really* need for effectiveness for MOST cli=
ents. The research on peer counselors and paraprofessionals (not confounded with the experience factor) IS POSITIVE. -- b jesness
P.S. Poor Ed, the mistaken "student"
Ed Anderson
This also supports Leslie's arguments that research re. PhDs and masters
is seriously flawed (which I'd been unaware of):
>Quoting Martin Seligman, from a mailing list:
>....The argument for equivalence derives from Christensen and
>Jacobson (1993) and their argument is seriously flawed. For they
>review studies, some badly flawed, in which manuals are used,
>mild and uncomplicated clinical problems are the diagnosis (but the
>diagnosis is made by doctoral level providers), and duration of
>therapy is brief and fixed. Precisely the situations in which clinical
>judgment, experience, and education matter very little. A seven-
>year-old may be able to fly a one-engine plane in clear weather, but
>this does not mean she can handle a 747 in a thunderstorm....
An apt analogy. Lord knows I've been in a number of thunderstorms.
ed
Cognitee
SAME THING, BETTER COPY:
Dear Ed,
You say: "This lends support to Leslie's claim that PhDs do better
(in some cases)
than master's level therapists. "
I say: No, not really. The issue is so confounded in the comparison
Seligman cites (between marriage counselors and others) that it **is**
meaningless. Who knows what all the counfounds might be.
Also note: (quoting Seligman): " Marriage counselors (presumably
mainly MFC's, MFT's, and other nondoctoral providers--although degree
was not queried),..." (end quote). Presumably! (why so presumed?) This
is such bullshit, even that most biased Seligman should realize it. A
you student like you, Ed should know much better.
All the best evidence does indicate that experience (or staying in
the field) is somewhat related to effectiveness (maybe marriage
counselors hang around less long and end up having less experience on
average). BUT (in any case) nothing in the research well addresses the
issue of training. Also we have virtually no evidence on WHAT TRAINING
is *really* need for effectiveness for MOST clients. The research on
Cognitee
Dear Leslie Packer.Ph.D.
Regarding the demonstration you asked for: The demonstration of a
lack of evidence is easy. Here it is:
[did you see it? ; OF COURSE NOT , SILLY !)
regards, b jesness
P.S. On the other matter: Needless flaws exist in virtually all
"therapy" efficacy research (as well as in the research you *are
willing* to criticize) because of the unethical cowardice of the field.
E.G.: No reasonable controls, nor any test of therapists against
reasonable alternatives. A sham and an disgrace. I will keep saying it
until it simply makes sense. I shall break through the denial just as
one would with a classic alcoholic (though now it is the power-addicted
we are dealing with). Have a good day, Leslie.
Ed Anderson
>....The CR dataset was conflicting on this issue. Social workers
>(presumably mainly MSW's, although degree was not queried) did
>as well as doctoral level providers. Marriage counselors
>(presumably mainly MFC's, MFT's, and other nondoctoral
>providers--although degree was not queried), on the other hand,
>did significantly worse. This was not an artifact of the fact that
>marriage counselors see couples in troubled relationships. CR
>compared doctoral level providers treating marital and sexual
>problems to marriage counselors treating the same problems, and
>the doctoral level providers did significantly better. Here are some
>relevant data: The marriage counselors (n=197) seeing
>respondents who checked "marital or sexual problems" had
>significantly poorer global outcome than other therapists (n=753)
>with these patients (p<.0017, controlling for initial severity)....
This lends support to Leslie's claim that PhDs do better (in some cases)
than master's level therapists. And, earlier, I stated that the finding
that MFTs etc. did worse than PhDs did not control for the fact that
couples are harder to treat than individuals; I was mistaken about that.
ed
Cognitee
P.S. Especially those who engage in denial would not believe in it.
Cognitee
Dear Leslie,
The Stein and Lambert "meta-analysis" (1995) is supposedly the most
recent assessment of the data on the matter of the "equivalence" or
"non-equivalence" of professionals vs. paraprofessionals, something
close to the issue you are addressing with me. It is bogus in many
ways, but probably has a fairly good bilbliography (including citations
of other more respectable meta-analyses).
I have been itching to again provide some review of this travesty,
that so well illustrates how standards of publication bend when it suits
the vested interests of the APA (political power and unwarranted
exclusive control). It is very close to pure propaganda, yet was
accepted for publication in a APA refereed journal:
The recent Stein and Lambert "meta-analysis", in the Spring, 1995
Jour. of Consulting and Clinical Psyc., is a professional disgrace.
Most studies cited were not on the issue they were trying to address in
2 senses. The issue they were trying to get at was the effectiveness of
trained professional counselors (or "therapists") VS.
"paraprofessionals." The first big problem is that MOST of the studies
they examined (by far) AND INCLUDED in the "meta-analysis" involved
comparing EXPERIENCED professionals to
INTERNS or counselors-in-practicum. These are not studies of
professionals vs. "paras" IN ANY SENSE, EVEN WITH THE EXPERIENCE
CONFOUND ASIDE. Second, the vast majority of the studies
also had this issue of trained vs. less trained (or whatever) as an
issue on the side (often something just examined in passing); i.e. most
of the studies were not really studies of the question being examined NO
MATTER HOW POORLY YOU DEFINE "PARAS." Finally there were only 3 studies
that were both GOOD by normal standards and used measures that could be
considered objective. These were 2 in favor of the "paras" and one
showed a tie. In both the 2 best studies, actually looking at what
rational people would call "paras" and having this as the actual focus
(purpose) of the study, I believe the "paras" faired better. I shall
elaborate more below on the great likelihood of confounds.
The authors' conclusions from their "meta-analysis," which were
actually contrary to the facts when looked at the way I did above are
inexplicable. I can conclude only that these researchers are totally
incompetent AND/OR this was an effort at subterfuge. For a more
detailed look at the study, read on.
Below is a compliation of past postings I have made about this
"meta-analysis" last summer; because it is a compliation it is
somewhat redundant:
I would like to address the recent Stein and Lambert study in the
Jour. of Consulting and Clinical Psyc., Spring, 1995. They concluded in
this "meta-analysis" that grad. trained therapists yield modestly better
results in outcome measures from clients than paraprofessionals.
Confounds are a big issue and very much so here as I shall describe
below. Furthermore, not only is there likelihood of serious confounds
that abound but no rating system for study quality was involved in their
meta-analysis. Often the better studies (including previous
meta-analyses) indicated results contrary to what they reported for
overall conclusions.
In this Stein and Lambert review ("meta-analysis"): Even in the
SELECT group of studies that had objective measures and supposedly did
show some effect for more TRAINED vs. LESS TRAINED, it is good to give
people a realistic and meaningful idea of the magnitude of those
differences found. On objective outcome measures (objective exit
"tests") where differences were found, the "effect size" was .2 (once
the 1 outlier study of the 10 was thrown out as the authors suggest).
THIS MEANS a one fifth of a standard deviation difference on average
between the groups (please see the "FOOTNOTE" AT THE END)**. (TO MAKE
THIS MEANINGFUL: This is *less than half* [(note corrected statistic in
this new edition of the review)] the level of difference shown between
males and females (where males and females differ at all) on several
objectively measured interpersonal traits. IN THIS SPHERE this level
of difference is NOT considered impressive (certainly it is not
considered differentiating); .2 s.d. is at most about a quarter of the
difference between males and females shown on conglomerate scales set up
to differentiate them.)
Another thing to note w/r to these 9 studies from Stein and Lambert I
referred to above: with an s.d. of the groups at .31, 1 or 2 of these 9
studies likely showed the "paras" doing better (i.e. due to variability
in results amongst the 9 studies -- and recall this is the select group
of studies that showed more than the typical outcome difference). AND I
must add that in these studies showing a difference with grad. training:
These differences could very well be due to confounding factors (BIG
ONES): perceived status of therapist, age of therapist, experience (a
matter different from training) and OTHERS! None of this was
controlled. The only controlled study w/o confounds showed untrained
listeners superior to therapists for a BROAD RANGE of college student
problems.
Also remember these are group data and with just a .2 of a standard
deviation difference between the grad. trained and the "paras (their
definition)," a sizeable number of the "paras" (in EACH of the nine
studies) were doing better than the professionals on average (act. just
slightly less true than the other way around).
Because some personal uncertainty still remained for me with respect
to these studies: I went back and looked at all the particular
studies where objective measures were involved. In 6 out of nine, the
comparison was actually between late stage grad. students (in practicum
or interns) vs. EXPERIENCED degreed professions. Obviously this is not
the comparison either I or S & L were supposedly out to address with
this study (again, we are really not looking at training, but
experience). The 3 studies that remained using objective, typical
psychological measures of symptom change found no difference paras vs
profs. in 2 cases AND 1 study favoring the paras.
Furthermore, INDEED it is still true today that the best research
available on professional psychologists vs. others, and the only
controlled study, is one that compared professionals to untrained
individuals. This is the Strupp and Hadley study, 1979. And here, it
was shown that intelligent good-listeners could help college students
with a broad range of problems at least as well as professional
psychologists (Strupp and Hadley, 1979). It is completely unacceptable
that a study comparing professionals to "paras" without major confounds
has not been done. It is worse than if a drug company did not do
placebo studies. Worse because we do not know that "paras" would
materially or substantially provide anything different than grad.
trained psychologists.
In spite of the grave deficiencies, weaknesses of the studies
presented, and unjustified conlusions of the authors, still this S & L
"meta-analysis" has been heralded and is the study on the basis of which
John Grohol and others have argued that there is a modest difference in
outcomes with clients when grad. trained therapists and
paraprofessionals are compared. Readers can see for themselves after
going to the source (Stein and Lambert in Jour. of Couns. and Clin.
Psyc., Spring, 1995) that what I have been able to say about this study
is true and it is a mess. This is a much worse than usual meta-analysis
(many are very, very good and useful). Most meta-analyses are summaries
of studies that were on the actual matter of concern. Again, here in
the typical study included in this meta-analysis, the primary focus of
the study was not the question at hand (not even: the effectiveness of
more trained VS. less trained) but this was either a secondary
hypothesis of the study OR results "that were almost presented as an
aside." Again, in fact only 1 study included in the whole report had a
similar primary focus (still not identical to the question at hand) and
was controlled for confounds (this is the old Strupp and Hadley study,
'79; NO MORE RECENT ONES HAVE BEEN DONE). This study showed no
difference in counseling outcomes between trained psychologists and
totally untrained "nice guy" professors doing counseling with college
students with a BROAD RANGE of problems. To quote the authors of the
study itself on these matters:
"Readers familiar with the outcome research in this area are aware
that authors typically did not design their studies to primarily
investigate the effects of therapist training or experience. Indeed,
examining the relationship between training or experience and outcome
was usually a secondary hypothesis, or results were presented almost as
an aside. Thus, it appears that the investigation of the relationship
between level of training and outcome was not planned as carefully as
procedures designed to study the central hypothesis. For example, as
noted earlier in the article, the typical study did not adequately
isolate the issue of professional training from confounding variables.
We are aware of only one published study that has quite reasonably
isolated the ingredient of therapist training by controlling some of its
inherent confounds and correlates (e.g. age, status, perceived
expertness, interpersonal skills, etc.). This was the Strupp and
Hadley's (1979) classic study comparing male university professors, who
were selected because of their reputation among students as being
approachable and easy to talk to, and experienced male
psychotherapists."
It appears that while the issue of fully grad. trained therapists
vs, "paras" may not be a new one, THE RESEARCH WOULD BE. IT HAS NOT YET
BEEN DONE!! Also it is VERY likely that a number of the major confounds
I cited as possibly present probably were (and some I didn't think of).
Some (I won't say "a lot") of good and reasonable work is yet to be done
before we have a clue as to whether well-selected, reasonably-trained
and supervised "paras" do as well as clinical and counseling
psychologists with the terminal degree with the majority of clients.
The best evidence we have indicates that in general they will probably
do as well. There is some suggestive evidence from other studies cited
by Stein and Lambert that fully trained therapists may be helpful with
diagnoses and that dropout rates with "paras" become higher only when
more than 10 or 15 counseling sessions are required. This is suggestive
for the role of a new type of supervising clinical psychologist. Still
the full merit of reasonably selected and trained "paras" has not been
addressed. One wonders whether a major research issue will ever be
addressed when it is not in the vested interests of therapists. It is
surely overdo.
How these authors concluded in this "meta-analysis" that grad.
trained therapists yield modestly better results in outcome measures on
clients than paraprofessionals is very hard or impossible to understand
on a reasonable basis. As I indicated, bad definitions, poor measures,
and confounds are a big issue (and very much so here). As I indicated,
the likelihood of confounds abound; and no rating system for study
quality was involved in their meta-analysis. Often the better studies
(including previous meta-analyses) indicated results contrary to what
they reported for overall conclusions.
** FOOTNOTE:
The "effect size" in the Stein and Lambert research for the studies
using objective measures was defined as follows: ((mean of more highly
trained ON THE OBJECTIVE MEASURE)-(mean of less trained ON THE OBJECTIVE
MEASURE)) divided by the STANDARD DEVIATION OF THE LESS TRAINED ON THE
MEASURE. I tried to gage the magnitude of this effect in meaningful
terms by assuming the s.d. shown by the "experimental" ("para") group
would be about that of the general population. (It is actually likely
LESS thus I'M INFLATING the actual "effect" the way I represent things,
though this is not certain). Anyhow, assuming s.d. of the para group
equal to the general population on the measures of symptoms involved, I
argued tha .2 standard deviation difference (more trained vs less) would
not typically be consider meaningful on the measures.
I do admit that it may be too much of an assumption to assume that
the standard deviation shown by subjects in the control group (para
group) is equal in magnitude to the standard deviation of the general
population on the outcome measures. Yet, again, if anything a measure
of range shown by a select group on a pertinent measure is at least
typically (though not invariably) smaller than that in the general
population AND would come out that way here, I think, unless the shift
in therapy for some was dramatic, while for others none or for the
worse. Typically they would start AND (equally treated) END more
similar to one another than those in the general population. With this
in mind, as I've said, the real denominator in the formula for "effect
size" as they defined it would be even less than the s.d. of the
population sample on the test. And, this would mean the difference in
outcome measures is likely less than .2 s.d. of the measure (generally
speaking), that is, when looking at what this would mean in the actual
general population, on which the s.d. of the measure is based. This
would make the difference between paras and profs. even less meaningful.
The only thing to argue against this is if you can find out that there
was greater variability in the client sample than in the general
population.
---------------------
Cognitee
Dear Leslie,
Regarding equivalence: Strupp and Hadley (1979) was the last of 3
controlled studies done. All indicate equivalence. Allen Ivey's read
of the literature agrees with me. Close examination of the unconfounded
and relevant studies in Stein and Lambert (1995) also shows my view is
correct (all 3 controlled studies I noted are noted there). (Disregard
Stein and Lambert's umnjustified conclusions in their report. Stein and
Lambert make conclusions without regard for confounds of experience and
training; also training differences are very small -- Ph.D. candidates
vs. practicing clinicians -- while experience diferrences (overlooked)
are large.) Thanks. Have fun getting real ! -- b jesness
Leslie E. Packer, PhD
On Jul 23, 1996 15:42:32 in article <Re: Another debate between BRAD and
Seligman, with BRAD the victor>, 'Cognitee <Cogn...@aol.com>' wrote:
>P.S. Especially those who engage in denial would not believe in it.
Sure... but so will those who know that they don't.
(Isn't this fun?)
Leslie E. Packer, PhD
Leslie E. Packer, PhD
On Jul 23, 1996 17:33:23 in article <Re: Another debate between BRAD and
>The Stein and Lambert "meta-analysis" (1995) is supposedly the most
>recent assessment of the data on the matter of the "equivalence" or
>"non-equivalence" of professionals vs. paraprofessionals
OK... I remember this one, but have to see if I can find my copy of it or
if I lent it out.... I've printed out your post so that I can formulate a
response. I'm glad to see you providing more substance, but I think you
overrate those who make decisions about what to publish. For example, you
say:
" I have been itching to again provide some review of this travesty, that
so well illustrates how standards of publication bend when it suits the
vested interests of the APA (political power and unwarranted exclusive
control). It is very close to pure propaganda, yet was accepted for
publication in a APA refereed journal:"
it's generally not for political reasons or economic reasons (unless
there's some hanky panky with the pharmaceutical companies providing
funding, advertising, etc., in which case I am generally suspicious).
Having served as a guest editor for journals and seeing how decisions about
publication are made, I think it would be more accurate to say that a lot
makes it into print that shouldn't, except that the reviewers don't know
enough stat and design themselves to realize what they're doing. That is
especially relevant when we talk about meta-analyses, which is a relatively
new area (new for those of us who completed our coursework over 20 years
ago). If you followed my conversation with Ed, I don't hold with
meta-analyses at all. So I may have an entirely different set of problems
with the S & L study, apart from issues that you think are important.
IAE, let me see if I can dig up my copy... if not, I'll have to wait to
respond to the substance of your post until I can get the whole thing in
front of me again.
I'll be back....
Leslie
--
Leslie E. Packer, PhD
Leslie E. Packer, PhD
On Jul 23, 1996 15:41:56 in article <Re: Another debate between BRAD and
>controlled studies done. All indicate equivalence. Allen Ivey's read
>of the literature agrees with me. Close examination of the unconfounded
>and relevant studies in Stein and Lambert (1995) also shows my view is
>correct (all 3 controlled studies I noted are noted there).
>(Disregard
>Stein and Lambert's umnjustified conclusions in their report.
the most part. It's one of the benefits of having expertise in design and
analysis... you don't have to trust or rely on the investigators'
conclusions and can form your own conclusions. (Of course, if people are
fudging data, that's another story....). Those who (merely) read abstracts
or summaries without critical reading of the full report (with particular
respect to D & A) are almost inevitably mislead.
>Stein and
>Lambert make conclusions without regard for confounds of experience and
>training; also training differences are very small -- Ph.D. candidates
>vs. practicing clinicians -- while experience diferrences (overlooked)
>are large.)
statement to make.
Have you ever read Tactics of Scientific Research, by any chance? (If you
haven't, no big deal, most people haven't)... but if you can get your hands
on it, there's a point where Sidman talks about such "confounds" and their
implications for research. It might give you a slightly different
perspective. (If I remember correctly, I think Basic Books published it in
around 1960... I don't know if it's still in print).
>> Thanks. Have fun getting real ! -- b jesness <<
that I'm quite real. And those reports aren't really going to help you
accomplish your goals... nor are your current approaches to advocacy.
Now, do YOU really want to get real?
Tell me... how many test cases have you taken to court? Where have you
testified? What political and legal changes have you been responsible for
so far -- that you can personally take credit for?
If you want to improve things for mental health patients, you're going
about it all wrong. But that's just my less than humble opinion. But I'm
not playing games with you or anyone else. When we (I) do advocacy, we
take a whole other approach.
Leslie
---
Leslie E. Packer, PhD
Leslie E. Packer, PhD
On Jul 23, 1996 17:43:15 in article <Re: Another debate between BRAD and>Sorry I went backk to comparing "paras" with professionals, but this is
>the resaearch I know of. It should have clear implications for the
>M.A./ Ph.D. comparison *YOU WANT* to make. I think you would agree.
Who was the above addressed to?
Cognitee
Sorry I went backk to comparing "paras" with professionals, but this is
the resaearch I know of. It should have clear implications for the
M.A./ Ph.D. comparison *YOU WANT* to make. I think you would agree.
Leslie E. Packer, PhD
On Jul 22, 1996 21:32:39 in article <Re: Another debate between BRAD and
>Regarding the demonstration you asked for: The demonstration of a
>lack of evidence is easy. Here it is:
>
>
>
>
>[did you see it? ; OF COURSE NOT , SILLY !)
>regards, b jesness
If you will go back and re-read my post, I did not ask for a demonstration
of lack of evidence. I asked for a demonstration of equivalence. Perhaps
the distinction was lost on you.
Kinda takes the sting out of your repartee when you're the only one who
doesn't understand the point, doesn't it? Ah, but don't worry. Just go
back to your seat and try again. We'll all wait patiently for you.
>
>P.S. On the other matter: Needless flaws exist in virtually all
>"therapy" efficacy research (as well as in the research you *are
>willing* to criticize) because of the unethical cowardice of the field.
(check the DSM for appropriate number... 300.03, perhaps?) doesn't give it
veracity.
>E.G.: No reasonable controls, nor any test of therapists against
>reasonable alternatives.
adequate designs, I disagree with you strongly as to why.
>I shall break through the denial
> Have a good day, Leslie.
>
Leslie
--
Leslie E. Packer, PhD
Peter
In article <4t151i$c...@news1.t1.usa.pipeline.com>,
lpa...@nyc.pipeline.com writes
>On Jul 22, 1996 21:32:39 in article <Re: Another debate between BRAD and
>Seligman, with BRAD the victor>, 'Cognitee <Cogn...@aol.com>'
<kill-filed, thus only responses to his responses can be responded to.>
>>Dear Leslie Packer.Ph.D.
>>Regarding the demonstration you asked for: The demonstration of a
>>lack of evidence is easy. Here it is:
>>[did you see it? ; OF COURSE NOT , SILLY !)
>>regards, b jesness
[Uh? Regards? After _that_ string of _un_therapeutic abuse, or was it
'parapro', or "Professorial" abuse] <Noch'n mal; es +gibt+ keine
Professorenherrlichkeit hier, susses <VBG> kind: ;Ś+Ś >
>Reading comprehension problems acting up again, m'dear? <concerned look>
Dr. P., 'Prof'' J?sness was never too good with comprehension, and I
often fear for the condition of his striate cortex. <Or did I 'hear'
frontal lobe syndrome...? ...perseveration, fo' _sho'_...?>
>If you will go back and re-read my post, I did not ask for a demonstration
>of lack of evidence. I asked for a demonstration of equivalence. Perhaps
>the distinction was lost on you.
Distinctions are rarely emitted from the undistinguished, whether
'Professorial' or or not, and 'Prof.' B??? is no exception, Dr. Packer;
you might as well correspond with a sandbag, as well we both know.
>Kinda takes the sting out of your repartee when you're the only one who
>doesn't understand the point, doesn't it? Ah, but don't worry. Just go
>back to your seat and try again. We'll all wait patiently for you.
_Repartee_? Nope; _bland repeatee_. <But you knew already what to
expect...> <g> <Waiting behaviours commence.>
>>P.S. On the other matter: Needless flaws exist in virtually all
>>"therapy" efficacy research (as well as in the research you *are
>>willing* to criticize) because of the unethical cowardice of the field.
<Cor, I'd be careful about supporting one such as this, one might be
tarred with the same J?sness, no?... ...'many' and 'all', as two
different posters are wont to say...>
>How unfortunate for you that just repeating something over and over again
>(check the DSM for appropriate number... 300.03, perhaps?) doesn't give it
>veracity.
[Click, Clack]
300.03?; fur mesen, aah prefers the .303, it's terminal, it's mah
feryend, it sees and reaches for miles*, and from the faces of united
buttheads it always removes smahles.
* <'specially with a 'scope>
Absolute Disclaimer: 'Prof.' B.J and his young Canadian friend should
not even ideate <let alone falsely and _hysterically_ [g] assume> that I
own, or have _***access***_ to, the weapon on which I gained my
marksman's badge - 'cos I don't': I don't have and would reject such
access, or ownership; I believe in kill-files rather than killing or
maiming, or even psychologically hurting <altho', I believe that 'Prof
Jesness' has done some of the last himself, and I have some postings to
show...>. I also believe in second-hand access to postings - the latter
make me grin; that's enough for me. Savourrr the flavourrr....
....OTOH... ...wot's the position, re. falsely declaring oneself to be a
Professor of Psychology?
[Click, clack]
I was just more than a tad p****d off with seeing direct posts by people
who have nothing new to offer, who will not entertain the restrictions
laid down by IRB's <problems with reality testing?>, and who
consistently come up with bullship when confronted with data. So, I
placed them in my kill-file. Now, because of the quality of responding
beh's., I wonder whether I should refrain from killing threads... ...the
eternal NG dilemma.
Responses? OK. They tend to teach me. I am also amazed that we can,
from a sow's ear, a silk purse make. My 'umble 'gratulatins to the
respondents, you are better than 'Bad' <or was that 'B???'?>
<Why don't we have a stats thread?>
B??? I wouldn't mind if he was 'RET to the bone', but he's not. He's
?r?d to the bone, and so it would seem is the axon grinder/rent-a-quote.
How _could_ _anyone_ defend such a _plonker_, I ask? Well? _HOW_?
<Sorry, shouting.> Not good for credibility, what, what, what?
>>E.G.: No reasonable controls, nor any test of therapists against
>>reasonable alternatives.
>While I actually agree with you about the absence of necessary controls and
>adequate designs, I disagree with you strongly as to why.
IRB's, ethics comittees, the actual _experiences_ and _feelings_ of
_patients <other side of psychology being the subject>, near whom I
wouldn't even let a parapro _imagine_ a fart?
B???, you're dangerous, and that's the truth. You have overextended an
idea; you have cuddled up to those who extol the virtues of social
medicine, you have cuddled up to those who attempt to propagate self
help, and you also ride on the bandwaggon of meta-analyses <been taking
tuition recently?>... ...never mind your earlier <uninformed> attempts
to snuggle up to those who opposed those whom you opposed... ...not
realising that said opponents actually had a clearly defined theoretical
viewpoint of their own... ...not yours.
>>I shall break through the denial
<This from a man in <g> denial.> <Or was that 'renial'?>
>Well, now you're in real trouble. Some of us don't believe in denial <G>.
Shame, there's a _good_ few miles in this one... ...Monsignor J?sness
_loathes_ denial! He's a BTTB'er, as opposed to an RET'er...
>> Have a good day, Leslie.
<Have a nice 'phone bill Bradley, using a web browser is costly, and
probably requires 'BTTB power', eh?>
><chuckling> Thanks for the laugh.
Moi aussi. [;Ś+Ś to you, B?????y]
CHEERS, eh? <Stick on the ice? PTSD? Explanation?>
--
Peter
Cognitee
Dear Leslie<
You said:
"On Jul 23, 1996 17:43:15 in article <Re: Another debate between BRAD
>Sorry I went backk to comparing "paras" with professionals, but this is
>the resaearch I know of. It should have clear implications for the
>M.A./ Ph.D. comparison *YOU WANT* to make. I think you would agree.
Leslie E. Packer, PhD"
(end quote) It was addressed to you (sorry for my lack of salutation)
-- b jesness
Cognitee
Dear Leslie,
"Unconfounded" may be a judgement call; "confounded" is sometimes
less so. -- b jesness
Cognitee
Dear Leslie,
Trouble is the things the APA (the referee groups, esp. the clinical
group) tends to accept are pro-status quo, pro-professional, even when
the quality is terrible. I have given 2 examples in another thread.
Presumptions and "groupthink" abide with these "long-sufferers"
(habitual "hoop jumpers"). Some of the mistakes made are so outrageous
they should NEVER (ever) happen and wouldn't if good science standards
were simply well entrenched and enforced. Stein and Lambert is one such
case of an inexcuasble error (and note how self-seving the [false]
conclusions of this "study" are).
Then following the biased "slips" of the APA referees, everyone jumps
on these as providing evidence that all is well. I believe the APA
ALONE cannot be trusted to judge quality. Proven results from true
science practioners working in *individual agencies* (or a few agencies
working together, as appropriate) is the way. There can always be
government (and media) oversight to regulate abuses here if false claims
of improved efficacy or improved treatment are made that are not so.
Agencies themselves can set-up quality control to keep from getting in
trouble. But we certainly can't expect all "good results" to appear in
[biased] referred journals, nor can we expect that what appears there is
very good at all. NOR can we expect that good work should be done with
an eye (particularly) to publication. Some things that seem small end up
big in the long run; only individuals working on the problems themselves
can likely see this. We should basically just do science work to help
ourselves, our colleagues, and CLIENTS, simply, sincerely, however
through all appropriate scientific means (through all means of gaining
higher inter-rater reliabilities). It is good business (good
advertizing), even for the HMOs. Research is now way to rare an
enterprize for clinical psychology to be considered a science by the
most reasonable standard I outline in another thread AND THIS IS WHY:
few true science practitioners. Just reading and understanding research
and making use of it does not make you a sceince practitioner. I don't
even think that makes you a professional. Some cooperative work with
others (and not just clients) that is distinctly scientific is trainly
meaningfully brought to fruition. Nothing else. -- b jesness
Leslie E. Packer, PhD
On Jul 24, 1996 01:39:15 in article <Re: Another debate between BRAD and>What is this the completion of the proof? Like: All male humans are
>men; all men are male humans. Or am I missing something?
--
I suspect you are (missing something, that is).
Would it have helped if I had reformulated it as "And when did you stop
beating your wife?"
Leslie
Leslie E. Packer, PhD
On Jul 23, 1996 23:40:03 in article <Re: Another debate between BRAD and
Seligman, with BRAD the victor>, 'lpa...@nyc.pipeline.com(Leslie E.
Packer, PhD)' wrote:
>Tell me... how many test cases have you taken to court? Where have you
>testified? What political and legal changes have you been responsible for
>so far -- that you can personally take credit for?
And I am still waiting for your reply, Cognitee. I really want to know
what _positive_ changes you think you've personally made for mental health
patients using your approach and arguments.
Leslie E. Packer, PhD
On Jul 24, 1996 01:15:28 in article <Re: Another debate between BRAD and
Seligman, with BRAD the victor>, 'Cognitee <Cogn...@aol.com>' wrote:
>It was addressed to you (sorry for my lack of salutation)
stated:
> It should have clear implications for the
>M.A./ Ph.D. comparison *YOU WANT* to make. I think you would agree.
between masters' level and doctoral level professionals. The contrast
exists in the nature of the training and what the different levels are
qualified to do (by law as well as by training).
But more on that when I respond to your other post.
Leslie E. Packer, PhD
On Jul 24, 1996 01:13:22 in article <Re: Another debate between BRAD and
>less so.
--
speculative statements, unless backed up by hard data. And in light of
potential confounds, the most conservative thing one can say is something
to the effect that the author's conclusions may not be warranted in light
of a,b,c. That does not mean that the author is wrong... merely that the
data don't _unequivocally_ support the conclusions. The entire system of
labelling or characterizing something as a "confound" or "not a confound"
is predicated on certain assumptions and hypotheses, as well as on
pre-existing experimental data.
I have done a quick read-through of your longer post critiquing the S& L
meta-analytic review, and will try to draft a reply to you tomorrow. I'm
not sure I understand who you think you're advocating for, but there's at
least one serious hole in your argument in that post, which I will try to
outline to you for your consideration.
Leslie E. Packer, PhD
Without going through your post line by line, I am curious as to why you
focus on the APA? Do you also make the same statements about the other
APA? What about the AMA?
And if you don't, why not? The DSM-IV is not under the control of the
psychologists, you know. Speak to the physicians about those problems.
Are you aware that when it comes to the quality of research, the
A_Psychol_A is light years ahead of the other organizations because
psychologists are better trained in design?
And why have you remained silent on the fact that most of these journals
encourage debate by having a "Letters to the Editor" section where those
who disagree can present their own data or objections to a published
article?
Leslie
On Jul 24, 1996 01:35:19 in article <Re: Another debate between BRAD and
Cognitee
Dear Leslie,
BRAD members do self-advocacy and work for fairness to clients and
themselves, directly and personally. We need no court process and YET
will triumph. We do seek to appropriately divide labor (when desired by
members and as prudent) and encourage cooperation among our declared
members. So adding your name to an official register is helpful.
Clients who are BRAD members insist on a fair amount of information
before going into a costly treatment. All who are thorough-going
scientists, believe very significant improvements in the
counseling/therapy field ARE NECESSARY, and back the basic tenents of
the BRAD manifesto are BRAD (honorary B.R.A.D. members).
When in Toronto, say you are BRAD (or back the ideas, if you do not
like the provisional name of the organization):
It is a client and science advocacy group,
dedicated to furthering science standards and practices in the therapy
field. We insist on fair and proper representation of treatments and on
providing information about costly or limited treatment options
available to clients "up front". We believe options and evidence of
their efficacies should be provided to clients before they enter a
course of counseling or therapy. The various treatments and programs
offered by each professional mental health service provider should be
outlined in some detail in a booklet made available to clients. Only
this would provide reasonable information before the expense of and
commitment to a course of treatment.
Also, techniques or methods used that have NOT been clearly shown to
have efficacy AND validated for a particular, reliably-identifiable
problem type (i.e. showing blind inter-rater reliability) are NOT be
referred to as "therapy." Correspondingly, when what is done is
COUNSELING, the cooperative nature of this should be made clear and it
should be properly represented, engendering appropriate expectations.
Counseling is considered a most noble cooperative endeavor, requiring
the most consideration, judgement, and intelligence. Those who are
well-adapted will be better counselors. For this reason, and considering
the rest of the evidence, counselors/therapists should have a long
history of good adaptation.
Moreover, BRAD believes daily standards in practice should provide
for on-going research (such as for the development of reliable
diagnoses) and this should be done within each large mental health
service agency. Furthermore, basic foundation research definitively
showing that graduate-school-trained counselors are superior to other
sources of help must be done to establish the range of problems for
which special treatment by professionals is actually better (and not
inferior to other more accessible and less costly sources of help, e.g.
peer counselors or paraprofessionals). BRAD also supports (given at
present there is no evidence against it and some good evidence in its
favor): peer counseling programs and counseling programs for
paraprofessionals. BRAD seeks to demystify mental health professions
and rid it of great myths. We hope for a sensible, delineated mental
health care SYSTEM, with the care often involving peers and
paraprofessionals and for care to be provided by individuals within a
client's working community.
Cognitee
DEAR Leslie:
The following is truely pathetic (quoting you): "Are you aware that
A_Psychol_A is light years ahead of the other organizations because
DEAL. What is needed is training in SCIENCE. DESIGN IS *DESIGN*. It
is basically just an "apparent form". While you may not feel the field
is (in general) established, you certainly FALSELY believe methodology
is. You are deluded. (You may turn back your Ph.D.) Anyone that
believes that "the method" can be "ok" without much more local work and
local (agency) investment in research (i.e. without more true science
practioners) will someday (VERY, VERY SOON, I HOPE) find themselves
dinosaurs. The reason is that all the little things only get done by
continuous research by persons with lifetime investments. It is as if
the APA would encourage everyone to do everything by committee. Do you
see the difference? It gets back to the point that many things that
"look little" to research evaluators are really important ("big") to the
insightful lifetime researcher with a developing clear vision. This
will be the science. Until we get there, there is no science. -- b
jesness
Leslie E. Packer, PhD
On Jul 23, 1996 15:41:56 in article <Re: Another debate between BRAD and
>controlled studies done. All indicate equivalence. Allen Ivey's read
>of the literature agrees with me. Close examination of the unconfounded
>and relevant studies in Stein and Lambert (1995) also shows my view is
>correct (all 3 controlled studies I noted are noted there).
with your critique of S&L and the whole issue. I will forego responding to
each example of your nonobjectivity except to note that it detracts
significantly from your post: it would be far better to stick with the
facts and omit the editorializing, IMO. But, from the top:
1. If I understand your position (?), you want to see a well-controlled
comparison of professionals to paraprofessionals.
2. You do not specify any context in this post for what scope of problems
you wish to use to assess any potential differences or equivalence. I am
therefore left wondering if you would demand that we compare well-spoken
and caring paraprofessionals to trained physicians for people who have
chest pains or sudden weakness in one limb, and if not, why not? Taking it
back to mental health, we still need a hypothetical scope or range of
clearly defined diagnoses for which you want the comparison made. Can you
provide me/us with such a list?
3. You cite Strupp and Hadley (1979) as the last of the "well-controlled"
and presumably unconfounded (IYO) studies on the issue of concern to you.
I don't have a copy of that article in my office, but if memory serves me,
their study used an extremely small sample size. Hence, any demonstration
of "equivalence" is immediately suspect in the absence of any statistical
power to find a difference between the two conditions had one existed.
IOW, they didn't look very hard to find a difference, and in fact, the
cards were stacked _against_ them finding a statistically significant
difference. The power of their statistical test was probably somewhere on
the order of .15 or so.
4. Additionally, didn't Strupp and Hadley report that the group means were
misleading because of the large within-group variability? Again, this
would argue for a real problem with drawing _any_ conclusions from their
study.
Thus, wrt to this study that you rely on to bolster your argument, it
doesn't really tell us anything helpful, except that there _may_ be (not
are) some nonprofessionals who _might_ do as well as _some_ (but not
other) professionals for certain types of problems (but see next point).
Such a statement is a far cry from saying that equivalence has been
demonstrated.
5. You keep repeating that the paraprofessionals performed equally well
over a "BROAD" range of problems exhibited by college students. That is
not my recollection. I cannot remember offhand _how_ the student
participants were recruited, and perhaps you could refresh my memory, but I
think that the type of problems they were presenting with were pretty
low-level problems (e.g., some anxiety, some depression (but not endogenous
depression), etc.). I don't recall there being any instances of more
serious diagnoses. But perhaps my memory fails me here...?
6. With respect to S&L's meta-analysis, I repeat what I said to Ed in
another thread: garbage in, garbage out. I was pleased to see that you
went and looked at the details of the articles that were included in the
analyses. I don't agree with much of what you said about them, but at
least you looked for the details.
7. Even if a finding is incidental to the purpose of a study, that does
not make the finding less valid or useful. You seem to want to dismiss or
discredit some data just because the main purpose of the study wasn't to
look at the difference between paraprofessionals (or lesser trained
individuals) vs. professionals.
That is not an acceptable justification for discounting data. There may be
other valid reasons to discount data, but that is not a valid one.
8. With respect to your analysis of the effect size, I think you yourself
are well aware of the problems due to the assumptions that you had to make
to generate any number. I won't challenge you on that because you do point
out different possibilities, and I give you credit for trying to calculate
a measure. BUT: you draw what I think is a totally erroneous conclusion.
If I followed your argument, the small effect size (which you estimated at
2 SD but suggest might be an overestimate) shouldn't be considered
important because it is small. Was I understanding your correctly?
Because if so:
First, I can only wonder what you would have argued if the difference had
been in the other direction. There is an important difference between
statistical significance and clinical significance, but one cannot truly
have the latter if the former does not exist. So demonstrating a reliable
effect is the first step, and given small sample size, any reliable effect
demonstrated may be an _underestimate_ of the true population effect.
Second, keep in mind your point about the "poor" comparison many of the
studies used: More highly trained vs. less trained (let's ignore the
length of experience confound for now). Given that the studies used two
levels of the independent variable (training) that are _closer_ in value
than values such as "extremely highly trained vs. Zero training), they have
produced a _lower_ estimate of the effect of the independent variable. Had
they actually selected the levels properly (by your definition of what
levels they should have used), it is more than likely that there would have
been an even _greater_ effect (in the direction of trained >
paraprofessionals). IOW, I disagree with you completely on where you
landed up in your conclusion.
All of the above aside, I am still at a loss as to what you think could
possibly be accomplished by using paraprofessionals (and I'm not sure what
level of training you're advocating they receive).
There are certainly some situations in which support groups, peer
counselors, etc. can be of value. I don't deny that, and frequently refer
patients to certain groups, etc. BUT: I don't think that _anyone_ should
rely on any paraprofessional for counseling or psychotherapy unless a
trained and qualified professional says that it would be of benefit. And
even then, there are tremendous risks, as I've seen in some situations.
If you really want to help people, then help them obtain quality care.
Don't fall into the trap of thinking that compassion or a couple of
training sessions is enough. With the exception of a few very
circumscribed situations, I don't think it is. Hell, I _know_ it isn't. I
see what happens in my own field(s) when I find a therapist who's had a
little bit of training and is confident that they know what to do. They
generally can't get beneficial results at all. And in some cases, they do
harm.
I'll repeat that: in some cases, they do harm. They don't mean to, but
they don't know enough to know when to refer out. Even if you give them
what you think are clear guidelines, there are a million things that could
happen or come up, and they will not have the same knowledge base as the
highly trained professional. And the patient will suffer because of it.
What also really concerns me is that in some sense, I think your actions
actually contribute to the stigma associated with mental health problems.
I know you don't intend to do that, but anyone who argues that mental
health problems don't require high level expertise is essentially agreeing
with those who would say "why should we pay for all this therapy? The
person should just try harder to get over their problems." Anyone who's
really worked with patients with mental illness knows how unfair and
demoralizing that whole attitude is.
I would rather see you work for parity between mental health problems and
other health problems than promote the idea that paraprofessionals should
be seriously investigated as a useful alternative to qualified clinicians.
Regards,
Cognitee
No Ed,
I think all the concerns about design matter. All these concerns
must be honored strictly. I do. My point is that there is more to good
science than "research design" as currrently considered. Good
observations and with the continuous establishment of a series of
excellent inter-rater reliabilities (and thus a basis for good validity)
DEPEND *AS YOU MUST KNOW OR SENSE* ON MORE THAN JUST "DESIGN." -- b
jesness
Cognitee
Dear Ed,
You say (at first quoting me):
"Cognitee <Cogn...@aol.com> wrote:
>No Ed,
> I think all the concerns about design matter. All these concerns
>must be honored strictly. I do. My point is that there is more to good
>science than "research design" as currrently considered.
Of course there is. My objection was to the way you offhandedly
dismissed the importance of research design." (end quote)
ED !, I, in no way, in any part "dismissed" research design. BUT this is usually a relatively easy matter, typically involving less=
theory or judgement. What to study and where to try to improve inter-rater reliabilities systematically is a greater matter and inv=
olves more judgement and wisdom. Still it is not hard if done (as it has to be) by a local working group. It is impossible "by com=
mittee" in most significant ways. And this is exactly why our progress in diagnoses has been advancing (at best) at a snail's pace:=
too much committee work ALONE. But (of course) the proof of quality of the sharing of the results is always in the inter-rater rel=
iabilities.
Quoting you again (at first quoting me):
">Good
>observations and with the continuous establishment of a series of
>excellent inter-rater reliabilities (and thus a basis for good validity)
>DEPEND *AS YOU MUST KNOW OR SENSE* ON MORE THAN JUST "DESIGN."
Yes, but let's see you get "good observations," "excellent inter-rater
reliabilities," or any sort of validity without a good research design.
ed" (end quote)
I AGREE you must have good design. You also must breathe, eat, and sleep. -- b jesness
Ed Anderson
Cognitee <Cogn...@aol.com> wrote:
>No Ed,
> I think all the concerns about design matter. All these concerns
>must be honored strictly. I do. My point is that there is more to good
>science than "research design" as currrently considered.
Of course there is. My objection was to the way you offhandedly
dismissed the importance of research design.
>Good
Ed Anderson
Leslie E. Packer, PhD wrote to Brad:
>....3. You cite Strupp and Hadley (1979) as the last of the "well-controlled"
>and presumably unconfounded (IYO) studies on the issue of concern to you.
>I don't have a copy of that article in my office, but if memory serves me,
>their study used an extremely small sample size. Hence, any demonstration
>of "equivalence" is immediately suspect in the absence of any statistical
That's a good point. The size of the treatment groups were 16 and 15. A
couple other things about that oft-cited study:
1. The "paraprofessionals" in Strupp & Hadley were university professors
specially selected by students "on the basis of their reputation for
warmth, trustworthiness, and interest in students." These were not even
average professors, much less average paraprofessionals.
2. The number of professional therapists in the study was five; the
number of "parapros" was four (I think -- it may have been five also).
3. The measures used were not particularly sensitive to change (e.g.,
MMPI).
4. While no significant differences appeared between pros & "parapros,"
differences in group means were all in the expected direction.
ed
Cognitee
Yes, Ed
Strupp and Hadley tried to minimize their results. Since this is the
last controlled study on the matter what do you conclude? What would you
propose? It was not a good study (some say), "therapists" are great,
forget studies. Is this your view. I would like your concrete
proposals without unfounded and unscientific cautions or groundless,
bogus ethical considerations stopping needed research cold. Without of
the cuff presumptions about the views of committees. Be bold, be
ethical, be a scientist. Forget "therapy" , its myth and presumptions.
-- b jesness
Peter
In article <4t5kig$k...@hamblin.math.byu.edu>, Ed Anderson
<eand...@cougarnet.byu.edu> writes
>Cognitee <Cogn...@aol.com> wrote:
>>DEAR Leslie:
>> The following is truely pathetic (quoting you): "Are you aware that
>>when it comes to the quality of research, the
>>A_Psychol_A is light years ahead of the other organizations because
>>psychologists are better trained in design?" (end quote) I say: NO BIG
>>DEAL. What is needed is training in SCIENCE. DESIGN IS *DESIGN*. It
>
>I think your dismissal of the importance of design is typical. As a
>"scientist," you suck.
Interesting, Ed, and I agree with you. I was under the impression that
all scientific experiments need to have a sound design, otherwise one
might as well go play in s.p.p.? <G>
OTOH, one could always become a parapro, or an adherent of German
Speculative Idealism. How apt.
Yours, in tears of laughter.... .....just like B??? is, I suspect, since
he's probably enjoying himself soooo much. This is just a game.
--
Peter
<trying hard to lurk, but failing intermittently, it's that damn 'send' button
thangy, it looks so professional!>
Ed Anderson
>3. ....You cite Strupp and Hadley (1979) as the last of the "well-controlled"
>and presumably unconfounded (IYO) studies on the issue of concern to you.
Well, since I have one handy...
>4. Didn't Strupp and Hadley report that the group means were
>misleading because of the large within-group variability? Again, this
>would argue for a real problem with drawing _any_ conclusions from their
>study.
Yes, they did: "...considerable variability in individual dyads.... As
our work progressed, we became impressed with the fact that group
comparisons obscure the very phenomenon that must be understood..."
(p. 1135).
Strupp published four articles in the 1980 Archives of General
Psychiatry, which examined these "individual dyads." In my opinion,
these are much more interesting articles (methodologically and
substantively) than S & H (1979). Furthermore, these 1980 articles
document (in places) how, confronted with difficult clients, the
university professors ("parapros") were often ineffective (not that the
pros had much luck with these cases, either, but the parapro's mistakes
were particularly glaring).
>5. You keep repeating that the paraprofessionals performed equally well
>over a "BROAD" range of problems exhibited by college students. That is
>not my recollection. I cannot remember offhand _how_ the student
>participants were recruited, and perhaps you could refresh my memory...
Allow me. They were selected based on elevations of depression, anxiety
(psychasthenia), and social introversion scores on the MMPI. Average T
scores on these were, respectively, 87, 82, and 70 for the
professional therapist group and 76, 74, and 69 for the "parapro" group.
Note, incidentally, the higher scores on intake among those assigned to
the therapists... As S & H note, randomization was not entirely
successful.
ed
Leslie E. Packer, PhD
On Jul 24, 1996 14:56:28 in article <Re: Another debate between BRAD and
>BRAD members do self-advocacy and work for fairness to clients and
>themselves, directly and personally. We need no court process and YET
>will triumph.
You and I are not playing in the same ball park, much less in the same
league.
I asked you what you've accomplished. You couldn't cite me one specific
accomplishment or change that you've produced. Think about it. Now you
can blame the powers that be for your failure to accomplish your goals, or
you can try a different approach.
You don't have to have a doctoral degree or be a health care professional
to achieve change in the system. But you do have to know how to advocate
effectively.
Leslie E. Packer, PhD
On Jul 24, 1996 15:07:07 in article <Re: Another debate between BRAD and
>is (in general) established, you certainly FALSELY believe methodology
>is. You are deluded. (You may turn back your Ph.D.)
Methodology is not established? Sheesh, thanks for telling me. Wonder
what I was teaching for lo, those many years.
And since my PhD was in _research_ and not _clinical_ psychology, I wonder
what I was studying for lo those many years.
> The reason is that all the little things only get done by
continuous
> research by persons with lifetime investments.
sounds good, even if it isn't accurate.
>It gets back to the point that many things that
"look little"
> to research evaluators are really important ("big") to the
insightful lifetime
> researcher with a developing clear vision.
and discovery.
> You are deluded.
Is that your professional diagnosis?
For the record, if you wish to dialogue or communicate with me, you will
need to modify your style to be less offensive. I am not willing to talk
with people who going around stereotyping people and attacking them. If
you want to have a respectful discussion, fine, but if all you want to do
is spout rhetoric and attack people, I will ignore your future posts.
Your choice.
Leslie
Cognitee
Dear Leslie,
Only SOME of what's involved in good methodology is established (THUS
OVERALL IT IS NOT). What you've VERY LIKELY been studying and
regurgitating (given your statements about research method being
"established") is "part of the story." Your vague protestations aside,
I stand by my remarks. -- b jesness
P.S. I will not be pleasant in the face of the "pleasnant acquiescence"
to the inadequate status quo. "I'm mad as hell and I'm not going to
take it anymore."
Cognitee
Dear Leslie,
Doing work in the courts is only less crude than a fist fight. BRAD
is "grassroots" everyday. -- b jesness
John Clark
In article <4t5vta$q...@hamblin.math.byu.edu>,>>....3. You cite Strupp and Hadley (1979) as the last of the "well-controlled"
>>and presumably unconfounded (IYO) studies on the issue of concern to you.
>>I don't have a copy of that article in my office, but if memory serves me,
>>their study used an extremely small sample size. Hence, any demonstration
>>of "equivalence" is immediately suspect in the absence of any statistical
>
>That's a good point. The size of the treatment groups were 16 and 15. A
>couple other things about that oft-cited study:
>
But then what of book writers who base theories on one or two patients,
who are not in any way 'picked at random'.
It seems that other that the Horatio Alger appeal, no other claim could be
made.
John M. Grohol Psy.D.
Cognitee wrote:
>
> DEAR Leslie:
> The following is truely pathetic (quoting you): "Are you aware that
> when it comes to the quality of research, the
> A_Psychol_A is light years ahead of the other organizations because
> psychologists are better trained in design?" (end quote) I say: NO BIG
> is basically just an "apparent form". While you may not feel the field
> is (in general) established, you certainly FALSELY believe methodology
> believes that "the method" can be "ok" without much more local work and
> local (agency) investment in research (i.e. without more true science
> practioners) will someday (VERY, VERY SOON, I HOPE) find themselves
> continuous research by persons with lifetime investments. It is as if
> the APA would encourage everyone to do everything by committee. Do you
> "look little" to research evaluators are really important ("big") to the
> will be the science. Until we get there, there is no science. -- b
> jesness
It is more than mildly hypocritical of you, Mr. Jesness, to talk
of "science" all the time and have done so little yourself.
As many people have pointed out many times in the past, why don't
you get off your ass and do some? It's one thing to sit there and
bitch and complain for months (almost a year now) on an Internet
newsgroup and quite another to get funding and subjects to actually
do all the "science" you claim is needed. If you're such the
expert in science, Mr. Jesness, you should truly be leading the
field in directing others and obtaining the means to do the research.
I'd very much like to see you in such a leadership role in this
field.
Last time I looked, research doesn't grow on trees or fall out
of the sky. Or get done by harassing others to do it.
-John
--
Mental Health Net
http://www.cmhc.com/
Top 5% of the Web - Point
Cognitee
Leslie Packer said:
"On Jul 23, 1996 15:42:32 in article <Re: Another debate between BRAD
and
Seligman, with BRAD the victor>, 'Cognitee <Cogn...@aol.com>' wrote:
>P.S. Especially those who engage in denial would not believe in it.
--
Sure... but so will those who know that they don't.
(Isn't this fun?)
Leslie E. Packer, PhD" (end quote)
What is this the completion of the proof? Like: All male humans are
Ed Anderson
>....Stein and Lambert is one such
The study you're referring to gives no more than half-hearted,
mild support for the relationship between therapist efficacy and
therapist experience. That fact alone contradicts your assertion that
the reason it was accepted was because it supports the status quo. The
"status quo," to the extent that such a thing exists in a field this new
and divisive, is that experienced therapists are MUCH, MUCH better than
inexperienced ones. Which is hardly what Stein & Lambert concluded.
ed
Cognitee
Dear Ed,
Your field is not so divisive as it is confused. True research
studies are do not occur in any integrated sequence and work is
disjointed. But this does not preclude the notion that there is a
rather common (or prevalent) set of ideas (beliefs) along with just bad
procedure. Ivey refers to the field of counseling/therapy as operating
in a "narrow fog." I think this is a most excellent metaphor. -- b
jesness
Cognitee
Dear Leslie,
Some things are so clearly confounded in ways that may well make
results unclear that such a judgement call is sometimes easy.
IN CONTRAST: The idea that one can always find confounds is
splitting hairs (kind of like saying we will never have ultimate proof).
True. But silly. Good enough is what is good and will work
significantly better and has the promise of being developed so it works
even better. This is the point of research. Some very useful research,
"proven" useful, can be still be seen to have confounds. But this is
sometimes certainly (at least relatively) trivial. -- b jesness
Cognitee
Dear Leslie,
The fact that there may be no really good studies that have been done
on the question of "paras" vs. "pros" only bolsters my point that there
are studies that need to be done. It defies reasoning (though makes
sense when viewed as EXTREME defensiveness) that these have not yet been
done. On the other hand, Strupp and Hadley's study used multiple
measures (both in time and otherwise) (many blind) with a great
consistency in the result. Furthermore there sample size was adequate.
Their sample group was well definded (and can be considered a "broad
range of clients") and could be used in future (PRESENT ?) research.
I will agree with you that people should not relay much on
non-professionals now. This role must be defined and brought "within
the system." Clinical psychology's failing here has been an outrage. I
am as mad as hell and I'm not going to take it anymore (your damned
frinki' lip-service aside!!).
You also make great presumptions about what "quality care" is. When
professional resources are non-specialized and spread thin and we are
failing (through not doing the very same research I've been begging for
above) to define where professional services are most needed and (really
needed) AND DEVELOPING THEM, WE ARE DENYING QUALITY CARE. You are
failing in your purpose. You may see yourself as the end-all-be-all,
but it's bullshit to me. And, Leslie, because you present no clarity
this is precisely what you are doing: insisting on the status quo.
Finally, you say (quoting): "What also really concerns me is that in
actually contribute to the stigma associated with mental health
associated with *your field* (and YOUR FIELD *AS IT IS* KEEPS THE STIGMA
ABOUT MENTAL HEALTH PROBLEMS (i.e. diagnoses) VERY MUCH ALIVE). Well, I
frankly think you deserve your stigma ! Your field frankly, simply HAS
NOT DONE THE STUDIES TO ALLOW YOU DO DEMONSTRATE THAT YOU HAVE "high
level expertise" OR EVEN ANY *PARTICULARLY* SPECIAL ABILITIES for many
counseling cases. Your entire position is so unclear we do not know
what a "high level expertise" looks like. It is because your field is
comprized of charlatans and cowards. I shall promote the
paraprofessional option as long as I and others (in the footsteps of
Carl Rogers) view it as a reasonable option. -- b jesness
P.S. When you realize, as Allen Ivey does, that the counseling/therapy
field is operating "in a narrow fog." A very apt metaphor. List to
BRAD.
Ed Anderson
Cognitee <Cogn...@aol.com> wrote:
>DEAR Leslie:
> The following is truely pathetic (quoting you): "Are you aware that
>when it comes to the quality of research, the
>A_Psychol_A is light years ahead of the other organizations because
>psychologists are better trained in design?" (end quote) I say: NO BIG
>DEAL. What is needed is training in SCIENCE. DESIGN IS *DESIGN*. It
I think your dismissal of the importance of design is typical. As a
"scientist," you suck.
ed
Cognitee
P.S. Leslie,
I am well aware of an American Psychological *Society*. Readers
should know that it split from the APsycholA due to problems (they say)
with science there. Still, I'm sure APAers will love their little
lovely convention though. Kinda of the teamsters, if you ask me (though
worse because they are disingenuous). -- b j
Cognitee
Dear John Grohol,
Here's the problem, it's the problem with the whole system: (quoting
you): " ..get funding and subjects to actually do all the "science" you
claim is needed." The problem is that "funding" must be obtained
through existing institutions with agendas and presuppositions or
presumptions and competition is both fierce and ingenuous for VERY, VERY
limited resources. The activity of research is not done as a matter of
course by individual agencies as part of their normal day to day
activity -- like drug companies or plastic manufacturers, or almost
anyone else with a specialized offering in society. I'd have to
brown-nose some status quo bastard. Someone like you.
I prefer my present course of insisting on professionism by all as a
matter of course. -- b jesness
Cognitee
Again Peter,
I agree totally with Ed. Sound design is essential. But "sound
design," as understood today in the very limited sense, is not all that
is essentail for good science. -- b jesness
P.S. You are not too smart when you're cogent, are you Pete?
Cognitee
Dear Ed,
15 or 16 subjects in each group is typically enough to find
significance when a meaningful result is sought (one where the
difference found would be considered meaningful AS WELL AS SIGNIFICANT).
And, yes: In a sense the "paras" used (act. university profs. with
good reputations) may have LESS THAN the skills you would find with well
selected and reasonably trained "paras". -- b jesness
Leslie E. Packer, PhD
On Jul 24, 1996 20:32:07 in article <Re: Another debate between BRAD and
Seligman (long post)>, 'Ed Anderson <eand...@cougarnet.byu.edu>' wrote:
>lpa...@nyc.pipeline.com(Leslie E. Packer, PhD) wrote
>(to brad):
>
>Well, since I have one handy...
I put an entire university or hospital library??? <G>
>>4. Didn't Strupp and Hadley report that the group means were
>>misleading because of the large within-group variability? Again, this
>>would argue for a real problem with drawing _any_ conclusions from their
>>study.
>
>Yes, they did: "...considerable variability in individual dyads.... As
>our work progressed, we became impressed with the fact that group
>comparisons obscure the very phenomenon that must be understood..."
>(p. 1135).
that there's tremendous variance and then continue to report group
statistics....
>
>Strupp published four articles in the 1980 Archives of General
>Psychiatry, which examined these "individual dyads." In my opinion,
>these are much more interesting articles (methodologically and
>substantively) than S & H (1979). Furthermore, these 1980 articles
>document (in places) how, confronted with difficult clients, the
>university professors ("parapros") were often ineffective (not that the
>pros had much luck with these cases, either, but the parapro's mistakes
>were particularly glaring).
publication(s) when he started advocating that we do away with outcome
measures. I think he truly saw that as the only way out of the dilemma of
what he saw as the pseudo-issue of therapist factors vs. professional
skills/techniques, but (IMO) he threw the baby out with the bathwater, and
in this day and age of managed care, we are seeing greater and greater
emphasis on outcome data. And I think that that is one of the _good_
consequences of managed care-- accountability.
>>5. You keep repeating that the paraprofessionals performed equally well
>>participants were recruited, and perhaps you could refresh my memory...
>
>Allow me.
> They were selected based on elevations of depression, anxiety
>(psychasthenia), and social introversion scores on the MMPI. Average T
>scores on these were, respectively, 87, 82, and 70 for the
>professional therapist group and 76, 74, and 69 for the "parapro" group.
>Note, incidentally, the higher scores on intake among those assigned to
>the therapists... As S & H note, randomization was not entirely
>successful.
the college students who participated? Can you remind me of the recruiting
procedure?
Leslie E. Packer, PhD
On Jul 24, 1996 20:26:56 in article <Re: Another debate between BRAD and
>OVERALL IT IS NOT). What you've VERY LIKELY been studying and
>regurgitating (given your statements about research method being
>"established") is "part of the story." Your vague protestations aside,
>I stand by my remarks. -- b jesness
you about in an earlier post (Tactics of Scientific Research)? Have you
ever read Barlow and Herson's book on single case designs? While I am
personally and professionally committed to rigorous experimental designs as
opposed to the quasi-experimental or qualitative designs, I have to wonder
exactly how much you've actually read on these topics. Your statements
about science and design show an incredible naivety.
>P.S. I will not be pleasant in the face of the "pleasnant acquiescence"
>take it anymore."
>
That is your choice, of course. But like all choices, there are
consequences. And as someone who had extensive training in behavioral
technologies, I could suggest that you chart your behavior and determine
how often it has been successful. If you wish to define "success" as
getting people to respond to you in this newsgroup, ok, but I would hope
that you would set more worthwhile goals. And on those goals, you have not
yet succeeded at all, by your own statements.
Leslie E. Packer, PhD
On Jul 24, 1996 20:46:57 in article <Re: Another debate between BRAD and
>is "grassroots" everyday.
because you don't know what you're doing.
Sometimes you have to break a few eggs to make an omelet. But try breaking
them into the frying pan instead of throwing them at people -- the results
are generally much more satisfying <g>.
Cognitee
Dear Leslie,
What is all this talk of court? What makes court so significant.
Power status? What? What wrong with the grassroots operation and
facilatation and coooperation and specialization I outlined? -- b
jesness
Leslie E. Packer, PhD
On Jul 24, 1996 14:46:38 in article <Re: Another debate between BRAD and
>significantly better and has the promise of being developed so it works
>even better. This is the point of research. Some very useful research,
>"proven" useful, can be still be seen to have confounds. But this is
>sometimes certainly (at least relatively) trivial.
--
dull" rats? Because the history of that line of research somewhat
contradicts your point above.
How about the results of thalidomide? That sure looked good and helpful
when it was first approved, right?
Sorry... but I tend to be much more conservative before offering things to
the public. And when I do experiment with/on my patients, it is generally
because all other options have failed, and it is with the patient's and the
referring MD's full information and consent.
Leslie
----
Leslie E. Packer, PhD
Leslie E. Packer, PhD
On Jul 24, 1996 19:45:22 in article <Re: Another debate between BRAD and
Seligman, with BRAD the victor>, 'Ed Anderson <eand...@cougarnet.byu.edu>'
wrote:
>Yes, but let's see you get "good observations," "excellent inter-rater
>reliabilities," or any sort of validity without a good research design.
Ed,
He seems to be stuck on inter-rater reliability as if that's the key to
solving some of these problems. You, of course, realize how far beyond
that we already are in some areas. Heck, my goal is to do away with the
need for observers/raters altogether <g>.
Leslie
--
Leslie E. Packer, PhD
On Jul 24, 1996 18:08:01 in article <Re: Another debate between BRAD and
>on the question of "paras" vs. "pros" only bolsters my point that there
>are studies that need to be done. It defies reasoning (though makes
>sense when viewed as EXTREME defensiveness) that these have not yet been
>done.
be done. And it doesn't defy reasoning at all if one considers the ethical
mandates of the various health care organizations. You're also
conveniently neglecting the role of Congress and the insurance industry and
the other professional organizations. Curiously, you keep attacking only
psychology. Wonder why.... did you ever apply to a doctoral program in
psychology and get rejected from it or something?
> On the other hand, Strupp and Hadley's study used multiple
>measures (both in time and otherwise) (many blind) with a great
>consistency in the result. Furthermore there sample size was adequate.
knows the first thing about power analysis would ever declare their sample
sizes as adequate to demonstrate "no difference" between two conditions.
I'm sorry, you can yell and scream all you want, but that doesn't change
the reality of what they did and what they found. You are trying to make
something out of their research that they explicitly negated themselves.
>Their sample group was well definded (and can be considered a "broad
>range of clients")
broad range of clients by any respectable scientist's definition of
sampling techniques. May I ask you what your credentials in research
design and statistics are? How many graduate courses in design and
statistics did you actually take, and where did you take them?
>and could be used in future (PRESENT ?) research.
students, then perhaps. But they do not represent a broad range of
problems, etc. That's not necessarily bad, but the limitations would need
to be kept in mind.
>I will agree with you that people should not relay much on
>non-professionals now. This role must be defined and brought "within
>the system."
you know what the answer will be and that it will be that they are suitable
or can be made suitable. Why bother with research at all, then?
>Clinical psychology's failing here has been an outrage. I
>am as mad as hell and I'm not going to take it anymore (your damned
>frinki' lip-service aside!!).
something that is rightfully yours and by evil forces, to boot. What is it
you really want? The right to be able to offer services to others and to
charge them money for it even though you have no real training to do so?
Is that what this all boils down to? You can't seriously expect anyone to
believe that your concern is for patients.
>You also make great presumptions about what "quality care" is.
I actually <cyberthrongs gasp> record objective data/measures during each
session so we can evaluate progress.
I also know when large segments of our society are being discriminated
against and when they are not receiving care that they need. And I know
that I would not for a minute allow anyone I cared about to be schloffed
(sp?) off on some nonprofessional if they had a real problem.
>You are
>failing in your purpose. You may see yourself as the end-all-be-all,
>but it's bullshit to me.
single person have been surprisingly (even to me) effective and are now
being expanded to a national level. But for all your rhetoric, you've
seemingly accomplished nothing... even with your group ("BRAD").
> And, Leslie, because you present no clarity
>this is precisely what you are doing: insisting on the status quo.
that would elucidate my position or what I would propose. Furthermore, my
advocacy efforts directly contradict your assertion that I insist on the
status quo. I'm the one who developed and proposed a very clear model to
address unmet needs and got that model implemented. Included in those
efforts are months of discussions with both the APA and NASP ethics
committees about improving standards and quality of care for school
children with neuropsychiatric disorders. Finally, if I insisted on the
status quo, I never would have gone to court to obtain the necessary court
orders, would I have?
Like I said, you and I are not in the same league. You seem to be one of
those people who whine and throw tantrums. I prefer to take effective
action. And the results speak for themselves.
>Finally, you say (quoting): "What also really concerns me is that in
>some sense, I think your actions
>actually contribute to the stigma associated with mental health
>problems." (end quote) No, my presentation contributes to the stigma
>associated with *your field* (and YOUR FIELD *AS IT IS* KEEPS THE STIGMA
>ABOUT MENTAL HEALTH PROBLEMS (i.e. diagnoses) VERY MUCH ALIVE).
and I don't know what the heck you think my field even is. But I will
tell you this, because you seem very badly out of touch with most of the
current research on mental health problems: every year, more and more
research identifies biological and nonenvironmental factors implicated in
problems that were originally thought to be psychogenic. This does not
mean that treatment or help for all these disorders must be or should be
medical, but it does mean that society needs to rethink some of its views
about people with "mental illness" and recognize that these are not just
people who lack self-control or willpower.
And to the extent that there are medical issues involved, any attempt to
use paraprofessionals is unwarranted and contraindicated, IMO. If you
would not use a paraprofessional to treat epilepsy, then you shouldn't
attempt to use a paraprofessional to treat a biologically rooted behavioral
problem. And before you assume anything else incorrectly, I will tell you
that I also had some medical training and held some medical certification.
And I will also tell you that I have had some patients who had very serious
neurological problems that I detected -- problems that a paraprofessional
never would have even recognized.
Finally, even if one looks at the more recent research on dysthymia (e.g.,
the NIH study), it is clear that there are large overlooked segments of our
population that need help. Until we have developed effective protocols for
helping them, it makes no sense to even consider trying to use
paraprofessionals... what on earth would we tell the paraprofessionals to
do? Just listen and be empathetic? Hell, these people have had that
already in their lives and it didn't work. They need real and professional
care.
>Well, I
>frankly think you deserve your stigma ! Your field frankly, simply HAS
>NOT DONE THE STUDIES TO ALLOW YOU DO DEMONSTRATE THAT YOU HAVE "high
>level expertise" OR EVEN ANY *PARTICULARLY* SPECIAL ABILITIES for many
>counseling cases.
problems you think falls within the domain of counseling as opposed to
psychotherapy or some other field?
> Your entire position is so unclear we do not know
>comprized of charlatans and cowards.
If you don't know my position it is because you haven't asked me specific
questions that would elucidate it. Don't assume that you know what I think
on anything.
> I shall promote the
>paraprofessional option as long as I and others (in the footsteps of
>Carl Rogers) view it as a reasonable option.
promoted the idea in the absence of research demonstrating efficacy and
protection of the patient's well-being.
>
>P.S. When you realize, as Allen Ivey does, that the counseling/therapy
>field is operating "in a narrow fog." A very apt metaphor.
(which may be more relevant sometimes than the degree, since you can have
PsyDs who practice CBT vs PsyDs who practice psychoanalytic approaches).
Cognitee
P.S. Leslie,
I might add that I believe much better reliable diagnostic systems,
starting from the DSM and remaining as close as appropriate, could be
arrived at rather easily with "in-house" (local) science practitioners
working as I have outlined. That this has not been done is one of the
major disgraces of clinical psychology. -- b jesness
Cognitee
Dear Leslie,
The "para" vs professional research is essential basic foundation
research. Without this question answered many major questions cannot be
answered: What different helper roles should exist and what sort of
mental health care SYSTEM should there be? AND: Where do we need to
have people especially professionally trained. Can you think of any
more important questions ????? See my point: basic FOUNDATION
RESEARCH !! lordy -- b jesness
Cognitee
Dear Leslie,
I have read "Tactics... " YEARS AGO. I respect the true experiment,
for its clarity BUT I feel VERY favorably about quasi-experimental
designs and important correlational studies ALSO ! I'd often take a
very strong correlation over a marginally significant experiment (or
even over an experiment with "very "significant" results", if dependent
on a large N). Insisting on the true experiment as the only thing
that's good is certainly true *only* in certain contexts (AND NOT
OFTEN). I was topic of my graduate class in research methodology. -- b
jesness
P.S. I've expressed some regard for single sugject designs elsewhere.
In today's mental health care system, I guess I love them, though
reversing conditions seems UNFORTUNATE. It would be often unnecessary
IF ONLY ONE HAD AMASSED DATA THAT SPOKE WELL GENERALLY TO A
WELL-IDENTIFIED (defined) GROUP OF PROBLEMS.
Cognitee
Hey, Leslie:
I am in a leadership role. You just don't realize it yet. -- b j
Peter Hood
<delurks>
In article: <31F6DB...@coil.com> "John M. Grohol Psy.D." <gro...@coil.com> writes:
> It is more than mildly hypocritical of you, Mr. Jesness, to talk
> of "science" all the time and have done so little yourself.
> As many people have pointed out many times in the past, why don't
> you get off your ass and do some? It's one thing to sit there and
> bitch and complain for months (almost a year now) on an Internet
> newsgroup and quite another to get funding and subjects to actually
> do all the "science" you claim is needed. If you're such the
> expert in science, Mr. Jesness, you should truly be leading the
> field in directing others and obtaining the means to do the research.
> I'd very much like to see you in such a leadership role in this
> field.
> Last time I looked, research doesn't grow on trees or fall out
> of the sky. Or get done by harassing others to do it.
" 'Tis the voice of the Para; I heard him declare,
'You have baked me too brown, I must sugar my hair.'
As a duck with its eyelids, so he with his nose
Trims his belt and his buttons, and turns out his toes.
When the sands are all dry, he is gay as a lark,
And will talk in contemptuous tones of the Thera-Shark,
But, when the tide rises and thera-sharks abound,
His voice has a timid and tremulous sound.
I passed by his garden, and marked, with one eye,
How the Student and the Professor were sharing a pie.
The Professor took pie-crust, and gravy, and meat,
While the Student had the dish as its share of the treat.
When the pie was all finished, the Student, as a boon,
Was kindly permitted to pocket the spoon;
While the Professor received knife and fork with a growl,
And concluded the banquet -" ...
That'sa spicy para-meata-balla, as all Alice afficionados will know.
Still laughing in my sleep.
--
Peter
<relurks>
Cognitee
Dear Mark Moore,
One must appreciate that "significance" is a statistical concept
having to do only with the statistical likelihood of finding the
between-group difference again (either 99 or 95 times out of 100). This
has to do with: group variabilities, difference in means, and group
size. It's easier to find differences reliably, that is again and
again, with large groups even if the difference is small. (see footnote)
In fact, if very large groups are used very, very small differences
between the group means can be found reliably (and this is all
"statisical significance" means). It is possible that I could get
"significant result" showing that eating beans affects school
performance (after lunch on a standardized test of some kind) in a
controlled study if I had 10,000 children in each group. So you see,
"statistical significance" does not equal (or mean) meaningfulness (in
a practical sense) UNLESS realistic sizes are compared.
Correlation coefficients are better indicators of absolute STRENGTH
of the actual results or findings. They do not rely on group size
(though the certainty of the exact value of the correlation increases
with group size). Correlation coefficients could be reported with
experimental results and probably should to allow for clearer
comparisons of strenth of results. (Correlation coefficients can be used
with experimenatal designs as well -- it is a statistic that is
calculated and is not linked up with a particular design). Hope this
helps. -- b jesness
** Here's a very apt analogy: It's easier to tell a meter from a yard
than it is to tell a 1/2-inch from a centimeter.
Cognitee
Dear Mark Moore:
You say: " It may not show up in a journal, been funded by a grant,
or
presented at a conference. However, on a case-by-case basis, it shows a
theory of what the problem is, a course of outline for treatment using
validated techniques, and the use of repeated objective measures to
assess
progress (or lack thereof). Again, just a little reminder that the
issue
of effectively evaluating treatment is often done one client/patient at
a
time and administered scientifically." (end quote)
Bravo, I have nothing much against this (and nothing necessarily). I
must caution though: One must be systematically sharing (passing on
results) and one should also be working to develop better inter-rater
reliabilities for diagnoses, methods, and treatments. This way YOUR
results will matter to others too (not only because you are working with
them, but because you are showing quality). Also developing diagnostic
systems and treatments means you are *NOT* just using what has been
"handed down." Otherwise, though, your point is well-taken. -- b
jesness
Leslie E. Packer, PhD
On Jul 25, 1996 05:17:56 in article <Re: Another debate between BRAD and
I could think of any more important questions? Sure, I can. For starters:
1. What type of alternative classroom environment(s) do we need to develop
so that children who have ADHD can function to their potential without
medication?
2. Some SSRIs that are highly selective for the 5HT-1A receptor have been
demonstrated to help reduce alcohol consumption in animal models. Can we
develop a comprehensive program of medication and environmental support
that will reduce alcoholism, a disease that is extremely costly to our
society and that wrecks a lot of lives?
3. What child-rearing and/or school-based experiences are effective in
promoting the "just say no" attitude towards drugs in our children?
4. With so many people dying of AIDS, how do we effectively promote safe
sex?
Shall I continue? Your concerns are not necessarily my concerns or
priorities, Brad. And my priorities and concerns are probably no one
else's.
So instead of screaming that people are being defensive, biased, etc. just
because they don't jump on your bandwagon, perhaps you might step back and
try to see other perspectives.
I am more concerned in seeing that everyone who needs help has access to
the system. But I am not willing to compromise and say that means that we
start by giving them less than professional help.
Leslie
Leslie E. Packer, PhD
On Jul 25, 1996 04:45:47 in article <Re: Another debate between BRAD and>Dear Leslie,
>Choosing professors by reputation is hardly necessarily "optimising
>..the "paraprofessionals"." For one thing they weren't even
>paraprofessionals, because they weren't trained. Also: reputation is
>not as good as the selection procedures anyone has in mind for "paras."
Oh. So now _you're_ claiming that the S&H study wasn't relevant for the
comparison you want to make? Interesting, since up until now you were
citing it as the last of three "controlled" (your word) studies on the
issue _you_ want to look at.
I do agree with you though that selecting by reputation does not
_necessarily_ optimize the "para" condition. It is a possibility, however,
particularly since I have not seen you list any specific screening
criteria that you think should be employed beyond being a mature,
presumably well-adapted adult.
>Just how badly does bias and defensiveness cloud your judgement?
Not badly at all, thank you. And you? <laughing> You apparently are still
failing to recognize that what you are proposing is in no way competition
for me professionally or economically. My only vested interest is in
protecting patients. Maybe if I repeat it another 20 times or so, it will
sink in. Then again, maybe it won't <g>.
>Well-adapted mature people in general counsel others frequently (not
>just professors).
Look, there are all kinds of people out there doing counseling with all
different levels of training. Religious figures frequently engage in
counseling without any formal training, and many do a good job. The
problem is that who determines whether a lower level of training will be
adequate and sufficient to help a particular individual if there is no
evaluation/diagnosis first by a highly trained professional who can rule
out other problems. And who regulates these paras to protect the patient.
I am aware of one case right now where a pastoral counselor is engaging in
a sexual relationship with his "patient." If this creep was a licensed
professional, there would be charges against him. But as someone who isn't
licensed, there is no real legal recourse. And he has done tremendous harm
-- and continues to do harm -- to a very fragile person.
>Also recall that it is not only to give "paras" jobs that this sort
>of helper needs to be examined and compared to professionals in
>controlled studies.
Whoa. I feel _no_ obligation to make sure that "paras" have jobs. Funny
thing about it is I was taught that if you wanted a job you learned the
skills that were necessary to acquire the job. I was not taught to expect
people to lower their standards so that I could find work. If people want
to do professional counseling, then they should take and complete a
professional training program.
>It is to find out where special professional
>services are most needed as well. Let's try to think of the client and
>not just our turf. OK ?
Why "most" needed? Are you buying into that allocation of limited
resources garbage again?
And I assure you that I _am_ thinking of the client. But the more you
write, the more you show your true colors, Brad, and I am becoming
increasingly convinced that you are _not_ thinking of the client but of
your own ego and those of a lot of wannabes. Instead of bringing your
level of expertise up, you have taken the tack of trying to discredit
those who hold the credentials and experience you don't. Chac'un a son
gout, but by your very posts and comments on research in this NG you have
more than amply demonstrated my point that those without advanced training
cannot be counted on to read research journals with an appropriately
critical eye. And that puts the patients at risk.
>
>P.S. If Strupp and Hadley is not ideal work (and it is not) it only
>begs more for the work to be done.
>
And they did it. Take a look at the rest of the studies Strupp published
on the topic.
Leslie E. Packer, PhD
On Jul 25, 1996 04:47:31 in article <Re: Another debate between BRAD and
Seligman, with BRAD the victor>, 'Cognitee <Cogn...@aol.com>' wrote:
>What is all this talk of court? What makes court so significant.
>Power status? What? What wrong with the grassroots operation and
>facilatation and coooperation and specialization I outlined?
life as an activist, usually at the grass roots level. But there comes a
time when you look at a situation and see what your options are to make the
system work for you. But I certainly wouldn't characterize your approach
as facilitation, cooperation, and/or specialization. All I've seen you do
is attack psychologists here.
As to the other part of your question, I couldn't give a fig for power
status. When I go after something, it's to accomplish some goal -- to get
something implemented or changed.
Without going into details, I will tell you one experience I had. I'd
spent two years working on something at a local level, and had gotten
stonewalled at every turn. Following policies, etc., I took my complaint
the next step up the ladder, etc. Each time, I got nowhere -- even though
I knew I was right and they were wrong. Finally, I decided to go into
federal court (thereby bypassing a couple of steps <g>). Within one week
of submitting the documents, we were before the federal judge. He never
even asked for one witness. He merely read all the affidavits and
supporting documents we submitted, turned to the other side, and ordered
them to do it immediately. That was it. It was all over but ironing out
the details. Two years of grass roots paid off in the sense that I had
developed quite a paper trail to prove my case, but the power of the court
to help a child was something that will be with me always. And judges have
a tremendous amount of discretionary power when it comes to the welfare of
children.
Since that case, a lot of things have changed. Not the least of which is
that the state learned that I was dead serious and that I didn't make idle
threats about court actions. As a consequence of that, they started paying
more attention to the issues I raised, and now, in a complete turnaround,
have not only asked me to help train educators throughout the state, but
have started their own efforts to train educators and to support the
development of new programs (the original program being one I developed and
that the state approved for implementation).
The beauty of it all is that now advocates in other states can point to us
and say, "see.... another state already does this," etc. And that's how
things spread.
So you can continue to spin your wheels here, Brad, or you can take the
same time and energy and try to channel it into something that will be
productive.
--
Leslie
Leslie E. Packer, PhD
Leslie E. Packer, PhD
On Jul 25, 1996 04:52:15 in article <Re: Another debate between BRAD and>Leslie,
>You do not even have excellent inter-rater reliability for step one: >in
diagnosis. Validity issues to a good extent hinges on that. "Good" is
>the best that is done with inter-rater reliabilities in diagnosing and
>that is likely for less than half of diagnoses, other diagnoses show
>poor to fair relaibility.
But validity does not need to hinge on inter-rater reliability. There are
other ways to deal with the validity of the diagnosis, for example. You
seem stuck in a very narrow way of designing research.
> You are beyond nothing, not even bullshit.
>Ignorant pretentious fool.-- b jesness
You seem to have a real problem with accepting limits, Brad. I had told
you that I had no desire or willingness to continue dialoguing with you if
you continued to emit offensive verbal behaviors, such as the personal
attack above.
I'll pass on responding to the rest of your posts. You're losing it,
kiddo. I hope you feel better soon.
Cognitee
Leslie,
Carl Rogers DID promote the idea of using paraprofessionals. Yet, I am
happy to see evidence gathered of their utility. QUESTION IS: ARE YOU?,
AND YOUR FIELD BE WILLING TO SUPPORT THIS EFFORT? Or do presumptions
outweigh scienctific respectibility? -- b jesness
Cognitee
And Leslie,
Given the conduct of clinical psychology, it would be lucky if we found
more and more and more problems that are not psychogenic !! -- b jesness
Cognitee
No Scot,
The professionals vs. the professors showed a 0.2 s.d. difference in
"effect size" in SOME set of better studies whose results FAVORED the
professionals (such "favoring" was not true overall of all the best the
studies in the report).
By the way: Clinicians show only about 0.5 (and at most a .7) effect
size (or difference in s.d units) FROM WAITLIST CONTROLS with their
treatment of conditions (ON AVERAGE). And "therapy" shows only about half
that when treatment is compared to placebo -- b jesness
Cognitee
Leslie,
For the record several respected psychologist have seen Strupp and
Hadley's sample as a broad range (you can find that with college
students). BUT I do not find it necessary to further debate Strupp and
Hadley to debate the issue of needed research and why it's needed. Must
we? -- b jesness
P.S. I do somewhat like your single subject research outlook at times and
your attitude towards being responsible in client assessment. Too bad
that is all I like about you
Cognitee
And Finally., Leslie:
Your concern about "no
evaluation/diagnosis first by a highly trained professional " is such a
joke. It is a joke because of the low inter-rater reliabilities shown by
professionals in using the DSM diagnoses and it is a joke because very
possibly most educated lay people could read the criteria and do as well
diagnosing (though I don't think studies have yet been done here either).
The criteria are at the same time both simple and vague. Any intelligent
person who has examined the DSM at a bookstore knows it is patently
non-technical. You simply have no basis to claim any special prowess in
diagnosis as a professional. It is yet another incredibly ridiculous,
pompous, pretentious CLAIM.
AND, what makes you think there are less creeps among Ph.D.s It is
all myth scare tactics and nonsense. -- b jesness
jupiterbowl
In article <4t5imf$2...@news1.t1.usa.pipeline.com>,
Leslie E. Packer, PhD <lpa...@nyc.pipeline.com> wrote:
>4. Additionally, didn't Strupp and Hadley report that the group means were
>misleading because of the large within-group variability? Again, this
>would argue for a real problem with drawing _any_ conclusions from their
>study.
I think I can confirm this. Strupp & Binder (1984; _Time Limited
Dynamic Therapy, Basic Books, I think) commented on this program of
research and did mention that they suspected the large within group
variability in outcome prevented between group differences from being
detected.
Another interesting comment they made was the reports of the
untrained college professors discomfort at running out of material to talk
about before the end of the allotted sessions.
>If I followed your argument, the small effect size (which you estimated at
>2 SD but suggest might be an overestimate) shouldn't be considered
Are you all talking about 2.0 SD? As in (control group mean) -
(treatment group mean)/sd(control-group scores) = 2.0? If I'm reading
this right, this is a huge effect size! (thinking of Cohen's d here)
>important because it is small. Was I understanding your correctly?
^^^^^?
>been in the other direction. There is an important difference between
>statistical significance and clinical significance, but one cannot truly
>have the latter if the former does not exist. So demonstrating a reliable
I don't think I understand this Dr. Packer: are you saying that
statistical significance is necessary for clinical significance? If you
mean that reliability of measurement and reliability of effect are
necessary to show clinical significance then I'm sure I agree.
But if you really mean statistical significance, as in some p <
alpha, I think I disagree. Since statistical significance is directly
influenced by one's decision about the level of Type 1 error one is
willing to make in one's inferences on the basis of a particular design
before the fact, and clinical significance is a judgment about the amount
of change in a patient or group of patients determined after the fact, I
think these are apples and oranges.
I imagine a counterexample where a small, but reliable, effect
would not attain significance due to low power in the design because of a)
small samples b) small Type 1 error rate, and/or c) high within-group
variability due to other reasons.
>than values such as "extremely highly trained vs. Zero training), they have
>produced a _lower_ estimate of the effect of the independent variable. Had
>they actually selected the levels properly (by your definition of what
>levels they should have used), it is more than likely that there would have
>been an even _greater_ effect (in the direction of trained >
>paraprofessionals).
This is a really interesting point.
>Regards,
>
>Leslie
>---
>
>Leslie E. Packer, PhD
Really interesting review, thanks.
Scot
--
\----/ \----/ \----/
|||| Scot McNary |||| sc...@wam.umd.edu ||||
|||| Graduate Student, UMCP |||| (bethany too) ||||
/----\ /----\ /----\
Cognitee
By the way, Leslie:
I do much more to suggest good research, having large publications on
applications of ethology concepts and methods to psychology. You'all
for the most part are not at the point where this would be meaningful to
you. Some counseling psychologists right now are in the process of
integrating their systems with an ethological view for assessment and
treatment, etc.-- b jesness (early human ethologist)
Ed Anderson
sc...@wam.umd.edu (jupiterbowl) wrote:
>....Another interesting comment they made was the reports of the
>untrained college professors discomfort at running out of material to talk
>about before the end of the allotted sessions.
Yes, and in the Strupp (1980) articles, there is the report of the
professor/"parapro" who got locked in a power struggle with an angry,
resistant client and kept trying to feed him advice (condescendingly).
> Are you all talking about 2.0 SD? As in (control group mean) -
>(treatment group mean)/sd(control-group scores) = 2.0? If I'm reading
>this right, this is a huge effect size! (thinking of Cohen's d here)
That was a typo, I think, Scot -- it was supposed to be 0.2.
ed
Ed Anderson
cogn...@aol.com (Cognitee) wrote:
>....By the way: Clinicians show only about 0.5 (and at most a .7) effect
>size (or difference in s.d units) FROM WAITLIST CONTROLS with their
Boy, no matter how many times we go over this territory, you just can't
get it right, can you?
0.82. Memorize it.
ed
ps, and Leslie, none of your snide remarks about meta-analyses <s>.
Cognitee
Leslie,
Exceptions occur for exceptional reasons. Sometimes we overlook
important confounds.
What's this about experimenting on patients? Single subject?
Single subject is a neglected design, yet one the individual
practitioner can do himself quite practically. Since practitioners
often essentially work alone, there should likely be more of this (if
you want to claim to be a science practitioner with the present system).
-- b jesness
Ed Anderson
Brad,
I am continually struck by how nonresponsive you are to the very good
points Leslie raises. She takes care to quote you at length and respond
specifically to each of the issues you raise. Why do you not
do the same? Why do you repeatedly fail to respond substantively to her
critiques? [rhetorical question]
ed
Cognitee
And, Leslie,
Unfortunately BECAUSE OF DEFIECIENCIES IN YOUR DAMNED FIELD, Strupp and
Hadley is among the very BEST evidence we have. -- b jesness
Cognitee
Oh, Leslie,
Strupp and Hadley is still relevant, though it does not perfectly or
exactly address the issue, does it? -- b jesness
Leslie E. Packer, PhD
On Jul 24, 1996 20:09:14 in article <Re: Another debate between BRAD and
Seligman (long post)>, 'Ed Anderson <eand...@cougarnet.byu.edu>' wrote:
>That's a good point. The size of the treatment groups were 16 and 15. A
>couple other things about that oft-cited study:
>
>1. The "paraprofessionals" in Strupp & Hadley were university professors
>specially selected by students "on the basis of their reputation for
>warmth, trustworthiness, and interest in students." These were not even
>average professors, much less average paraprofessionals.
Right... so there was some "stacking" or optimizing in terms of the
paraprofessionals used. But that doesn't concern me so much as long as we
recognize that these "paras" had been selected this way. We also need to
realize that as university professors who routinely provide academic advice
and counseling to their students, most of these professors already had some
experience in counseling. As a former academic, I know that students
frequently drop in to talk when they're upset or in trouble if they
like/trust the faculty member. Hence, these "paras" _may_ not have been
equivalent to "new" paras in their abilities.
>
>3. The measures used were not particularly sensitive to change (e.g.,
>MMPI).
I couldn't remember the measures. Thanks, Ed.
>
>4. While no significant differences appeared between pros & "parapros,"
>differences in group means were all in the expected direction.
>
Which supports the concern about the power of the design and the tests.
Thanks again.
Leslie E. Packer, PhD
On Jul 24, 1996 21:53:53 in article <Re: Another debate between BRAD and
Seligman (long post)>, 'jcl...@cts.com (John Clark)' wrote:
>
>
>But then what of book writers who base theories on one or two patients,
>who are not in any way 'picked at random'.
John,
There needs to be some relationship between the type of design and the size
of sample needed.
If you think back to some of the best work done (in the sense that it's
stood up to time and generated much meaningful work), you can think of
Fechner and Ebbinghaus. Neither researcher used group designs. They
studied _one_ person under a variety of conditions with direct and
systematic replications built in to their design. When you have the
opportunity to really study one individual that way _and_ are
experimentally controlling the sequence of conditions and controls, etc.,
you actually have more information than you might by studying 1000 people,
each under only two conditions.
Part of the problem in selecting the design is to fit the design to the
nature of the question being posed, and this is an all-too-common failing
in research, IMO.
That may be more than you wanted to know, but I hope it answers your
question. If not, ask again.
Mark Moore
On Jul 25, 1996 05:13:10 in article <Re: Another debate between BRAD and
>I have read "Tactics... " YEARS AGO. I respect the true experiment,
>for its clarity BUT I feel VERY favorably about quasi-experimental
>designs and important correlational studies ALSO ! I'd often take a
>very strong correlation over a marginally significant experiment (or
>even over an experiment with "very "significant" results", if dependent
>on a large N). Insisting on the true experiment as the only thing
>that's good is certainly true *only* in certain contexts (AND NOT
>jesness
>
>In today's mental health care system, I guess I love them, though
>reversing conditions seems UNFORTUNATE. It would be often unnecessary
>IF ONLY ONE HAD AMASSED DATA THAT SPOKE WELL GENERALLY TO A
>WELL-IDENTIFIED (defined) GROUP OF PROBLEMS.
>
>
correlation over .... significant results." Surely you know that one
cannot tell cause and effect with a correlation (or have I totally missed
your point?).
BTW, my newsreader appears to be delayed so if I've been "behind" in my
couple of responses tonight (i.e., I'm stating something that has already
been addressed or responded to), sorry.
Thanks.
Mark Moore
Mark Moore
On Jul 25, 1996 04:52:15 in article <Re: Another debate between BRAD and
>You do not even have excellent inter-rater reliability for step one:
>in diagnosis. Validity issues to a good extent hinges on that. "Good" is
>the best that is done with inter-rater reliabilities in diagnosing and
>that is likely for less than half of diagnoses, other diagnoses show
>
>
"cooperation" that was noted in your previous post?
Just to add my two cents in here, there are many Ph.D.'s (including myself)
that adhere very strongly to the scientist-practioner model of
psychotherapy. It may not show up in a journal, been funded by a grant, or
presented at a conference. However, on a case-by-case basis, it shows a
theory of what the problem is, a course of outline for treatment using
validated techniques, and the use of repeated objective measures to assess
progress (or lack thereof). Again, just a little reminder that the issue
of effectively evaluating treatment is often done one client/patient at a
time and administered scientifically.
Mark Moore
Cognitee
Dear Leslie,
Choosing professors by reputation is hardly necessarily "optimising
..the "paraprofessionals"." For one thing they weren't even
paraprofessionals, because they weren't trained. Also: reputation is
not as good as the selection procedures anyone has in mind for "paras."
trained would be selected. Also: academic counseling is not especially
good practice often.
of helper needs to be examined and compared to professionals in
Cognitee
But, Leslie,
My concern is linked up with many of yours. If we do not know what
the difficult problems are (by presuming, in effect, that they are all
difficult) and do not know what sort of help and helpers are really
required, we cannot best answer many of the questions you raise or solve
the problems practically (or, in fact, AT ALL). You really don't believe
that one-type of helper can do everything do you?? Things must be
sorted out more than that. Or, indeed you are operating in what Ivey
calls the "narrow fog." You're "BIG," but confused. Finding the degree
of help needed and the nature of help that may be provided by others in
other mental health helper roles helps both define and deal with the
problems. (This is true of your: 1, 3 and 4. 3 is a medical question,
where neither you NOR I are needed; M.D.s can manage to do experiments
without our help.)
So you see: IT'S THE WORK I OUTLINE THAT IS FOREMOST IN IMPORTANCE.
I still see you (and the APA in general) defending your turf only at the
expense of clients.-- b jesness
Cognitee
Ed Anderson
lpa...@nyc.pipeline.com(Leslie E. Packer, PhD) wrote:
>...Ed... if you still have the article handy, do me a favor: how did they get
>the college students who participated? Can you remind me of the recruiting
>procedure?
Sure, let's see...(fumbling through pages)...Five came from applicants to
the local counseling center; the rest were recruited via "letters
announcing the availability of therapy" which were "sent to random
samples of the male student body. Respondents were then screened for T
scores above 60 on the three MMPI scales mentioned earlier; also, they
had to be between 17 and 24, male, single, and undergraduates.
Hopes this helps to put Brad's statement re. a "broad range of clients"
into perspective. S & H themselves note that they made a "concerted
effort to obtain a truly homogenous patient population..." (p. 1126)
ed
Leslie E. Packer, PhD
On Jul 25, 1996 05:45:18 in article <Re: Another debate between BRAD and
Seligman, with BRAD the victor>, 'wmm...@usa.pipeline.com(Mark Moore)'
wrote:
>On Jul 25, 1996 05:13:10 in article <Re: Another debate between BRAD and
>>I have read "Tactics... " YEARS AGO. I respect the true experiment,
>>for its clarity BUT I feel VERY favorably about quasi-experimental
>>designs and important correlational studies ALSO ! I'd often take a
>>very strong correlation over a marginally significant experiment (or
>>even over an experiment with "very "significant" results", if dependent
>>on a large N). Insisting on the true experiment as the only thing
>>that's good is certainly true *only* in certain contexts (AND NOT
>>OFTEN). I was topic of my graduate class in research methodology. -- b
>>jesness
>>
>>P.S. I've expressed some regard for single sugject designs elsewhere.
>>In today's mental health care system, I guess I love them, though
>>reversing conditions seems UNFORTUNATE. It would be often unnecessary
>>IF ONLY ONE HAD AMASSED DATA THAT SPOKE WELL GENERALLY TO A
>>WELL-IDENTIFIED (defined) GROUP OF PROBLEMS.
>>
>>
>In your statement above, you stated that you would "take a very strong
>correlation over .... significant results." Surely you know that one
>cannot tell cause and effect with a correlation (or have I totally missed
>your point?).
Nope, you didn't misread him. Amazing, huh? And don't you just love his
statement about reversing conditions? Shows a real... how shall we say
this diplomatically.... lack of sophistication about research? And based
on these impeccable credentials, he naturally presumes to tell people how
to run research <laughing>.
Pull up a chair, Mark, and put up your feet. By the time you get done
reading the rest of his posts, you'll probably be dizzy <G>.
Leslie E. Packer, PhD
On Jul 25, 1996 05:34:37 in article <Re: Another debate between BRAD and>On Jul 25, 1996 04:52:15 in article <Re: Another debate between BRAD and
>>You do not even have excellent inter-rater reliability for step one:
>>in diagnosis. Validity issues to a good extent hinges on that. "Good" is
>>the best that is done with inter-rater reliabilities in diagnosing and
>>that is likely for less than half of diagnoses, other diagnoses show
>>poor to fair relaibility. You are beyond nothing, not even bullshit.
>>Ignorant pretentious fool.-- b jesness
>>
>>
>How does the name calling above support your call for "facilitation" and
>"cooperation" that was noted in your previous post?
>Just to add my two cents in here, there are many Ph.D.'s (including
myself)
>that adhere very strongly to the scientist-practioner model of
>psychotherapy. It may not show up in a journal, been funded by a grant,
or
>presented at a conference. However, on a case-by-case basis, it shows a
>theory of what the problem is, a course of outline for treatment using
>validated techniques, and the use of repeated objective measures to assess
>progress (or lack thereof). Again, just a little reminder that the issue
>of effectively evaluating treatment is often done one client/patient at a
>time and administered scientifically.
Well put, Mark.
Cognitee
Dear Leslie,
Do I have to repeat my brief statistics lesson FOR YOU ?? Well, here
it is:
One must appreciate that "significance" is a statistical concept
having to do only with the statistical likelihood of finding the
between-group difference again (either 99 or 95 times out of 100). This
has to do with: group variabilities, difference in means, and group
size. It's easier to find differences reliably, that is again and
again, with large groups even if the difference is small. (see footnote)
In fact, if very large groups are used very, very small differences
between the group means can be found reliably (and this is all
"statisical significance" means). It is possible that I could get
"significant result" showing that eating beans affects school
performance (after lunch on a standardized test of some kind) in a
controlled study if I had 10,000 children in each group. So you see,
"statistical significance" does not equal (or mean) meaningfulness (in
a practical sense) UNLESS realistic sizes are compared.
Correlation coefficients are better indicators of absolute STRENGTH
of the actual results or findings. They do not rely on group size
(though the certainty of the exact value of the correlation increases
with group size). Correlation coefficients could be reported with
experimental results and probably should to allow for clearer
comparisons of strenth of results. (Correlation coefficients can be used
with experimenatal designs as well -- it is a statistic that is
calculated and is not linked up with a particular design). Hope this
helps. -- b jesness
** Here's a very apt analogy: It's easier to tell a meter from a yard
than it is to tell a 1/2-inch from a centimeter.
NOW THERE ISN'T TOO MUCH "AMAZING" ABOUT WHAT I SAY, HUH? -- b jesness
Leslie E. Packer, PhD
On Jul 24, 1996 19:29:18 in article <Re: Another debate between BRAD and
Seligman, with BRAD the victor>, '"John M. Grohol Psy.D."
<gro...@coil.com>' wrote:
>
>As many people have pointed out many times in the past, why don't
>you get off your ass and do some? It's one thing to sit there and
>bitch and complain for months (almost a year now) on an Internet
>newsgroup and quite another to get funding and subjects to actually
>do all the "science" you claim is needed. If you're such the
>expert in science, Mr. Jesness, you should truly be leading the
>field in directing others and obtaining the means to do the research.
>I'd very much like to see you in such a leadership role in this
>field.
<spitting out my coffee, laughing>
John,
_Please_ tell me that you're not serious or that some unapproved person got
hold of your account. You wouldn't really like to see him in a leadership
role, would you? I can just imagine the quality of the research he would
produce....
<laughing hysterically.... can't type...wait....wait....>
<smack> OK, I'm back now.
<no, I'm not.... I can't stop laughing at the idea.... sorry....><G>
Cognitee
And Leslie,
Given one does not have evidence of the nature and reliablilty for an
effect of a treatment from generally available data: in a single
subject design to see if the effect was due to your intervention, you
would reverse conditions A-B-A. Anything factually wrong here? -- b
jesness
Leslie E. Packer, PhD
On Jul 25, 1996 09:00:26 in article <Re: Another debate between BRAD and
Seligman (long post)>, 'sc...@wam.umd.edu (jupiterbowl)' wrote:
>In article <4t5imf$2...@news1.t1.usa.pipeline.com>,
>Leslie E. Packer, PhD <lpa...@nyc.pipeline.com> wrote:
>
>>If I followed your argument, the small effect size (which you estimated
at
>>2 SD but suggest might be an overestimate) shouldn't be considered
>
>(treatment group mean)/sd(control-group scores) = 2.0?
We seem to have lost a decimal point somewhere. No, what he had calculated
was 0.2 SD. I don't have my original post handy, so if I left out the
decimal, I apologize. Or maybe the tricky little thing didn't show up on
the screen and vanished into cyberspace <g>. You are, of course, correct,
that had it been 2.0 SD, it would have been a large effect. But we were
talking about 0.2 SD.
>>been in the other direction. There is an important difference between
>>statistical significance and clinical significance, but one cannot truly
>>have the latter if the former does not exist. So demonstrating a
reliable
>
> I don't think I understand this Dr. Packer: are you saying that
>statistical significance is necessary for clinical significance?
something really be important (clinical significance) if it was (merely)
due to sampling error?
>If you
>mean that reliability of measurement and reliability of effect are
>necessary to show clinical significance then I'm sure I agree.
>
> But if you really mean statistical significance, as in some p <
>alpha, I think I disagree. Since statistical significance is directly
>influenced by one's decision about the level of Type 1 error one is
>willing to make in one's inferences on the basis of a particular design
>before the fact, and clinical significance is a judgment about the amount
>of change in a patient or group of patients determined after the fact, I
>think these are apples and oranges.
you're a good researcher, you also set your beta risk and then calculate
what N you need so that your beta risk is acceptably small. Given that
you've done those things, your alpha level is your statement as to what you
will consider "statistically significant" or "reliable," yes? _If_ your
results are (statistically) significant, _then_ it makes sense to ask the
next question which relates to the other definition of significance.
Let me insert one thought here: I agree with you that "clinical"
significance or "importance" are somewhat judgemental. A reliable but
_small_ effect may be very important for someone who's engaged in theory
testing. That small but reliable effect may also lead the researcher to
another study that attempts to maximize or boost the effect, etc. But the
smaller the effect is, the less likely it is to be of "clinical"
significance at that point (in terms of changing what practitioners do --
but even that's debatable). The larger the _apparent_ effect is, and
assuming that the results are statistically significant, the more likely
the results are to be clinically significant.
But again, this is not written in stone. One could (hypothetically) obtain
a set of data that exceed the alpha level (e.g., p
<.000000000000000000000000000001 <g>) and demonstrate a large effect (in
terms of explained variance), but it all be due to a confound. And of
course, there's always that 1 in a zillion chance that it _is_ a Type 1
error.
> I imagine a counterexample where a small, but reliable, effect
>would not attain significance due to low power in the design because of a)
>small samples b) small Type 1 error rate, and/or c) high within-group
>variability due to other reasons.
Whoa. You just made an error above. "Reliable" is the _same_ as
"statistically significant." "Reliable" is one of a number of synonyms
that are used for "statistically significant," the other most common phrase
being "not due to chance or random error." I think that perhaps what you
were trying to say is that there is a real effect (in the population) but
you don't recognize it in your research because.... etc.
All of the parameters you cited in your example above would, indeed, work
against an outcome achieving statistical significance. That's why I always
taught my students to _set_ their power and then use power analysis
procedures to determine their N. I am probably one of the few people who
do not automatically set alpha and/or beta. I really try to think of the
implications of each type of error for the particular question I'm looking
at.
>
><snip>
>
>Really interesting review, thanks.
Thanks for the kind words.
--
Leslie
Leslie E. Packer, PhD