A point I only realised recently is that many of these spoof answers could
give a more accurate answer than the `textbook' method.

Peter

The first few that I remember.

1) Drop the barameter and time its fall.

2) Tie it to a long piece of string and use it as a lead-line, measuring the
     length of the string.

        I *presume* that "politically incorrect" above means "nonstandard", not
"involving stereotypes of gender, religion, ethnicity, sexual
orientation, or hair color".  The idea of a whole subgenre of "Scottish
Barometer Jokes" or "Blonde Barometer Jokes" is just too mind-boggling.
<grin>

        -Robert Dawson

But here are words from what was originally posted:
========== part of post, extracted
Do you think the question is actually OK? Is the wording good enough
as it stands?  Or, as worded, could there be a legitimate uncertainty
about which answer is correct?

  BEGIN QUESTION TEXT

37. When Matt's and Damien's broad jumps were measured accurately to
the nearest foot, each measurement was 21 feet. Which statement best
describes the greatest possible difference in the lengths of Matt¹s
jump and Damien's jump?

A. One jump could be up to 1/4 foot longer than the other.
B. One jump could be up to 1/2 foot longer than the other.
C. One jump could be up to 1 foot longer than the other.
D. One jump could be up to 2 feet longer than the other.

  END QUESTION TEXT

ObPuzzle: Assume that the wording needs improvement.  Assume that the
concept to be tested is < rounding off  ... >
======== end of extract from post.

There are at least 3 distractors in there, which I identified before.
Maybe they make this a better item for General Intelligence (or,
Intelligence and acculturation).  (Stupid phraseology of rounding;
rounding of SIZE rather than numbers; "greatest possible difference"
in the Q  invokes "at least 1 foot" as the minimum - before presenting
answers without the proper alternative.)  (Is it possible to test
"rounding"  without using the word "rounding"? - I suspect that the
simple attempt, like this, may be something that breeds scorn and
contempt into the hearts and minds of mathophobics, everywhere.)

They make this a sloppy test of Rounding, i.e,
Does the pupil understand the concepts of rounding off numbers?
 - well, there are rational plus neurotic reasons to resist.  
The most obvious reason to answer C, in my opinion, is that the item
is an obvious probe, "Do you understand Rounding"?

Further:  I have had trouble making this point to people, but I am
pretty sure that "measured accurately to the nearest foot"  is context
dependent, or an idiomatic expression.  The 2nd reason to answer C is
that the test-item is a probe, "Do you understand  *that*  idiom?"  -
and if you haven't paid attention in math-class, it's likely you
don't.

math would also reinforce meaning of the use of the english ( ____ input
other language if appropriate) language ... the term "accurate" does have
some meaning ...

in this context ... and after all, every item is in some context ... it
says accurate for BOTH ... to the nearest foot

so, i think a perfectly legitimate interpretation of that is ... could be
off 1/2 foot down ... or up ...

i don't see anything mathematically wrong with deducing the answer to be C
given the context of the item ...

so, if you are saying that C is the best in this context ... good ... if
you are arguing for D ...

i disagree

my math is not great ... but, it ain't that bad either

reliability is all about that ... reliability of the measured value ... the
measured jump of 21 feet ... can we depend on this to be correct and if
not, how "off" could it be from a bad measurement standpoint ...

so, this question interests me not so much from the standpoint of what
specifically IT is measuring ... but, from the standpoint of what makes a
decent question ... or a poor one

this kind of item  is one that gives "tests" some of the bad name they get

in a case like this, we have to look at the question being asked ... ie,
the stem ... and first list out what are "facts" of the stem and what are
"logical inferences" that an examinee could make (maybe should make)

let's assume up front that the objective of the item really is ... concept
of measurement error ... that is, when folks take measurements ... they can
be wrong ... and wrong either way but, consider the following in this case

FACTS

1. you are given that the jumps were measured to the nearest foot
2. you are given that the jumps were measured accurately

LOGICAL INFERENCES on the part of the examinee

1. this is a contest ... broad jump ... and a tape measure was used that
had at least inch subdivisions ... or even greater (ever see a tape measure
at a track meet only with FOOT tick marks? ever have in your hand, a tape
measure that is 25 feet or 50 feet ... that did NOT have at least inch or
probably FINER subdivisions?)
2. contests are important so ... the measurers are assumed to be doing
their best to read "jumps" accurately ... if they don't ... they get water
bottles tossed at them by the irate parents
3. typical tapes could be extended in a somewhat slack mode  ... when
extended to make the measurement ... but there is a limit to how TAUT or
lengthened they can be made to go ... so, if an error is likely to occur
(forget the fact that "accurate" is given in the stem) ... then it would be
most likely and sensibly in the slackened condition ... which means the
measured jump would be recorded LONGER than it should be ...

given the FACTS and what i believe to be sensible inferences one can and
should  make in a broad jump contest which this is assumed to be like ...
would lead me as a measurement person ... to say this:

if the tape were extended in the taut(est) condition ... AND, measurements
were done accurately ... then if the landing mark were really between 20.5
and 21 ... we assume it will be rounded up/reported as 21 ... max error 1/2
foot ... or ... if the landing mark fell between just less than 21.5 and 21
... it would be rounded down or reported to be 21 ... this max gap sensibly
would be 1 foot

however, if the tape were slackened ... either a little or a lot, whether
it be measurer's error or not  ... for a particular measurement (which
means it is not an accurate measurement by definition but, lets let that
slide for a moment) ... then the gap (and hence max error) between the tape
mark and the landing mark becomes harder to discern ... perhaps impossible
to discern

because ... we don't know how much slack there might be in the tape

but, regardless, it will be seen by the measurer as being LONGER than it
really is ...

thus, under the slackened condition ... errors could make the measurment
longer than it should be ... but, in the taut condition ... the error is
not likely to make the measurement shorter  than it should be ...

the liklihood of an LONGER error (if anything) is much greater than the
liklihood of a SHORTER error

if in the taut condition ... the MINIMUM "max" error could sensibly be
called (rounding of course considered) 1 foot ... between the two ACTUAL
jumps ... BUT WHAT COULD THE SENSIBLE MAXIMUM MAX ERROR BE BETWEEN THE 2
ACTUAL JUMPS?

i say that this canNOT be sensibly determined from the facts and logical
inferences made in the question ... and while i now say that choice C would
be a "possible choice" it should actually read (C: One jump could be up to
1 foot longer than the other <<<< could be 1 foot ... but not could be UP
TO 1 foot)  for the MIN MAX ... choice D of 2 feet is NOT a good choice
either (nor can any be deduced) for the MAX MAX error in the actual jumps

thus, i now don't believe C is correctly stated ... and therefore is not
correct ... and D is not correct because we cannot determine what might be
the largest error that could be made ... it might be 1.3 feet or 1.7 ... or
2.1 ... but we do NOT know that the max error could or would be 2 feet

bottom line:

A and B are incorrect for sure ... C is not good ... and D can't be proved
to be correct
none of the choices is correct ... C is probably the BEST choice but still
not a good one

this might be a good question for assessing an inappropriate objective ...
or, an inappropriate question to test a legitimate objective

but as it stands ...  it surely is a poor item that fails to keep straight
... appropriateness of the item GIVEN some  objective

==============================================================
dennis roberts, penn state university
educational psychology, 8148632401
http://roberts.ed.psu.edu/users/droberts/drober~1.htm

On Thu, 01 Feb 2001 14:22:21 GMT, kak21...@excite.com (J. Williams)
wrote:

JW >

(1) There is (something like) "Is the right answer given by someone
with a good IQ?"  I think that we are all agreed that (C) should meet
that requirement.  Further, I imagine that the item was validated
*statistically*  by this standard -- marking "C" goes along with
higher scores on other test items.

(2) There is a narrower approach -- which, indeed, was the question
specified when this item was posted.  "Does the item show whether the
student understands rounding?"  Will it be answered correctly by
everyone who does, or could naive respondents be led astray?
Since a "broad jump measured accurately to the nearest foot"
is not something that anyone in the Western world has ever heard
of,  is it really fair to ask an 8th grader to interpret what it might
mean?  (I assume, the 8th grader is suppose to translate this,
immediately, into "This is a ROUNDING problem,"  and the rest
of us statisticians know what the item's answer is, because we
have overlearned exactly that same response.)

You demonstrate possible difficulties, perhaps, by debriefing
students who missed the item; or by comparing to other, related items;
or by noting that there are unexpected item-loadings in a large scale
factor analysis.  But you usually will discover them by careful
face-inspection, which is what I provided (I hope) in earlier posts.

"If you can imagine a way that someone would misread the item,
then someone will."  This is a mild version of Murphy's law.  It is
practically a truism when you are designing items or forms -- the hard
part of your judgement is,  figuring how much "problem" is too-much
problem.  In the recent Florida election, we learned that "punched
cards"  have an inherent error rate of over 1%.  And a "butterfly
ballot" has a rate over 5%.  How much does it matter that most of
these errors should befall that 15% of the voters in Florida who were
voting for the first time? - well, it means that our subjective
account should not assume that every voter is cool and experienced.

"Professionally speaking,"  the butterfly punch-ballot  has to be
regarded as awful, no matter how much Jay Leno, etc., make fun
of the Florida voters instead.  

Similarly for the test-item.  If you are making assumptions about the
pupil's experience, vocabulary, acculturation, IQ,  and attitude, then
you may forget to rate the item by how it measures "rounding."

Here is a minor question or observation.  In the real world, does
anyone ever perform rounding, and blandly expect for it to be
recognized as such?  Or don't we *explicitly*  state that "this is
rounding and not truncation or estimation."

(3) The third approach is, "Is the answer technically correct?"
So far, it remains embarrassing and something-to-be-corrected,
when the keyed answer violates physics, or careful logic.  Or if,
on close inspection, the question does not make good sense.
This is more important than slightly misleading some students.  
Bad logic likely will be reflected in errors of the previous type,
but errors of (#3)  need to be corrected, where (#2) do not.
It is harder to show test-makers that *they*  are "wrong."

I have not had many people agree with me that, instead of being
purely logical,  this item relies on well-understood jargon or idiom.
I 'm trying one more time.

It says, "measured accurately to the nearest foot."  People keep
claiming that "accurate" must mean "it's 100% accurate"  - so that
this conflation of accuracy (of measurement) and precision (of
reporting) is entirely expected and natural.

What if another item said that a blimp at 1000 feet saw the two jumps,
and "estimated each at 21 feet, accurate only to the nearest foot."
What does this imply about the maximum difference between the two
jumps?  
What if it said, "estimated each at 21 feet and 6 inches, measuring
accurately only to the nearest foot"?
 - actually, the occasional use of half-units (like 6 inches)  is
probably a give-away that someone thinks that their *accuracy*  is
about one-unit; they are promising not to err by more than 1/2, so
they refuse to round off, between .40 and .60, say.

 - I think that I have just presented, in those last two things,
comments that are much more "real-life"  than the statement in the
original test item.  And "accurately" is ambiguous to the 14-year-old.

Finally, we round  *numbers*  if we don't want to fret about
measurement error.  And we keep that language clear.

-Robert Dawson

On 5 Feb 2001 08:52:28 -0800, Robert.Daw...@STMARYS.CA (Robert J.

Or, is that what we are supposed to teach?  I do wonder whether
my  complaint  is fundamentally against bad teaching.  I do imagine
that concept-insensitive teachers are using words and examples that
are just as sloppy as the Item.  And then they wonder why some
students, who insist on their own poetic or neurotic interpretations,
Don't Get It.

[ snip, rest ]