Teachers and student test data

[Dick]

The story out west today is the release of test scores by teacher in
LAUSD. A simplistic notion is provided by the LA Times that these
score gains will help parents find good teachers for their kids. If
only it were that easy. Obviously, as seems to be the case with both
President Obama and Secretary Duncan, the LA Times reporters have also
never taken a course in psychometrics. The achievement tests now
available were not designed to show reliable scores for individuals.
Nope, they were designed to reliable scores not for individuals but
for schools. The technical manuals that are available for every
standardized test all indicate that it is unacceptable to use the test
scores to make judgements about individuals. I can't be too hard on
the reporters though since far too many school officials seem to have
no idea about this either (maybe they were asleep during that tests
and measurements class).

John Papay of Harvard has a study that will soon appear in the
American Educational Research Journal that demonstrates the problem
succinctly. He used the Houston school districts value-added formula
and applied to three different reading tests given in grades 3 through
8 in a northeastern school district. Basically, 40% of the "best"
teachers on one test had the worst performing students (these are the
same students) when a different test was used (even though it was a
testing given the same semester as the other one). Likewise 45% of the
"worst" teachers on one test became the "best" teachers on the other
test.

If you want to create a crapshoot and make it look scientific,
participate in any of the value-added schemes now in marketplace.