Why is lower spoilage a benefit?

[Clay Shentrup]

Someone recently sent me an email containing this response to my ballot spoilage figures. My reply is everything after the blue quoted text:

I don’t think you can make the case that spoiled ballots are a sign that voters have more to say than having one preference.. over-voting is usually the result of *error*, possibly in the ballot design

I had to do quite a thought experiment to provide a suitable response here.

Let's say a mistake which prevents a ballot from being counted produces a "spoiled" ballot, whereas a mistake which causes the ballot to be counted, but in a way other than what the voter meant, is "inaccurate". And let's say that we roughly want to minimize the sum of "spoiled_ballots + 2 * inaccurate_ballots" (because "accurate & counted" is better than "not counted" is better than "inaccurate and counted").

So then it seems like your point is that Approval Voting is really just causing us to count a bunch of inaccurate ballots that Plurality Voting would have (rightly) discarded. Let's go with that for the sake of argument, and say that X is the number of spoiled ballots with Approval Voting.

That would mean that the number of spoiled/bad ballots with Plurality Voting would be about 5X. But since we're agreeing that Approval Voting turned those spoiled ballots into inaccurate ballots, then Approval Voting actually has something more like 9X bad ballots.

Now, IRV experimentally results in about 7 times as many spoiled ballots as Plurality Voting. So that means 35X bad ballots. So IRV is about 4 times as bad as Approval Voting according to this ballpark estimate. (And that goes up appreciably if it turns out that that a significant fraction of the Plurality over voting is actually an intentional act of self-expression rather than inaccurate/accidental.)

One might argue that this "7 times" figure is only a result of bad ballot design, but...

1. This figure is based off of analogous (even concurrent) IRV/Plurality races and ballots, thus Plurality Voting also suffered from bad ballot design. I.e. if you redo the experiment with an improved IRV ballot, you have to also use an improved Plurality ballot to be fair, presumably leaving you roughly where you started.
2. Even if we can hypothetically improve IRV ballot design, we have to evaluate policies based on the current and foreseeable reality. I don't know of any overwhelming evidence that ballot design is going to get radically better in the near future.

Another (perhaps equally) important issue here is public perception. Increased ballot spoilage rates have been cited as grounds for the repeal of Instant Runoff Voting here in San Francisco, and it seems this was also the case in Burlington, Vermont.

http://www.uvm.edu/~vlrs/IRVassessment.pdf

[Aaron Hamlin]

It may be possible to throw a few more reasons on there, too:

1. Ballot spoilage can act as a proxy measure for the voting system's complexity, with less spoilage indicating more simplicity and understandability
2. The feel-good nature of having more votes counted is better (overlaps with your point #2)
3. At best, spoilage happens randomly among groups meaning it is random error. This error averages out, but it does sacrifice some accuracy
4. At worst, the spoilage is systematic among groups. This means the error can go in a focused direction helping or hurting specific groups. This significantly hurts accuracy.

Also, as a rebuttal, if this was a ballot design issue, then we would expect the spoilage rates to average out over different samples. Then we'd see similar spoilage rates, on average, regardless of the voting system. Instead, we consistently see higher spoilage rates with ranked systems regardless of the ballot design. I'm not holding my breath for a spectacular ranked ballot design that is able to reduce a ranked ballot's spoilage rate. And I don't think there's any ranked ballot design that will get its spoilage rate in the neighborhood of an Approval ballot.

[Abd ul-Rahman Lomax]

At 06:08 PM 3/6/2012, Aaron Hamlin wrote:
>I'm not holding my breath for a spectacular ranked ballot design
>that is able to reduce a ranked ballot's spoilage rate. And I don't
>think there's any ranked ballot design that will get its spoilage
>rate in the neighborhood of an Approval ballot.

Except, of course, for a ranked Approval ballot, or any other ballot
that does not discard overvotes and consider them spoilage. Even an
IRV ballot could count the overvotes. The method would be the better
for it. It would allow counting IRV as a pure Approval method (i.e.,
if all voters only vote in first rank.)

This raises the obvious question, but apparently not-so-obvious: why
are overvotes not counted?

I don't know, it would take a major, and difficult historical
examination, and I only know a piece of this.

The default election method under Robert's Rules is repeated election
until a candidate gains a majority, on vote-for-one ballots. The
instructions for clerks explicitly refer to the overvotes as
"errors," where the "true intention of the voter" cannot be
discerned. Now, technically, the "true intention of the voter cannot
*ever* be discerned. The voter might mistakenly vote for the wrong candidate.

In any case, there is no reason given, anywhere that I've found,
justifying the rule against voting for more than one. In fact, the
default in Robert's Rules is an open vote, by voice, or, if the
result is not clear, by show of hands. There is no process for
preventing overvotes in that, I have never seen an open instruction
to not vote for more than one, and I'm sure that overvotes happen.
They are harmless! So the overvoting issue only arises with paper
ballots. The standard paper ballot, again, is just a blank piece of
paper on which the voter writes the vote. The name of the candidate.
What if the voter writes "Candidate X *and* "Candidate Y" Would this
still be treated as a mistake? My guess is that a good clerk would
simply make a report of the votes, as is, and would note the
"overvote," explicitly, unless it had been established as a practice,
and the *meeting* would decide the issue. "Shall the overvote be
discounted?" might be the question.

But there is a huge safeguard in Robert's Rules elections that is
missing in public election rules, as far as anything I've seen. A
true majority *of ballots* is required. If that ballot contains *any
mark*, even if unintelligible, it is considered a "vote," for the
purposes of considering "majority." So even if all the votes on the
ballot are discounted, *the ballot counts as a vote, effectively,
*against all the candidates.* It simply can create majority failure,
thus a *repeated election*, the first election becomes completely
moot, except for how it naturally informs the voters in the repeated
election. There are no eliminations. There is a new process with new
nominations, which can be the same, or different.

It should be understood that the standard Robert's Rules election,
given the details of the rules, is a far more powerful system than is
commonly recognized.

But a rule from within that context, where it doesn't do much harm,
it only decreases efficiency a little, is being applied to public
elections. This may have happened from people simply assuming that
the same methods, known to be highly democratic, should work in
public elections. But, of course, desiring to avoid lots of trips to
the polls, they simply truncated the process.

This, again, in most contexts, does less harm than we think. In
nonpartisan elections, simple vote-for-one Plurality is quite likely
to produce the same result as a more advanced system, *in most
contexts.* It's the exceptions that are killers, and it's largely
moot in a two party system, except for those damn spoilers. They are
called that name because, indeed, they sometimes spoil the function
of Plurality. And many people blame those candidates, not the system.
Why don't those idiots work within a party?

Back to the issue. I'd suggest adopting public rules that more
closely match Robert's Rules. Use a runoff system to seek a majority,
and consider *all ballots* in determining "majority."

In theory, there would be an unlimited series of elections, but that,
perhaps quite properly, will be considered impractical. So, in light
of the practicality of requiring the election to complete in two
polls, maximum, shift the rules to allow what takes place under
Roberts' Rules to be simulated in the two ballots.

First of all, go back and notice that Approval Voting was used in the
past, and worked. It merely means "Count All the Votes." Get rid of
discarding ballots for overvoting. Count them. Only discard blank
ballots, Robert's Rules does not count them, calling them "mere
scraps of paper." A ballot that reads, "None of the Above" is a
ballot, and can properly be counted as such. It really is a vote
against all the candidates, preventing them equally from gaining a
majority of votes, and thus possibly throwing the election into a
runoff. If write-ins are allowed in the runoff, there really would be
a chance for someone better to win. (And this has happened, for those
who might, knee-jerk, assume it's impossible.)

But wouldn't it be better for the voter to simply write in the name
of a preferred candidate, if the voter strongly disapproves of all on
the ballot? Not necessarily. This might be a libertarian who thinks
the office should remain vacant, and is expressing that. It's fair as
an expression, especially in a primary. In a runoff, None of the
Above, as a write-in candidate, might spoil a majority but should not
be prevented as a vote. And what if a majority vote "None of the
Above" or the equivalent? Because that was not the question being
presented, I'd think of still awarding the election to the plurality
winner, but ... I'd want an advanced method in the runoff as well as
in the primary, so a voter can express this preference without harm.

And if a majority did vote that way, and this was real as the
sentiment of the majority of voters, then there is an obvious next
political step to take, having been validated as probably the
majority position!

What will simulate vote-for-one, repeated, first of all, is allowing
multiple approvals on the ballots. It will be even more effectively
simulated with a Range ballot, that analyzes each rating, seeking a
majority for that rating or higher, and that then steps down the
ratings, until a majority is found. Such a ballot should have an
Approval Cutoff, ratings below that mean "I'm not yet willing to
support the election of this candidate, but I prefer this candidate
to others at lower ratings." If there is no majority approval, the
election, in a primary, would fail. For efficiency, then, the runoff
ballot would list all reasonable candidates from the primary, and if
write-ins are still allowed, problems with this can be fixed. A
spoiler-protective method like Bucklin should be used, then, for the
runoff as well. The ballots could be the same ballots, i.e., same form.

Original Bucklin can be understood as a Range 4 ballot, with these
restrictions: Ratings 0 and 1 were collapsed to a single unapproved
rank. The three Bucklin ranks then can be seen as ratings 2, 3, and
4. Some Bucklin methods, such as the one passed but never used in
Oklahoma, assigned fractional vote values to those ranks. That was a
Range method! Non-linear, but so what?

In any case, definitely, ballot spoilage can be reduced by following
a reasonable interpretation for all ballot variations. All
discernable votes should be counted, it's that simple.

IRV only results in so many spoiled ballots because of rules that
unnecessarily disregard votes. I have never seen a clean
justification for those rules. What I've seen is a claim that it will
violate the majority criterion or that somehow it is violating
one-person, one-vote. While the majority criterion violation
technically exists under some rather weird interpretations of that
criterion (which if applied consistently would mean that *no method*
satisfies the criterion), it's very clear that there is no
one-person, one-vote violation, since the most that the ballot can
do, with every pairwise election, is to support one candidate over
the other with one vote, or with no vote, i.e., if the voter has
voted for neither or voted for both.

The UnFairVote arguments only stand because the process by which
decisions are made is not allowed to complete with full discussion
and consideration of all points. UnFairVote is actually brilliant in
coming up with Fair-Seeming arguments that fall apart when examined
carefully, and they actually lie on occasion. I don't know how many
times I've seen FairVote propaganda repeated by election officials as
if it were the truth, such as "Robert's Rules of Order recommends IRV." Not.

1. What Roberts' Rules recommends is crucially different from IRV, it
recommends a preferential ballot with counting rules that follow the
IRV method, but which differ in that critical step, the determination
of a majority. Robert's Rules follows the true "majority of the vote"
interpretation, and that's clear to any parliamentarian, who would
expect nothing else, but the *description* itself can be
misunderstood by a non-parliamentarian to mean that "majority" means
"majority of unexhausted ballots." Which, from other comments in the
coverage of the method, *it does not mean.* They count all the
ballots, that are not completely blank, and so the method actually
does seek a true majority, unlike the faux "instant runoff" using
UnFairVote's recommended rules.

2. Robert's Rules then points out the defects in the Preferential
Voting that they describe, *even if a true majority is sought.* This
is only a method to be used if repeated ballot isn't practical. And
*even then* they will require a repeat of the election -- no
eliminations! -- if there is majority failure. They tell the clerks,
for the PV method, to advise voters to rank all the candidates, "or
else the election may have to be repeated."

3. Does Robert's Rules prefer the STV method over all other
preferential voting methods. No. They merely describe that one
because it has seen the widest user. They are not voting systems
experts, they are parliamentarians, and Robert's Rules reports on
actual practice, it is not a handbook for reform. They do, however,
very accurately point out the problems with PV as they describe it,
and IRV is worse, quite a bit worse.

And when the facts have been pointed out, I've seen them lie. They
simply repeat that the method in Roberts' Rules is exactly the same
as what they are proposing. Technically, that my not be a lie, it may
simply represent the tendency of people to believe what is convenient
for their agenda, and they simply assume that someone saying
something different is lying, so they don't bother to look and check.
After all, it was on the UnFairVote web site, and that is surely a
reliable source!

Anyone can see the rules in RRO. I'll say this, though. I'd first
read the UnFairVote description, and when I first read the real
rules, I thought, "well, they are right." Even with all that I knew,
-- I've been a parliamentarian, though I'm far from expert -- I
missed the critical difference, so effective had been the UnFairVote
propaganda. So to communicate this, the information must be *very
specific* and clear as to the crucial differences, and as to what
Robert's Rules actually says. If anyone needs assistance with this,
please write me.

I only have the 10th edition, though. There is now an 11th edition.
I'll see about getting a copy. Donations appreciated.

Looking at what has been announced as changes, though, I'm a little
worried. These are not, as I've mentioned, voting systems experts,
but the Devil will depend on the details. What I see that might
*possibly* be a problem:

*Recognition of "one person, one vote" as a fundamental principle of
parliamentary law [407].
*Restates the rules on recording the votes in a ballot vote [415-17].
*Eliminates the requirement that detailed rules for the procedure for
conducting preferential voting must be in the bylaws [428].
*Clarifies the circumstances under which a nominee may be dropped
from a succeeding ballot, and the effect of so doing [441].
*Notes that cumulative voting violates the fundamental principle of
"one person, one vote" [444].
*Adds a table summarizing the rules for counting ballots [tinted page 48].

None of these are a definite problem. I was already aware that
cumulative voting was considered a violation of one-person, one-vote,
but that issue is complex. The 10th edition does describe cumulative
voting fairly, as a method for proportional representation, but then
it states, "However, this method of voting, which permits a member to
transfer votes, must be viewed with reservation since it violates a
fundamental principle of parliamentary law."

It violates a common *understanding* of one-person, one-vote, but the
fundamental principle is equality of participation. I don't expect
the parliamentarians, as a group, to necessarily understand this.
There is long *tradition* of only allowing one-vote-for-one in
practically everything, which can make sense and is at least
relatively harmless with the rest of the rules in place. Remember,
they use an iterative process, seeking a majority for a single
result. If one only looks at a single ballot, it looks like
Plurality. It is not. It's Majority, and they are quite firm on that.

Cumulative voting was allowed by the 10th edition, and I'd think that
continues, so the comment was dicta, merely urging "reservation."

Bylaw changes are difficult to come by, sometimes. Bylaws may be
amended at any regular business meeting, with previous notice, by a
two-thirds vote (of a quorum, of course), or without notice, at any
regular meeting *by an absolute majority of members of the
organization.* I'm guessing that the editors or RRO wanted to make it
simpler to make procedural changes in voting procedures.

(Improved voting systems will tend to remove some level of power from
incumbents electted under the old system, hence they may tend to
resist change, making it substantially more difficult to get a
two-thirds majority, and getting an absolute majority of members
present to approve a measure at a meeting can be somewhere between
difficult and impossible. RRO doesn't allow proxy voting, based on
the idea that proxy voting is useful where there are property
interests, but democratic procedure is not about property, and
depends on members being present in order to vote. That is a very
narrow view of what's possible for proxies, and simply reflects,
again, that RRO is not foward-looking, it is only describing what is
actually done in organizations, and certainly not recommending
change; the comments made are, then, dicta, rationalizations of why
things are the way they are, not actually coherent recommendations
regarding something else, possible changes. In any case, the change
described on the implementation of preferential voting is probably
sound. And then we could get into *what kind of preferential voting*
I assume they still mention there are many methods!)

[Clay Shentrup]

On Tuesday, March 6, 2012 3:08:54 PM UTC-8, Aaron Hamlin wrote:

1. Ballot spoilage can act as a proxy measure for the voting system's complexity, with less spoilage indicating more simplicity and understandability

You're missing her brilliantly simple point. Approval Voting gets lower ballot spoilage by COUNTING over votes that would be ignored by Plurality Voting. So that's only a good thing if you think those votes were accurate/meaningful. Otherwise you're just adding noise to the result. That's why I went into incredible detail about the mathematics of accurate vs. valid votes.

The feel-good nature of having more votes counted is better (overlaps with your point #2)

Right. This seems to have a lot to do with public perception. You don't want a better system to get repealed simply because it has some minor flaws that are unfortunately too visible and easy for critics to talk up.

At best, spoilage happens randomly among groups meaning it is random error. This error averages out, but it does sacrifice some accuracy

But if the data is bad, you don't want to count it.

Also, as a rebuttal, if this was a ballot design issue, then we would expect the spoilage rates to average out over different samples. Then we'd see similar spoilage rates, on average, regardless of the voting system.

Well that's a very good point.

[Abd ul-Rahman Lomax]

At 12:59 AM 3/8/2012, Clay Shentrup wrote:
>On Tuesday, March 6, 2012 3:08:54 PM UTC-8, Aaron Hamlin wrote:

> * Ballot spoilage can act as a proxy measure for the voting

> system's complexity, with less spoilage indicating more simplicity
> and understandability
>You're missing her brilliantly simple point. Approval Voting gets
>lower ballot spoilage by COUNTING over votes that would be ignored
>by Plurality Voting. So that's only a good thing if you think those
>votes were accurate/meaningful. Otherwise you're just adding noise
>to the result. That's why I went into incredible detail about the
>mathematics of accurate vs. valid votes.

Sure. I didn't read that, unfortunately, so this is my
possibly-ignorant comment.

People will make mistakes and overvote when they don't intend to.
However, it is likely that they intended to vote one of these.

It is also relatively likely that they did not consider the other vote harmful.

This is just overall, individual situations could obviously vary.

Often, as in Florida, the "overvote" wasn't really one, the voter's
intention was clear. With the confusing butterfly ballot, some voters
voted for Gore and wrote in "Gore." Those were counted as overvotes,
and discarded.

Now, suppose we have four somewhat-viable candidates, call them Gore,
Bush, Nader, and Buchanan.

Gore and Bush are frontrunners. Most voters will prefer one of them.

So they vote for one. But they accidentally vote for one of the
others. What's the effect of counting the vote? Let's look at the
possible errors.

Gore. The voter preferred Bush. Counting the vote has the same effect
as not counting it.
Bush. Ditto.
Nader. The vote increases the Nader vote count, but that's moot,
almost certainly.
Buchanan. Ditto.

Only if the voter intended to vote for Nader or Buchanan but
accidentaly adds a vote for Gush or Bore, so to speak, is some
possible harm done, that could affect the outcome. These are, by
definition, not very many voters. But, in this case, the overvoting
is more likely a strategy, not an error. These are the people
motivated to vote for one of the frontrunners. If they are a Nader
supporter and accidentally vote for Bush, it have have an effect on
the real outcome, but the voter, in this case, may care enough to get
another ballot!

(Voters are voting, and may be in a hurry to get someplace. If they
add an additional vote, accidentally and without intention, they may
then consider whether or not this vote does some real harm. A Gore
supporter who accidentally votes for Buchanan might well say, "this
isn't going to make any difference, I'll get to work.")

I see little harm and much good from counting all the votes, so....

Count All the Votes.

The information on these "spoiled ballots" is not garbage, especially
if voters know that overvotes are allowed. Do we do Robson rotation
in the U.S.? If not, then we are certainly collecting "garbage," and
it is still garbage with Robson rotation, but it tends to average
out, that's all.

At worst, counting all the votes is superior to discarding them, for
if they are purely error, they would, with fair ballot design, average out.

Definitely, we should count the votes. How they are used is actually
a separate question, but, I'm confident, it's far better to use them.
And it provides a fix for the spoiler effect, at no significant cost.

[Stephen Unger]

...

> Often, as in Florida, the "overvote" wasn't really one, the voter's intention
> was clear. With the confusing butterfly ballot, some voters voted for Gore
> and wrote in "Gore." Those were counted as overvotes, and discarded.

...

The famous butterfly ballot did NOT lead to overvotes. It led to
people voting for Buchanan when they intended to vote for Gore because
of the peculiar staggered arrangement of the ballot. (Because Gore was
listed second in the left column, many thought they should punch the
second hole to vote for him, whereas they should have punched the third
hole.) See http://arts.bev.net/roperldavid/politics/fl2000.htm

Steve

[Jameson Quinn]

2012/3/9 Stephen Unger <un...@cs.columbia.edu>

It led to both misvotes and overvotes. Some simply misvoted. Some people made the mistake and then tried to correct it. Others were confused and added a write-in for Gore "just to be sure". I'm sure there were others who ended up discouraged from voting altogether.

Of course, the butterfly ballot was only one of multiple irregularities, almost any one of which could have swung the election if it had been resolved.

Jameson

Steve

[Aaron Hamlin]

You're missing her brilliantly simple point. Approval Voting gets lower ballot spoilage by COUNTING over votes that would be ignored by Plurality Voting. So that's only a good thing if you think those votes were accurate/meaningful. Otherwise you're just adding noise to the result. That's why I went into incredible detail about the mathematics of accurate vs. valid votes. 

I understand what you're saying here and agree. No data is better than bad data. And more meaningful data is better than not.

[Abd ul-Rahman Lomax]

My undertanding was that some voters voted mistakenly for Buchanan,
then added votes for Gore. The example I gave apparently did occur.
Voter voted for Gore, correctly, but had some doubt because of the
butterfly ballot, so added the write-in for Gore. And this was
considered an overvote.

So, if that memory is correct, yes, the butterfly ballot did lead to
overvotes. it also led to just plain mistaken votes for Buchanan.

the question here is whether we should Count All the Votes, or not.