Voter Satisfaction Efficiency

[Spenser Kearns]

https://docs.google.com/document/d/1dzt-q6Gb8PHAgkLBv2j7UhiijbWGSaQeoeYy16YOt4Y/edit

This was posted in reddit.com/r/endfptp , and I'm curious to know how this is different from Bayesian Regret. The results from this have Schulze's method performing extremely well, and I'm wonder how the results managed to differ so much from Warren's study.

[Steve Cobb]

VSE is just BR with the sign flipped, and normalized to a range of 0 to 100%.

Jameson, could you summarize the key differences between your simulation and Warren's?

[Jameson Quinn]

Key differences:

-My simulation generated a more-realistic range of voting scenarios. Warren used two voter models, one of which had all candidates always artificially close to a multy-way tie (and thus an inflated chance of Condorcet cycles), and the other of which had almost 0 chance of Condorcet cycles. My model had a realistic chance of Condorcet cycles (around 2%), which is just a symptom of more overall realism.

-Warren's system arbitrarily designated two candidates as "frontrunners" for voters to strategize around. My system used a kind of simulated polling for that purpose, so that the frontrunners were an imperfect but meaningful approximation of the true honest winners.

-Warren's system used different strategies for some methods.

-I broke down my results by "scenario type", Warren didn't.

-I included "one-sided strategy" as a possible strategic configuration, Warren didn't.

My simulation is clearly better than Warren's on all but the third point (specific strategies used). On the third point, you could argue either way, but I'm still working on improving my sim to make it clearly better than Warren's in that way too.

I don't want to denigrate Warren's groundbreaking work here. If I couldn't do better than him, 16 years later, that would be surprising.

--
You received this message because you are subscribed to the Google Groups "The Center for Election Science" group.
To unsubscribe from this group and stop receiving emails from it, send an email to electionscience+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Warren D Smith]

All of what Jameson is saying are his improvements, sound good to me.

IEVS is a later program by me which
includes a lot more capabilities than my original code.
In some ways IEVS is more advanced than what Jameson is saying
his code does.
http://rangevoting.org/IEVS/IEVS.c

There also is a program by Billy Tetrud, which simulates MULTIwinner
voting systems to compute "2-stage" Bayesian Regret.
Tetrud's program is quite primitive.
However, the main point is that it does 2-stage BR and multiwinner
voting systems, which is exploring a new land.
http://rangevoting.org/multiWinnerBaysianRegret.html





--
Warren D. Smith
http://RangeVoting.org <-- add your endorsement (by clicking
"endorse" as 1st step)

[Warren D Smith]

I suggest to JQ that he add "score voting with subsequent honest 2-man runoff"
to his voting method set. I.e. the top two score finishers participate
in a later runoff.

The trouble with the breakdown into "scenarios" is it is a bit confusing.
One wants one number which is the "quality" of
a voting method. If you provide 10 numbers, then one gets confused.
Also, the scenarios probably are not mutually disjoint; they overlap
to different degrees, which also is confusing.

Incidentally, my old sims also had "scenarios" e.g. "number of issues"
and "strategic vs honest" voters, and "ignorance level" knob. They
however were not the same as Jameson's "scenarios."
I think one could, however, try to choose the most realistic-looking
among my "scenarios" fairly simply (and indeed Poundstone did so).
For Jameson's set of scenarios, in contrast, they are sort of
purified and therefore all unrealistic (?).

The truth is probably that the one number is a linear combination of his
10 numbers, but it is not obvious what that linear combination ought to
be. If one were to try to assess the frequency
of all the scenarios in real life, that perhaps might help
tell us the right linear combination, but this assessment might
be hard, and also in view of the overlaps it still would
not tell us the right linear combination (at least not trivially simply).
But I guess eventually JQ ought to make such an attempt (and probably
he was planning to anyway).

[Jameson Quinn]

My scenarios are disjoint. They are just a classification of the natural output of the voter model; that is, you first generate an election without considering "scenario", and you only classify it afterwards. So the scenarios are no more or less purified/unrealistic than the voter model.

Thus the "correct" linear combination depends on the voter model, but for any given voter model, it's clear. Also, as you vary the parameters of a voter model, the percentages by scenario will probably vary more than the VSE within any given method/scenario combination; though this is not an iron law.

[Warren D Smith]

On 1/20/17, Jameson Quinn <jameso...@gmail.com> wrote:
> My scenarios are disjoint. They are just a classification of the natural
> output of the voter model; that is, you first generate an election without
> considering "scenario", and you only classify it afterwards. So the
> scenarios are no more or less purified/unrealistic than the voter model.

--huh? I mean, "condorcet cycle" can occur SIMULTANEOUSLY with "spoiler."
Hence, the two not disjoint. Right?

And I also saw something about how you always pick the first scenario
that applies
from an ordered list. If so, then that would make them artificially disjoint,
but it is a bad idea in that is it extremely deceptive. I.e. "spoiler" then
might actually mean "spoilers, but only those that do not have any
condorcet cycles"
which is extremely deceptive in the sense that a normal reader was not
expecting that meaning, never wanted to know about that meaning, and
is now worried it might
differ massively from the meaning she wanted,
hence not willing to assign any credibility to your results!
To avoid that fate you would in my opinion be far better off making your
scenarios indeed overlap and discarding any "artificial first on list" nonsense.

Also, one more thing:
I recommend adding "random winner" to your list of voting methods.
That is an easy and good sanity check.

[Jameson Quinn]

"Random winner" has 0% VSE by definition.

"Cycle" refers only to top-cycles.

Yes I use an ordered list and apply the first one that works. I think that's as good a definition as any. When I say "spoiler", I don't mean any possible spoiler scenario, but a scenario with a CW and with a potential plurality spoiler who's not in the top 3 Condorcet candidates.

[Spenser Kearns]

Warren, does this change your views any on Schulze methods or Score Runoff?

[Clay Shentrup]

Jameson,

I'd still very much like to see a simple graph with X-axis being ratio of strategic/honest voters, and Y being VSE, with each voting method having its own colored line.

I suppose you could have three or so different graphs; one with symmetric strategy, one with mildly asymmetric strategy, and one with highly asymmetric strategy.

[Brian Olson]

This all sounds good to me.

I've always liked a positive-utility measure more; it's more intuitive to me. 'Bayesian regret' was always confusing to me.

I glanced at the source and see it's in Python.

What's the run speed like?

How many randomized trials are you able to run and over what population sizes?

In my simulations I found that there was some difference in how the population statistics worked out when there were 100, 1000, or 10000 simulated voters.

On Fri, Jan 20, 2017 at 9:39 AM, Jameson Quinn <jameso...@gmail.com> wrote:

Key differences:

-My simulation generated a more-realistic range of voting scenarios. Warren used two voter models, one of which had all candidates always artificially close to a multy-way tie (and thus an inflated chance of Condorcet cycles), and the other of which had almost 0 chance of Condorcet cycles. My model had a realistic chance of Condorcet cycles (around 2%), which is just a symptom of more overall realism.

-Warren's system arbitrarily designated two candidates as "frontrunners" for voters to strategize around. My system used a kind of simulated polling for that purpose, so that the frontrunners were an imperfect but meaningful approximation of the true honest winners.

-Warren's system used different strategies for some methods.

-I broke down my results by "scenario type", Warren didn't.

-I included "one-sided strategy" as a possible strategic configuration, Warren didn't.

My simulation is clearly better than Warren's on all but the third point (specific strategies used). On the third point, you could argue either way, but I'm still working on improving my sim to make it clearly better than Warren's in that way too.

I don't want to denigrate Warren's groundbreaking work here. If I couldn't do better than him, 16 years later, that would be surprising.

[Warren D Smith]

On 1/22/17, Spenser Kearns <kear...@mymail.vcu.edu> wrote:
> Warren, does this change your views any on Schulze methods or Score Runoff?

--I don't know enough about Jameson's sims and code.

His sim sure does look good for "score runoff" SRV,
so I guess that makes me like it more. I disliked SRV
not for reasons of Bayesian Regret but for reasons of
non-simplicity, poor properties, sacrificed enhanced-media-focus & info
with a SEPARATE 2nd round, and bullshit illogical and outright
false underlying rationales.

E.g, it seems clear that whatever SRV can do about
Bayesian Regret, range voting with a separate
(and necessarily 100% honest) top-2 runoff -- call that
"score+run" for short -- ought to do even
better. Jameson has not verified or denied that statement as
yet with his sim since he did not include score+run.
But it probably would be a good sanity check for him to add it.
On the other hand, if it were found that the BR difference
between score+run and SRV were small, that could be
viewed as a reason to want SRV. I.e: "score+run is good,
and we can sacrifice very little quality by its approximate version
SRV, which happens to be cheaper."
That would be a nice statement. If so.

Albeit the loss of voter knowledge because of lost media focusing
in the period between the two rounds, would not be visible in
Jamesonian sims. It might be very important, especially
for 3rd-party candidates... in fact it might even be so important
it causes third parties to thrive or always to die...
and you just will not know about that from a sim.

Incidentally, the CES paid for a commercial poll, run online,
fr the 2016 USA presidential election, and occasionally
Aaron Hamlin releases more info about it. But there are
many questions about it I do not know, and their raw data seems (?)
unavailable. Anyhow, said poll apparently found Sanders
was the Condorcet winner if they voted honestly.
And I presume (based on other polls) Sanders was the score-voting winner,
now with real votes, too. So I presume Sanders would have been
the SRV winner. But I do not currently know.

[Warren D Smith]

And actually, in the best of all possible worlds, Jameson would write
enough extra
Warren-like code to pretty much duplicate all my results, and I would write
enough extra Jameson-like code to pretty much duplicate all of his results,
thus independently verifying each other (except in hopefully rare cases
of not, we'd track down whatever the issue was).

Unfortunately I doubt that'll all happen in the actual world.

[Mark Frohnmayer]

On Mon, Jan 23, 2017 at 9:40 AM Warren D Smith <warre...@gmail.com> wrote:

On 1/22/17, Spenser Kearns <kear...@mymail.vcu.edu> wrote:
> Warren, does this change your views any on Schulze methods or Score Runoff?

E.g, it seems clear that whatever SRV can do about
Bayesian Regret, range voting with a separate
(and necessarily 100% honest) top-2 runoff -- call that
"score+run" for short -- ought to do even
better.  Jameson has not verified or denied that statement as
yet with his sim since he did not include score+run.
But it probably would be a good sanity check for him to add it.
On the other hand, if it were found that the BR difference
between score+run and SRV were small, that could be
viewed as a reason to want SRV.  I.e: "score+run is good,
and we can sacrifice very little quality by its approximate version
SRV, which happens to be cheaper."

Brian Kelly simulated both of these and by his utility measure, SRV outperformed score + separate top two. I'd be curious to see if Jameson's simulations show something different.

[Jameson Quinn]

In a simulation, there actually isn't much difference between SRV and score with a separate runoff. The only ballots that aren't the same are those where a voter happened to give exactly the same rating to both frontrunners; which, as the number of rating categories increases, becomes vanishingly rare.

I can think of ways to tweak my simulation to include the effects of additional scrutiny on the frontrunners and different turnout in the runoff election. But any such tweak would involve additional assumptions, which could then be criticized. 

--

[Brian Olson]

Hi Jameson,

I checked out your simulator and tried to extend it with another voting method but I'm having some trouble.

I build a new Method class "IRNR" with results() honBallot() and stratBallotFor(); but now I'm getting this message about PersonalityVoter and I don't see where to go in the code to implement what it needs:

  File "/Users/bolson/psrc/vse-sim/dataClasses.py", line 175, in resultsFor

    for voter in voters],

  File "/Users/bolson/psrc/vse-sim/dataClasses.py", line 175, in <listcomp>

    for voter in voters],

  File "/Users/bolson/psrc/vse-sim/dataClasses.py", line 268, in ballotChooser

    return getattr(voter, cls.__name__ + "_" + chooserFun(cls, voter, tally))

AttributeError: 'PersonalityVoter' object has no attribute 'IRNR_strat'

[Jameson Quinn]

I believe the issue may be that the nested stratBallot function is misnamed? Unfortunately, that matters... I know it shouldn't.

I'm working on refactoring the code. It needs it badly. 

Meanwhile, feel free to send a pull request with the code you have, even if it causes failed tests, as long as it doesn't break other voting methods. I'll fix it up; I expect it's something simple but undocumented.

[Warren D Smith]

> Brian Kelly simulated both of these and by his utility measure, SRV
> outperformed score + separate top two. I'd be curious to see if Jameson's
> simulations show something different.

--oh, well, I am extremely dubious of that.
Either (1) my intuition and supporting reasoning are totally wrong,
or (2) this so-called simulation with so-called quality measure,
is completely wrong and/or misleading.

Which? Well, if (1), then it'd sure be interesting to know
what the hell was wrong with this intuition & reasoning.

But I doubt it. I think (2): some stupid decisions must have
been made when designing that simulator;
"garbage in garbage out" as the saying goes.

[Warren D Smith]

And re GIGO, this is one reason to run sanity checks of this ilk, and
when they come out with result "sanity test failed" then you
need to repair your simulator, or else figure out why it wasn't
really a bug due to some subtle reason, which anyhow you need to
figure out before anybody should trust you.

[Mark Frohnmayer]

Kelly's simulator was for honest voters, so by using the quantized score in the runoff instead of a second vote using the full resolution score, runoff votes are discarded for voters who like both candidates pretty much the same. Overall utility benefits when the voter who sees the two as a 4.9999999 and a 4.99999998 is not counted the same as the voter who sees the two as a 0 and a 9.

This seems consistent with your results on honest score plus top two vs. pure score.

--
You received this message because you are subscribed to the Google Groups "The Center for Election Science" group.

To unsubscribe from this group and stop receiving emails from it, send an email to electionscien...@googlegroups.com.

[Mark Frohnmayer]

Warren, do you have the original code you used to generate this: http://rangevoting.org/RVstrat3.html -- it's an interesting data set, and it'd be really interesting to re-run it with SRV. If I read it right, it seems to suggest that sufficiently large voter blocs would see a strategic advantage in bullet voting under score. My guess is that the numbers would look different, and probably more even with SRV.

[Warren D Smith]

On 1/23/17, Mark Frohnmayer <mark.fr...@gmail.com> wrote:
> Kelly's simulator was for honest voters, so by using the quantized score in
> the runoff instead of a second vote using the full resolution score, runoff
> votes are discarded for voters who like both candidates pretty much the
> same. Overall utility benefits when the voter who sees the two as a
> 4.9999999 and a 4.99999998 is not counted the same as the voter who sees
> the two as a 0 and a 9.
>
> This seems consistent with your results on honest score plus top two vs.
> pure score.

--that actually made sense!
So if we are to believe this explanation, then we must
believe that for honest voters, SRV actually gets BETTER with FEWER
allowed score levels (infinity is not best) and keeps getting better as you keep
decreasing, until some optimum number
is reached, where it seems a priori clear the optimum number must
be in {3,4,5,6,7,8,9} although the simulator would need to be
used to tell which one it is.

That is quite interesting! I had not before seen a voting method
with this property that fewer levels, i.e. less expressiveness,
is better (or maybe I had seen, but had not thought of this whole idea
so I did not notice).

One possible example was this: C.Benham had a voting method he called
MCA -- which was kind of like Bucklin voting except with only 3 allowed
score levels. It seemed to work well in my BR tests. And it seemed as though
there was something especially good about "3." Which was mysterious to me.
Well, maybe that now is not as mysterious, because this idea
explains why?!

However, his whole logical design-rationale that Frohnmayer had at one
time told me, for SRV, was for STRATEGIC voters, not honest ones --
and his claim had been that these voters would be inspired by fear of the
runoff, to vote with an honest-ordering in the score-voting part of SRV,
which supposedly would be an advantage versus score+run.
And I then riposted that this was very dubious if using a small number
of score levels, since real voters are scared of
1%-margin victories, since those are common, hence
for voters to behave in the way Frohnmayer wanted,
it was better to employ over 100, perhaps 1000, score-levels.
Which he himself was refusing to do, which was against his entire rationale.

I still say that, and I still think score+run ought to be
superior for strategic voters, than SRV. With score+run the runoff
stage will be 100% honest even with strategic voters, and will
help compensate for strategy-caused distortions in the
score-voting 1st round.

[Mark Frohnmayer]

> --that actually made sense!

Nice!

Regarding strategy, it'd be awesome to be able to run the numbers here: http://rangevoting.org/RVstrat3.html with score+separate runoff and SRV, and add in the strategy of giving your favorite a top score and your second favorite top-1. It should show more conclusively whether particular strategies give voters more of an advantage under SRV, score+top two or score, and then we can put the strategic voting debate with SRV fully to rest.

So where's that code at? :-).