Which game do you hate the least?
Walter S.
wrong with this.
a) It reinforces an already thriving atmosphere of bickering and petty
vindictiveness. ("Man, do I wish I hadn't entered this year just so
that I would have had the pleasure of voting this a 1." Emily Short on
a 2001 comp entry)
b) It rewards the games that are the least offensive, where "least
offensive" is a euphemism for "most inane". In practice this means
that Luminous Horizon (9 ten-votes) wins over Blue Chairs (16
ten-votes) and that Splashdown (2 ten-votes) wins over The Great Xavio
(10 ten-votes). There were 5 games that got more than 9 ten-votes,
which makes me wonder if the winner was in fact a good reflection of
the voters' intentions.
c) It encourages reviewers to review every single entry, even the ones
they obviously don't like, which often serves as an excuse for
pursuing a personal vendetta. ("Content apparently written by A. P.
Hill's younger brother." Emily Short on a 2000 comp entry that appears
to have had nothing to do with A.P. Hill, nor his "younger brother")
I propose that instead of having as many votes as there are entries,
voters should have only three votes: Best (3 points), Second Best (2
points) and Third Best (1 point).
lemonalle
been intentionally giving low scores to undeserving games just to tip the
scales in favor of the one they want to win.
Sidney Merk
news:72c3472c9762ddc9...@localhost.talkaboutgaming.com...
> That's actually a really good idea. It seems to me that some people have
> been intentionally giving low scores to undeserving games just to tip the
> scales in favor of the one they want to win.
Without knowing what each person voted, how would you know, though? Of those
who posted reviews, none seemed unbalanced in that way. The weighting would
have to be done by judges lurking in the shadows. Maybe I'm naive, but I'm
not seeing as much in the score distribution as others might. I can't tell
that any game was purposely sabotaged, or that any game was purposely voted
up. It seems that it would take a pretty large group effort to keep these
individual attempts from being engulfed by the remainder of votes,
anyway....
Did you have a specific example from this year's comp?
---- Mike.
John Prevost
Second Best (2 points) and Third Best (1 point)" idea is that it
prevents people from participating in the judging if they don't have
time to play every one of the submitted games.
It's true that those people (including myself) can skew the
results--for example, I might have been either more or less critical
than most, skewing the games I did play to one side or the other. But,
I think it's worthwhile to allow a larger audience to feel involved in
things. This is the first year that I was actually paying enough
attention to get in when the competition started, instead of just
learning about it after the fact. And, I have to say, this has
encouraged me to try to get more involved in the community. Perhaps
even try my own hand at writing.
If the rules of the competition had required me to play each and every
game or have my votes not count at all, well, I still would have played
as many as I had time to during the competition. But I certainly would
not have felt as happy about having done everything I could do.
For myself, I'd say that having the ifcomp be a "popular award" isn't
that bad. Sure, it adds a certain "lowest common denominator" factor.
But any judging rules are going to impact that. So, the ifcomp is
really a popular competition for fairly short IF. The XYZZY awards are
more of a reward from "the academy". And, of course, people can always
look at individual game reviews and make their own judgement.
Trying to make the ifcomp into a "perfectly balanced" competition is
rather pointless, as any competition will have its own skew.
Do I agree with many peoples' opinions that "Luminous Horizon" probably
should not have been first place? Well, yes. Do I think that means
the competition is dreadfully flawed? Well, no. And really, I thought
I was doing my part in judging the competition just as well when I
voted low on entries that I found to be awful as when I voted high on
entires that I thought were great. (And personally, I reserved 1s and
2s for things that I found well-nigh unplayable, and games that I felt
were well-crafted but unenjoyable rated higher.)
So anyway, it's kind of silly to change things in the way you describe.
Instead, consider setting up a separate award if you think there's a
need.
John.
Graham Grant
"I propose that instead of having as many votes as there are entries,
voters should have only three votes: Best (3 points), Second Best (2
points) and Third Best (1 point)."
Not a bad idea. Not because personal vendettas are much of a problem ,
though. The main problem , I think , is that people have a tendency to
dislike things they don't understand. _Goose , Egg , Badger_ got 19 <5
votes not because they "hated" the author , but because they couldn't
figure out what polysemy is about. With only three votes voters won't
feel forced to vote on games they don't have the slightest clue about.
That way authors won't be punished for coming up with original
concepts.
Graham Grant
Walter S.
>One specific problem with the "only three votes: Best (3 points),
>Second Best (2 points) and Third Best (1 point)" idea is that it
>prevents people from participating in the judging if they don't have
>time to play every one of the submitted games.
Not what I meant. The ten games rule would still stand. "Best" would
thus mean "best game of the ten I've played", assuming you hadn't had
time to play more.
>It's true that those people (including myself) can skew the
>results
You're welcome to skew the results as much as you like, as long as
your reasons are legitimate. The only practically enforceable way of
ensuring that is making the votes too valuable to squander.
Andrew Plotkin
> The main problem , I think , is that people have a tendency to
> dislike things they don't understand. _Goose , Egg , Badger_ got 19 <5
> votes not because they "hated" the author , but because they couldn't
> figure out what polysemy is about. With only three votes voters won't
> feel forced to vote on games they don't have the slightest clue about.
> That way authors won't be punished for coming up with original
> concepts.
This is yet another variation of the "people should only submit
*correct* opinions to the IFComp" idea.
--Z
"And Aholibamah bare Jeush, and Jaalam, and Korah: these were the borogoves..."
*
I'm still thinking about what to put in this space.
Tomasz Pudlo
Sidney Merk <sid-ne...@hot-mail.com (remove dashes)> skrev i
diskussionsgruppsmeddelandet:boCAd.4234$2_4.3590@okepread06...
Assuming that a ten-vote equals 3 points, a nine-vote 2, and an eight-vote
1, (and counting total votes, not average) this is how the top-ten would
look like.
1. Blue Chairs (131)
2. Luminous Horizon (106)
3. All Things Devours (85)
4. Gamlet (83)
5. Sting of the Wasp (77)
6. Square Circle (70)
7. The Orion Agenda (68)
8. The Great Xavio (63)
9. MingSheng (52)
10. Splashdown (42)
The only game that could be suspected of falling victim to "sabotage" is
Gamlet and (perhaps) Xavio. As far as I can see, the only significant change
would be that the comp would seize to be a means of pursuing a personal
vendetta. I'm not sure, however, if that would necessarily be a good thing.
The so-called "IF community" has always been a mighty tempest inside a tiny
tea-pot. What better expression of this than a competition where you can
have the "pleasure" (Emily Short) of voting 1 for a game whose author you
detest?
Quintin Stone
> Assuming that a ten-vote equals 3 points, a nine-vote 2, and an eight-vote
> 1, (and counting total votes, not average) this is how the top-ten would
> look like.
Unfortunately, the existing data cannot be retro-fit into this new
proposed model, since there were no restrictions on the numbers of 10's,
9's, or 8's that judges were allowed to award under the existing system.
There's simply no way to tell how the Comp 04 games would have fared with
this revised scoring.
==--- --=--=-- ---==
Quintin Stone "You speak of necessary evil? One of those necessities
st...@rps.net is that if innocents must suffer, the guilty must suffer
www.rps.net more." - Mackenzie Calhoun, "Once Burned" by Peter David
Stephen Bond
> bickering and petty
> vindictiveness [...] an excuse for
> pursuing a personal vendetta.
Don't worry, maybe _Gamlet_ will do better in the XYZZYs.
Stephen.
vincenzo vinciguerra
Andrew Plotkin <erky...@eblong.com>
> Here, Graham Grant <ggr...@europe.com> wrote:
> > The main problem , I think , is that people have a tendency to
> > dislike things they don't understand. _Goose , Egg , Badger_ got 19 <5
> > votes not because they "hated" the author , but because they couldn't
> > figure out what polysemy is about. With only three votes voters won't
> > feel forced to vote on games they don't have the slightest clue about.
> > That way authors won't be punished for coming up with original
> > concepts.
>
> This is yet another variation of the "people should only submit
> *correct* opinions to the IFComp" idea.
Why is that so hard to swallow? This isn't a political election where you
vote for Bush because he's got a better make-up than Kerry. The IF
competition should be a place where people express well-founded esthetical
convictions, not their ignorance.
Sidney Merk
news:E8FAd.125639$dP1.4...@newsc.telia.net...
>> Did you have a specific example from this year's comp?
>
> Assuming that a ten-vote equals 3 points, a nine-vote 2, and an eight-vote
> 1, (and counting total votes, not average) this is how the top-ten would
> look like.
>
> 1. Blue Chairs (131)
> 2. Luminous Horizon (106)
> 3. All Things Devours (85)
> 4. Gamlet (83)
> 5. Sting of the Wasp (77)
> 6. Square Circle (70)
> 7. The Orion Agenda (68)
> 8. The Great Xavio (63)
> 9. MingSheng (52)
> 10. Splashdown (42)
Hmmm... Xavio bumps Punches, and they're a little shuffled around. Granted,
there are prize and prestige differences that result from that, but in
general, the best games still came up as being voted the best games. I'm not
sure it works by using the actual results (10's, 9's, and 8's), though. It's
likely that many judges voted the same score for multiple games, and that
many judges started (for example) at 8 or 9 and went down from there. The
only way to see a real picture would be if we knew every voter's top three
games, and gave 3, 2, or 1 point accordingly. Presumably the info exists
(since the votes were compiled), but I'm not sure it's worth nagging the
comp coordinators about it. :)
> The only game that could be suspected of falling victim to "sabotage" is
> Gamlet and (perhaps) Xavio. As far as I can see, the only significant
> change
> would be that the comp would seize to be a means of pursuing a personal
> vendetta. I'm not sure, however, if that would necessarily be a good
> thing.
> The so-called "IF community" has always been a mighty tempest inside a
> tiny
> tea-pot. What better expression of this than a competition where you can
> have the "pleasure" (Emily Short) of voting 1 for a game whose author you
> detest?
Honestly, I wasn't even thinking about your game when I asked. You and I
talked about it before the results were even announced, so you probably
remember that I was confused even as to who really wrote it. I initially
thought somebody might be taking a one-sided jab at you, via the competition
entry. On this point, though, you might be right. You (not the game) may
have been voted down regardless of the merits of Gamlet. I'd like to think
those people simply refrained from casting a vote at all, though. Some
probably did. The Gamlet histogram isn't too bottom-heavy, evidenced by a
placing in the top 10.
One way or another, there will be friction. It's a very sedate group in
which there isn't. Further, entering the comp (or releasing a game at all,
for that matter) is an open invitation for all manner of criticism. Even a
3-2-1 scoring guildeline might not have that big an impact on the results.
Instead of getting 1's and 2's, a game would simply receive no votes at all.
The game that gets more votes will still beat the game that gets fewer, just
like the game that gets more 10's than 1's will beat the game that gets more
1's than 10's. I think it works as well as it can, right now.
Fortunately, the comp already allows people to avoid sabotage; enter under
an alias, and don't use subject-matter that links the game back to you. If
you remove the reason for grudge-voting, grudge-voting can't happen. :)
---- Mike
Sidney Merk
news:HxFAd.125643$dP1.4...@newsc.telia.net...
> Why is that so hard to swallow? This isn't a political election where you
> vote for Bush because he's got a better make-up than Kerry. The IF
> competition should be a place where people express well-founded esthetical
> convictions, not their ignorance.
But doesn't the current voting method already make it possible to vote
without ignorance?
---- Mike
John Prevost
> Why is that so hard to swallow? This isn't a political election where
> you vote for Bush because he's got a better make-up than Kerry. The
IF
> competition should be a place where people express well-founded
> esthetical convictions, not their ignorance.
Well, if you want to do that, you had better start restricting the set
of judges to only that set of people who have "well-founded esthetical
convictions", and not letting any old person (like me) judge games. My
votes were based on: 1) Does the game have serious mechanical flaws?
2) Is the concept of the game interesting? 3) Did I enjoy the game?
In general, stuff that did really poorly in part 1 got a very low score
from me--that was along the lines of "the parser works so poorly that
this is unplayable" or "ten completely unreasonable causal chains in
the first ten minutes of the game." That kind of flaw was most likely
to make me put a game down quite quickly, and give it a score of 1 or
2. Assuming no horrible horrible flaws like that, it mainly just meant
+/- 1 or 2 for things that were done especially well or especially
awkwardly.
Parts two and three varied a lot. For example, I found the whole
concept of Gamlet to be quite interesting, but found it to be pretty
un-fun in practice. Contrariwise, I thought Luminous Horizon was
somewhat fun but not actually very interesting. Baseline here is "if
it was fun or if it was interesting, but not both, start at around 5.
If it was both fun and interesting, start around 8."
Of course, my opinions are almost certainly going to be different from
anybody else's. I'm sure other folks found Luminous Horizon to be more
original (and also more fun) than I did. I'm just as sure that other
folks found Gamlet to be less un-fun than I did. Or, more likely, they
had different ways they were looking at games. I was looking for
"working, interesting, fun". Somebody who's been doing this for longer
might be looking for for "different from what's been done before."
ANYway, the key thought is: this is a competition that is *well
publicized*, which has judging open to the world. As such, you're
going to get a pretty wide range of viewpoints. You are certainly
going to get people voting who haven't played much recent IF. So, it's
a little hard to say that "people [should] express well-founded
esthetical convictions, not their ignorance" without saying "We must
exclude those who have not proven that they are aesthetes."
So make a new competition with more restrictions if you want that--with
enforced anonymity of authors, and picked judges who are respected
members of the community. Ifcomp is obviously the populist contest
here, where hoi polloi gets to come in and have a voice.
Walter S. wrote:
> Not what I meant. The ten games rule would still stand. "Best" would
> thus mean "best game of the ten I've played", assuming you hadn't had
> time to play more.
Yes... but I think that works worse than the current system. With the
"rate every game and take the average" model, if you get people to "do
the right" thing and play in random order as they've been asked to, you
get at least a reasonable amount of statistical goodness coming out.
If you ask for "what are the top three games you've rated out of the
at-least ten that you've played, and please play in random order", then
a person who happens to play only ten *truly awful* games is supposed
to say that three of those games were the best three? I mean, I guess
it sort of works--but it doesn't make much sense to me. Or at least,
no more sense than the idea that "If you didn't like any of them, don't
vote at all" is really voting.
John.
Andrew Plotkin
> [...]
Here, vincenzo vinciguerra <vvi...@milano.uu.it> wrote:
> [...]
Here, Tomasz Pudlo <tom...@nycmail.com> wrote:
> [...]
Same troll, new fake names, new attempts to fool people by arguing
with himself.
Sidney Merk
news:1104359247.9...@f14g2000cwb.googlegroups.com...
> vincenzo vinciguerra wrote:
> If you ask for "what are the top three games you've rated out of the
> at-least ten that you've played, and please play in random order", then
> a person who happens to play only ten *truly awful* games is supposed
> to say that three of those games were the best three? I mean, I guess
That's another point I hadn't thought of. If by chance you played what would
turn out to be the bottom 10 games, you'd be voting three of them as the
best. With the current vote, you could simply vote them all low, as
appropriate.
Here's something else to consider (and of personal concern to me). My game
got the fewest number of total votes, save one. If the final score was the
result of cumulative 1's, 2's, and 3's (as opposed to an average), I would
have been at a severe disadvantage. To have a chance in the contest, I
couldn't use anything *but* TADS and Inform...
Mike.
Quintin Stone
> Hmmm... Xavio bumps Punches, and they're a little shuffled around.
Actually, you can count on Punches dropping a lot further if everyone went
back and revoted using the proposed (but poorly thought-out) system. Why?
Trading Punches was coded in Hugo and saw fewer votes because of it.
Under Waltroll S's system, the fewer judges play your game, the less
likely you are to score well. This means that any game not coded in
Inform or TADS2 is at an immediate disadvantage. Adrift? Hugo? Alan?
All screwed from the start. When you look at the numbers, you actually
see that Glulx, TADS3, and yes, even TADS2 games fail to bring in as many
votes as the z-code games. Under the proposed system, judge count is
everything. With the existing system, non-votes aren't factored.
Graham Grant
> Same troll, new fake names, new attempts to fool people by arguing
> with himself.
Thanks for filling us in on your paranoid delusions. We're all *so* interested.
Sidney Merk
news:Pine.LNX.4.58.04...@yes.rps.net...
> On Wed, 29 Dec 2004, Sidney Merk wrote:
>
>> Hmmm... Xavio bumps Punches, and they're a little shuffled around.
> Actually, you can count on Punches dropping a lot further if everyone went
> back and revoted using the proposed (but poorly thought-out) system. Why?
> Trading Punches was coded in Hugo and saw fewer votes because of it.
Yeah, I caught on to that in another portion of the thread. Even with a 1,
2, and 3 voting scheme, you'd still have to average the votes to make it
fair to all entries. So.... I'm officially opposed to the idea. :)
My first replies probably sounded like I'm not open to change at all. It's
not really that. It's just that I'm not sure things would be easier, better,
or more fair if the judging worked differently. In fact, hasn't this come up
every year, with the same conclusion?
---- Mike.
Andrew Krywaniuk
"John Prevost" <j.pr...@gmail.com> wrote in message
news:1104359247.9...@f14g2000cwb.googlegroups.com...
> > Why is that so hard to swallow? This isn't a political election where
> > you vote for Bush because he's got a better make-up than Kerry. The
> IF
> > competition should be a place where people express well-founded
> > esthetical convictions, not their ignorance.
>
> Well, if you want to do that, you had better start restricting the set
> of judges to only that set of people who have "well-founded esthetical
> convictions", and not letting any old person (like me) judge games.
Just because people are free to vote how they want doesn't mean that you
can't encourage them to use a better way. Zarf complains that we have this
discussion every year. So what? Maybe that's what's required to maintain the
equilibrium. Every year, a certain number of people post their voting
systems (some bad, some good), then flamewars ensue and this causes some
rgif readers put more thought into their votes next year. Hopefully, this
ensures that in any given year, the majority of judges will vote sensibly.
Take away the feedback cycle, and who knows...
Andrew
John Prevost
> Hopefully, this ensures that in any given year, the majority of
> judges will vote sensibly. Take away the feedback cycle, and
> who knows...
A good reason to talk about why people voted in what way. Not really a
good reason to change the voting system, though. :) You'll note that
I wasn't saying that we shouldn't talk about voting--just that trying
to change the voting system to *force* enlightened votes is probably a
bad thing to do.
John.
Andrew Plotkin
>
> Just because people are free to vote how they want doesn't mean that you
> can't encourage them to use a better way. Zarf complains that we have this
> discussion every year.
I'm not complaining, I'm pointing out that it *is* the same discussion.
Context.
Andrew Plotkin
> Here, Walter S. <sis...@africamail.com> wrote:
> > [...]
>
> Here, vincenzo vinciguerra <vvi...@milano.uu.it> wrote:
> > [...]
>
> Here, Tomasz Pudlo <tom...@nycmail.com> wrote:
> > [...]
And Graham Grant, I suppose, considering the evidence.
> Same troll, new fake names, new attempts to fool people by arguing
> with himself.
(Or by agreeing with himself, whichever he's in the mood for.)
Joshua Houk
> c) It encourages reviewers to review every single entry, even the ones
> they obviously don't like, which often serves as an excuse for
> pursuing a personal vendetta.
This is, of course, completely humorous given that it's readily apparent
that you have a personal vendetta against someone who you just happened
to single out twice in your post. (Is that "double out", then? But, I
digress.)
Assuming that your post is a valid query (which it's not, but I'm
in a humoring mood), I'd only say that it's pointless to wish for what
could have been. Did games get lower rankings than I think they
deserved? Ya, you betcha. Did games give higher rankings than I think
they earned? Yup, that too. Different people like different things.
You can either accept it or you can whine incessantly about how life
isn't fair. Speaking from experience, the latter option is the most
boring.
Joshua Houk
jlh...@comcast.net
Reese Warner
>
>
> The only game that could be suspected of falling victim to "sabotage" is
> Gamlet and (perhaps) Xavio. As far as I can see, the only significant change
> would be that the comp would seize to be a means of pursuing a personal
> vendetta. I'm not sure, however, if that would necessarily be a good thing.
> The so-called "IF community" has always been a mighty tempest inside a tiny
> tea-pot. What better expression of this than a competition where you can
> have the "pleasure" (Emily Short) of voting 1 for a game whose author you
> detest?
>
>
>
I can't imagine anyone voted to sabotage The Great Xavio. Some voters
were more impressed with what I hoped was a good idea; some were less
tolerant of the relatively weak implementation. Short of getting one
judge to do it all, different judges are just going to have different
criteria.
I should have implemented better. Oh, well. Next time. I know a lot more
now than I did then.
Reese
ems...@mindspring.com
> The only game that could be suspected of falling victim to "sabotage"
is
> Gamlet and (perhaps) Xavio. As far as I can see, the only significant
change
> would be that the comp would seize to be a means of pursuing a
personal
> vendetta. I'm not sure, however, if that would necessarily be a good
thing.
> The so-called "IF community" has always been a mighty tempest inside
a tiny
> tea-pot. What better expression of this than a competition where you
can
> have the "pleasure" (Emily Short) of voting 1 for a game whose author
you
> detest?
Okay, Tomasz. I sense a genuine concern here, and on a subject I've
been wanting to talk to you about since you left ifMUD.
I didn't vote on Gamlet; I didn't play it. I only had time this year to
play about a third of the games. You should know this, because we
discussed it while you were hanging about ifMUD under the name of
Vektor. (No, I didn't have time to play after our conversation; no, I
didn't vote on any games I didn't play.)
As I also said, I don't detest you. In fact, I find this situation
frustrating, since as far as I can tell you have talent as a writer and
a good grounding in IF. I won't say I am happy to have you trolling the
group, but considering that few people have been taking the trolling
seriously for the last few years, it doesn't seem a huge problem any
more, either, except inasmuch as polarizing the community may
discourage people from posting legitimate criticism of games they found
flawed. That would be bad, but it's hard to assess how much it's
happening. I *am* distressed to know that a potential author of
considerable skill and reviewer of considerable insight is wasting his
time on the not-very-interesting-or-difficult enterprise of pissing
people off.
I did, once, expose your sock-puppet personae and suggest that people
not respond further to your jibes; I did notify Google and Telia, once,
that you might be breaking their terms of service by generating so many
fake IDs. That's the sum total of things I have done to you, and I told
you what I was up to and why. (Not exactly "personal vendetta"
territory even then -- you hadn't said much to offend me personally.
But you were setting innocent people against each other by
impersonation; you were causing mass confusion and refusing to quit
when people asked you to. I wanted to put a stop to that if possible.
If I hurt your feelings, I'm sorry for that side-effect, but I still
think what you were doing was wrong and that revealing your method was
the most effective way to limit the damage. It was also my assessment
at the time that you had no intention beyond stirring up bile and that
if people ignored you, you would be too bored to go on; obviously I was
incorrect about that part, but the fact that you do have an active
interest in upsetting people makes it hard to respond fruitfully to any
of your newsgroup remarks.)
Still, I am not sabotaging your IF work. To the best of my knowledge,
neither is anyone else. I saw one person had marked you down on the
grounds that Gamlet was a troll game, but I'm not sure whether that was
because it was you who wrote it, or because he didn't like the ending,
which (as I gather from reviews) might have come across as a slap at
the player. If it's the former, I think that's not a common reaction.
Indeed, several regulars beta-tested for you in good faith.
For the record, I've never voted on or reviewed anything negatively out
of personal dislike for the author. Kallisti was magnificently and
(here's the key) intentionally bad; I knew nothing about James
Mitchelhill other than that he'd written a good IF-Review piece, which
is hardly likely to fill my heart with wrathy flames, but the *game*
practically begged to be colorfully shot down. (He admitted as much
later -- there was a thread about this on alt.games.x-trek, as I
recall.) As for Coffee Quest, it reminded me of the writing in
Amissville, with the same flaws and most of the same virtues -- minus a
bit of the vigor. Hence that comment, which may have been too curt, but
wasn't arbitrary. However, if it would make you feel better, I'll
promise not to vote on or review any of your work. If you choose not to
exercise that option, then you'll just have to have faith in my
integrity -- and since my single vote isn't worth much, you'll have to
have the same faith in the community at large, if you're going to be
involved at a productive level. In my opinion, it's a safe gamble, but
if you're deeply concerned, you can use (heh) yet a different pseudonym
for any future entries.
You clearly have something to contribute here. I hope you'll do so, and
not waste energy on the fear that your efforts are being undercut.
-- Emily
Chaotic Harmony
music contest does (http://dwellingoduels.rock-out.net). In that
contest, everyone enters anonymously. You might think that most people
can guess an author they know by their style, but it's surprisingly
difficult when all the entrys are anonymous... Once voting is over, the
authors are revealed.
Adam Thornton
Tomasz Pudlo <tom...@nycmail.com> wrote:
>Assuming that a ten-vote equals 3 points, a nine-vote 2, and an eight-vote
>1
Don't you bastards dare.
I'm *never* going to write a top-ten game.
I *might* sieze the mighty Golden Banana of Discord by its fleshy shaft
one day, if I, I dunno, get fired and suddenly have oodles of time and
whiskey in order to write _Mentula Macanus: Apocolocyntosis_.
But in order for that to work, I need a bunch of 10s from people who
think this is a spectacular abuse of the z-System, or who think that
Classical Roman Pr0n is really great, and *also* a bunch of 1s from
people who aren't into antler jobs or dead languages, let alone a
combination of the two. Obviously a simple 1-2-3 system isn't going to
help me there.
Adam
Andrew Plotkin
difficult to guess *many* of the authors, but easy to guess some.
("Earth and Sky 3 -- by Anonymous!" doesn't work.) And if someone
wanted to leave clues about his identity, it would be impossible to
prevent.
So, in practice, it would wind up being the same as the current
situation -- people enter anonymously if they want to, or not.
Kevin Venzke
"Walter S." <sis...@africamail.com> wrote in message
news:3c4994a3.04122...@posting.google.com...
> I propose that instead of having as many votes as there are entries,
> points) and Third Best (1 point).
Do you think a majoritarian method would help? It would consider
whether more judges ranked A to be better than B than vice versa,
instead of considering an average ranking, which can be sunk or elevated
by a minority of judges.
Tideman's method seems particularly useful since it necessarily
generates a ranking of all the alternatives.
http://en.wikipedia.org/wiki/Ranked_Pairs
Kevin Venzke
aph...@altavista.com
> Amissville, with the same flaws and most of the same virtues -- minus
a
Ahhh, Amissville, doesn't the name ring bells and a host of angels
singing upon hearing? It does for me.
Wow! (notices Emily's skirt is crooked)
*yanks skirt back on straight making sure seem is perfectly aligned
with buttcrack*
Closed eye and Raising hand.."NO need to thank me"
Santoonie.com has a celebratory movie in honor of New Year's eve. A
little music and some hilariousness:
Tomasz! *Tips Hat* My apologies for intruding on your host with an
ashameful santoonie plug. Perhaps this will defer any ill
repute..*tosses a hefty gold purse*.
http://www.santoonie.com/videos/bash.wmv
http://www.santoonie.com/videos/aphill.wmv
I received a most glorious box of Biltmore Estates Wines this past
holiday to which I am currently enjoying. A toast to all.
*raises glass*
A.P. Hill
Happy 2005
Jake Wildstrom
>tea-pot. What better expression of this than a competition where you can
>have the "pleasure" (Emily Short) of voting 1 for a game whose author you
>detest?
Emily Short already said a lot of this, but it's worth emphasizing
one side point which was critical for me.
I encountered three games by authors I had prior negative
preconceptions of: Ninja, Zero, and Gamlet. The former two were
reasonably easily judged on their own merits (and I doubt my low
opinion was actually significantly affected by their authors'
reputations), but I chose specifically not to rate Gamlet for one
reason: the appearance in-game of your identity. Your name, and your
agenda as a poster on this group, are extrinsic information. I may
have a strong feeling on them, but they're not part of the game. But
putting yourself (or your identity, or some sort of pastiche like the
one you've assembled as your identity here) in the game necessarily
broke that down, and basically _asked_ people to import their
preconceptions. I didn't think I could do so impartially, and
refrained from voting.
I'm not saying that self-insertion is forbidden -- Chris Klimas did it
subtly enough that almost nobody actually noticed -- but if you have a
strong real-world personality and you associate a game character with
the name attached to that character, you're basically _asking_ people
to judge, not just your work, but you. Some of us surely felt
uncomfortable doing so and like myself refrained from voting. Others
may not, but I can hardly speak for them.
To reiterate one of Emily's points: you've established that you have
significant talent and imagination and interesting ideas to share. Why
are you wasting your energy on the trivial and pointless task of
pissing people off? I for one would be interested in seeing your next
work of IF.
--
D. Jacob (Jake) Wildstrom, Math monkey and freelance thinker
"A mathematician is a device for turning coffee into theorems."
-Alfred Renyi
The opinions expressed herein are not necessarily endorsed by the
University of California or math department thereof.
David Whyld
"Jake Wildstrom" <dwil...@zeno.ucsd.edu> wrote in message
news:cr4f3g$enb$1...@news1.ucsd.edu...
> To reiterate one of Emily's points: you've established that you have
> significant talent and imagination and interesting ideas to share. Why
> are you wasting your energy on the trivial and pointless task of
> pissing people off?
For the same reason that all trolls troll: they get a kick out of annoying
people.
m...@df.lth.se
David Whyld wrote:
> For the same reason that all trolls troll: they get a kick out of
annoying
> people.
I don't think it's quite that simple. Most trolls I've encountered have
some
sort of agenda. Pudlo seems to have at least two.
(Of course, having - or seeming to have - an agenda is one thing that
tends
to annoy people).
--
Magnus Olsson (m...@df.lth.se)
PGP Public Key available at http://www.df.lth.se/~mol
Magnus Olsson
Andrew Plotkin <erky...@eblong.com> wrote:
>Here, Chaotic Harmony <blinkthe...@gmail.com> wrote:
>> Perhaps there's a case for anonymous entrys, like the dwellingduels
>> music contest does (http://dwellingoduels.rock-out.net). In that
>> contest, everyone enters anonymously. You might think that most people
>> can guess an author they know by their style, but it's surprisingly
>> difficult when all the entrys are anonymous... Once voting is over, the
>> authors are revealed.
>
>With the IFComp, if everyone entered anonymously, it would be
>difficult to guess *many* of the authors, but easy to guess some.
>("Earth and Sky 3 -- by Anonymous!" doesn't work.) And if someone
>wanted to leave clues about his identity, it would be impossible to
>prevent.
That's true, but I still think anonymizing the Comp would be a good
idea. With the current system, you have to take active steps to become
anonymous - create a pseudonym, maybe register an email addy for your
pesudonym - and there is also a psychological resistance to doing it:
"If everybody else is entering under their own names, then why should
I enter under a pseudonym?".
With an anonymized system, you'd have to take active steps to make
your game identifiable. And why would you want to?
Sequels are a problem, but I'm not sure that entering sequels in the
Comp is a good idea at all (no shadow on _Luminous Horizon_ intended).
Mike Snyder
news:33vaumF...@individual.net...
> That's true, but I still think anonymizing the Comp would be a good
> idea. With the current system, you have to take active steps to become
> anonymous - create a pseudonym, maybe register an email addy for your
> pesudonym - and there is also a psychological resistance to doing it:
> "If everybody else is entering under their own names, then why should
> I enter under a pseudonym?".
I like the idea of everyone entering under a pseudonym, but...
> With an anonymized system, you'd have to take active steps to make
> your game identifiable. And why would you want to?
...I think in some cases it would take active steps *not* to enter an
identifiable game. For instance, wouldn't most people be able to figure out
a Robb Sherwin game, based on the style of it and his customizations to the
Hugo defaults? Likewise, I'd probably have to avoid including any rendered
graphics or music (especially if I do more of it between now and the next
comp) to avoid making a link back to me.
I suppose I could enter in Tads or Inform, but I'm more inclined to get
better with what I've started, rather than switch gears. :)
> Sequels are a problem, but I'm not sure that entering sequels in the
> Comp is a good idea at all (no shadow on _Luminous Horizon_ intended).
I guess if the rule was "your entry must not do anything that makes your
real identity obvious", then it would be up to the authors to not enter
sequels, and take whatever steps necessary to disassociate the entry with
any of their previous work. I'd suspect a lot of disqualifications,
though...
---- Mike.
Magnus Olsson
Mike Snyder <wy...@prowler-pro.com> wrote:
>"Magnus Olsson" <m...@df.lth.se> wrote in message
>news:33vaumF...@individual.net...
>> your game identifiable. And why would you want to?
>
>...I think in some cases it would take active steps *not* to enter an
>identifiable game. For instance, wouldn't most people be able to figure out
>a Robb Sherwin game, based on the style of it and his customizations to the
>Hugo defaults?
Maybe, but such author identification is usually harder than you'd
think. People may say "That author's style is very similar to Robb
Sherwin's," or "this feels like a Zarf game," but it's usually quite
hard to tell with any certainty.
ANyway, I think the point with anonymizing the entries is not to make
it impossible to figure out who wrote a certain game, but to minimize
prejudice. What I'd like to see eliminated is the judges getting
preconceptions and/or expectations about a game because they recognize
the author's name. If they can guess who wrote it after they've played
it, well, then they have played it so hopefully they've already formed
their opinion.
This is also why I don't like sequels: anybody seeing that a certain
entry is a sequel to a previous entry (or sharing the same world or
characters) is going to get certain expectations, good or bad.
And, for the record: I don't think the expectations or preconceptions
are a bg problem per se. The big problem is that the very possibility
of prejudice makes the judgin process suspect. If, say, Zarf wins the
next Comp, there will be people saying that he won just because he's
Zarf. If the game was entered anonymously, people wouldn't be able to
say that.
>I guess if the rule was "your entry must not do anything that makes your
>real identity obvious", then it would be up to the authors to not enter
>sequels, and take whatever steps necessary to disassociate the entry with
>any of their previous work. I'd suspect a lot of disqualifications,
>though...
I'm afraid such a rule would be virtually impossible to enforce.
However, *if* the well-known author X enters an anonymous game but
lets the command XYZZY print out the text "This game was written by X",
then he'll deserve any accusations that he won just because of his name :-).
aph...@altavista.com
It's real simple, this community just needs a panel of bipartisan
judges, People who care less about IF but possess common karnal
knowledge enough to judge games. People like the staff at the Weather
Channel could judge our games, or.. editors from Sports Illustrated.
Voting, judging and dancing amongst ourselves simply spins in circles.
That's why I don't hide behind a psudeonism, I come to the party pants
off.
A.P. Hill
Mike Snyder
> ANyway, I think the point with anonymizing the entries is not to make
> it impossible to figure out who wrote a certain game, but to minimize
> prejudice. What I'd like to see eliminated is the judges getting
> preconceptions and/or expectations about a game because they recognize
> the author's name. If they can guess who wrote it after they've played
> it, well, then they have played it so hopefully they've already formed
> their opinion.
I'm getting a sense of deja vu -- was this same topic brought up recently?
It seems like I remember reading (and possibly responding to) another thread
about entering the comp with pseudonyms. If I'm right, my thinking then was
that authors can already use pseudonyms if they want to avoid prejudice. But
I see what you're getting at, I think -- it would help combat the
possibility that a popular author will be voted highly, even if the vote is
out of sync with the entry. Or, at least serve to counter any arguments
after the results that this must be what happened.
> This is also why I don't like sequels: anybody seeing that a certain
> entry is a sequel to a previous entry (or sharing the same world or
> characters) is going to get certain expectations, good or bad.
Yikes -- I'm working on a Trading Punches prequel for Spring Thing '05...
although it'll be a non-issue if I don't get cracking on it again.
> And, for the record: I don't think the expectations or preconceptions
> are a bg problem per se. The big problem is that the very possibility
> of prejudice makes the judgin process suspect. If, say, Zarf wins the
> next Comp, there will be people saying that he won just because he's
> Zarf. If the game was entered anonymously, people wouldn't be able to
> say that.
Point taken (although I think his prior entries have been anonymous,
anyway).
In a way, this is more fair to the judges too. I know it's tounge-in-cheek
when an IF "cabal" or inner-circle is mentioned (at least, I hope it is --
maybe I need to play Stephen Bond's game), but to some of us, it sometimes
seems very real. This was especially true for me after IF-Comp '99 (and even
prior, when I was gung-ho on QB IF development). Even if misperceived, it
seemed real enough. If the entry is written by an author recognized as one
of the best, it seems possible (even likely) that many of us are going to
think the fault is with our ability to understand the game, rather than with
the game's ability to make sense. It seems like a burden that *could* be
avoided if every game in the comp was written by an unknown author.
> >I guess if the rule was "your entry must not do anything that makes your
> >real identity obvious", then it would be up to the authors to not enter
> >sequels, and take whatever steps necessary to disassociate the entry with
> >any of their previous work. I'd suspect a lot of disqualifications,
> >though...
>
> I'm afraid such a rule would be virtually impossible to enforce.
Well, it could become very subjective.
During Comp '04, I dropped the pseudonym for many of the people I talked to.
It just felt weird and misleading sometimes.
> However, *if* the well-known author X enters an anonymous game but
> lets the command XYZZY print out the text "This game was written by X",
> then he'll deserve any accusations that he won just because of his name
:-).
But, he'll still have won... :)
---- Mike.
PJ
> "Magnus Olsson" <m...@df.lth.se> wrote in message
> > This is also why I don't like sequels: anybody seeing that a
certain
> > entry is a sequel to a previous entry (or sharing the same world or
> > characters) is going to get certain expectations, good or bad.
>
> Yikes -- I'm working on a Trading Punches prequel for Spring Thing
'05...
> although it'll be a non-issue if I don't get cracking on it again.
I think a good rule would be that you can't do prequels or sequels in
the same comp, but you could enter them in different comps, of course.
Since a sequel from the IF Comp to the C32 comp would be quite
different, it wouldn't matter that it was a sequel. The same thing for
Spring Thing vs. IF Comp -- they are supposedly different enough in
their format that there shouldn't be a problem. But if someone wrote
one really long, good game, divided it into thirds (a la Zork) for the
IF Comp, then won every year, the question would be, why not just enter
it into the Spring Thing as a single entry? A no sequel/prequel idea
for each comp would also foster the goal of getting people to develop
new ideas, not just expanding on old ones.
> > And, for the record: I don't think the expectations or
preconceptions
> > are a bg problem per se. The big problem is that the very
possibility
> > of prejudice makes the judgin process suspect. If, say, Zarf wins
the
> > next Comp, there will be people saying that he won just because
he's
> > Zarf. If the game was entered anonymously, people wouldn't be able
to
> > say that.
>
> Point taken (although I think his prior entries have been anonymous,
> anyway).
There really is no way to make the authors anonymous. The anonymous
entries always seem to be known by some of the judges and people can
always think of ways to let folks know they have an entry and which one
it is. I don't think the top authors get more benefit from their names
as much as the scoring seems to contribute not to identifying the
"best" game, but an "optimax" game being selected. People have
commented over and over how EAS3: LH won because it was most people's
2nd or 3rd choice. I don't think Paul's name got votes that high, but
rather it was the quality of the game that led most folks to put it in
their top three.
On the other hand, either ATD or Blue Chairs arguably could've,
should've won, maybe. So considering how the voting system should be
weighted is probably the real solution. I'm not familiar with the
current algorithm, but if it's just highest average score then
obviously that's going to hurt otherwise superb but challenging games
like those two. Introducing a partial pairs or similar scoring scheme,
or weighting 10 votes higher than 9 votes, etc., would cure a lot of
that problem.
PJ
Mike Snyder
news:1104849677.0...@c13g2000cwb.googlegroups.com...
> On the other hand, either ATD or Blue Chairs arguably could've,
> should've won, maybe. So considering how the voting system should be
> weighted is probably the real solution. I'm not familiar with the
> current algorithm, but if it's just highest average score then
> obviously that's going to hurt otherwise superb but challenging games
> like those two. Introducing a partial pairs or similar scoring scheme,
> or weighting 10 votes higher than 9 votes, etc., would cure a lot of
> that problem.
Somewhere else in this topic -- or in another topic entirely -- it was
pointed out that maybe the Comp results aren't a reflection of what the
judges intended. I'm pretty sure the comp score *is* the average of all
votes given for a game (i.e., (v1+v2+...vn) / n). I'd have to learn more
about those alternate scoring methods to understand what you mean. As it is,
it seems fair as-is. Everybody's vote merits the same weight. If a judge
strongly dislikes a game, can't there also be a case made that the game
should score low instead? Take a split decision -- half low votes, half high
votes. Was the intention of the judgining community that the game win, or
that it lose? Averaging the score takes care of that. It finishes in the
middle.
So the notion that the results don't reflect voter intentions is primarily
true of games with lower standard deviations, correct? In other words, a
game that most voters felt was a second, third, or fourth place game can win
because better games are dragged down with low votes? I still have trouble
with that, primarily because enough voters were in disagreement to bring
down the better games's average (i.e., they didn't consider it to be
better). Again, unweighted averaging seems the most fair reflection of all
voter's scores.
I've never been particularly strong in math, though. :) Maybe there is a way
to revise the average score that is more fair and more indicative of voter's
intentions. I just don't know what it would be.
---- Mike.
Magnus Olsson
Mike Snyder <wy...@prowler-pro.com> wrote:
>"PJ" <pete_...@hotmail.com> wrote in message
>news:1104849677.0...@c13g2000cwb.googlegroups.com...
>
>> On the other hand, either ATD or Blue Chairs arguably could've,
>> should've won, maybe. So considering how the voting system should be
>> weighted is probably the real solution.
But a solution to which problem?
I mean, everybody can find fault with the Comp and the voting system.
And it seems that more or less everybody does. But is there any real
agreement about what the problem is - that is, is there a consensus on
what the Comp *ought* to be and which game *ought* to win?
Obviously not - because if there were such a consensus, everybody would
give that game a high score and it would win even with the current
system.
>> I'm not familiar with the
>> current algorithm, but if it's just highest average score then
>> obviously that's going to hurt otherwise superb but challenging games
>> like those two. Introducing a partial pairs or similar scoring scheme,
>> or weighting 10 votes higher than 9 votes, etc., would cure a lot of
>> that problem.
But is it so obvious that a "superb but challenging" game really should
win?
In other words, are the people who give a game a low score because they
don't get it (for whatever reason) necessarily *wrong*?
It all boils down to this: If the best game is to win, then who
decides what is best?
>Somewhere else in this topic -- or in another topic entirely -- it was
>pointed out that maybe the Comp results aren't a reflection of what the
>judges intended.
But how do you decide what a collective of judges really intended?
>I'm pretty sure the comp score *is* the average of all
>votes given for a game (i.e., (v1+v2+...vn) / n).
That's correct.
>As it is,
>it seems fair as-is. Everybody's vote merits the same weight.
I think you've hit the nail on the head.
The entire point of the current voting system is that it's *fair*,
in the sense that all votes are given the same weight. Many people
think the wrong game won (that is, their personal preferences don't
agree with the results of the vote); theoretically, we can get a
Comp where *everyone* thinks the wrong game won. But at least they
can't complain that the voting procedure was unfair.
Yes, this turns the Comp into a popularity contest. So what? Let me
remind you that the original purpose of the Comp was to promote interest
in text adventures and to give people an incentive to write more games,
*not* to decide which game is in any way the "best".
And, if I may end on a cynical note, *any* competition with more than
one judge will in some sense be a popularity contest. The only way to
ensure that the game *you* think is the best will win is to hold your
own contest with yourself as the only judge. Nothing stops you from
doing that. But as soon as there is more than one judge, you have the
potential for "unfair" results where "quality" has to play second fiddle
to finding the common denominator of the judges.
Mike Snyder
>>Somewhere else in this topic -- or in another topic entirely -- it was
>>pointed out that maybe the Comp results aren't a reflection of what the
>>judges intended.
>
> But how do you decide what a collective of judges really intended?
That's what I was getting at. :)
---- Mike.
Andrew Plotkin
>
> On the other hand, either ATD or Blue Chairs arguably could've,
> should've won, maybe. So considering how the voting system should be
> weighted is probably the real solution. I'm not familiar with the
> current algorithm, but if it's just highest average score
It is.
> then obviously that's going to hurt otherwise superb but challenging
> games like those two. Introducing a partial pairs or similar scoring
> scheme, or weighting 10 votes higher than 9 votes, etc., would cure
> a lot of that problem.
I am fond of the ranked-pairs voting system; I've used it for a
(non-IF) competition, and it seems to work pretty well. However, it
does nothing to solve the superb-but-challenging "problem".
I put "problem" in quotes because it's a biased way to describe the
situation. If you gave the game a 10, the "problem" is that
unperceptive voters didn't understand the game, and gave it lower
scores, dragging it down. If you gave the game a 5, the "problem" is
that narrowly-focussed voters ignored the real weaknesses in the game
and scored it only on its best points, thus pushing it incorrectly
high.
Objectively, what's *happened* is that voters *disagree* on the
quality of the game. The voting system has to reconcile that and come
up with a single result. Averaging the scores is brutally simple and
fair.
A ranked-pairs system would do more or less the same thing by a
different algorithm. The voters who ranked A above C would be
counterbalanced by the voters who ranked C above A. Game B (which
everyone put in second place) would do well against both of them. The
exact outcome would depend on the exact margins. (Just as, with score
averaging, the exact outcome depends on where the decimal places fall
out -- 7.85 versus 7.39, for IFComp04.)
The advantage of ranked pairs is that there's no normalization problem
-- you don't have to worry about whether one player's "10" is the same
as another player's "9". The disadvantage is that it's easier for ties
to occur.
(Another disadvantage is that it becomes impossible to compare scores
between competitions. I think that's already shaky -- you can say that
_Slouching Towards Bedlam_'s 8.39 (in 2003) beats _Luminous Horizon's_
7.85, but I wouldn't put a lot of weight on that, because I think most
voters normalize within a given IFComp. However, in a ranked-pairs
voting system, you couldn't even say that much. You'd just know that
each game won against its competitors.)
PJ
> I am fond of the ranked-pairs voting system; I've used it for a
> (non-IF) competition, and it seems to work pretty well. However, it
> does nothing to solve the superb-but-challenging "problem".
>
> I put "problem" in quotes because it's a biased way to describe the
> situation. If you gave the game a 10, the "problem" is that
> unperceptive voters didn't understand the game, and gave it lower
> scores, dragging it down. If you gave the game a 5, the "problem" is
> that narrowly-focussed voters ignored the real weaknesses in the game
> and scored it only on its best points, thus pushing it incorrectly
> high.
Yes. I should've started my post by saying that, before you change the
voting system, the organizers (or the public could vote I guess) should
decide whether the outcomes represented a "problem" or not. If the
public is truly convinced that EAS3:LH was the BEST game, then there is
no problem. If the public isn't convinced of that, but thinks the
process was fair (see Paul Hamm, Olympics), then maybe you don't have a
problem but you might want to tweak the comp design a bit. But that
might also imply posting a list of judging rules over what constitutes
"good" that would start a different war over what bias means.
> Objectively, what's *happened* is that voters *disagree* on the
> quality of the game. The voting system has to reconcile that and come
> up with a single result. Averaging the scores is brutally simple and
> fair.
I agree that it is conceptually fair and in practice has a "process"
equity to it. I would say that because not all voters vote on all
games, there is a slight mathematical bias (either for or against,
depending on how good or bad the game is and what its standard
deviation is) in favor of games that have fewer voters (which was the
case for EAS3:LH vs. ATD & Blue Chairs). But since the number of votes
is relatively small in this case, it would be ridiculous to say that
this actually affected the comp this year.
> A ranked-pairs system would do more or less the same thing by a
> different algorithm. The voters who ranked A above C would be
> counterbalanced by the voters who ranked C above A. Game B (which
> everyone put in second place) would do well against both of them. The
> exact outcome would depend on the exact margins. (Just as, with score
> averaging, the exact outcome depends on where the decimal places fall
> out -- 7.85 versus 7.39, for IFComp04.)
That is correct. But what you would have (depending on how you did the
analysis) is the ranked choices of each judge with a clearly preferred
winner, 2nd, etc. for each judge in each case. This is important
because while some folks reserve a 10 for their top game only, others
distribute them to every game they feel is excellent. This inherently
will result in some degree of mathematical bias, which a partial pairs
approach would limit. And I'm not sure that the low votes balance out
the high votes because people are probably more reluctant to give out
1s and 2s than they are 10s (see US universities, grade inflation over
the last 20 years).
> The advantage of ranked pairs is that there's no normalization
problem
> -- you don't have to worry about whether one player's "10" is the
same
> as another player's "9". The disadvantage is that it's easier for
ties
> to occur.
Yes. And yes. I don't think ties are a real problem, though. It
happens all the time in other types of competitions and I don't think
it really gripes anybody. And with even a few hundred judges, I'd be
surprised if ties occurred a lot of the time.
> (Another disadvantage is that it becomes impossible to compare scores
> between competitions. I think that's already shaky -- you can say
that
> _Slouching Towards Bedlam_'s 8.39 (in 2003) beats _Luminous
Horizon's_
> 7.85, but I wouldn't put a lot of weight on that, because I think
most
> voters normalize within a given IFComp. However, in a ranked-pairs
> voting system, you couldn't even say that much. You'd just know that
> each game won against its competitors.)
Lifetime achivement and "Best of the Last 10 Years" are different
comps, I think. Deciding how to rate winners from later years vs.
games that helped define the genre is always tricky. Certainly a "10"
does not mean the same thing in a weak year as a strong year for the
comp.
In a nutshell, I'm not sure there's anything wrong with IF Comp either.
But if there is some perception of either bias or incorrect results,
there are many other ways to set up judging than a "raw" popularity
contest. I think the first thing you'd have to do is poll people and
see whether there is a "problem" or not. But then the poll might be
biased as well ...
PJ
Andrew Plotkin
> Andrew Plotkin wrote:
>
> > I am fond of the ranked-pairs voting system; I've used it for a
> > (non-IF) competition, and it seems to work pretty well. However, it
> > does nothing to solve the superb-but-challenging "problem".
> >
> > I put "problem" in quotes because it's a biased way to describe the
> > situation. If you gave the game a 10, the "problem" is that
> > unperceptive voters didn't understand the game, and gave it lower
> > scores, dragging it down. If you gave the game a 5, the "problem" is
> > that narrowly-focussed voters ignored the real weaknesses in the game
> > and scored it only on its best points, thus pushing it incorrectly
> > high.
>
> Yes. I should've started my post by saying that, before you change the
> voting system, the organizers (or the public could vote I guess) should
> decide whether the outcomes represented a "problem" or not.
The IFComp voting system has remained essentially unchanged since
1996, so it's safe to say that the organizers think it works.
> If the public is truly convinced that EAS3:LH was the BEST game,
> then there is no problem. If the public isn't convinced of that, but
> thinks the process was fair (see Paul Hamm, Olympics), then maybe
> you don't have a problem but you might want to tweak the comp design
> a bit.
I think you're making an unconscious assumption that "the public" has
a simply ordered preference scale, just like a person does, with one
game as "the best". Not that people have simple orderings either! But
a person can simplify his preferences *into* an ordering, or a list of
numbers, and satisfy himself that he's done it "right". A group of
people (with varying opinions) will *never* have that sense.
There is no such thing as a voting system which guarantees that the
majority of voters get their first-place choice in first place. If
that's what you're envisioning, then pick that thought out and make it
stand in the corner.
Really the case of any given Comp is the same as the case of the Comp
rules themselves. People frequently post and express dissatisfaction
with the current Comp rules. However, that does not mean that the
rules are broken, or even that they are not what "the public" wants.
No other rules proposal has ever gotten a significant amount of
support. In fact, a lot of the dissatisfaction never even gets boiled
down to a concrete, practical proposal. So it's not de facto evidence
that anything's wrong.
> And I'm not sure that the low votes balance out the high votes
> because people are probably more reluctant to give out 1s and 2s
> than they are 10s
That's not an obvious assumption. I typically give one 10 and
two-to-four 1s. I see a lot of reviewers saying things like "10 means
best in the competition", or "10 means it's as good as the best I've
ever seen", or "I didn't think any game this year was worth a 10."
I've never seen anyone be that shy about 1s. Some people say "1 means
unplayable, or not an IF game" but they're still willing to bestow
that honor on as many entries as deserve it.
PJ
> I think you're making an unconscious assumption that "the public" has
> a simply ordered preference scale, just like a person does, with one
> game as "the best". Not that people have simple orderings either! But
> a person can simplify his preferences *into* an ordering, or a list
of
> numbers, and satisfy himself that he's done it "right". A group of
> people (with varying opinions) will *never* have that sense.
Actually, I'm not making any such assumption. I'm just responding to
this thread, which started with the idea that Luminous Horizon wasn't
the best game in the Comp. I happen to agree with that. The question
isn't my preference, however, but whether a majority of the judges, if
asked to vote for the "single best game" instead of scoring all the
games they played, would have come up with Luminous Horizon as the
winner. It's a bit of an esoteric question, since the comp is over.
You could design scoring systems that answer that question and have the
same sense of "process fairness." I'm not sure that anyone but
disappointed authors is lobbying for that, though. And probably not
the vast majority of them, either. It's just an interesting exercise
to discuss alternatives.
> There is no such thing as a voting system which guarantees that the
> majority of voters get their first-place choice in first place. If
> that's what you're envisioning, then pick that thought out and make
it
> stand in the corner.
I'm not sure that's the case. Where you are going wrong here is
assuming that voters voted on their "first place choice." They didn't.
They gave numerical scores, basically using any system they wanted to,
which may have had modest amounts of bias in them. It results in a
"first place outcome," but voters weren't asked to rank the games in
order and have a score assigned to each game based on that ranking. AP
sports polls for the NCAA do this all the time, and while there is
certainly subjective bias, you can look at the results and see that the
team with the most number 1 votes is usually rated "#1." It all
depends on what you *want* to measure. Right now, IF Comp basically
measures overall opinion about multiple facets of each game in the Comp
rather than "single best game." I personally don't have a problem with
that. But the public-at-large in the IF Community might gripe about it
from time to time, e.g., this thread.
> Really the case of any given Comp is the same as the case of the Comp
> rules themselves. People frequently post and express dissatisfaction
> with the current Comp rules. However, that does not mean that the
> rules are broken, or even that they are not what "the public" wants.
> No other rules proposal has ever gotten a significant amount of
> support. In fact, a lot of the dissatisfaction never even gets boiled
> down to a concrete, practical proposal. So it's not de facto evidence
> that anything's wrong.
I agree. You know the judging rules coming in, so there are probably
optimum strategies to follow if you are really trying to "game" the
voting. I don't think most people are thinking that way, however, and
the randomness of scoring as well as the sophisitication of the game
required to get a high score necessarily means that any game that
"wins" the comp deserves it.
> > And I'm not sure that the low votes balance out the high votes
> > because people are probably more reluctant to give out 1s and 2s
> > than they are 10s
>
> That's not an obvious assumption. I typically give one 10 and
> two-to-four 1s. I see a lot of reviewers saying things like "10 means
> best in the competition", or "10 means it's as good as the best I've
> ever seen", or "I didn't think any game this year was worth a 10."
> I've never seen anyone be that shy about 1s. Some people say "1 means
> unplayable, or not an IF game" but they're still willing to bestow
> that honor on as many entries as deserve it.
>From reviews I've read, I wouldn't say everyone follows the same
strategy. But certainly any bias in either direction is relatively
small. One person can't turn a game's scores by giving everyone a 1,
and the same is true for 10s. But if people were really spamming the
judging (e.g., Yao Ming wins the most NBA All-Star votes), I think it
would have to be more obvious.
As I said in my other post, there doesn't appear to be a real problem
with outcomes. But it might be interesting to take a post-Comp poll of
judges on which of the Top 3 games they believed to be the "single best
game." I'm not going to bother with that idea myself. EAS3:LH was an
excellent game, and a well-deserved win. Kudos to Paul. Personally,
I'm done with this thread.
PJ
Brent VanFossen
<erky...@eblong.com> wrote:
>The IFComp voting system has remained essentially unchanged since
>1996, so it's safe to say that the organizers think it works.
For what it's worth, I think the system works. This discussion comes
up every year (as you well know) and we never gather a concensus on
whether and how to change the judging. Part of the beauty of the
current system is that it is easy to understand, and easy to use. That
brings in more voters and more players to the community.
As for anonymous entries, there are many games entered in the comps
that are never released in a post-comp version. That means that had
the author been required to release it with a pseudonym, it would
reside there in the if-archive forever and never show the author's
real name. Many of those games will remain on hard-drives and on other
websites in their original version, even if the author releases
post-comp.
I would hate to see a pseudonym requirement on the comp, because as
the author of a creative work released to the public, I should have
the right to have my name associated with it (or not, as I choose).
---
Brent VanFossen
Cirk R. Bejnar
> Andrew Plotkin wrote:
> > There is no such thing as a voting system which guarantees that the
> > majority of voters get their first-place choice in first place. If
> > that's what you're envisioning, then pick that thought out and make
> > it stand in the corner.
>
> I'm not sure that's the case.
I think Andrew's point is just that in situations where there are
multiple alternatives that there will be situations such as
Game A 30% think its the best
Game B 45% think its the best
Game C 25% think its the best
So that whatever the outcome of the voting (which can vary depending on
the system used) a majority (over 50%) of the people will agree that
the best game did not win.
Cirk R. Bejnar
carolyn...@yahoo.com
> ...if you have a
> strong real-world personality and you associate a game character with
> the name attached to that character, you're basically _asking_ people
> to judge, not just your work, but you. Some of us surely felt
> uncomfortable doing so and like myself refrained from voting. Others
> may not, but I can hardly speak for them.
Before the rest of my life opened its maw (raar) and ate my head
(chomp), I had set the personal goal for myself of playing every IF
game and then reviewing at length, with notes on what was good and what
was bad in every game along with suggestions on how to improve. (I
like reviewing things in contexts where their creators will see the
reviews. It makes me feel useful.)
Gamlet threw me for an utter spin. I didn't WANT to review the author
along with the game, but I couldn't figure out how to separate the two.
I found the self-insertion very dismaying.
If you're curious, you can find my review below the line, which was
written immediately after playing and prior to reading anyone else's
opinions on Gamlet; I now have the answers to all the questions it
poses, but I didn't when I was writing. (I should note that it somehow
never occurred to me to try the WALKTHROUGH command to get... duh... a
walkthrough. I kept looking for a separate file. I also still haven't
finished Gamlet, though I do still mean to do so, insulting or no.)
Carolyn
------------------------------------------
I really didn't know what to make of this game. Judging by the "about"
and the opening sequence, it looks to me as though it is most likely
one of two things:
1) someone taking a peculiar slap at an unpopular raif poster, or
2) the same unpopular poster taking a slap at everyone else through
bizarre self-mockery.
Whichever way it may be, reading the "about" left me feeling rather on
shaky ground. I don't want to make a statement about the poster; I
only want to play and review the game. I feel rather as if I've been
included in a conflict that I didn't want to be part of, and that
dismays me in and of itself. Can't the acrimony stay on the newsgroup?
To explain a bit more, for those who haven't loaded the game, this game
is apparently a rendition of Hamlet as executed in a Jewish context --
"the House of Pudlo". I found it pretty strange. I don't know how
those who are actually part of the Jewish culture would respond to it,
and I myself am not sufficiently familiar with it to figure out whether
this should be seen as appropriate or insulting. A lot seems to depend
upon the context in which it was written... just as in the issue of the
author's true identity.
Questions of appropriateness and possible hostile undercurrents aside,
the game is stunning. The writing is extraordinary (I didn't find even
one error) and the mechanics are wonderous. You enter rooms, and they
show you a first impression before changing their description to
reflect your current perspective and opinion. You examine objects, and
they give you subtle hints about what to do next. You type "attack"
and the character curses the object instead of taking a swing at it --
it's much more appropriate to the character! Things are implemented
layers and layers deep.
I only found one bug, which related to the game prompting me repeatedly
to look on top of an object but informing me that the object wasn't
implemented when I did try to look on top of it (despite the fact that
the object WAS implemented, which I could verify when it responded
properly to commands like "open" and "examine"). Compared to all the
things that did work right, and very right, that was pretty minor.
But... there was no walkthrough, and the hint offered to me by the HINT
command, while presented in a very appropriate fashion, did not help me
adequately. I ran up against a puzzle and found that I could not
proceed any further. I think I know what the correct next course of
action was in the game, but I couldn't figure out the syntax, and I
might even have been wrong about what the game wanted. If some sort of
game-ruining bug existed later in the game, and I never found it
because I couldn't continue further in the game, what would be the
merit of the high rating? I really dislike games without walkthroughs.
Is it fair to rate a game higher or lower because of uncertainty about
its author's intentions? People who are unfamiliar with the
situational context may cherish this game. Arguably, for mechanics and
depth, all walkthrough issues aside, it could receive a 9 quite fairly
from me. But if one of the purposes of the game is to make someone
unhappy... I can't support that. Maybe I'll owe the author an apology
later for this, but I'm lowering the score accordingly because I just
don't know what's going on.
SCORE: 6 out of 10.
Brent VanFossen
>An unhelpful suggestion would be: if you feel that strongly about it,
>then don't enter a competition with an anonymity requirement. Yes,
>I know that it's unhelpful. But if you had to choose between two evils:
>a competition with an anonymity requirement, or a competition that's
>tainted by allegations of favouritism, which would you choose?
>
>(Note: I'm not saying that the Comp is tainted today).
I understand your point, but as I said, I'm not convinced the Comp is
broken. I've seen reviewers go out of their way to be fair, even to
some of the present agitators. I think most judges really try to be
fair. The best authors seem to me to score appropriately high, whether
anonymous or not. My top ten usually matches the Comp's top ten,
though not in exactly the same order, and those are the games I'm most
interested in anyway.
But if you're asking me what I'd choose, I'd probably abstain from a
Comp with an anonymity requirement and just release my game at another
time. (This in a comp where the judges are in fact the public at
large. I'd happily submit an anonymous work to a panel for judging,
provided that when the winners are posted, my name would be in the
appropriate place. I think this is how most fairs judge quilting and
cookie baking and wine tasting and whatnot. But I don't want a panel
of judges for the IF Comp, as I think that goes against the spirit of
the Comp itself.)
---
Brent VanFossen
Magnus Olsson
Brent VanFossen <brentva...@NOSPAM.hotmail.com> wrote:
>On 5 Jan 2005 08:49:12 GMT, m...@df.lth.se (Magnus Olsson) wrote:
>
>>An unhelpful suggestion would be: if you feel that strongly about it,
>>then don't enter a competition with an anonymity requirement.
Sorry for my wording above: it came out as unnecessarily harsh.
>>Yes,
>>I know that it's unhelpful. But if you had to choose between two evils:
>>a competition with an anonymity requirement, or a competition that's
>>tainted by allegations of favouritism, which would you choose?
>>
>>(Note: I'm not saying that the Comp is tainted today).
>
>I understand your point, but as I said, I'm not convinced the Comp is
>broken. I've seen reviewers go out of their way to be fair, even to
>some of the present agitators. I think most judges really try to be
>fair.
What can I say, more than that I agree?
The reason that I'd prefer an anonymous Comp is not that the Comp
is broken as it is today, but that it could become broken. "Ceasar's
wife must be above suspicion".
>But if you're asking me what I'd choose, I'd probably abstain from a
>Comp with an anonymity requirement and just release my game at another
>time. (This in a comp where the judges are in fact the public at
>large. I'd happily submit an anonymous work to a panel for judging,
>provided that when the winners are posted, my name would be in the
>appropriate place.
You mean because then there is no public release of the anonymized
game?
>But I don't want a panel
>of judges for the IF Comp, as I think that goes against the spirit of
>the Comp itself.)
Neither do I, and for the same reason.
Having a select panel of judges would of course be a way of solving
one of the "problems" people are talking about, viz. that of excellent
but narrow games getting lower scores than (allegedly) mediocre games
with broad public appeal (or, alternatively formulated, it would
prevent the unwashed masses from voting down my favourites).
But it would be a very different Competition and one which I think would
attract much less interest.
Brent VanFossen
>Sorry for my wording above: it came out as unnecessarily harsh.
I didn't take it as harsh. I understood you as you meant it.
I wrote:
>>But if you're asking me what I'd choose, I'd probably abstain from a
>>Comp with an anonymity requirement and just release my game at another
>>time. (This in a comp where the judges are in fact the public at
>>large. I'd happily submit an anonymous work to a panel for judging,
>>provided that when the winners are posted, my name would be in the
>>appropriate place.
>
>You mean because then there is no public release of the anonymized
>game?
Yes. If my game is out for public consumption, I'd like to have my
name on it. I'm proud of the work I do, both professionally and in the
things I do as hobbies. I mean, for all of us here, this is a hobby.
The Comp prizes are a very nice thank you from the community to the
authors, but the value of the prizes (even the most generous, and they
are quite generous - please don't anyone think I'm knocking the
donors) compared to the time spent preparing a game is not high. We do
this because we enjoy the process and we enjoy sharing the results. I
enjoy getting emails from people I've never met or heard of who thank
me for the game I released. My wife and I receive emails all the time
from people who have discovered our website and thank us for the
content. That makes it worthwhile.
Once released, that version of a game lives forever. The casual player
will not go searching for an updated version. Even if I wanted to set
up a hotmail account with the pseudonym, I wouldn't really want to
maintain that account forever. So any feedback that might have come
from that game is lost, and while I'll never know that, it kind of
rubs me the wrong way.
The good news is we have a community that will play games outside of
the Comp, so I have realistic options. I'd probably just release it
during the year.
>
>>But I don't want a panel
>>of judges for the IF Comp, as I think that goes against the spirit of
>>the Comp itself.)
>
>Neither do I, and for the same reason.
...
>But it would be a very different Competition and one which I think would
>attract much less interest.
We already have panelled competitions in the IF community. They do
attract less interest, and they don't really interest me much either.
Perhaps this is because I can't participate in the voting, or perhaps
for other reasons.
---
Brent VanFossen
Cirk R. Bejnar
> Here, PJ <pete_...@hotmail.com> wrote:
> > And I'm not sure that the low votes balance out the high votes
> > because people are probably more reluctant to give out 1s and 2s
> > than they are 10s
>
> That's not an obvious assumption. I typically give one 10 and
> two-to-four 1s. I see a lot of reviewers saying things like "10 means
> best in the competition", or "10 means it's as good as the best I've
> ever seen", or "I didn't think any game this year was worth a 10."
> I've never seen anyone be that shy about 1s. Some people say "1 means
> unplayable, or not an IF game" but they're still willing to bestow
> that honor on as many entries as deserve it.
>
> --Z
I have also noted the same anecdotal evidence as Zarf but thought it
would be intersting to look at the actual numbers. I only crunched the
current comp since I pulled the raw data off the website by hand. They
reveal that judges gave out a total of 107 tens and 199 ones. Tens
were by far the least common score accounting for only 3.1% of all
scores given. And nearly 40% of judges gave out no tens at all. From
another angle 6 games recieved no ones but 15 no tens and 9 neither a
ten nor a nine. The full stats are copied below with appologies about
the lack of formating.
1 2 3 4 5 6 7 8 9 10
Total 199 279 398 438 549 489 468 314 210 107 3451
Games Without 6 2 0 0 0 2 0 5 9 15
Average Per
Voter 1.144 1.603 2.287 2.517 3.155 2.810 2.690 1.805 1.207 0.615 19.833
Percentage 5.77% 8.08% 11.53% 12.69% 15.91% 14.17% 13.56% 9.10% 6.09% 3.10%
Average Score 5.253549696
1 though 5 1863 53.98%
6 through 10 1588 46.02%
Kevin Venzke
"Cirk R. Bejnar" <eluch...@yahoo.com> wrote in message
> I have also noted the same anecdotal evidence as Zarf but thought it
> would be intersting to look at the actual numbers. I only crunched the
> current comp since I pulled the raw data off the website by hand. They
> reveal that judges gave out a total of 107 tens and 199 ones. Tens
> were by far the least common score accounting for only 3.1% of all
> scores given. And nearly 40% of judges gave out no tens at all. From
> another angle 6 games recieved no ones but 15 no tens and 9 neither a
> ten nor a nine. The full stats are copied below with appologies about
> the lack of formating.
Does this mean you have access to the actual individual ballots? When
you say nearly 40% of judges gave no tens, this makes me think that
you must.
If the individual votes are available, the rankings according to
different methods could be found.
Kevin Venzke
Cirk R. Bejnar
> "Cirk R. Bejnar" <eluch...@yahoo.com> wrote in message
> > I have also noted the same anecdotal evidence as Zarf but thought
it
> > would be intersting to look at the actual numbers. I only crunched
the
> > current comp since I pulled the raw data off the website by hand.
They
> > reveal that judges gave out a total of 107 tens and 199 ones. Tens
> > were by far the least common score accounting for only 3.1% of all
> > scores given. And nearly 40% of judges gave out no tens at all.
From
> > another angle 6 games recieved no ones but 15 no tens and 9 neither
a
> > ten nor a nine. The full stats are copied below with appologies
about
> > the lack of formating.
>
> Does this mean you have access to the actual individual ballots? When
> you say nearly 40% of judges gave no tens, this makes me think that
> you must.
It sounds that way, but if you look at the actual numbers you can see
what I really did. The public results page reveals that 174 people
voted and that they collectively gave out 107 tens. Since judges can
only give out integer votes at least 67 ballots were turned in with no
tens at all. 67 is 38.5% of 174. Since a few people probably gave out
more than one ten I am quite confident that the number of judges who
handed out no tens at all is around 70 judges or 40%.
Cirk R. Bejnar
PJ
Cirk R. Bejnar wrote:
> It sounds that way, but if you look at the actual numbers you can see
> what I really did. The public results page reveals that 174 people
> voted and that they collectively gave out 107 tens. Since judges can
> only give out integer votes at least 67 ballots were turned in with
no
> tens at all. 67 is 38.5% of 174. Since a few people probably gave
out
> more than one ten I am quite confident that the number of judges who
> handed out no tens at all is around 70 judges or 40%.
I'm missing where you see the collective 107 tens on the IF Comp's
public results page. When I navigate to that page, I see the total
number of voters and the average score, the standard deviation, and the
total voters per game. I can't see how you extracted the total number
of 10s and 1s votes, either for the game or the competition as a whole.
If you can explain that, I'd appreciate it, as it would be interesting
to analyze the voting patterns a bit.
PJ
Mike Snyder
> I'm missing where you see the collective 107 tens on the IF Comp's
> public results page. When I navigate to that page, I see the total
> number of voters and the average score, the standard deviation, and the
> total voters per game. I can't see how you extracted the total number
> of 10s and 1s votes, either for the game or the competition as a whole.
> If you can explain that, I'd appreciate it, as it would be interesting
> to analyze the voting patterns a bit.
The histogram chart shows how many of each score were cast. Mouse-hover over
that bar in the grid if your browser supports pop-up labels, and you should
see it.
---- Mike.
PJ
Zhou Fang
> Thanks, Mike. I can see it in IE; couldn't in Mozilla Firefox. PJ
>
Interesting story:
http://www.gadgetopia.com/2004/11/09/FirefoxALTTagsAndTooltips.html
"The ALT tag for images is NOT supposed to produce a little tooltip when
you mouseover an image, according to the HTML spec. This is supposed to
be the job of the TITLE tag.
Firefox has never done this for ALT tags, correctly obeying the spec.
This always confused me, because some images would pop a tooltip and
others wouldn't. Apparently the ones that did also had a TITLE tag,
unbeknowst to me."
In any case, this will make the things appear 'correctly':
Mike Snyder
news:cs1qsg$nfq$1...@wisteria.csv.warwick.ac.uk...
> PJ wrote:
> "The ALT tag for images is NOT supposed to produce a little tooltip when
> you mouseover an image, according to the HTML spec. This is supposed to
> be the job of the TITLE tag.
>
> Firefox has never done this for ALT tags, correctly obeying the spec.
> This always confused me, because some images would pop a tooltip and
> others wouldn't. Apparently the ones that did also had a TITLE tag,
> unbeknowst to me."
>
> In any case, this will make the things appear 'correctly':
Hmmm.... I thought that the "title" tag only works when it's a link -- and
it'll work on text links as well. I wasn't aware that you could even do a
title on a plain image without a link. Interesting.
---- Mike.