Puzzles fairness ratings

8 views
Skip to first unread message

Herbert Mouse

unread,
Feb 10, 2007, 5:23:59 PM2/10/07
to rec.games.int-fiction
I'm wondering if having some sort of record of how players have rated
a games puzzles as far as fairness would be a good idea. The reason
that I can see for using it, if it existed, would be as a way of
guessing how much time to spend at a puzzle before going to the hints
or walkthrough for the game. I've been burned several times by wasting
time on a puzzle whose solution I'd already tried using a different
and obvious verb, and also on puzzles that required an ability to read
the author's mind. I've been burned the other way by puzzles as well,
going to the hints/walkthroughs too early because I suspected these
problems were there for puzzles which had good solutions, and ended up
cheating myself out of the chance to solve them on my own. Puzzle
fairness ratings might be a way to help reduce this problem.

Emily Short

unread,
Feb 10, 2007, 5:59:50 PM2/10/07
to

There is no such formal thing, and I'm not sure how one could be
established that would be at all objective, but for the games it
covers, the IF ratings site ( http://www.carouselchain.com/if/ ) often
contains comments from players about how fair they thought the puzzles
were.

Alternatively, if you're willing to wait a few hours for feedback,
asking on rgif is often a good solution.

Chitterbits

unread,
Feb 10, 2007, 10:23:27 PM2/10/07
to
I find Herbert's question important to IF theory. It is a difficult
question and may garner few responses outside Emily's since it is not
clear how to assess a puzzle as fair -- but that is just what makes it
an interesting question.

Awards are given for best puzzles. Puzzles make or break a work of IF.
Indeed, the solving of puzzles constitutes the interactive business of
interactive fiction -- the absence of which, for better or worse, is a
short story.

So, Herbert, let me give it a shot.

First, there is need for a working description of a puzzle. I suggest
that a puzzle has both mental (problem-solving) and literary (story-
progressing) aspects.

One the one hand, a puzzle has to be perplexing for the player,
producing a sense of confusion. In this regard, a puzzle is baffling;
it has no obvious solution. The room is dark, so flip the light
switch. That is trivial and as such not a puzzle. The room is dark,
but the protagonist can feel a glass tube (neon bulb) and metal rod
(magnet), the combination of which results in a glow. Illumination in
this regard is not so obvious and gives rise to a constructive puzzle
solved by manipulation of resources.

On the other hand, a puzzle is an obstacle to progression of the
narrative. Perhaps this is what Dennis Jerz's means in describing a
puzzle as a management tool to separate movements the the plot. Solve
this puzzle and more of the story is revealed. Solve all the puzzles
to uncover the complete composition, rendering it no longer
interactive, but now a narrative with a determined story path.

OK, now I need a test puzzle. Fetch and swap will do.

Fetch and swap, although it goes by other names, is essentially
exploring to find X, swapping it for Y, swapping Y for Z. For
instance, imagine a story in which the protagonist must ask a
proprietor about an amulet. He gives it to her, which she gives to a
bum, who turns out to be a doctor, who gives her a key in exchange.
The protagonist uses the key and later gets the amulet back to give to
her husband to ward off some antagonist or other.

If the amulet fetch and swap is perplexing and an obstacle to
progression of the narrative, then it constitutes a puzzle. Let's
suppose for sake of discussion that it does constitute a puzzle. Now
to apply Herbert's question. Is the amulet fetch and swap puzzle fair?
More generally, since it could be a key and door rather than an
amulet, does "fetch and swap" constitute a fair puzzle?

Fairness can mean equity, in the sense of conforming to standards. In
this sense we are asking whether "fetch and swap" is impartial, free
from favoritism, or prejudice. For instance, does it favor special
knowledge. Is it the kind of puzzle that requires skill in
cryptography or an understanding of genetics?

Fairness can also be used in the sense of playing fairly; that is,
according to the rules. In this sense we are asking whether "fetch and
swap" is consistent with an immersive counter-factual world -- or does
it require the game to step in ex machina to guide the player?

Drawing upon both senses of fairness, I propose a "first draft" test
for puzzle fairness: "would a reasonable person inevitably find the
necessary resources and be led from the context to make connections
that result in solving the problem?" A reasonable person is not an
expert. Expertise in a particular field breaks the first sense of
fairness. Note that connections need be made from the context, the
scenario, and not by access to an external list of clues (which breaks
the second sense of fairness).

Applying the test, is the fetch and swap puzzle fair?

No. One can accept the logic internal to a horror story that amulets
can protect against evil, but there is nothing to make the leap that
amulets make good barter for arbitrary keys. Thanks for the amulet
lady; here, have a key for your troubles. How should such a puzzle be
solved -- by having the protagonist offer each NPC each item of
inventory in turn in the off-chance that the latter just happens to
swap it for some other bit of merchandise which might or might not be
needed in some other scene?

Even highly regarded stories can have unfair puzzles.

Jim Aikin

unread,
Feb 10, 2007, 11:53:34 PM2/10/07
to

Well, it's unlikely that every puzzle in every game is going to be rated, so
you're probably on your own. Rating a game as a whole based in part on fhe
fairness/unfairness of its puzzles is certainly appropriate, and I have the
impression reviewers do it pretty routinely.

(a) A puzzle is not fair if it relies on the author intentionally describing
something in a vague or confusing way so as to disguise its intended
utility. If the object is something the player character would logically
recognize, it must be named. Even if it's not named, it should be described
clearly!

(b) A puzzle is unfair if it involves hiding something in a location that is
not, based on its description, a plausible hiding-place. (For instance, I
recently tested an unreleased game in which the player begins the game
standing on a platform, and has to 'search platform' to find the first
object. Now, a "platform" is flat, and you can see its entire surface while
standing on it. I felt that requiring the player to search a platform was
unfair.)

(c) Guess-the-verb puzzles are inherently unfair. All of the reasonable
grammar for a puzzle object should be implemented.

(d) A puzzle is not fair if a random number generator causes a given action
to succeed sometimes and not succeed other times.

(e) A puzzle is not fair if it requires that the player have cultural
knowledge that some players will lack. This is a moving target, though. I
have no idea what a TiVO is, so a puzzle that required this knowledge would
be incomprehensible to me.

I'm sure there are other unfair types.

--Jim Aikin


Jeff Nyman

unread,
Feb 11, 2007, 11:17:57 AM2/11/07
to
"Herbert Mouse" <hombr...@yahoo.com> wrote in message
news:1171146239.8...@q2g2000cwa.googlegroups.com...

> I'm wondering if having some sort of record of how players have rated
> a games puzzles as far as fairness would be a good idea.

I'd be worried that it's too subjective, although you could take an
"average," I suppose, of a weighted score. The problem is that if only two
people play the game (or, at least, only two play *and* report on it) and
one rates it "unfair" and the other rates it "fair" -- what does that tell
you? Not much. If twenty people play the game and all of them report on it,
let's say five people report a given puzzle as "totally unfair" and the
other's say it wasn't. Now do you ask how much time they all spent on it?
What their background is? Maybe the ones who solved it "easily" played a
similar game and that made them think of the puzzle solution. Or maybe
they're more used to playing interactive fiction. Or maybe the five people
who didn't get it just aren't good at solving puzzles.

> cheating myself out of the chance to solve them on my own. Puzzle
> fairness ratings might be a way to help reduce this problem.

Maybe, but, honestly, I doubt it. Again, what's the scale for the rating?
Just "fair" and "unfair"? That tells you nothing about *why* and that's
often a key component to how someone rates a puzzle in this situation.

This subject of unfairness and fairness is way too amorphous for too many
pronouncements to be made, in my opinion. People will say what is "fair" and
"unfair" based on their own likes and dislikes, although no doubt there are
some common threads. What makes a puzzle unfair or fair (not just in games)
is how much the puzzle fits into the context that's been built up. (This
sort of matches one aspect of the definition of "unfair", which is
"disproportionate.")

For example, what people call "guess-the-verb" can be unfair, but it sort of
depends, because what verbs you (as the author) thought to implement might
have been perfectly generous. However, one person comes along who doesn't
think along those lines and they might say "Sheesh, I had to guess the verb.
What an unfair puzzle." The act of "guessing-the-verb" is a little vague to
me, because you're *always* guessing what verbs to use in an IF puzzle.

That said, let's say the puzzle would *only* allow you to get past a door if
the you said "use door", rather than "open door" or "unlock door" or even
"push door". In that case, I would say that is unfair simply because a good
(but not airtight) argument could be made that more people, on average, will
tend to think of "open" or "unlock" or "push" (or maybe even "pull") than
"use" in the context of a door. (This can be helped by testing as well,
where you find out how people try to solve a given puzzle.)

I usually find these situations of "unfairness" to come in because the
author simply wasn't thinking rather than trying to craft a puzzle. In other
words, it's not so much that the puzzle itself is unfair; it's how the
author implemented the puzzle that's unfair. (As a puzzle, qua puzzle,
putting a towel over a grate to stop a babel fish from falling down it isn't
unfair, even if the author only allowed "lay towel on grate" as the
appropriate way to do this. In this case, it's the author's implementation:
not the puzzle.) If "guess-the-verb" is used *as the puzzle* itself, I agree
that this can be largely unfair becuase you're not asking the player to use
their reasoning skills, per se, but rather just a brute force approach.

I agree with one of Jim's comments somewhat: the random nature of some
puzzles. That said, things happen in life sometimes based on random (or
fortuitous) circumstances. If you're trying to simulate that, then perhaps
that's not unfair *in the context of your world model* but then I, as the
author, would advise the player of this in some fashion, such as making it
clear that certain things may be the result of a random outcome. (For
example, I can imagine someone writing a game that simulates the quantum
uncertainty as part of the puzzle, perhaps using this to introduce a
probabilistic puzzle.) A way around the "unfairness" of this might be that
the game can't get into an unwinnable state, regardless of which way the
random event fires.

Consider time limits. Are these considered unfair? Personally, I very much
dislike time limit puzzles, at least in text-based IF. I'm playing to
immerse myself in the story and enjoy the experience. If I'm being rushed
into solutions, I don't like that. (In the same way that I wouldn't like if
someone told me I had to read four chapters of a given novel in ten
minutes.) That said, I can't say this is "unfair" just because I don't like
it.

What about having to examine things multiple times? Is this "unfair?" Some
players might not know to re-examine objects or re-look at rooms and yet
doing so might give key clues that help solve some puzzles. This is a form
of information-hiding that can be very effective: allowing a player to
explore a world and incrementally gain more knowledge. The key there is
making sure that the player knows they should re-examine and re-look at
things. Reinforce this behavior. Again, it's an implementation thing on the
author's part. Puzzles that rely on this incremental information are not
necessarily unfair; how the author presented the information (or failed to
do so) is more the issue.

Along with the above point, I think a puzzle can be fair if it requires the
player to figure out what some object is in order to determine its ulitity
(i.e., how it might be used) but I would argue an unfair puzzle is one that
relies on a specific use of this thing where no in-game clues (not hints; I
mean clues in the narrative) have been presented as to what it is or how to
use it.

Domain knowledge vs. domain expertise. This is a tricky one. An engineer
type might write a game that has very "complex" puzzles based on engineering
concepts. Other engineers might get that. Others, who are not engineers, may
not. Is that unfair? It obviously depends on the class of people you are
dealing with. (Just like a novel. Some so-called "hard" science-fiction
requires you to already have some knowledge of physics concepts. That's not
unfair, but it's nice if the author tells you that upfront in some fashion.)
So how much knowledge is your player expected to have? Since you can in no
way know the knowledge base of all your players, you have to decide how to
present the domain to them. That said, authors of conventional fiction deal
with this all the time. I don't think you can say an engineering puzzle in a
game that's based on engineering concepts is unfair any more than you could
blame, say, Peter Hamilton for assuming some background knowledge about
wormholes, black holes or quantum theory.

That said, if the puzzle does require some aspect of engineering, to be
"fair" I would argue that *all* relevant information has to have been
conveyed to the player, regardless of their skill set, to allow them to draw
the appropriate conclusions based on the application of some logical
thinking. (*How much* logical thinking can determine how "hard" -- not
necessarily "unfair" -- the puzzle is.)

So, for me, I'm more interested in the game as a whole, not one or two
individual puzzles. Was the author a good enough writer that they fit the
puzzles into the context of their overall story? By this I mean, the puzzles
in the story are "fair" to me when, assuming only knowledge that I get while
reading/playing, I should be able to solve the puzzles. (Whether I fail to
do so because I didn't draw the appropriate connections doesn't
automatically mean the puzzle(s) is/are unfair.)

- Jeff


Herbert Mouse

unread,
Feb 11, 2007, 4:51:32 PM2/11/07
to
On Feb 10, 4:59 pm, "Emily Short" <emsh...@mindspring.com> wrote:
>
> Alternatively, if you're willing to wait a few hours for feedback,
> asking on rgif is often a good solution.

Asking others their opinions on the puzzles as you encounter them
sounds like a good idea, but I'm not sure if rgif is the ideal place
to do so. Maybe a puzzle-support channel on the MUD might be better?

"Help! I've tried typing unblock passage, enable navigability, cause
point of entry to become ajar, and agape egress. What do I need to
do?"
"Try Open Door"
"Of course! Stupid guess-the-verb puzzle. Thanks for the help!"

Andrew Plotkin

unread,
Feb 11, 2007, 5:04:31 PM2/11/07
to
Here, Herbert Mouse <hombr...@yahoo.com> wrote:
> On Feb 10, 4:59 pm, "Emily Short" <emsh...@mindspring.com> wrote:
> >
> > Alternatively, if you're willing to wait a few hours for feedback,
> > asking on rgif is often a good solution.
>
> Asking others their opinions on the puzzles as you encounter them
> sounds like a good idea, but I'm not sure if rgif is the ideal place
> to do so. Maybe a puzzle-support channel on the MUD might be better?

Both are available. It's a matter of how fast you want feedback and
which group of people you hang out with.

--Z

--
"And Aholibamah bare Jeush, and Jaalam, and Korah: these were the borogoves..."
*
9/11 did change everything. Since 9/12, the biggest threat to American
society has been the American president. I'd call that a change.

greg

unread,
Feb 11, 2007, 7:01:38 PM2/11/07
to
Jeff Nyman wrote:

> For example, what people call "guess-the-verb" can be unfair, but it sort of
> depends, because what verbs you (as the author) thought to implement might
> have been perfectly generous. However, one person comes along who doesn't
> think along those lines

I don't think that's what people mean by a guess-the-verb
puzzle. A true guess-the-verb is when the author intentionally
chooses an obscure verb just to slow the player down. When
it happens accidentally, it might be considered a bug, but
not an unfairly designed puzzle.

> I usually find these situations of "unfairness" to come in because the
> author simply wasn't thinking rather than trying to craft a puzzle.

That's probably true. Rather than "fair" and "unfair", it
might be better to think of ranking in terms of well-designed
vs. poorly-designed.

--
Greg

Autymn D. C.

unread,
Feb 12, 2007, 4:46:06 AM2/12/07
to
Jim Aikin wrote:
> (d) A puzzle is not fair if a random number generator causes a given action
> to succeed sometimes and not succeed other times.

This is pretty anfair and natural to me, if the player should expect
it.

Autymn D. C.

unread,
Feb 12, 2007, 5:48:18 AM2/12/07
to
Jeff Nyman wrote:
> I'd be worried that it's too subjective, although you could take an
> "average," I suppose, of a weighted score. The problem is that if only two
> people play the game (or, at least, only two play *and* report on it) and
> one rates it "unfair" and the other rates it "fair" -- what does that tell
> you? Not much. If twenty people play the game and all of them report on it,
> let's say five people report a given puzzle as "totally unfair" and the
> other's say it wasn't. Now do you ask how much time they all spent on it?
> What their background is? Maybe the ones who solved it "easily" played a
> similar game and that made them think of the puzzle solution. Or maybe
> they're more used to playing interactive fiction. Or maybe the five people
> who didn't get it just aren't good at solving puzzles.

It's a raw score, like any other raw score. However, none ever put a
category for it. (Nor does one for length, except for GameSpot.)
Fairness matches the PC's experience.

> Maybe, but, honestly, I doubt it. Again, what's the scale for the rating?
> Just "fair" and "unfair"? That tells you nothing about *why* and that's
> often a key component to how someone rates a puzzle in this situation.

The labels come after the score. How about a scale between 0 and 4:

0: nofair 1: unfair 2: fair; 3: anfair 4: alfair

> That said, let's say the puzzle would *only* allow you to get past a door if
> the you said "use door", rather than "open door" or "unlock door" or even
> "push door". In that case, I would say that is unfair simply because a good
> (but not airtight) argument could be made that more people, on average, will
> tend to think of "open" or "unlock" or "push" (or maybe even "pull") than
> "use" in the context of a door. (This can be helped by testing as well,
> where you find out how people try to solve a given puzzle.)

It's unfair because the author would need to reprogram the game to
make it work harder and wronger for all. Verbs are already as
specific as they need be.

>use door
You'll have to tell me how to do that.

>push door
You push the door.

>n
You walk into the door.

> author implemented the puzzle that's unfair. (As a puzzle, qua puzzle,
> putting a towel over a grate to stop a babel fish from falling down it isn't
> unfair, even if the author only allowed "lay towel on grate" as the
> appropriate way to do this. In this case, it's the author's implementation:
> not the puzzle.) If "guess-the-verb" is used *as the puzzle* itself, I agree

How could the author only allow one odd verb unless it was unfairly
programmed? (Lay is a huponum of Put.) If one forgets the libraries,
the player must throw out all interpreters but the author.

> conveyed to the player, regardless of their skill set, to allow them to draw

The player has no their and is not a them.

-Aut

Jeff Nyman

unread,
Feb 12, 2007, 8:05:28 AM2/12/07
to
On Feb 11, 6:01 pm, greg <g...@cosc.canterbury.ac.nz> wrote:
> Jeff Nyman wrote:
> > For example, what people call "guess-the-verb" can be unfair, but it sort of
> > depends, because what verbs you (as the author) thought to implement might
> > have been perfectly generous. However, one person comes along who doesn't
> > think along those lines
>
> I don't think that's what people mean by a guess-the-verb
> puzzle. A true guess-the-verb is when the author intentionally
> chooses an obscure verb just to slow the player down. When
> it happens accidentally, it might be considered a bug, but
> not an unfairly designed puzzle.

I agree. That's why I had said that to me this is only unfair if the
author tries to use "guess-the-verb" *as a puzzle* in itself. In other
words, a deliberate choice by the author.

- Jeff

Jeff Nyman

unread,
Feb 12, 2007, 8:09:15 AM2/12/07
to
On Feb 12, 4:48 am, "Autymn D. C." <lysde...@sbcglobal.net> wrote:
>
> It's a raw score, like any other raw score. However, none ever put a
> category for it. (Nor does one for length, except for GameSpot.)
> Fairness matches the PC's experience.

It still sounds unworkable to me, at least for being truly indicative
of anything. I have no doubt this rating scale (score) could be used.
I'm just not sure how much it would really tell anyone.

> > author implemented the puzzle that's unfair. (As a puzzle, qua puzzle,
> > putting a towel over a grate to stop a babel fish from falling down it isn't
> > unfair, even if the author only allowed "lay towel on grate" as the
> > appropriate way to do this. In this case, it's the author's implementation:
> > not the puzzle.) If "guess-the-verb" is used *as the puzzle* itself, I agree
>
> How could the author only allow one odd verb unless it was unfairly
> programmed? (Lay is a huponum of Put.) If one forgets the libraries,
> the player must throw out all interpreters but the author.

I agree. That's why I said if this sort of thing was used *as the
puzzle* itself. In other words, the author would have had to take some
action to actively make a puzzle out of the action by only allowing a
given verb to be used in that particular situation.

- Jeff

Adam Thornton

unread,
Feb 12, 2007, 6:49:18 PM2/12/07
to
In article <1171277298.7...@a75g2000cwd.googlegroups.com>,
Autymn D. C. <lysd...@sbcglobal.net> wrote:

>Jeff Nyman wrote:
>> conveyed to the player, regardless of their skill set, to allow them to draw
>The player has no their and is not a them.

You are my hero, or heroine, or possibly heroin. That's a mighty sexy
grasp of number you've got there, anyway.

Adam

Reiko

unread,
Feb 13, 2007, 5:21:13 PM2/13/07
to
On Feb 12, 7:09 am, "Jeff Nyman" <jeffny...@gmail.com> wrote:
> I agree. That's why I said if this sort of thing was used *as the
> puzzle* itself. In other words, the author would have had to take some
> action to actively make a puzzle out of the action by only allowing a
> given verb to be used in that particular situation.

By that logic, wordplay games are inherently unfair. I'm thinking of
games like Ad Verbum or Goose, Egg, Badger, where to accomplish many
things, only one specific command is allowed due to the nature of the
wordplay. But these games, while I found somewhat difficult to come up
with the correct command sometimes, were very fun and very consistent
in their puzzles. I think perhaps a qualification for a true "guess
the verb" puzzle is if the author required a particular command for no
good reason. In the wordplay games, there are ways to figure out which
words are acceptable and which aren't. But guessing the verb involves
trying to figure out without any information which one of several
possible commands is the one the author enabled. And once you figure
it out, there's still no reason why it should be that particular
command.

~Reiko

Jeff Nyman

unread,
Feb 13, 2007, 6:02:11 PM2/13/07
to
"Reiko" <tel...@gmail.com> wrote in message
news:1171405273.2...@v33g2000cwv.googlegroups.com...

> in their puzzles. I think perhaps a qualification for a true "guess
> the verb" puzzle is if the author required a particular command for no
> good reason.

A good point and a good distinction. I agree.

So this goes back to what I said here maybe: "Was the author a good enough

writer that they fit the
puzzles into the context of their overall story?"

If those puzzles that you mention fit the context of the story, and that
aspect of the story is made clear to the player (either upfront or through
experimentation), I would consider this fair, at least in my perspective.
(Note, however, that if somebody thinks *any* form of guess-the-verb is
unfair, then they would think these games unfair. Ultimately that's what I
meant about a simple rating of "fair" or "unfair" without consideration of
why the person feels that way.)

- Jeff


Mark Tilford

unread,
Feb 26, 2007, 8:17:00 AM2/26/07
to
On 2007-02-13, Jeff Nyman <jeff...@gmail.com> wrote:
>
>
> "Reiko" <tel...@gmail.com> wrote in message
> news:1171405273.2...@v33g2000cwv.googlegroups.com...
>> in their puzzles. I think perhaps a qualification for a true "guess
>> the verb" puzzle is if the author required a particular command for no
>> good reason.
>
> A good point and a good distinction. I agree.
>
> So this goes back to what I said here maybe: "Was the author a good enough
> writer that they fit the
> puzzles into the context of their overall story?"
>

I'd say gameplay instead of story. For example, requiring precise
timing or movement within a first-person shooter game is perfectly fair;
requiring precise timing or movement within a Sokoban type game is not.

Reply all
Reply to author
Forward
0 new messages