Giving players what they like?
Julian Togelius
Michael Mateas, Mark Riedl, and Kate Compton) and there were several
interesting topics raised there which probably should be brought to
the attention of this group so we can get more people's views on them.
One of them is the following: my work, as well as the work of e.g.
Erin and Ken, somehow models what the player enjoys and gives the
player more of it. That is, it implicitly assumes that you should give
players more of what they like. Mark pointed out that you should
sometimes give players content they don't like as well, otherwise they
would get bored. My take on this is that it simply means that your
model of the player's entertainment is somehow faulty or too shallow;
with a good model of player entertainment, we should at any point
choose the content that optimizes expected long-term enjoyment.
What's your take on this? Mark, would you care to elaborate on your
view (which I probably misrepresented)?
Julian
--
Julian Togelius
Assistant Professor
IT University of Copenhagen
Rued Langgaards Vej 7
2300 Copenhagen S
Denmark
jul...@togelius.com
http://julian.togelius.com
+46-705-192088
Lucas, Simon M
Is like / dislike is a bit one-dimensional?
An alternative is to take a multi-objective
view that factors out various dimensions
such as familiar ... unfamiliar, easy ... hard,
consistent ... variable and so on.
Simon
Kenneth Stanley
don't like from time to time is interesting, but it may not really be
an issue that needs to be resolved in practice, since the hard part is
to give players what they actually do like; it's easy to produce lots
of garbage that people don't like. Any algorithm that aims to produce
novelty (i.e. not just give players more of the same) by necessity
must take some risks to explore the space of possibilities. With such
risk an occasional undesirable outcome is inevitable. It is a
property of search in general that while a conservative search can
potentially minimize the amount of unintended surprises, it also
reduces the probability of finding anything interesting. Thus there
will always be such an exploitation/exploration trade-off. Given
these slippery conditions, my feeling is that the risk of producing
too much undesirable content is much greater than the risk of creating
too little.
ken
Mark Riedl
Julian, as you point out, if one has a well-formed model of global
experience, then one just searches, using whatever technique is most
appropriate, for the solution that maximizes long-term payoff. By
"well-formed", I use Newell's definition of "a solver’s ability to
identify a specific answer." Dramatic storytelling, in general, is
still an ill-defined domain. The short answer is: we do not have a
model that we can employ.
Consider a class of agents called drama managers. The goal of a drama
manager is to manipulate a virtual world so that our experience takes
on dramatic qualities. The user/player has agency to act in the world
to achieve his or her goals. The agent must be reactive in real-time
because we also do not have a perfect model of the user. What we know
of humans is that they are bounded rational agents. That is, we try
to maximize payoff. For example, when I travel by plane, I enact a
policy that minimizes negative situations. I get to the airport early,
I make sure my bags are taken care of, etc.
Drama is a complex phenomenon. But one thing that most narratologists
and narrative psychologists agree upon is that drama occurs when there
are "obstacles" that are eventually overcome. For simplicity, let's
consider an "obstacles" as any force that impedes the achievement of a
goal (or violates an achieved goal state that is to be maintained,
like health or happiness). Anxiety mounts as obstacles mount.
Catharsis occurs when all obstacles are overcome through means other
than happenstance.
Thus, if a drama management agent were to manipulate the world so that
I have a dramatic airplane travel experience, it would make
moment-to-moment decisions that would make me late to the airport,
increase the probability that my bags are lost, etc. Why do we need
the drama manager? Because most people will attempt to minimize
obstacles. But note that the drama management agent is not purely
adversarial: it cannot obstruct me such that I have no chance to
complete my travel goal. It must make be believe that I cannot but
then also facilitate my eventual successful goal accomplishment.
Narratives are temporal in nature. At any given moment it may /appear/
that a system is /locally/ giving the user content that they do not
like, but /globally/ will like. However, the user will not be aware
that they like the global experience until after-the-fact. In situ,
they will -- and should -- feel frustration, anxiety, and tension. If
this interesting, I invite you to read a paper that my colleagues and
I wrote exploring some of these issues in the context of a
hypothetical system that plays a "fun" game of chess:
http://www.cc.gatech.edu/~riedl/pubs/digra09.pdf.
Mark Riedl
---
Assistant Professor
College of Computing
Georgia Institute of Technology
---
ri...@cc.gatech.edu
http://www.cc.gatech.edu/~riedl/
Andrew Doull
> One of them is the following: my work, as well as the work of e.g.
> Erin and Ken, somehow models what the player enjoys and gives the
> player more of it. That is, it implicitly assumes that you should give
> players more of what they like. Mark pointed out that you should
> sometimes give players content they don't like as well, otherwise they
> would get bored. My take on this is that it simply means that your
> model of the player's entertainment is somehow faulty or too shallow;
> with a good model of player entertainment, we should at any point
> choose the content that optimizes expected long-term enjoyment.
which doesn't appear a feature that players especially like. However
it is important for the game design triangle of roguelike play. (see
http://www.escapistmagazine.com/articles/view/issues/issue_209/6235-Infinite-Caves-Infinite-Stories)
I'm not sure whether that qualifies as a mechanic that optimizes
expected long term play, given your definition.
Andrew
Jim Whitehead
levels (or other content) is a cardinal sin.
It would be great to develop procedural algorithms that had the ability
to generate levels that had some kind of unique twist to them.
For example, how could one develop an algorithm that could create
something as unique and interesting as the puzzle challenge in the first
level of Braid? It involves standard collection actions, but (warning,
spoiler) viewing the puzzle as both a puzzle, as well as having the
image in the puzzle act as a platform, seems well beyond the
capabilities of any generation algorithm at present.
Perhaps it would be useful to document such "generation challenges" as a
way of helping articulate research directions.
- Jim
E. Hastings
since you don't know what players prefer. As the game progresses
though, and the players have some favorite content items, seems like
it wouldn't hurt to do a limited amount of novelty searching to throw
out new possibilities. Novelty search meaning, not randomization, but
specifically generating new content with distant genotypes from that
of content players have shown to favor.
Magy Seif El-Nasr
I just joined this group. Great discussions. sorry for chiming in the
middle of conversation here,
I may not have the right context.
This is an interesting discussion. I think we should differentiate
between long term enjoyment and moment
to moment reward systems. I do agree with Mark that there should be
ups and downs in the system
otherwise you won't get a long term enjoyment. That is just how human
psychology work.
The reward would seem of so much value when it follows a high struggle
period to get it.
If you look at theories of emotion like Soloman's work and others you
will see why this may be the case.
In my game design class, we were discussing the application of
skinner's reward schedules to
level design. There is an article discussing this which is quite
interesting,
it may be of interest to look at it as well.
I can provide some references there if this is of interest.
Julian Togelius
Yes, there is definitely a need to differentiate between short-term
and long-term enjoyment. My point about the model was that if it
doesn't take this difference into account, it isn't complete, but if
it does take it account, you can just use the model and optimize
long-term enjoyment. However, while short-time enjoyment can often
gauged from questionnaires and player interaction, learning long-term
enjoyment purely from data might be a tall task. So there's probably
need for considerable hand-crafting here. Either that, or using data
from hundreds of thousands of players playing entire games - which can
be hard to get hold of.
Reward schedules are definitely interesting. Something that's always
fascinated me is Skinner's random reward scheme that induces
"superstition" in pigeons (and, it is claimed, in humans). Those
results go against so many naive assumptions about how learning
work... I suppose it's quite applicable to game design as well, but
I'm not sure about exactly how.
Do you have any links to papers?
Julian
2009/11/13 Magy Seif El-Nasr <ma...@sfu.ca>: