Agile Game Development
Phlip
From a brief study of the game industry, I might list a few impediments to
"agile" software development. Whereas this essay will define "agile" as
"good", we should not make the mistake to think that any agile "best
practice" must slavishly mirror into game development. Achieving the results
shown by "agile" development in other industries - reduced odds of failure,
accurate schedule estimates, and absurdly low defect rates - will require
adjusting the practices to fit game programming topologies.
Planning Game, Early Delivery
With a few exceptions, the game industry could not sustain Early Delivery.
Games are entertainment, and changing levels every few months would destroy
the tenuous addictions that designers so carefully foster. The game industry
has more in common with the motion picture industry than the other sectors
of programming. A game starts as a pitch, very similar to a screenplay
pitch, and ends with a rollout. If the game succeeds it funds its studio's
failures. Like a screenplay, a big game design, up front, must pitch a known
genre to a very clearly defined demographic. That helps a studio's successes
exceed its failures. This practice excuses each game's development from much
internal risk management. A successful game generates enough cheddar to fund
rewriting the next game's engine. (A studio simultaneously shipping two
games generates trade literature headlines.)
Frequent, regular internal releases have many more benefits than merely
preparing for frequent delivery. Frequent releases depend on a steady stream
of feature requests, where each feature can cover any subset of database,
middleware, engine, logic, art (assets), and front-end. Sorting these
requests in order of business value has a powerful effect on design, because
the high-priority items get tested, reviewed, integrated, and reinforced the
most during the remaining development. However...
Continuous Integration, Daily Deployment, Frequent Releases
Game development works with very large teams, split by function into groups
with similar abilities. The business world, working on projects requiring a
narrower range of abilities, has learned to avoid this tactic, and to
colocate very small teams of diverse abilities. Splitting teams by function
requires goal donors (designers and technical leaders) to split feature
requests by function, not by business value. So these forces, coupled with
the incredibly high volume of data born in each game title, powerfully
dis-integrates production, making continuous integration much more difficult
than it could be. A game's source code and data file can take several
computer-hours to rebuild.
So these forces, in turn, inspire game developers to re-invent a practice
that business programmers are learning to avoid - rampant code branching. A
game team might have a mini-branch for the sound guys, a mini-branch for the
AI guys, etc. The cost of re-integrating each teams' new outputs, to finally
create the integrated feature the designer requested, delays the most
important feedback. Designers should review the effect of each request, on
all game aspects, as soon as possible to permit that review to accurately
and usefully drive the new requests.
Pair Programming, Shared Code Ownership
The game industry is widely reputed to program game engines using only the
worst practices of both Waterfall and Code-and-Fix. Designers request engine
abilities far in advance of reviewing the levels that exploit them.
Engineers perform Big Design Up Front to the best of their abilities. When
any Waterfall system runs off the end of its planned design, developers are
forced to switch to Code-and-Fix. Those of us experienced with the long-term
results of Waterfall have all seen projects with many clever designs, many
over-designed parts, and many parts riddled with spaghetti, cross-patches,
and horrid cruft.
I lack the tact that other agilistas exercise daily when I admit that game
development attracts and retains programmers with ... special skills. Pride
of code ownership must preceed the all-night coding sessions that keep such
code afloat. Game developers learn their systems very well, and the industry
relies on their experience shipping games to request them to ship again,
with a slightly better engine each time. Within these mega-iterations,
source code loses many opportunities for collaboration to force in new
flexibilities. "We can't do X until Q completes Y" is a common impediment.
Test-Driven Development, Customer Tests
Over time, game projects grow a huge "decompression debt" of easily-fixed
bugs. Shipping a game requires an early feature-freeze and then a very long
party to stomp insects. However, within the many layers that make up a game
are many latent tests, and an extraordinarily huge number of side-effects
that could easily become "characterization tests". But only pure TDD from
scratch can beat legacy test retrofits. Pure TDD will generate full-feature
test fixtures capable of testing arbitrarily complex aspects of play that
entry-level testers could only dream of.
And all games have a scripting layer of some kind. The Lua language is the
leading candidate for this role. And the great irony of game development is
this: Most business projects don't have a scripting layer, and have Onsite
Customers who should not write source code. Agile development relies on
Customer Tests, written by customers, to constrain each feature they
request. Game projects, by contrast, require designers to author Lua scripts
to add the hooks and triggers to each level. So game projects have Onsite
Customers who can write scripting code, and game projects also have the
scripting layer for them to write with. To close this gap, game projects
need only require designers to author the tests for each high-level feature
they request. And game projects must tune their scripting layers to make
these techniques easy and robust.
A Prescription
Start a game small, not super-big. Maybe start with a designer, two engine
guys, and two asset guys. They finish a level, incrementally adding
challenges, and maintaining a zero-tolerance for bugs. They measure their
velocity, and their gold-owner adds resources until the velocity levels off.
The game remains playable and deliverable at all times. It might not have
all its levels and its resolution cinematics, but it will have zero bugs and
all playability features for the levels it grows.
This recommendation turns traditional development side-ways. The storyboard
becomes the schedule, not the bottleneck, and levels not yet completed are
the ones most easy to change. Shortening the feedback between feature
requests and results, by any means necessary, will lead to agility for the
game industry.
--
Phlip
http://industrialxp.org/community/bin/view/Main/TestFirstUserInterfaces
Paul Sinnett
> From a brief study of the game industry, I might list a few
> impediments to "agile" software development. Whereas this essay
> will define "agile" as "good", we should not make the mistake to
> think that any agile "best practice" must slavishly mirror into
> game development.
I'd like to point out a couple of areas where your brief study is
inconsistent with my own experience, but first...
> Achieving the results shown by "agile" development in other
> industries - reduced odds of failure, accurate schedule estimates,
> and absurdly low defect rates - will require adjusting the
> practices to fit game programming topologies.
You're assuming that these are big problems in game development. In
my experience, other problems dwarf these factors.
Failure:
Few games that pass the green light stage get canceled before
completion. In my experience I have only worked on 1 game in 10 that
got a green light and was later canceled. Getting a green light is
usually quite tough so this weeds out most poor game designs before
they get too far. The most common cause for canceled projects (in my
admittedly limited experience of such things) are for marketing and
budget reasons unconnected with the game's development progress. I
suspect this is outside the scope of things agile methods are likely
to cover. That's not to say that failures due to technical problems
don't occur, but I've not heard of any. As far as I'm aware, all of
the recent studios that closed in the UK did so in spite of
delivering their games, not because they couldn't.
Accurate Schedule Estimates:
Contrary to popular belief, most game development teams are fairly
good at estimating project schedules. The problem is usually that
these accurate estimates aren't used. Instead most publishers
substitute ridiculous estimates based around delivery dates dictated
for (usually very questionable) marketing reasons. For example, in a
project that I worked on, our technical director came up with a spot
on estimate for development time based on years of development
experience. This was then cut in half by the publisher and later
extended month by month until we delivered pretty much bang on our
estimate date - in other words late! This is how most experienced
teams deliver accurate estimates and yet paradoxically miss their
deadlines. Again, it's difficult to imagine where agility would help
here.
Low Defect Rates:
I've played a lot of console games over the years, and in that time
I've only ever found one crash bug in a published game. That was
after playing continuously for many hours and then starting over. On
the other hand, I've lost count of the amount of times I've lost
work and had to reboot my PC while using perfectly standard "mature"
development tools. Now it's true that PC games also suffer from this
instability, but these games are mostly written and tested by the
exactly the same people who write the console games. That makes me
think this is a PC / OS problem rather than defects inherent in PC
game code or the game development process.
A defect in game development is a vague concept. A bug can mean
anything from a wrong shade of green on a texture to an inability to
complete the game. The priority of bugs is usually somewhat
arbitrary: a single incorrect spelling of some on screen text can
render the game unshippable; while extremely annoying consequences
of bizarre design choices are often waived as "unimportant" as the
deadline closes. In fact, many of the "bugs" that ship with a game
are emergent in nature. They don't arise from a mistake in the
coding, but from an unintended consequence of the specification of
rules of the game.
Okay, back to your essay:
> (A studio simultaneously shipping two games generates trade
> literature headlines.)
I'm not sure where you got this impression. It does happen. (And
probably more than you realise precisely because it does not
generate trade literature headlines when it does?)
> Game development works with very large teams, split by function
> into groups with similar abilities.
True. And so many game development teams are anything but agile by
definition. However, there are still many smaller game developments
that could benefit.
> So these forces, in turn, inspire game developers to re-invent a
> practice that business programmers are learning to avoid - rampant
> code branching.
I've never seen that myself. Almost every project I've worked on, or
even heard about used simple source control with everybody working
on the same branch. Although I'm not sure what branching has to do
with agility. Is there some pet peeve you have with this practice?
> Pride of code ownership must preceed the all-night coding sessions
> that keep such code afloat. Game developers learn their systems
> very well, and the industry relies on their experience shipping
> games to request them to ship again, with a slightly better engine
> each time.
In my experience, code ownership is imposed on programmers. In that
sense it's more like code responsibility than ownership. Also, in my
experience, programmers frequently collaborate (in secret if
necessary) to get the job done. It's not pride on the part of
programmers, it's a manager's need for a name to blame.
> Shipping a game requires an early feature-freeze and then a very
> long party to stomp insects.
This is usually the policy, but in practice most games are adding
features right up to the burn of the final master. It's particularly
easy to introduce emergent bugs at this point.
> Shortening the feedback between feature requests and results, by
> any means necessary, will lead to agility for the game industry.
But does the game industry need agility? And what for?
In my experience, speeding up the feedback loop hasn't improved much.
If anything, it led to experimentation without purpose. At
best it has had no effect. I once made every predefined gameplay
constant editable from a single text file. The designer took that
version away to "tweak" the design. When he brought it back the
following week he had changed one parameter by 0.01 of a second.
Nobody noticed.
I think game development could benefit from many of the techniques
of agile methods, but I don't think agility in itself offers much.
Laurent Bossavit
> With a few exceptions, the game industry could not sustain Early Delivery.
It worked for Id Software's Quake 3. Perhaps that's one of the
exceptions...
You could always release (early and often) to a small, select, trusted
population of early players.
Laurent
Phlip
One myth of XP is it requires early and frequent delivery.
You get all the benefits if you release regularily to a small crowd of
users.
One goal for my essay was skipping over some of these myths, before they
caused the usual diversionist threads.
--
Phlip
http://industrialxp.org/community/bin/view/Main/TestFirstUserInterfaces
Stede Troisi
complicated form of programming there is! Period. Most of the games out
there today are at the cutting-edge and do well compared to the lame crap
business developers write.
Maybe we should be learning something from them? Also, are you coming from
the philosophy that XP and agile development is the "right" way to program?
If so, I think game developers show it is at least not the only way.
Stede
"Phlip" <phli...@yahoo.com> wrote in message
news:mu3md.21811$C74....@newssvr31.news.prodigy.com...
WTH
before with:
> You need to understand Phlip that Game development is by far the most
> complicated form of programming there is! Period. Most of the games
> out there today are at the cutting-edge and do well compared to the
> lame crap business developers write.
>
> Maybe we should be learning something from them? Also, are you coming
> from the philosophy that XP and agile development is the "right" way
> to program? If so, I think game developers show it is at least not
> the only way.
>
> Stede
Sorry, but there's plenty of non-game development software that is just as
complex. Maybe you meant to say 'pressure driven and competitive' instead
of 'by far the most complicated form of programming there is.' It is
certainly equal to other development tasks, but it is the
time/money/resource constraints that make development harder than most other
forms of development, not the complexity of the code itself.
WTH
Phlip
> Phlip wrote:
> > From a brief study of the game industry, I might list a few
> > impediments to "agile" software development. Whereas this essay
> > will define "agile" as "good", we should not make the mistake to
> > think that any agile "best practice" must slavishly mirror into
> > game development.
>
> I'd like to point out a couple of areas where your brief study is
> inconsistent with my own experience, but first...
I listed the three major reasons why "agile" development has been reputed to
be "good". Having never seen a game project fail, up close (for various
definitions of "fail"), I will personally rely on "good". Maybe "avoiding
all-nighters" should be the primary goal.
If I had seen such a project fail, I suppose I could find a way to recast
each failure point back onto my list of agile results just as easily as I
could recast them away from them. It's all "good".
> > Achieving the results shown by "agile" development in other
> > industries - reduced odds of failure, accurate schedule estimates,
> > and absurdly low defect rates - will require adjusting the
> > practices to fit game programming topologies.
>
> You're assuming that these are big problems in game development. In
> my experience, other problems dwarf these factors.
What top three problems would you list? Be brutal - many shops grow
accustomed to some situations, just as ones eyes grow accustomed to the
dark.
> > (A studio simultaneously shipping two games generates trade
> > literature headlines.)
>
> I'm not sure where you got this impression. It does happen. (And
> probably more than you realise precisely because it does not
> generate trade literature headlines when it does?)
My untenable premise is a shop builds their engine, builds a game, and
starts a re-write on the engine. I'm not saying that's rampant (or
especially bad). I suspect that truly flexible and bug-free code would allow
such an engine to obey the "Software Product Lines" style.
> > Game development works with very large teams, split by function
> > into groups with similar abilities.
>
> True. And so many game development teams are anything but agile by
> definition. However, there are still many smaller game developments
> that could benefit.
I don't understand. Are you saying the teams _must_ split, and that therefor
process improvement must be ... "less" agile?
> > So these forces, in turn, inspire game developers to re-invent a
> > practice that business programmers are learning to avoid - rampant
> > code branching.
>
> I've never seen that myself. Almost every project I've worked on, or
> even heard about used simple source control with everybody working
> on the same branch. Although I'm not sure what branching has to do
> with agility. Is there some pet peeve you have with this practice?
Why yes. I have seen code forking churn non-game projects into complete mud.
Proper respects for observing unbranched game projects, and I should not
behave as if I had determined it were the industry average.
> > Pride of code ownership must preceed the all-night coding sessions
> > that keep such code afloat. Game developers learn their systems
> > very well, and the industry relies on their experience shipping
> > games to request them to ship again, with a slightly better engine
> > each time.
>
> In my experience, code ownership is imposed on programmers. In that
> sense it's more like code responsibility than ownership. Also, in my
> experience, programmers frequently collaborate (in secret if
> necessary) to get the job done. It's not pride on the part of
> programmers, it's a manager's need for a name to blame.
Ah, I forgot to blame the managers, too. Point.
> > Shipping a game requires an early feature-freeze and then a very
> > long party to stomp insects.
>
> This is usually the policy, but in practice most games are adding
> features right up to the burn of the final master. It's particularly
> easy to introduce emergent bugs at this point.
Per the "crunch mode" thread, it seems I tried to describe "crunch mode"
here. Feature freezes are not bad (it's just a game, not air traffic
control).
But if scheduling were working, were are the "crunch modes" coming from? My
non-game crunch modes came from not maintaining a zero-tolerance policy
against even minor bugs, early in development.
> > Shortening the feedback between feature requests and results, by
> > any means necessary, will lead to agility for the game industry.
>
> But does the game industry need agility? And what for?
Let's assume they currently have "bad", and need "good". Neither of us have
disagreed, so far, on those fundamental issues. However,
> In my experience, speeding up the feedback loop hasn't improved much.
> If anything, it led to experimentation without purpose. At
> best it has had no effect. I once made every predefined gameplay
> constant editable from a single text file. The designer took that
> version away to "tweak" the design. When he brought it back the
> following week he had changed one parameter by 0.01 of a second.
> Nobody noticed.
That's not feedback. You didn't get early feedback on whether your tool was
needed.
The recommendations, again:
- start small, colocate a team, and pair
- mix the team; design, art, and math
- seamless integration, no code forking
- leverage the script layer for design-tests
- one level at a time, one trigger at a time
- TDD, refactoring, shared code, daily deploment, etc.
- absolute zero tolerances for anything remotely resembling a bug
Within that cycle, I suspect that a design could tweak a parameter via
several different scenarios, including user stories, story tests, pairing
with a developer, tweaking the scripts, etc.
If you complain about any one item in the list, then you play CTips's game
of describing some practice going out of balance, while ignoring the
opposing forces from the other practices.
> I think game development could benefit from many of the techniques
> of agile methods, but I don't think agility in itself offers much.
Maybe I didn't say, "we should not make the mistake to think that any agile
'best
practice' must slavishly mirror into game development," in the first
paragraph.
--
Phlip
http://industrialxp.org/community/bin/view/Main/TestFirstUserInterfaces
sted...@yahoo.com
of its
own. First, almost anyone can write a payroll system (think C3) but
inventing cutting-edge game graphics and sounds that have never been
done
before is tough. Add all the other factors that go into game design and
I
think it would be almost impossible to do XP at companies like EA,
Activision and Blizzard.
Then again, your math is probably better than mine and it probably is
easier
for you. It would be interesting to see a AAA title done with XP
though.
Working 90 hour weeks minimum and enough teenagers to make 1 game a day
for
the next 500 years I don't think it will happen any time soon.
Chris Dollin
> You need to understand Phlip that Game development is by far the most
> complicated form of programming there is! Period. Most of the games out
> there today are at the cutting-edge and do well compared to the lame crap
> business developers write.
You really *are* the offensive twit you appeared to be on the xp mailing
list. At least *consistency* is some kind of virtue.
--
Chris "electric hedgehog" Dollin
John Roth
> Paul Sinnett wrote:
[Paul's message doesn't appear to have made it to my server...]
>
>> Phlip wrote:
>
>
> My untenable premise is a shop builds their engine, builds a game, and
> starts a re-write on the engine. I'm not saying that's rampant (or
> especially bad). I suspect that truly flexible and bug-free code would
> allow
> such an engine to obey the "Software Product Lines" style.
I'm not sure where you're going with this one, but several examples
of engines that underlie multiple games come to mind. LucasArts'
SCUMM engine, for example. The Z-machine that not only ran
all the early Zork games, but was reimplemented, ported to just
about every platform in existance and is ubiquitous in the
interactive fiction camp. There are numerous other examples.
I suspect that the norm is to use an existing engine to crank out
mid-level games quickly. It's only the real high end games where
the intention is to push the state of the art where you really want
to do wholesale revisions to existing engines.
>> > Game development works with very large teams, split by function
>> > into groups with similar abilities.
>>
>> True. And so many game development teams are anything but agile by
>> definition. However, there are still many smaller game developments
>> that could benefit.
>
> I don't understand. Are you saying the teams _must_ split, and that
> therefor
> process improvement must be ... "less" agile?
I don't necessarilly think of this as bad. While I've heard of
XP teams that are larger than about a dozen people, the
"conventional wisdom" seems to put an upper limit on
an effective team that is somewhere around that point.
Beyond that, you need a strategy to coordinate multiple
teams, or operate in some other fashion.
>> > Shipping a game requires an early feature-freeze and then a very
>> > long party to stomp insects.
>>
>> This is usually the policy, but in practice most games are adding
>> features right up to the burn of the final master. It's particularly
>> easy to introduce emergent bugs at this point.
>
> Per the "crunch mode" thread, it seems I tried to describe "crunch mode"
> here. Feature freezes are not bad (it's just a game, not air traffic
> control).
>
> But if scheduling were working, were are the "crunch modes" coming from?
> My
> non-game crunch modes came from not maintaining a zero-tolerance policy
> against even minor bugs, early in development.
IMO, crunch mode usually comes from not having a production
quality deployable at frequent intervals throughout the development
process.
For me, the question simply comes down to:
"Would you rather be able to ship a bug-free product that's missing
a couple of levels that you had on the storyboards in time for Christmas,
or would you rather ship the complete game your designers had in mind
and miss the Christmas shopping window?"
This is simply one of the basic XP questions, with the blanks
filled in from the game development lexicon.
However, the question only makes sense if the process is
capable of producing that production quality deployable
every couple of weeks, with a new level or two, fixes
for previous levels from the play testers, game engine
enhancements and so forth.
>> In my experience, speeding up the feedback loop hasn't improved much.
>> If anything, it led to experimentation without purpose. At
>> best it has had no effect. I once made every predefined gameplay
>> constant editable from a single text file. The designer took that
>> version away to "tweak" the design. When he brought it back the
>> following week he had changed one parameter by 0.01 of a second.
>> Nobody noticed.
The basic issue is: which feedback loop needs tweaking? The
only way you tell that is to find the bottlenecks and work on
them. In this example, it sounds like gameplay engine change
requests weren't a bottleneck.
On the other hand, they may well have been. I can very
easily envision a scenario where the engine people were
being burried under a load of change requests for constants
that didn't actually do anything. Externalizing that would
have removed the change request queue.
John Roth
>
> --
> Phlip
> http://industrialxp.org/community/bin/view/Main/TestFirstUserInterfaces
>
>
Tom Plunket
> You need to understand Phlip that Game development is by far the most
> complicated form of programming there is! Period. Most of the games out
> there today are at the cutting-edge and do well compared to the lame crap
> business developers write.
Heh, I love this attitude. This is exactly why game development is
mired in the shit that it is right now. "Game development isn't
rocket science, it's /way/ harder than rocket science." LOL. Fuck
yeah!
> Maybe we should be learning something from them? Also, are you coming from
> the philosophy that XP and agile development is the "right" way to program?
> If so, I think game developers show it is at least not the only way.
Actually, what we can learn is how to make software better. The state
of the software in games is miserable, in general. Game studios
regularly ignore the masters; Brooks told us that adding more people
to a late project is death, yet we still do it. We all know that
developers' efficiency goes down the longer they go without rest, yet
we (as an industry) still "mandate" seven-day work weeks.
Game development is NOT rocket science. Game development is easy.
That's the only reason why we're able to do it with the hackery that
we did. If game development were actually hard, we wouldn't be able
to hack through it like we do, and either a sustainable process would
have come to us earlier, or we wouldn't be making games.
-tom!
sted...@yahoo.com
> That's the only reason why we're able to do it with the hackery that
> we did. If game development were actually hard, we wouldn't be able
> to hack through it like we do, and either a sustainable process would
> have come to us earlier, or we wouldn't be making games.
Tom, Easy? Can you at least say unique or specialized? I mean game
programmers have to know Physics and a heavy, heavy dose of math. That
doesn't even touch on the optimization wizardry they produce?
Maybe programming is hard when the field you are working in is hard.
Maybe a complex stock market simulator would also be hard. It just
seems to me that game programming requires a much deeper understanding
of the machine then a client/server application.
> Actually, what we can learn is how to make software better.
How when you cannot see that game development and traditional business
development is different?
> Are you a game developer? If so what commercial games have you
produced?
Stede
Tom Plunket
> > (A studio simultaneously shipping two games generates trade
> > literature headlines.)
>
> I'm not sure where you got this impression. It does happen. (And
> probably more than you realise precisely because it does not
> generate trade literature headlines when it does?)
November 2004 issue of Game Developer.
-tom!
Tom Plunket
> You have a good point but I still feel game development is in a field
> of its own. First, almost anyone can write a payroll system (think C3) but
> inventing cutting-edge game graphics and sounds that have never been done
> before is tough.
Show me a game that has done some sort of programming "cutting-edge"
graphics work that has never been done before. Maybe something from
id or Epic, but other than that, everyone's reinventing the wheel but
it's not exactly an unsolved problem anymore. (Please note that I
don't consider reinventing the wheel to be doing something that has
never been done before. Despite the fact that NotInventedHere is
rampant in the game industry, we're still all building on the work of
those who have come before us.)
What are some examples of cutting-edge game sounds that have never
been done before? How does the implementation of these sounds
illustrate technology that has never been done?
> Add all the other factors that go into game design and I think it would be
> almost impossible to do XP at companies like EA, Activision and Blizzard.
He's talking programming, not game design. BTW, Blizzard says they're
doing XP. Do we not believe them?
> It would be interesting to see a AAA title done with XP though.
The company that I work for will be starting down this path very soon.
We've already moved toward many agile processes, and each switch we
flip makes development go that much faster.
> Working 90 hour weeks minimum and enough teenagers to make 1 game a day
> for the next 500 years I don't think it will happen any time soon.
???
Anyway, it's not hard to think of systems that have much more
stringent requirements than game development. Consider the back-ends
for credit card companies. Despite perhaps billions of dollars in
transactions a day, I have never bought something on my credit card
and not ended up getting the bill for it. I can go to my credit
card's website and see my transaction history without waiting for more
than a couple of seconds. Consider the software behind any
spacecraft; sure, the tech isn't super-advanced, but the requirements
of not killing people make the software's functionality that much more
important.
Philippa Cowderoy
> never been done before. Despite the fact that NotInventedHere is
When WikiWords attack! :-)
> What are some examples of cutting-edge game sounds that have never
> been done before? How does the implementation of these sounds
> illustrate technology that has never been done?
>
*Hah*. That said, I'd love to see better reference material available on
the sort of stuff that goes into eg the average Lexicon or TC effects
unit. And one of these days I'll learn enough DSP to write a half-decent
softsynth, honest...
> Anyway, it's not hard to think of systems that have much more
> stringent requirements than game development.
Quite. Games might be a step above typical business code, but so're 101
other things.
Stede Troisi
You win this argument. I must agree. Blizard doing XP makes sense. Their
games are always better designed and contain less bugs then most of their
competitors.
Can I assume if Blizzard is doing XP they can't have programming sweatshops
like EA?
Would anything in your opinion need to be changed/amended to XP to make it
easier to fit into the game development lifecycle?
Stede
"Tom Plunket" <plu...@gmail.com> wrote in message
news:7d209056.04111...@posting.google.com...
Nathan Mates
Stede Troisi <st...@verizon.net> wrote:
>Can I assume if Blizzard is doing XP they can't have programming sweatshops
>like EA?
Reading some of the recent postings, it seems that the sweatshop
mentality at EA comes from a management attitude of "you will work
long hours." [I can't comment on the validity of those assumptions,
just presenting my limited understanding of them.] The number of hours
worked seems to be irrelevant to XP-- it can be combined with few or
many hours. A manager of a company using XP could very easily turn it
into a sweatshop. Or, the employees could do it to themselves-- giving
dirty looks to people who leave early.
Don't use hours worked as a plus/minus for XP. It's tangential, and
irrelevant. Silver bullets are usually tarnished.
Nathan Mates
--
<*> Nathan Mates - personal webpage http://www.visi.com/~nathan/
# Programmer at Pandemic Studios -- http://www.pandemicstudios.com/
# NOT speaking for Pandemic Studios. "Care not what the neighbors
# think. What are the facts, and to how many decimal places?" -R.A. Heinlein
Stede Troisi
I can't stop playing it! I wish it had its own level editor though and
worked with a joystick.
Stede
"Nathan Mates" <nat...@visi.com> wrote in message
news:419aa140$0$218$a186...@visi.com...
Phlip
> Reading some of the recent postings, it seems that the sweatshop
> mentality at EA comes from a management attitude of "you will work
> long hours." [I can't comment on the validity of those assumptions,
> just presenting my limited understanding of them.] The number of hours
> worked seems to be irrelevant to XP-- it can be combined with few or
> many hours. A manager of a company using XP could very easily turn it
> into a sweatshop. Or, the employees could do it to themselves-- giving
> dirty looks to people who leave early.
>
> Don't use hours worked as a plus/minus for XP. It's tangential, and
> irrelevant. Silver bullets are usually tarnished.
Suppose, instead of rescuing those meek programmers from their tyrant
bosses, or proving XP is good for everything, we instead set a goal of
"rapidly produce awesome games, and kick our competition's butt."
How would you (plural) target that goal?
--
Phlip
http://industrialxp.org/community/bin/view/Main/TestFirstUserInterfaces
John Roth
> The number of hours
> worked seems to be irrelevant to XP-- it can be combined with few or
> many hours. A manager of a company using XP could very easily turn it
> into a sweatshop. Or, the employees could do it to themselves-- giving
> dirty looks to people who leave early.
>
> Don't use hours worked as a plus/minus for XP. It's tangential, and
> irrelevant. Silver bullets are usually tarnished.
I wouldn't be so sure about that. There's quite a bit of evidence
that you can't work more than about six hours doing straight-out
TDD and pairing. If you try to, you start making so many many
mistakes from mental fatigue that it isn't worth it.
In other words, trying to get more than six hours a day out of
your developers with XP is going to send your velocity plumeting
toward zero.
The other two hours, of course, are e-mail, meetings and other
stuff that has to be done but isn't development work.
John Roth
Nathan Mates
John Roth <newsg...@jhrothjr.com> wrote:
>I wouldn't be so sure about that. There's quite a bit of evidence
>that you can't work more than about six hours doing straight-out
>TDD and pairing. If you try to, you start making so many many
>mistakes from mental fatigue that it isn't worth it.
There's quite a bit of evidence that there's a limit without
XP. Usually a bit higher (and depends on the person), but still under
the number of hours worked during crunch mode on games. Once you work
past a certain number of hours, productivity goes negative as you
have to spend time cleaning up after the mistakes of yesterday.
You can argue this point till you're blue in the face. But, when
management doesn't care, then XP, non-XP, whatever you want to call
your development process is irrelevant. XP is *NOT* a silver bullet
for working sane hours. It's tangential to the real issue.
John Roth
> In article <10pld9e...@news.supernews.com>,
> John Roth <newsg...@jhrothjr.com> wrote:
>>I wouldn't be so sure about that. There's quite a bit of evidence
>>that you can't work more than about six hours doing straight-out
>>TDD and pairing. If you try to, you start making so many many
>>mistakes from mental fatigue that it isn't worth it.
>
> There's quite a bit of evidence that there's a limit without
> XP. Usually a bit higher (and depends on the person), but still under
> the number of hours worked during crunch mode on games. Once you work
> past a certain number of hours, productivity goes negative as you
> have to spend time cleaning up after the mistakes of yesterday.
>
> You can argue this point till you're blue in the face. But, when
> management doesn't care, then XP, non-XP, whatever you want to call
> your development process is irrelevant. XP is *NOT* a silver bullet
> for working sane hours. It's tangential to the real issue.
In that sense, you're quite right. If you've got that issue, there
are a number of formal studies (beginning with some at the
US DoD) that demonstrate that error rates go sky high as
fatigue sets in.
The medical community doesn't seem to have learned this either,
from how they schedule interns and residents.
John Roth
Phlip
> There's quite a bit of evidence that there's a limit without
> XP. Usually a bit higher (and depends on the person), but still under
> the number of hours worked during crunch mode on games. Once you work
> past a certain number of hours, productivity goes negative as you
> have to spend time cleaning up after the mistakes of yesterday.
>
> You can argue this point till you're blue in the face. But, when
> management doesn't care, then XP, non-XP, whatever you want to call
> your development process is irrelevant. XP is *NOT* a silver bullet
> for working sane hours. It's tangential to the real issue.
Not working long hours is the silver bullet for working long hours.
Studies (by game shops) have shown that if you actually track velocity, it
does not go up as the hours get longer.
I suggested "design code and finish one level at a time". I'm not sure, but
I suspect games grow breadth-first.
Just like working sane hours, there are people who will reject such radical
ideas simply because they have never been done.
Look to "lean hardware design", aka lean manufacturing. You delay decisions,
and empower the workers who collected the raw data about things.
--
Phlip
http://industrialxp.org/community/bin/view/Main/TestFirstUserInterfaces
WTH
with:
> Nathan Mates wrote:
You're missing Nathan's point Philip. If XP was a magic bullet, and you
gave it to EA, they'd just double their game output and continue to work
people into the ground. Hours are 'tangential' to software methodology in
EA's case (copyright NM 2004)
WTH
Phlip
> > Suppose, instead of rescuing those meek programmers from their tyrant
> > bosses, or proving XP is good for everything, we instead set a goal of
> > "rapidly produce awesome games, and kick our competition's butt."
> >
> > How would you (plural) target that goal?
>
> You're missing Nathan's point Philip. If XP was a magic bullet, and you
> gave it to EA, they'd just double their game output and continue to work
> people into the ground. Hours are 'tangential' to software methodology in
> EA's case (copyright NM 2004)
Good management, in general, might help EA-style projects.
Pure XP would help too - the practice called variously "40 hour weeks",
"sustainable pace", and "energetic work" forbids chronic overtime.
Whether or not XP is a magic bullet, if EA did XP with chronic overtime then
it wouldn't be XP.
However, some folks see "agile" or "XP", don't read any farther, and then
assume someone thinks they have a magic bullet.
I don't know about EA's management, but managers who can be helped are
advised to read /Slack/ by Tom DeMarco.
--
Phlip
http://industrialxp.org/community/bin/view/Main/TestFirstUserInterfaces
Christer Ericson
plu...@gmail.com says...
Oh please, that's an article submitted by the developer,
not a news item.
As Game Developer postmortems go, it's a particularly
vacuous one, and reads mostly like an ad for the
developer. (GD really needs to change the format of
the postmortem articles -- many don't contain any
interesting info at all. IMO.)
Christer Ericson
Sony Computer Entertainment, Santa Monica
Laurent Bossavit
> The number of hours worked seems to be irrelevant to XP -- it can
> be combined with few or many hours.
That's an odd statement. One of the very first things that attracted me
to XP was its officially recommended practice "Sustainable Pace", which
at the time was named "Forty Hour Week".
It seemed good to me that the method, considered as a contract, should
include such a provision. If I had at some point struck a bargain with
my management that we were going to use XP, then there would be
something in there to cover overtime. Unlike previous occasions, I would
have an early line of defense before it came to "Work harder or we fire
you."
Since then, my position has shifted a bit - in all circumstances I make
my own hours, as appropriate for the situation at hand - but being
encouraged by XP to reason through the issue has been a valuable aid
along the way.
Laurent
Paul Sinnett
> What top three problems would you list? Be brutal - many shops
> grow accustomed to some situations, just as ones eyes grow
> accustomed to the dark.
Okay, my top three would be:
1. Like the music and movie industry, it is now dominated by a few
massive companies that routinely eat up independents or starve
them out of existence by leaving little room for competition.
2. Again, like the big movie and music companies, what they produce
is largely celebrity vehicles, licenses, sequels, and "me too"'s.
And this tripe outsells vastly superior original games.
3. The people management is ridiculous. This means a low quality of
life for developers and a high staff turnover. For huge companies
this is increasingly unsustainable. (As EA is finding out.)
Paul Sinnett
nit picking.
John Roth wrote:
> I suspect that the norm is to use an existing engine to crank out
> mid-level games quickly. It's only the real high end games where
> the intention is to push the state of the art where you really
> want to do wholesale revisions to existing engines.
That was broadly correct a few years ago. However, as the gaming
public started detecting this, game-o-matic engines started to
decline. (At the same time, original games started re-using more
from existing code.) Today, most games are built on at least some
legacy code. Some of that evolved into middleware, some is still
kept proprietary. I don't know of any recent games that where
built from the ground up (although the ground is very well
known at it probably wouldn't take so long if they did!)
> I don't necessarilly think of this as bad. While I've heard of
> XP teams that are larger than about a dozen people, the
> "conventional wisdom" seems to put an upper limit on
> an effective team that is somewhere around that point.
> Beyond that, you need a strategy to coordinate multiple
> teams, or operate in some other fashion.
That's right. And in games they are usually divided by job
category rather than along other lines. There are a number of
reasons for this that have to do with the practicalities of
making games; it's not an arbitrary decision and could not
be split another way without addressing many of those issues.
However, that's probably an article in itself.
> IMO, crunch mode usually comes from not having a production
> quality deployable at frequent intervals throughout the
> development process.
IME that's not the case. Even games teams that are crunching
like mad still produce high quality builds from very early in
the development, right up to the final master. In fact, the
only thing that makes it the final master is that you've run
out of time.
> For me, the question simply comes down to:
>
> "Would you rather be able to ship a bug-free product that's
> missing a couple of levels that you had on the storyboards
> in time for Christmas, or would you rather ship the complete
> game your designers had in mind and miss the Christmas
> shopping window?"
>
> This is simply one of the basic XP questions, with the blanks
> filled in from the game development lexicon.
>
> However, the question only makes sense if the process is
> capable of producing that production quality deployable
> every couple of weeks, with a new level or two, fixes
> for previous levels from the play testers, game engine
> enhancements and so forth.
Broadly speaking, that is what happens. The game you buy in the
shops at Christmas is indeed the planned game minus levels and
so forth.
If you've read the recent EA story, you'll know that it is
fairly common for projects that are on time and within budget
to be crunching like mad for the duration. It's not that they
are trying to catch-up. It's that they are trying to cram in
as much as possible. (The irony being, that by pacing them-
selves, they'd probably get more done anyway!)
> The basic issue is: which feedback loop needs tweaking? The
> only way you tell that is to find the bottlenecks and work on
> them. In this example, it sounds like gameplay engine change
> requests weren't a bottleneck.
>
> On the other hand, they may well have been. I can very
> easily envision a scenario where the engine people were
> being burried under a load of change requests for constants
> that didn't actually do anything. Externalizing that would
> have removed the change request queue.
Yes. The changes were becoming a bottleneck. Or at least, an
annoyance. Once instant feedback was available - that is, you
could make edits while playing the game - it was no longer a
bottleneck. However, the paradox is that without the bottle-
neck, there were no more changes.
It reminds me of when I first tried to write using a WYSIWYG
word processor. I wrote a heading and a couple of paragraphs,
and then spent the next few hours fiddling with fonts and
sizes and layout. Writing then became a constant struggle of
resisting the temptation to fiddle. Now I don't bother. Most
of what I write, I write in plain text.
I'm guessing that my customer felt the same way. Once he had
the ability to fiddle he realized that it just got in the
way of doing any real work; and so stopped doing it.
Phlip
> 1. Like the music and movie industry, it is now dominated by a few
> massive companies that routinely eat up independents or starve
> them out of existence by leaving little room for competition.
Okay. The ability to rapidly deploy a non-standard game in an alternative
genre would help here.
> 2. Again, like the big movie and music companies, what they produce
> is largely celebrity vehicles, licenses, sequels, and "me too"'s.
> And this tripe outsells vastly superior original games.
The Onion's book /Our Dumb Century/, in their headlines for 1952 or so,
announced that a new product, "Tele-vision", would be an enlightened
alternative to all the banality on the radio, and that schools would be
obsolete by 1970.
> 3. The people management is ridiculous. This means a low quality of
> life for developers and a high staff turnover. For huge companies
> this is increasingly unsustainable. (As EA is finding out.)
After a dumb generic title is green-lighted, what are the common technical
problems you see in development?
I suspect that applying a methodology with as many technical recommendations
as XP, out of the box, would cause trouble without more checks and balances
for game development's special needs.
--
Phlip
http://industrialxp.org/community/bin/view/Main/TestFirstUserInterfaces
Paul Sinnett
Phlip wrote:
> After a dumb generic title is green-lighted, what are the common
> technical problems you see in development?
I don't. Most of the problems I see are non-technical. If there
are technical problems (and I'm sure there are many) I don't get
to see them or deal with them because of far more pressing non-
techincal problems.
> I suspect that applying a methodology with as many technical
> recommendations as XP, out of the box, would cause trouble
> without more checks and balances for game development's special
> needs.
Quite likely. But then, in general, game development doesn't fit
the minimum requirements on the box.
Paul Sinnett
> Paul Sinnett wrote:
> > 1. Like the music and movie industry, it is now dominated by a
> > few massive companies that routinely eat up independents or
> > starve them out of existence by leaving little room for
> > competition.
>
> Okay. The ability to rapidly deploy a non-standard game in an
> alternative genre would help here.
You'd think. But it hasn't been the case so far. However, changes
in distribution technology might shake things up a bit. The next
few years should be interesting for all the entertainment industries.
Christer Ericson
paul_s...@yahoo.co.uk says...
>
> Phlip wrote:
> > After a dumb generic title is green-lighted, what are the common
> > technical problems you see in development?
>
> I don't. Most of the problems I see are non-technical. If there
> are technical problems (and I'm sure there are many) I don't get
> to see them or deal with them because of far more pressing non-
> techincal problems.
Exactly. The problems we have are very much non-technical, and
just as Paul says, while there may be technical problems, they
are much too heavily outweighed by the non-technical problems
to even be noticed.
Paul Sinnett
Funny. I have that magazine. I even read the article. And I still
didn't realise that was what Phlip was referring to. I thought he
was writing rhetorically.
True it is on the front cover. But it's a postmortem, and they are
always on the cover. And as Christer says, it isn't news. The last
company I worked for is smaller than Timegate but still had 3 (or
maybe more) projects on the go at the same time. I also worked for
Acclaim but I have no idea how many games they were working on
simultaneously. Quite a few I'd guess.
The article itself is about the advantages and disadvantages of moving
from single title development to multiple titles. But it's
not about breaking new ground in game development.
Paul Sinnett
> In that sense, you're quite right. If you've got that issue,
> there are a number of formal studies (beginning with some at
> the US DoD) that demonstrate that error rates go sky high as
> fatigue sets in.
>
> The medical community doesn't seem to have learned this either,
> from how they schedule interns and residents.
Paul Sinnett
> BTW, Blizzard says they're doing XP. Do we not believe them?
Did you ever get any answers from Blizzard? I seem to remember
that one team in one studio of the group (Blizzard North?) used
XP on a tool once. Did it ever get any further than that?
I remember asking to quiz their teams on their XP experience
and getting this rather ironic reply from their PR manager:
I do know that the World of Warcraft team will not be able to
participate, as they are already working 12-14-hour days as it
is trying to get the game to beta.
Tom Plunket
> > IMO, crunch mode usually comes from not having a production
> > quality deployable at frequent intervals throughout the
> > development process.
>
> IME that's not the case. Even games teams that are crunching
> like mad still produce high quality builds from very early in
> the development, right up to the final master. In fact, the
> only thing that makes it the final master is that you've run
> out of time.
Where do you work? This sure isn't my experience in corporate games.
At my (one) previous game job and my present one, the game is very
difficult to package up as a "deliverable," and as such it's really
not ready to go out. Change is barely managed, and snapshotting the
game at any given point in the cycle will usually yield something that
looks like it might be a game but isn't robust in any sense of the
word.
> > The basic issue is: which feedback loop needs tweaking? The
> > only way you tell that is to find the bottlenecks and work on
> > them. In this example, it sounds like gameplay engine change
> > requests weren't a bottleneck.
> >
> > On the other hand, they may well have been. I can very
> > easily envision a scenario where the engine people were
> > being burried under a load of change requests for constants
> > that didn't actually do anything. Externalizing that would
> > have removed the change request queue.
>
> Yes. The changes were becoming a bottleneck. Or at least, an
> annoyance. Once instant feedback was available - that is, you
> could make edits while playing the game - it was no longer a
> bottleneck. However, the paradox is that without the bottle-
> neck, there were no more changes.
I feel differently on this one also, since the instant-change,
instant-feedback system that we have in place is critical for most of
the systems in the game I'm presently working on. Perhaps one
difference is that the instant feedback system doesn't require
restarting the game. :) Another certainly is that our designers have
grown up with this system and are accustomed to initially authoring
content in "ballpark" mode, and then tuning while running the game.
> I'm guessing that my customer felt the same way. Once he had
> the ability to fiddle he realized that it just got in the
> way of doing any real work; and so stopped doing it.
The subtle point is that the work must be directed. Exactly what is
it that they're trying to accomplish? If it's just fiddling then it's
probably wasting time. If it's "make the enemies shoot faster", then
there's a reason behind it and the content author can twiddle 'til
everything feels right.
-tom!
Tom Plunket
> > Game development is NOT rocket science. Game development is easy.
> > That's the only reason why we're able to do it with the hackery that
> > we did.
>
> Tom, Easy? Can you at least say unique or specialized?
Nope, it's not unique, and it's only specialized insofar as people
reinvent the wheel over and over again, and each wheel is subtly
different.
> I mean game programmers have to know Physics and a heavy, heavy dose of math.
> That doesn't even touch on the optimization wizardry they produce?
And missles need to fly to their targets and blow up to most
effectively kill people. I would surmise that the software inside a
missle is every bit as math intensive as the software in a game.
Games are easy because the domain is so narrow. Well, ok, actually
you have hard things in games like PC compatibility issues and
networking issues that come up with online multiplayer games, but most
of a game's development is focused on knowing exactly what inputs
could possibly come in and having a highly constrained set of possible
outputs.
It's just database programming at the end of the day. Our front ends
are just snazzy.
> Maybe programming is hard when the field you are working in is hard.
Programming is hard when a person makes it hard. IMHO, the act of
programming isn't the hard part in (most?) software development. The
hard part is understanding what "the customer" wants sufficiently.
Delivering what the customer wants is trivial if you understand the
problem.
> It just seems to me that game programming requires a much deeper understanding
> of the machine then a client/server application.
Console (e.g. PlayStation or Xbox) programming doesn't require dealing
with browser interop issues, nor does it require dealing with (much)
internationalization.
We've got some constraints that other domains don't have, but those
are workable. "The machine has 32M of RAM and that's it" is one.
"The game loop should run in 16ms" is one usually as well, although
this one is somewhat soft for many games.
One thing that makes programming games easier than other domains in
some respects is the fact that our customer actually is actually
on-site and often only a few feet away.
> > Actually, what we can learn is how to make software better.
>
> How when you cannot see that game development and traditional business
> development is different?
???
> Are you a game developer?
Yep.
-tom!
Nathan Mates
Tom Plunket <plu...@gmail.com> wrote:
>Programming is hard when a person makes it hard. IMHO, the act of
>programming isn't the hard part in (most?) software development. The
>hard part is understanding what "the customer" wants sufficiently.
>Delivering what the customer wants is trivial if you understand the
>problem.
Bingo-- the main problem is that the requirements are mutable and
seemingly random at times. No amount of rejiggering the development
process will fix that, as the problem doesn't lie within what's
tweakable by the developers. As long as people can come to you and say
"we know you've got a great first-person WWII squad combat simulator;
the focus group wants it to play more like Bejeweled by this Friday"
then bugs will be introduced and the schedule will slip.
Those who have never shipped a game will think that some
methodology will allow them to account for such micromanaging
interference from outside. Moats and boiling oil may be the only thing
that might work, but as long as developers aren't self-funded, that's
hard to do.
Phlip
> Those who have never shipped a game will think that some
> methodology will allow them to account for such micromanaging
> interference from outside. Moats and boiling oil may be the only thing
> that might work, but as long as developers aren't self-funded, that's
> hard to do.
The attitude "Those who have never shipped a game" is part of the problem
here. Because the software engineering and design of games are _not_
reproducible or predictable, everyone depends on heros who have "shipped
games before".
I personally refuse to listen to anyone who has never shipped a distributed
object database written in Python that can quantize genetic codes more
efficiently than common industry benchmarks before.
--
Phlip
http://industrialxp.org/community/bin/view/Main/TestFirstUserInterfaces
WTH
> distributed object database written in Python that can quantize
> genetic codes more efficiently than common industry benchmarks before.
I would agree if they're posting diatribes about what is wrong with doing it
the way you're going about it. It makes them look like a bit of an ass,
what with their not actually knowing what they're talking about and all.
WTH
Phlip
> The subtle point is that the work must be directed. Exactly what is
> it that they're trying to accomplish? If it's just fiddling then it's
> probably wasting time. If it's "make the enemies shoot faster", then
> there's a reason behind it and the content author can twiddle 'til
> everything feels right.
I have this friend who works on a game that's nearing "alpha" phase.
Operating the bench build, my friend has made the main character...
- shoot an enemy out from under its head, which floats there
- fight invisible characters wielding visible weapons
- shoot at visible characters riding invisible horses
- fall thru solid floor, and keep falling forever
- throw dynamite at an enemy, convincing its AI to run it off a cliff
They will probably leave the last behavior active, because it fits the
context charmingly. The question here is _not_ "let's see if we can make
design emergent". The big dirty secret is game design is ALREADY EMERGENT.
Getting a handle on this process is much more healthy than denying it.
--
Phlip
http://industrialxp.org/community/bin/view/Main/TestFirstUserInterfaces
Paul Sinnett
> The attitude "Those who have never shipped a game" is part of the
> problem here.
It's certainly a problem. But it's a problem of groups not of games.
The extreme programming group responds to all criticism with the
question "have you tried it?" too. Bion did some useful work in this
area that you might want to take a look at.
----
Remove mote from own eye before applying!
Paul Sinnett
> Where do you work? This sure isn't my experience in corporate
> games. At my (one) previous game job and my present one, the game
> is very difficult to package up as a "deliverable," and as such
> it's really not ready to go out. Change is barely managed, and
> snapshotting the game at any given point in the cycle will usually
> yield something that looks like it might be a game but isn't
> robust in any sense of the word.
It might be a difference in terms of contract. Almost every game
that I have worked on required a deliverable with every milestone.
No deliverable, no money. Automating the build process is usually
one of the first things we do. YMMV.
> > Yes. The changes were becoming a bottleneck. Or at least, an
> > annoyance. Once instant feedback was available - that is, you
> > could make edits while playing the game - it was no longer a
> > bottleneck. However, the paradox is that without the bottle-
> > neck, there were no more changes.
>
> I feel differently on this one also, since the instant-change,
> instant-feedback system that we have in place is critical for most
> of the systems in the game I'm presently working on. Perhaps one
> difference is that the instant feedback system doesn't require
> restarting the game. :) Another certainly is that our designers
> have grown up with this system and are accustomed to initially
> authoring content in "ballpark" mode, and then tuning while
> running the game.
I have an experience to bring to this discussion that you'd be
hard pushed to beat even under laboratory conditions. I recently
worked on a game engine which powered two significantly different
games for the same company. One was designed in detail up front and
was a joy to play and work on. The other was very vague up front,
designed as it was developed, and was awful to play and boring to
work on. The game code and the development processes were otherwise
identical. The only difference was in the design. I'd love to think
that a game design could emerge from the development process. I've
seen it attempted numerous times. But all my experience tells me
that it just doesn't work.
> The subtle point is that the work must be directed. Exactly what
> is it that they're trying to accomplish? If it's just fiddling
> then it's probably wasting time. If it's "make the enemies shoot
> faster", then there's a reason behind it and the content author
> can twiddle 'til everything feels right.
Have you actually measured the changes that are happening. I was
as surprised as anyone when it turned out no significant changes
were needed.
The system I put in place recorded all the changes in a text file.
When the designer had fiddled to his hearts content he returned
the file to me. Perhaps he felt after some considerable fiddling
that he had perfected the balance. It was only when I checked this
file against the original that I noticed there was only one tiny
modification that nobody could seriously be expected to detect
given the games side by side. I may not even have made the change,
I don't recall.
I have noticed in game designers a certain reluctance to release
their game. I'd imagine it's the same for all artistic endeavours.
As an open question to all game developers*: how many times has
the playability of your game been tweaked to it's detriment by
your own game designer in the last few days before the game's
release?
* Disclaimer: Phlip, this is not in an attempt to exclude, but
because only game developers would have the
experience to answer the question.
Phlip
> I have an experience to bring to this discussion that you'd be
> hard pushed to beat even under laboratory conditions. I recently
> worked on a game engine which powered two significantly different
> games for the same company. One was designed in detail up front and
> was a joy to play and work on. The other was very vague up front,
> designed as it was developed, and was awful to play and boring to
> work on. The game code and the development processes were otherwise
> identical. The only difference was in the design. I'd love to think
> that a game design could emerge from the development process. I've
> seen it attempted numerous times. But all my experience tells me
> that it just doesn't work.
The game industry uses "design" to mean "play scripting".
Is Big Play Scripting Up Front evil? It seems to work for movies...
--
Phlip
http://industrialxp.org/community/bin/view/Main/TestFirstUserInterfaces
Phlip
> Phlip wrote:
> > The attitude "Those who have never shipped a game" is part of the
> > problem here.
>
> It's certainly a problem. But it's a problem of groups not of games.
> The extreme programming group responds to all criticism with the
> question "have you tried it?" too. Bion did some useful work in this
> area that you might want to take a look at.
That is the difference, not the similarity. The XP group says, "when people
try this technique, they often claim it feels more successful and
reproducible."
The game developers say, "we won't listen to anyone who has not been
spectacularily successful before, because we don't feel our process is
reproducible."
--
Phlip
http://industrialxp.org/community/bin/view/Main/TestFirstUserInterfaces
phli...@gmail.com
> * Disclaimer: Phlip, this is not in an attempt to exclude, but
> because only game developers would have the
> experience to answer the question.
Excluding is fine when you ask a question, as a survey. It's not fine
when listening to an answer.
Tom Plunket
> "we know you've got a great first-person WWII squad combat simulator;
> the focus group wants it to play more like Bejeweled by this Friday"
> then bugs will be introduced and the schedule will slip.
LOL, beauty.
> the main problem is that the requirements are mutable and seemingly random at
> times. No amount of rejiggering the development process will fix that, as the
> problem doesn't lie within what's tweakable by the developers.
Not exactly true, as my current experience (and long-time assumption
before that) has illustrated fairly well.
> Those who have never shipped a game will think that some methodology will
> allow them to account for such micromanaging interference from outside. Moats
> and boiling oil may be the only thing that might work, but as long as
> developers aren't self-funded, that's hard to do.
Speaking as one who's shipped a few games and millions of units, I
have discovered that the agile methods help alleviate this problem in
an amazing fashion.
Micromanaging goes away as soon as the tech team works with the
mandate, "we're working on these features this month, period. If you
want anything new, that's cool. Anything new goes on the big list,
roughly prioritized against everything else. When the next month
comes around, we'll look at the list, our designers will evaluate the
priorities, and we'll start work on the next month's worth of
features."
It's actually been working fairly well. We have been growing into
this system since April maybe, and people are still learning to work
with it, but generally the only people who are having problems with it
are the ones who act like that first quoted bit above about the WW2
shooter into Bejeweled. Sticking all feature requests on a backlog
means that people will think about those features more between "now"
and when the task comes up to be implemented, and it seems to be
helping us deliver higher quality features earlier. I can't wait to
actually start a project with all of this stuff in place.
-tom!
Phlip
> Micromanaging goes away as soon as the tech team works with the
> mandate, "we're working on these features this month, period. If you
> want anything new, that's cool. Anything new goes on the big list,
> roughly prioritized against everything else. When the next month
> comes around, we'll look at the list, our designers will evaluate the
> priorities, and we'll start work on the next month's worth of
> features."
One important thing those airhead commie "agile" methodologies enforce is
very similar to Marx's "worker's control of the means of production".
In both XP and Scrum, you get a religiously enforced mandate: The business
side may set priorities for each feature, but may not interrupt a sprint
(iteration). The technical side owns time estimates for each feature, but
may not implement the features without wall-to-wall tests and review.
I am most curious how to define "feature" in the game context. Is blowing an
opponent's intestines out a feature? Is falling at exactly 32 feet per
second squared a feature? Is "your player gets killed 66% of the time in
Advanced Mode" a feature? Note that art, middleware, and game design all
influence these features, and changes can derail existing features.
XP works exactly because the testing makes new features safe to specify
without absurdly high time estimates or long bug hunts. Game programming
tempts with the question - how far can you push testing? Can you write a
test for _anything_ a game designer thinks to request?
Part of the answer there is simple: The tester role's main job is making
sure everyone writes tests at every level, including designers. This implies
design will slide into "that which is testable".
> It's actually been working fairly well. We have been growing into
> this system since April maybe, and people are still learning to work
> with it, but generally the only people who are having problems with it
> are the ones who act like that first quoted bit above about the WW2
> shooter into Bejeweled. Sticking all feature requests on a backlog
> means that people will think about those features more between "now"
> and when the task comes up to be implemented, and it seems to be
> helping us deliver higher quality features earlier. I can't wait to
> actually start a project with all of this stuff in place.
And there are those who are so defensive of their high bug rates (caused by
years of squirming under a boss's gaze) that when we tell them "XP teams
report defect rates in the range of 1 every 6 months", they will shriek that
we are "zealots" who think we have a "silver bullet".
Blam! The vampire slumps to the ground and dissolves. But it ain't no silver
bullet, nope!
--
Phlip!
http://industrialxp.org/community/bin/view/Main/TestFirstUserInterfaces
Gerry Quinn
paul_s...@yahoo.co.uk says...
> I've played a lot of console games over the years, and in that time
> I've only ever found one crash bug in a published game. That was
> after playing continuously for many hours and then starting over. On
> the other hand, I've lost count of the amount of times I've lost
> work and had to reboot my PC while using perfectly standard "mature"
> development tools. Now it's true that PC games also suffer from this
> instability, but these games are mostly written and tested by the
> exactly the same people who write the console games. That makes me
> think this is a PC / OS problem rather than defects inherent in PC
> game code or the game development process.
I'd say it is a combination of three issues:
1. Every PC is different, varying both in hardware and software - that
can cause problems especially if you are doing anything fancy with
graphics etc., which games often do. [This is the reason that
developers like to blame.]
2. PC games can be patched. I wonder has anyone ever done a study on
the prevalence of serious and even show-stopping bugs since internet
access becamew widespread? If this is an issue, they should have
increased.
3. Console game developers have the console manufacturers looking over
their shoulder. They will refuse to release a game with serious
problems. In theory, PC publishers should do this too, but for obvious
reasons (fewer titles means less to lose, no hardware to sell means a
bigger discount on future sales as against sales now) they won't be half
so tough on the developer.
Gerry Quinn
http://bindweed.com
Games, Kaleidoscopes, Screensavers
Gerry Quinn
> Micromanaging goes away as soon as the tech team works with the
> mandate, "we're working on these features this month, period. If you
> want anything new, that's cool. Anything new goes on the big list,
> roughly prioritized against everything else. When the next month
> comes around, we'll look at the list, our designers will evaluate the
> priorities, and we'll start work on the next month's worth of
> features."
Maybe 'agile' programmers have that on their list of good things. But
there is hardly anything especially 'agile' about it.
Indeed, in many environments it might be the tech team who have to be
disciplined into working on the features they are supposed to be working
on.
- Gerry Quinn
Gerry Quinn
phli...@yahoo.com says...
> Paul Sinnett wrote:
> >
> > It's certainly a problem. But it's a problem of groups not of games.
> > The extreme programming group responds to all criticism with the
> > question "have you tried it?" too. Bion did some useful work in this
> > area that you might want to take a look at.
>
> That is the difference, not the similarity. The XP group says, "when people
> try this technique, they often claim it feels more successful and
> reproducible."
>
> The game developers say, "we won't listen to anyone who has not been
> spectacularily successful before, because we don't feel our process is
> reproducible."
Maybe they say "I'm glad that you are feeling empowered and successful.
Now show me an actual success."
- Gerry Quinn
Phlip
> Maybe they say "I'm glad that you are feeling empowered and successful.
> Now show me an actual success."
http://www.gamasutra.com/resource_guide/20030714/demachy_01.shtml
I'm not sure if that article describes their success in terms that other
teams could follow and improve their odds. For example, they list the
standard "post mortem" problems:
- Technology evolves too rapidly.
- Developers always want to do their best.
- Publishers frequently change their minds.
People always get used to problems they don't realize they could fix, and
they don't always list them on post-mortems. Problems like a poorly tuned
asset pipeline could bring down a project, even if it follows the progress
metric that article recommends:
"Progress is measured by counting-unit test and functional tests."
I suspect that progress should be measured by reviewing gameplay features,
within their tests.
A team that viciously attacks any reported "impediment", instead of living
with it, would also generate a good metric: A short impediment resolution
backlog.
--
Phlip
http://industrialxp.org/community/bin/view/Main/TestFirstUserInterfaces
Gerry Quinn
phli...@yahoo.com says...
> Gerry Quinn wrote:
>
> > Maybe they say "I'm glad that you are feeling empowered and successful.
> > Now show me an actual success."
>
> http://www.gamasutra.com/resource_guide/20030714/demachy_01.shtml
>
> I'm not sure if that article describes their success in terms that other
> teams could follow and improve their odds. For example, they list the
> standard "post mortem" problems:
Did you even READ it?
"At Titus Interactive Studio near Paris, France, we are currently
starting two game projects using XGD."
[...]
"As I said earlier, XGD is currently being rolled out at Titus
Interactive Studio on two projects. I'll let you know in a few months
how we fared using XGD -- which practices worked and which did not."
It doesn't describe their success. It describes them feeling empowered
and successful, right before they actually start...
- Gerry Quinn
Tom Plunket
> > Micromanaging goes away as soon as the tech team works with the
> > mandate, "we're working on these features this month, period. If you
> > want anything new, that's cool. Anything new goes on the big list,
> > roughly prioritized against everything else. When the next month
> > comes around, we'll look at the list, our designers will evaluate the
> > priorities, and we'll start work on the next month's worth of
> > features."
>
> Maybe 'agile' programmers have that on their list of good things. But
> there is hardly anything especially 'agile' about it.
It actually makes the development team more agile in a narrower focus.
It also keeps the developers working on the things deemed most
important by the stakeholders, because the developers are actually
working on the things that have been given the highest priority rather
than working off of any person's daily whims. It means that you can't
be working on an FPS one day and then someone decide you should do an
RTS the next, only to realize on the third day that maybe an FPS
wasn't a bad idea. Nobody is efficient if they're jumping all over
the map of possible things they could be working on.
Over the course of a sprint (an iteration period of rougly one month),
changes happen. However, changes that get implemented that sprint are
changes that are actually related to the work that's being done that
month as the features are being developed. A 'team' has a set of
goals for the month, and the team consists of programmers, artists,
and designers.
Overall it may be less agile in some respects; you're explicitly not
allowed to throw random things at the programming team and expect with
them to do this new random stuff in addition to the old random stuff
that was already on the list. However, it has the side effect of
making the team much more focused so that team can be more efficient.
One month intervals for large-scale steering of the ship aren't all
that long when you've got a team of 70 people working on a project for
24-30 months. For shorter projects you could go to shorter
iterations, but quite honestly this period seems just about ideal to
me. The whole development team knows what's in development at any
given time, people know what to expect, and they know what might
effect them.
> Indeed, in many environments it might be the tech team who have to be
> disciplined into working on the features they are supposed to be working
> on.
This process helps on that front too. We've got all of the tasks for
the current iteration up on the wall in a grid. The rows are "goals",
fairly high-level requirements. The columns are "To Do" for tasks
that haven't been started on, "In Progress" for things that are being
actively worked on, and "To Merge" for tasks that are completed in the
'home' code branch, and "Done" when the task makes it into the
mainline.
If an individual developer (be it a coder, an artist, or a designer)
fails to update anything for too long, the team will notice. This
peer pressure keeps people focused on the tasks that they themselves
have moved into the "In Progress" column, and if the task is taking
way longer than they estimated that it would, then people will notice.
Taking notice means that other team members may offer to help get it
done, but the mere presence of it up there encourages people to get
the things done that they say they're working on.
-tom!
Nathan Mates
Tom Plunket <plu...@gmail.com> wrote:
>It actually makes the development team more agile in a narrower focus.
> It also keeps the developers working on the things deemed most
>important by the stakeholders, because the developers are actually
>working on the things that have been given the highest priority rather
>than working off of any person's daily whims. It means that you can't
>be working on an FPS one day and then someone decide you should do an
>RTS the next, only to realize on the third day that maybe an FPS
>wasn't a bad idea. Nobody is efficient if they're jumping all over
>the map of possible things they could be working on.
If "agile" means "having a plan and sticking to it," then Chaucer,
Shakespeare, and Wordsworth must be spinning like jet engines in their
graves. I'm not disputing the value of having a plan; I'm just
somewhat aghast at yet another butchering of the English language.
Phlip
> It actually makes the development team more agile in a narrower focus.
> It also keeps the developers working on the things deemed most
> important by the stakeholders, because the developers are actually
> working on the things that have been given the highest priority rather
> than working off of any person's daily whims. It means that you can't
> be working on an FPS one day and then someone decide you should do an
> RTS the next, only to realize on the third day that maybe an FPS
> wasn't a bad idea. Nobody is efficient if they're jumping all over
> the map of possible things they could be working on.
>
> Over the course of a sprint (an iteration period of rougly one month),
> changes happen. However, changes that get implemented that sprint are
> changes that are actually related to the work that's being done that
> month as the features are being developed. A 'team' has a set of
> goals for the month, and the team consists of programmers, artists,
> and designers.
>
> Overall it may be less agile in some respects; you're explicitly not
> allowed to throw random things at the programming team and expect with
> them to do this new random stuff in addition to the old random stuff
> that was already on the list. However, it has the side effect of
> making the team much more focused so that team can be more efficient.
I can say "agile" in much fewer words:
Minimize the time between making a decision about gameplay
and operating a feature based on that decision in a real game.
The *.design group is on the cross-posting because this is entirely a design
issue. Designers should _demand_ that this pipeline is as short as possible
(and should work with the team to make it so). The fact that game shops can
deliver nice games without this metric is a testament to human fortitude.
> One month intervals for large-scale steering of the ship aren't all
> that long when you've got a team of 70 people working on a project for
> 24-30 months. For shorter projects you could go to shorter
> iterations, but quite honestly this period seems just about ideal to
> me. The whole development team knows what's in development at any
> given time, people know what to expect, and they know what might
> effect them.
That's why Scrum is a nice training for even shorter iterations. I'd vote
for 1 week mini iterations, within each team, and 1 month macro-iterations,
where everyone firms up a release candidate.
> This process helps on that front too. We've got all of the tasks for
> the current iteration up on the wall in a grid. The rows are "goals",
> fairly high-level requirements. The columns are "To Do" for tasks
> that haven't been started on, "In Progress" for things that are being
> actively worked on, and "To Merge" for tasks that are completed in the
> 'home' code branch, and "Done" when the task makes it into the
> mainline.
Right. There are those who think the "To Merge" column represents Fear
creating an impediment. But I'l shut up about that because it seems to be a
sore spot.
> If an individual developer (be it a coder, an artist, or a designer)
> fails to update anything for too long, the team will notice. This
> peer pressure keeps people focused on the tasks that they themselves
> have moved into the "In Progress" column, and if the task is taking
> way longer than they estimated that it would, then people will notice.
> Taking notice means that other team members may offer to help get it
> done, but the mere presence of it up there encourages people to get
> the things done that they say they're working on.
But the "theory" is that designers are the ones pulling for features. Is
that enforced?
--
Phlip
http://industrialxp.org/community/bin/view/Main/TestFirstUserInterfaces
Phlip
> > http://www.gamasutra.com/resource_guide/20030714/demachy_01.shtml
> >
> > I'm not sure if that article describes their success in terms that other
> > teams could follow and improve their odds. For example, they list the
> > standard "post mortem" problems:
>
> Did you even READ it?
Only enough to critique it. I don't predict much "success" above average. I
do think they'l have fun.
Oh, I'm sorry. I forgot. I'm supposed to be rabidly pro-XP, and only
reporting facts that support my conclusions, right?
--
Phlip
http://industrialxp.org/community/bin/view/Main/TestFirstUserInterfaces
Phlip
> If "agile" means "having a plan and sticking to it," then Chaucer,
> Shakespeare, and Wordsworth must be spinning like jet engines in their
> graves. I'm not disputing the value of having a plan; I'm just
> somewhat aghast at yet another butchering of the English language.
Agile generally means "plan your infrastructure so that your actual product
can follow 'adaptive planning'." You code a few features at a time, but
completely finish them, before starting the next set of features.
TP and me are discussing how to map this (admittedly trite) definition onto
game programming. If the business definition of a "feature" is "anything you
can write on a card and specify a test for", how can you stretch that
definition - and the tests - into complex things that games do?
--
Phlip
http://industrialxp.org/community/bin/view/Main/TestFirstUserInterfaces
WTH
> spectacularily successful before, because we don't feel our process is
> reproducible."
Wow, you don't like to distort the opposition, do you? Did anybody ask you
if you were a spectacularly successful game developer or did they just ask
you if you have done ANY professional game development?
LOL, you take offense at the slightest provocation and yet you have no
problem with hyperbole when it suits you.
WTH
Christer Ericson
phli...@yahoo.com says...
> Gerry Quinn wrote;
>
> > > http://www.gamasutra.com/resource_guide/20030714/demachy_01.shtml
> > >
> > > I'm not sure if that article describes their success in terms that other
> > > teams could follow and improve their odds. For example, they list the
> > > standard "post mortem" problems:
> >
> > Did you even READ it?
>
> Only enough to critique it. I don't predict much "success" above average. I
> do think they'l have fun.
>
> Oh, I'm sorry. I forgot. I'm supposed to be rabidly pro-XP, and only
> reporting facts that support my conclusions, right?
Nice try. You posted the link in reply to Gerry's 'Maybe they say
"I'm glad that you are feeling empowered and successful. Now show
me an actual success."', then you referred to "their success"
as if it existed.
From where I'm sitting, you OBVIOUSLY thought the article described
a success scenario. Oops.
Christer Ericson
Sony Computer Entertainment, Santa Monica
Christer Ericson
> I have discovered that the agile methods help alleviate this
> problem in an amazing fashion.
You being into that sort of thing, I think you should hire
Phlip at Sammy! He's clearly an asset: in addition to being
into all things agile, he also knows exactly what your
development problems are and how to fix them. (That he
doesn't grok stuff like how matrix transforms work shouldn't
be much of a problem, right?)
Phlip
> phli...@yahoo.com says...
> > Gerry Quinn wrote;
> >
> > > > http://www.gamasutra.com/resource_guide/20030714/demachy_01.shtml
> > > >
> > > > I'm not sure if that article describes their success in terms that
other
> > > > teams could follow and improve their odds. For example, they list
the
> > > > standard "post mortem" problems:
> > >
> > > Did you even READ it?
> >
> > Only enough to critique it. I don't predict much "success" above
average. I
> > do think they'l have fun.
> >
> > Oh, I'm sorry. I forgot. I'm supposed to be rabidly pro-XP, and only
> > reporting facts that support my conclusions, right?
>
> Nice try. You posted the link in reply to Gerry's 'Maybe they say
> "I'm glad that you are feeling empowered and successful. Now show
> me an actual success."', then you referred to "their success"
> as if it existed.
>
> From where I'm sitting, you OBVIOUSLY thought the article described
> a success scenario. Oops.
Guilty!
But note I didn't try to deny it. You read that into my verbiage. ;-)
--
Phlip
http://industrialxp.org/community/bin/view/Main/TestFirstUserInterfaces
Phlip
> plu...@gmail.com says...
> > I have discovered that the agile methods help alleviate this
> > problem in an amazing fashion.
>
> You being into that sort of thing, I think you should hire
> Phlip at Sammy!
Eeeew! I'd rather work for Sony!!
> He's clearly an asset: in addition to being
> into all things agile, he also knows exactly what your
> development problems are and how to fix them.
If I knew how to fix them why would I ask these newsgroups about them?
> (That he
> doesn't grok stuff like how matrix transforms work shouldn't
> be much of a problem, right?)
Uh, are you refering to my previous postings around town? They were for a
personal project.
However, you are _again_ betraying the "game programming cliche" mentality
when you scoff that a game shop would hire someone who doesn't know 3D
graphics theory. If game programming success were quantizable and
reproducible...
--
Phlip
http://industrialxp.org/community/bin/view/Main/TestFirstUserInterfaces
Christer Ericson
> > Phlip at Sammy!
>
> Eeeew! I'd rather work for Sony!!
Based on the "skills" you've displayed around here, don't
get your hopes up too high.
> However, you are _again_ betraying the "game programming cliche" mentality
> when you scoff that a game shop would hire someone who doesn't know 3D
> graphics theory.
I'm afraid _you_ have it backwards, showing your ignorance of
games yet again. FYI, matrix transformations are ubiquitous
in game code; they are not limited to 3D graphics. It may
surprise you, but it's not a coincidence that (3D) math skills
are stressed at all game development programs across the
world.
Tom Plunket
> If "agile" means "having a plan and sticking to it," then Chaucer,
> Shakespeare, and Wordsworth must be spinning like jet engines in their
> graves. I'm not disputing the value of having a plan; I'm just
> somewhat aghast at yet another butchering of the English language.
I'm not a master of this stuff yet, and haven't memorized the
propaganda. I can assure you, however, that as we adopt more bits of
what the agile community says we should be doing, we are becoming more
receptive to outside input, we can change course easier, and in
general we can meet the needs of our customers (the designers) better.
How the agile Xealotry defends their use of the word 'agile' I cannot
even guess at, but neither do I care because it works for me. :)
-tom!
Tom Plunket
> You being into that sort of thing, I think you should hire
> Phlip at Sammy! He's clearly an asset: in addition to being
> into all things agile, he also knows exactly what your
> development problems are and how to fix them.
He knows what problems exist in software development, presumably from
having been a software developer in the past. The fact that most game
developers have most of the problems he's talking about clearly
doesn't distract anyone from discounting his lack of documentable
evidence that he may at one time have worked on some game that
nobody's heard of.
We're not doing Rocket Science here. We're making games for fuck's
sake. How can we make games easier? I happen to know first-hand that
what Phlip is talking about helps in general. Most of the industry
gives most of these things lip service, too: "oh, we know it'd be
great to have tools that didn't crash all the time, but we don't have
the time to do it (because we're so busy fighting them)", and "oh,
it'd be great to have an easy process for building assets rather than
this manual click-click-click, but it would take so much time (and
would save our artists from getting carpal tunnel syndrome, and would
eliminate mistakes...)". Ahh well, my win I guess; as long as I can
make games faster and cheaper than the competition I don't have to
worry about where the next meal comes from.
(And I forgot that Sony's policy is, "we want to make good games, but
not so good that other publishers can't make better ones otherwise
they'll abandon us like they have Nintendo." I love Sony for many
things, but making good games for little money is not one of their
strengths.)
> (That he doesn't grok stuff like how matrix transforms work
> shouldn't be much of a problem, right?)
A thorough knowledge of 3D transformations is an asset, as are
thorough understandings of any of the other things that we do in game
development. Certainly you need more than 3D transformations to make
a game, no? Indeed, it is my feeling that not everyone on the team
actually needs to really grok 3D matrix math; as long as you've got
someone on the engine that knows it and the animation person knows it
and there's someone on the tools team who understands it, you're
probably set. Hell, I didn't understand it for the first couple of
years I was in the biz and I still successfully spit out animation
tools (we were one of the first single-skin skeletally-deformed
characters games on the PSX) and used 3D math by trial and error.
It's not that hard, after all; one order of operations doesn't do what
you expect, try another one.
WTH
> thorough understandings of any of the other things that we do in game
> development. Certainly you need more than 3D transformations to make
> a game, no?
Uh, matrices are used in many places besides just 3D rendering. The point
being that such a thing is a basic skill for someone manipulating objects in
2D or 3D. Also useful in image manipulation, et cetera.
> Indeed, it is my feeling that not everyone on the team
> actually needs to really grok 3D matrix math; as long as you've got
> someone on the engine that knows it
Yeah, and that person goes on vacation, quits/leaves, gets fired, gets hit
by a truck...
> and the animation person knows it
> and there's someone on the tools team who understands it, you're
> probably set. Hell, I didn't understand it for the first couple of
> years I was in the biz and I still successfully spit out animation
> tools (we were one of the first single-skin skeletally-deformed
> characters games on the PSX) and used 3D math by trial and error.
> It's not that hard, after all; one order of operations doesn't do what
> you expect, try another one.
That's another part of the point. It isn't hard, it is strange at first,
but you can learn it if you want to. Philip isn't interested in being a
professional game developer, he's interested in promoting XP.
WTH
Phlip
> > I'd rather work for Sony!!
>
> Based on the "skills" you've displayed around here, don't
> get your hopes up too high.
Ah, so you guys don't want an test framework tuned to host, process, edit,
and browse the results of thousands of acceptance tests.
What do you all do, hire kids to play the game? How often do they report
it's "hosed"?
--
Phlip
http://industrialxp.org/community/bin/view/Main/TestFirstUserInterfaces
Christer Ericson
phli...@yahoo.com says...
> > > I'd rather work for Sony!!
> >
> > Based on the "skills" you've displayed around here, don't
> > get your hopes up too high.
>
> Ah, so you guys don't want an test framework tuned to host, process, edit,
> and browse the results of thousands of acceptance tests.
No, not really. As several people have already pointed out to
you, that would be a mostly misguided effort. We don't have a
problem with large code-bug counts. The problems are predominantly
non-code related.
Your XP zealotry is making you bark up the wrong tree.
Christer Ericson
plu...@gmail.com says...
> [...] The fact that most game developers have most of
> the problems he's talking about [...]
Really? What is this statement based on? Game Developer
postmortems seem to indicate that problems are--as I and
others here have suggested--production-based, not
programming-based.
> "oh, we know it'd be great to have tools that didn't crash
> all the time, but we don't have the time to do it (because
> we're so busy fighting them)"
What nonsense. I don't know of any developers having problems
with their tools crashing all the time (or even at all).
Unless that was a strawman comment, please point out some
developers that do!
> (And I forgot that Sony's policy is, "we want to make good games, but
> not so good that other publishers can't make better ones otherwise
> they'll abandon us like they have Nintendo."
More nonsense.
> Certainly you need more than 3D transformations to make a
> game, no?
Of course. That doesn't change the fact that 3D math is
just about everywhere in a modern game and that's its an
important skill (one of many).
If Philip is truly interested in getting into game
development he would do better displaying he possesses
the important everyday skills (if indeed he does) rather
than behaving like a loony zealot.
> It's not that hard, after all; one order of operations doesn't
> do what you expect, try another one.
Ah, so that's what TDD means to you! Well, we prefer hiring
people who don't practice trial-and-error programming but
that actually know what they're doing. That's how we keep
our bug counts down.
Phlip
> plunket says...
> > [...] The fact that most game developers have most of
> > the problems he's talking about [...]
>
> Really? What is this statement based on? Game Developer
> postmortems seem to indicate that problems are--as I and
> others here have suggested--production-based, not
> programming-based.
All software engineering has a common, unique problem: Bugs. The problem is
so common that everyone simply builds their culture around it. For example,
Microsoft's XBox developer model isn't called the "Test Kit", it is called
the "Debug Kit". The expectation is you write them, find them, and kill
them.
I wouldn't have used Tom's verbiage, but I suspect that high-end game
development has very common burdens. The biggest difference between game
programming and other kinds might be the huge amount of content, and how
changes in content might affect gameplay.
> > "oh, we know it'd be great to have tools that didn't crash
> > all the time, but we don't have the time to do it (because
> > we're so busy fighting them)"
>
> What nonsense. I don't know of any developers having problems
> with their tools crashing all the time (or even at all).
> Unless that was a strawman comment, please point out some
> developers that do!
Calling an opinion "nonsense" and then immediately asking a question about
it is hardly a good way to conduct a dialog.
> > (And I forgot that Sony's policy is, "we want to make good games, but
> > not so good that other publishers can't make better ones otherwise
> > they'll abandon us like they have Nintendo."
>
> More nonsense.
I thought Tom worked at Sony. Oh, sorry - I get it. You mean that Sony's
corporate policy is nonsense. If that's true, I agree.
> > Certainly you need more than 3D transformations to make a
> > game, no?
>
> Of course. That doesn't change the fact that 3D math is
> just about everywhere in a modern game and that's its an
> important skill (one of many).
>
> If Philip is truly interested in getting into game
> development he would do better displaying he possesses
> the important everyday skills (if indeed he does) rather
> than behaving like a loony zealot.
Game shops don't need more coders capable of transforming matrices all
night. They might need a few more guys familiar with sound software
engineering practices.
Go ahead - say "nonsense"! ;-)
> > It's not that hard, after all; one order of operations doesn't
> > do what you expect, try another one.
>
> Ah, so that's what TDD means to you! Well, we prefer hiring
> people who don't practice trial-and-error programming but
> that actually know what they're doing. That's how we keep
> our bug counts down.
Who brought up TDD?
And what are, in fact, your bug rates?
Apparently, anyone who says "you can write software without the neurotic
cycle of creating bugs and then fixing them" must be a looney zealot around
here. What a culture!
--
Phlip
http://industrialxp.org/community/bin/view/Main/TestFirstUserInterfaces
Nathan Mates
Phlip <phli...@yahoo.com> wrote:
>All software engineering has a common, unique problem: Bugs. The problem is
>so common that everyone simply builds their culture around it. For example,
>Microsoft's XBox developer model isn't called the "Test Kit", it is called
>the "Debug Kit". The expectation is you write them, find them, and kill
>them.
Pop quiz: Sony has some units which sit in a similar place in the
tool chain. What are they called? Not the DTLH-10000, but the smaller
units. I repeat the question: what is the name written into the metal
on them? What is are those Sony units used for? If you're going to
make a "point" (more like a ugly "splat") based on names, then you're
going to make a fool of yourself when you run into those who actually
have a clue about the games industry.
WTH
with:
> In article <mk5od.9098$27....@newssvr16.news.prodigy.com>,
> Phlip <phli...@yahoo.com> wrote:
>> All software engineering has a common, unique problem: Bugs. The
>> problem is so common that everyone simply builds their culture
>> around it. For example, Microsoft's XBox developer model isn't
>> called the "Test Kit", it is called the "Debug Kit". The expectation
>> is you write them, find them, and kill them.
>
> Pop quiz: Sony has some units which sit in a similar place in the
> tool chain. What are they called? Not the DTLH-10000, but the smaller
> units. I repeat the question: what is the name written into the metal
> on them? What is are those Sony units used for? If you're going to
> make a "point" (more like a ugly "splat") based on names, then you're
> going to make a fool of yourself when you run into those who actually
> have a clue about the games industry.
>
> Nathan Mates
He's not going to understand what you're trying to explain to him.
WTH
Phlip
> Phlip wrote:
> >All software engineering has a common, unique problem: Bugs. The problem
is
> >so common that everyone simply builds their culture around it. For
example,
> >Microsoft's XBox developer model isn't called the "Test Kit", it is
called
> >the "Debug Kit". The expectation is you write them, find them, and kill
> >them.
>
> Pop quiz: Sony has some units which sit in a similar place in the
> tool chain. What are they called? Not the DTLH-10000, but the smaller
> units. I repeat the question: what is the name written into the metal
> on them? What is are those Sony units used for? If you're going to
> make a "point" (more like a ugly "splat") based on names, then you're
> going to make a fool of yourself when you run into those who actually
> have a clue about the games industry.
Ah, so Sony doesn't write bugs, manually test them, report them, and then
use a debugger to kill them.
That sounds preposterous; if Sony has found some way to develop without
using manual tests, or a debugger.
Please tell us what they are doing, so we can learn how they achieve these
remarkable results!
--
Phlip
http://industrialxp.org/community/bin/view/Main/TestFirstUserInterfaces
Philippa Cowderoy
> That sounds preposterous; if Sony has found some way to develop without
> using manual tests, or a debugger.
>
I believe the term is something like "programming by construction".
Variations on the theme can be pretty useful when coding something tricky,
though most will want to skip the majority of the maths.
Phlip
> Phlip wrote:
>
> > That sounds preposterous; if Sony has found some way to develop without
> > using manual tests, or a debugger.
> >
>
> I believe the term is something like "programming by construction".
> Variations on the theme can be pretty useful when coding something tricky,
> though most will want to skip the majority of the maths.
That looks translationist. How does it work with the classic "changing
requirements" problem?
--
Phlip
http://industrialxp.org/community/bin/view/Main/TestFirstUserInterfaces
Philippa Cowderoy
> Philippa Cowderoy wrote:
>
> > Phlip wrote:
> >
> > > That sounds preposterous; if Sony has found some way to develop without
> > > using manual tests, or a debugger.
> > >
> >
> > I believe the term is something like "programming by construction".
> > Variations on the theme can be pretty useful when coding something tricky,
> > though most will want to skip the majority of the maths.
>
> That looks translationist. How does it work with the classic "changing
> requirements" problem?
>
Taken with excessive formalism, pretty painfully. If you've got a language
with a good type system, lighter weight versions (ie skipping half the
working, not showing any of it) can be pretty effective - but then it also
starts to sound a lot like an effective coder simply doing what they do.
Or maybe I have a biased sample due to the number of FP folks I talk to?
Phlip
Philippa Cowderoy wrote:
> Taken with excessive formalism, pretty painfully. If you've got a language
> with a good type system, lighter weight versions (ie skipping half the
> working, not showing any of it) can be pretty effective - but then it also
> starts to sound a lot like an effective coder simply doing what they do.
> Or maybe I have a biased sample due to the number of FP folks I talk to?
Ah, the old "my proof can beat up your proof" methodology.
FP is used in life-critical systems. Here's a great post by one of its
thought leaders:
Rod Chapman wrote:
> John Roth wrote:
> > The thing to understand is that the latter facility is critical:
> > to do TDD, the language system (including the prover)
> > needs to be able to cycle within a few seconds.
>
> Absolutely. If I can be so bold as to quote myself:
>
> "The efficiency of any such analysis is crucial. If you want someone to
use
> a tool in preference to compiling and testing, then the tool must be fast!
> Again, the intractable nature of many static analysis problems on
> traditional languages has limited the adoption of these tools."
>
> Basically, SPARK is very carefully designed so that all the analyses
> are sound and computable in Polynomial time. The Examiner can do its
> static semantics and information flow analysis very, very fast - a couple
> of seconds for a single body is typical. Analysis of the entire
> Examiner (about 60kloc) takes about 2 minutes on a modest PC.
>
> Theorem proving is not quite so fast, but not too shabby either -
> we have a 20kloc system here that generates about 18000 proofs.
> The prover takes about 43 minutes to churn through those on a
dual-processor
> P4 box. This brings "regression proof" into scope for reasonable size
> embedded projects at least in a lunch-time or overnight time-frame.
>
> - Rod Chapman, SPARK Team, Praxis Critical Systems
Notice Rod's team tuned their system for rapid turnaround.
--
Phlip
http://industrialxp.org/community/bin/view/Main/TestFirstUserInterfaces
Philippa Cowderoy
> Do any game projects use Formal Proofs?
>
I know a couple of folks who've bashed that out at the design stage,
though IIRC they tend not to prove the final code. Generally this is when
they've been working out something that's mathematically intense.
> Philippa Cowderoy wrote:
>
> > Taken with excessive formalism, pretty painfully. If you've got a language
> > with a good type system, lighter weight versions (ie skipping half the
> > working, not showing any of it) can be pretty effective - but then it also
> > starts to sound a lot like an effective coder simply doing what they do.
> > Or maybe I have a biased sample due to the number of FP folks I talk to?
>
> Ah, the old "my proof can beat up your proof" methodology.
>
> FP is used in life-critical systems. Here's a great post by one of its
> thought leaders:
>
Believe it or not I follow an amount of this stuff - it's pretty much my
subject at uni, though I'm still nominally an undergrad. There're ways and
means, certainly - dependant types are starting to look seriously
interesting (have a play with something called Epigram), and it's
reasonably easy to catch a shockingly large number of errors at
compile-time with some of the more modern derivatives of the
Hindley-Milner type system (yes, I'm working on one bit-by-bit).
Christer Ericson
phli...@yahoo.com says...
>
>[...]
> > > not so good that other publishers can't make better ones otherwise
> > > they'll abandon us like they have Nintendo."
> >
> > More nonsense.
>
> I thought Tom worked at Sony. Oh, sorry - I get it. You mean that Sony's
> corporate policy is nonsense. If that's true, I agree.
You're being a retard. Let me spell it out for you. (a) Sony
does not have any such policy. (b) That was a poor attempt
at sarcasm from Tom.
> Game shops don't need more coders capable of transforming matrices all
> night. They might need a few more guys familiar with sound software
> engineering practices.
For the Nth time, the software engineering problems you are
harping on about are non-problems. The primary problems are
production related, NOT software related.
> Apparently, anyone who says "you can write software without the neurotic
> cycle of creating bugs and then fixing them" must be a looney zealot around
> here. What a culture!
Like your more famous loon counterpart you are tilting at
windmills; you're stuck up on a non-issue.
Phlip
> You're being a retard. Let me spell it out for you. (a) Sony
> does not have any such policy. (b) That was a poor attempt
> at sarcasm from Tom.
Okay. At the risk of pulling this discussion out of the recess playground
level, may I ask what is Sony's general software development methodology?
> > Apparently, anyone who says "you can write software without the neurotic
> > cycle of creating bugs and then fixing them" must be a looney zealot
around
> > here. What a culture!
>
> Like your more famous loon counterpart you are tilting at
> windmills; you're stuck up on a non-issue.
Who is that? And what issue do you think I was discussing?
(Apologies for losing track, there have been several of them so far!)
--
Phlip
http://industrialxp.org/community/bin/view/Main/TestFirstUserInterfaces
Christer Ericson
phli...@yahoo.com says...
> [...]
> > You're being a retard. Let me spell it out for you. (a) Sony
> > does not have any such policy. (b) That was a poor attempt
> > at sarcasm from Tom.
>
> Okay. At the risk of pulling this discussion out of the recess playground
> level, may I ask what is Sony's general software development methodology?
There is no such thing as "Sony's general software development
methodology".
> > Like your more famous loon counterpart you are tilting at
> > windmills; you're stuck up on a non-issue.
>
> Who is that? And what issue do you think I was discussing?
Don Quixote. You were yet again implying that game developers have
a problem with rampant bug counts and that agile methodologies
(and perhaps XP and TDD in particular) would rectify those
problems.
However, as I'm now repeating for the N^Nth time, the accounts
of rampant bug counts are highly exaggerated. The majority of
game development problems are production issues, NOT software
engineering issues. Why do you have such a hard time getting
this?
In other words, your favorite newfangled software methodology
of the day does nothing to address what are the real problems
that we have in game development.
Thus, you appearing in these newsgroups behaving like the new
messiah has met with the obvious response, and things will
continue that way until you get a clue or go away.
Phlip
> There is no such thing as "Sony's general software development
> methodology".
Because computers don't receive mental telepathy very well yet, you can't
get software without implementing it. Everyone who writes software follows
some kind of pattern or technique.
> > > Like your more famous loon counterpart you are tilting at
> > > windmills; you're stuck up on a non-issue.
> >
> > Who is that? And what issue do you think I was discussing?
>
> Don Quixote. You were yet again implying that game developers have
> a problem with rampant bug counts and that agile methodologies
> (and perhaps XP and TDD in particular) would rectify those
> problems.
All software has a problem with high bug rates. If game programming doesn't,
then maybe I _do_ need to learn more about what people are doing!
> However, as I'm now repeating for the N^Nth time, the accounts
> of rampant bug counts are highly exaggerated. The majority of
> game development problems are production issues, NOT software
> engineering issues. Why do you have such a hard time getting
> this?
Because I never saw a very large scale project, with multiple Software
Product Lines, and with a long build process, that did not have delays and
process waste caused by debugging. Hey, what can I say? It's a windmill that
looks a _lot_ like a giant!
(Please also keep in mind I'm also reluctant to dish on my employer. Our
loyalty should not be taken as obfuscation.)
> In other words, your favorite newfangled software methodology
> of the day does nothing to address what are the real problems
> that we have in game development.
"Newfangled"? TDD was used for the software on the Gemini missions.
I have heard that Scrum is useful for aligning the developers and the
producers. I don't see how it could hurt.
I have also seen first hand how tuning a build process, and switching
debugging to bug-prevention, can stabilize extraordinarily large projects,
reducing the turnaround time. This can't but help relations with producers.
At one gig, when the linguists generated new content, I could build it, test
it, and wrap it up into an MSI and deliver it within a few minutes. The
build scripts formerly required several days of mucking. These efficiencies
gave producers more freedom in selecting short-term goals; more steering.
I'm not saying that agile techniques will automatically shorten a build
process. I'm saying that as it shortens, producers get feedback from their
own decisions faster. When that feedback is very slow, process waste will
_always_ occur, but it may take many forms.
(You may want to open the book /The Game Asset Pipeline/ by Ben Carter, turn
to page 6, and read the last sentence of the subsection "The Game Asset
Pipeline" for a second opinion here...;)
--
Phlip
http://industrialxp.org/community/bin/view/Main/TestFirstUserInterfaces
WTH
Phlip <phli...@yahoo.com> loquated like no one had ever loquated before
with:
> Christer Ericson wrote:
>
>> There is no such thing as "Sony's general software development
>> methodology".
>
> Because computers don't receive mental telepathy very well yet, you
> can't get software without implementing it. Everyone who writes
> software follows some kind of pattern or technique.
I usually write of your more stupid responses to pedagogical rhetoric, but
I'm starting to think that you really do not understand what Christer meant
by *** There is no such thing as "Sony's general software development
methodology". ***
You're exhibiting one of two potential problems. Please let us know which
one applies. First, you honestly don't understand that "no such thing as
'Sony's general software development methodology" means there's no establish
corpus of processes for every game development team at Sony. Second, you're
a typical evangelizing zealot who constantly deflects counter-points and
disputations with feigned confusion and/or itentional misinterpretation
(i.e. as above where you ridiculously assume that because there's no GENERAL
software development methodology that there would, de facto, then be no
methodologies used by anyone if there wasn't a 'general' one.)
Which of those two people would you like to be? Stupid, or an ass?
Personally, I think you're just being an ass.
> All software has a problem with high bug rates. If game programming
> doesn't, then maybe I _do_ need to learn more about what people are
> doing!
Maybe you should define high bug rates first, and software second.
Personally, I've worked on several projects with VERY low bug rates;
however, it always seems to follow the philosophy of fast, cheap, good:
Pick any two.
> Because I never saw a very large scale project, with multiple Software
> Product Lines, and with a long build process, that did not have
> delays and process waste caused by debugging. Hey, what can I say?
> It's a windmill that looks a _lot_ like a giant!
What a strawman... You're in the game development ng evangelizing for XP,
and using a non game development example for your pontificating. I wonder
why people (like myself) find you actually detrimental to the adoption of
'agile' methodologies.
> (Please also keep in mind I'm also reluctant to dish on my employer.
> Our loyalty should not be taken as obfuscation.)
Who cares, you have never done any professional game development, why would
'dishing' on your current employer convince people that you know what is
wrong with game development as a whole?
>> In other words, your favorite newfangled software methodology
>> of the day does nothing to address what are the real problems
>> that we have in game development.
>
> "Newfangled"? TDD was used for the software on the Gemini missions.
I'm sorry, is the only thing you're talking about now TDD? I mean, you've
been throwing around the word 'agile' for a while now. Most people who know
anything about XP know that it is generally just a mix of practices from
other methodologies.
> I have also seen first hand how tuning a build process, and switching
> debugging to bug-prevention, can stabilize extraordinarily large
> projects, reducing the turnaround time. This can't but help relations
> with producers. At one gig, when the linguists generated new content,
> I could build it, test it, and wrap it up into an MSI and deliver it
> within a few minutes. The build scripts formerly required several
> days of mucking. These efficiencies gave producers more freedom in
> selecting short-term goals; more steering.
Funny, now 'tuning a build process' only happens when embracing 'agile'
methods. My build process currently (23 projects, roughly 600-700k loc)
takes 0 minutes and 0 seconds to add a foreign language to our product.
Beat that.
> I'm not saying that agile techniques will automatically shorten a
> build process. I'm saying that as it shortens, producers get feedback
> from their own decisions faster. When that feedback is very slow,
> process waste will _always_ occur, but it may take many forms.
This is where the fundamental purpose of XP comes to light. XP is most
valuable as an emergency methodology (although parts of it are useful in any
scenario if they appeal to you), in order to accomodate for shitty
management. XP grew because product managers wouldn't make decisions, sales
personnel were involved beyond the initial spec, because development began
without a decent design, because feature creep became problematics, yadda
yadda yadda. All the problems that arise from everybody and their dog
wanting to make software w/o learning how to do it (on the non-technical
side most importantly.)
> (You may want to open the book /The Game Asset Pipeline/ by Ben
> Carter, turn to page 6, and read the last sentence of the subsection
> "The Game Asset Pipeline" for a second opinion here...;)
A second opinion which is talking about production issues? Well, now you're
making Christer's points for him.
WTH
Philippa Cowderoy
> All software has a problem with high bug rates.
Horseshit. Unless, of course, you're going to keep shifting the goals on
what counts as high. Some of us have decent techniques for avoiding bugs
already.
WTH
loquated before with:
> On Tue, 23 Nov 2004, Phlip wrote:
Well, surely you must be fully embracing 'agile' methodologies to accomplish
such a wondrous feat...
WTH
Phlip
> Phlip wrote:
>
> > All software has a problem with high bug rates.
>
> Horseshit. Unless, of course, you're going to keep shifting the goals on
> what counts as high. Some of us have decent techniques for avoiding bugs
> already.
I saw an XP team that had finished a moderately complex visualization tool
for gas chromatographic tools in 6 months once. They had 2 defects on their
board. Teams working like that have the option to never use their debugger.
I am not going to shift any goals on what counts as too high.
--
Phlip
http://industrialxp.org/community/bin/view/Main/TestFirstUserInterfaces
Tom Plunket
> > > Based on the "skills" you've displayed around here, don't
> > > get your hopes up too high.
> >
> > Ah, so you guys don't want an test framework tuned to host, process, edit,
> > and browse the results of thousands of acceptance tests.
>
> No, not really. As several people have already pointed out to
> you, that would be a mostly misguided effort. We don't have a
> problem with large code-bug counts. The problems are predominantly
> non-code related.
>
> Your XP zealotry is making you bark up the wrong tree.
...and here I was thinking that one could automate tests to catch
asset problems also. Hmm, weird, I wonder what that pipeline that we
have in place is actually doing...
(FWIW, XP calls for acceptance testing but acceptance testing does not
require XP. The fact that we've got a huge pile of acceptance tests
that fire up the game and do automated things in spite of the fact
that we don't do XP would seem to indicate a few things [one of which
is likely the fact that it was a lot harder bolting them on after the
fact].)
Again I wish to remind people that when flaming someone, it's good to
actually be flaming them on something that they actually said rather
than something you wish they said. In this case, Christer flames
Phlip on both XP and the testing of software, when in fact Phlip said
not two words about XP and didn't specify what these automated tests
were to be testing.
Good luck!
-tom!
Philippa Cowderoy
Sort of. More like I'm hacking on single-coder projects and have an
aversion to building on bug-ridden code though :-)
Rainer Deyke
> ...and here I was thinking that one could automate tests
> to catch
> asset problems also.
I'd love to have an automated test that can find problems like "this level
is too hard", "this animation looks ugly", and "this dialogue doesn't make
sense if the player already completed task A before talking to the npc".
However, I don't see any reasonable way to automate these tests. That
doesn't mean that automated tests arem't useful for finding other types of
bugs.
--
Rainer Deyke - rai...@eldwood.com - http://eldwood.com
Phlip
> Tom Plunket wrote:
> > ...and here I was thinking that one could automate tests
> > to catch
> > asset problems also.
>
> I'd love to have an automated test that can find problems like "this level
> is too hard", "this animation looks ugly", and "this dialogue doesn't make
> sense if the player already completed task A before talking to the npc".
> However, I don't see any reasonable way to automate these tests. That
> doesn't mean that automated tests arem't useful for finding other types of
> bugs.
You could hire a test engineer, then excuse them from the bug-chasing
everyone else does.
This is a very simple formula. As the cost of writing a test remains higher
than just fixing the bug, nobody will capture bugs with tests. But as you
invest in test infrastructure, the cost goes down below the threshold. Then
writing the test to capture the bug is as fast as debugging (and works very
well as a platform for debugging). Then each bug leads to more tests, and
the project turns around.
The goal is not to avoid debugging; the goal is to avoid the long bug-hunts,
and the bugs that you can't fix because "everyone else uses this code and I
might break all of them".
I suspect that large game projects have side-effects, cheats, trace
statements, manual test levels, and extra scripts, all waiting to be mined
for testage. To get down to "this level is too hard", until you have the
full-featured AI bot to run the level as the player, you can make due with
testing side-effects. To test "this animation is ugly", you can record the
animation and upload it to a wiki. Browsing a few wiki pages might be more
efficient than running a bunch of manual tests.
--
Phlip
http://industrialxp.org/community/bin/view/Main/TestFirstUserInterfaces
Ronald E Jeffries
Great! Please tell us what your techniques are, and what your defect
rates are. I'd also be interested in the time delay between defect
injection and discovery (or prevention). And if you have data from
before you used the technique you use now, those would be fascinating.
Thanks,
--
Ron Jeffries
www.XProgramming.com
I'm giving the best advice I have. You get to decide if it's true for you.
Ronald E Jeffries
>This is where the fundamental purpose of XP comes to light. XP is most
>valuable as an emergency methodology (although parts of it are useful in any
>scenario if they appeal to you), in order to accomodate for shitty
>management. XP grew because product managers wouldn't make decisions, sales
>personnel were involved beyond the initial spec, because development began
>without a decent design, because feature creep became problematics, yadda
>yadda yadda. All the problems that arise from everybody and their dog
>wanting to make software w/o learning how to do it (on the non-technical
>side most importantly.)
This is simply not the case.
Rainer Deyke
> This is a very simple formula. As the cost of writing a
> test remains higher than just fixing the bug, nobody will
> capture bugs with tests.
In the cases I decribed, the cost of writing a fully automated test is
higher than the cost of the entire project (including bug fixing).
Phlip
> Phlip wrote:
> > This is a very simple formula. As the cost of writing a
> > test remains higher than just fixing the bug, nobody will
> > capture bugs with tests.
>
> In the cases I decribed, the cost of writing a fully automated test is
> higher than the cost of the entire project (including bug fixing).
Now read the second half of my post.
--
Phlip
http://industrialxp.org/community/bin/view/Main/TestFirstUserInterfaces
Tom Plunket
> > ...and here I was thinking that one could automate tests
> > to catch asset problems also.
>
> I'd love to have an automated test that can find problems like "this level
> is too hard", "this animation looks ugly", and "this dialogue doesn't make
> sense if the player already completed task A before talking to the npc".
> However, I don't see any reasonable way to automate these tests. That
> doesn't mean that automated tests arem't useful for finding other types of
> bugs.
Exactly. Could it be that the resistance to automated testing in
games comes down to something as simple as, "we can't figure out how
to automate the testing of everything, so we won't bother automating
any tests."
It's not that nobody is testing, of course. The Sims 2 had a lot of
automated tests (or so I'm told), and the game I'm working on now has
a few that test various things that used to break a lot. The fact
that the tests are automated means if those things ever break again,
we know in the morning when we come in. (The tests themselves run
right after the nightly build finishes.)
So that nobody gets on me for saying "TDD", these tests catch data
errors as frequently as code bugs. On the other hand, I think TDD for
higher-level game logic would be interesting. Phlip once proposed to
me the idea that designers could script a sequence the way they wanted
to script it, but before the scripting support was in place. In this
way, the scripts coupled with what the designers say they want could
lead development much like TDD does for "pure code" development.
(Obviously there's give and take; if something in a designer's script
needs adjustment to be efficient or work at all, you make those
changes with the designer present.) Hell, you could even do something
akin to pair programming with one designer sitting at a desk with one
gameplay programmer.
Oh well, all you who "know better" won't bother with it, but it is
interesting for me to consider and is likely something we'll try in
the near future...
-tom!
Tom Plunket
> I'd love to have an automated test that can find problems like "this level
> is too hard", "this animation looks ugly", and "this dialogue doesn't make
> sense if the player already completed task A before talking to the npc".
How about these:
For testing if the level is too hard, hook an AI up to your main
character and lightly drag that character through the level on a path.
Verify that the full level is completed X% of the time.
For testing to see if an animation looks ugly, one thing you could do
is inspect the continuity of their curves (including/especially at the
loop points). If anything is not C1 continuous there'll be obvious
pops to the user. Anything that isn't C2 continuous would probably
appear to have pops, or may have 'robotic' looking movements, so the
test would probably want to verify that all animations have C3
continuity. If you wanted to exclude some animations from these
restrictions, that should be easy to do as well.
For testing if dialog doesn't make sense... That's a tough one,
although there may be a way to author tools such that the designers
putting dialog trees together could more easily run through weirdo
permutations to make sure the flow is ok.
Not everything is testable in a code-only way. For instance, you
can't test code to turn "rumble" on in your joystick unless you build
some sort of hardware that wraps the joystick and plugs into the
computer somehow. Probably not worth it. On the other hand, as I've
done more and more TDD, I've discovered ways to test things that I
previously thought untestable.
-tom!
Phlip
> Sort of. More like I'm hacking on single-coder projects and have an
> aversion to building on bug-ridden code though :-)
[I'm not saying you did, but] carefully planning a design before coding it
has better odds of working for games than business coding. Here's why:
In business coding, your customers ask for what they /want/, and you try to
give them what they /need/. What they want is nebulous, but what they need
is exact. If you listen to what they want, plan a design, implement it, and
deliver it, it will differ from their needs. This leads to the dire
"changing requirements" problem that bedevils so much planned software
efforts. Studies have shown that the larger a business project, the longer
it goes without delivering, the more likely it will fail. Carefully
collecting requirements ("wants"), up-front, _increases_ the odds of
failure.
Planned designs generally resist change. Changing things that have never
been changed before adds risk. So planned designs turn a corner to
"code-and-fix", which is all the hacking and debugging that planning the
design tried to avoid.
In game programming, customers have serious /wants/, but not exact /needs/.
This changes everything. I suspect that while some shops still use
code-and-fix all night, some have successfully moved to a "phasist"
approach, where they carefully design and implement an engine, while
targeting it to the art and scripts for some game levels. The final game
play combines up-front play design, up-front engine design, and all the
features and interactions that simply emerge during development.
(I thought of this today when my character jumped on top of a landscape
feature he wasn't supposed to, the enemies couldn't get to him, and he
simply picked each off one by one. Until he ran out of ammo.)
Replacing excessive planning with excessive testing helps business
developers rapidly and safely target customer needs. This is not sophistry;
it's a deliberate result of techniques like early delivery, frequent
iterations, and implementing features in order of business priority.
I am curious what would happen if a game project replaced early design
planning with excessive testing. Ideals like "write a test that a level is
balanced" make for great speculation, and will probably happen as we learn
to design by testing. An easier goal is simply the Lean Manufacturing ideal
of delaying irreversible decisions until gathering the most information. If
a game started with themes, but not level designs, and if designers delayed
building levels and plots until an engine matured, they could then design
from a scripting console, with some of the art installed. This effort would
allow gameplay design to benefit from immediate feedback of the results of
the interactions of its finished engine components.
So my experiment today, jumping on top of a landscape feature and using it
as a sniper's nest, could have lead to an inspiration about the interplay of
the physics layer and the AI layer, and this could have lead to an amusing
optional twist in this level, instead of just a curious dead-spot.
--
Phlip
http://industrialxp.org/community/bin/view/Main/TestFirstUserInterfaces
Christer Ericson
phli...@yahoo.com says...
> > There is no such thing as "Sony's general software development
> > methodology".
>
> Because computers don't receive mental telepathy very well yet, you can't
> get software without implementing it. Everyone who writes software follows
> some kind of pattern or technique.
Again you're being a retard. Do you understand the difference
between a corporation-adopted methodology and personal
coding style? I commented on the former, and you reply
on the latter.
Since you're unable to distinguish two so simple concepts,
have you ever considered that perhaps you've screwed up your
ideas about game development and agile methodologies too?
It would probably be a good idea if you learned the meaning
of the word "methodology" before you continue ranting.
Also, WTH supplied you with such a good reply, that I feel
compelled to repeat it here (in case you missed it):
#You're exhibiting one of two potential problems. Please let us
#know which one applies. First, you honestly don't understand
#that "no such thing as 'Sony's general software development
#methodology" means there's no establish corpus of processes for
#every game development team at Sony. Second, you're a typical
#evangelizing zealot who constantly deflects counter-points and
#disputations with feigned confusion and/or itentional
#misinterpretation (i.e. as above where you ridiculously assume
#that because there's no GENERAL software development methodology
#that there would, de facto, then be no methodologies used by
#anyone if there wasn't a 'general' one.)
#
#Which of those two people would you like to be? Stupid, or an
#ass? Personally, I think you're just being an ass.
> > Don Quixote. You were yet again implying that game developers have
> > a problem with rampant bug counts and that agile methodologies
> > (and perhaps XP and TDD in particular) would rectify those
> > problems.
>
> All software has a problem with high bug rates. If game programming doesn't,
> then maybe I _do_ need to learn more about what people are doing!
How does "All software has a problem with high bug rates" rhyme
with "I saw an XP team that had finished a moderately complex
visualization tool for gas chromatographic tools in 6 months once.
They had 2 defects on their board."
Do you consider the people in your example to have high bug
rates too? (I assume not.) Then, clearly you lie when you say
all software has a problem with high bug rates.
It would help your case if you didn't make contradictory
statements.
> > In other words, your favorite newfangled software methodology
> > of the day does nothing to address what are the real problems
> > that we have in game development.
>
> "Newfangled"? TDD was used for the software on the Gemini missions.
TDD now? It was XP before, when it wasn't agile. Do you even know
what you stand for? Did I say "newfangled?" Sorry, I guess I meant
"dayfly." Is "newfangled" the only objection you have to my statement?
I must assume you agree with it otherwise then.
> I'm not saying that agile techniques will automatically shorten a build
> process.
So why are you ranting about applying it to games then?
> I'm saying that as it shortens, producers get feedback from their
> own decisions faster. When that feedback is very slow, process waste will
> _always_ occur, but it may take many forms.
Why are you stating the obvious?
> (You may want to open the book /The Game Asset Pipeline/ by Ben Carter, turn
> to page 6, and read the last sentence of the subsection "The Game Asset
> Pipeline" for a second opinion here...;)
A second opinion of the obvious?
Establishing a good asset pipeline is completely orthogonal to
what software development methodology you're using.
As WTH said, you are making my point for me!
Christer Ericson
Tools and technology lead