Game development in Clojure

1,430 views
Skip to first unread message

Mike Anderson

unread,
Aug 13, 2010, 9:51:41 AM8/13/10
to Clojure
Hello all,

I've recently been working on a game development project in Clojure
which is now starting to bear fruit. I thought people here might be
interested, and that it would be worthwhile to share some experiences
and perspectives.

The project is a steampunk-themed strategy game, and a playable in-
development version is available here:

http://mikera.net/ironclad/


Overall, I think Clojure is a fantastic language for game development.
Some thought on what makes it really compelling:

1. Immutable data structures work surprisingly well for games. I
represent the entire game state with a big defrecord, and the main
engine loop is basically:
a) Get commands from player / AI (e.g. "Move unit from A to B")
b) Convert each command into set of atomic updates ("Remove unit
from A", "Add unit to B")
c) Apply updates sequentially to the game state
d) Trigger any feedback to player e.g. animations, sounds

2. Concurrency support has been very helpful. With immutable game
state, it has been trivial to separate the game engine from the
renderer from the AI calculations - each effectively gets to operate
on its own "snapshot" of the entire game state. This can be a big
headache in many game engines that have to mediate access to a single,
mutable game state. I can also see even greater benefits when I
finally start adding some multi-player features.

3. High level, functional programming works great for rapid, iterative
and dynamic development. Of all the languages I've used, Clojure has
probably been quickest in terms of time required to add a given new
piece of functionality. For example, the map generator code probably
only took about 30 minutes to write, pretty good for a fractal
landscape generation system!

4. The Java interoperability support is fantastic - definitely counts
as one of the "killer features" of Clojure. I've been able to reuse a
lot of Java code libraries, both my own and standard libraries such as
Swing. As an example, I have a well optimised immutable persistent 2D
grid data structure in Java (basically a very efficient, spatially
partitioned map from (int,int) to Object) that I have been able to use
pretty much seamlessly in Clojure thanks to extend-protocol and
similar.


Some interesting observations / discoveries / surprises in the
process:

1. The game worked first time on Mac and Linux, despite having been
tested exclusively on Windows. Great kudos to to both the Java
platform and the Clojure libraries!

2. Reflection is *really slow*. After getting 20-100x speedups from
eliminating reflection warnings in the AI code, I now treat any
reflection warning in code other than one-time setup code as a bug to
be fixed immediately.


Finally, here are some things I think could be great improvements to
Clojure in the future from a game developer's perspective:

1. Better support for primitive parameters / return values / unchecked
arithmetic - in performance sensitive code, these are pretty much
essential. In a few cases in the graphics and AI code, I've had to
drop back to Java to get the performance I need.

2. It would be great to reduce the amount of memory allocations. Yes,
I know memory is plentiful and GC is very cheap, but it's still not as
cheap as stack allocation and any noticeable GC pauses are not good
for the player experience in interactive games. For this reason, I
find myself using reduce and indexed loops a lot more than I guess
would normally be idiomatic, and conversely tend to avoid some of the
lazy constructs and functions that generate sequences. While Clojure
is great for a strategy game, I'd probably hesitate to use it for a
real-time 3D game.

3. I think it would be great to have better support for circular
references - perhaps a two-pass compile? The reason this is
particularly acute in game development is that different subsystems
have quite a lot of inter-dependencies. AI evaluation system needs to
understand game state/engine so it can evaluate a position. Game state/
engine needs to understand units so it can manipulate them. Units need
to understand AI evaluation system so they can decide which actions to
take...... obviously it's possible to work around all this, but it's a
major pain, adds complexity and means that you need to structure code
to manage dependencies rather than in logical modules (which would be
easier to manage and maintain!)


Would love to hear thoughts, and particularly any other experiences
people have had in using Clojure for game development!

Mike.

Mac

unread,
Aug 13, 2010, 11:45:47 AM8/13/10
to Clojure


On Aug 13, 9:51 am, Mike Anderson <mike.r.anderson...@gmail.com>
wrote:
> 2. It would be great to reduce the amount of memory allocations. Yes,
> I know memory is plentiful and GC is very cheap, but it's still not as
> cheap as stack allocation and any noticeable GC pauses are not good
> for the player experience in interactive games. For this reason, I
> find myself using reduce and indexed loops a lot more than I guess
> would normally be idiomatic, and conversely tend to avoid some of the
> lazy constructs and functions that generate sequences. While Clojure
> is great for a strategy game, I'd probably hesitate to use it for a
> real-time 3D game.

There really is no way around this. We preallocate space for all our
OpenGL vertex, as well as all the FSM for the AI engine. This is
something we do even in our C++ code. We avoid creating objects
whenever possible, and preallocate at init time. This is especially
true in devices with smaller footprint like console/iphone/android,
etc.

Having said that, for prototyping OpenGL realtime 3d stuff on our
desktop,
clojure has been ok.

> 3. I think it would be great to have better support for circular
> references - perhaps a two-pass compile? The reason this is
> particularly acute in game development is that different subsystems
> have quite a lot of inter-dependencies. AI evaluation system needs to
> understand game state/engine so it can evaluate a position. Game state/
> engine needs to understand units so it can manipulate them. Units need
> to understand AI evaluation system so they can decide which actions to
> take...... obviously it's possible to work around all this, but it's a
> major pain, adds complexity and means that you need to structure code
> to manage dependencies rather than in logical modules (which would be
> easier to manage and maintain!)

I'm not sure what you mean by this, can you expand on this?

Brian Carper

unread,
Aug 13, 2010, 2:06:40 PM8/13/10
to Clojure
On Aug 13, 6:51 am, Mike Anderson <mike.r.anderson...@gmail.com>
wrote:
> Hello all,
>
> I've recently been working on a game development project in Clojure
> which is now starting to bear fruit. I thought people here might be
> interested, and that it would be worthwhile to share some experiences
> and perspectives.
>

Looks great. Thanks for sharing your experiences.

Do you plan to share the source code? Any reason you went with Swing
instead of OpenGL?

--Brian

Wilson MacGyver

unread,
Aug 13, 2010, 2:09:41 PM8/13/10
to clo...@googlegroups.com
I assumed he didn't use OpenGL because it's a 2d tile game?
Using OpenGL for 2d or 2.5d (isometric) is really only a good idea
if you can assume the target has hardware OpenGL acceleration.

Even then you may not want to do that, due to battery concerns.

On Fri, Aug 13, 2010 at 2:06 PM, Brian Carper <brian...@gmail.com> wrote:
> Do you plan to share the source code?  Any reason you went with Swing
> instead of OpenGL?
>


--
Omnem crede diem tibi diluxisse supremum.

Nicolas Oury

unread,
Aug 13, 2010, 2:16:16 PM8/13/10
to clo...@googlegroups.com
On Fri, Aug 13, 2010 at 2:51 PM, Mike Anderson
<mike.r.an...@gmail.com> wrote:
> 2. It would be great to reduce the amount of memory allocations. Yes,
> I know memory is plentiful and GC is very cheap, but it's still not as
> cheap as stack allocation and any noticeable GC pauses are not good
> for the player experience in interactive games. For this reason, I
> find myself using reduce and indexed loops a lot more than I guess
> would normally be idiomatic, and conversely tend to avoid some of the
> lazy constructs and functions that generate sequences. While Clojure
> is great for a strategy game, I'd probably hesitate to use it for a
> real-time 3D game.
>

This can be made a bit better by turning Escape Analysis on? Have you
tried that?

The G1 collector is supposed to have lower latency. Have you tried it?

http://research.sun.com/jtech/pubs/04-g1-paper-ismm.pdf

Peter Schuller

unread,
Aug 13, 2010, 2:56:20 PM8/13/10
to clo...@googlegroups.com
> This can be made a bit better by turning Escape Analysis on? Have you
> tried that?
>
> The G1 collector is supposed to have lower latency. Have you tried it?
>
> http://research.sun.com/jtech/pubs/04-g1-paper-ismm.pdf

(GC choice/tuning is a loooong story, but some select suggestions follow)

First off, make sure you're at least using CMS
(-XX:+UseConcMarkSweepGC) or G1 at all to begin with - the default
collector is the throughput collector which will always do full GC:s
of the old generation so that you are guaranteed to get pause times
that scale linearly with heap size (unless you're generating *zero*
non-young garbage).

With CMS, you can force shorter young generation pauses by limiting
the size of the young generation (usually at the cost of more GC work
in total; i.e., less throughput, because the older generation is more
expensive to collect). I'm not sure at all how much control over VM
options you have with Java WebStart though, but in general that would
be my suggestion if the young generation pauses are too long.
-XX:NewSize=SIZE and -XX:MaxNewSize=SIZE should be your friend here
(and -XX:+UseParNewGC though I think that is always the default on
modern VM:s anyway on multi-core machines). I'm making the assumption
here that the allocation pattern is not such that the concurrent
mark/sweep:s are the problem.

With G1, you can directly specify target pause times that G1 tends to
be pretty good at maintaining in my experience. A recurring issue for
me and at least some others seems to be that your application may have
a workload that disallows certain memory regions to be collected
within the pause time goal; this can lead to a build-up of
uncollectable regions that may eventually cause a fallback to a full
stop-the-world GC. The likelyhood of this happening is higher the
lower the pause time goal you specify. An starter command line to play
with (preferably with jdk 1.7) might be:

-XX:+UnlockExperimentalVMOptions -XX:+UseG1GC
-XX:MaxGCPauseMillis=15 -XX:GCPauseIntervalMillis=20

--
/ Peter Schuller

Alan

unread,
Aug 13, 2010, 12:33:10 PM8/13/10
to Clojure
Funny you should mention this - I was about to post a question about
my own game when I saw your article. My issue is, I assume someone has
written minimax and/or alpha-beta pruning in Clojure (or a java
library that's easy to interop with). My case is slightly different in
that the turn order is not fixes - sometimes player X goes twice in a
row - but it ought to be pretty simple to plug into a standard AI
library. Does anyone know where I can find such a thing?

On Aug 13, 6:51 am, Mike Anderson <mike.r.anderson...@gmail.com>
wrote:

Wilson MacGyver

unread,
Aug 13, 2010, 4:01:36 PM8/13/10
to clo...@googlegroups.com
Paradigms of artificial intelligence programming: case studies in Common LISP
By Peter Norvig has a full chapter on this (ch18),
complete with code in Common LISP.

his "Artificial Intelligence: A Modern Approach" is also a good book
if you are interested in game AI. 3rd edition just came out recently

> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com
> Note that posts from new members are moderated - please be patient with your first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en

Brian Carper

unread,
Aug 13, 2010, 4:51:30 PM8/13/10
to Clojure
On Aug 13, 11:09 am, Wilson MacGyver <wmacgy...@gmail.com> wrote:
> I assumed he didn't use OpenGL because it's a 2d tile game?
> Using OpenGL for 2d or 2.5d (isometric) is really only a good idea
> if you can assume the target has hardware OpenGL acceleration.
>
> Even then you may not want to do that, due to battery concerns.
>

But there are some good OpenGL 2D game engines for Java, e.g.
Slick[1]. There's also Penumbra[2] which nicely wraps LWJGL for
Clojure. I ask this mostly because I'm making my own 2D game and
somewhat torn between Java2D and OpenGL. Penumbra is pretty solid in
my experience.

What do you mean by "battery concerns"?

--Brian

[1] http://slick.cokeandcode.com/
[2] http://github.com/ztellman/penumbra

Wilson MacGyver

unread,
Aug 13, 2010, 5:13:24 PM8/13/10
to clo...@googlegroups.com
On Fri, Aug 13, 2010 at 4:51 PM, Brian Carper <brian...@gmail.com> wrote:
> But there are some good OpenGL 2D game engines for Java, e.g.
> Slick[1].  There's also Penumbra[2] which nicely wraps LWJGL for
> Clojure.  I ask this mostly because I'm making my own 2D game and
> somewhat torn between Java2D and OpenGL.  Penumbra is pretty solid in
> my experience.

We don't use Penumbra. On Java, we use JOGL, and on
other targets we use pure OpenGL or OpenGL ES API
for both compatibility and performance reasons. We have to ensure
whatever OpenGL stuff we do, will work everywhere.

>
> What do you mean by "battery concerns"?

Laptops, Mobile devices and such. If you use OpenGL on a laptop,
it will use the GPU assuming it's hardware accelerated. GPU is a big
battery drain. On mobile devices such as iPhone and Android, it's
even more so. Using OpenGL unnecessarily causes battery drain
on your hardware.

Plus, if you use OpenGL, and your target is using those Intel Extreme Graphics,
where the only thing it's Extreme at, is Extremely slow. :) You end up gain
nothing but lose lots.

BerlinBrown

unread,
Aug 13, 2010, 7:38:25 PM8/13/10
to Clojure
I played it, it was pretty fun.

I have UI recommendations. I couldn't tell the difference between the
enemy and my units.

I wish maybe you had some quick tips and recommended next actions so I
could get used to how the game works.

On Aug 13, 9:51 am, Mike Anderson <mike.r.anderson...@gmail.com>
wrote:

Eric Lavigne

unread,
Aug 13, 2010, 11:49:22 PM8/13/10
to clo...@googlegroups.com
>> 3. I think it would be great to have better support for circular
>> references - perhaps a two-pass compile? The reason this is
>> particularly acute in game development is that different subsystems
>> have quite a lot of inter-dependencies. AI evaluation system needs to
>> understand game state/engine so it can evaluate a position. Game state/
>> engine needs to understand units so it can manipulate them. Units need
>> to understand AI evaluation system so they can decide which actions to
>> take...... obviously it's possible to work around all this, but it's a
>> major pain, adds complexity and means that you need to structure code
>> to manage dependencies rather than in logical modules (which would be
>> easier to manage and maintain!)
>
> I'm not sure what you mean by this, can you expand on this?

Though I didn't write that paragraph, I have faced the same issue.

Suppose I have two functions in the same file, and one depends on the other:

(defn foo [x] (+ 1 x))
(defn bar [x] (* 2 (foo x)))

I can't switch their order without adding extra forward-declaration
code, which is redundant:

(declare foo)
(defn bar [x] (* 2 (foo x)))
(defn foo [x] (+ 1 x))

This example is just a minor irritation, that I need to make sure all
the functions in a file are in the right order.

A bigger problem is when the two functions are in different
files/packages. Suppose I have files a.clj and b.clj and place some of
my functions in each of those files based on some arbitrary
categorization that makes intuitive sense to me. Then I realize that
some of the functions in a.clj depend on some functions in b.clj,
while some functions in b.clj depend on some functions in a.clj. Can I
still use declare to resolve this circular dependency? Can I even
"require" a.clj from b.clj when I have already "require"d b.clj from
a.clj?

My solution so far has been to make sure that dependencies between
packages are never two-way. I deliberately choose categorizations that
won't result in circular dependencies between packages. I would much
prefer to just refer to whatever function I wish, from whichever
function I wish, and not need to think about which packages I am
allowed to depend on or in what order functions must appear in a file.

Wilson MacGyver

unread,
Aug 14, 2010, 12:13:42 AM8/14/10
to clo...@googlegroups.com
I rarely run into this. The few times I have, I just do

(def g) ;creates a var g that is unbound

(defn f []
(g)) ;ok

(defn g [] ;f will call this
nil)

as shown by Rich at

http://markmail.org/message/vuzvdr4xyxx53hwr#query:+page:1+mid:tzsd3k6tvvc4ahoq+state:results

On Fri, Aug 13, 2010 at 11:49 PM, Eric Lavigne <lavign...@gmail.com> wrote:
> Suppose I have two functions in the same file, and one depends on the other:
>
>   (defn foo [x] (+ 1 x))
>   (defn bar [x] (* 2 (foo x)))
>
> I can't switch their order without adding extra forward-declaration
> code, which is redundant:
>
>   (declare foo)
>   (defn bar [x] (* 2 (foo x)))
>   (defn foo [x] (+ 1 x))
>

--

Eric Lavigne

unread,
Aug 14, 2010, 12:39:19 AM8/14/10
to clo...@googlegroups.com
The (def g) in your example has the same effect as the (declare foo)
in my example.

I discussed two problems. The first problem, which you addressed, was
mostly just a warm-up for discussing a related problem that is more
severe. Where can I put (def g) so that two files can "require" each
other?

This is not a rare problem for me. Like Mike Anderson, I work around
it by putting extra thought into which package-level dependencies I
will allow, which sometimes necessitates creating more or fewer
packages than I otherwise would have created.

Tim Daly

unread,
Aug 14, 2010, 12:49:41 AM8/14/10
to clo...@googlegroups.com
Suppose you make a file containing a
(def foo)
form for every defn in every file and then load that first?
Does that solve the circular reference problem?

Tim Daly

Wilson MacGyver

unread,
Aug 14, 2010, 1:37:43 AM8/14/10
to clo...@googlegroups.com
I realize that. I was pondering why I don't run into the the 2nd problem.

In your code, how many files/name spaces are you creating?
And how many lines of code are in each file? I'm curious how you
organize your code.

Mark Engelberg

unread,
Aug 14, 2010, 3:21:34 AM8/14/10
to clo...@googlegroups.com
On Fri, Aug 13, 2010 at 9:39 PM, Eric Lavigne <lavign...@gmail.com> wrote:
> This is not a rare problem for me. Like Mike Anderson, I work around
> it by putting extra thought into which package-level dependencies I
> will allow, which sometimes necessitates creating more or fewer
> packages than I otherwise would have created.

Me three. I find it to be a *lot* of work to avoid circular dependencies.

Nicolas Oury

unread,
Aug 14, 2010, 3:34:58 AM8/14/10
to clo...@googlegroups.com
I have a similar problem with deftype/defecord

If I need to mutually defined types...

(declare new-a)


(deftype B....

... (new-a)...))

(deftype A)

(defn new-a [] (A.))

Btsai

unread,
Aug 14, 2010, 1:05:32 AM8/14/10
to Clojure
Continuing this train of thought...

1. The "declare" macro may be handy for declaring multiple names at
once.

2. Maybe one could use the functions in clojure.repl or clojure-
contrib.ns-utils to write something that automatically forward
declares everything needed?

On Aug 13, 10:49 pm, Tim Daly <d...@axiom-developer.org> wrote:
> Suppose you make a file containing a
> (def foo)
> form for every defn in every file and then load that first?
> Does that solve the circular reference problem?
>
> Tim Daly
>
>
>
> Eric Lavigne wrote:
> > The (def g) in your example has the same effect as the (declare foo)
> > in my example.
>
> > I discussed two problems. The first problem, which you addressed, was
> > mostly just a warm-up for discussing a related problem that is more
> > severe. Where can I put (def g) so that two files can "require" each
> > other?
>
> > This is not a rare problem for me. Like Mike Anderson, I work around
> > it by putting extra thought into which package-level dependencies I
> > will allow, which sometimes necessitates creating more or fewer
> > packages than I otherwise would have created.
>
> > On Sat, Aug 14, 2010 at 12:13 AM, Wilson MacGyver <wmacgy...@gmail.com> wrote:
>
> >> I rarely run into this. The few times I have, I just do
>
> >> (def g) ;creates a var g that is unbound
>
> >> (defn f []
> >>      (g)) ;ok
>
> >> (defn g [] ;f will call this
> >>      nil)
>
> >> as shown by Rich at
>
> >>http://markmail.org/message/vuzvdr4xyxx53hwr#query:+page:1+mid:tzsd3k...

Eric Lavigne

unread,
Aug 14, 2010, 10:33:26 AM8/14/10
to clo...@googlegroups.com
One example is a contract programming job I did recently, writing
software for identifying radioactive isotopes based on measurements of
their emission spectra. In the end I had 7 Clojure source files (not
including tests). I will show them in such an order that each file
depends only on previous files.

math (11 definitions, 80 lines)
matching (5 definitions, 54 lines)
units (2 definitions, 15 lines)
io (7 definitions, 74 lines)
physics (4 definitions, 28 lines)
metrics (7 definitions, 174 lines)
main (1 definition, 35 lines)

I originally wanted to include the contents of the units file in the
physics file, but then there would be a circular dependency between
physics and io. The io code needs to interpret half-life values in
files that provide information about isotopes, and those half-life
values are meaningless if you don't know the relationship between
seconds, hours, and years. Physics code needs access to io code quite
frequently because, for example, you can't determine what portion of
photons will penetrate a 1/4" thick iron plate without reading iron's
transmission coefficients as a function of photon energy from a file.
It's not a big deal to have a units package, but I don't typically
create a new category for just two definitions that already fit well
in one of the existing categories.

Also, there were two functions in the physics package that I moved to
the math package so that functions in the io package could depend on
them. These functions were responsible for sorting emission lines by
energy and filtering out emission lines that would not be included in
the analysis because their energies are often associated with false
positives. These operations need to be performed on every isotope that
is read from the application's isotope library, and the easiest way to
arrange that is to call them from the function that reads the isotope
library. Again, not a big deal, but I would not ordinarily include
code in the math package that can only operate on emission data - that
belongs in physics.

Tim Daly

unread,
Aug 14, 2010, 1:09:13 PM8/14/10
to clo...@googlegroups.com
grep for defn and write out def forms to a file?

Brian Hurt

unread,
Aug 15, 2010, 10:43:37 AM8/15/10
to clo...@googlegroups.com
On Sat, Aug 14, 2010 at 10:33 AM, Eric Lavigne <lavign...@gmail.com> wrote:

I originally wanted to include the contents of the units file in the
physics file, but then there would be a circular dependency between
physics and io.
 
So what was the problem with splitting units out into it's own name space?

Circular dependencies between modules is a major code smell.  Code where everything depends upon everything, or even close to that, is unmaintainable.

Brian

Brian Hurt

unread,
Aug 15, 2010, 10:45:50 AM8/15/10
to clo...@googlegroups.com
Sorry, hit send too soon.


Circular dependencies are then the canary in the coal mine, warning you that your code is trending (hard) towards unmaintainability, You ignore the warning at your peril.

Brian

Mark Engelberg

unread,
Aug 15, 2010, 9:09:28 PM8/15/10
to clo...@googlegroups.com
On Sun, Aug 15, 2010 at 7:45 AM, Brian Hurt <bhu...@gmail.com> wrote:
> Circular dependencies are then the canary in the coal mine, warning you that
> your code is trending (hard) towards unmaintainability, You ignore the
> warning at your peril.

Avoiding circular dependencies can result in either:
a) Everything ends up in one file.
b) Everything ends up in zillions of small files with very complex
dependency structures (and every time you move a function from one
namespace to another to avoid a circular dependency, you have to
locate every file that uses the relocated function and change the
references to that function, all without the help of IDEs that do this
kind of refactoring for you -- it can be a lot of work).

Both of these scenarios result in their own significant
maintainability problems. I think it's much more ideal to be able to
divide modules of functions into the appropriate granularity for the
task at hand. I have found that for some tasks, circular dependencies
arise very naturally at the desired level of granularity, and I don't
like having to rework everything into something that avoids circular
dependencies. This problem seems to be especially acute in rapidly
evolving systems, because you're forced to constantly think about
conforming to an acyclic structure at a time when you're focusing on
something other than overall structure. I'd rather get things to
work, and *then* when the code base becomes stable, figure out a more
optimal organization.

Martin DeMello

unread,
Aug 16, 2010, 5:51:11 AM8/16/10
to clo...@googlegroups.com
On Sun, Aug 15, 2010 at 8:13 PM, Brian Hurt <bhu...@gmail.com> wrote:
>
> Circular dependencies between modules is a major code smell.  Code where
> everything depends upon everything, or even close to that, is
> unmaintainable.

Sometimes there's simply no way around it. For instance, I recently
had some python code that (stripped to its simplest form) had two
classes, Document and Attachment, where Attachment was a specialised
subclass of Document, but Document had a list of Attachments as one of
its members. There was no way around packing the Document and
Attachment classes into one big file, simply because I couldn't do
circular module dependencies (python files are modules), but any
refactoring of the code to remove the circular dependency left it
*more* complicated than before.

martin

Nicolas Oury

unread,
Aug 16, 2010, 6:07:04 AM8/16/10
to clo...@googlegroups.com
On Mon, Aug 16, 2010 at 10:51 AM, Martin DeMello
<martin...@gmail.com> wrote:
> Sometimes there's simply no way around it. For instance, I recently
> had some python code that (stripped to its simplest form) had two
> classes, Document and Attachment, where Attachment was a specialised
> subclass of Document, but Document had a list of Attachments as one of
> its members. There was no way around packing the Document and
> Attachment classes into one big file, simply because I couldn't do
> circular module dependencies (python files are modules), but any
> refactoring of the code to remove the circular dependency left it
> *more* complicated than before.
>

In Clojure, a protocol Attachable, could have been made explaining
what are the things you need for an attachment.

Martin DeMello

unread,
Aug 16, 2010, 6:23:56 AM8/16/10
to clo...@googlegroups.com

How would that have helped? The problem lay in the fact that there
could be many subclasses of Document, but only one specific subclass,
Attachment, could go into the attachments[] field. So if we had to
split the code into two files, we'd have

class Attachment(Document) # <-- attachment.py needs to import from document.py
...

and

class Document
def create_attachment
self.attachments += Attachment.new() # <-- document.py needs to
import from attachment.py

I did think for a while that it was a code smell, but after some
attempts at refactoring decided it was a "genuine" circular reference
in that this was the conceptually simplest and clearest way to write
it.

martin

Mike Anderson

unread,
Aug 16, 2010, 6:43:11 AM8/16/10
to Clojure
On Aug 14, 6:37 am, Wilson MacGyver <wmacgy...@gmail.com> wrote:
> I realize that. I was pondering why I don't run into the the 2nd problem.
>
> In your code, how many files/name spaces are you creating?
> And how many lines of code are in each file? I'm curious how you
> organize your code.

Sure - I'll give a quick sketch. Would love any hints if you think
this is a good structure or if there is anything I should be doing
differently!

It's about 6,000 lines of code, 16 Clojure source files of 50-1600
lines, which typically each define their own namespace. Structure is
modular, probably reflecting my preference structuring code around
conceptually related subsystems.

Everything gets loaded in approximately this order:

protocols.clj - common protocol definition used throughout the source
(e.g. defines the PGame protocol for game state)

command.clj - message formatting functions for comms
serial.clj - serialisation functions, wrappers for JSON save games
etc.
graphics.clj - graphics functions and image resource loading
player.clj - utility functions for player management
sounds.clj - wrapper for Java sound functions and sound resource
loading

map.clj - code for handling the game map and terrain
units.clj - code for all the units in the game, including unit
definitions and unit AI
game.clj - code for managing the overall game state and updates
(implements the PGame protocol)
gamefactory.clj - game/landscape generator functions (depends heavily
on game and map)

ui.clj - common GUI functions
dialogs.clj - library of common dialogs
renderer.clj - library for drawing / animating the main game view
interface.clj - GUI interface while playing the game, click handling
etc.
frame.clj - overall game window
main.clj - main menu GUI and navigation

The main pain points are:
1. Game state is pretty integral, but all the files above game.clj
can't access it except to the extent that I expose functions in the
common PGame protocol.
2. User interface comes last, which is good in general but makes it a
royal pain to pass notifications back to the UI. In Java I would
simply have e.g. units.clj call a simple notification function in
interface.clj, in Clojure I can't do that because I can't mutually
require the namespaces.... so I end up either passing a callback
function or polling some atom, neither of which are particularly
elegant.
3. Although it works, I don't really like declaring all the common
protocols up front because it seems to violate modularity - it would
seem nicer if the protocols could be bundled with their implementation
in a library file and included with just one :require

Mike Anderson

unread,
Aug 16, 2010, 5:00:39 AM8/16/10
to Clojure
On Aug 13, 7:16 pm, Nicolas Oury <nicolas.o...@gmail.com> wrote:
> On Fri, Aug 13, 2010 at 2:51 PM, Mike Anderson
>
Sadly I feel that I'm going to have to target development at the
lowest common
denominator that I think users will have (hence I'm targeting Java
1.5+)

Though I am excited about the prospect for the future! I think both
escape analysis
and G1 will help Java/Clojure massively for game development.

The killer case for me is having a simple 2D or 3D point vector as a
return value. It's
such a common pattern in games/graphics, it can happen millions of
times per second,
and currently there are no nice options: it seems you either have to
put up with excessive
garbage, hack the return value into primitive arrays or manually
"inline" the inner loop function.

If escape analysis can reliably guarantee stack allocation for this
kind of return value,
it would be a massive win.


Mike Anderson

unread,
Aug 16, 2010, 6:47:36 AM8/16/10
to Clojure
On Aug 14, 12:38 am, BerlinBrown <berlin.br...@gmail.com> wrote:
> I played it, it was pretty fun.
>
> I have UI recommendations. I couldn't tell the difference between the
> enemy and my units.
>
> I wish maybe you had some quick tips and recommended next actions so I
> could get used to how the game works.

Thanks!

Great to get the feedback, I'll work on improving this in the next
version.

Adam Burry

unread,
Aug 16, 2010, 8:23:25 AM8/16/10
to Clojure
On Aug 14, 2:39 am, Eric Lavigne <lavigne.e...@gmail.com> wrote:
> I discussed two problems. The first problem, which you addressed, was
> mostly just a warm-up for discussing a related problem that is more
> severe. Where can I put (def g) so that two files can "require" each
> other?

I would say this is not a clojure specific problem but a general
programming problem. John Lakos wrote extensively on this issue in
"Large Scale C++".

I would say that if you have two files that require eachother then
those things are obviously not independent and should be in the same
file. If you cannot use them independently then why separate them.
Either roll the files into a single file, or extract the tangled part
into a third file which can require the other two.

Adam

Mike Anderson

unread,
Aug 16, 2010, 4:47:56 AM8/16/10
to Clojure
On Aug 13, 7:06 pm, Brian Carper <briancar...@gmail.com> wrote:

> Looks great.  Thanks for sharing your experiences.
>
> Do you plan to share the source code?  Any reason you went with Swing
> instead of OpenGL?
>

Main reason I went with Swing was wanting to get something up and
running quickly (this is my first Clojure project after all!) and I
didn't really see the need for OpenGL for a graphically quite simple
game like this. It would always be possible to switch the renderer
later if needed.

I wasn't planning on making the code open source, but if you're just
after a quick peek you can always just take a look at the .clj files
inside the jar :-)



Mike Anderson

unread,
Aug 16, 2010, 6:08:10 AM8/16/10
to Clojure
On Aug 13, 5:33 pm, Alan <a...@malloys.org> wrote:
> Funny you should mention this - I was about to post a question about
> my own game when I saw your article. My issue is, I assume someone has
> written minimax and/or alpha-beta pruning in Clojure (or a java
> library that's easy to interop with). My case is slightly different in
> that the turn order is not fixes - sometimes player X goes twice in a
> row - but it ought to be pretty simple to plug into a standard AI
> library. Does anyone know where I can find such a thing?
>

I don't actually use alpha-beta: I opted to spend the effort to
develop
a decent evaluation function and then do some simple local
optimisation
on the gradient of said function.

My reasoning was that alpha-beta usually works best when the
branching
factor is low and the evaluation function pretty cheap to calculate,
sadly my situation was pretty much the reverse :-)

For your case it may be different. Two turns in a row works fine for
minimax or alpha-beta with a little tweaking (although it is likely
to
cut your search depth).

Be warned though - my experience is that it's rather hard to find an
AI library that will just "plug in" nicely to your code. Most of the
challenge in AI tends to be around special cases, embedding "expert"
knowledge and heuristics, plumbing in the right data representations
etc.
The search algorithm itself is usually the easy bit....

If you're after resources, there's a decent free online book on AI in
Java that might be useful, has lots of code examples that should be
pretty easy to convert to Clojure:

http://www.scribd.com/doc/6995538/Practical-AI-in-Java

Nicolas Oury

unread,
Aug 16, 2010, 9:21:24 AM8/16/10
to clo...@googlegroups.com
> Sadly I feel that I'm going to have to target development at the
> lowest common
> denominator that I think users will have (hence I'm targeting Java
> 1.5+)
>

Escape Analysis and GC tweeking does not need to change your code.
It is activated by option to the JVM.

It would be interesting to try it on your computer and if some options
helps really a lot,
dynamically detect if they are available in a launch script.

For the allocations of points in return values, macro can probably
helps a bit with the inlining,
it would be fun to throw one or two examples on the list to see if
there is a pattern.
I wouldn't advocate this solution if Escape Analysis removes the
problem, though.

Best,

Nicolas.

Timo Mihaljov

unread,
Aug 20, 2010, 1:00:28 PM8/20/10
to clo...@googlegroups.com
On Mon, Aug 16, 2010 at 03:43:11AM -0700, Mike Anderson wrote:
> 2. User interface comes last, which is good in general but makes it a
> royal pain to pass notifications back to the UI. In Java I would
> simply have e.g. units.clj call a simple notification function in
> interface.clj, in Clojure I can't do that because I can't mutually
> require the namespaces.... so I end up either passing a callback
> function or polling some atom, neither of which are particularly
> elegant.

It's often helpful to have one component that acts as a message bus
between the other components.

A quick and dirty example:

;;; --------------------------------------------------

;;; The message bus
(ns game.event)

;; A map storing the functions that should be called when an event
;; occurs
(def callbacks (atom {}))

;; Functions for subscribing to, and dispatching events
(defn call-when [event callback]
...)

(defn dispatch [event & args]
...)

;;; --------------------------------------------------

;;; User interface
(ns game.ui
(:require [game.event :as event]))

;; An example of an event listener
(event/when :unit-destroyed
(fn [unit]
(comment Update the UI here)))

;; An example of an event dispatcher. Sends an event to be handled by
;; the component whose responsibility it is. Note that at this point
;; we don't know which component that is.
(defn request-reinforcements-button-clicked []
(let [amount (.getValue reinforcements-amount-input-field)]
(event/dispatch :reinforcements-requested amount)))

;;; --------------------------------------------------

I've found this approach to work quite well in game development. As
the components don't care about who reacts to the messages they send,
or who sent the messages they are interested in, it's very easy to
plug in and remove components as needed. For example, you can add a
sound component that plays appropriate sounds when events occur in the
game without requiring any changes to any other components.

This approach is especially helpful when unit testing. You can create
a new bus and plug in just the component under test, and then simulate
all the other components by sending the same messages as they do.

--
Timo

Alan

unread,
Aug 20, 2010, 2:21:27 PM8/20/10
to Clojure
Thanks, I'll look into it. I know minimax ought to be easy to do but
it's a bit of a weak spot of mine - I can never seem to get it right,
and the poorish debug support in clojure, even with slime/swank,
doesn't make it easier.

I'm reasonably confident minimax/alpha-beta is right for me, since
it's not really an AI - I'm writing a "solution finder" which works
with perfect knowledge and has all the time it needs (though obviously
faster is better) to find the optimal solution. If anyone's
interested, it's specifically a double-dummy solver for the card game
bridge.

On Aug 16, 3:08 am, Mike Anderson <mike.r.anderson...@gmail.com>
wrote:

Ben Mabey

unread,
Aug 20, 2010, 4:47:40 PM8/20/10
to clo...@googlegroups.com
On 8/20/10 12:21 PM, Alan wrote:
> Thanks, I'll look into it. I know minimax ought to be easy to do but
> it's a bit of a weak spot of mine - I can never seem to get it right,
> and the poorish debug support in clojure, even with slime/swank,
> doesn't make it easier.
>
> I'm reasonably confident minimax/alpha-beta is right for me, since
> it's not really an AI - I'm writing a "solution finder" which works
> with perfect knowledge and has all the time it needs (though obviously
> faster is better) to find the optimal solution. If anyone's
> interested, it's specifically a double-dummy solver for the card game
> bridge.

I stumbled across this implementation of alpha-beta in clojure on github:

http://github.com/rcrr/reversi/blob/master/strategies.clj#L251-282

It is a re-write of PAIP's CL version. (I didn't write it- just found it.)

-Ben

Nicolas Oury

unread,
Aug 21, 2010, 3:42:57 AM8/21/10
to clo...@googlegroups.com
Just my 2 cents:

sometimes these algorithms are easier to implement by constructing an
infinite lazy trees of all the game, and
then independently writing a few strategy to explore the tree.

John Hughes gives an example of that in Section 5 of "Why functional
programming matters", that can be found on the internet.

If it can be done that way, this is nice has it separates the tree
construction, which is about the rule of the game, from the
evaluation/exploration part.

Laurent PETIT

unread,
Sep 1, 2010, 10:49:15 AM9/1/10
to clo...@googlegroups.com
A little bit OT, but for files where you prefer to write your
declarations "top down" rather than "bottom up":

(defmacro top-down [ & body ]
`(do ~@(reverse body)))

then

(defn utility-fn [])
(defn high-level-fn [] (utility-fn))

becomes

(top-down

(defn high-level-fn [] (utility-fn))

(defn utility-fn [])

)

:)
2010/8/14 Btsai <benny...@gmail.com>:

tsuraan

unread,
Sep 1, 2010, 11:58:08 AM9/1/10
to clo...@googlegroups.com
> How would that have helped? The problem lay in the fact that there
> could be many subclasses of Document, but only one specific subclass,
> Attachment, could go into the attachments[] field. So if we had to
> split the code into two files, we'd have
>
> class Attachment(Document) # <-- attachment.py needs to import from document.py
>  ...
>
> and
>
> class Document
>  def create_attachment
>    self.attachments += Attachment.new() # <-- document.py needs to
> import from attachment.py

What python allows to break the circular dependency is function-level imports:

attachment.py:
from document import Document
class Attachment(Document):
pass

document.py:
class Document(object):
def create_attachment(self):
from attachment import Attachment
self.attachments += Attachment.new()

I don't think clojure has anything like this; it looks like imports
are at the file-level, rather than the function level as in python.
Perhaps there is a way though?

Reply all
Reply to author
Forward
0 new messages