Notes on implementing FBP in a game engine

[James Hofmann]

Hello, I've followed FBP things off and on for some time and made toy implementations but am only recently convinced I have found a great application for it. The game engine is called "Casten" and is written in Haxe, using the Kha framework. It is currently being used in the development of a 2D, GUI-intensive game. The engine as initially implemented is nothing too outrageous: game assets are organized in an XML script file and then handed off to a hard-coded game loop. Other details of the build process are handled by Kha.

The major change to the engine that motivates FBP - I'm doing it classic-style, and it's gradually being refactored into play now - is the introduction of a global addressing system with a strong resemblance to the Unix filesystem: a tree structure type called "MixedNode" with each node being either a leaf node, or a container for a list, string map, or ordered string map, and optionally containing arbitrary(dynamic typed) data or string tags. In addition to the basic structure, route objects may be constructed and functions to traverse, search, or match routes are included - enabling "soft" or "symbolic" linking, partially-completed paths to assets, and miscellaneous other "late-binding" techniques.

At first MixedNode was just compiler technology for a "story engine" - translating XML script into state-machine behaviors and instructions for rendering content. The structure started off very strict at first, using only integers as the allowed data type, intending it to output flat arrays of integers for runtime efficiency. At the same time I did this, I was looking for ways to expose all of the different data types used by the engine(source assets, new assets derived at runtime, input devices, etc.), and I iterated through a few things, but gradually sensed that I could bump up the power level of MixedNode to do it. So I switched to the dynamic types and added tagging. This gave me the necessary expression to equally describe assets from files, assets generated at runtime, and assets received over the network, and to describe some assets as compositions of other assets(e.g. "monster entity" is a monster sprite, a monster AI behavior, a monster sound).

Once I did this, I found a way to make FBP make sense in the engine: I could write a "flow compiler" that builds small FBP programs at runtime based on new assets or user configuration, and have it convert MixedNode addresses into the input and output data for the flow graph. The most trivial example of this, which I'm still working on now in spare moments, is input device configuration: It's possible to plug in many different input devices, and sometimes hotswap them. But at the other end, gameplay code demands input data in a limited, more tautological form: "the jump button is defined as the jump button". In between there's a binding of controller data to gameplay input, which is usually hand-massaged code written towards particular devices and use cases; some games support gamepads, others keyboards and mice, touchscreens, spinners, wheels, trackballs, dance pads, accelerometers, microphones, etc. In making the two ends of the system addressable, and mapping the two together with a program rather than a simple binding, there is a suitable system for describing all kinds of user configurations, and even doing additional filtering. 

Dataflow is already used in some engines to facilitate art creation or certain game design tasks, but it's not used throughout as the core style - which I believe is erroneous in that so much of engine coding is entangled with "pipeline" creation, just the simple steps needed to load, process assets and make them usable at runtime. Hard-coding the pipeline in sequence is sufficient for many simple cases, but it falls short as more configuration becomes necessary and more dependencies appear(such as the compositional assets I mentioned earlier). It's the addressing that presented the difficulty, since engines tend to use different data containers to optimize for different tasks, and don't naturally fall into a single hierarchical organizational model.

The combination of addressing + FBP also describes a means of achieving a more modular, exploratory game engine: With modules that operate on common packet types and can easily "hotswap" sources and destinations, it's possible to get similar kinds of results to the "Ming Mecca" analog system, which uses analog control voltages and a small SOC to achieve glitch art effects and arcade-style gameplay in tandem with synthesizer sounds.

I should note, though, that all my previous experimentation also concluded that for the innermost gameplay coding, access to imperative style is still essential. The problem definition of most video games - requiring past game state to feed into present state - tends to lead to this style intrinsically, creating a monolithic imperative feedback loop broken up into periodic "ticks" or "frames". That part is not what I'm trying to address. But what can be broken out into FBP-friendly modules is mostly everything else - the details of the rendering, the types of assets used, where they are not gameplay-critical, anything I/O related, anything that might entail hard-coded algorithms combined in unique ways(procedural content generation).

If all goes well, I may be able to report back in a few months with more progress and hopefully some validation!

James

[Paul Morrison]

Hi James,

This sounds interesting!  You mention that you are doing it "classic-style" - by this I assume you mean "classic FBP" as described in my book, web page, http://www.jpaulmorrison.com/fbp/ , and related pages...  I hadn't heard of Haxe, but I assume it doesn't have a virtual machine, but is more like C++.  If so, what does it/you use for pipelines, asynchronism, etc.?  I used Boost for my C++ FBP implementation - https://github.com/jpaulm/cppfbp - and found it worked pretty well!  Or does Casten take care of that?

Keep us posted on how this is going!

JPaul

[Paul Morrison]

Hi James,

If I understand you correctly, you have a division of labour between functions that are time-critical and functions that can be more asynchronous using streaming - which can use "classical" FBP concepts.  You mention I/O and rendering for the latter, which reminds me of Jing Chen's video on Facebook Flux - https://youtu.be/nYkdrAPrdcw - where, among other points, she describes doing diffs between successive scenes, and just rendering the differences.  I also assume that you can look after logging and error lists this way. If I'm right, it makes sense to split functions this way, as a lot of the FBP-like systems out there let the requirement for time-critical functions force the overall architecture to be more synchronous, which prevents the designer from taking advantage of many of the benefits of "classical" FBP.   Or am I reading too much (or the wrong things) into what you wrote? :-)

I'm afraid I didn't understand: "At first MixedNode was just compiler technology for a "story engine"" - later I read MixedNode as being (or becoming) a tree-structured database...  which would seem more passive...?

Also, I was confused by: "the jump button is defined as the jump button".  ?

A lot of good material in your post - I should probably reread it several times!

Best regards,

JPaul