Entity–component–system and Clojure
518 views
Skip to first unread message
Didier
Aug 15, 2017, 8:52:38 PM8/15/17
to Clojure
I recently stumbled upon the entity-component-system design pattern which is popular in game engine design: https://en.wikipedia.org/wiki/Entity%E2%80%93component%E2%80%93system, and really liked what I saw, thought it could be a good fit for Clojure.
Basically, it has three concepts:
1) Components are pure data grouped together as per a given domain. In a game engine that would be for example the 3D positional data related to positioning of objects in the 3D scene. So one component would be PositionComponent and it would have :X, :Y.
2) Entities are collections of Components with a unique ID.
3) Systems are processing functions that take an entity, transforming their components' data, or performing side effects from them.
Generally, in games, they inverse the entities, so instead of having entities contain components, they have components stored in an array with the index being the entity ID, and another array which contains the set of components for the entity at that index. All of this is kept track of by a world container.
(def world
{:entities []
:comp1 []
:comp2 []
...})So say you want to create an entity which is composed of comp1 and comp2, you would just add to the world :entities at index 0 a set #{:comp1 :comp2}, and to the world :comp1 and :comp2 vectors at index 0 an initial component1 and component2 data structure. In games, for performance, they use a bitmask instead of a set for the entry of :entities.
I'm not sure this structure is necessary if trying to use the pattern not for game, but it doesn't hurt either I think.
What I like about this, is I'm thinking its possible to use it to do data-driven functional object modeling in Clojure. A problem I face, and I feel other faces in Clojure, is how do you model entities without OOP? I find this creates a kind of OO that is functional and data driven.
You would spec a bunch of component, they're pure data. Then you'd define systems (aka functions) which take an entity, and operate on the entity's components (aka its data). At first glance, this appears to just be OOP, but there's no inheritance here, and functions that operate or related data are decoupled from the data. Systems are implicitly mapped to components, based on what they work on. So you can extend all entities with more functionality easily. You can also create entities from components on the fly.
On second glance, I wonder what's different about this from just functions operating over data. I think its just a more rigid means to do so when you need the concept of entities. In a way, entities act as a class, in that they're a template of data. A system works over that template.
Has anyone experimented with this in Clojure?
Jeaye
Aug 15, 2017, 9:05:28 PM8/15/17
to clo...@googlegroups.com
Alex Kehayias gave a talk about designing a functional game engine in ClojureScript, using components, here: https://www.youtube.com/watch?v=TW1ie0pIO_E I thought it was a superb talk.
There's also Arcadia, here: https://github.com/arcadia-unity/Arcadia which wrap's Unity3D in Clojure, but Unity takes about the most imperative approach to components as possible.
In general, and as Alex shows, the entity-component approach can work very well with game engines. The issues of coupling can either be tackled with explicit dependency declaration or by a more flexible async notification/inbox system. I haven't yet built something like this, though I'd like to, but, if/when I do, my design would be very much along the lines of Alex's.
J
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com
> Note that posts from new members are moderated - please be patient with your first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
There's also Arcadia, here: https://github.com/arcadia-unity/Arcadia which wrap's Unity3D in Clojure, but Unity takes about the most imperative approach to components as possible.
In general, and as Alex shows, the entity-component approach can work very well with game engines. The issues of coupling can either be tackled with explicit dependency declaration or by a more flexible async notification/inbox system. I haven't yet built something like this, though I'd like to, but, if/when I do, my design would be very much along the lines of Alex's.
J
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com
> Note that posts from new members are moderated - please be patient with your first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
James Reeves
Aug 15, 2017, 9:50:10 PM8/15/17
to clo...@googlegroups.com
I wrote Ittyon a little while ago, which is a database partially inspired by the entity-component model, but uses a more clojurey architecture.
There are also experiments with trying to make rule systems like Clara fast enough to use in games.
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscribe@googlegroups.com.
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscribe@googlegroups.com.
Mark Mandel
Aug 15, 2017, 10:13:12 PM8/15/17
to clojure
A while ago, I wrote a library called brute as an ECS library for Clojure + ClojureScript.
Articles:
Github:
Mark
Thomas
Aug 16, 2017, 9:44:50 AM8/16/17
to Clojure
If I am not mistaken Lightable uses it as well. There is also a plug in for it which helps with the ECS.
Hope that helps.
Thomas
joinr
Aug 18, 2017, 8:40:34 AM8/18/17
to Clojure
Didier, I think your synopsis is right on the mark. I shifted from the canonical class-based data model for entities (primarily for discrete event simulation, with applications to games) toward a functional ECS approach a while back - even before I found clojure in 2010. The biggest utilities I find are a) flexibility in data definition(s) - add/remove components as needed, b) unified storage medium for entity information (like an in-memory database, with the ability to easily define aggregate queries and other relational stuff), and c) the ability to operate on domain-specific slices of entities (via selected components). There's a similarity between shifting to an ECS model and using clojure maps for generic data storage; lots of flexbility and unintentional re-use.
My (work in progress, but useful) implementation of ECS (from a melange of libraries called spork): spork.entitysytem.store.
This is useful, albeit research-quality software that I'm still adapting to my needs and documenting. If nothing else, there may be pedagogical value for other folks.
I took the approach of pushing the entitystore behind some fundamental protocols, and evolved functionality as I ran into problems. Like relational databases, you can take a column or row-based approach to implementing an entitystore (where component data is defined). Depending on your needs and use-cases, either may be appropriate. I stuck with a column store, with IEntityStore protocols implemented on a record that stores information about entity component membership (has-a?) and the actual component domain data. So, we end up with a structure like {:entities {:id1 #{:name :domain2 ...}} :domains {:name {:id1 "SomeName"}}} as our "store". From there, we can define operations on how to add, drop, update, modify entries in the store. Convenience functions, like implementing sql-like queries are easy to implement too. You can probably stick datalog or other unification-based queries on the store since the structure is pretty easy to pick apart. I opted for clojurey names for operating on entity components, ala "assoce, updatee, mergee, assoc-ine, ..." which provide a familiar layer (for me) over the like-named clojure functions.
When you stick with the intended use-case [designing "systems", i.e. functions that operate on one or more domain of component data, to compute new entity stores and the like] the scheme works really nicely. Specifically, systems that are tied to one component, or entity stores with a relatively small number of components that are cheap to join. About 1/2 way into fleshing out the design and putting it through its paces, I realized that i) I often don't care about all the components of an entity.....but I may not know that information ahead of time; it'd be nice to not have to compute joins for every entity every time on certain systems ii) updating entities by-component can be very inefficient, so batching updates would be nice....
This cried out for either a row-based implementation, or a pseudo-row based representation of an entity that avoided paying costs for computing joins across components, while allowing for the column-based properties of the backing store. I ended up implementing a lazy entity record, which provides a map-like reference to the entity's components but performs joins lazily and on-demand, and kept track of changed or dirty components smartly. This alleviated most of the performance concerns I ran into, and enabled me to define and additional set of operations on entities as if they were row-based, i.e. clojure maps.
Regarding inheritance and some of the class-based things you brought up, I baked up a little DSL for defining entity constructors (really just functions making maps) that's deriving partially from the CLOS way of defining structs (with inherited fields and the like). I thought it was pretty slick when I first cooked it up (after reading Land of Lisp), but I don't really use it all that much. In practice, you can just create a map and shove it into the entitystore, which implies dissecting the key-vals of the map and associng like component entries for the entity (assuming a :name key exists). Or, you can create complex "class-like" hierarchies and use that to alleviate some of the burden. Or you can make your own composition of functions - operating on data - to achieve the same result.
Currently, I have the entitystore paired with a behavior tree system that I use for entity behaviors / "ai". I'm still feeling it out, but it works and provides a way to compose sophisticated behaviors that gets away from the FSM spaghetti I ran into originally. In this case, the entitystore provides another advantage, since you can use it as a blackboard for communicating information between multiple concurrent entities. Rather than "firing" events and side-effecting, you can use the presence/absence of data via components to communicate. You can still implement messaging via component data as well. Communication (or simulating communication between entities) was one of the harder problems that seemed to be left as an "exercise for the reader" in much of the ECS literature/presentations...
For me, the combination of an entitystore and behavior trees has ended up fairly robust in practice. It's even surprisingly performant for some real-time 60fps visualizations I've done (tied to a simulation), where I thought I'd have framerate problems. I'm still optimizing (for faster simulation runs), looking into varying implementations for the store - mutable, async, row-store.
My (work in progress, but useful) implementation of ECS (from a melange of libraries called spork): spork.entitysytem.store.
This is useful, albeit research-quality software that I'm still adapting to my needs and documenting. If nothing else, there may be pedagogical value for other folks.
I took the approach of pushing the entitystore behind some fundamental protocols, and evolved functionality as I ran into problems. Like relational databases, you can take a column or row-based approach to implementing an entitystore (where component data is defined). Depending on your needs and use-cases, either may be appropriate. I stuck with a column store, with IEntityStore protocols implemented on a record that stores information about entity component membership (has-a?) and the actual component domain data. So, we end up with a structure like {:entities {:id1 #{:name :domain2 ...}} :domains {:name {:id1 "SomeName"}}} as our "store". From there, we can define operations on how to add, drop, update, modify entries in the store. Convenience functions, like implementing sql-like queries are easy to implement too. You can probably stick datalog or other unification-based queries on the store since the structure is pretty easy to pick apart. I opted for clojurey names for operating on entity components, ala "assoce, updatee, mergee, assoc-ine, ..." which provide a familiar layer (for me) over the like-named clojure functions.
When you stick with the intended use-case [designing "systems", i.e. functions that operate on one or more domain of component data, to compute new entity stores and the like] the scheme works really nicely. Specifically, systems that are tied to one component, or entity stores with a relatively small number of components that are cheap to join. About 1/2 way into fleshing out the design and putting it through its paces, I realized that i) I often don't care about all the components of an entity.....but I may not know that information ahead of time; it'd be nice to not have to compute joins for every entity every time on certain systems ii) updating entities by-component can be very inefficient, so batching updates would be nice....
This cried out for either a row-based implementation, or a pseudo-row based representation of an entity that avoided paying costs for computing joins across components, while allowing for the column-based properties of the backing store. I ended up implementing a lazy entity record, which provides a map-like reference to the entity's components but performs joins lazily and on-demand, and kept track of changed or dirty components smartly. This alleviated most of the performance concerns I ran into, and enabled me to define and additional set of operations on entities as if they were row-based, i.e. clojure maps.
Regarding inheritance and some of the class-based things you brought up, I baked up a little DSL for defining entity constructors (really just functions making maps) that's deriving partially from the CLOS way of defining structs (with inherited fields and the like). I thought it was pretty slick when I first cooked it up (after reading Land of Lisp), but I don't really use it all that much. In practice, you can just create a map and shove it into the entitystore, which implies dissecting the key-vals of the map and associng like component entries for the entity (assuming a :name key exists). Or, you can create complex "class-like" hierarchies and use that to alleviate some of the burden. Or you can make your own composition of functions - operating on data - to achieve the same result.
Currently, I have the entitystore paired with a behavior tree system that I use for entity behaviors / "ai". I'm still feeling it out, but it works and provides a way to compose sophisticated behaviors that gets away from the FSM spaghetti I ran into originally. In this case, the entitystore provides another advantage, since you can use it as a blackboard for communicating information between multiple concurrent entities. Rather than "firing" events and side-effecting, you can use the presence/absence of data via components to communicate. You can still implement messaging via component data as well. Communication (or simulating communication between entities) was one of the harder problems that seemed to be left as an "exercise for the reader" in much of the ECS literature/presentations...
For me, the combination of an entitystore and behavior trees has ended up fairly robust in practice. It's even surprisingly performant for some real-time 60fps visualizations I've done (tied to a simulation), where I thought I'd have framerate problems. I'm still optimizing (for faster simulation runs), looking into varying implementations for the store - mutable, async, row-store.
Reply all
Reply to author
Forward
0 new messages