settings/configs/defs systems for data driven games

[vazor]

I was asked about settings/configs/defs - data driven systems for allowing a game to be changed in settings and with data at runtime instead of being all code driven. This can be useful for example if you want a game to be able to change dynamically based on A/B testing groups, or to be able to update something or disable a feature/section of the game while you fix an issue without having to wait for the app store review.

I'm posting to see if anyone else has experience with these and wants to chime in on their best practices with them, and also in case it helps anyone as they build one or if anyone knows of an e.g. unity asset that does this.

Such a system has three major parts:

1. Design time packaging

At design time, you need some way to gather all the settings data. 

At one company, we called them "defs" and we had a custom tool with a GUI that was basically a properties file editor but with a way to visualize images and animations as you set the properties. Then it would "export" to a proprietary binary or string format and upload that to the server.

At another company, we called them "configs" and people wanted to enter the values in excel sheets. The name of the sheet would determine what kind of data structure it would be packed into, and the column headers would be the field names, and then each row would be an entry/instance of that structure/type. This was packed into a proprietary format using python to scan the sheets, but you could use any wire data format like json/bson, thrift, or protobuf to pack the data.

The upload might also need to be packed a certain way if you want to use unity bundles to download them.

2. Server storage

The places I've seen these stored were self-hosted servers, CDNs, and AWS S3 w/ a CDN front to access them.

3. App/client programming

The hardest part, of course, is writing/reworking your client code to be data driven and show images and UI based on the settings data and even turn on and off features based on the settings data. You can have versions or timestamps you check with the server on client startup to see if your current settings are outdated, and then if so you can fetch the latest data and process it. Both of the places I worked at even had logic in data form to parse and make decisions- in the tool there were logic drop downs for comparison operators and references to other properties, and in the excel sheets they could put in basic expressions we had to parse.

Thanks to Yotingo I got this rundown of unity assets that could probably be useful in building this kind of system:

Backend Services

As discussed during our call, here's some backend solutions.
My current favorite is Xsolla Backend (XBE) but Beamable is the one that supports a content delivery model out of the box.

Here's a good comparison list: https://i42.quest/baas-compare
There's also a GBaaS Discord channel if they are interested in that.

CBS - Cross-platform Backend Solution
Pre-built Playfab. Very comprehensive but Playfab is in the middle of a silent sunset.
https://assetstore.unity.com/packages/tools/game-toolkits/cbs-cross-platform-backend-solution-playfab-200638

Easy UGS
Integration for Unity Game Services - Unity's in-house backend solution. Expect more support from the developer of the asset than from Unity themselves.
https://assetstore.unity.com/packages/tools/game-toolkits/easy-ugs-228849

EzCloud
Unified Firebase API.
https://assetstore.unity.com/packages/tools/integration/ezcloud-the-complete-cloud-solution-for-unity-213605

Beamable
Ready-to-go prefabs. C# microservices. Good documentation but slow support.
https://beamable.com/

Xsolla Backend (previously AXR)
Modern architecture that supports BYOC (Bring Your Own Container). Poor documentation but great support.
https://www.acceleratxr.com/

Thanks all!

  -vazor

[Ross Przybylski]

Hi Josh -

I'm all-to-familiar with the data-driven architecture you're describing and happy to chime in and answer any additional questions folks may have.

1. We also call these configuration/definition files "defs". At my previous company, they were stored in JSON format; for my current work, I store them as sqlite (.db) files. I've found the best practice is indeed to create a custom editor that's suited to the needs of the specific game. That's because your custom editor can validate that the data structure meets the unique needs of the game - any kind of universal editor runs the risk of corrupting data when making edits.
2. Defs should be packaged both locally in the client and server-side. That way, the game always runs in the event the server cannot be reached. The client and server both have a manifest file that describes all the files that can be dynamically updated by the server (defs, assets, sfx, etc.). When the application launches, it compares client and server manifest, and updates all the files that are out of date by downloading them.
3. All Appstore/Google Play client updates require human approval, which can delay releases, and you cannot add executable code dynamically via server. But, using this architecture, defs and assets can be updated dynamically because there is no machine code to pose security risks. As you pointed out, they can contain logic, which is interpreted by your game, to create conditional effects. This is another reason why it's best to custom develop an editor tool specific to the project: you validate the data and your game can interpret your data as a script. Think of it like a map editor: the levels are your definitions, and those levels can contain any amount of logic to run a full fledged game (like the how the Defense of the Ancients map in Warcraft III was capable of producing an entirely new "League of Legends" game without adding any new code).
4. The silent sunset of back-end solutions like Playfab are a good example of why I advise approaching out-of-the-box solutions with caution. While these services can provide unparalleled ability to scale applications, there is risk of obstufucatation in their documentation/actual operations and potential for future headache depending on how the platform evolves. The mechanics of applying this architecture are fairly simple and can be handled with basic web hosting, MySQL, and PHP. If you're building a new application from the ground up, it sometimes makes sense to start with the simplest solution, design the client/server model together and fully understand its operation. Then, as the application scales, isolate specific services you need to handle more demanding tasks. For example, nearly all of my game services are handled via PHP/MySQL on a VPS. If I get more traffic, I upgrade to a dedicated server. Then I consider other options (like AWS/Azure). The most demanding service for a game is live multiplayer, and for that I use PlayerIO to handle unlimited scaling of players with minimal code intersections and dependencies. In other words: build your solution all-a-carte so that you have direct control over as much of your service as possible (especially your data) - and create interfaces to outside services that can be easily swapped with another service if your third-party solution decides to sunset.

Hope this helps!

Ross Przybylski, Founder & Game Developer

D20Studios, LLC

--
You received this message because you are subscribed to the Google Groups "Utah IGDA Members" group.
To unsubscribe from this group and stop receiving emails from it, send an email to igda_utah+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/igda_utah/CANwk9dsOG2dbH5bqNX_zFLB2JELkHGdGirtuRH-7_OGX%2B19yiw%40mail.gmail.com.

[Spencer Buchanan]

Super cool convo. My part is nowhere as technical, but I recall a GDC talk in 2012 by the folks that made Diner Dash and how they kept their game in the top 10, or almost top 3, for over 3 years straight in the app store. 

Aside from the notion that launch day is day1 of real development for your game, the part that contributed to most of their success is solving the for problem here of not letting the App store determine when you get to push updates to the players. They said the turnaround of pushing an XML file and assets to the user was almost daily to keep content fresh for users, if not weekly. Back then the way they explained it was almost a "dont tell apple we are doing this because they might get mad and ban our app" tone. 

It is interesting, 12 years later, it is still a very relevant issue to solve for and apparently Ross and Josh know a lot about the technical aspects. 

A question for you, I recall when ClusterTruck came out, I was watching streamers playing it that knew the dev. (This was a fascinating indie game that had a lot of community interaction during development and was fascinating how he grew his user base by posting updates so often on reddit.) The fascinating thing that the dev was doing was changing parameters of the game live while people streamed. He gave the streamers a build of the game that allowed for him to push changes live where the gravity got turned off, or all the trucks turned to ducks. He even posted messages on screen of the streamer in game as ways to troll them while they were playing. It was a very fascinating way to engage with players as the dev was literally pulling strings live. 

The practical use of doing that is probably low, but would the systems Josh and Ross mention above allow for that level of manipulation? 

I have barely been following Hell Divers 2 but what I have heard is it seems to have garnered a lot of community engagement to create events, along with game changing evolution or "on the fly designing" (https://www.pcgamer.com/we-have-an-actual-person-with-the-title-of-game-master-a-single-helldivers-2-dev-named-joel-is-pulling-the-strings-on-its-galactic-war-like-an-all-powerful-dandd-dungeon-master-war-will-become-more-and-more-sophisticated-over-time/) of new content to keep engagement up. 

Regardless of how you do it technically, that always seems to be one of the hardest parts of long term engagement with games. 

Spencer Buchanan

(801) 448-6055 

buch...@gmail.com

To view this discussion on the web visit https://groups.google.com/d/msgid/igda_utah/CANRde5cMUF_wa293tm7peVG%3DdUaOOvZ%2BGWdbgUrM-kr3DYZZ0w%40mail.gmail.com.

[vazor]

Often the checks for the "defs" type systems are only on startup, so if the game is easy to hop in and out of or the streamers you're working with are fine restarting it whenever you ask them, it would work. There's nothing stopping you from building this system to have a realtime check though. You would probably want to separate the realtime defs from the startup defs though.

The other way that comes to mind to make a streamer interaction happen at realtime is to set up a server that is connected to the game using a socket or other realtime protocol, and then a way for you from the developer side to flip switches on the server (usually a web based admin tool).

Yeah the conventional wisdom in long term engagement in f2p/mobile/IAP/gacha games is: events, events, events.

[vazor]

Yotingo found a more direct asset solution (less live-service oriented) for the use-case of not submitting new builds to a storefront. It's essentially a replacement of Addressables but it's supposed to be easier and has built in support for some remote servers.

https://assetstore.unity.com/packages/templates/systems/dream-bit-bundles-142117#description

Or this one has support for distributed builds, flexible packaging, and even version management. The author doesn't speak english natively, however.

https://www.yooasset.com/en-US/docs/Introduce