Saving and loading

[Alberth]

Upon request, an explanation of what I found out about load and save.
Feel free to comment and/or improve on it, I am likely to be incomplete, missing some key feature of the problem, or just talking nonsense.

The goal of save (and load at some future point in time) is that a user can continue where he left off the last time.
In addition (and this is what makes it difficult), a saved game can be given to another user who may load it in a completely different environment, or it may be loaded after a looooong time, where FreeRCT is evolved to something else (much better, hopefully).
The latter uses make "load" a much more difficult problem than "save".

Save seems however a good starting point to get an understanding of what is involved in storing a game to a file.
Basically, the game state needs to be serialized onto a file. With save there is only one version you can save, namely the current one. (If you want to be able to load a file in an older version of the game, you need to be able to write older serializations too, but to me it does not seem useful enough to support.)

The next question is what to save exactly. A simple but non-feasible answer is "everything". A memory dump would obviously work. However, it does not seem the right solution, since that also means all data structures must stay the same, and get restored at the exact same place in memory. Further evolution would be tricky at least.

So what to save then? Do we save the set of windows opened at the game interface? Do we save that the user was buying a shop, and had just placed it? I don't know whether we should, but as a starting point, I would think just the current simulation engine state is enough.
Opened windows, loaded RCD files, and their sprites and other data is not saved.

Also, pointers in the data structures are converted to numeric indices in some way to decouple them. Furthermore, the game engine has duplicate data storage for easier/faster access at run time. To make life easier during loading, there should not be any duplication in the stored data. It may make save and load a bit more complicated, but I believe solving a consistency problem in the saved data is much worse to handle.


A scenario save is more difficult. To save a game, you save all game state. For a large part, game state depends on the RCD data available at the time of the save, eg a shop of type X in the game. X and its behavior is defined in an RCD file. If you ensure the same RCD file is available during loading, you can continue the game.
A scenario however can be seen as a template. Some of the things are pre-defined, but other things are still free. One thing I expect to happen is that users will want to play with their favorite set of RCD files. Loading such a scenario thus has to be able to load the saved state, yet allow the user to add/modify some aspects.


Loading has the problem that you don't know what you got until you load it. Since I would like to support old versions (otherwise all saved games and all scenarios would stop working with each new release), the loader has to be ready for loading an old version.
Thus it may get file data about concepts that has changed a lot in the mean time, or parts of the game state are missing entirely (when we introduce something new).
Secondly, the file may need RCD files that are not at the system or (more likely) a more modern version exists. (Off-topic, but a reliable way to detect RCD file version is thus needed, experience shows that a manually maintained version number will not work.)

The game may also have RCD files that were not around when the game was saved.
Thus basically, loading has to deal with three problems
- Old file data that needs to be updated to the current version
- Different RCD file environment
- De-serialization of the save, in particular mapping indices back to pointers, and restoring the data duplication in the run time.

[mac coder]

Save seems however a good starting point to get an understanding of what is involved in storing a game to a file.
Basically, the game state needs to be serialized onto a file. With save there is only one version you can save, namely the current one. (If you want to be able to load a file in an older version of the game, you need to be able to write older serializations too, but to me it does not seem useful enough to support.)

There is absolutely no need for saving in old formats. The game is free, so everyone can keep it up to date.

The next question is what to save exactly. A simple but non-feasible answer is "everything". A memory dump would obviously work. However, it does not seem the right solution, since that also means all data structures must stay the same, and get restored at the exact same place in memory. Further evolution would be tricky at least.
So what to save then? Do we save the set of windows opened at the game interface? Do we save that the user was buying a shop, and had just placed it? I don't know whether we should, but as a starting point, I would think just the current simulation engine state is enough.
Opened windows, loaded RCD files, and their sprites and other data is not saved.

Everything that can't be calculated or loaded from files (sprites, terrain data) and represents the game's current state. Windows should be saved. There isn't to much to save anyways (window). You just save the list of opened windows and their screen position, the data displayed in window is part of the selected object (shop, ride, person) anyways. For shops, I would only save it's location and current values (price, sales, position...). I don't see any point in saving if a shop was bought - can you explain how you meant it?

A scenario save is more difficult. To save a game, you save all game state. For a large part, game state depends on the RCD data available at the time of the save, eg a shop of type X in the game. X and its behavior is defined in an RCD file. If you ensure the same RCD file is available during loading, you can continue the game.
A scenario however can be seen as a template. Some of the things are pre-defined, but other things are still free. One thing I expect to happen is that users will want to play with their favorite set of RCD files. Loading such a scenario thus has to be able to load the saved state, yet allow the user to add/modify some aspects.

A scenario save doesn't seem that very much different to me. It has everything that a save game has, except some additional parameters (like goals or limits) that define the way how the game is played. A scenario can contain already built rides or shops. It can also have already visitors. The only thing that I see right now that a scenario doesn't need to contain and a save game must, is history data (sales stats, ratings, guests) - my opinion is that a game should always start from year 1, without any history, even if there are rides already built.

Loading has the problem that you don't know what you got until you load it. Since I would like to support old versions (otherwise all saved games and all scenarios would stop working with each new release), the loader has to be ready for loading an old version.
Thus it may get file data about concepts that has changed a lot in the mean time, or parts of the game state are missing entirely (when we introduce something new).
Secondly, the file may need RCD files that are not at the system or (more likely) a more modern version exists. (Off-topic, but a reliable way to detect RCD file version is thus needed, experience shows that a manually maintained version number will not work.)

A compatibility list can be made. It should specify which file versions can be loaded with specified game version. If those two are not compatible, the saved game cannot be loaded. Until features are added (a shop gets a new item to sell), this shouldn't be the case, but if you make a change like remove a shop (won't happen, is just an example :)), it changes the direction of the gameplay, so it can't/shouldn't be loaded. It could also occur that a scenario would change. In this case, the save game and the scenario wouldn't be compatible to.

[Alberth]

On Wednesday, July 17, 2013 10:47:52 AM UTC+2, mac coder wrote:

Save seems however a good starting point to get an understanding of what is involved in storing a game to a file.
Basically, the game state needs to be serialized onto a file. With save there is only one version you can save, namely the current one. (If you want to be able to load a file in an older version of the game, you need to be able to write older serializations too, but to me it does not seem useful enough to support.)

There is absolutely no need for saving in old formats. The game is free, so everyone can keep it up to date.

Exactly :)

So what to save then? Do we save the set of windows opened at the game interface? Do we save that the user was buying a shop, and had just placed it? I don't know whether we should, but as a starting point, I would think just the current simulation engine state is enough.
Opened windows, loaded RCD files, and their sprites and other data is not saved.

Everything that can't be calculated or loaded from files (sprites, terrain data) and represents the game's current state.

 

Windows should be saved.

Why, your screen is different from mine. Your preferences are different from mine. It makes sense to store these values in a local config file, but copying them into a saved game does not seem so useful.
 

There isn't to much to save anyways (window). You just save the list of opened windows and their screen position, the data displayed in window is part of the selected object (shop, ride, person) anyways.

Pressed buttons, toggle switches? Position of scrollbars? Selected tab in the window?  Tbh I don't see much value in saving this data, it's easy enough to open the window again.
 

For shops, I would only save it's location and current values (price, sales, position...). I don't see any point in saving if a shop was bought - can you explain how you meant it?

The path and shop (and coaster) windows have state. You have to select what to buy, place it in the window, and press some button to actually buy it. So you have to store you selected X shop, were busy placing it, and had the mouse at position (x,y,z), or whatever.

To make it more fun, this state is distributed over the window, the viewport, and a mouse mode. I have a hard time to construct the code when it starts from a closed window. I really wouldn't know how to handle arbitrary restoring of that state, especially in the context of changing code.
 

A scenario save is more difficult. To save a game, you save all game state. For a large part, game state depends on the RCD data available at the time of the save, eg a shop of type X in the game. X and its behavior is defined in an RCD file. If you ensure the same RCD file is available during loading, you can continue the game.
A scenario however can be seen as a template. Some of the things are pre-defined, but other things are still free. One thing I expect to happen is that users will want to play with their favorite set of RCD files. Loading such a scenario thus has to be able to load the saved state, yet allow the user to add/modify some aspects.

A scenario save doesn't seem that very much different to me. It has everything that a save game has, except some additional parameters (like goals or limits) that define the way how the game is played. A scenario can contain already built rides or shops. It can also have already visitors. The only thing that I see right now that a scenario doesn't need to contain and a save game must, is history data (sales stats, ratings, guests) - my opinion is that a game should always start from year 1, without any history, even if there are rides already built.

The big difference is that you want to allow other RCD files added at least, and if possible, also replaced (but not sure if that's possible).

Loading has the problem that you don't know what you got until you load it. Since I would like to support old versions (otherwise all saved games and all scenarios would stop working with each new release), the loader has to be ready for loading an old version.
Thus it may get file data about concepts that has changed a lot in the mean time, or parts of the game state are missing entirely (when we introduce something new).
Secondly, the file may need RCD files that are not at the system or (more likely) a more modern version exists. (Off-topic, but a reliable way to detect RCD file version is thus needed, experience shows that a manually maintained version number will not work.)

A compatibility list can be made. It should specify which file versions can be loaded with specified game version. If those two are not compatible, the saved game cannot be loaded. Until features are added (a

I would like to aim for loading all versions. Adding a new feature is not a problem, old games simply don't have that (but once loaded they do). Removing a feature is also doable, just delete the feature from the file (or rather, just ignore it while loading). Changing a feature means you have to also define how an old game is loaded in that case. A bad solution (but it always works) is to see it as removal of the old version of the feature, and adding of the new version of the feature.
 

shop gets a new item to sell), this shouldn't be the case, but if you make a change like remove a shop (won't happen, is just an example :)), it changes the direction of the gameplay, so it can't/shouldn't be loaded. It could also occur that a scenario would change. In this case, the save game and the scenario wouldn't be compatible to.

 
Having such cuts is bad for continuity. It means all scenarios and savegames that exist become unusable.
Users can throw away all their saved data. Bug reports with savegames becomes unsolvable. Scenario authors have to start over from scratch.

What's worse, it also means you can never even retrieve the contents of the old data anymore. That rollercoaster design you worked at for 4 months, or that beautiful landscape, and that tricky three rollercoasters with paths completely intertwined with each other.... all gone.

I don't see that as a feasible solution.

[devnoname120]

Pressed buttons, toggle switches? Position of scrollbars? Selected tab in the window?  Tbh I don't see much value in saving this data, it's easy enough to open the window again.

I think that currently opened Windows should not be saved, that's overcomplicated and mainly useless (except if the user is currently building a rollercoaster).

The path and shop (and coaster) windows have state. You have to select what to buy, place it in the window, and press some button to actually buy it. So you have to store you selected X shop, were busy placing it, and had the mouse at position (x,y,z), or whatever.

I think it would be just fine not to save it: if a user is stupid enough to save while building a shop, then he'll have to select it again to place it.

I would like to aim for loading all versions. Adding a new feature is not a problem, old games simply don't have that (but once loaded they do). Removing a feature is also doable, just delete the feature from the file (or rather, just ignore it while loading). Changing a feature means you have to also define how an old game is loaded in that case. A bad solution (but it always works) is to see it as removal of the old version of the feature, and adding of the new version of the feature.

I think these will be isolated cases, you can define rules for the objects that were deleted (load something else instead). However, when an object has a different size than it used to have, how would you behave?

Having such cuts is bad for continuity. It means all scenarios and savegames that exist become unusable.
Users can throw away all their saved data. Bug reports with savegames becomes unsolvable. Scenario authors have to start over from scratch.

What's worse, it also means you can never even retrieve the contents of the old data anymore. That rollercoaster design you worked at for 4 months, or that beautiful landscape, and that tricky three rollercoasters with paths completely intertwined with each other.... all gone.

I fully agree, that would disappoint many players.

Old file data that needs to be updated to the current version

I think we should actually load it using a compatibility mode and then when time comes to save it, we can save it to the new format just like any regular game.

 

 

[devnoname120]

By the way, about the format, we can do it two ways:

• Use a generic well-known format: for example XML, allows humans to read it, easy to debug (spotting mistakes would be a piece of cake), and easy to use (many good parsers already exist, for instance TinyXml). The major drawback of this is that it would make savedatas bigger than they can be.
• Make our own format: using a magic number, a header, and the content we can have a fast and optimized parser (because adapted to our use case), the main advantage would be the speed and the size of savedatas, major disadvantage is that would be harder to code, and nasty to debug.

About the savedata version, it is very important to keep it in the header or in a special node of the XML file, otherwise, we will have serious trouble loading old savedatas on newer game versions.

[Alberth]

What do you mean with "an object changes size"? You mean number of bytes for storage in memory, or number of tiles in the game, or ...?

The former will happen, but should be handled by version conversion while loading. The latter should not happen I think. If it changes appearence, it's a new object, and the old one ceases to exist.
If you want to continue to play that game, it means you also need to have the old data files, or there is no information about the old object any more. Data-files are thus also attached to the save game.

As for the data format, I think XML is overrated as storage format. We do not need to be compatible with other systems, or exchange savegames with them.
I'd prefer a binary format. Judging by openttd savegames, which can be 6MB heavily compressed binary data, I would hate to see how much that would be in text.

[devnoname120]

What do you mean with "an object changes size"?

Sorry about the imprecision, I meant the room that an object needs. For example a toboggan had a size of 2x2, and you decide that finally, 2x2 is really too small for a toboggan and thus you change its size to 3x3.

As for the data format, I think XML is overrated as storage format. I'd prefer a binary format.

I agree with you.

We do not need to be compatible with other systems, or exchange savegames with them.

That pops a question in my mind, do you think there will be some kind of Roller Coaster Tycoon savedata importing tool at some point?

Judging by openttd savegames, which can be 6MB heavily compressed binary data

I don't remind the size of Roller Coaster Tycoon savedata, and I don't have a copy of this game right now. Can you please give me some size examples?

[Alberth]

On Wednesday, January 1, 2014 2:04:15 PM UTC+1, devnoname120 wrote:

What do you mean with "an object changes size"?

Sorry about the imprecision, I meant the room that an object needs. For example a toboggan had a size of 2x2, and you decide that finally, 2x2 is really too small for a toboggan and thus you change its size to 3x3.

In my view, that would be a different object. You cannot expect a save game to have room for expanding an existing object. If you want to stay compatible and not make a new object, this is just not possible, imho.
 

As for the data format, I think XML is overrated as storage format. I'd prefer a binary format.

I agree with you.

Technically, there are binary storage formats for xml too, but the goal of xml doesn't match with our needs.
 

We do not need to be compatible with other systems, or exchange savegames with them.

That pops a question in my mind, do you think there will be some kind of Roller Coaster Tycoon savedata importing tool at some point?

I looked for format descriptions a few years back. Other than the graphics format, and a few bytes here and there in the save games, I haven't been able to find information on it.
Without it, making such a tool is not possible. You'll probably also need to convert all graphics and object properties as well. FreeRCT doesn't make it impossible (or if it does, it's fixable as we have the source code), but I think you'll have a hard time to find out all properties.
 

Judging by openttd savegames, which can be 6MB heavily compressed binary data

I don't remind the size of Roller Coaster Tycoon savedata, and I don't have a copy of this game right now. Can you please give me some size examples?

Just like you cannot compare original Transport Tycoon Deluxe with OpenTTD, I think size of original Roller coaster files has little meaning. Inevitably, FreeRCT will start to expand with new and memory eating features, and file size will grow. In the case of OpenTTD, just making a map of max 2048*2048 instead of max 256*256 exploded the number of tiles to store by a factor 64.
Something like that will also happen in FreeRCT. I don't see 128*128 as enough for all users (even though it would be nice if it was).

[devnoname120]

I think you'll have a hard time to find out all properties.

Yes, I think it would need a great time to reverse the whole savedata system (maybe we could ask Chris Sawyer to hand us some documentation about it?)

Something like that will also happen in FreeRCT. I don't see 128*128 as enough for all users (even though it would be nice if it was).

Especially for a co-op mode, if ever made.

[Alberth]

On Wednesday, January 1, 2014 8:31:13 PM UTC+1, devnoname120 wrote:

I think you'll have a hard time to find out all properties.

Yes, I think it would need a great time to reverse the whole savedata system (maybe we could ask Chris Sawyer to hand us some documentation about it?)

The first thing we need is the ability to save and load games ourselves. Once we have that, you can think about compatibility with other software.

Mr Sawyer is not going to give that information. First of all, he does not own that information, it was owned by Atari. Atari is bankrupt now, so the owner is now very much unclear. Second, I don't believe Mr Sawyer would like us interfacing with his game.

Last but not least, I am not terribly interested into compatibility with that game. So much work, just to be able to do what the program can already do on its own (namely saving and loading games). Adding this capability will also add restrictions to what we can change, at some point.

If you really want to realize this, maybe the best solution is to write a converter to the FreeRCT savegame file format (which currently is non-existent) ?

[Hazzard]

I assume that the savefile would have some sort of per-tile array save data, but I was wondering if rollercoasters will be saved by their parts per-tile or as a group. I'm also interested in how FreeRCT stores objects that take up more then one tile (ferris wheel, etc, including rollercoasters).

[Lord Aro]

If you really want to realize this, maybe the best solution is to write a converter to the FreeRCT savegame file format (which currently is non-existent) ?

Or just give FRCT the ability to load old savegames, like how OTTD does it currently

It seems to me that a converter will be unfeasible as, like OTTD, i expect the savegame file format to change somewhat regularly 

[Alberth]

@Hazzard:
Currently, you cannot save a game. Most is thus not yet decided.

Currently, I think the rides, people, and other 'objects' have to be saved, and the world voxels only need to be saved for as far as the data is not available in the objects, to avoid consistency problems.

For loading, it's probably better to first load the world voxels before loading the ojects.