New Caches, time and Events (obs, db purge needed)

[Griatch]

If you haven't read the commit logs, you will need to purge your
database with rev819 due to the persistent cache additions.

... And I really /do/ recommend you purge your database and start over
from scratch rather than just resyncing. The initial_setup changed has
changed - it is now once and for all initializing the cache, and is
responsible for the initial creation and launching of the persistent
system events (this is no longer done every time the server starts).
This means that the normal server startup now takes care of loading
caches and setting up events, which will not be there if you didn't
run the new initial_setup ... so yeah, you will most likely need to
start from scratch or you'll be in a world of hurt.

... So what is this about now? Well, Evennia has seen a few small but
critical changes previously that led to rather large repercussions.
The foremost of those was the object-caching of player objects. Before
you could do things like object.myvar = data and be "reasonably" sure
that you could get that data back a few minutes later. This was more
of a quirk than a real cache though, and as part of a move to make
sure the objects were always in sync with the databse, this fluke is
now gone completely (so obj.var = data is now a sure way to throw away
your data). Evennia's data model is fully persistent and you should
use obj.set_attribute() and obj.get_attribute_value() whenever you
want to store objects persistently on objects.

However, there ARE some valid uses for wanting a more temporary cache
that is not directly tied to the database. For example you might have
a complex near-realtime combat system (hi rcaskey!) that wants to
store and shuffle lots of data every second, and remember most of it
at least for a while. You don't want to save all that in the database
at every moment, you don't really care to keep it for posterity and
database access is an overhead that might be too much to take if you
have many users and/or much data to store per unit time.

For this reason Evennia now has a volatile cache. This is simple to
use: You access it through the object model using dot notation:

object.cache.myvar = data

This data has no "contact" with the database (and thus no overhead)
and you can get it back just as easily with

data = object.cache.myvar

The cache is guaranteed to be consistent across a server session and
different objects will allow separate non-colliding cache storage
(there are ways to build your own caches too without having to go
through the objects if you want, I might cover that in the wiki). The
volatile cache will however not survive a reboot (which is the whole
meaning of 'volatile' after all).

With rev819 Evennia now also has what I call a persistent cache
(pcache). This is a sort of middle-way between the volatile cache and
Evennia's fully persistent object model. The pcache works in a similar
way as the volatile cache:

obj.pcache.myvar = data
data = obj.pcache.data

The difference is that pcache is regularly saved to disk at an
interval defined by settings.CACHE_BACKUP_INTERVAL. It is re-loaded
back when the server is re-started. So this is a sort of save-point
scheme that you can use if you want, the denser your do your backups
the better "security" you'll have, but the more you might just as well
have used the full object model instead.

The pcache allows for rapid cache access while retaining persistence
across sessions. The main use for this is for events, which
traditionally were completely volatile but which now can take a
variable "persistent=True" in order to store it to pcache and thus get-
backed up (also their current times will be backed-up so they will
restart at (roughly) the same time as they left off). The main server
events use this. Another use is a global time counter, which now
resides in the src/gametime.py module. This implements an in-game time
(seconds the server has been running actively since first startup)
which can be used for all sorts of in-game time measuring (passage of
seasons etc), also across server downtimes (the downtime doesn't
count. Also, persistent events need syncing with a persistent time in
order to work correctly across downtimes, this was the original reason
for creating one).

Speaking of events, those are now much more stable and allows for
proper deletion without any confusions and multiples showing up (I
hope). This allows for things like events that just repeat a certain
number of times before they cleanly delete themselves. The default
event has a "repeats" property specifically for this particular use
and the @ps command shows some more information to reflect this.

I'll try to get down to writing up examples for most of this in the
wiki one of those days ... for now I hope that this gave an overview.
And as usual, more testing is needed by many more people to iron out
those pesky unexpected situations ...
.
Griatch

[George Oliver]

On Nov 22, 2:11 pm, Griatch <gria...@gmail.com> wrote:
> If you haven't read the commit logs, you will need to purge your
> database with rev819 due to the persistent cache additions.

This is more a curiosity question than anything, but what is the
reason for using three storage models? Is it to limit the number of
calls to the persistent DB?

[Griatch]

Yes, this is the main reason, as mentioned there are situations when
you want to store temporary data but don't care about persistence
between reboots - you just want to keep it from moment to moment. Many
skill systems or combat systems are like this - they need plenty of
stuff remembered while they are used, but the data being shifted
around, while necessary at the moment, is not important in itself -
it's just an uncesseray overhead to store it only to re-read it to/
from disk every moment.

The cache is also a way to store things without actually tying it to a
particular object model at all (this can be achieved by using the
cache access functions). This can be useful when you implement big
autonomous systems.

Lastly, the pcache system can easily be used to build your own custom
persistent cache scheme if you want (so you can have a fully
persistent, separate storage scheme if you want). So to set up - say,
a separately running and persistent global economic system that
handles its state independent from in-game objects, this is a good
place to build from.
.
Griatch

[Griatch]

... Oh, and I should probably repeat, that you don't have to use any
of the caches at all in your coding. The default way to use Evennia up
to now has been to use the full persistent model and that works fine
for most types of MUDs. It's all about choices.

.
Griatch

On Nov 23, 2:46 am, George Oliver <georgeolive...@gmail.com> wrote:

[Griatch]

The wiki has been updated with more info about the caches.
.
Griatch