Eventtools reboot
Greg Turner
Now we've used EventTools a few times, we've discovered these significant issues:
1) Generating EventOccurrences far into the future is slower than retrieving them from the database would be. However we don't want to store infinite occurrences in the database, though storing thousands would be OK. Also, occurrences are persisted as soon as you view them in admin which is less than ideal.
2) We need PURLs for, and increasingly to FK to, EventOccurrences. For example, we often want iCal versions of EventOccurrences, or we might want to indicate which users are attending a particular occurrence. The PURLs/FKs should be constant, even if the date and time of the Occcurrence changes.
3) The current structure is difficult to understand and use, for Content editors and coders alike.
* most event sites are walnut-shaped. Current EventTools is a sledgehammer.
* many fks to traverse, for often unclear purposes
* test coverage isn't great
* unpersisted occurrences is not Djangonic
* deleting occurrences doesn't do what you expect
* we're exposing events and generators to the front-end, where simple navigation depends on occurrences.
* The admin interface is rather hacky and incomplete
* Ambiguous definitions (e.g. is_varied - is that time or EventVariation)?
* Slightly too many 'voodoo rules' in there, particularly relating to time-shifting and variations.
4) Unless you are very careful, EventVariations are generally not included in queries (e.g. 'show all events in the lecture theatre' will not show eventoccurrences with variations that have been moved to the lecture theatre).
5) We have deviated sufficiently from django-schedule to be unable to merge changes, yet there is still API cruft and backwards-compatibility that makes the API difficult to document and test.
6) There are common navigation patterns for events and occurrences that are reimplemented in each project, because of the issues described above. The project would benefit from sample views and urlconfs that implement common navigation patterns. Navigating Events, not Occurrences, introduces problems, like displaying the dates, pointing towards possible variations, etc.
7) There are too many different ways to query. Event.occurrences_between, Month.occurrences, Event.Occurrences.between, etc. None of these take variations into account.
8) shell_plus produces an warning when it can't find the injected classes.
PROPOSAL:
I think eventtools should be reconstructed from the ground up (neweventtools?), abandoning it's django-schedule heritage, with the following backwards-incompatible changes:
ALL OCCURRENCES ARE PERSISTENT
------------------------------
All occurrences would be stored in the database. This would make PURLs and queries quick and easy, and simplify the Occurrence model and logic. Generators would require either an end date, or a max number of new occurrences to generate (reset to 0 after they have been created). When you change the date/time/rule of a generator, all its occurrences can be deleted and recreated. It is possible to 'detatch' an occurrence from a generator (by nulling the FK), so it is not affected by the generator.
When you edit an event, you see inlines for both Occurrences and Generators. Most of the time, you can just create one-off occurrences directly. Otherwise you can generate them by saving an inline generator. If you change an occurrence it is detatched from its generator.
You can delete occurrences too, though if you modify the generator, the deleted one will come back. This means that, rather than make a rule that excludes Easter, you can just delete the occurrences by hand. This will be easier for most editors on smaller sites.
EVENT OVERRIDES ARE JUST OTHER EVENTS
-------------------------------------
(Deprecating 'variation' terminology, as it doesn't make sense to content editors - are all concerts variations of each other?)
To create an override, you'd first follow the same process as currently in NFSA - from the occurrence, choose an override from the available list, or add a new one. The new override would need to be prepopulated with the contents of the original event.
However, when you save the override event, the occurrence with the override is 'moved' from the original event to the new event, and linked to the original event through a second FK. Other occurrences can be moved across in the same way, or new occurrences can be added to the override event.
Changes to the original event cascade, where appropriate, to override events.
Advantages of this approach:
* Easy to teach - the models and relations are simpler, clearer, less magical and more Djangonic.
* Easy to query - overrides and their occurrences work in the same way as originals.
* Easy to find/see/create/delete overrides. None of this MergedObject voodoo.
* Easy to attach several overrides to an event, or the same override to several occurrences.
* Can do cool stuff like
QUERIES THROUGH AN OBVIOUS API
------------------------------
Let's make it so all queries go through Event, then we don't need a complex Model or manager, or bizarre injection code.
All queries run through the Event manager e.g.
Event.objects.filter(venue=the_library) # would return all Events (or variations) in the library.
To get occurrences, you take your event queryset and add a occurrences* method, which returns occurrences.
Event.objects.filter(venue=the_library).occurrences(cancelled=True)
Event.objects.filter(venue=the_library).occurrences_between(dt1, dt2, cancelled=False)
Event.objects.filter(venue=the_library).occurrences_this_weekend()
Event.objects.filter(venue=the_library, cancelled=True).occurrences_from_GET_params(request.GET, cancelled=True)
concievably,
Event.objects.occurrences_this_weekend().filter(venue=the_library)
might work (like django's values()).
URLS AND VIEWS
--------------
Let's assume EventOccurrences are always presented in chronological order, grouped by date. Multi-day occurrences (like a camp) are only displayed on the first day (unless they have seperate occurrences on each day).
That means we can define a pattern of URLs and views for:
* navigating by date range, showing date range on calendar, etc.
* rendering events as cancelled/fully booked
* viewing an individual occurrence, and getting an ical download.
DEMO
----
There are useful patterns emerging that depend on a fleshed-out model
* faceting events, and getting ical/rss feeds for the facets
GOING FORWARD
-------------
It is worth moving quickly, since we're working on a few sites that use this, and are running into the problems outlined above. The first step is to draft out the API and write tests, and send for approval. Second step is to adapt a site in development to this. Implementation should be straightforward - most of the difficult code is already written and needn't change.
The main impediments are:
* Can we live without infinitely repeating events? I say yes, since we are already living without them since they are so slow. The new alternative would be to generate n new occurrences at a time, when a generator is saved.
* Migration for our current large site will be difficult, but not impossible. EventVariations need to be coerced into being other events, and the occurrences moved across.
--. --- .. -..- -.-. --- -. ... --- .-. - .. ..- --
Dr Greg Turner
Director, the Interaction Consortium
http://interaction.net.au
Phone: +61 2 8060 1067
skype: gregturner
Follow us on twitter:
http://twitter.com/theixc
http://twitter.com/gsta
Greg Turner
Still working on views/urls.
TEST EVENTS
"""
When you create an Event, you get an Occurrence class.
You can attach occurrences to events with a FK called 'event'.
You can query the occurrences for an event by date(datetime) range etc.
e.occurrences(status='cancelled')
e.occurrences_between(dt1, dt2, status=None))
You can query occurrences for an event queryset by date range etc.
Event.objects.filter(venue=the_library).occurrences(status='cancelled')
Event.objects.filter(venue=the_library).occurrences_between(dt1, dt2, status=None))
And dates (clamped to time.min and .max).
Event.objects.filter(venue=the_library).occurrences_between(d1, d2, status=None))
Occurrences are sorted by date, then time by default.
If date_order = -1 then occurrences are sorted by reverse date, then time.
There are shortcut occurrence queries, which define date_order and date range as appropriate:
e.occurrences_this_weekend()
Event.objects.filter(venue=the_library).occurrences_this_weekend()
today(forthcoming_only=True)
today(forthcoming_only=False)
on_day()
tomorrow
yesterday
this week(forthcoming_only)
next week
last week
this weekend(forthcoming_only)
next weekend
last weekend
month(forthcoming_only)
etc.
year(forthcoming_only)
etc.
fortnight(forthcoming_only)
etc.
forthcoming
etc.
most_recent (come back)
Weeks can start on sunday, monday, etc. This is defined in settings.
a (GET) dictionary, containing date(time) from and to parameters can be
Event.objects.filter(venue=the_library, cancelled=True).occurrences_from_GET_params(request.GET, 'from', 'to')
Events are in an mptt tree, which indicates parents (more general) and children (more specific).
Events have a char field called 'difference' that can be used to characterise the difference between parent and child.
The custom admin occurrence view lists the occurrences of an event and all its children. Each occurrence shows which event it is linked to.
The custom admin view can be used to assign a different event to an occurrence. The drop-down list only shows the given event and its children.
When you save a parent event, every field cascades to all children events (and not to parent events).
If the child event has a different value to the original, then the change doesn't cascade.
If we create a new child, it takes all of its parents' fields (but not occurrences or generators).
When you view an event, the diff between itself and its parent is shown, or fields are highlighted, etc, see django-moderation.
"""
TEST OCCURRENCES
"""
Occurrences have an fk to 'event'
Occurrences must have a start datetime and end datetime. (We might have to make a widget to support entry of all-day events).
If start.time is 'ommitted', it is set to time.min.
If end is omitted, then:
end.date is start.date, then apply rule below for time.
If end.time is 'ommitted' it is set to start.time, unless start.time is time.min in which case end.time is set to time.max.
If an occurrence's times are min and max, then it is an all-day event.
End datetime must be >= start datetime.
Occurrences have a status (e.g. cancelled, fully_booked).
Occurrences can span more than one day.
Occurrences that span more than one day only appear on the first day. (to have occurrences appear on several days, create several occurrences).
Occurrences that span more than one day are not matched on queries for day+1 days (to have occurrences appear on several days, create several occurrences).
Occurrences have a duration.
Occurrences have a robot description.
Occurrences are sorted by date.
"""
TEST GENERATORS
"""
Occurrences can be generated by a Generator. And event can have many Generators.
A Generator has event, start datetime, end datetime, rule, repeat_until, and 'exceptions'.
exceptions is a JSONfield of start/end datetime pairs that the generator skips over if it was about to generate them.
A generator generates occurrences by by repeating start/end datetimes according to the rule, until repeat_until is about to be exceeded.
A generator without a rule generates one occurrence.
If the start time of a generator is time.min and the end time is time.max, then the generator generates all_day occurrences.
If the start time of a generator is time.min and the end time is time.max and the repetition rule is hourly, then the generator generates one all_day occurrences followed by (e.g. 23 timed occurrences).
Occurrences are saved to the database.
If a generated occurrence exists in the database already, it is not overwritten.
It is not valid to have a repeat_until without a rule.
(We need a special admin widget for entering the datetimes.)
If repeat_until is omitted (and rule is set) then repetitions are created upto a preset period into the future. The preset is in settings.
(This preset period is continually updated)
Every time a generator is saved, it does its generating.
Every time an event is saved, generators with a rule but no repeat_until do their generating.
Occurrences are saved to the database, and have an FK to the generator that did so. The FK can be set to None so that an occurrence can be detatched from a generator.
When you change a generator and update existing occurrences:
* we update the times of occurrences and exceptions to match the generator's times.
* we generate occurrences as normal.
This means that data is never lost except through an actual delete.
If an occurrence is deleted, that occurrence's datetimes are added to the generator's exceptions. That prevents deleted occurrences from being regenerated.
If an occurrence's generator FK is changed, that occurrence is added as an exception to the generator. That prevents reassigned/detatched occurrences from being regenerated.
If the generator's datetimes are modified, it is possible to update occurrence datetimes (where unmodified).
And exception datetimes.
If the generator's dates are modified, it is possible to update occurrence dates (where unmodified).
And exception dates.
If the generator's times are modified, it is possible to update occurrence times (where unmodified).
And exception times.
If the generator is modified from being all-day to not all-day, it is possible to update (trivially).
And exceptions.
If the generator is modified from being not all-day to all-day, it is possible to update (trivially).
And exceptions.
If a generator's rule is changed, then no times are changed (unless as above), and no occurrences are deleted but any 'extra' occurrences are still generated.
If a generator's repeat until value is changed then then no times are changed (unless as above), and no occurrences are deleted, but any 'extra' occurrences are still generated.