TI-58: Exclusions and ids

[Thomas Lauf]

Hi,

some time ago I posted some questions that came up during my analysis on
TI-58 (delete command is not always deleting). However I never got an
answer to them (maybe post them on the Q&A site? But these are
architectural questions, not user questions...).

After some further analysis of TI-58, I came to the conclusion that one
problem is that the summary command is lying about the ids. Consider the
following scenario:

- Exclusions defined in timewarrior.cfg:

exclusions.monday = 12:00-13:00
exclusions.tuesday = 12:00-13:00
exclusions.wednesday = 12:00-13:00
exclusions.thursday = 12:00-13:00
exclusions.friday = 12:00-13:00
exclusions.saturday = 12:00-13:00
exclusions.sunday = 12:00-13:00

- I start a task at 2017-03-08T06:00

timew start 2017-03-08T06:00

- I look at my summary two days later and I see

Wk Date Day ID Tags Start End Time Total
W10 2017-03-08 Wed @4 6:00:00 12:00:00 6:00:00
@3 13:00:00 0:00:00 11:00:00 17:00:00
W10 2017-03-09 Thu @3 0:00:00 12:00:00 12:00:00
@2 13:00:00 0:00:00 11:00:00 23:00:00
W10 2017-03-10 Fri @2 0:00:00 12:00:00 12:00:00
@1 13:00:00 - 0:56:14 12:56:14

52:56:14

This summary table suggests that there are 4 intervals, however, my
data-file only contains one (open) interval.

I think a more honest display would be this:

Wk Date Day ID Tags Start End Time Total
W10 2017-03-08 Wed @1 6:00:00 12:00:00 6:00:00
@1 13:00:00 0:00:00 11:00:00 17:00:00
W10 2017-03-09 Thu @1 0:00:00 12:00:00 12:00:00
@1 13:00:00 0:00:00 11:00:00 23:00:00
W10 2017-03-10 Fri @1 0:00:00 12:00:00 12:00:00
@1 13:00:00 - 0:56:14 12:56:14

52:56:14

All intervals have the same id.

Sidenote: timew delete @1 still does not work, but you are no longer
mislead to think that there are 4 intervals in the database.

Another approach would be to immediately write completed/closed
intervals into the database, i.e. the start command above would not
write one open interval, 3 closed and one open.

Comments welcome.

Regards,
Thomas


PS: And here is the old email with some other details and questions that
came up with TI-58:

# Is "timew delete @1" equal to "timew cancel"?

Imagine defined daily exclusions from 12:00-13:00
The command "timew start today" will write 1 interval into the database,
but display two in "timew export" and "timew summary":

Wk Date Day ID Tags Start End Time Total
W10 2017-03-06 Mon @2 foo 0:00:00 12:00:00 12:00:00
@1 foo 13:00:00 - 7:13:58 19:13:58

19:13:58

"timew cancel" would remove the open interval, i.e. remove both, "timew
delete @1" would (be expected to) remove the last (open) interval. Or not?

# The delete command serializes the interval to be deleted and searches
for it in the respective datafile. This depends on defined exclusions

- Defined exclusion 12:00 - 13:00
- Create open interval: timew start today foo
=> Entry in datafile: inc 20170305T230000Z # foo
=> Result of "timew export":
[
{"start":"20170305T230000Z","end":"20170306T110000Z","tags":["foo"]},
{"start":"20170306T120000Z","tags":["foo"]}
]

- The delete command would search for "inc 20170306T120000Z # foo"
which does not exist in the datafile

My proposed fix for TI-58 would be to expand the exclusion when the
interval is started and write them to the database. This would also
alter (fix?) the following scenario (performed at some point in the
afternoon):

timew config rc.exclusions.<current weekday>=12:00-13:00*
timew start today
timew summary

Wk Date Day ID Tags Start End Time Total
W10 2017-03-06 Mon @2 0:00:00 12:00:00 12:00:00
@1 13:00:00 - 7:33:30 19:33:30

19:33:30

timew config rc.exclusions.<current weekday>=11:00-14:00*
timew summary

Wk Date Day ID Tags Start End Time Total
W10 2017-03-06 Mon @2 0:00:00 11:00:00 11:00:00
@1 14:00:00 - 6:34:38 17:34:38

17:34:38

The interval is altered by the change in the configuration. Is this
expected/desired? If yes, you cannot (currently!) change the exclusion
for the current weekday without possibly altering the current
timetracking...

[Paul Beckingham]

Hi Thomas,

You’re right, I didn’t answer you. I see your message from Monday. That was my big libshared fix day - I ignored everything. Sorry.

I’ll provide detailed answers later today.

P

> --
> You received this message because you are subscribed to the Google Groups "taskwarrior-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to taskwarrior-d...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Paul Beckingham]

> However I never got an
> answer to them (maybe post them on the Q&A site? But these are

> architectural questions, not user questions…).

Agreed, architectural questions belong here.

> This summary table suggests that there are 4 intervals, however, my
> data-file only contains one (open) interval.
>
> I think a more honest display would be this:
>
> Wk Date Day ID Tags Start End Time Total
> W10 2017-03-08 Wed @1 6:00:00 12:00:00 6:00:00
> @1 13:00:00 0:00:00 11:00:00 17:00:00
> W10 2017-03-09 Thu @1 0:00:00 12:00:00 12:00:00
> @1 13:00:00 0:00:00 11:00:00 23:00:00
> W10 2017-03-10 Fri @1 0:00:00 12:00:00 12:00:00
> @1 13:00:00 - 0:56:14 12:56:14
>
> 52:56:14
>
> All intervals have the same id.

Yes, that would indeed make the id match reality. But that wasn’t the original intent. Here’s a scenario that illustrates what was *intended*, and clearly not properly implemented:

Given these exclusions:
Thursday = <9:00 >17:00
Friday = <9:00 >17:00

Assume this command, is run at noon on Thursday:
timew start tag1

Then obviously there is one open interval written to the data file. It is intended that open intervals are just a start time, and exclusions are used to calculate total tracked time. When an interval is closed, that one entry gets modified, and turns into N closed intervals, where N >= 1. It the interval is closed on Friday at noon, I expect these intervals to be stored:

20170309T120000Z - 20170309T170000Z # tag1
20170310T090000Z - 20170310T120000Z # tag1

In this situation, there is @1 and @2. No problem.



Now in the other case, suppose the interval remains open at noon on Friday, which means one stored entry. When the summary command (or export, or anything else) is run, it is intended that two intervals are shown, the first, @2 being closed, and the second, @1 being still open, like this:

20170309T120000Z - 20170309T170000Z # tag1
20170310T090000Z - # tag1

To say it another way, open intervals are represented by one entry in the data file, and N simulated intervals for the purposes of reports, addressable in the usual way, @1, @2. If a simulated interval is modified in some way, then the simulated intervals, after adjustment, should be written to the data file. In this example, a closed followed by an open interval are written.

Saying it yet another way: Open intervals span exclusions. Reports are always shown a mix of real intervals, and simulated intervals if there are any open intervals at the time.

> Sidenote: timew delete @1 still does not work, but you are no longer
> mislead to think that there are 4 intervals in the database.
>
> Another approach would be to immediately write completed/closed
> intervals into the database, i.e. the start command above would not
> write one open interval, 3 closed and one open.

Yes, this is the intended behavior. Something is clearly not working, otherwise the bug wouldn’t exist. I believe the simulated intervals are correctly created (otherwise “summary” would be wrong), but the functions that modify data are where the problem lies. I must have simply not tested this, and assumed I was finished when all the tests passed.

> Comments welcome.
>
> Regards,
> Thomas
>
>
> PS: And here is the old email with some other details and questions that
> came up with TI-58:

I’m going to answer these, although it is be covered by the above. I’m answering for additional clarity.

> # Is "timew delete @1" equal to "timew cancel”?

No, but it’s close. If @1 is open, then @2 (etc) may be simulated, and will need to be written. But the overall result in both cases is that @1 goes away.

> Imagine defined daily exclusions from 12:00-13:00
> The command "timew start today" will write 1 interval into the database,
> but display two in "timew export" and "timew summary":
>
> Wk Date Day ID Tags Start End Time Total
> W10 2017-03-06 Mon @2 foo 0:00:00 12:00:00 12:00:00
> @1 foo 13:00:00 - 7:13:58 19:13:58
>
> 19:13:58
>
> "timew cancel" would remove the open interval, i.e. remove both, "timew
> delete @1" would (be expected to) remove the last (open) interval. Or not?

Not both. @1 would be removed, @2 is simulated, and would need to replace the written open interval.

> # The delete command serializes the interval to be deleted and searches
> for it in the respective datafile. This depends on defined exclusions
>
> - Defined exclusion 12:00 - 13:00
> - Create open interval: timew start today foo
> => Entry in datafile: inc 20170305T230000Z # foo
> => Result of "timew export":
> [
> {"start":"20170305T230000Z","end":"20170306T110000Z","tags":["foo"]},
> {"start":"20170306T120000Z","tags":["foo"]}
> ]
>
> - The delete command would search for "inc 20170306T120000Z # foo"
> which does not exist in the datafile

Correct. This is the bug.

> My proposed fix for TI-58 would be to expand the exclusion when the
> interval is started and write them to the database. This would also
> alter (fix?) the following scenario (performed at some point in the
> afternoon):

Close. Described above.

> timew config rc.exclusions.<current weekday>=12:00-13:00*
> timew start today
> timew summary
>
> Wk Date Day ID Tags Start End Time Total
> W10 2017-03-06 Mon @2 0:00:00 12:00:00 12:00:00
> @1 13:00:00 - 7:33:30 19:33:30
>
> 19:33:30
>
> timew config rc.exclusions.<current weekday>=11:00-14:00*
> timew summary
>
> Wk Date Day ID Tags Start End Time Total
> W10 2017-03-06 Mon @2 0:00:00 11:00:00 11:00:00
> @1 14:00:00 - 6:34:38 17:34:38
>
> 17:34:38
>
> The interval is altered by the change in the configuration. Is this
> expected/desired? If yes, you cannot (currently!) change the exclusion
> for the current weekday without possibly altering the current

> timetracking…

The interval would be altered by config changes only if the interval is open. Once closed, it’s fixed.

Hope this clarifies and doesn’t instead add another misty layer.

P

[Thomas Lauf]

Hi Paul!

Based on your reply I tried formulate some statements to define the
correct behaviour of TimeWarrior concerning ids and exclusions. This
should help me with the fix of TI-58...

1) Ids reference intervals and are assigned after exclusions but
before filters are applied
2) Intervals can be open or closed
3) There is only one or none open interval
4) There are real and simulated intervals
5) The database only contains real intervals
6) Simulated intervals only exist, if an open interval spans over
exclusions
7) Exclusions are not applied to closed intervals
8) Simulated intervals are converted to real intervals if an open
interval is closed
9) Cancelling active time tracking converts all simulated intervals to
real intervals and removes the last interval
10) Deleting a simulated interval converts all simulated intervals to
real intervals and deletes the referenced one

I hope these cover all cases...

Regards,
Thomas

--
Dr. Thomas Lauf * thoma...@tngtech.com * +49 174 3180 086
TNG Technology Consulting GmbH, Betastr. 13a, 85774 Unterföhring
Geschäftsführer: Henrik Klagges, Christoph Stock, Dr. Robert Dahlke
Sitz: Unterföhring * Amtsgericht München, HRB 135082

[Paul Beckingham]

Hi Thomas,

There is one item I think is wrong, (9), details below.

> Based on your reply I tried formulate some statements to define the
> correct behaviour of TimeWarrior concerning ids and exclusions. This
> should help me with the fix of TI-58...
>
> 1) Ids reference intervals and are assigned after exclusions but
> before filters are applied

Correct. Ids are 1-based array indexes from the end of the list of simulated intervals.

> 2) Intervals can be open or closed

Correct.

> 3) There is only one or none open interval

Correct.

> 4) There are real and simulated intervals

Correct. If there are any simulated intervals, they will always have the lowest id[s].

> 5) The database only contains real intervals

Correct. Closed intervals, and possibly one open interval, which would be the last line in the most recent file.

> 6) Simulated intervals only exist, if an open interval spans over
> exclusions

Correct.

> 7) Exclusions are not applied to closed intervals

Correct. They can only be modified using one of the commands that specifically modifies intervals (move, split, join, modify, shorten …). Changing configuration has no effect on closed intervals.

> 8) Simulated intervals are converted to real intervals if an open
> interval is closed

Correct.

> 9) Cancelling active time tracking converts all simulated intervals to
> real intervals and removes the last interval

No. The entire open [non-simulated] interval is [supposed to be] removed. It’s not, and this is the bug.

If you think the intended behavior is wrong in this case, let’s talk, you may have put more thought into this than I have at this point.

> 10) Deleting a simulated interval converts all simulated intervals to
> real intervals and deletes the referenced one

Correct. This is what it should do. I don’t think it does that, based on the bugs. I’m not sure though.


> I hope these cover all cases…

Yes, I can’t think of another case. Thanks for doing this. Perhaps your numbered rules should be pasted into code somewhere.

P

[Thomas Lauf]

Hi Paul,

I changed 9) to

9) Cancelling active time tracking removes the open interval, thus
removing all simulated intervals (if any)

I wonder if we can add these rules somewhere to the documentation under
https://taskwarrior.org/docs/timewarrior/

Regards,
Thomas

[Paul Beckingham]

I wonder if we can add these rules somewhere to the documentation under
https://taskwarrior.org/docs/timewarrior/

Done.

https://taskwarrior.org/docs/timewarrior/interval_behavior.html

P