Days operator offset by 1 day in timezones west of GMT

[Hubert]

Hello,

I'm aware this topic was raised many times before, so my apologies for the sprawl.

TiddlyWiki treats all dates in fields as UTC, which makes sense. We have the view widget to 'translate' that UTC date to our local time (depending on time zone, daylight saving time offset etc). All good so far.

However, the days filter operator by definition skips any time portion of a date field and only evaluates its date portion. At the same time, the days operator still treats date fields as UTC, which results in a full day negative offset of the interpreted date in timezones east of GMT.

To illustrate my point:

<$list filter="[all[tiddlers]tag[task]days:date[0]]">
{{!!title}}<br>
</$list>

This will list all tasks that have today's date in its date field. So, on 16 February 2019, all tasks dated 20190216 will be listed by the above list filter, provided that your timezone is GMT or ahead (Asia, Australia etc). In timezones behind GMT (The USA, South & North America, etc), the filter will not list any 20190216 tasks on 16 February 2019, because the days operator has offset the UTC date of 20190216 by whatever number of hours behind the midnight of 20190216 that match the relative UTC offset of that timezone. This means that if you're in, say, New York, 20190216 will be offset from UTC by 5 hours and result in 201902151900 and the date is now interpreted as 20190215 (time is skipped), so essentially 1 full day.

As far as I'm aware and if my understanding is correct, there's no way to work around this behaviour using the days operator but I would still like to hear about your ideas.

What I'd like to suggest as well is that the date operator only works on full days and doesn't produce a negative offset from UTC in timezones east of Greenwich (the negative offset will always amount to 1 full day, because time is ignored by this operator). If we're working with full days I don't believe we should need to factor in any time offsets.

I understand the design principle behind interpreting all date fields as UTC and I'm in favour of it, when it comes to time. But I also think that using the days filter operator may result in unexpected behaviour with no workable or easy workaround if the local time is behind GMT.

I'd be happy to hear about your thoughts or suggestions to work around this problem.

Thank you,

Hubert

[Mark S.]

I think you're talking about using your own dates, in the short form e.g. 20190216. Because you don't use all the digits of the time, TW pads the number and assumes that it is 20190216000000000 or midnight GMT. Since  at midnight GMT, it's still the 15th on days west of timezone GMT, the days operator doesn't show it.

A trick, which I haven't thoroughly tested, is to add "9" after the bespoke date:

201902169

Whether that causes trouble on the 17th, I don't know.

-- Mark

[TonyM]

Mark,

I wonder if it makes sence setting a date only actualy to midday, yyyymmdd1200

This possibly can be done within the date format. If its 00:04 do we realy think 5 mins ago was a day ago when a day implies 24 hours?

Regards
Tony

[Mark S.]

Are we still talking about rolling our own dates?

I think 12 would work almost everywhere (there might be some fringe cases).

 

Yes a single minute can make a difference for bills, taxes, assignments. Which is why the deadline time is often at midnight, so there's a great deal of overlap with actual waking hours. What annoys me is a certain cloud-based service that resets subscriptions per a different timezone, even though I'm in the same timezone as the corporate office. Apparently wherever the server is located becomes the time standard.

-- Mark

[Hubert]

Thank you for your responses, Mark and Tony.

 

I think you're talking about using your own dates, in the short form e.g. 20190216. Because you don't use all the digits of the time, TW pads the number and assumes that it is 20190216000000000 or midnight GMT. Since  at midnight GMT, it's still the 15th on days west of timezone GMT, the days operator doesn't show it.

Yes, that's exactly what I was talking about.

A solution with yyyymmdd1200 or yyyymmdd9 might work in most cases but is not universal. The reason being, time zones stretch up to -12 hours from (behind) UTC if we move west of GMT (so an added offset of 1200 could work here), however the time zone farthest to the east of GMT has an offset of +14 hours, which could then shift the date once again, this time in the other direction (even part of New Zealand is +12:45 as per this Wikipedia article; so, it's not just a case of scarcely populated islands).

Adding artificial offsets to dates has no 'safe margin' because the sum of all the time zone offsets appears to be 26 hours. Either way, there is going to be an inaccurate date interpretation by the days filter operator, whether that's 1 day behind or 1 day ahead.

Thanks,

Hubert

[S. S.]

We should rename the operator to days±1
Cheekily yours!

[TonyM]

I believe there is a workaround.

The problem is now not the compare date if it is yyyymmdd1200

I will see what I can find
Regards

[Mark S.]

I did mention there were fringe cases. The kludge covers about 99% of real life cases.  But the tweak would be, if you happen to be someone in one of those fringe areas who wants to roll their own dates then use an offset value that puts you back on the same day. For instance, if you live in TZ +14, you can use any offset hour between 15 and 23, so just use 15. Those really are the outer fringes. If you live in that part of NZ with TZ+12:45 then you can use offset 13.

-- Mark

[Hubert]

Thanks Mark. I don't have an issue with adding a time offset to my own dates to prevent this, even though I'd need to rewrite my js macros that I use for pushing my recurrent tasks forward.

The only reason I've raised the issue here is that I think the days operator could be 'fixed' easily and that's its default mode of operation is counterintuitive, considering that it works on dates alone.

Thanks,
Hubert

[TonyM]

Hubert,

Unfortunately the days operator was a quick solution to an important problem but can be a little hard to understand. I did a lot of work trying to simplify it, but this complexity makes it time consuming to return to it. The table attached needs a thorough test for example;

The truth is there should be no problem relating to dates and UTC when we manually stamp a date field with a date only or a yyyymmdd1200 except if the days filter is sourcing today incorrectly.

Now when a date is stamped with now, there is a chance it is a local time or UTC s future comparisons need to know to which that date is referring. It appears the timezone offset is retrieved automatically, and their is now aye to obtaining it independently. 

<<now>>

{{!!modified}}

{{!!created}}

Are returning the correct date with time zone, because there is special handling for the modified and created dates, A method I have wanted to emulate for other fields. If this was known we could ensure our own date fields comply and work this way as well.

My Suspicion is Days sees a number as UTC but custom dates are not stored as UTC but the local time, so the comparison becomes invalid as the Time zone is falsely included. 

As a result I have raised this GitHub issue https://github.com/Jermolene/TiddlyWiki5/issues/3779

Regards

Tony