Missing timezone support in built-in time template filter

[Warren Smith]

Is anyone aware of a reason why the built-in time template filter does not have timezone support?  The built-in date template filter has timezone support, but I wanting to use the TIME_FORMAT default stuff linked to the time filter.

Was timezone considered to be not relevant for the time filter? I found the bit in the Time Zone documentation about Django only supporting naive datetime.time objects, so I suppose this might have been the justification for not supporting timezones in the time filter. However, the time filter can be applied to datetime.datetime objects as well, to display only the time portion. That might even be the most common use case. It is at least my use case. In that context, I would think the timezone IS relevant.

What I'm wanting to be able to do is set settings.TIME_FORMAT = 'g:i a T' and then use <datetime_var>|time in my templates to get times with timezones at the end.

I would prefer not to have to resort to a custom filter for what seems to be very generic behavior. What am I doing wrong?

[Russell Keith-Magee]

Hi Warren,

I can see how what you're describing is a problem, and as far as I can make out, you haven't missed anything. 

The issue is how to handle the ambiguity it introduces; Django has punted on the problem because what you're describing is an edge case with some ambiguity, and there are ways to work around the problem.

The problem with datetime.time is that time objects *can't* have a timezone -- at least, not reliably. For example - If your timezone is EST, and the time is 10:00 UTC - what time is it in EST? You can't answer that question reliably because it might be 0500, or it might be 0600, depending on the date. Although common vernacular often describes the timezone as EST and EDT, there's actually only one timezone, and part of that timezone definition is the rules for switching for switching forward and back the extra hour. Without a date associated with the time, timezone calculations are error-prone at best.

As a result, since the default usage of |time is on time objects, the implementation of the time filter doesn't support the use of the T format specifier.

However, you *can* use exclusively time-related format specifiers in the date filter; for example:

{{ mydatetime|date:"g:i a T" }}

will work and give you the output you expect, and if you set DATETIME_FORMAT to "g:i A T", you'll get all datetimes displayed in the format you describe. The problem is that *all* datetimes will be displayed like that by default, and you won't ever see the actual date printed.

So - short term, the fix is to use an explicit format string wherever you want this behaviour. Yes, it's annoying that you can't just set a default, but it works. Defining a custom filter that implements the default could be done in about 5 lines of code (including tag registration) if the repetition of the format string bugs you enough. You can even call that filter |time if you want - Django's template engine takes the 'newest' tag registration as the canonical one, so you can overwrite system-provided tags.

Longer term, I can see that there is something we could address - the question is how.

What you've described is the ability to set TIME_FORMAT to a value that won't be valid for the default case (using time objects on the time filter), so we'd need to come up with an interpretation for how to ignore T (and any other timezone-sensitive format values) when it can't be used. I'm not sure how I feel about ignoring formatters as an approach -- my gut reaction is that it's a bad idea, but I can see how it *might* work.

Another approach might be to make |time call on the |date format - however, this means you would need to set DATETIME_FORMAT

A third approach would be a new setting, for AWARE_TIME_FORMAT that would only get used if you pass a datetime object to |time; this *would* be able to use T as an option. However, this means introducing a new setting, which is rarely a good answer to a problem.

If you've got any other ideas, I'd be interested in hearing them. Either way, this is worth a ticket so that the idea/problem isn't forgotten.

Yours,

Russ Magee %-)

[Warren Smith]

On Thu, Jun 27, 2013 at 7:54 PM, Russell Keith-Magee <rus...@keith-magee.com> wrote:

I can see how what you're describing is a problem, and as far as I can make out, you haven't missed anything.

Cool. This thing has been perplexing me for the better part of a day.

 

The issue is how to handle the ambiguity it introduces; Django has punted on the problem because what you're describing is an edge case with some ambiguity, and there are ways to work around the problem.

I was afraid of that. Not unexpected, but unwelcome news just the same. :-(

 

The problem with datetime.time is that time objects *can't* have a timezone -- at least, not reliably. For example - If your timezone is EST, and the time is 10:00 UTC - what time is it in EST? You can't answer that question reliably because it might be 0500, or it might be 0600, depending on the date. Although common vernacular often describes the timezone as EST and EDT, there's actually only one timezone, and part of that timezone definition is the rules for switching for switching forward and back the extra hour. Without a date associated with the time, timezone calculations are error-prone at best.

Agreed.

 

As a result, since the default usage of |time is on time objects, the implementation of the time filter doesn't support the use of the T format specifier.

Where is this default usage established? By the filter name? It certainly is not spelled out in the documentation. In fact, the only example of input to the time filter uses a datetime object, not a time object. If a time object really was the default case, wouldn't it make sense to say so in the documentation? The fact that it doesn't seems to indicate that at least one other person (the person who wrote the docs) seems to think that a more common input to the time filter is a datetime object.

In practice, I think that datetime.datetime objects are actually very common as input to both filters, perhaps more common than using datetime.date for the date filter and datetime.time for the time filter. Still, who are we to judge? The date filter fully supports both date and datetime objects. IMHO, the time filter should fully support time and datetime objects as well.

 

However, you *can* use exclusively time-related format specifiers in the date filter; for example:

{{ mydatetime|date:"g:i a T" }}

will work and give you the output you expect, and if you set DATETIME_FORMAT to "g:i A T", you'll get all datetimes displayed in the format you describe. The problem is that *all* datetimes will be displayed like that by default, and you won't ever see the actual date printed.

I'm a little confused by that last paragraph. I think what you are saying is that since the date filter can handle datetime objects with timezones just fine, I can abuse the DATETIME_FORMAT setting and only put time format specifiers in it, then litter my code with |date:"DATETIME_FORMAT" everywhere I would have used |time, is that it? Hmm. I think I'll pass on that.

 

So - short term, the fix is to use an explicit format string wherever you want this behaviour. Yes, it's annoying that you can't just set a default, but it works. Defining a custom filter that implements the default could be done in about 5 lines of code (including tag registration) if the repetition of the format string bugs you enough. You can even call that filter |time if you want - Django's template engine takes the 'newest' tag registration as the canonical one, so you can overwrite system-provided tags.

The overridden |time filter seems like a much better approach and is what I already had in mind should my fears about django punting on this issue be confirmed, as they now have been.

 

Longer term, I can see that there is something we could address - the question is how.

What you've described is the ability to set TIME_FORMAT to a value that won't be valid for the default case (using time objects on the time filter), so we'd need to come up with an interpretation for how to ignore T (and any other timezone-sensitive format values) when it can't be used. I'm not sure how I feel about ignoring formatters as an approach -- my gut reaction is that it's a bad idea, but I can see how it *might* work.

How *might* it work?  Could we just return a blank string?  It appears there is a precedent for this sort of behavior in the  django.utils.dateformat.DateFormat.e() method.

 

Another approach might be to make |time call on the |date format - however, this means you would need to set DATETIME_FORMAT

Not necessarily, if we did the delegation at a low enough level. If we did some type detection and branching in django.utils.dateformat.time_format(), we would already be underneath where the format strings are determined.

However, doing this sort of delegation would allow the time filter to honor date format specifiers, which feels wrong. Even if we left it undocumented, somebody would find it, depend on it, and then would get upset if we ever tightened things up and broke their stuff. Better to just not go there in the first place.

A third approach would be a new setting, for AWARE_TIME_FORMAT that would only get used if you pass a datetime object to |time; this *would* be able to use T as an option. However, this means introducing a new setting, which is rarely a good answer to a problem.

Agreed. Adding new settings == generally bad. Explaining why we need the new setting == fairly hard. Hard to explain == bad idea.

 

If you've got any other ideas, I'd be interested in hearing them.

I think the correct solution is to add proper handling of datetime timezones to the django.utils.dateformat.TimeFormat class, similar to the way they are supported in the DateFormat class, and perhaps do some refactoring of both classes so that common methods are inherited from an additional base class.

Thoughts?

Either way, this is worth a ticket so that the idea/problem isn't forgotten.

Will do.

-- 

Warren Smith

[Russell Keith-Magee]

On Sat, Jun 29, 2013 at 3:53 AM, Warren Smith <wsmi...@gmail.com> wrote:

On Thu, Jun 27, 2013 at 7:54 PM, Russell Keith-Magee <rus...@keith-magee.com> wrote:

I can see how what you're describing is a problem, and as far as I can make out, you haven't missed anything.

Cool. This thing has been perplexing me for the better part of a day.

 

The issue is how to handle the ambiguity it introduces; Django has punted on the problem because what you're describing is an edge case with some ambiguity, and there are ways to work around the problem.

I was afraid of that. Not unexpected, but unwelcome news just the same. :-(

 

The problem with datetime.time is that time objects *can't* have a timezone -- at least, not reliably. For example - If your timezone is EST, and the time is 10:00 UTC - what time is it in EST? You can't answer that question reliably because it might be 0500, or it might be 0600, depending on the date. Although common vernacular often describes the timezone as EST and EDT, there's actually only one timezone, and part of that timezone definition is the rules for switching for switching forward and back the extra hour. Without a date associated with the time, timezone calculations are error-prone at best.

Agreed.

 

As a result, since the default usage of |time is on time objects, the implementation of the time filter doesn't support the use of the T format specifier.

Where is this default usage established? By the filter name?

Well, it's established by virtue that the most obvious data type to which you'd want to apply a time filter is time. "Default" is perhaps the wrong word.

 

It certainly is not spelled out in the documentation. In fact, the only example of input to the time filter uses a datetime object, not a time object. If a time object really was the default case, wouldn't it make sense to say so in the documentation? The fact that it doesn't seems to indicate that at least one other person (the person who wrote the docs) seems to think that a more common input to the time filter is a datetime object.

In practice, I think that datetime.datetime objects are actually very common as input to both filters, perhaps more common than using datetime.date for the date filter and datetime.time for the time filter. Still, who are we to judge? The date filter fully supports both date and datetime objects. IMHO, the time filter should fully support time and datetime objects as well.

Well, it does, depending on your definition of "fully". All features available to time objects are available to datetime objects. 

 

However, you *can* use exclusively time-related format specifiers in the date filter; for example:

{{ mydatetime|date:"g:i a T" }}

will work and give you the output you expect, and if you set DATETIME_FORMAT to "g:i A T", you'll get all datetimes displayed in the format you describe. The problem is that *all* datetimes will be displayed like that by default, and you won't ever see the actual date printed.

I'm a little confused by that last paragraph. I think what you are saying is that since the date filter can handle datetime objects with timezones just fine, I can abuse the DATETIME_FORMAT setting and only put time format specifiers in it, then litter my code with |date:"DATETIME_FORMAT" everywhere I would have used |time, is that it? Hmm. I think I'll pass on that.

I'm not saying it's a *good* workaround - I'm saying its *a* workaround, and one that doesn't require you to make every |date usage on a datetime display only the time.

 

So - short term, the fix is to use an explicit format string wherever you want this behaviour. Yes, it's annoying that you can't just set a default, but it works. Defining a custom filter that implements the default could be done in about 5 lines of code (including tag registration) if the repetition of the format string bugs you enough. You can even call that filter |time if you want - Django's template engine takes the 'newest' tag registration as the canonical one, so you can overwrite system-provided tags.

The overridden |time filter seems like a much better approach and is what I already had in mind should my fears about django punting on this issue be confirmed, as they now have been.

 

Longer term, I can see that there is something we could address - the question is how.

What you've described is the ability to set TIME_FORMAT to a value that won't be valid for the default case (using time objects on the time filter), so we'd need to come up with an interpretation for how to ignore T (and any other timezone-sensitive format values) when it can't be used. I'm not sure how I feel about ignoring formatters as an approach -- my gut reaction is that it's a bad idea, but I can see how it *might* work.

How *might* it work?  Could we just return a blank string?  It appears there is a precedent for this sort of behavior in the  django.utils.dateformat.DateFormat.e() method.

 

Hrm. That's an interesting point… I hadn't considered that.

 

Another approach might be to make |time call on the |date format - however, this means you would need to set DATETIME_FORMAT

Not necessarily, if we did the delegation at a low enough level. If we did some type detection and branching in django.utils.dateformat.time_format(), we would already be underneath where the format strings are determined.

However, doing this sort of delegation would allow the time filter to honor date format specifiers, which feels wrong. Even if we left it undocumented, somebody would find it, depend on it, and then would get upset if we ever tightened things up and broke their stuff. Better to just not go there in the first place.

A third approach would be a new setting, for AWARE_TIME_FORMAT that would only get used if you pass a datetime object to |time; this *would* be able to use T as an option. However, this means introducing a new setting, which is rarely a good answer to a problem.

Agreed. Adding new settings == generally bad. Explaining why we need the new setting == fairly hard. Hard to explain == bad idea.

I don't think it's *that* hard to explain - one is a format that will be used for naive time objects; one is a format that will be used for timezone-aware time objects (primarily datetimes).

However, given that there's precedent for blanking formats when they don't/can't apply, that would appear to be the better solution.

Yours,

Russ Magee %-)

[Warren Smith]

On Fri, Jun 28, 2013 at 5:49 PM, Russell Keith-Magee <rus...@keith-magee.com> wrote:

A third approach would be a new setting, for AWARE_TIME_FORMAT that would only get used if you pass a datetime object to |time; this *would* be able to use T as an option. However, this means introducing a new setting, which is rarely a good answer to a problem.

Agreed. Adding new settings == generally bad. Explaining why we need the new setting == fairly hard. Hard to explain == bad idea.

I don't think it's *that* hard to explain - one is a format that will be used for naive time objects; one is a format that will be used for timezone-aware time objects (primarily datetimes).

However, given that there's precedent for blanking formats when they don't/can't apply, that would appear to be the better solution.

Agreed.

Also, just to clarify, I think we should stick with Django's policy of only supporting naive time objects, for the reasons you cited earlier in this thread.

So, any timezone support we would add to the time filter should only support aware datetime objects, not aware time objects.

Correct?

 

--
You received this message because you are subscribed to the Google Groups "Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
Warren Smith

[Russell Keith-Magee]

On Mon, Jul 1, 2013 at 10:47 PM, Warren Smith <wsmi...@gmail.com> wrote:

On Fri, Jun 28, 2013 at 5:49 PM, Russell Keith-Magee <rus...@keith-magee.com> wrote:

A third approach would be a new setting, for AWARE_TIME_FORMAT that would only get used if you pass a datetime object to |time; this *would* be able to use T as an option. However, this means introducing a new setting, which is rarely a good answer to a problem.

Agreed. Adding new settings == generally bad. Explaining why we need the new setting == fairly hard. Hard to explain == bad idea.

I don't think it's *that* hard to explain - one is a format that will be used for naive time objects; one is a format that will be used for timezone-aware time objects (primarily datetimes).

However, given that there's precedent for blanking formats when they don't/can't apply, that would appear to be the better solution.

Agreed.

Also, just to clarify, I think we should stick with Django's policy of only supporting naive time objects, for the reasons you cited earlier in this thread.

So, any timezone support we would add to the time filter should only support aware datetime objects, not aware time objects.

Correct?

Yes - that sounds like the right approach to me.

Yours,

Russ Magee %-)

[Warren Smith]

On Mon, Jul 1, 2013 at 6:18 PM, Russell Keith-Magee <rus...@keith-magee.com> wrote:

Agreed.

Also, just to clarify, I think we should stick with Django's policy of only supporting naive time objects, for the reasons you cited earlier in this thread.

So, any timezone support we would add to the time filter should only support aware datetime objects, not aware time objects.

Correct?

Yes - that sounds like the right approach to me.

Done. https://code.djangoproject.com/ticket/20693 is ready for somebody to review it. It would be great if this could get into 1.6.

--
Warren Smith

[gilberto dos santos alves]

well done.

2013/7/3 Warren Smith <wsmi...@gmail.com>:

> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-develop...@googlegroups.com.
> To post to this group, send email to django-d...@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>



--

gilberto dos santos alves
+55.11.98646-5049
sao paulo - sp - brasil

[Warren Smith]

Any chance that https://code.djangoproject.com/ticket/20693 can make it into 1.6?

It has been sitting in ready-for-checkin for several weeks now. I thought I had done everything I could, but then I realized that my branch was out of date and it was skewing the diff on https://github.com/django/django/pull/1327. I'm a git newbie, so it took me a while to figure out how to remedy that.

I'm not sure if that was why it was getting held up, but it is fixed now.

Warren Smith

[Carl Meyer]

Hi Warren,

On 08/26/2013 03:32 PM, Warren Smith wrote:
> Any chance that https://code.djangoproject.com/ticket/20693 can make it
> into 1.6?

Thanks for your work on the patch! Since it's a new feature, it's a
couple months late for 1.6; it would have needed to go in before the
first 1.6 beta was released back in June (which was before you created
the patch).

I've reviewed and merged the pull request, so it'll go into 1.7.

> It has been sitting in ready-for-checkin for several weeks now. I
> thought I had done everything I could, but then I realized that my
> branch was out of date and it was skewing the diff on
> https://github.com/django/django/pull/1327. I'm a git newbie, so it took
> me a while to figure out how to remedy that.
>
> I'm not sure if that was why it was getting held up, but it is fixed now.

Thanks for doing that, but I doubt it was the reason for the delay, more
likely just didn't catch anyone's attention for review and merge. If
someone had sat down to review it and been blocked by it not merging
cleanly, they'd have commented to that effect on the ticket.

Thanks again for the patch!

Carl