[Python-ideas] datetime: Support infinity
Thomas Güttler
postgreSQL supports infinity for datetime:
http://www.postgresql.org/docs/current/static/datatype-datetime.html#AEN6027
{{{
infinity date, timestamp later than all other time stamps
-infinity date, timestamp earlier than all other time stamps
}}}
Mapping this to python is not possible at the moment.
See:
http://initd.org/psycopg/docs/usage.html#infinite-dates-handling
{{{
PostgreSQL can store the representation of an “infinite” date, timestamp, or interval. Infinite dates are not available
to Python, so these objects are mapped to date.max, datetime.max, interval.max. Unfortunately the mapping cannot be
bidirectional so these dates will be stored back into the database with their values, such as 9999-12-31.
}}}
I don't know the internals of the datetime module. I guess it is not possible to support infinity.
What do you think?
Thomas Güttler
_______________________________________________
Python-ideas mailing list
Python...@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
M.-A. Lemburg
> Hi,
>
> postgreSQL supports infinity for datetime:
>
> http://www.postgresql.org/docs/current/static/datatype-datetime.html#AEN6027
>
> {{{
> infinity date, timestamp later than all other time stamps
> -infinity date, timestamp earlier than all other time stamps
> }}}
>
>
> Mapping this to python is not possible at the moment.
>
> See:
>
> http://initd.org/psycopg/docs/usage.html#infinite-dates-handling
>
> {{{
> PostgreSQL can store the representation of an “infinite” date, timestamp, or interval. Infinite
> dates are not available to Python, so these objects are mapped to date.max, datetime.max,
> interval.max. Unfortunately the mapping cannot be bidirectional so these dates will be stored back
> into the database with their values, such as 9999-12-31.
> }}}
>
> I don't know the internals of the datetime module. I guess it is not possible to support infinity.
>
> What do you think?
would be, you can use the PG work-around for supporting these
in Python as well.
In mxDateTime, we have special values which can be used for this:
>>> mx.DateTime.MaxDateTime
<mx.DateTime.DateTime object for '25200470046051299-12-31 00:00:00.00' at 7fd7c4b3b2b8>
>>> mx.DateTime.MinDateTime
<mx.DateTime.DateTime object for '-25200470046051299-01-01 00:00:00.00' at 7fd7c4b3b390>
--
Marc-Andre Lemburg
eGenix.com
Professional Python Services directly from the Source (#1, Jan 26 2015)
>>> Python Projects, Coaching and Consulting ... http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
________________________________________________________________________
::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
http://www.egenix.com/company/contact/
Antoine Pitrou
Thomas Güttler
<guet...@thomas-guettler.de> wrote:
> Hi,
>
> postgreSQL supports infinity for datetime:
>
> http://www.postgresql.org/docs/current/static/datatype-datetime.html#AEN6027
>
> {{{
> infinity date, timestamp later than all other time stamps
> -infinity date, timestamp earlier than all other time stamps
> }}}
>
>
> Mapping this to python is not possible at the moment.
>
> See:
>
> http://initd.org/psycopg/docs/usage.html#infinite-dates-handling
>
> {{{
> PostgreSQL can store the representation of an “infinite” date, timestamp, or interval. Infinite dates are not available
> to Python, so these objects are mapped to date.max, datetime.max, interval.max. Unfortunately the mapping cannot be
> bidirectional so these dates will be stored back into the database with their values, such as 9999-12-31.
> }}}
Unless someone has a real-world use for the values of date.max,
datetime.max, interval.max, I find it rather counter-productive to not
store them back as infinities.
Adding infinities to the datetime module would probably be possible but
someone has to figure out the arithmetic rules. Do we need "not a time"
when adding infinity to -infinity?
Regards
Antoine.
Steven D'Aprano
> On Mon, 26 Jan 2015 09:24:07 +0100
> Thomas Güttler <guet...@thomas-guettler.de> wrote:
> > PostgreSQL can store the representation of an “infinite” date, timestamp, or interval. Infinite dates are not available
> > to Python, so these objects are mapped to date.max, datetime.max, interval.max. Unfortunately the mapping cannot be
> > bidirectional so these dates will be stored back into the database with their values, such as 9999-12-31.
> Unless someone has a real-world use for the values of date.max,
> datetime.max, interval.max, I find it rather counter-productive to not
> store them back as infinities.
That would make them de-facto infinities that weirdly don't look like
infinities:
py> datetime.date.max
datetime.date(9999, 12, 31)
If I'm reading this page correctly, PostgreSQL supports a lot of dates
that Python doesn't, up to 5874897AD:
http://www.postgresql.org/docs/9.4/static/datatype-datetime.html
so it might not matter if 9999-12-31 gets turned into infinity rather
than treated as a normal date.
> Adding infinities to the datetime module would probably be possible but
> someone has to figure out the arithmetic rules. Do we need "not a time"
> when adding infinity to -infinity?
I shouldn't think so. The purpose of NANs in IEEE-754 maths is to allow
the programmer to delay dealing with the failed operation until the end
of the calculation. I don't think that date calculations tend to be
anywhere as complicated as mathematical ones, so it would be acceptable
to just raise an exception.
Besides, Postgresql doesn't have a "NotATime" value, so if we're trying
to match what they do, we don't need one either.
--
Steven
Antoine Pitrou
Steven D'Aprano <st...@pearwood.info> wrote:
> On Mon, Jan 26, 2015 at 10:14:23AM +0100, Antoine Pitrou wrote:
> > On Mon, 26 Jan 2015 09:24:07 +0100
> > Thomas Güttler <guet...@thomas-guettler.de> wrote:
>
> > > PostgreSQL can store the representation of an “infinite” date, timestamp, or interval. Infinite dates are not available
> > > to Python, so these objects are mapped to date.max, datetime.max, interval.max. Unfortunately the mapping cannot be
> > > bidirectional so these dates will be stored back into the database with their values, such as 9999-12-31.
>
> > Unless someone has a real-world use for the values of date.max,
> > datetime.max, interval.max, I find it rather counter-productive to not
> > store them back as infinities.
>
> That would make them de-facto infinities that weirdly don't look like
> infinities:
That would only make them infinities in PostgreSQL. That sounds like a
reasonable compromise for a probably little-used feature, and uncommon
values.
> > Adding infinities to the datetime module would probably be possible but
> > someone has to figure out the arithmetic rules. Do we need "not a time"
> > when adding infinity to -infinity?
>
> I shouldn't think so. The purpose of NANs in IEEE-754 maths is to allow
> the programmer to delay dealing with the failed operation until the end
> of the calculation. I don't think that date calculations tend to be
> anywhere as complicated as mathematical ones, so it would be acceptable
> to just raise an exception.
Note Numpy datetimes and timedeltas do have a concept of "NotATime".
But, yes, it's an unexpected feature.
Regards
Antoine.
Chris Barker - NOAA Federal
generally useful,
Useful enough that I wrote them for my code base. And early the
postgress folks think it's a useful construct.
So +1 on adding this -- and not just hacked into the postgress adapter.
My use case is thus:
We have a model that does a computation through time. Various objects
can be active for a given time period. We use datetimes to store the
active_start and active_stop. For objects that are always active up to
a particular stop time, or become active at a particular time, and
then stay active forever, we needed a way to express -infTime and
infTime.
It probably would have mostly worked to use the min and max datetime
values, but then we lose information to pass back to the user. We also
convert to-from numpy datetimes, which have a different range.
I have a sample implementation if anyone wants to look amateur it,
though I don't think it does reflected comparisons correctly. it's
really only just good enough for our use case.
-Chris
rand...@fastmail.us
> If I'm reading this page correctly, PostgreSQL supports a lot of dates
> that Python doesn't, up to 5874897AD:
You know, as long as we're talking about expanding the range of
datetime.
Alexander Belopolsky
On Mon, Jan 26, 2015, at 06:44, Steven D'Aprano wrote:
> If I'm reading this page correctly, PostgreSQL supports a lot of dates
> that Python doesn't, up to 5874897AD:
Why doesn't Python support these?
Yawar Amin
On 2015-01-26 03:24, Thomas Güttler wrote:
> [...]
> possible to support infinity.
>
> What do you think?
in any modern date handling library). If I read you right, the problem
with mapping back to Postgres is that infinity dates are being
represented in-band in Python, so they get mapped to non-infinite dates
in Postgres.
The solution is to represent infinity dates as out-of-band values. My
take on this is to represent them as subtypes of datetime and handle
those subtypes appropriately. A rough (but working) sketch of the idea:
import datetime as dt
class datetime_ext(dt.datetime):
def __new__(cls, *args, **kwargs):
return dt.datetime.__new__(cls, *args, **kwargs)
# Getting the infinity dates.
@staticmethod
def pos_inf(): return datetime_inf(1)
@staticmethod
def neg_inf(): return datetime_inf(-1)
# More fun names.
@staticmethod
def big_bang(): return datetime_ext.neg_inf()
@staticmethod
def big_crunch(): return datetime_ext.pos_inf()
# Now the fun part.
def _are_we_inf(self, other):
return (
isinstance(self, datetime_inf),
isinstance(other, datetime_inf)
)
def __lt__(self, other):
awi = self._are_we_inf(other)
if awi == (True, True): return self.sign < other.sign
elif awi == (False, True): return other.sign > 0
elif awi == (True, False): return self.sign < 0
else: return self < other
def __gt__(self, other):
awi = self._are_we_inf(other)
if awi == (True, True): return self.sign > other.sign
elif awi == (False, True): return other.sign < 0
elif awi == (True, False): return self.sign > 0
else: return self > other
def __eq__(self, other):
awi = self._are_we_inf(other)
if awi == (True, True): return self.sign == other.sign
elif awi == (False, False): return self == other
else: return False
def __ne__(self, other): return not (self == other)
def __le__(self, other): return (self < other) or (self == other)
def __ge__(self, other): return (self > other) or (self == other)
class datetime_inf(datetime_ext):
def __new__(cls, sign):
# The numbers passed in to the constructor are meaningless; they
# should never be used.
retval = datetime_ext.__new__(cls, 1, 1, 1)
retval.sign = sign
return retval
def _raise_err(self): raise TypeError("Infinity date")
def date(self): return datetime_inf(self.sign)
def time(self): _raise_err()
def timetz(self): _raise_err()
def replace(self): _raise_err()
def astimezone(self): _raise_err()
# ... all the others ...
# All True:
print(datetime_ext.big_bang() == datetime_ext.big_bang())
print(datetime_ext.big_crunch() == datetime_ext.big_crunch())
print(datetime_ext.big_bang() < datetime_ext.big_crunch())
print(datetime_ext.big_crunch() > datetime_ext.big_bang())
print(datetime_ext.big_bang() < dt.datetime.today())
print(datetime_ext.big_crunch() > dt.datetime.today())
# Added bonuses (boni?):
print(dt.datetime.max < datetime_ext.big_crunch())
print(dt.datetime.min > datetime_ext.big_bang())
So the basic idea here is to override the comparison methods to treat
the datetime_inf type specially. The implementation of datetime_inf
isn't particularly elegant, but I think it works for its intended
purpose.
Now with these, I'm pretty sure you can write an InfDateWrapper class
that checks for datetime_ext.big_bang() and datetime_ext.big_crunch()
and returns the corresponding Postgres '-infinity::date' and
'infinity::date'; and that the mapping will be fully bidirectional.
And as I said, I believe something like this should be added to the
Python stdlib. If no one has a better implementation I'll start working
on a proposal.
Regards,
Yawar
Thomas Güttler
>
> And as I said, I believe something like this should be added to the
> Python stdlib. If no one has a better implementation I'll start working
> on a proposal.
in our code base, and a solution in Python stdlib would be very good.
Up to now we must use Python 2.7, so a backport (like subprocess32) would be needed.
Is this possible?
Please CC your proposal to me.
Regards,
Ben Finney
writes:
> Unless someone has a real-world use for the values of date.max,
> datetime.max, interval.max, I find it rather counter-productive to not
> store them back as infinities.
The values are useful to know exactly how far in the future one can
store date values in Python. It is important that this be discoverable
programmatically because the maximum values might change in future
versions of the ‘datetime’ library.
Because it remains true (and presumably will continue to be true) that
the maximum value is *not* infinity, it is important that the following
real-world cases are distinct:
* Query the ‘datetime’ library for the largest (furthest-in-the-future)
value that it can handle.
* Store a value explicitly meaning “infinitely far in the future” or
“infinitely far in the past” compared to any actual date or datetime
value.
I have needed both, sometimes in the same program.
An example is to be able to represent the timestamp field of a journal
entry that is still being composed and has not yet gained a timestamp.
‘None’ is not sufficient, because it must be “in the future” compared to
any actual point in time, even greater than ‘datetime.date.max’.
To use the same value attempting to represent both “maximum” and
“infinite” is a fragile, and needlessly confusing, hack. Having distinct
“infinitely far in the future” and “infinitely far in the past” values,
that are *not* themselves particular points on the timeline, would be a
valuable addition for this reason.
> Adding infinities to the datetime module would probably be possible but
> someone has to figure out the arithmetic rules. Do we need "not a time"
> when adding infinity to -infinity?
Adding dates (or datetimes) is not a valid operation today, so I don't
see why we would need to change behaviour there.
--
\ “This world in arms is not spending money alone. It is spending |
`\ the sweat of its laborers, the genius of its scientists, the |
_o__) hopes of its children.” —Dwight Eisenhower, 1953-04-16 |
Ben Finney
Antoine Pitrou
Ben Finney <ben+p...@benfinney.id.au> wrote:
> Antoine Pitrou <soli...@pitrou.net>
> writes:
>
> > Unless someone has a real-world use for the values of date.max,
> > datetime.max, interval.max, I find it rather counter-productive to not
> > store them back as infinities.
>
> The values are useful to know exactly how far in the future one can
> store date values in Python. It is important that this be discoverable
> programmatically because the maximum values might change in future
> versions of the ‘datetime’ library.
So what? Again, the values wouldn't change in Python. It would only be
the PostgreSQL mapping that would change.
> > Adding infinities to the datetime module would probably be possible but
> > someone has to figure out the arithmetic rules. Do we need "not a time"
> > when adding infinity to -infinity?
>
> Adding dates (or datetimes) is not a valid operation today, so I don't
> see why we would need to change behaviour there.
We're talking about both datetimes and timestamps here. Please follow.
Regards
Antoine.
Alexander Belopolsky
> Adding dates (or datetimes) is not a valid operation today, so I don't
> see why we would need to change behaviour there.
We're talking about both datetimes and timestamps here. Please follow.
Antoine Pitrou
Alexander Belopolsky
> On Tue, Jan 27, 2015 at 5:47 PM, Antoine Pitrou <soli...@pitrou.net> wrote:
>
> > > Adding dates (or datetimes) is not a valid operation today, so I don't
> > > see why we would need to change behaviour there.
> >
> > We're talking about both datetimes and timestamps here. Please follow.
>
>
> I think Antoine meant "timedeltas".
And for the record:
>>> import numpy as np
>>> np.timedelta64(0) / np.timedelta64(0)
__main__:1: RuntimeWarning: invalid value encountered in true_divide
nan
>>> np.timedelta64(0) / np.timedelta64(0) == np.timedelta64('nat')
True
>>> type(np.timedelta64('nat'))
<class 'numpy.timedelta64'>
Ben Finney
writes:
> On Wed, 28 Jan 2015 09:42:16 +1100
> Ben Finney <ben+p...@benfinney.id.au> wrote:
> > The [minimum and maximum] values are useful to know exactly how far
> > in the future [and past] one can store date values in Python. It is
> > important that this be discoverable programmatically because the
> > maximum [or minimum] values might change in future versions of the
> > ‘datetime’ library.
>
> So what? Again, the values wouldn't change in Python. It would only be
> the PostgreSQL mapping that would change.
That's the point, though. We should at least allow for “datetime.date.max”
and “datetime.date.infinitely_far_in_the_future” to be distinct, and
allow for the former to change value in a backward-compatible way.
So the maxima and minima should map to their specific points in time in
both directions between PostgreSQL and Python.
“Maximum” does not mean “infinite”, and conflating them robs us not only
of the backward-compatible ability to change the maximum in future, but
also of the ability to later add a distinct value to represent infinity.
--
\ “You don't need a book of any description to help you have some |
`\ kind of moral awareness.” —Dr. Francesca Stavrakoloulou, bible |
_o__) scholar, 2011-05-08 |
Ben Finney
Chris Barker
On Wed, 28 Jan 2015 09:42:16 +1100
Ben Finney <ben+p...@benfinney.id.au> wrote:
So what? Again, the values wouldn't change in Python. It would only be
the PostgreSQL mapping that would change.
> > Adding infinities to the datetime module would probably be possible but
> > someone has to figure out the arithmetic rules. Do we need "not a time"
> > when adding infinity to -infinity?
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
Chris....@noaa.gov
Steven D'Aprano
Same for use-cases, which Antoine asked for. The primary use-case is to
improve compatibility with Postgresql while still treating the full
range of dates up to and including date.max as regular dates.
I'm not convinced that numpy is the right library to follow here. Unless
there are cases where you would perform calculations with Not-A-Time, an
exception would probably be better suited.
--
Steve
Antoine Pitrou
>
> Same for use-cases, which Antoine asked for. The primary use-case is to
> improve compatibility with Postgresql while still treating the full
> range of dates up to and including date.max as regular dates.
>
> I'm not convinced that numpy is the right library to follow here. Unless
> there are cases where you would perform calculations with Not-A-Time, an
> exception would probably be better suited.
knowledgeable people I talked to generally agreed that Numpy's
datetime64/timedelta64 design isn't very good (actually, it has an
even wonkier feature that's only exposed internally AFAIK). So
"Not-A-Time" need not necessarily be implemented in the stdlib ;)
Regards
Antoine.
Yawar Amin
On 2015-01-27 18:04, Chris Barker wrote:
> On Tue, Jan 27, 2015 at 2:47 PM, Antoine Pitrou <soli...@pitrou.net> wrote:
> > > Adding infinities to the datetime module would probably be
> > > possible but
> > > someone has to figure out the arithmetic rules. Do we need
> > > "not a time"
> > > when adding infinity to -infinity?
> not_a_timedelta, I suppose, or simply disallow it.
>
> But if you do allow it -- then following the rules for floating point
> inf, -inf, and NaN would make sense.
datetime.now().
Regards,
Yawar
Yawar Amin
On 2015-01-26 11:31, Chris Barker - NOAA Federal wrote:
> [...]
> though I don't think it does reflected comparisons correctly. it's
> really only just good enough for our use case.
Regards,
Yawar
Alexander Belopolsky
Chris Barker
On 2015-01-26 11:31, Chris Barker - NOAA Federal wrote:
> I have a sample implementation if anyone wants to look at it,
> though I don't think it does reflected comparisons correctly. it's
> really only just good enough for our use case.
I'd like to take a look. Can you send it to me, or post it somewhere?
Yawar Amin
> [...]
>
> https://github.com/NOAA-ORR-ERD/PyGnome/blob/master/py_gnome/gnome/utilities/inf_datetime.py
https://groups.google.com/d/msg/python-ideas/G3jeWoa6h14/ELpDLFu28QcJ
I believe the ideal design is the following type hierarchy:
datetime_base
datetime_neg_inf
datetime_pos_inf
datetime
But in the interests of pragmatism, I think this one can be made to
work:
datetime
datetime_neg_inf
datetime_pos_inf
So we'd keep using the existing datetime type, and whenever we need
infinity datetimes we'd just pass in the appropriate subtypes. The
advantage to this is that we don't have to switch everything over to
using a new derived type--just use the normal datetime for the majority
of cases where we don't need inifinity dates.
Regards,
Yawar
Chris Barker
Cool, the comparison operator logic looks very similar to mine:
https://groups.google.com/d/msg/python-ideas/G3jeWoa6h14/ELpDLFu28QcJ
Did you get it to work both ways? i.e.
datetime.now() > datetime.neg_inf
I believe the ideal design is the following type hierarchy:
datetime_base
datetime_neg_inf
datetime_pos_inf
datetime
But in the interests of pragmatism, I think this one can be made to
work:
datetime
datetime_neg_inf
datetime_pos_inf
The
advantage to this is that we don't have to switch everything over to
using a new derived type--just use the normal datetime for the majority
of cases where we don't need inifinity dates.
rand...@fastmail.us
> I had trouble with that, because the regular datetime doesn't know how to
> compare itself to a neg_inf object.
3 returns NotImplemented.
Chris Barker
Python 2 raises exception from __lt__,
Ethan Furman
>>
>> Python 2 raises exception from __lt__,
>
> yeah, that's what I got -- we're on 2 for now. You'd think there wold
> be a way to do it -- but I don't see a "reflected" versions of the
> comparison operators.
--
~Ethan~
Chris Barker
>> Python 2 raises exception from __lt__,
>
> yeah, that's what I got -- we're on 2 for now. You'd think there wold
> be a way to do it -- but I don't see a "reflected" versions of the
> comparison operators.
Raising an exception was a bug. The comparison operators are their own reflection.
Yawar Amin
rand...@fastmail.us
reflection of __lt__).
This works fine in 2.7 for types that properly return NotImplemented.
This is just a bug in Python 2's datetime module.
Skip Montanaro
So does that mean it could be fixed in 2.* ? (or already has been?)
rand...@fastmail.us
> Hi Chris, sorry for the top-posting; replying from my phone.
>
> My test implementation works for me on Python 2.7, see http://repl.it/9Wz
>
> Notice especially the last few assertions, where normal datetime objects
> are on the LHS.
operators when working with a base class and a derived class?
Ethan Furman
> On Thu, Jan 29, 2015, at 14:14, Ethan Furman wrote:
>> On 01/29/2015 11:09 AM, Chris Barker wrote:
>>> On Thu, Jan 29, 2015 at 10:10 AM, random832 wrote:
>>>>
>>>> Python 2 raises exception from __lt__,
>>>
>>> yeah, that's what I got -- we're on 2 for now. You'd think there wold
>>> be a way to do it -- but I don't see a "reflected" versions of the
>>> comparison operators.
>>
>> Raising an exception was a bug. The comparison operators are their own
>> reflection.
>
> Or, more precisely, they are *each other's* reflection (__gt__ being the
> reflection of __lt__).
--
~Ethan~
Chris Barker
My test implementation works for me on Python 2.7, see http://repl.it/9Wz
Ethan Furman
> On Thu, Jan 29, 2015, at 14:19, Yawar Amin wrote:
>> Hi Chris, sorry for the top-posting; replying from my phone.
>>
>> My test implementation works for me on Python 2.7, see http://repl.it/9Wz
>>
>> Notice especially the last few assertions, where normal datetime objects
>> are on the LHS.
>
> How'd you do that? Does python always know to use the derived class's
> operators when working with a base class and a derived class?
So the broken datetime behavior would only be noticed by a non-datetime derived class.
--
~Ethan~
Chris Barker
> How'd you do that? Does python always know to use the derived class's
> operators when working with a base class and a derived class?
I can't find the reference, but yes.
So the broken datetime behavior would only be noticed by a non-datetime derived class.
--
~Ethan~
_______________________________________________
Python-ideas mailing list
Python...@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/