model: date and datetime types stored with wrong offset

[der_On]

Ok. I have several datetime properties in my model. They get stored as timestrings within the LOCAL time zone. This is very critical as they should be stored as GMT timestrings instead. Because when reloading them from the database the local time offset gets added to them. Next problem is that local timestrings will break everything as soon as the server's time zone changes or we input times from different time zones.

I've tried to understand where exactly the dates get converted back and forth but wasn't able to find the right place in code.

[der_On]

This is also true for the build-in createdAt and updatedAt datetimes. The created at datetime even stores the wrong day!

[der_On]

Okay found out where the conversion from date to datestring happens. Actually the serialize method for dates creates a new Date from an existing date, which is actually nonsense in my eyes as a Date object already contains time offset. Please correct me if I'm wrong. Then this date is automatically cast to a string which automaticly calls Date.toString() this string contains the timezone, but for me (using postgres) this part get's trimmed away as the data type in the Database is "timestamp without time zone".

So actually the postgres data type for the datetime fields is wrong. It should be "timestamp with time zone" instead.

[der_On]

After changing the datatype in postgres to timestamp with time zone the timezone gets attached, however with the time offset added already. This most propably comes from the date serialize method which simply ignores time zones by creating a new local Date from a date objects UTC times/dates. The date serialize method should instead return a Date.toUTCString().

[der_On]

Oh boy. Found out that actually everything works fine out of the box if I set model.useUTC to true! I thought this is true by default. The docs are completly missing this one. Question is where I have to set model.useUTC = true to make it persistante within my geddy app?

[der_On]

Also this is become a monolog...

The problem is as follows: The serailize method of the date datatype checks for model.useUTC, however at that point model is simply an empty object. I can see within the model index.js that model gets mixed with an immediately called function containing all the props and also this.useUTC = true. However this does not seem to work when doing a simple var model = require('./index') as dpne in the datatypes.js

[Matthew Eernisse]

Dates and timezones are hard. I wrote a very early timezone library for JavaScript (https://github.com/mde/timezone-js, still active), and set of date utilities (now in our utilities lib: https://github.com/mde/utilities/blob/master/lib/date.js) and it still confuses the shit out of me.

There are some unit-tests for the date datatype, but since they are unit tests they don't make a serialize/deserialize round-trip. Maybe the right place for this sort of test would be the shared adapter tests, so we could ensure that we actually get the round-trip behavior right, and never regress it. (I know that updating the pg module right now breaks dates.)

Do you think UTC should be on by default?

[der_On]

Basically UTC timestrings are a good thing. Generally I prefer saving times/dates as integers in seconds/milliseconds after the UNIX epoche as those are easy comparable, have no time zone difficulties and can be converted to dates pretty easy and are independent of time/date implementations in databases as they are simple integers. However they are not human readable.

I've tried to hardoce model.useUTC = true within datetime.serialize() and it stored the correct timestring (in UTC time) in the database and also retrieved the correct date objects afterwards. However as model = undefined within datetime.serialize() useUTC (which is true by default according to the model/lib/index.js) is also falsy. That's the problem here, not dates/times and time zones.

[Matthew Eernisse]

Okay, now I'm really confused -- it's been awhile since I had date issues, so I had forgotten that useUTC is set to true by default on the model object. It has been since Aug of 2012 (commit 49b234cb0befc8b623f66bd8c788ab7719aec8e6), where I made a bunch of UTC and date-related fixes.

I've also added a test to the shared adapter tests that makes sure dates are persisted and round-trip correctly, and that we don't regress it.

Where are you trying to use the datatype serialization that it doesn't have access to the normal model object?

[der_On]

Actually it is used internally when a datetime model property will be written to the database. And I've just added a console.log(model) there and it showed an empty object {}. So I've made a really quick hack to make it work for me in my fork: https://github.com/der-On/model/commit/77766d7139cd45ef59df68c8e0360863c016947f

[Matthew Eernisse]

Is this happening inside a Geddy app? All the Model integration tests show it working properly (and set to 'true').

--
The official community discussion group.
website: geddyjs.org, source: https://github.com/mde/geddy, group: https://groups.google.com/d/forum/geddyjs?hl=en
---
You received this message because you are subscribed to the Google Groups "GeddyJS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to geddyjs+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Matthew Eernisse]

I created an app (using current Geddy master/HEAD), and ran the console (which just loads up the app environment in a console), and useUTC is indeed correctly set to true:

$ ./geddy/bin/cli.js gen app foo
Created app foo.
$ cd foo
$ ../geddy/bin/cli.js jake console
>>> geddy.model.useUTC;

true

[der_On]

I did just the same too and it works.

I think I've found the problem. It is within my fork of the "model" module. I'm doing datatypes = require('./datatypes') in the index.js. This causes a circular dependency which is not good in node.js.

I found out that you had the same problem before and solved using lazy require of datatypes. I've now did the same and it works. :)