force ravenDB to store all dates in UTC

[Karl Cassar]

Hi,

Is it possible to force RavenDB to store all dates in UTC format?  For example, considering the case below:

        [Test]

        public void test()

        {

            DateTime dt = new DateTime(2012, 1, 1, 15, 0, 0, DateTimeKind.Utc); //16:00 malta time

            TestObj item1 = new TestObj();

            item1.Date1 = dt; //same date, UTC

            item1.Date2 = dt.ToLocalTime(); //same date, LocalTime

            _session.Store(item1);

            _session.SaveChanges();

            TestGeneral_v4.Util.RavenDbUtil.WaitForUserToContinueTheTest();

            disposeSessionAndCreateNewOne();

            var loadedItem1 = _session.Load<TestObj>(item1.Id);

            int k = 5;

        }

Both dates should ideally be saved as the same item, and when loaded back, they are loaded as 'local time'.

Regards,

Karl

[Kijana Woodard]

Wait. What are your Asserts?

--
You received this message because you are subscribed to the Google Groups "ravendb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Oren Eini (Ayende Rahien)]

RavenDB accept the date in the format you give it.

If you care about how it gets store, use a listener to mdify that.

[Kijana Woodard]

Local to what, the web/app machine?

Personally, I've had enough 'fun' with systems trying to be helpful by converting dates to local.

I don't want the db guessing what I need. If I give you data, give it back as is. :-)

DateTimeOffset or NodaTime when I need to be precise. Otherwise, I'll store utc myself.

[Justin A]

Karl - all your -code- should use UtcNow for dates, IMO.

eg. foo.CreatedOn = DateTime.UtcNow;

foo.AnotherDate = DateTime.UtcNow.AddHours(40);

etc.

that's how I roll, at least.

[Kijana Woodard]

I can see situations where only local time matter, but in general, yes.

I've been burned by systems with out of band time zone information. Oh, that field is UTC. No no no, the other field is EST. Grrrr.

--

[Karl Cassar]

The thing is having to remember each time you pass a date, to change it to UTC can greatly introduce mistakes.

Basically, as you know a DateTime has a .Kind property, which lets you know if it is UTC or not.  

This seems to be more something from JSON.Net, as it seems you can specify the serialization type.  Basically, when serializing a date, I would like it to be serialized to UTC time.  When deserializing, it is deserialized to Local time (as in client machine-time). 

Is it possible to specify the 'JsonSerializerSettings' that RavenDB uses when serializing to the server, and de-serializing data from the server?

The below code in native Json.net works just like I would like RavenDB to work, automatically.

            string sSerializedDate = null;
            {
                JsonSerializerSettings settingsSerialize = new JsonSerializerSettings();
                settingsSerialize.DateTimeZoneHandling = DateTimeZoneHandling.Utc;


                var jsonSeriazlier = Newtonsoft.Json.JsonSerializer.Create(settingsSerialize);
                DateTime d = new DateTime(2012, 1, 1, 15, 0, 0, DateTimeKind.Local);
                StringBuilder sbSerializedDate = new StringBuilder();
                StringWriter swSerializedDate = new StringWriter(sbSerializedDate);
                jsonSeriazlier.Serialize(swSerializedDate, d);
                sSerializedDate = sbSerializedDate.ToString();  //value here is: 2012-01-01T14:00:00Z (UTC)
            }
            DateTime deserializedDate;
            {
                JsonSerializerSettings settingsDeserialize = new JsonSerializerSettings();
                settingsDeserialize.DateTimeZoneHandling = DateTimeZoneHandling.Local;
                var jsonDeserializer = Newtonsoft.Json.JsonSerializer.Create(settingsDeserialize);
                
               
                StringReader srDeserializedDate = new StringReader(sSerializedDate);
                deserializedDate = (DateTime)jsonDeserializer.Deserialize(srDeserializedDate, typeof(DateTime));
                //date here is "01/Jan/12 3:00:00 PM" Kind=Local
            }

Regards,

Karl

[Kijana Woodard]

What about when the date originates from a client in yet another time zone?

What about when you really want a local time in the db?

What about when you really trying to model a date without a time: March 3rd?

There are a lot of cases raven can't make universal assumptions about.

That said, maybe you can set the serializer for raven. It is json.net.

--

[Karl Cassar]

Hi,

Yes, ideally you set these settings for the JsonSerializer.  However, considering this is done internally using the RavenDB Client, I don't know if this can be set.  Normally, when you create a JsonSerializer, you can pass it the settings - and this way it can be customized.  Similar to the test code in my last post.

Regards,

Karl

[Kijana Woodard]

Like this:
http://ravendb.net/docs/client-api/advanced/custom-serialization

--

[Oren Eini (Ayende Rahien)]

And the best way to do that is to use DateTimeOffset

[Karl Cassar]

Somehow I had never heard about DateTimeOffset - thanks for that!  It definitely makes more sense to use that rather than DateTime as it stores the exact point in time, unambiguously.

Regards,

Karl

[Matt Johnson]

Hi Karl,

Sorry I didn't jump into this conversation earlier.  Been a bit busy lately. :)

Yes - DateTimeOffset is your friend here.  For a deeper understanding, see my post on StackOverflow:

http://stackoverflow.com/a/14268167/634824

Keep in mind a couple of other things as dates relate to JSON and RavenDB:

1) There is no mention of dates in the JSON specification.  At first, people used Microsoft's screwy convention of ticks wrapped in some escape chars, such as "\/Date(628318530718)\/" - which entirely contrived and unreadable.  The only benefit is how it can be finagled into a JavaScript Date object - which are also screwy.  Most people have abandoned this now, and use the ISO8601 format instead.

2) ISO8601 has multiple representations.  The ones we care about for RavenDB are:

    YYYY-MM-DDTHH:MM:SS.fffffff  - which we use to represent a DateTime with Unspecified kind

    YYYY-MM-DDTHH:MM:SS.fffffffZ - which we use to represent a DateTime with UTC kind

    YYYY-MM-DDTHH:MM:SS.fffffffZZZ - where ZZZ is an offset like +05:30 or -07:00.  We use this to represent a DateTimeOffset

Note that there is no representation that can adequately represent a DateTime with Local kind.  This is actually a pretty screwy concept if you think about it.  We only ever get one of these from DateTime.Now - and the information about which clock was local is lost as soon as we retrieve it.  We really can't expect that it stay in Local kind once we hand it off to someone else - like storing it in the database.  Who knows what "local" will mean when it is retrieved.  What if we have switched time zones or crossed a daylight savings transition since it was restored?

3) What is really important when you are working with dates is that you think about context.  What does the date/time mean?  If it is the time something occurred or will occur  (an event time) one usually wants to be able to know that point in time unambiguously.  Often, the answer is to store as UTC - but storing as a DateTimeOffset is also unambiguous and it has the added benefit of preserving the perspective context that it was recorded in.  Any DateTime math you might need to do (adding hours or days, subtracting for duration, etc.) or any range lookups (such as querying a database with a where or orderby) will certainly need to be unambiguous.  With RavenDB - any DateTimeOffset you store is persisted as a DateTimeOffset, but indexed as a UTC DateTime so you can sort and filter properly.

HOWEVER - sometimes your context is not a specific event.  Perhaps it is a recurring event, such as the operating hours of a business, or the time a television show is to air every week.  In those cases, you really do want to store just an unspecified DateTime.  At some point, you will combine this information with some other context (like the timezone of the location of the business, or the station the show is airing on).

So my point is, think about the context.  Not all dates are equal, and the web is full of blanket responses like "just store everything in UTC" - or conversely "I'm not global so I can just use local dates for everything".  Both statements are rash and not well thought out.  Never do math on DateTimes that are ambiguous.  Never subtract two local dates.  It might *work* right up until you cross a DST boundary and then who knows what consequences it will bring.

4)  If you plan on being able to edit a document and change an event time that is stored with a DateTimeOffset - if you want that offset to be meaningful then you also need to store a time zone identifier string.  Many time zones can change their offset, so the only thing a DTO gives you is the offset that happened to be in effect at that particular time.  If you change the time, then you might (or might not) also need to change the offset.

5) Lastly, realize that Microsoft Windows and .Net use a timezone database that is proprietary.  The rest of the world has standardized on the IANA/Olson/TZ/ZoneInfo database (lots of names - same source data).  The best solution if you want to use this in .Net is NodaTime.  Presently, if you want to use NodaTime with RavenDB, you have to translate to .Net native DateTime and DateTimeOffset classes.  I am working on a contrib project that will add native NodaTime support to RavenDB, but it requires some hooks that won't make it in to Raven until the upcoming 2.5 release.

And if you really want to understand why DateTime is screwy with it's .Kind property - have a read: http://noda-time.blogspot.com/2011/08/what-wrong-with-datetime-anyway.html

Cheers!

-Matt

[Kijana Woodard]

Thanks Matt for this write up. Community wiki? :-D

[Matt Johnson]

The best community wiki I know of for this kind of stuff is:

http://stackoverflow.com/questions/2532729/daylight-saving-time-and-timezone-best-practices

There are a few things in there that aren't so great, but at least it gets you thinking along the right lines.

Perhaps something RavenDB specific is in order.  I'll see about a KB article for the RavenDB website when I get some time.

[Kijana Woodard]

pun intended. ;-)

[Karl Cassar]

Thanks a lot Matt for the very detailed post! DateTimeOffset makes much sense for most of the times I want to deal with dates in RavenDB.  Your post and it's links are extremely helpful in helping one decide which one to use over the other.

Regards,

Karl

[Kijana Woodard]

Since Matt turned me onto DateTimeOffset and the perils of time generally, it makes sense to use DateTimeOffset often...with or without raven db. :-)

--

[Karl Cassar]

Yes definitely, thanks for bringing this to my attention as had never heard about that data structure, even though been using .Net for a couple of years now!

Regards,

Karl