2.5 to 3.0 upgrade -- Invalid cast with DateTimeOffset values of 0001-01-01T00:00:00.0000000

[Kamran Ayub]

I'm using the Streaming API and I am getting a new error I never encountered in 2.5.3599, which is Raven is failing to deserialize one of my nested objects due to some casting issue.

Document sample (imported from my 2.5 DB):

{

  Id: "Etc/1",

  Prop1: {

    SomeDate: "0001-01-01T00:00:00.0000000"

  }

}

My C# class has SomeDate defined as a DateTimeOffset.

public DateTimeOffset SomeDate { get; set; }

Any ideas? I am using TypeNameHandling.None. This error doesn't occur with the same data in my 2.5 code. If I paste in a "normal" date, it works OK (e.g. 2015-02-01T14:58:11.9044321)... but unfortunately many of my docs have "blank" (min value) values since it was a new property I added later.

The error I get is Specified cast invalid.

       InnerException: Raven.Imports.Newtonsoft.Json.JsonSerializationException

            HResult=-2146233088

            Message=Could not read value for property: SomeDate

            Source=Raven.Abstractions

            StackTrace:

                 at Raven.Imports.Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id) in c:\Builds\RavenDB-Stable-3.0\Imports\Newtonsoft.Json\Src\Newtonsoft.Json\Serialization\JsonSerializerInternalReader.cs:line 1739

                 at Raven.Imports.Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue) in c:\Builds\RavenDB-Stable-3.0\Imports\Newtonsoft.Json\Src\Newtonsoft.Json\Serialization\JsonSerializerInternalReader.cs:line 384

                 at Raven.Imports.Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue) in c:\Builds\RavenDB-Stable-3.0\Imports\Newtonsoft.Json\Src\Newtonsoft.Json\Serialization\JsonSerializerInternalReader.cs:line 254

                 at Raven.Imports.Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty containerProperty, JsonReader reader, Object target) in c:\Builds\RavenDB-Stable-3.0\Imports\Newtonsoft.Json\Src\Newtonsoft.Json\Serialization\JsonSerializerInternalReader.cs:line 743

                 at Raven.Imports.Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id) in c:\Builds\RavenDB-Stable-3.0\Imports\Newtonsoft.Json\Src\Newtonsoft.Json\Serialization\JsonSerializerInternalReader.cs:line 1729

            InnerException: Raven.Imports.Newtonsoft.Json.JsonSerializationException

                 HResult=-2146233088

                 Message=Error setting value to 'SomeDate' on 'SomeClass'.

                 Source=Raven.Abstractions

                 StackTrace:

                      at Raven.Imports.Newtonsoft.Json.Serialization.DynamicValueProvider.SetValue(Object target, Object value) in c:\Builds\RavenDB-Stable-3.0\Imports\Newtonsoft.Json\Src\Newtonsoft.Json\Serialization\DynamicValueProvider.cs:line 88

                      at Raven.Imports.Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty containerProperty, JsonReader reader, Object target) in c:\Builds\RavenDB-Stable-3.0\Imports\Newtonsoft.Json\Src\Newtonsoft.Json\Serialization\JsonSerializerInternalReader.cs:line 752

                      at Raven.Imports.Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id) in c:\Builds\RavenDB-Stable-3.0\Imports\Newtonsoft.Json\Src\Newtonsoft.Json\Serialization\JsonSerializerInternalReader.cs:line 1729

                 InnerException: System.InvalidCastException

                      HResult=-2147467262

                      Message=Specified cast is not valid.

                      Source=Etc

                      StackTrace:

                           at SetLastUpdated(Object , Object )

                           at Raven.Imports.Newtonsoft.Json.Serialization.DynamicValueProvider.SetValue(Object target, Object value) in c:\Builds\RavenDB-Stable-3.0\Imports\Newtonsoft.Json\Src\Newtonsoft.Json\Serialization\DynamicValueProvider.cs:line 84

                      InnerException: 

[Kamran Ayub]

So it looks like it expects the MinValue roundtrip format:

0001-01-01T00:00:00.0000000+00:00

But instead right now the document is using the DateTime.MinValue roundtrip format:

0001-01-01T00:00:00.0000000

But looking through the history, the code has always used DateTimeOffset (since it was added not too long ago). And 2.5 doesn't choke on it either, so I am not sure what's up.

[Oren Eini (Ayende Rahien)]

You can do a PATCH to update this, it would probably be the easiest to solve it.

Note that the date time format is not a date time offset, because we don't know what to put there.

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

--
You received this message because you are subscribed to the Google Groups "RavenDB - 2nd generation document database" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Kamran Ayub]

I just tested saving a test class to my local Raven 2.5.3599 instance and it wrote the DateTimeOffset.MinValue round-trip format. So I cannot explain why in my production database when I only used DateTimeOffset it saved the "wrong" roundtrip format.

On Monday, February 23, 2015 at 10:55:25 PM UTC+1, Kamran Ayub wrote:

[Kamran Ayub]

OK, I am not crazy.

I imported my production 2.5 dump locally and my DateTimeOffsets are all in the correct format:

0001-01-01T00:00:00.0000000+00:00

2015-02-01T15:00:23.3088773+00:00

The same document in 3.0 production (RavenHQ) is missing the timezones:

0001-01-01T00:00:00.0000000

2015-02-01T15:00:23.3088773

This is why it is failing.

However, to make it even stranger, my staging database which I upgraded the other day (on RavenHQ) imported the correct data in 3.0! I created the dump the same way and imported the same way (using the GUI).

On Monday, February 23, 2015 at 10:55:25 PM UTC+1, Kamran Ayub wrote:

[Kamran Ayub]

Apologies, my 2.5 instance has been 2952. My 3.0 is 3599.

On Monday, February 23, 2015 at 10:55:25 PM UTC+1, Kamran Ayub wrote:

[Kamran Ayub]

OK, I can confirm importing a 2.5.2952 ravendump file containing DateTimeOffsets fails to import successfully in 3.0.3599. I just tested importing both my production and staging dumps locally. My staging dump was taken Feb 21 (and imported to RavenHQ same day) and my production dump was taken today (23rd).

I don't know why importing a few days ago to staging RavenHQ 3.0 instance has correct data when importing locally doesn't work (same dump). The only explanation is magic gnomes.

On Monday, February 23, 2015 at 10:55:25 PM UTC+1, Kamran Ayub wrote:

[Kamran Ayub]

Hey Ayende,

I tried fixing this at the import level by writing a quick transform script but it seems the issue might lie in the smuggler API.

I basically did something like:

function (doc) {

if (doc.SomeDate && doc.SomeDate.toString().indexOf("+00:00") < 0) {

doc.SomeDate += "+00:00";

}

    return doc;

}

And I got an error:

System.AggregateException: One or more errors occurred. ---> System.ArgumentOutOfRangeException: Year, Month, and Day parameters describe an un-representable DateTime.
   at System.DateTime.DateToTicks(Int32 year, Int32 month, Int32 day)
   at System.DateTime..ctor(Int32 year, Int32 month, Int32 day)
   at Jint.Native.Date.DatePrototype.DaylightSavingTa(Double t)
   at Jint.Native.Date.DatePrototype.Utc(Double t)
   at Jint.Native.Date.DateConstructor.FromDateTime(DateTime dt)
   at Raven.Database.Json.JintOperationScope.ToJsInstance(Engine engine, RavenJToken value, String propertyKey)
   at Raven.Database.Json.JintOperationScope.ToJsObject(Engine engine, RavenJObject doc, String propertyName)
   at Raven.Database.Smuggler.SmugglerJintHelper.Transform(String transformScript, RavenJObject input)
   at Raven.Database.Smuggler.SmugglerEmbeddedDatabaseOperations.TransformDocument(RavenJObject document, String transformScript)
   at Raven.Abstractions.Smuggler.SmugglerDatabaseApiBase.<ImportDocuments>d__99.MoveNext()
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
   at Raven.Abstractions.Smuggler.SmugglerDatabaseApiBase.<>c__DisplayClass6e.<ImportData>b__67()
   at Raven.Abstractions.Smuggler.SmugglerDatabaseApiBase.<ImportData>d__70.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Raven.Abstractions.Smuggler.SmugglerDatabaseApiBase.<ImportData>d__5c.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Raven.Database.Server.Controllers.StudioTasksController.<>c__DisplayClassf.<<ImportDatabase>b__b>d__14.MoveNext()
---> (Inner Exception #0) System.ArgumentOutOfRangeException: Year, Month, and Day parameters describe an un-representable DateTime.
   at System.DateTime.DateToTicks(Int32 year, Int32 month, Int32 day)
   at System.DateTime..ctor(Int32 year, Int32 month, Int32 day)
   at Jint.Native.Date.DatePrototype.DaylightSavingTa(Double t)
   at Jint.Native.Date.DatePrototype.Utc(Double t)
   at Jint.Native.Date.DateConstructor.FromDateTime(DateTime dt)
   at Raven.Database.Json.JintOperationScope.ToJsInstance(Engine engine, RavenJToken value, String propertyKey)
   at Raven.Database.Json.JintOperationScope.ToJsObject(Engine engine, RavenJObject doc, String propertyName)
   at Raven.Database.Smuggler.SmugglerJintHelper.Transform(String transformScript, RavenJObject input)
   at Raven.Database.Smuggler.SmugglerEmbeddedDatabaseOperations.TransformDocument(RavenJObject document, String transformScript)
   at Raven.Abstractions.Smuggler.SmugglerDatabaseApiBase.<ImportDocuments>d__99.MoveNext()<---

On Monday, February 23, 2015 at 10:55:25 PM UTC+1, Kamran Ayub wrote:

[Oren Eini (Ayende Rahien)]

What does it means, fails to import?

It gets into RavenDB, right?

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

[Oren Eini (Ayende Rahien)]

Can you send us a smuggler dump with a document that has this issue?

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

[Kamran Ayub]

Yes, but I will have to send it separately, maybe through a RavenHQ support ticket. What I mean is that the date time offset values import incorrectly (without the +00:00) but import correctly in 2.5.2952. I can explain more when I send the dump.

You received this message because you are subscribed to a topic in the Google Groups "RavenDB - 2nd generation document database" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ravendb/5tctvUkw1MM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ravendb+u...@googlegroups.com.

[Kamran Ayub]

I was able to repro with a single document. I've sent it in a support ticket but since it's not proprietary data, here it is along with repro steps!

Steps to reproduce: 
1. Create a 2.5.2952 database 
2. Create a document that uses a DTO value and set to valid offset: 2015-01-18T05:41:11.0000000+00:00 
3. Export the database to a dump file using Raven Studio 
4. Create a new 3.0.3599 database 
5. Import the dump file 
6. Observe the imported document has a different value: 2015-01-18T05:41:11.0000000

To test this dump, just import it into a blank 2.5 and 3.0 database to see the behavior.

On Monday, February 23, 2015 at 10:55:25 PM UTC+1, Kamran Ayub wrote:

[Oren Eini (Ayende Rahien)]

You sent that to ravenhq, which is very different.

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

[Oren Eini (Ayende Rahien)]

This has nothing to do with import / export.

The issue is here:

    "LastUpdated": "2015-01-18T08:41:11.0000000"

This is the format of a DateTime, not a DateTimeOffset.

[Kamran Ayub]

I don't think you understand... the issue is very clear, I tried to explain it the best I can.

Import the dump to a brand new 2.5 database. The value is 2015-01-18T08:41:11.0000000+00:00. Import the dump to a new 3.0 database, the value is 2015-01-18T08:41:11.0000000 (note the lack of offset).

You received this message because you are subscribed to a topic in the Google Groups "RavenDB - 2nd generation document database" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ravendb/5tctvUkw1MM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ravendb+u...@googlegroups.com.

[Kamran Ayub]

Here's the same document, side-by-side, after importing to a 2.5 and 3.0 database. There is an issue.

On Monday, February 23, 2015 at 10:55:25 PM UTC+1, Kamran Ayub wrote:

[Oren Eini (Ayende Rahien)]

Found the issue. We seems to be setting date time handling on the server side, and we shouldn't.

This will be fixed soon

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

[Kamran Ayub]

Thank you!!! Glad I could help. Sorry for the back and forth; should have just started with a screenshot ;)

You received this message because you are subscribed to a topic in the Google Groups "RavenDB - 2nd generation document database" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ravendb/5tctvUkw1MM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ravendb+u...@googlegroups.com.