That "Unexpected end of content while loading JObject" error

[Ayende Rahien]

Okay, the story for that is quite interesting, and requires digging through the annals of history for RavenDB.

In short, it is a combination of several things together that trigger this bug:

* The use of Distributed Transactions.

* Deletion of a document withing DTC tx.

* A specific sequence of bytes in the serialized BSON format of the document metadata.

That has made this bug tremendously difficult to isolate and reproduce.

In particular, finding out that it was deletion that was triggering the issue wasn't fun at all.

If you care, you can look at the fix for this issue at: 53f28f491ebb135f9fd8e53333abee1aa3c27462

This is a one line fix, believe it or not. 

The underlying reason is that a long time ago, we saved metadata in JSON format to the Esent storage, but a while ago we switched to using BSON internally.

Unfortunately, I forgot to update one location (delete in a dtc tx), and that still treated that value as a string. In general, it wasn't an issue, because the place only copy the value from one place to another, converting it to & from unicode string.

Given a particular sequence of bytes, however, the unicode conversion wouldn't result in a seamless transfer, and would corrupt the internal structure of data inside the database.

This was surfaced as the "Unexpected end of content while loading JObject" error, upon startup, because this was a DTC issue, RavenDB attempted to recover the failed transaction, and attempted to read the same corrupted data again, resulting in a failure to start.

This issue has been fix, and is currently in the build box.

I would like to apologize for the trouble, and for the long time that it took to isolate and fix it.

And I would like to thank Rob Ashton if helping me fixing  

[Ayende Rahien]

And I would like to thank Rob Ashton if helping me fixing  this bug.

[John Simons]

Ayende,

Congrats for figuring this one out :)

Since this was just a one line code fix, are you planning to merge
this fix into the stable release?

Cheers
John

On Mar 22, 9:22 am, Ayende Rahien <aye...@ayende.com> wrote:
> And I would like to thank Rob Ashton if helping me fixing  this bug.
>
>
>
>
>
>
>
> On Tue, Mar 22, 2011 at 12:22 AM, Ayende Rahien <aye...@ayende.com> wrote:
> > Okay, the story for that is quite interesting, and requires digging through
> > the annals of history for RavenDB.
>
> > In short, it is a combination of several things together that trigger this
> > bug:
> > * The use of Distributed Transactions.
> > * Deletion of a document withing DTC tx.
> > * A specific sequence of bytes in the serialized BSON format of the
> > document metadata.
>
> > That has made this bug tremendously difficult to isolate and reproduce.
>
> > In particular, finding out that it was deletion that was triggering the
> > issue wasn't fun at all.
>
> > If you care, you can look at the fix for this issue
> > at: 53f28f491ebb135f9fd8e53333abee1aa3c27462

> > This is *a one line fix*, believe it or not.

[nabils]

That's great news!!!

[Ayende Rahien]

Yes

[Ayende Rahien]

Probably so.

That said, I'll probably do so for the specific people who go to prod soon, and wait a bit and then make another stable release.

[DanPlaskon]

Any idea when we can expect the next stable release?

On Mar 21, 7:10 pm, Ayende Rahien <aye...@ayende.com> wrote:
> Probably so.
> That said, I'll probably do so for the specific people who go to prod soon,
> and wait a bit and then make another stable release.
>

> > > > And I would like to thank Rob Ashton if helping me fixing- Hide quoted text -
>
> - Show quoted text -

[Ayende Rahien]

Monday