Maddening error: "marshal data too short"

[Kyle Fox]

I'm occasionally getting an odd Rails error that says "marshal data
too short".

Our entire web app seems to be working fine -- we can post forms, save
models, redirect, login, etc, without any problems. Then out of
nowhere a request will result in the 500 error "marshal data too
short". Subsequent requests to *any* page then result in the same
error -- even pages that were previously working.

The only way to (temporarily) make the problem go away is to `rake
db:sessions:clear` -- but I need to determine what the root cause is.

Some forum posts led me to believe the `data` column in the Rails
sessions table was too short. I have it as 'text', which should
support roughly 65,000 characters. The data in my actual session is
only 305 characters, so I doubt this is the problem.

It's basically impossible for me to replicate the problem exactly, as
I have no idea what's triggering it. If anyone could shed some light
on how I could investigate further, I'd be ever so thankful :)

Kyle.

[Frederick Cheung]

On Aug 7, 12:07 am, Kyle Fox <kyle....@gmail.com> wrote:
>
> Some forum posts led me to believe the `data` column in the Rails
> sessions table was too short.  I have it as 'text', which should
> support roughly 65,000 characters.  The data in my actual session is
> only 305 characters, so I doubt this is the problem.
>

I wouldn't use a text column (use a blob column instead). If you use a
text column then the database is expecting the content to be actual
text. If it's expecting utf8 and you give it invalid utf8 data it may
well truncate your data at the first occurrence of illegal data.

Fred

[Kyle Fox]

> I wouldn't use a text column (use a blob column instead).

AFAIK, we just used the default table and column types that Rails
creates for sessions. Why in the world wouldn't it use a more robust
column type in the first place?

[Ilan Berci]

Kyle,

How much stuff are you storing in user sessions?
ilan
--
Posted via http://www.ruby-forum.com/.

[Frederick Cheung]

What session store are you using? if it's the default activerecord
store then it's base64 encoding everything and so you can ignore my
comment.
If I were you I'd instrument the session store code to log when it
updates the sessions table and when it loads from it: if you can spot
how the data changes when this error occurs that may be a valuable
clue into why it changes.

Fred

[Kyle Fox]

> How much stuff are you storing in user sessions?

We're storing basic authentication info and then a little object that
holds some data to pre-populate a form. I wouldn't think that's too
much data to be storing in a session....

The last time I encountered this error I ran `SELECT LENGTH(data) FROM
sessions;` and it returned "305", which from what I understand is the
number of bytes -- well below the max length of TEXT columns.

I'm going to try using a BLOB, and we'll see if the error occurs
again. However, I have a feeling the problem isn't too much data in
the session, so feel free to post any other suggestions :)

[Kyle Fox]

Fred: we're using activerecord. Perhaps I'll try adding some code to
log what's happening behind the scenes.