Custom serialization

[Ariel Raunstien]

Hey.

Say I have a serializable object (graph) and I get poor performance for the serailization process (speed-wise).

Is there a rule of thumb on when I should implement the serialization myself to gain speed?

Arielr

http://blogs.microsoft.co.il/blogs/ariel/

[Ken Egozi]

q: when should I implement myself to gain speed?

a: when the default approach is too slow

:)

--
Ken Egozi.
http://www.kenegozi.com/blog
http://www.delver.com
http://www.musicglue.com
http://www.castleproject.org
http://www.idcc.co.il - הכנס הקהילתי הראשון למפתחי דוטנט - בואו בהמוניכם

[Ariel Raunstien]

Smartass.

For the *specific* feature of serialization, with the *specific* problem of speed - do any of you have any *specific* rules of a *specific* thumb?

Arielr

http://blogs.microsoft.co.il/blogs/ariel/

[Lior Friedman]

1)      Locate the slowest part during the serializtion

2)      Optimize that part until

a.       It’s not the slowest part

b.      You can’t optimize it anymore.

Repeat until *specific* problem of speed is no longer a problem.

 

Just so you’ll know that we (I) don’t understand your question.

 

Lior Friedman
Blog  - http://imistaken.blogspot.com

[Ken Egozi]

I agree with Lior - to your super generic problem, our generic solution apply.

you are being too fluid.

do you have a dotTrace output showing that the bottleneck is in Serialize? in which part of it? can you share code?

[Ariel Raunstien]

I have some module.

That module creates (and caches) some objects during its life time, and some of these objects are the same for every run.

Creating and caching X objects takes Y time.

I've decided to pre-build these objects, serialize and deploy them with the app, so that in runtime - I could just deserialize them into the cache.

Problem - deserializing the objects takes very close to Y.

"Should I write custom serialization code"?

Arielr

http://blogs.microsoft.co.il/blogs/ariel/

[hmemcpy]

How about just trying a different serializer?
http://code.google.com/p/protobuf-net/wiki/Performance

[Ariel Raunstien]

Been there, done that. :)

Arielr

http://blogs.microsoft.co.il/blogs/ariel/

[Lior Friedman]

Hi Ariel,

 

At the risk of being very annoying.

 

Is there a possibility that your chasing your tail in the wrong direction?

 

(Roy is going to love this)

 

The original question was “How can I optimize serialize/deserialize?”

 

1)      Why do you want to optimize them?

Answer: cause I’m preloading a cache and it’s not fast enough

2)      Why are you preloading the cache?

Answer: Cause creating the cache in run time is too long.

3)      Why are you creating a cache?

Answer: Cause some of the objects are the same for every run.

4)      Why not create them at runtime?

Answer: (pure assumption) creating them in runtime takes too long.

 

Possible solution:

Optimize the creation of your object to the point it becomes a non-issue to create them at runtime.

 

 

Now I assume you went through all this and the actual object creation is completely optimized, and still it’s too slow, and you need to preload the cache.

 

If I were you I would start by analyzing which parts of the object tree is causing you grief. It might be that you only need to optimize only some of the classes you are using.

 

Also this sounds a lot like something that in memory object model data base might solve.

 

Lior

 

 

[onemenny]

it sound like you've been trying for a while (different serializers,
attributes) and i see no other option then writing your own serlzr.
there are no rule o thumb beside discarding unnecessary dependencies,
and using the right srlzr - in your case one that srlzes a graph.

writing a good graph srlzr is not an easy process... (depending on the
complexity of the graph+depth)

On Sep 22, 12:12 pm, Ariel Raunstien <arie...@gmail.com> wrote:
> Been there, done that. :)
>
> Arielr
>
> http://blogs.microsoft.co.il/blogs/ariel/
>

> On Tue, Sep 22, 2009 at 11:14 AM, hmemcpy <hmem...@gmail.com> wrote:
> > How about just trying a different serializer?
> >http://code.google.com/p/protobuf-net/wiki/Performance
>

> > On Tue, Sep 22, 2009 at 11:13 AM, Ariel Raunstien <arie...@gmail.com>wrote:
>
> >> I have some module.That module creates (and caches) some objects during

> >> its life time, and some of these objects are the same for every run.
> >> Creating and caching X objects takes Y time.
>
> >> I've decided to pre-build these objects, serialize and deploy them with
> >> the app, so that in runtime - I could just deserialize them into the cache.
>
> >> Problem - deserializing the objects takes very close to Y.
>
> >> "Should I write custom serialization code"?
>
> >> Arielr
>
> >>http://blogs.microsoft.co.il/blogs/ariel/
>

> >> On Tue, Sep 22, 2009 at 9:24 AM, Ken Egozi <egoz...@gmail.com> wrote:
>
> >>> I agree with Lior - to your super generic problem, our generic solution
> >>> apply.
> >>> you are being too fluid.
>
> >>> do you have a dotTrace output showing that the bottleneck is in
> >>> Serialize? in which part of it? can you share code?
>

> >>> On Tue, Sep 22, 2009 at 9:18 AM, Lior Friedman <lfried...@gmail.com>wrote:
>
> >>>>  1)      Locate the slowest part during the serializtion
>
> >>>> 2)      Optimize that part until
>
> >>>> a.       It’s not the slowest part
>
> >>>> b.      You can’t optimize it anymore.
>

> >>>> Repeat until **specific** problem of speed is no longer a problem.

>
> >>>> Just so you’ll know that we (I) don’t understand your question.
>

> >>>> *Lior Friedman
> >>>> *Blog  -http://imistaken.blogspot.com
>
> >>>> *From:* altnet...@googlegroups.com [mailto:
> >>>> altnet...@googlegroups.com] *On Behalf Of *Ariel Raunstien
> >>>> *Sent:* Monday, September 21, 2009 11:10 PM
> >>>> *To:* altnet...@googlegroups.com
> >>>> *Subject:* Re: Custom serialization

>
> >>>> Smartass.
>
> >>>> For the *specific* feature of serialization, with the *specific* problem
> >>>> of speed - do any of you have any *specific* rules of a *specific* thumb?
>
> >>>> Arielr
>
> >>>>http://blogs.microsoft.co.il/blogs/ariel/
>
> >>> --
> >>> Ken Egozi.
> >>>http://www.kenegozi.com/blog
> >>>http://www.delver.com
> >>>http://www.musicglue.com
> >>>http://www.castleproject.org

> >>>http://www.idcc.co.il- הכנס הקהילתי הראשון למפתחי דוטנט - בואו בהמוניכם

[Ariel Raunstien]

onemenny - I'm not trying to write a serializer, just to implement per-object serialization.

Lior - good point, however:

1. I serialize these objects across app-domains too, so I'd gain more by improving the serialization;

2. Improving performance of serialization, in my mind, seems easier than improving other processes.

3. Since these objects are created exactly the same each and every run - it seemed like a proper path to take. (a counter-argument here is the fact that the serialized objects are not the only ones created by this process, and that improving the creation time would benefit other parts of the application as well).

Arielr

http://blogs.microsoft.co.il/blogs/ariel/