Serializing the configuration

[Tuna Toksoz]

Yesterday I played with the idea of Serializing the configuration. People complain about sessionFactory initialization takes forever and serialization is an option most of the time.
I tried the idea of serializing SessionFactory which wasn't possible since it has dependency on many nonserializable classes within NH (and marking them as serializable is time consuming).  Instead, I tried serializing Configuration right after the buildSessionFactory operation

            BinaryFormatter fm = new BinaryFormatter();
            Stopwatch sw = Stopwatch.StartNew();
            FileStream fs = new FileStream("myfile.dat", FileMode.OpenOrCreate, FileAccess.Write);
            Configuration cfg = new Configuration();
            cfg.Configure();
            cfg.AddAssembly("

DomainWith75EntityClass");
            cfg.Configure();
            cfg.BuildSessionFactory();
            fm.Serialize(fs, cfg);
            sw.Stop();
            Console.WriteLine(sw.ElapsedMilliseconds);

before BuildSessionFactory, serialization wasn't possible as System.Xml.XmlSchema is nonserializable.

This took 3998 ms

and then I tried

            BinaryFormatter fm = new BinaryFormatter();
            Stopwatch sw = Stopwatch.StartNew();
            FileStream fs = new FileStream("myfile.dat", FileMode.Open, FileAccess.Read);
            var cfg = fm.Deserialize(fs) as Configuration;
            cfg.BuildSessionFactory();
            sw.Stop();
            Console.WriteLine(sw.ElapsedMilliseconds);

this took 1578 ms on average.

Should I make Configuration serializable, or do you think it is useless?

Tuna Toksöz
http://tunatoksoz.com
http://twitter.com/tehlike

Typos included to enhance the readers attention!

[Fabio Maulo]

Configuration useless.

Which are the classes, used in sessionFactory state, that are not serializable ?

2009/2/5 Tuna Toksoz <teh...@gmail.com>

--
Fabio Maulo

[Tuna Toksoz]

That I don't remember, I had to do one at a time and there are more than 10 class, I gave up on that then.

Tuna Toksöz
http://tunatoksoz.com
http://twitter.com/tehlike

Typos included to enhance the readers attention!

[Tuna Toksoz]

Serializing configuration saves you from XML validation, at least.

Tuna Toksöz
http://tunatoksoz.com
http://twitter.com/tehlike

Typos included to enhance the readers attention!

On Thu, Feb 5, 2009 at 3:53 PM, Fabio Maulo <fabio...@gmail.com> wrote:

[Tuna Toksoz]

I can play more with serializing SessionFactory if you find it useful

Tuna Toksöz
http://tunatoksoz.com
http://twitter.com/tehlike

Typos included to enhance the readers attention!

[Fabio Maulo]

2009/2/5 Tuna Toksoz <teh...@gmail.com>

Serializing configuration saves you from XML validation, at least.

Serializing SessionFactory saves you from "nh-compilation-process"

--
Fabio Maulo

[Tuna Toksoz]

I know that it is better :)

Tuna Toksöz
http://tunatoksoz.com
http://twitter.com/tehlike

Typos included to enhance the readers attention!

[Fabio Maulo]

The sessionFactory should be serializable.

If I well remember there is a method to upload a serialized sessionFactory

2009/2/5 Tuna Toksoz <teh...@gmail.com>

--
Fabio Maulo

[Tuna Toksoz]

Ok, I'll check that tonight.

Tuna Toksöz
http://tunatoksoz.com
http://twitter.com/tehlike

Typos included to enhance the readers attention!

[Ayende Rahien]

And that tends to be the most expensive part of the operation, IIRC.

On Thu, Feb 5, 2009 at 3:56 PM, Tuna Toksoz <teh...@gmail.com> wrote:

[Tuna Toksoz]

yeap, I got upto 60% improvement with 75 entities.

Tuna Toksöz
http://tunatoksoz.com
http://twitter.com/tehlike

Typos included to enhance the readers attention!

[s_tristan]

Can you make a speed benchmark with traditional method and
serialization?

[Tuna Toksoz]

Sure I will, serializign configuration speeds up but not as much as serializing SessionFactory. You'll need to wait for it.

Tuna Toksöz
http://tunatoksoz.com
http://twitter.com/tehlike

Typos included to enhance the readers attention!

[Tuna Toksoz]

But in order to have a meaningful benchmark, I will need to have several hundreds of entities, I guess, which scares me :D

Tuna Toksöz
http://tunatoksoz.com
http://twitter.com/tehlike

Typos included to enhance the readers attention!

[Fabio Maulo]

The serialization of the sessionFactory is useful only in few cases.

In practice...

The BuildSessionFactory is the "in-ram compilation" of all mappings and persistence stuff.

Serializing the sessionFactory and saving it to your hard-drive is equivalent to create an "EXE" of the persistence stuff.

2009/2/5 Tuna Toksoz <teh...@gmail.com>

--
Fabio Maulo

[Tuna Toksoz]

So it comes back to "serializing Configuration" idea? as in ram operations are way cheaper than xml validation.

Tuna Toksöz
http://tunatoksoz.com
http://twitter.com/tehlike

Typos included to enhance the readers attention!

[Fabio Maulo]

What is needed is the SessionFactory serialization.

The Conf should have a method to upLoad a serialized sessionFactory and you can use this method instead call BuildSessionFactory.

2009/2/5 Tuna Toksoz <teh...@gmail.com>

--
Fabio Maulo

[Stephen Bohlen]

I confess to some confusion about how this proposed change would actually be used in a practical example.  Perhaps my ignorance is clouding my vision :)

 

It seems to me it would have to be either....

 

1) call BuildSessionFactory() once at the beginning of your app init (e.g., Main() or Application_Start() in the global.asax) and then persist the serialized already-built session factory to disk so that subsequent calls to get the session factory elsewhere in your code would be able to use the pre-built serialized instance of the session factory

 

2) create the serialized session factory as some kind of pre-build compilation step so that the serialized instance of the pre-built session factory would be part of the deployment package for the app and the app would then NEVER need to do other than load the session factory from disk and then deserialize it to make use of it

 

If #1 then I must be in the minority of people who have setup my apps to build the session factory just once then provide the same instance of it back to the rest of the app any time its needed to get a new session.  In a web app, this 'penalty' is only experienced by the first visitor to the site after the IIS app pool is restarted while in a win client app its perhaps more penalizing since each user each time they launch the app would of course re-experience the 'pain' of building the session factory.  But even in this kind of case (where the serialized session factory is built once on first-launch and then serialized to disk for later rapid-loading) wouldn't you nearly always have to rebuild it again on re-launch just to ensure that any config changes (e.g., connection string, etc.) made since it was last serialized are reflected in the 'version' of the session factory that your app is using?

 

If its #2, then this would seem to me to have some merit because in this scenario the pre-built serialized session factory would just be part of the binary deployment package of the app, but in a very real sense it seems to me that this has two big issues:

 

a) how would someone be creating this serialized session factory as part of a build process --?  perhaps through some kind of invoked unit test that calls BuildSessionFactory and then persists it to disk? (if so, then we now are saying "you cannot actually build your app unless you also invoke this 'special' unit test here to create the serialized session factory")...?

 

b) NOTHING about the deployment could be changed via a config file (e.g., not even the DB connection string would be part of the config so you would need the 'dev' serialized session factory, the 'test/staging' serialized session factory, the 'prod' serialized session factory, etc. (and that doesn't even touch on the issues of what you do if/when you're building a *product* where each customer needs a different connection string to their own DB --!)

 

In short, while I think I proprerly understand the technical goal here (avoid the performance penalty of calling BuildSessionFactory() by persisting the results of the call to disk for later re-use), I'm not entirely sure I fully understand the practical application of this capability even if we were to build it.

 

I must be overlooking some rather obvious aspect of how this capability would be used in the real world -- can someone take a sec and explain to me what I'm overlooking ???

 

More confused that usual,

 

-Steve B.

[Steve Strong]

Thinking aloud, I'm guessing that you'd want it to be fully automatic.  i.e., option 1 from your choices.  Ideally, BuildSessionFactory would do this itself internally, so that the user doesn't have to know.  You could, perhaps, disable this behaviour through configuration if you really wanted (perhaps you app doesn't have permission to write to disk, for example).

To handle config changes, I'd suggest the a hash of the config is stored along with the serialised factory; that way, changes can be detected automatically and the persisted version can be trashed.

Cheers,

Steve

[Tuna Toksoz]

Inline

I confess to some confusion about how this proposed change would actually be used in a practical example.  Perhaps my ignorance is clouding my vision :)

 

It seems to me it would have to be either....

 

1) call BuildSessionFactory() once at the beginning of your app init (e.g., Main() or Application_Start() in the global.asax) and then persist the serialized already-built session factory to disk so that subsequent calls to get the session factory elsewhere in your code would be able to use the pre-built serialized instance of the session factory

 

2) create the serialized session factory as some kind of pre-build compilation step so that the serialized instance of the pre-built session factory would be part of the deployment package for the app and the app would then NEVER need to do other than load the session factory from disk and then deserialize it to make use of it

 

If #1 then I must be in the minority of people who have setup my apps to build the session factory just once then provide the same instance of it back to the rest of the app any time its needed to get a new session.  In a web app, this 'penalty' is only experienced by the first visitor to the site after the IIS app pool is restarted while in a win client app its perhaps more penalizing since each user each time they launch the app would of course re-experience the 'pain' of building the session factory.  But even in this kind of case (where the serialized session factory is built once on first-launch and then serialized to disk for later rapid-loading) wouldn't you nearly always have to rebuild it again on re-launch just to ensure that any config changes (e.g., connection string, etc.) made since it was last serialized are reflected in the 'version' of the session factory that your app is using?

You're right, it is the advised way of using session factory.  However, as you said this penalty can sometimes bit seconds if not minutes for SessionFactory to  initialize, reducing this is a good option.  We can leave it upto the developer to make this change and remove the stuff if there is a change. Another option would be to have timestamps stored elsewhere to make sure the Domain and .Config file didn't change, and this wouldn't take so long, I guess.

My proposal would be to have a STATIC SessionFactory per application, which is initialized like the following

1. Check if there is a SessionFactory Serialization Binary
2. If yes, then check .config file and related assembly Last Modified Date, if those are newer than the Serialized factory, then go to step 4. If not step 3
3. Now we have a valid session factory, we can safely deserialize it.
4. Means that we either don't have a session factory, or it is pretty old. Create the session factory once more, serialize it to the disk.

[Stephen Bohlen]

That makes sense to me -- and I agree that sometimes on large apps with many persistent classes (and usually a correspondingly high number of mapping files) building the session factory can be an (embarrasingly) time-consuming process.

 

Both your and Steve's suggestions for ways to handle some of my related 'infrastructure behavior' concerns seem reasonable -- I just wanted to ensure that we were properly considering the 'real-world' use of this option in the context of the rest of what would need to happen for its implementation to make sense for practical use.

 

If the serialized session factory is auto-built and then serialized on first-run and we use either your or Steve's suggested approach to 'determine the obsolesence of the existing serialized session factory instance', what it seems like we are solving is ONLY the cold-start situation (new app pool in IIS or app-launch in win client), correct?  This would seem to have much more applicability to the win client scenario than the web app story except for those very high-traffic, high-availability web apps where the app pool is scheduled for a nightly restart to wipe out the effects of some resource leak that the devs are unable to properly find and fix (which we all know to be a real-world use case!).

 

As a side note, we need to also be cognizant of the fact that its many time the case that there are (of course) multiple session factories per app (ea for a diff DB, ea with a different caching strategy, etc.) and so whatever we come up with as a way to deserialize the factory instance of course needs to reflect this need too (multiple names session factories).

 

-Steve B.

[Tuna Toksoz]

I guess it wasn't really inline :)

Tuna Toksöz
http://tunatoksoz.com
http://twitter.com/tehlike

Typos included to enhance the readers attention!

[Ayende Rahien]

Just one note,
for big web app, the benefit isn't for the users, it is for the developers.
Imagine that we can reduce startup time from NH from 15 seconds to 2 seconds. Now we save the developers 13 seconds each time they hit F5.

[Tuna Toksoz]

Indeed.

Tuna Toksöz
http://tunatoksoz.com
http://twitter.com/tehlike

Typos included to enhance the readers attention!

[Fabio Maulo]

2009/2/6 Tuna Toksoz <teh...@gmail.com>

My proposal would be to have a STATIC SessionFactory per application, which is initialized like the following

1. Check if there is a SessionFactory Serialization Binary
2. If yes, then check .config file and related assembly Last Modified Date, if those are newer than the Serialized factory, then go to step 4. If not step 3
3. Now we have a valid session factory, we can safely deserialize it.
4. Means that we either don't have a session factory, or it is pretty old. Create the session factory once more, serialize it to the disk.

Too many responsibilities for NH.

Considering that, after BuildSessionFactory, you can discard the Configuration instance what we must do is only ensure that the SessionFactoryImpl class is serializable and let the user to choose how, where, when y can save-restore the serialized SessionFactory instance.

After that, if you want, we can provide a "default" mode implemented in some Helper class in NH-Core or in NH-Contrib.
--
Fabio Maulo

[Tuna Toksoz]

I am just thinking out loud, what I was to do is just making SessionFactory serializable, and let them choose what to do.

Tuna Toksöz
http://tunatoksoz.com
http://twitter.com/tehlike

Typos included to enhance the readers attention!

On Fri, Feb 6, 2009 at 3:32 PM, Fabio Maulo <fabio...@gmail.com> wrote:

onsidering that, aft

[Fabio Maulo]

2009/2/6 Tuna Toksoz <teh...@gmail.com>

I am just thinking out loud, what I was to do is just making SessionFactory serializable, and let them choose what to do.

Perfect.

For sure we will have more than one SessionFactoryLoader provided by NH's users useful in various app environments (as usual)

--
Fabio Maulo

[Stephen Bohlen]

I admit to not having considered this as a design-time (dev cycle) optimization vs a run-time (production) optimization -- excellent point!  I know *I* would take immediate advantage of just such a thing in several of my web apps even if I actually then 'reverted' to traditional behavior on-deployment.

 

-Steve B.

[Tuna Toksoz]

Hello again,

I took a look at sessionfactory & serialization and found an interesting thing.

SessionFactory is serializable but only it's name and uuid is serialized(maybe some other minor stuff).

SessionFactory implements IObjectReference(which I didn't know previously). After serialization takes place, the method "GetRealObject" is called to get the real object.

NH executes the following in that method:

        public object GetRealObject(StreamingContext context)
        {
            ISessionFactory result = SessionFactoryObjectFactory.GetInstance(uuid);
            if (result == null)
            {
                result = SessionFactoryObjectFactory.GetNamedInstance(name);
                if (result == null)
                    throw new NullReferenceException("Could not find a SessionFactory named " + name + " or identified by uuid " + uuid);
            }
            return result;
        }

if sesisonFactory is deserialized into a different app domain, then it checks if that appdomain has the factory with name, if deserialized into same appdomain, then it is taken from the objectSessionFactory.



In our case, this approach won't work.

What do you think I should do? Just make session factory fully serializable? Then what's the purpose of SessionFactoryObjectFactory? A custom serializer?

Tuna Toksöz
http://tunatoksoz.com
http://twitter.com/tehlike

Typos included to enhance the readers attention!

[Tuna Toksoz]

Or have a Real reference Object?

Tuna Toksöz
http://tunatoksoz.com
http://twitter.com/tehlike

Typos included to enhance the readers attention!

[Julian]

The idea of serializing a factory object (SessionFactory) grates with
me. Instinctively it doesn't seem right, but also it will require a
public, no-arg constructor and making its public properties read/
writeable.

Another idea for you to consider: If the compiled mapping data could
be represented by a serializable object it could be passed to the
Configuration object and used to start the factory:

CompiledMaps mappings = <<get from database/filesystem/wherever>>
var config = new Configuration();
config.AddPrecompiledMaps(mappings);
var factory = config.BuildSessionFactory();

In other words, the SessionFactory can be started with uncompiled
mapping files (in which case the factory needs to compile them and the
performance penalty is incurred), or with precompiled mapping data. To
obtain the compiled mappings:

CompiledMaps mappings = factory.CompiledMaps;

It may not make sense to represent the compiled mapping data as an
object if it has no actual behaviours...in which case, the compiled
maps could just be exposed as XML.

On Feb 6, 10:32 pm, Fabio Maulo <fabioma...@gmail.com> wrote:
> 2009/2/6 Tuna Toksoz <tehl...@gmail.com>

>
> > My proposal would be to have a STATIC SessionFactory per application, which
> > is initialized like the following

> > *

> > 1. Check if there is a SessionFactory Serialization Binary
> > 2. If yes, then check .config file and related assembly Last Modified Date,
> > if those are newer than the Serialized factory, then go to step 4. If not
> > step 3

> > **3. **Now we have a valid session factory, we can safely deserialize it.

> > 4. Means that we either don't have a session factory, or it is pretty old.

> > Create the session factory once more, serialize it to the disk.*

[Tuna Toksoz]

BinarySerialization doesn't need default ctor or public properties IIRC.

What you say is make absolute sense and it is like serializing the Configuration, which I tried and was successful, however, there is some validation going on during factory initialization which we want to get rid of.

 
Tuna Toksöz
http://tunatoksoz.com
http://twitter.com/tehlike

Typos included to enhance the readers attention!

[Tuna Toksoz]

anybody?

Tuna Toksöz
http://tunatoksoz.com
http://twitter.com/tehlike

Typos included to enhance the readers attention!

[Fabio Maulo]

2009/2/10 Tuna Toksoz <teh...@gmail.com>

anybody?

What happens ? 

--
Fabio Maulo

[Tuna Toksoz]

http://groups.google.com/group/nhibernate-development/msg/d201071ffca5a09d

Tuna Toksöz
http://tunatoksoz.com
http://twitter.com/tehlike

Typos included to enhance the readers attention!

[Fabio Maulo]

Well step-by-step...

The first part of the session-factory should be serializable.

By part I mean all metadata and not the cache and queryplan stuff

2009/2/10 Tuna Toksoz <teh...@gmail.com>

--
Fabio Maulo