Why Rebuild SessionFactory for Every Mapper Change?

[GANESHKUMAR M]

Hello Everyone,

I hope you're all doing well.

I'm currently working with NHibernate and have been reviewing some of its code on GitHub. I noticed that every time we add or modify mappers in the configuration, it seems necessary to rebuild the SessionFactory.

Why can't we simply update the newly added mappers in the existing SessionFactory? Is it mandatory to rebuild the SessionFactory each time? What is the rationale behind maintaining the SessionFactory as immutable? If there are any vulnerabilities or risks with this approach, could you provide example scenarios?

[Frédéric Delaporte]

Hi,

A session factory is thread safe once built. Thread safety is hard to get right with mutable objects. And thread safe mutability incurs using locks, which hurts performances.

[Paulo Quicoli]

Hi! 

Do your mappings change at run time? To me that's the question, why do you need to change your database class mappings at run time? 

If we take NHibernate out of the scene it's like your database is changing at runtime, is that true? 

Sorry but I'd like to understand your reason for something that's not common to see (at least to me) in software design... 

Regards 

---

From: nhu...@googlegroups.com <nhu...@googlegroups.com> on behalf of GANESHKUMAR M <mareegan...@gmail.com>
Sent: Thursday, August 22, 2024 6:20:38 AM
To: nhusers <nhu...@googlegroups.com>
Subject: [nhusers] Why Rebuild SessionFactory for Every Mapper Change?

 

--
You received this message because you are subscribed to the Google Groups "nhusers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nhusers+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nhusers/0a516299-fea1-45c8-ab4a-31a81aa8d94cn%40googlegroups.com.

[GANESHKUMAR M]

Hello,

Thank you so much for the answer. You explained the reason behind implementing an immutable SessionFactory—primarily due to thread safety and performance concerns. Making the SessionFactory immutable was a key decision.

Let me explain my current scenario.

I’m facing significant challenges with NHibernate in my .NET application, particularly around dynamically creating tables and altering schemas at runtime. My application needs to create millions of tables concurrently and occasionally modify their schemas. Currently, each table creation or schema alteration involves creating new Configuration and SessionFactory instances, leading to severe memory overhead and performance bottlenecks.

I understand that SessionFactory is immutable, but in my scenario, adding newly persistent entities to the existing SessionFactory is preferable to building a new one. Despite my efforts, rebuilding the SessionFactory is costly (e.g., over 6 GB for 50,000 mappers and 12.27 GB for 100,000 mappers).

Based on this, we need a mutable SessionFactory. I am actually trying to alter the existing NHibernate code to add newly added PersistentClass instances and CollectionModels as well. It is working well, but we are not entirely sure about the implications of this alteration, given the fundamental reason for maintaining an immutable SessionFactory—specifically the performance aspect. I need to explain that while we may reduce memory complexity, we could potentially introduce other issues.

Could you please verify the code changes I have attached below and also explain any cases where this implementation might break or throw exceptions? Locking the SessionFactory for a few milliseconds is crucial for managing our application to run with low memory usage. Thank you.

[Frédéric Delaporte]

> I am actually trying to alter the existing NHibernate code to add newly added PersistentClass instances and CollectionModels as well.

You take the risk of freezing the NHibernate version you are using due to the complexity of the conflicts you may have to resolve for updating your modified factory implementation to the latest NHibernate version. Take the current latest version of the source code and give it a try, to see what I mean.

Your solution is a bold endeavor in my opinion. You will be mostly on your own, I fear.

Another solution could be to re-architecture your application to use many small independent session factories, instead of having a single huge one. Are all these entities all inter-related through associations? Or could they be spill in many independent schema? With the later, independent smaller session factories should work well. And a schema change would need to rebuild only one small session factories instead of the huge single one. It should solve your current troubles.

> Could you please verify the code changes I have attached below and also explain any cases where this implementation might break or throw exceptions

Doing this thoroughly would require much more time than what I am willing to spend. I already do not understand why have you removed most of the readonly modifiers, while seemingly not assigning anything new to most of the members you have changed. (Just in case you would not know this: the readonly modifier does not cause instances assigned to readonly members to be immutable. It only causes the members to be assignable only at class initialization, and no more at a later time.)

Your updating code also looks like:

 - Running operations, probably disabled by your config, that should not be supported for an incremental update, like creating the whole schema.

 - Running initialization code on the whole persisters list. Is that the right choice, or should it be run only on the newly added ones? Or something else? Analyzing what does these initializations is required to know what is needed. I do not want to dive into this potential rabbit hole.

 - Recreating the second level query cache entirely (thus needing to remove the readonly modifiers for these members). It looks unnecessary to me. Again, I have not checked thoroughly. Moreover the code forgets to dispose the old one prior to re-assigning a new one. (See Close method.)

 - Other small "reintializations" seem likewise unneeded.

[GANESHKUMAR M]

Hi,

Thank you for sharing your insights. I now better understand the scenario. I appreciate your information related to my queries. I agree that my implementation could potentially cause problems within other NHibernate internals at runtime, which is why I mentioned that I'm not entirely sure about its broader implications. My focus was primarily on developing this implementation for a specific case. However, I understand that NHibernate isn't limited to just these scenarios—it has a wealth of features and internal APIs. Therefore, I wouldn't recommend using the exact code implementation for broader use cases without thorough validation and testing.

In real-world scenarios, we're often not limited to static tables and views; sometimes, database schemas need to change based on user requirements. This creates challenges that extend beyond the existing NHibernate implementation. Adding features like dynamic schema alterations to an ORM as robust as NHibernate could significantly impact the development process.

Regarding your point about Dispose() and Close() methods in NHibernate, I understand that they clear the cache allocated for specific persistent and collection models. I acknowledge that I overlooked handling this in my method.

As for the reason behind removing the readonly modifiers, it was to allow altering existing mapper properties and adding the altered mapper into the SessionFactory. Since both would have the same EntityName, I needed to replace the existing key-value pair (instead of removing it) when altering the schema.

I also appreciate your suggestion of maintaining multiple small SessionFactory instances. While some of our tables are related, I plan to explore splitting them into multiple SessionFactory instances based on your advice. I'll analyze the memory footprints and associated challenges in my application accordingly.

I realize that I'm not fully familiar with some of NHibernate's internals, which is why I'm seeking verification on the exact purpose of immutability and whether there's a way to test it using your test suite.

My implementation idea is straightforward: I disabled the entire second-level cache in the configuration to avoid certain issues in our application, so preparing the second-level cache isn't necessary for us. We build the SessionFactory with our application's POCO objects and XML mappers, and at that point, the initialization is complete, so it doesn't affect the flow. After that, I add or alter tables and handle it according to your documentation on dynamic persistent models (by creating an XML mapper string at runtime using System.Xml.Linq). This works well. Then, I prepare the configuration, call the Dispose method for the old SessionFactory, and build a new one. My concern is that re-preparing all caches, HQL, etc., for all my entities again seems unnecessary. I'm only adding a single mapper, but the iteration goes to n+1, which increases both time and memory consumption. That's why I'm looking for a solution from within NHibernate.

Every development needs to be consistent and tolerant. I believe this could be an implementable feature for the existing NHibernate code and could be considered for future updates, especially concerning dynamic persistent models.

Lastly, I have another question related to your documentation and code:

In the NHibernate documentation (Chapter 4.5. Tuplizers), you mention that it's possible to extends NHibernate.Tuple.Entity.DynamicMapEntityTuplizer and create a custom tuplizer. However, in the code, this tuplizer has an internal constructor. If I'm using NHibernate as a NuGet package, how can I implement this custom tuplizer? Could you please verify this and consider changing the constructor to public?

[Frédéric Delaporte]

> In the NHibernate documentation (Chapter 4.5. Tuplizers), you mention that it's possible to extends NHibernate.Tuple.Entity.DynamicMapEntityTuplizer and create a custom tuplizer. However, in the code, this tuplizer has an internal constructor. If I'm using NHibernate as a NuGet package, how can I implement this custom tuplizer? Could you please verify this and consider changing the constructor to public?

Indeed, it is internal since 2007, and that was overlooked when this example was introduced in the documentation in 2009. Since the PocoEntityTuplizer has its constructor public, it would also be more consistent. May you open an issue about this at https://github.com/nhibernate/nhibernate-core/issues, since you are the one having seen this?

> I believe this could be an implementable feature for the existing NHibernate code and could be considered for future updates, especially concerning dynamic persistent models.

Dynamic persistent models does not look to me as an usual use case. It also seems to me that most of the applications having this need are likely doing well enough by rebuilding the affected session factories, since I do not remember having already seen that request during the few years I have contributed to NHibernate. Introducing locking mechanisms in a class which currently works well without, and is a central piece of NHibernate, is likely to be seen as a "no-go". You may still open a feature request at https://github.com/nhibernate/nhibernate-core/issues to get feedback about this idea from other contributors, and see if they would see it better than me.

[GANESHKUMAR M]

  Okay, thanks for your feedback and information about NHibernate. I will definitely raise all the queries and problems related to NHibernate as we discussed before.