NHibernate eats memory - again (memory leak in unmanaged code...)

[Paul Allington]

Please excuse me for sending this again, but the last one descended into comments on my test code rather than the issue - which is that there's an unmanaged memory leak somewhere in nhibernate, and I'd like to find it as it's costing me a lot of memory

So far:

- a simple 1 page web app uses about 40-50MB more when nhibernate is introduce.  Around 30MB in unmanaged memory, and about 20-30 in "Unused memory allocated to .NET" which indicates memory fragmentation.

- Entity objects are being held in memory (second level caching is off)

    - this is not because of bad code or not closing sessions.  The example I've written couldn't be simpler so there's little to no chance of coder error.

    - A lot of the objects are "IUserType" classes (don't know if this is relevant)

    - A number of objects are only being held on to by "System.Object[]" - which is being held by something in unmanaged memory (this is what I can gather) meaning there's a memory leak in something unmanaged that nhibernate has introduced.

Someone must know what's going on surely?

Paul

[kor]

without the code is impossible, anyhow it's more probable that the bug
is in your code (for example iusertype).
try to investigate if you have same static variables.

anyhow try to split your code to find exactly what is allocating and
not releasing the memory and then submit code that reproduce it with a
jira ticket

[Jason Meckley]

Is it that you have a memory leak, or you think NH should just use less memory? try removing the IUserTypes and just map to primitive values. See what happens to the memory consumption then. If it drops, then you can start by investigating the IUserTypes. Kor's correct, without code or context (quantity of data) there isn't much we can do.

[Paul Allington]

Ok - have included some code here (please please don't comment on the code itself - I wrote this purely to test this problem, it's not supposed to be production ready code).

I've added a GC.Collect() to the end of the module so that anything in the memory snapshot is part of the problem.

I've only included relevant parts of the code (if you'd like the whole project, please email me)

Not using the CustomType isn't an option.

** After running this code, I profiled the memory.  In it was left an instance of UserConfig (the User instance has been GC'd) - as the user had gone, I would have expected an instance of UserConfig to have gone as well.  The "instance retention graph" showed:

UserConfig

 |

NHibernate.Type.CustomType

 |

NHibernate.Type.IType[]

 |

NHbernate.Persister.Entity.SingleTableEntityPersister

 |

System.Collections.....

 |

System.Collections.....

 |

NHibernate.Impl.SessionFactoryImpl

 |

SessionSource

Session factory...

                SessionFactory = Fluently.Configure()

                    .Database(MsSqlConfiguration.MsSql2008.ConnectionString(ConfigurationManager.ConnectionStrings["ConnectionString"].ToString()))

                    .Mappings(x => GetFluantMappings(x))

                    .ExposeConfiguration(c =>

                                             {

                                                 c.SetProperty("generate_statistics", "true");

                                                 c.SetProperty("current_session_context_class", contextClass);

                                                 c.SetProperty("cache.use_second_level_cache", "false");

                                                 c.SetProperty("cache.use_query_cache", "false");

                                                 

                                             })

                    .BuildSessionFactory();

            }

            catch(FluentConfigurationException ex)

            {

                throw new SessionSourceException("Error initializing session factory", ex);

            }

Session Per Request httpmodule

        private static void BeginRequest(object sender, EventArgs e)

        {

            ISession session = SessionSource.Instance.SessionFactory.OpenSession();

            ManagedWebSessionContext.Bind(HttpContext.Current, session);

        }

        private static void EndRequest(object sender, EventArgs e)

        {

            ISession session = ManagedWebSessionContext.Unbind(HttpContext.Current, SessionSource.Instance.SessionFactory);

            if (session == null) return;

            session.Close();

            session.Dispose();

            GC.Collect();

        }

The aspx Page

        protected void Page_Load(object sender, EventArgs e)

        {

            int userId = CreateUser(SessionSource.Instance.SessionFactory);

            GetUser(SessionSource.Instance.SessionFactory, userId);

        }

        private static void GetUser(ISessionFactory sessionFactory, int userId)

        {

            ISession session = sessionFactory.GetCurrentSession();

            User user = session.Get<User>(userId);

            //User user = session.Query<User>().Single(u => u.Id == userId);

        }

        private static int CreateUser(ISessionFactory sessionFactory)

        {

            User user = new User();

            user.Created = DateTime.Now;

            user.Email = "fi...@lastname.com";

            user.Enabled = true;

            user.FirstName = "first name";

            user.LastName = "last name";

            user.Password = "password";

            user.ScreenName = "firstname lastname";

            ISession session = sessionFactory.GetCurrentSession();

            session.SaveOrUpdate(user);

            return user.Id;

        }

The User class and User Map

    public class User

    {

        public virtual int Id { get; set; }

        public virtual string Email { get; set; }

        public virtual string ScreenName { get; set; }

        public virtual string Password { get; set; }

        public virtual string FirstName { get; set; }

        public virtual string LastName { get; set; }

        public virtual bool Enabled { get; set; }

        public virtual DateTime Created { get; set; }

        public virtual DateTime? LastLogin { get; set; }

        public virtual DateTime? LastActivity { get; set; }

        public virtual UserConfig Config { get; set; }

    }

    [Serializable]

    public class UserConfig : List<string>, IUserType

    {

        public UserConfig() { }

        public UserConfig(IEnumerable<string> items) : base(items)

        {

        }

        #region IUserType Members

        public object Assemble(object cached, object owner)

        {

            return DeepCopy(cached);

        }

        public object DeepCopy(object value)

        {

            return value;

        }

        public object Disassemble(object value)

        {

            return DeepCopy(value);

        }

        public new bool Equals(object x, object y)

        {

            if (ReferenceEquals(x, y)) return true;

            if (x == null || y == null) return false;

            return x.Equals(y);

        }

        public int GetHashCode(object x)

        {

            return x.GetHashCode();

        }

        public bool IsMutable

        {

            get { return false; }

        }

        public object NullSafeGet(IDataReader rs, string[] names, object owner)

        {

            object obj = NHibernateUtil.String.NullSafeGet(rs, names[0]);

            if (obj == null)

            {

                return new UserConfig();

            }

            return XmlSerialiser.FromXml<UserConfig>(obj as string);

        }

        public void NullSafeSet(IDbCommand cmd, object value, int index)

        {

            var parameter = (IDataParameter)cmd.Parameters[index];

            if (value == null)

            {

                parameter.Value = DBNull.Value;

            }

            else

            {

                parameter.Value = XmlSerialiser.ToXml<UserConfig>(value);

            }

        }

        public object Replace(object original, object target, object owner)

        {

            return original;

        }

        [XmlIgnore]

        public Type ReturnedType

        {

            //the .Net type that this maps to

            get { return typeof(UserConfig); }

        }

        [XmlIgnore]

        public SqlType[] SqlTypes

        {

            //the sql type that this maps to

            get { return new[] { SqlTypeFactory.GetString(int.MaxValue), }; }

        }

        #endregion

    }

    public class XmlSerialiser

    {

        public static XmlSerializerNamespaces GetNamespaces()

        {

            XmlSerializerNamespaces ns = new XmlSerializerNamespaces();

            ns.Add(string.Empty, string.Empty);

            return ns;

        }

        //Creates an object from an XML string.

        public static T FromXml<T>(string xml) where T : new()

        {

            return FromXml<T>(xml, () => new T());

        }

        public static T FromXml<T>(string xml, Func<T> func)

        {

            if (xml == "<nil-classes type=\"array\" />")

            {

                return func();

            }

            XmlSerializer ser = new XmlSerializer(typeof(T));

            StringReader stringReader = new StringReader(xml);

            XmlTextReader xmlReader = new XmlTextReader(stringReader);

            try

            {

                object obj = ser.Deserialize(xmlReader);

                xmlReader.Close();

                stringReader.Close();

                return (T) obj;

            }

            finally

            {

                stringReader.Dispose();

            }

        }

        public static string ToXml(Type type, object obj)

        {

            XmlSerializer ser = new XmlSerializer(type);

            MemoryStream memStream = new MemoryStream();

            XmlTextWriter xmlWriter = new XmlTextWriter(memStream, Encoding.Unicode);

            try

            {

                xmlWriter.Namespaces = true;

                ser.Serialize(xmlWriter, obj, GetNamespaces());

                xmlWriter.Close();

                memStream.Close();

                string xml = Encoding.Unicode.GetString(memStream.GetBuffer());

                xml = xml.Substring(xml.IndexOf(Convert.ToChar(60)));

                xml = xml.Substring(0, (xml.LastIndexOf(Convert.ToChar(62)) + 1));

                return xml;

            }

            finally

            {

                memStream.Dispose();

            }

        }

        //Serializes the <i>Obj</i> to an XML string.

        public static string ToXml<T>(object obj)

        {

            return ToXml(typeof(T), obj);

        }

    }

    public sealed class UserMap : ClassMap<User>

    {

        public UserMap()

        {

            Id(x => x.Id).Column("UserId");

            Map(x => x.Created);

            Map(x => x.Email);

            Map(x => x.Enabled);

            Map(x => x.FirstName);

            Map(x => x.LastActivity);

            Map(x => x.LastLogin);

            Map(x => x.LastName);

            Map(x => x.Password);

            Map(x => x.ScreenName);

            Map(x => x.Config).CustomType(typeof(UserConfig));

        }

    }

On Thu, Feb 10, 2011 at 3:17 PM, Jason Meckley <jasonm...@gmail.com> wrote:

Is it that you have a memory leak, or you think NH should just use less memory? try removing the IUserTypes and just map to primitive values. See what happens to the memory consumption then. If it drops, then you can start by investigating the IUserTypes. Kor's correct, without code or context (quantity of data) there isn't much we can do.

--
You received this message because you are subscribed to the Google Groups "nhusers" group.
To post to this group, send email to nhu...@googlegroups.com.
To unsubscribe from this group, send email to nhusers+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/nhusers?hl=en.

[John Davidson]

I would change your CreateUser call so that it is not static. This may be why the reference is being retained.

John Davidson

[Greg Young]

I would not expect this to be related.

On Thu, Feb 10, 2011 at 11:24 AM, John Davidson <jwdav...@gmail.com> wrote:
> I would change your CreateUser call so that it is not static. This may be
> why the reference is being retained.
> John Davidson
>
> On Thu, Feb 10, 2011 at 11:10 AM, Paul Allington
> <pa...@intelligentpenguin.co.uk> wrote:
>>
>> Ok - have included some code here (please please don't comment on the code
>> itself - I wrote this purely to test this problem, it's not supposed to be
>> production ready code).

>> Idn't 've added a GC.Collect() to the end of the module so that anything in the

--
Les erreurs de grammaire et de syntaxe ont été incluses pour m'assurer
de votre attention

[kor]

the isertype must not be used in your domain object, you use for the
property User.Config a List<string> (or a sub type if you need same
custom behaviour) and in an external class you implement the iuser
type.

then on the mapping file you associate your implementation of
iusertype with that property (ie with xml config
<property name="Config" type="YourIUSerType"/>)

i don't know if the problem is that but surely this is the good way to
use an user type.

ps. before callig GC.Collect set the "session" variable to null, i
don't remember if gc will also remove an object already referenced by
a variable that will not be used after the call to gc.collect)

[Paul Allington]

Afraid that's not it, thanks though

Paul

[Greg Young]

ISression session =
ManagedWebSessionContext.Unbind(HttpContext.Current,
SessionSource.Instance.SessionFactory);

if (session == null) return;

session.Close();
session.Dispose();
GC.Collect();

whether or not session is still held at the GC.Collect depends on
optimizations and debug/release mode. You can get in and view with SOS
where the JIT will hold session until. Either way making it set to
null first would remove a variable.

[Jason Meckley]

Kor may be on to something with your IUserType implementation. IUserTypes are not part of the domain, they are part of the infrastructure. a user type tells NH when give object X convert it to schema Y.

[Oskar Berggren]

Yes, surely the UserConfig instance you show in the retention graph is
the single instance created as a consequence of the CustomType() call
in your mapping. I'm not certain, but I would expect this single
instance to stay as long as the session factory remains - destroying
it and creating it again would mean less performant.

Map(x => x.Config).CustomType(typeof(UserConfig));

However, when you actually load the User, you would get another
UserConfig instance, this time as part of your model. This instance is
in fact garbage collected.

As Kor said, you are using the same UserConfig class for two
completely different purposes. What I usually have would be a
Domain.UserConfig class, and a corresponding
Persistence.UserConfigUserType class.

On the other hand... it does not seem obvious that this would create
any serious memory issues. Though your DeepCopy() is clearly not doing
a deep copy, but perhaps it looks better in the production code.

/Oskar

2011/2/10 Jason Meckley <jasonm...@gmail.com>:

> Kor may be on to something with your IUserType implementation. IUserTypes
> are not part of the domain, they are part of the infrastructure. a user type
> tells NH when give object X convert it to schema Y.
>

[Paul Allington]

Thanks to you all! That makes sense, I'll sort that issue - that makes sense to me (I wonder if IUserType should be called IUserTypeMapper, just to make it easier to understand :)

Doesn't seem to solve my big memory problem though.  So peculiar.  Although, with all this I've been doing in the last couple of days the profiler is now saying 5MB actually used, 60MB in total...which means it's probably not my app, it's now more the amount of memory that's been allocated.  Has anyone else had this problem? (is it even a problem?)

Paul

[Oskar Berggren]

60MB isn't that much for many cases - depends of course on the
intended deployment scenario. Though it would be irritating if it's
unnecessary of course.

A more important question is: is it growing over time? That could
indicate a more serious memory leak.

/Oskar

2011/2/10 Paul Allington <pa...@intelligentpenguin.co.uk>:

[Paul Allington]

If left for a while, it can grow to about 200MB - but still only aroun 5/6MB actually used...which is a little frustrating.

ANTS say it's a fragmentation issue

Paul Allington  T: 01799 522 665 M: 07973 145 754 E: pa...@intelligentpenguin.co.uk W: www.intelligentpenguin.co.uk W: www.creative-penguin.co.uk
For highly creative & technically brilliant websites, and on-line management systems ------------------------------------------------------------------------------

My profiles:
Contact me: phallington paul-allington paul_allington

[Gunnar Liljas]

Could you provide us with a ready to go project? I'd be happy to investigate.

/g

Sent from my iPad

--