Google Groups

Re: Many Exceptions + OOM/Heap


project2501 May 7, 2012 10:17 AM
Posted in group: mongodb-user
Thanks Scott. Seems to be working now. I've seen the light!
Also, I need to RTFM more....

On May 7, 11:33 am, Scott Hernandez <scotthernan...@gmail.com> wrote:
> You can use the Mongo.Holder instance which is a "static" singleton.
>
> Mongo yourMongo = Mongo.Holder.instance().connect(...);http://api.mongodb.org/java/current/com/mongodb/Mongo.Holder.html#con...)
>
>
>
>
>
>
>
> On Mon, May 7, 2012 at 8:09 AM, project2501 <darreng5...@gmail.com> wrote:
> > Scott,
> >   Thanks for the help. Appreciate it. The logs do indeed indicate too
> > many connections.
>
> > I changed the use of Mongo mongo = new Mongo(), to become a
> > <jsp:useBean ... with scope set to "application" for all my JSP's.
>
> > Watching mongostat it seems a new connection is still made every time
> > the application is visited or refreshed.
>
> > Is there some other code change I can do to fix this? Should there
> > only be one connection for all JSP's,sessions,etc? I'm new to mongo,
> > so not sure how the connection management is to be handled.
>
> > On May 7, 9:49 am, Scott Hernandez <scotthernan...@gmail.com> wrote:
> >> You need to start it before, if you have connection problems. Please post
> >> the logs.
> >> On May 7, 2012 9:43 AM, "project2501" <darreng5...@gmail.com> wrote:
>
> >> > When I try to run mongostat on the jammed server it says:
>
> >> > DBClientCursor::init call() failed
> >> > DBClientBase::findN: transport error: .....
>
> >> > mongod is running still.
>
> >> > On May 7, 7:34 am, project2501 <darreng5...@gmail.com> wrote:
> >> > > Gotcha. Will report back. Thanks!
>
> >> > > On May 7, 12:19 am, Scott Hernandez <scotthernan...@gmail.com> wrote:
>
> >> > > > I don't think the server is crashing at all. I suspect you are simply
> >> > > > just using all the connections available.
>
> >> > > > You are probably seeing a combination of the objects staying in scope
> >> > > > too long (holding connections open) for the few number of connections
> >> > > > which the server support based on your system.
>
> >> > > > It is best not to guess and for you to just provide the logs and
> >> > > > stats. The answers will be there.
>
> >> > > > On Sun, May 6, 2012 at 8:50 PM, project2501 <darreng5...@gmail.com>
> >> > wrote:
> >> > > > > I will make those adjustments, but even still, re-creating a Mongo()
> >> > > > > object should not have the ill side effect of crashing the server
> >> > > > > after a few thousand times.
> >> > > > > When those objects go out of scope, the driver should not leak
> >> > > > > resources. It should act gracefully.
>
> >> > > > > I will try the mongostat and report back. Thanks for the tip!
> >> > > > > Hopefully, I find the solution because its a serious production
> >> > system
> >> > > > > experiencing this error.
>
> >> > > > > On May 6, 12:47 pm, Scott Hernandez <scotthernan...@gmail.com>
> >> > wrote:
> >> > > > >> I'd guess it is because you should not create a new Mongo instance
> >> > on
> >> > > > >> each page. You should treat the Mongo class as a Singleton and
> >> > > > >> basically only use one for your (whole) application.
>
> >> > > > >> Have you looked at the server logs, or run mongostat as this
> >> > happens?
>
> >> > > > >> On Sun, May 6, 2012 at 9:37 AM, project2501 <darreng5...@gmail.com>
> >> > wrote:
> >> > > > >> > I will post some code, but its only 1 jsp that is doing this.
> >> > There is
> >> > > > >> > no heavy logic or state being maintained. It seems the mongo
> >> > driver
> >> > > > >> > originates the exception well before the OOM. Also, as I
> >> > mentioned, I
> >> > > > >> > can no longer reach mongo when this happens, even from mongo
> >> > shell.
> >> > > > >> > Mongo dies.
>
> >> > > > >> > Here is my JSP pseudo-code, simply.
>
> >> > > > >> > Mongo mongo = new Mongo();
> >> > > > >> > DB db = mongo.getDB('testdb');
> >> > > > >> > DBCollection items = db.getCollection('items');
>
> >> > > > >> > for i = 0; i< items.length; i++ {
>
> >> > > > >> > BasicDBObject dbobject = new BasicDBObject();
> >> > > > >> > dbobj.put("key","value");
> >> > > > >> > ....
> >> > > > >> > ....
>
> >> > > > >> > items.insert(dbobj);
>
> >> > > > >> > }
>
> >> > > > >> > Ack, perhaps I need a close or commit or something?
>
> >> > > > >> > On May 6, 11:31 am, Scott Hernandez <scotthernan...@gmail.com>
> >> > wrote:
> >> > > > >> >> It seems like this is completely a client-side (java) issue. Can
> >> > you
> >> > > > >> >> please use jconsole/jvisualvm or a profiler to see where the
> >> > memory is
> >> > > > >> >> going. The odds are very good this has nothing to do with mongo
> >> > per se
> >> > > > >> >> and is just related to your java code.
>
> >> >http://java.sun.com/developer/technicalArticles/J2SE/jconsole.htmlhtt...
>
> >> > > > >> >> If you have a sample which causes the problem then post that
> >> > could
> >> > > > >> >> would be a good idea.
>
> >> > > > >> >> On Sun, May 6, 2012 at 8:11 AM, project2501 <
> >> > darreng5...@gmail.com> wrote:
> >> > > > >> >> > Hi,
> >> > > > >> >> >  I'm using mongo 2.0.0 and java driver 2.6.5
>
> >> > > > >> >> > I do periodic insert of small documents of about 20-50 every
> >> > couple
> >> > > > >> >> > minutes. Before long, mongo throws exceptions and the
> >> > java-driver
> >> > > > >> >> > loses its heap and gives OOM (even though app server was
> >> > allocated 8GB
> >> > > > >> >> > ram and had plenty).
>
> >> > > > >> >> > Java driver complains many times:
>
> >> > > > >> >> > Connection reset,
>
> >> > > > >> >> > then
>
> >> > > > >> >> > Exception determining maxBSON size using0
>
> >> > > > >> >> > then
>
> >> > > > >> >> > OutOfMemory: Java heap space
>
> >> > > > >> >> > I boosted the heap 5 times and its now 8GB, yet this still
> >> > happens.
>
> >> > > > >> >> > mongod is still running, but can no longer connect to it from
> >> > shell or
> >> > > > >> >> > anything. I'm running on a NUMA and launch using numactl.
>
> >> > > > >> >> > Any idea why mongo isn't working properly here?
>
> >> > > > >> >> > thanks.
>
> >> > > > >> >> > --
> >> > > > >> >> > You received this message because you are subscribed to the
> >> > Google Groups "mongodb-user" group.
> >> > > > >> >> > To post to this group, send email to
> >> > mongod...@googlegroups.com.
> >> > > > >> >> > To unsubscribe from this group, send email to
> >> > mongodb-user...@googlegroups.com.
> >> > > > >> >> > For more options, visit this group athttp://
> >> > groups.google.com/group/mongodb-user?hl=en.
>
> >> > > > >> > --
> >> > > > >> > You received this message because you are subscribed to the
> >> > Google Groups "mongodb-user" group.
> >> > > > >> > To post to this group, send email to
> >> > mongod...@googlegroups.com.
> >> > > > >> > To unsubscribe from this group, send email to
> >> > mongodb-user...@googlegroups.com.
> >> > > > >> > For more options, visit this group athttp://
> >> > groups.google.com/group/mongodb-user?hl=en.
>
> >> > > > > --
> >> > > > > You received this message because you are subscribed to the Google
> >> > Groups "mongodb-user" group.
> >> > > > > To post to this group, send email to mongod...@googlegroups.com.
> >> > > > > To unsubscribe from this group, send email to
> >> > mongodb-user...@googlegroups.com.
> >> > > > > For more options, visit this group athttp://
> >> > groups.google.com/group/mongodb-user?hl=en.
>
> >> > --
> >> > You received this message because you are subscribed to the Google Groups
> >> > "mongodb-user" group.
> >> > To post to this group, send email to mongod...@googlegroups.com.
> >> > To unsubscribe from this group, send email to
> >> > mongodb-user...@googlegroups.com.
> >> > For more options, visit this group at
> >> >http://groups.google.com/group/mongodb-user?hl=en.
>
> > --
> > You received this message because you are subscribed to the Google Groups "mongodb-user" group.
> > To post to this group, send email to mongod...@googlegroups.com.
> > To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.
> > For more options, visit this group athttp://groups.google.com/group/mongodb-user?hl=en.