Google Groups

Re: [mongodb-user] Re: Many Exceptions + OOM/Heap


Scott Hernandez May 7, 2012 8:33 AM
Posted in group: mongodb-user
You can use the Mongo.Holder instance which is a "static" singleton.

Mongo yourMongo = Mongo.Holder.instance().connect(...);
http://api.mongodb.org/java/current/com/mongodb/Mongo.Holder.html#connect(com.mongodb.MongoURI)

On Mon, May 7, 2012 at 8:09 AM, project2501 <darre...@gmail.com> wrote:
> Scott,
>   Thanks for the help. Appreciate it. The logs do indeed indicate too
> many connections.
>
> I changed the use of Mongo mongo = new Mongo(), to become a
> <jsp:useBean ... with scope set to "application" for all my JSP's.
>
> Watching mongostat it seems a new connection is still made every time
> the application is visited or refreshed.
>
> Is there some other code change I can do to fix this? Should there
> only be one connection for all JSP's,sessions,etc? I'm new to mongo,
> so not sure how the connection management is to be handled.
>
> On May 7, 9:49 am, Scott Hernandez <scotthernan...@gmail.com> wrote:
>> You need to start it before, if you have connection problems. Please post
>> the logs.
>> On May 7, 2012 9:43 AM, "project2501" <darreng5...@gmail.com> wrote:
>>
>>
>>
>>
>>
>>
>>
>> > When I try to run mongostat on the jammed server it says:
>>
>> > DBClientCursor::init call() failed
>> > DBClientBase::findN: transport error: .....
>>
>> > mongod is running still.
>>
>> > On May 7, 7:34 am, project2501 <darreng5...@gmail.com> wrote:
>> > > Gotcha. Will report back. Thanks!
>>
>> > > On May 7, 12:19 am, Scott Hernandez <scotthernan...@gmail.com> wrote:
>>
>> > > > I don't think the server is crashing at all. I suspect you are simply
>> > > > just using all the connections available.
>>
>> > > > You are probably seeing a combination of the objects staying in scope
>> > > > too long (holding connections open) for the few number of connections
>> > > > which the server support based on your system.
>>
>> > > > It is best not to guess and for you to just provide the logs and
>> > > > stats. The answers will be there.
>>
>> > > > On Sun, May 6, 2012 at 8:50 PM, project2501 <darreng5...@gmail.com>
>> > wrote:
>> > > > > I will make those adjustments, but even still, re-creating a Mongo()
>> > > > > object should not have the ill side effect of crashing the server
>> > > > > after a few thousand times.
>> > > > > When those objects go out of scope, the driver should not leak
>> > > > > resources. It should act gracefully.
>>
>> > > > > I will try the mongostat and report back. Thanks for the tip!
>> > > > > Hopefully, I find the solution because its a serious production
>> > system
>> > > > > experiencing this error.
>>
>> > > > > On May 6, 12:47 pm, Scott Hernandez <scotthernan...@gmail.com>
>> > wrote:
>> > > > >> I'd guess it is because you should not create a new Mongo instance
>> > on
>> > > > >> each page. You should treat the Mongo class as a Singleton and
>> > > > >> basically only use one for your (whole) application.
>>
>> > > > >> Have you looked at the server logs, or run mongostat as this
>> > happens?
>>
>> > > > >> On Sun, May 6, 2012 at 9:37 AM, project2501 <darreng5...@gmail.com>
>> > wrote:
>> > > > >> > I will post some code, but its only 1 jsp that is doing this.
>> > There is
>> > > > >> > no heavy logic or state being maintained. It seems the mongo
>> > driver
>> > > > >> > originates the exception well before the OOM. Also, as I
>> > mentioned, I
>> > > > >> > can no longer reach mongo when this happens, even from mongo
>> > shell.
>> > > > >> > Mongo dies.
>>
>> > > > >> > Here is my JSP pseudo-code, simply.
>>
>> > > > >> > Mongo mongo = new Mongo();
>> > > > >> > DB db = mongo.getDB('testdb');
>> > > > >> > DBCollection items = db.getCollection('items');
>>
>> > > > >> > for i = 0; i< items.length; i++ {
>>
>> > > > >> > BasicDBObject dbobject = new BasicDBObject();
>> > > > >> > dbobj.put("key","value");
>> > > > >> > ....
>> > > > >> > ....
>>
>> > > > >> > items.insert(dbobj);
>>
>> > > > >> > }
>>
>> > > > >> > Ack, perhaps I need a close or commit or something?
>>
>> > > > >> > On May 6, 11:31 am, Scott Hernandez <scotthernan...@gmail.com>
>> > wrote:
>> > > > >> >> It seems like this is completely a client-side (java) issue. Can
>> > you
>> > > > >> >> please use jconsole/jvisualvm or a profiler to see where the
>> > memory is
>> > > > >> >> going. The odds are very good this has nothing to do with mongo
>> > per se
>> > > > >> >> and is just related to your java code.
>>
>> >http://java.sun.com/developer/technicalArticles/J2SE/jconsole.htmlhtt...
>>
>> > > > >> >> If you have a sample which causes the problem then post that
>> > could
>> > > > >> >> would be a good idea.
>>
>> > > > >> >> On Sun, May 6, 2012 at 8:11 AM, project2501 <
>> > darreng5...@gmail.com> wrote:
>> > > > >> >> > Hi,
>> > > > >> >> >  I'm using mongo 2.0.0 and java driver 2.6.5
>>
>> > > > >> >> > I do periodic insert of small documents of about 20-50 every
>> > couple
>> > > > >> >> > minutes. Before long, mongo throws exceptions and the
>> > java-driver
>> > > > >> >> > loses its heap and gives OOM (even though app server was
>> > allocated 8GB
>> > > > >> >> > ram and had plenty).
>>
>> > > > >> >> > Java driver complains many times:
>>
>> > > > >> >> > Connection reset,
>>
>> > > > >> >> > then
>>
>> > > > >> >> > Exception determining maxBSON size using0
>>
>> > > > >> >> > then
>>
>> > > > >> >> > OutOfMemory: Java heap space
>>
>> > > > >> >> > I boosted the heap 5 times and its now 8GB, yet this still
>> > happens.
>>
>> > > > >> >> > mongod is still running, but can no longer connect to it from
>> > shell or
>> > > > >> >> > anything. I'm running on a NUMA and launch using numactl.
>>
>> > > > >> >> > Any idea why mongo isn't working properly here?
>>
>> > > > >> >> > thanks.
>>
>> > > > >> >> > --
>> > > > >> >> > You received this message because you are subscribed to the
>> > Google Groups "mongodb-user" group.
>> > > > >> >> > To post to this group, send email to
>> > mongod...@googlegroups.com.
>> > > > >> >> > To unsubscribe from this group, send email to
>> > mongodb-user...@googlegroups.com.
>> > > > >> >> > For more options, visit this group athttp://
>> > groups.google.com/group/mongodb-user?hl=en.
>>
>> > > > >> > --
>> > > > >> > You received this message because you are subscribed to the
>> > Google Groups "mongodb-user" group.
>> > > > >> > To post to this group, send email to
>> > mongod...@googlegroups.com.
>> > > > >> > To unsubscribe from this group, send email to
>> > mongodb-user...@googlegroups.com.
>> > > > >> > For more options, visit this group athttp://
>> > groups.google.com/group/mongodb-user?hl=en.
>>
>> > > > > --
>> > > > > You received this message because you are subscribed to the Google
>> > Groups "mongodb-user" group.
>> > > > > To post to this group, send email to mongod...@googlegroups.com.
>> > > > > To unsubscribe from this group, send email to
>> > mongodb-user...@googlegroups.com.
>> > > > > For more options, visit this group athttp://
>> > groups.google.com/group/mongodb-user?hl=en.
>>
>> > --
>> > You received this message because you are subscribed to the Google Groups
>> > "mongodb-user" group.
>> > To post to this group, send email to mongod...@googlegroups.com.
>> > To unsubscribe from this group, send email to
>> > mongodb-user...@googlegroups.com.
>> > For more options, visit this group at
>> >http://groups.google.com/group/mongodb-user?hl=en.
>
> --
> You received this message because you are subscribed to the Google Groups "mongodb-user" group.
> To post to this group, send email to mongod...@googlegroups.com.
> To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
>