[mongodb-user] OutOfMemoryError using java-driver 1.4

[niedhui]

Hi, 

  I'm using java-driver-1.4  ,and recently our tomcat server become very slow due to some memory issue, every full gc will spent 7s +-,so I add -XX:+DisableExplicitGC .

and i found mongo-java-driver call System.gc() in SimplePool,  I didn't know will it be a problem when after i add XX:+DisableExplicitGC ...

today ,an error occured,

Exception in thread "pool-2-thread-1160" java.lang.OutOfMemoryError: Direct buffer memory

        at java.nio.Bits.reserveMemory(Bits.java:633)

        at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:95)

        at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:288)

        at sun.nio.ch.Util.getTemporaryDirectBuffer(Util.java:57)

        at sun.nio.ch.IOUtil.read(IOUtil.java:205)

        at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:236)

        at sun.nio.ch.SocketAdaptor$SocketInputStream.read(SocketAdaptor.java:176)

        at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:86)

        at com.mongodb.DBPort._read(DBPort.java:219)

        at com.mongodb.DBPort.go(DBPort.java:91)

        at com.mongodb.DBPort.call(DBPort.java:58)

        at com.mongodb.DBTCPConnector.call(DBTCPConnector.java:158)

        at com.mongodb.DBApiLayer$MyCollection.find(DBApiLayer.java:297)

        at com.mongodb.DBCollection.findOne(DBCollection.java:330)

and i googled , found ```The buffer memory for a direct ByteBuffer is allocated outside of the normal heap (so that the GC doesn't move it!!)```

so the purpose of the SimplePool is for this?

--
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To post to this group, send email to mongod...@googlegroups.com.
To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.

[Eliot Horowitz]

Not quite.

The reason for the System.gc() in simple pool is to see if anyone didn't correctly release a connection.

I think the root problem is you just don't have enough memory allocated.

Can you try setting -Xmx to something higher?

[niedhui]

our server config is like 

-Xmx2048m -Xms2048m -Xmn768m -XX:PermSize=80m 

and i googled ,likely the Direct buffer memory is non-heap memory,i'm not quite familiar with java-nio...

i use jstat,and see there is enough heap memory left , 

although the -Xmx is 2048,but the jvm will use almost 4G memory,i don't figure out why

i chang to the java-driver 1.3,and it likely be better than driver-1.4,still watching ...

[Eliot Horowitz]

There aren't any relevant changes between 1.3 and 1.4, so that
shouldn't make a difference.

[niedhui]

thanks , continue to check the issue :)

[tony tam]

You probably should do some GC tuning to avoid massive GCs which can
lock up the VM. Look here:

http://ch.sun.com/sunnews/events/2009/apr/adworkshop/pdf/5-1-Java-Performance.pdf

You can probably stand to do parallel GCs with something like this:

-Xms1024m -Xmx2048m -XX:NewSize=256m -XX:MaxNewSize=256m -XX:
+UseConcMarkSweepGC -XX:+UseParNewGC -XX:PermSize=128m -
XX:MaxPermSize=128m

And if you have a real leak in your app, you're going to have to fix
it. Try using jmap & jhat to see what object references you have
dangling-- I'm pretty confident that the mongo java driver isn't
leaking memory with normal fetching and saving operations.

Tony

On Apr 26, 8:10 pm, niedhui <nied...@gmail.com> wrote:
> thanks , continue to check the issue :)
>
> On Apr 27, 2010, at 9:46 AM, Eliot Horowitz wrote:
>
>
>
>
>
> > There aren't any relevant changes between 1.3 and 1.4, so that
> > shouldn't make a difference.
>

> > On Mon, Apr 26, 2010 at 9:40 PM, niedhui <nied...@gmail.com> wrote:
> >> our server config is like
> >> -Xmx2048m -Xms2048m -Xmn768m -XX:PermSize=80m
> >> and i googled ,likely the Direct buffer memory is non-heap memory,i'm not
> >> quite familiar with java-nio...
> >> i use jstat,and see there is enough heap memory left ,
> >> although the -Xmx is 2048,but the jvm will use almost 4G memory,i don't
> >> figure out why
> >> i chang to the java-driver 1.3,and it likely be better than driver-1.4,still
> >> watching ...
> >> On Apr 26, 2010, at 9:47 PM, Eliot Horowitz wrote:
>
> >> Not quite.
> >> The reason for the System.gc() in simple pool is to see if anyone didn't
> >> correctly release a connection.
> >> I think the root problem is you just don't have enough memory allocated.
> >> Can you try setting -Xmx to something higher?
>

> > For more options, visit this group athttp://groups.google.com/group/mongodb-user?hl=en.

>
> --
> You received this message because you are subscribed to the Google Groups "mongodb-user" group.
> To post to this group, send email to mongod...@googlegroups.com.
> To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.

> For more options, visit this group athttp://groups.google.com/group/mongodb-user?hl=en.