Mongo connection doesn't close despite the close() using java driver 2.0

[YoannCh]

Hi there,

I'm currently using mongo 1.4.3 for my project, with the mongo 2.0
java driver in my app and have a question.

I recently noticed that, despite the close I was doing on my
connection at the end of a request, the number of threads used by the
mongod server still increased at every call(new connection) as my
client application was alive, which results by an increase of memory
used by the mongod processus serverside.

My application is a web app, made using Spring 3.0.2, deployed on a TC
server.

Is it normal that the close() call does nothing serverside and that
the number of threads still grow up at each new connection? (this may
result in a "memory leak" for the server).
Did I forget something?

My app is supposed to handle a big ammount of requests per minute, and
I noticed this problem doing a stress test on a method (about 20 000
calls in 10 minutes).

A sample of code :

[code]
Mongo mong = new Mongo();
DB db = mong.getDB("mypersodb");
ObjectId idAccount = new ObjectId();

BasicDBObject query = new BasicDBObject();
query.put("NumberOfEmployees", new BasicDBObject("$gt", 10));
query.put("sic", 452);
BasicDBObject keys = new BasicDBObject();
keys.put("Name", 1);

DBCursor cur = db.getCollection("account").find(query, keys);
while (cur.hasNext()) {
BasicDBObject obj = (BasicDBObject) cur.next();
idAccount = (ObjectId) obj.get("_id");
}

BasicDBObject query2 = new BasicDBObject();
query2.put("Account", idAccount);
BasicDBObject keys2 = new BasicDBObject();
keys2.put("Name", 1);

DBCursor cur2 = db.getCollection("contact").find(query2, keys2);
while (cur2.hasNext()) {
BasicDBObject obj2 = (BasicDBObject) cur2.next();
}

mong.close();
[/code]

I removed some logic that was useless for this example.

[Eliot Horowitz]

2 things.

1) I can't reproduce this. How are you measuring threads on the server?

2) You should have Mongo instance per jvm. It has internal connection
pooling, so you don't need to create and disconnect on every request.

-Eliot

> --
> You received this message because you are subscribed to the Google Groups "mongodb-user" group.
> To post to this group, send email to mongod...@googlegroups.com.
> To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
>
>

[Juan Ignacio Borda]

Hi there i found out that findOne make php crash if in keys array some key is not a string.

mongo

MongoDB Support	enabled
Version	1.0.7

Directive	Local Value	Master Value
mongo.allow_persistent	On	On
mongo.auto_reconnect	On	On
mongo.chunk_size	262144	262144
mongo.cmd	$	$
mongo.default_host	localhost	localhost
mongo.default_port	27017	27017
mongo.utf8	1	1

PHP Version 5.2.10-2ubuntu6.4

System	Linux juanb-desktop 2.6.31-21-generic #59-Ubuntu SMP Wed Mar 24 07:28:56 UTC 2010 i686
Build Date	Jan 6 2010 22:15:47
Server API	Apache 2.0 Handler
Virtual Directory Support	disabled
Configuration File (php.ini) Path	/etc/php5/apache2
Loaded Configuration File	/etc/php5/apache2/php.ini
Scan this dir for additional .ini files	/etc/php5/apache2/conf.d
additional .ini files parsed	/etc/php5/apache2/conf.d/adodb.ini, /etc/php5/apache2/conf.d/curl.ini, /etc/php5/apache2/conf.d/gd.ini, /etc/php5/apache2/conf.d/mongo.ini, /etc/php5/apache2/conf.d/mysql.ini, /etc/php5/apache2/conf.d/mysqli.ini, /etc/php5/apache2/conf.d/pdo.ini, /etc/php5/apache2/conf.d/pdo_mysql.ini, /etc/php5/apache2/conf.d/xdebug.ini, /etc/php5/apache2/conf.d/xmlrpc.ini
PHP API	20041225
PHP Extension	20060613
Zend Extension	220060519
Debug Build	no
Thread Safety	disabled
Zend Memory Manager	enabled
IPv6 Support	enabled
Registered PHP Streams	https, ftps, compress.zlib, compress.bzip2, php, file, data, http, ftp, zip
Registered Stream Socket Transports	tcp, udp, unix, udg, ssl, sslv3, sslv2, tls
Registered Stream Filters	zlib.*, bzip2.*, convert.iconv.*, string.rot13, string.toupper, string.tolower, string.strip_tags, convert.*, consumed

This server is protected with the Suhosin Patch 0.9.7 Copyright (c) 2006 Hardened-PHP Project

[Kristina Chodorow]

Can you send an example?

[YoannCh]

Thanks for the answer Eliot.

1) I am running all this stuff on a windows 7 x64 that I use as a
server. With the resource monitor of the system, I can see how many
threads are used by the mongod process. While running my app, I can
see the connections created in the command term that opens when I
start the server, but don't see the destruction of them, which seems
to agree with the fact that the threads are still alive. When I kill
the mongod term, then I can see all the threads being killed in the
term. Is this behavior normal?

2) I was ignorant to that, thanks for the advice. Then I will expose
my Mongo instance as a singleton Spring bean.

Yoann.

[Eliot Horowitz]

There is (was) an issue
Mongo.close() cleared the references to the sockets, but they weren't
closed until gc happened.

Just fixed http://jira.mongodb.org/browse/JAVA-124

[Juan Ignacio Borda]

Of course yes! anything to help!
document:
{
    "_id" : ObjectId("4c23625e8ead0ee5734b3301"),
    "id" : 2961062598,
    "status" : "borrada",
    "owner" : 526627235,
    "group" : 5,
    "4836" : [
        "1"
    ],
    "4842" : "prueba135 mod 2",
    "4843" : "2009-05-13 10:44:39",
    "4844" : [
        828284489
    ],
    "4845" : [
        1426616061
    ],
    "4846" : [
        2406118494
    ],
    "4847" : [
        "3"
    ],
    "4848" : [
        "2"
    ],
    "4850" : "cvb",
    "4851" : "vbv",
    "4874" : [
        727058681
    ],
    "4886" : "vcx",
    "4887" : "5",
    "4905" : "1",
    "4906" : "1",
    "4970" : [
        "00"
    ],
    "5069" : "2009-04-20",
    "5372" : [
        "2010"
    ]
}

php code:

bla blablaaa....
$query=array();
$fields=array('id',4836);
$db->mycollection->findOne($query, $fields);

it doesn't crash using find() or converting int to strings (maybe a simple cast will do the trick)

[Kristina Chodorow]

Ah, I see.  You need to do:

$fields=array('id','4836'); //'4836' as a string, not an int

...to retrieve that field.  You should never pass an int as part of the fields array (notice that all of your keys are strings).  However, it shouldn't crash if you do, so I'll fix that.

[Kristina Chodorow]

Okay, fixed at http://github.com/mongodb/mongo-php-driver.  It'll throw an exception if you try to pass it an int field, now.

[Juan Ignacio Borda]

Thaks a lot!
(i double checked up my code to never pass an int anyway)