MongoError: server instance pool was destroyed

2,234 views
Skip to first unread message

S Friend

unread,
Mar 28, 2017, 1:16:34 PM3/28/17
to Mongoose Node.JS ODM
I'm seeing some behaviour I don't understand that seems related to mongoose/mongo connection pooling - 

My application asynchronously scrapes data from another server in chunks of 100, and then writes it to my mongo instance.  My connection pool size is 100.  When I run this on my server (and connect to mongo over localhost) I see MongoError: server instance pool was destroyed after about 2000 chunks.  When I run I change the connection pool size to be 200 - the error is thrown almost immediately.  At poolSize = 50, it has not thrown yet and I'm 100,000 chunks in.  

Why might changing my connection pool size cause this error?  

This is the code I'm using to connect:

initConnection(uri) {
    console.log(uri);
    // initiate connection with mongo
    mongoose.connect(uri,
       { server: { socketOptions: { keepAlive: 300000,    connectTimeoutMS: 30000 }, poolSize: 100 } })
    .then(
        // ready to use. The `mongoose.connect()` promise resolves to undefined.
        () => { this.log('Db is open!'); },
        // log any initial connection errors
        (err) => {
            console.log(err);
            this.errorLog(`Connection error ${err}`);
        },
    );
}


Using mongoose 4.8.1 and mongo 2.2.22.  

I'm seeing this in the mongodb driver changelog, but I don't know if this woud cause the behaviour I'm seeing

2.2.23 2017-02-13
Updated mongodb-core to 2.1.8.
NODE-925 ensure we reschedule operations while pool is < poolSize while pool is growing and there are no connections with not currently performing work.
Any advice much appreciated!

S Friend

unread,
Mar 28, 2017, 1:26:02 PM3/28/17
to Mongoose Node.JS ODM
My mongo logs are showing mostly this type of thing - the final things before the error are all connections ending

2017-03-28T17:08:05.341+0000 I NETWORK  [initandlisten] connection accepted from 172.17.0.1:34310 #1196 (45 connections now open)
2017-03-28T17:08:05.341+0000 I NETWORK  [initandlisten] connection accepted from 172.17.0.1:34312 #1197 (46 connections now open)
2017-03-28T17:08:05.341+0000 I NETWORK  [initandlisten] connection accepted from 172.17.0.1:34308 #1198 (47 connections now open)
2017-03-28T17:08:05.341+0000 I NETWORK  [initandlisten] connection accepted from 172.17.0.1:34314 #1199 (48 connections now open)
2017-03-28T17:08:05.341+0000 I NETWORK  [initandlisten] connection accepted from 172.17.0.1:34316 #1200 (49 connections now open)
2017-03-28T17:08:05.342+0000 I NETWORK  [initandlisten] connection accepted from 172.17.0.1:34318 #1201 (50 connections now open)
2017-03-28T17:08:05.342+0000 I NETWORK  [initandlisten] connection accepted from 172.17.0.1:34320 #1202 (51 connections now open)
2017-03-28T17:08:05.607+0000 I ACCESS   [conn1199] Successfully authenticated as principal admin on mydb
2017-03-28T17:08:06.302+0000 I ACCESS   [conn1200] Successfully authenticated as principal admin on mydb
2017-03-28T17:08:06.303+0000 I ACCESS   [conn1201] Successfully authenticated as principal admin on mydb
2017-03-28T17:08:06.303+0000 I ACCESS   [conn1196] Successfully authenticated as principal admin on mydb
2017-03-28T17:08:06.303+0000 I ACCESS   [conn1197] Successfully authenticated as principal admin on mydb
2017-03-28T17:08:06.303+0000 I ACCESS   [conn1198] Successfully authenticated as principal admin on mydb
2017-03-28T17:08:06.304+0000 I ACCESS   [conn1202] Successfully authenticated as principal admin on mydb
2017-03-28T17:08:06.304+0000 I ACCESS   [conn1195] Successfully authenticated as principal admin on mydb
2017-03-28T17:11:07.638+0000 I NETWORK  [conn1202] end connection 172.17.0.1:34320 (50 connections now open)
2017-03-28T17:11:07.640+0000 I NETWORK  [conn1198] end connection 172.17.0.1:34308 (49 connections now open)
2017-03-28T17:11:07.640+0000 I NETWORK  [conn1148] end connection 172.17.0.1:35140 (49 connections now open)
2017-03-28T17:11:07.640+0000 I NETWORK  [conn1195] end connection 172.17.0.1:34292 (48 connections now open)
2017-03-28T17:11:07.640+0000 I NETWORK  [conn1152] end connection 172.17.0.1:35164 (47 connections now open)
2017-03-28T17:11:07.640+0000 I NETWORK  [conn1146] end connection 172.17.0.1:35136 (45 connections now open)
2017-03-28T17:11:07.641+0000 I NETWORK  [conn1155] end connection 172.17.0.1:35170 (44 connections now open)
2017-03-28T17:11:07.642+0000 I NETWORK  [conn1153] end connection 172.17.0.1:35168 (43 connections now open)
2017-03-28T17:11:07.642+0000 I NETWORK  [conn1161] end connection 172.17.0.1:35190 (42 connections now open)
2017-03-28T17:11:07.643+0000 I NETWORK  [conn1170] end connection 172.17.0.1:35226 (41 connections now open)
Reply all
Reply to author
Forward
0 new messages