PHP MongoDB Driver High Connection Count

[J...@poweriep.com]

Hello all,

We just implemented MongoDB via MongoDB's Atlas product and are seeing a high number of connections with a relatively low number of users in our app. For about 400 users in our app, we're seeing around 180 connections. Compared to MySQL, where we see about 5 connections, this seems really high.

I did some research and saw a thread on GitHub (https://github.com/mongodb/mongo-php-driver/issues/393) talking about a similar issue. I understand that the driver does not support closing the connection. I also read that the connection is hashed based on a number of parameters, including the process ID.

So, all that being said...should we be concerned about hitting the max connection limit of 2000? We're at about 10% traffic right now (we serve school districts). We're running PHP via Apache (so not php-fpm as mentioned in the GitHub article). Or, should the connections be reused by Apache workers the die off when the worker is no longer valid?

Any insight would be great.

Thanks!

[Jeremy Mikola]

On Wed, Jul 12, 2017 at 3:04 PM, <J...@poweriep.com> wrote:

Hello all,

We just implemented MongoDB via MongoDB's Atlas product and are seeing a high number of connections with a relatively low number of users in our app. For about 400 users in our app, we're seeing around 180 connections. Compared to MySQL, where we see about 5 connections, this seems really high.

I did some research and saw a thread on GitHub (https://github.com/mongodb/mongo-php-driver/issues/393) talking about a similar issue. I understand that the driver does not support closing the connection. I also read that the connection is hashed based on a number of parameters, including the process ID.

Connection and topology persistence from the PHP.net documentation is the authority on this. The driver uses libmongoc for handling connections/topologies, and we persist those libmongoc client objects based on the arguments to MongoDB\Driver\Manager's constructor. The process ID is only included in case a PHP process is forked, so that we can avoid re-using a parent process' persisted clients in a child process (this is more common in CLI scripts than a web environment).

If you're no forking, I would just expect to see each PHP worker process maintain one connection to each mongod. Based on Atlas' documentation, their connection limits appear to be the value used for net.maxIncomingConnections in the server configuration, so I expect that each mongod will allow 2000 connections.

I'm not sure how application users translates to Apache workers in your application, but from what you said above I assume you have at most 180 PHP processes (i.e. Apache workers if PHP is being run as an Apache module) at play. In order to avoid hitting the 2000 connection limit, you'd want to ensure that Apache doesn't scale its worker pool beyond that number (again, assuming each process will open at most one connection to each mongod in the replica set).

If you end up deploying updated application code, where the MongoDB\Driver\Manager constructor arguments may have changed, it would be prudent to also restart Apache. Otherwise, you will still have lingering connections from previous requests served by the Apache workers, which will compete with new connections needed for the new MongoDB\Driver\Manager objects you'll be constructing.

 

So, all that being said...should we be concerned about hitting the max connection limit of 2000? We're at about 10% traffic right now (we serve school districts). We're running PHP via Apache (so not php-fpm as mentioned in the GitHub article). Or, should the connections be reused by Apache workers the die off when the worker is no longer valid?

Once the Apache worker dies, any previous connections will no longer be used. Assuming Apache cleanly shuts down PHP and its extensions, we will destroy the libmongoc clients while freeing the internal hash table that was used to persist them across multiple requests.

Any insight would be great.

Thanks!

--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
 
For other MongoDB technical support options, see: https://docs.mongodb.com/manual/support/
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+unsubscribe@googlegroups.com.
To post to this group, send email to mongod...@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/985b8d42-2e50-4dad-a994-ec3fce7aa8ad%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[J...@poweriep.com]

I wanted to follow-up on this issue for anyone searching in the future.

First off, Jeremy, thanks for the info.

Second, as school is getting closer, we are seeing more users in the app - about 1,000 concurrent right now. The connections have moved up to around 200 on a regular basis, which, based on the info from Jeremy, makes complete sense. Ultimately, what it comes down to is that, at least in our app, active users does cause connections to scale, but not at all in a linear fashion. We should have no problem staying well below the limit of 2,000 connections during our busy season.

Thanks again,

Joe