Sharing sockets for connection strings

[Jon]

Hi,

We use dozens of databases on each of our MongoDB clusters and we've gotten to a point with both the number of databases and number of processes which are connecting to those instances that we're hitting MongoDB connection limits and it's causing a lot of problems with our design.

The way our code works is that we dynamically generate Mongoid sessions for each database, but each Mongoid session is creating its own connection to the host:port.

Does anyone have any immediate thoughts around the possibility/issues/problems of sharing the socket on a per-thread basis? I made an commit at https://github.com/jonhyman/moped/commit/aafec12b6b1eb067dceaa6a19cc49ac2408bc308 which does this, and basic testing with lsof and dynamic sessions (in MRI Ruby 2.1.5) does show that it now only makes one connection to each host:port pair instead of N, where N is the number of sessions that point at host:port that each have a different database. It also passes all the tests.

I'm going to continue doing some testing, but if it goes well, do you think you would accept a pull request into Moped for this?

Thanks,

Jon

[Daniel Doubrovkine]

I think a cleaner design would be for the connections to be pooled and reused as long as it has been properly released back to the pool rather than to make such an assumption. I don't think you can safely assume that a connection will be used on the same thread, but it may very well be the case "right now".

--

---
You received this message because you are subscribed to the Google Groups "Mongoid" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongoid+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

dB. | Moscow - Geneva - Seattle - New York
code.dblock.org - @dblockdotorg - artsy.net - github/dblock

[Jonathan Hyman]

I definitely agree with that, I'd much prefer a connection pool as well. I should caveat saying that my commit is for Moped 1.5.x, not Moped 2.0, where connection pools are actually supported. 

For 2.0, I think you'd want to put the pool to a given host:port on Thread.current instead of the socket.

--

---
You received this message because you are subscribed to a topic in the Google Groups "Mongoid" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mongoid/txEtsgB7im4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mongoid+u...@googlegroups.com.