I notice that you've set the max_retries_on_connection_failure. This could be causing you issues. Imagine you have a connection failure between an initial query and a getmore. The driver will reconnect, but it's not guaranteed that it will be routed to the same mongos. Do you whether these kinds of failures are happening?
Is there any way you can share your sidekiq code via email, perhaps a link to a private gist?
Another step would be to enable verbose logging on the mongos instances. That way, we can verify that the initial query and getmore are in fact being sent to different mongos nodes.
On Feb 28, 1:06 pm, fbjork <fredrik.bj...@gmail.com> wrote:
> Taking a step back for a while. I've removed connection pooling and only > talking to one mongos host (DNS resolution happens when the Ruby process > starts up and connects), but getMore is still not working. So either the > driver is messed up or mongos/mongod isn't working as expected. This > happens on small dataset too, which means it's not the data size.
> Could it be some incompatibility with Sidekiq > (https://github.com/mperham/sidekiq) and the Ruby mongo driver? SIdekiq > uses Celluloid for thread concurrency.
> On Tuesday, February 28, 2012 7:06:07 AM UTC-8, Kyle Banker wrote:
> > Probably because we weren't warning ;)
> > On Tuesday, February 28, 2012 10:03:08 AM UTC-5, fbjork wrote:
> >> Yes, the app starts, but I want to make sure Mongoid still works with the > >> driver from github. Not sure why mongoid would pass those values to the > >> driver?
> >> On Tuesday, February 28, 2012 6:59:21 AM UTC-8, Kyle Banker wrote:
> >>> Those are warnings, not errors. Are they preventing your application > >>> from starting? Basically, we're starting to warn when users pass invalid > >>> options to Connection.new.
> >>> On Monday, February 27, 2012 7:28:21 PM UTC-5, fbjork wrote:
> >>>> I tried using the latest commit from Github, but when I start rails I > >>>> get these errors:
> >>>> allow_dynamic_fields is not a valid option for Mongo::Connection > >>>> max_retries_on_connection_failure is not a valid option for > >>>> Mongo::Connection > >>>> use_activesupport_time_zone is not a valid option for > >>>> Mongo::Connection > >>>> identity_map_enabled is not a valid option for Mongo::Connection > >>>> host is not a valid option for Mongo::Connection > >>>> databases is not a valid option for Mongo::Connection > >>>> host is not a valid option for Mongo::Connection
> >>>> On Feb 27, 3:27 pm, Kyle Banker <kyleban...@gmail.com> wrote: > >>>> > Hi Fredrik,
> >>>> > We've posted the fixes that we believe will help. We'll release the > >>>> > gem tomorrow, but if you're eager to test now, you can build the gem > >>>> > and install yourself from this latest commit: > >>>>https://github.com/mongodb/mongo-ruby-driver/commit/349d37f3db86dfe5b...
> >>>> > I'd like you to re-enable the connection pooling and try again with > >>>> > the new release. If that doesn't solve the problem, we'll have to > >>>> look > >>>> > at your DDNS setup.
> >>>> > Kyle
> >>>> > On Feb 27, 5:02 pm, fbjork <fredrik.bj...@gmail.com> wrote:
> >>>> > > And I'm still getting the following error after removing connection > >>>> > > pooling:
> >>>> > > Query response returned CURSOR_NOT_FOUND. Either an invalid cursor > >>>> was > >>>> > > specified, or the cursor may have timed out on the server.
> >>>> > > On Feb 27, 1:48 pm, fbjork <fredrik.bj...@gmail.com> wrote:
> >>>> > > > I removed connection pooling and now I get this error:
> >>>> > > > getMore: cursor didn't exist on server, possible restart or > >>>> timeout?
> >>>> > > > On Feb 27, 1:31 pm, fbjork <fredrik.bj...@gmail.com> wrote:
> >>>> > > > > So connection pooling should stick to one host with your > >>>> upcoming > >>>> > > > > change?
> >>>> > > > > On Feb 27, 12:14 pm, Kyle Banker <k...@10gen.com> wrote:
> >>>> > > > > > Hi Kurt,
> >>>> > > > > > I think I know what's going on. Let me commit a fix to the > >>>> Ruby driver. > >>>> > > > > > We'll deploy, and you can test. Should have something for you > >>>> by EOD.
> >>>> > > > > > Kyle
> >>>> > > > > > On Monday, February 27, 2012 1:17:43 PM UTC-5, Kurt wrote:
> >>>> > > > > > > Oh, no, this is just DNS round robin. Once a connection's > >>>> established, > >>>> > > > > > > it will always go to that one mongos.
> >>>> > > > > > > However, there's no guaranty that all the connections in a > >>>> given pool > >>>> > > > > > > will be to the same mongos. So will the getMore op use > >>>> different > >>>> > > > > > > connections in a pool?
> >>>> > > > > > > On Feb 27, 11:47 am, Kyle Banker <kyleban...@gmail.com> > >>>> wrote: > >>>> > > > > > > > Hi Kurt,
> >>>> > > > > > > > If you're connecting to a load balancer instead of > >>>> connecting directly > >>>> > > > > > > > to mongos, and if that load balancer regularly > >>>> round-robins requests > >>>> > > > > > > > to a variety of mongos instances, then the GETMORE op may > >>>> be hitting > >>>> > > > > > > > different mongos instances. And that would explain the > >>>> errors.
> >>>> > > > > > > > Is that what you mean by pooled connections?
> >>>> > > > > > > > Kyle
> >>>> > > > > > > > On Feb 27, 12:35 pm, Kurt <mrk...@gmail.com> wrote:
> >>>> > > > > > > > > Hey Kyle,
> >>>> > > > > > > > > So if I understand correctly, basically pooled > >>>> connections might be > >>>> > > > > > > > > getting mongos addresses, and getMore might use a > >>>> different connection > >>>> > > > > > > > > each time it's called?
> >>>> > > > > > > > > > No, don't use round robin. You must ensure that the > >>>> client > >>>> > > > > > > application > >>>> > > > > > > > > > always hits the same server unless there's an actual > >>>> failure. For > >>>> > > > > > > > > > example, in HAProxy, the balancing algorithm you want > >>>> is "source."
> >>>> > > > > > > > > > > > > Could the issue be that our mongos hosts are > >>>> DNS round robin? > >>>> > > > > > > This would mean that a connection from the pool could get a > >>>> new mongos host > >>>> > > > > > > which would break the cursor?
-- You received this message because you are subscribed to the Google Groups "mongodb-user" group. To post to this group, send email to mongodb-user@googlegroups.com. To unsubscribe from this group, send email to mongodb-user+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
On Tuesday, February 28, 2012 10:35:06 AM UTC-8, Kyle Banker wrote:
> I notice that you've set the max_retries_on_connection_failure. This > could be causing you issues. Imagine you have a connection failure > between an initial query and a getmore. The driver will reconnect, but > it's not guaranteed that it will be routed to the same mongos. Do you > whether these kinds of failures are happening?
> Is there any way you can share your sidekiq code via email, perhaps a > link to a private gist?
> Another step would be to enable verbose logging on the mongos > instances. That way, we can verify that the initial query and getmore > are in fact being sent to different mongos nodes.
> On Feb 28, 1:06 pm, fbjork <fredrik.bj...@gmail.com> wrote: > > Taking a step back for a while. I've removed connection pooling and only > > talking to one mongos host (DNS resolution happens when the Ruby process > > starts up and connects), but getMore is still not working. So either the > > driver is messed up or mongos/mongod isn't working as expected. This > > happens on small dataset too, which means it's not the data size.
> > Could it be some incompatibility with Sidekiq > > (https://github.com/mperham/sidekiq) and the Ruby mongo driver? SIdekiq > > uses Celluloid for thread concurrency.
> > On Tuesday, February 28, 2012 7:06:07 AM UTC-8, Kyle Banker wrote:
> > > Probably because we weren't warning ;)
> > > On Tuesday, February 28, 2012 10:03:08 AM UTC-5, fbjork wrote:
> > >> Yes, the app starts, but I want to make sure Mongoid still works with > the > > >> driver from github. Not sure why mongoid would pass those values to > the > > >> driver?
> > >> On Tuesday, February 28, 2012 6:59:21 AM UTC-8, Kyle Banker wrote:
> > >>> Those are warnings, not errors. Are they preventing your application > > >>> from starting? Basically, we're starting to warn when users pass > invalid > > >>> options to Connection.new.
> > >>> On Monday, February 27, 2012 7:28:21 PM UTC-5, fbjork wrote:
> > >>>> I tried using the latest commit from Github, but when I start rails > I > > >>>> get these errors:
> > >>>> allow_dynamic_fields is not a valid option for Mongo::Connection > > >>>> max_retries_on_connection_failure is not a valid option for > > >>>> Mongo::Connection > > >>>> use_activesupport_time_zone is not a valid option for > > >>>> Mongo::Connection > > >>>> identity_map_enabled is not a valid option for Mongo::Connection > > >>>> host is not a valid option for Mongo::Connection > > >>>> databases is not a valid option for Mongo::Connection > > >>>> host is not a valid option for Mongo::Connection
> > >>>> On Feb 27, 3:27 pm, Kyle Banker <kyleban...@gmail.com> wrote: > > >>>> > Hi Fredrik,
> > >>>> > We've posted the fixes that we believe will help. We'll release > the > > >>>> > gem tomorrow, but if you're eager to test now, you can build the > gem > > >>>> > and install yourself from this latest commit:
> > >>>> > I'd like you to re-enable the connection pooling and try again > with > > >>>> > the new release. If that doesn't solve the problem, we'll have to > > >>>> look > > >>>> > at your DDNS setup.
> > >>>> > Kyle
> > >>>> > On Feb 27, 5:02 pm, fbjork <fredrik.bj...@gmail.com> wrote:
> > >>>> > > And I'm still getting the following error after removing > connection > > >>>> > > pooling:
> > >>>> > > Query response returned CURSOR_NOT_FOUND. Either an invalid > cursor > > >>>> was > > >>>> > > specified, or the cursor may have timed out on the server.
> > >>>> > > On Feb 27, 1:48 pm, fbjork <fredrik.bj...@gmail.com> wrote:
> > >>>> > > > I removed connection pooling and now I get this error:
> > >>>> > > > getMore: cursor didn't exist on server, possible restart or > > >>>> timeout?
> > >>>> > > > On Feb 27, 1:31 pm, fbjork <fredrik.bj...@gmail.com> wrote:
> > >>>> > > > > So connection pooling should stick to one host with your > > >>>> upcoming > > >>>> > > > > change?
> > >>>> > > > > On Feb 27, 12:14 pm, Kyle Banker <k...@10gen.com> wrote:
> > >>>> > > > > > Hi Kurt,
> > >>>> > > > > > I think I know what's going on. Let me commit a fix to > the > > >>>> Ruby driver. > > >>>> > > > > > We'll deploy, and you can test. Should have something for > you > > >>>> by EOD.
> > >>>> > > > > > Kyle
> > >>>> > > > > > On Monday, February 27, 2012 1:17:43 PM UTC-5, Kurt > wrote:
> > >>>> > > > > > > Oh, no, this is just DNS round robin. Once a > connection's > > >>>> established, > > >>>> > > > > > > it will always go to that one mongos.
> > >>>> > > > > > > However, there's no guaranty that all the connections > in a > > >>>> given pool > > >>>> > > > > > > will be to the same mongos. So will the getMore op use > > >>>> different > > >>>> > > > > > > connections in a pool?
> > >>>> > > > > > > On Feb 27, 11:47 am, Kyle Banker <kyleban...@gmail.com>
> > >>>> wrote: > > >>>> > > > > > > > Hi Kurt,
> > >>>> > > > > > > > If you're connecting to a load balancer instead of > > >>>> connecting directly > > >>>> > > > > > > > to mongos, and if that load balancer regularly > > >>>> round-robins requests > > >>>> > > > > > > > to a variety of mongos instances, then the GETMORE op > may > > >>>> be hitting > > >>>> > > > > > > > different mongos instances. And that would explain > the > > >>>> errors.
> > >>>> > > > > > > > Is that what you mean by pooled connections?
> > >>>> > > > > > > > Kyle
> > >>>> > > > > > > > On Feb 27, 12:35 pm, Kurt <mrk...@gmail.com> wrote:
> > >>>> > > > > > > > > Hey Kyle,
> > >>>> > > > > > > > > So if I understand correctly, basically pooled > > >>>> connections might be > > >>>> > > > > > > > > getting mongos addresses, and getMore might use a > > >>>> different connection > > >>>> > > > > > > > > each time it's called?
> > >>>> > > > > > > > > > > > > Could the issue be that our mongos hosts > are > > >>>> DNS round robin? > > >>>> > > > > > > This would mean that a connection from the pool could > get a > > >>>> new mongos host > > >>>> > > > > > > which would break the cursor?
-- You received this message because you are subscribed to the Google Groups "mongodb-user" group. To view this discussion on the web visit https://groups.google.com/d/msg/mongodb-user/-/PKWPQ0Ra-9EJ. To post to this group, send email to mongodb-user@googlegroups.com. To unsubscribe from this group, send email to mongodb-user+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
> Let me know if you need more information on sidekiq.
> On Tuesday, February 28, 2012 10:35:06 AM UTC-8, Kyle Banker wrote:
>> I notice that you've set the max_retries_on_connection_failure. This >> could be causing you issues. Imagine you have a connection failure >> between an initial query and a getmore. The driver will reconnect, but >> it's not guaranteed that it will be routed to the same mongos. Do you >> whether these kinds of failures are happening?
>> Is there any way you can share your sidekiq code via email, perhaps a >> link to a private gist?
>> Another step would be to enable verbose logging on the mongos >> instances. That way, we can verify that the initial query and getmore >> are in fact being sent to different mongos nodes.
>> On Feb 28, 1:06 pm, fbjork <fredrik.bj...@gmail.com> wrote: >> > Taking a step back for a while. I've removed connection pooling and >> only >> > talking to one mongos host (DNS resolution happens when the Ruby >> process >> > starts up and connects), but getMore is still not working. So either >> the >> > driver is messed up or mongos/mongod isn't working as expected. This >> > happens on small dataset too, which means it's not the data size.
>> > Could it be some incompatibility with Sidekiq >> > (https://github.com/mperham/sidekiq) and the Ruby mongo driver? >> SIdekiq >> > uses Celluloid for thread concurrency.
>> > On Tuesday, February 28, 2012 7:06:07 AM UTC-8, Kyle Banker wrote:
>> > > Probably because we weren't warning ;)
>> > > On Tuesday, February 28, 2012 10:03:08 AM UTC-5, fbjork wrote:
>> > >> Yes, the app starts, but I want to make sure Mongoid still works >> with the >> > >> driver from github. Not sure why mongoid would pass those values to >> the >> > >> driver?
>> > >> On Tuesday, February 28, 2012 6:59:21 AM UTC-8, Kyle Banker wrote:
>> > >>> Those are warnings, not errors. Are they preventing your >> application >> > >>> from starting? Basically, we're starting to warn when users pass >> invalid >> > >>> options to Connection.new.
>> > >>> On Monday, February 27, 2012 7:28:21 PM UTC-5, fbjork wrote:
>> > >>>> I tried using the latest commit from Github, but when I start >> rails I >> > >>>> get these errors:
>> > >>>> allow_dynamic_fields is not a valid option for Mongo::Connection >> > >>>> max_retries_on_connection_failure is not a valid option for >> > >>>> Mongo::Connection >> > >>>> use_activesupport_time_zone is not a valid option for >> > >>>> Mongo::Connection >> > >>>> identity_map_enabled is not a valid option for Mongo::Connection >> > >>>> host is not a valid option for Mongo::Connection >> > >>>> databases is not a valid option for Mongo::Connection >> > >>>> host is not a valid option for Mongo::Connection
>> > >>>> On Feb 27, 3:27 pm, Kyle Banker <kyleban...@gmail.com> wrote: >> > >>>> > Hi Fredrik,
>> > >>>> > We've posted the fixes that we believe will help. We'll release >> the >> > >>>> > gem tomorrow, but if you're eager to test now, you can build the >> gem >> > >>>> > and install yourself from this latest commit:
>> > >>>> > I'd like you to re-enable the connection pooling and try again >> with >> > >>>> > the new release. If that doesn't solve the problem, we'll have >> to >> > >>>> look >> > >>>> > at your DDNS setup.
>> > >>>> > Kyle
>> > >>>> > On Feb 27, 5:02 pm, fbjork <fredrik.bj...@gmail.com> wrote:
>> > >>>> > > And I'm still getting the following error after removing >> connection >> > >>>> > > pooling:
>> > >>>> > > Query response returned CURSOR_NOT_FOUND. Either an invalid >> cursor >> > >>>> was >> > >>>> > > specified, or the cursor may have timed out on the server.
>> > >>>> > > On Feb 27, 1:48 pm, fbjork <fredrik.bj...@gmail.com> wrote:
>> > >>>> > > > I removed connection pooling and now I get this error:
>> > >>>> > > > getMore: cursor didn't exist on server, possible restart or >> > >>>> timeout?
>> > >>>> > > > On Feb 27, 1:31 pm, fbjork <fredrik.bj...@gmail.com> wrote:
>> > >>>> > > > > So connection pooling should stick to one host with your >> > >>>> upcoming >> > >>>> > > > > change?
>> > >>>> > > > > On Feb 27, 12:14 pm, Kyle Banker <k...@10gen.com> wrote:
>> > >>>> > > > > > Hi Kurt,
>> > >>>> > > > > > I think I know what's going on. Let me commit a fix to >> the >> > >>>> Ruby driver. >> > >>>> > > > > > We'll deploy, and you can test. Should have something >> for you >> > >>>> by EOD.
>> > >>>> > > > > > Kyle
>> > >>>> > > > > > On Monday, February 27, 2012 1:17:43 PM UTC-5, Kurt >> wrote:
>> > >>>> > > > > > > Oh, no, this is just DNS round robin. Once a >> connection's >> > >>>> established, >> > >>>> > > > > > > it will always go to that one mongos.
>> > >>>> > > > > > > However, there's no guaranty that all the connections >> in a >> > >>>> given pool >> > >>>> > > > > > > will be to the same mongos. So will the getMore op use >> > >>>> different >> > >>>> > > > > > > connections in a pool?
>> > >>>> > > > > > > On Feb 27, 11:47 am, Kyle Banker <kyleban...@gmail.com>
>> > >>>> wrote: >> > >>>> > > > > > > > Hi Kurt,
>> > >>>> > > > > > > > If you're connecting to a load balancer instead of >> > >>>> connecting directly >> > >>>> > > > > > > > to mongos, and if that load balancer regularly >> > >>>> round-robins requests >> > >>>> > > > > > > > to a variety of mongos instances, then the GETMORE >> op may >> > >>>> be hitting >> > >>>> > > > > > > > different mongos instances. And that would explain >> the >> > >>>> errors.
>> > >>>> > > > > > > > Is that what you mean by pooled connections?
>> > >>>> > > > > > > > Kyle
>> > >>>> > > > > > > > On Feb 27, 12:35 pm, Kurt <mrk...@gmail.com> wrote:
>> > >>>> > > > > > > > > Hey Kyle,
>> > >>>> > > > > > > > > So if I understand correctly, basically pooled >> > >>>> connections might be >> > >>>> > > > > > > > > getting mongos addresses, and getMore might use a >> > >>>> different connection >> > >>>> > > > > > > > > each time it's called?
>> > >>>> > > > > > > > > > > > > Could the issue be that our mongos hosts >> are >> > >>>> DNS round robin? >> > >>>> > > > > > > This would mean that a connection from the pool could >> get a >> > >>>> new mongos host >> > >>>> > > > > > > which would break the cursor?
-- You received this message because you are subscribed to the Google Groups "mongodb-user" group. To view this discussion on the web visit https://groups.google.com/d/msg/mongodb-user/-/9HZm3TXpT0QJ. To post to this group, send email to mongodb-user@googlegroups.com. To unsubscribe from this group, send email to mongodb-user+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
>> Let me know if you need more information on sidekiq.
>> On Tuesday, February 28, 2012 10:35:06 AM UTC-8, Kyle Banker wrote:
>>> I notice that you've set the max_retries_on_connection_failure. This >>> could be causing you issues. Imagine you have a connection failure >>> between an initial query and a getmore. The driver will reconnect, but >>> it's not guaranteed that it will be routed to the same mongos. Do you >>> whether these kinds of failures are happening?
>>> Is there any way you can share your sidekiq code via email, perhaps a >>> link to a private gist?
>>> Another step would be to enable verbose logging on the mongos >>> instances. That way, we can verify that the initial query and getmore >>> are in fact being sent to different mongos nodes.
>>> On Feb 28, 1:06 pm, fbjork <fredrik.bj...@gmail.com> wrote: >>> > Taking a step back for a while. I've removed connection pooling and >>> only >>> > talking to one mongos host (DNS resolution happens when the Ruby >>> process >>> > starts up and connects), but getMore is still not working. So either >>> the >>> > driver is messed up or mongos/mongod isn't working as expected. This >>> > happens on small dataset too, which means it's not the data size.
>>> > Could it be some incompatibility with Sidekiq >>> > (https://github.com/mperham/sidekiq) and the Ruby mongo driver? >>> SIdekiq >>> > uses Celluloid for thread concurrency.
>>> > On Tuesday, February 28, 2012 7:06:07 AM UTC-8, Kyle Banker wrote:
>>> > > Probably because we weren't warning ;)
>>> > > On Tuesday, February 28, 2012 10:03:08 AM UTC-5, fbjork wrote:
>>> > >> Yes, the app starts, but I want to make sure Mongoid still works >>> with the >>> > >> driver from github. Not sure why mongoid would pass those values to >>> the >>> > >> driver?
>>> > >> On Tuesday, February 28, 2012 6:59:21 AM UTC-8, Kyle Banker wrote:
>>> > >>> Those are warnings, not errors. Are they preventing your >>> application >>> > >>> from starting? Basically, we're starting to warn when users pass >>> invalid >>> > >>> options to Connection.new.
>>> > >>> On Monday, February 27, 2012 7:28:21 PM UTC-5, fbjork wrote:
>>> > >>>> I tried using the latest commit from Github, but when I start >>> rails I >>> > >>>> get these errors:
>>> > >>>> allow_dynamic_fields is not a valid option for Mongo::Connection >>> > >>>> max_retries_on_connection_failure is not a valid option for >>> > >>>> Mongo::Connection >>> > >>>> use_activesupport_time_zone is not a valid option for >>> > >>>> Mongo::Connection >>> > >>>> identity_map_enabled is not a valid option for Mongo::Connection >>> > >>>> host is not a valid option for Mongo::Connection >>> > >>>> databases is not a valid option for Mongo::Connection >>> > >>>> host is not a valid option for Mongo::Connection
>>> > >>>> On Feb 27, 3:27 pm, Kyle Banker <kyleban...@gmail.com> wrote: >>> > >>>> > Hi Fredrik,
>>> > >>>> > We've posted the fixes that we believe will help. We'll release >>> the >>> > >>>> > gem tomorrow, but if you're eager to test now, you can build >>> the gem >>> > >>>> > and install yourself from this latest commit:
>>> > >>>> > I'd like you to re-enable the connection pooling and try again >>> with >>> > >>>> > the new release. If that doesn't solve the problem, we'll have >>> to >>> > >>>> look >>> > >>>> > at your DDNS setup.
>>> > >>>> > Kyle
>>> > >>>> > On Feb 27, 5:02 pm, fbjork <fredrik.bj...@gmail.com> wrote:
>>> > >>>> > > And I'm still getting the following error after removing >>> connection >>> > >>>> > > pooling:
>>> > >>>> > > Query response returned CURSOR_NOT_FOUND. Either an invalid >>> cursor >>> > >>>> was >>> > >>>> > > specified, or the cursor may have timed out on the server.
>>> > >>>> > > On Feb 27, 1:48 pm, fbjork <fredrik.bj...@gmail.com> wrote:
>>> > >>>> > > > I removed connection pooling and now I get this error:
>>> > >>>> > > > getMore: cursor didn't exist on server, possible restart or >>> > >>>> timeout?
>>> > >>>> > > > On Feb 27, 1:31 pm, fbjork <fredrik.bj...@gmail.com> >>> wrote:
>>> > >>>> > > > > So connection pooling should stick to one host with your >>> > >>>> upcoming >>> > >>>> > > > > change?
>>> > >>>> > > > > On Feb 27, 12:14 pm, Kyle Banker <k...@10gen.com> wrote:
>>> > >>>> > > > > > Hi Kurt,
>>> > >>>> > > > > > I think I know what's going on. Let me commit a fix to >>> the >>> > >>>> Ruby driver. >>> > >>>> > > > > > We'll deploy, and you can test. Should have something >>> for you >>> > >>>> by EOD.
>>> > >>>> > > > > > Kyle
>>> > >>>> > > > > > On Monday, February 27, 2012 1:17:43 PM UTC-5, Kurt >>> wrote:
>>> > >>>> > > > > > > Oh, no, this is just DNS round robin. Once a >>> connection's >>> > >>>> established, >>> > >>>> > > > > > > it will always go to that one mongos.
>>> > >>>> > > > > > > However, there's no guaranty that all the connections >>> in a >>> > >>>> given pool >>> > >>>> > > > > > > will be to the same mongos. So will the getMore op >>> use >>> > >>>> different >>> > >>>> > > > > > > connections in a pool?
>>> > >>>> > > > > > > > If you're connecting to a load balancer instead of >>> > >>>> connecting directly >>> > >>>> > > > > > > > to mongos, and if that load balancer regularly >>> > >>>> round-robins requests >>> > >>>> > > > > > > > to a variety of mongos instances, then the GETMORE >>> op may >>> > >>>> be hitting >>> > >>>> > > > > > > > different mongos instances. And that would explain >>> the >>> > >>>> errors.
>>> > >>>> > > > > > > > Is that what you mean by pooled connections?
>>> > >>>> > > > > > > > Kyle
>>> > >>>> > > > > > > > On Feb 27, 12:35 pm, Kurt <mrk...@gmail.com> >>> wrote:
>>> > >>>> > > > > > > > > Hey Kyle,
>>> > >>>> > > > > > > > > So if I understand correctly, basically pooled >>> > >>>> connections might be >>> > >>>> > > > > > > > > getting mongos addresses, and getMore might use a >>> > >>>> different connection >>> > >>>> > > > > > > > > each time it's called?
>>> > >>>> > > > > > > > > > > > > Could the issue be that our mongos hosts >>> are >>> > >>>> DNS round robin? >>> > >>>> > > > > > > This would mean that a connection from the pool could >>> get a >>> > >>>> new mongos host >>> > >>>> > > > > > > which would break the cursor?
-- You received this message because you are subscribed to the Google Groups "mongodb-user" group. To view this discussion on the web visit https://groups.google.com/d/msg/mongodb-user/-/ORf3HOtMN9YJ. To post to this group, send email to mongodb-user@googlegroups.com. To unsubscribe from this group, send email to mongodb-user+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
If the queries are taking a long time, that's probably because MongoDB itself is being slow to respond and because you have a lot more threads than socket connections.
On Mar 14, 9:58 pm, fbjork <fredrik.bj...@gmail.com> wrote:
> >> Let me know if you need more information on sidekiq.
> >> On Tuesday, February 28, 2012 10:35:06 AM UTC-8, Kyle Banker wrote:
> >>> I notice that you've set the max_retries_on_connection_failure. This > >>> could be causing you issues. Imagine you have a connection failure > >>> between an initial query and a getmore. The driver will reconnect, but > >>> it's not guaranteed that it will be routed to the same mongos. Do you > >>> whether these kinds of failures are happening?
> >>> Is there any way you can share your sidekiq code via email, perhaps a > >>> link to a private gist?
> >>> Another step would be to enable verbose logging on the mongos > >>> instances. That way, we can verify that the initial query and getmore > >>> are in fact being sent to different mongos nodes.
> >>> On Feb 28, 1:06 pm, fbjork <fredrik.bj...@gmail.com> wrote: > >>> > Taking a step back for a while. I've removed connection pooling and > >>> only > >>> > talking to one mongos host (DNS resolution happens when the Ruby > >>> process > >>> > starts up and connects), but getMore is still not working. So either > >>> the > >>> > driver is messed up or mongos/mongod isn't working as expected. This > >>> > happens on small dataset too, which means it's not the data size.
> >>> > Could it be some incompatibility with Sidekiq > >>> > (https://github.com/mperham/sidekiq) and the Ruby mongo driver? > >>> SIdekiq > >>> > uses Celluloid for thread concurrency.
> >>> > On Tuesday, February 28, 2012 7:06:07 AM UTC-8, Kyle Banker wrote:
> >>> > > Probably because we weren't warning ;)
> >>> > > On Tuesday, February 28, 2012 10:03:08 AM UTC-5, fbjork wrote:
> >>> > >> Yes, the app starts, but I want to make sure Mongoid still works > >>> with the > >>> > >> driver from github. Not sure why mongoid would pass those values to > >>> the > >>> > >> driver?
> >>> > >> On Tuesday, February 28, 2012 6:59:21 AM UTC-8, Kyle Banker wrote:
> >>> > >>> Those are warnings, not errors. Are they preventing your > >>> application > >>> > >>> from starting? Basically, we're starting to warn when users pass > >>> invalid > >>> > >>> options to Connection.new.
> >>> > >>> On Monday, February 27, 2012 7:28:21 PM UTC-5, fbjork wrote:
> >>> > >>>> I tried using the latest commit from Github, but when I start > >>> rails I > >>> > >>>> get these errors:
> >>> > >>>> allow_dynamic_fields is not a valid option for Mongo::Connection > >>> > >>>> max_retries_on_connection_failure is not a valid option for > >>> > >>>> Mongo::Connection > >>> > >>>> use_activesupport_time_zone is not a valid option for > >>> > >>>> Mongo::Connection > >>> > >>>> identity_map_enabled is not a valid option for Mongo::Connection > >>> > >>>> host is not a valid option for Mongo::Connection > >>> > >>>> databases is not a valid option for Mongo::Connection > >>> > >>>> host is not a valid option for Mongo::Connection
> >>> > >>>> On Feb 27, 3:27 pm, Kyle Banker <kyleban...@gmail.com> wrote: > >>> > >>>> > Hi Fredrik,
> >>> > >>>> > We've posted the fixes that we believe will help. We'll release > >>> the > >>> > >>>> > gem tomorrow, but if you're eager to test now, you can build > >>> the gem > >>> > >>>> > and install yourself from this latest commit:
> >>> > >>>> > I'd like you to re-enable the connection pooling and try again > >>> with > >>> > >>>> > the new release. If that doesn't solve the problem, we'll have > >>> to > >>> > >>>> look > >>> > >>>> > at your DDNS setup.
> >>> > >>>> > Kyle
> >>> > >>>> > On Feb 27, 5:02 pm, fbjork <fredrik.bj...@gmail.com> wrote:
> >>> > >>>> > > And I'm still getting the following error after removing > >>> connection > >>> > >>>> > > pooling:
> >>> > >>>> > > Query response returned CURSOR_NOT_FOUND. Either an invalid > >>> cursor > >>> > >>>> was > >>> > >>>> > > specified, or the cursor may have timed out on the server.
> >>> > >>>> > > On Feb 27, 1:48 pm, fbjork <fredrik.bj...@gmail.com> wrote:
> >>> > >>>> > > > I removed connection pooling and now I get this error:
> >>> > >>>> > > > getMore: cursor didn't exist on server, possible restart or > >>> > >>>> timeout?
> >>> > >>>> > > > On Feb 27, 1:31 pm, fbjork <fredrik.bj...@gmail.com> > >>> wrote:
> >>> > >>>> > > > > So connection pooling should stick to one host with your > >>> > >>>> upcoming > >>> > >>>> > > > > change?
> >>> > >>>> > > > > On Feb 27, 12:14 pm, Kyle Banker <k...@10gen.com> wrote:
> >>> > >>>> > > > > > Hi Kurt,
> >>> > >>>> > > > > > I think I know what's going on. Let me commit a fix to > >>> the > >>> > >>>> Ruby driver. > >>> > >>>> > > > > > We'll deploy, and you can test. Should have something > >>> for you > >>> > >>>> by EOD.
> >>> > >>>> > > > > > Kyle
> >>> > >>>> > > > > > On Monday, February 27, 2012 1:17:43 PM UTC-5, Kurt > >>> wrote:
> >>> > >>>> > > > > > > Oh, no, this is just DNS round robin. Once a > >>> connection's > >>> > >>>> established, > >>> > >>>> > > > > > > it will always go to that one mongos.
> >>> > >>>> > > > > > > However, there's no guaranty that all the connections > >>> in a > >>> > >>>> given pool > >>> > >>>> > > > > > > will be to the same mongos. So will the getMore op > >>> use > >>> > >>>> different > >>> > >>>> > > > > > > connections in a pool?
> >>> > >>>> > > > > > > > > > > > > Could the issue be that our mongos hosts > >>> are > >>> > >>>> DNS round robin? > >>> > >>>> > > > > > > This would mean that a connection from the pool could > >>> get a > >>> > >>>> new mongos host > >>> > >>>> > > > > > > which would break the cursor?
-- You received this message because you are subscribed to the Google Groups "mongodb-user" group. To post to this group, send email to mongodb-user@googlegroups.com. To unsubscribe from this group, send email to mongodb-user+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
