Ruby driver: multiple connections confuses cursors?

[Justin Dossey]

Hi all,

I'm writing a script to migrate data from one MongoDB replica set to
another. There are some schema changes, hence this script. Is it
possible to connect to multiple replica sets in the same program using
the Mongo Ruby driver?

The issue I'm running into is that opening two connections (one for
the old replica set, one for the new) seems to confuse the Ruby driver
when refreshing a cursor: the wrong connection receives the cursor
refresh request and this generates a message,
"Mongo::OperationFailure: Failure to continue iterating cursor because
the the replica set member persisting this cursor at new-mongo-server:
27017 cannot be found"

Sample code (this breaks at time of cursor refresh):

old_replica_set = Mongo::ReplSetConnection.new(OLD_SERVERS, :read
=> :secondary)
new_replica_set = Mongo::ReplSetConnection.new(NEW_SERVERS, :read
=> :primary)

old_replica_set[mydb][mycollection].find({}) do |cursor|
cursor.each do |doc|
print '-'
old_replica_set[mydb][othercollection].find({:doc_id =>
doc['_id']}) do |subcursor|
subcursor.each do |subdoc|
print '.'
end
end
end
end

At no point does this code interact with the new replica set, but the
OperationFailure indicates that the cursor refresh attempt was from
the new replica set and not the old one.

The exception trace:
Mongo::OperationFailure: Failure to continue iterating cursor because
the the replica set member persisting this cursor at new-mongo-server:
27017 cannot be found.
from vendor/bundle/ruby/1.8/gems/mongo-1.6.2/lib/mongo/
cursor.rb:563:in `checkout_socket_from_read_pool'
from vendor/bundle/ruby/1.8/gems/mongo-1.6.2/lib/mongo/
cursor.rb:541:in `checkout_socket_for_op_get_more'
from vendor/bundle/ruby/1.8/gems/mongo-1.6.2/lib/mongo/
cursor.rb:504:in `send_get_more'
from vendor/bundle/ruby/1.8/gems/mongo-1.6.2/lib/mongo/
cursor.rb:460:in `refresh'
from vendor/bundle/ruby/1.8/gems/mongo-1.6.2/lib/mongo/
cursor.rb:128:in `next'
from vendor/bundle/ruby/1.8/gems/mongo-1.6.2/lib/mongo/
cursor.rb:290:in `each'

This is MongoDB 2.0.4 64-bit, running on Ubuntu 12.04 LTS, Mongo gem
version 1.6.2, ruby 1.8.7 (2012-02-08 MBARI 8/0x6770 on patchlevel
358) [x86_64-linux], MBARI 0x6770, Ruby Enterprise Edition 2012.02

[Justin Dossey]

I have confirmed that this is a bug in the mongo ruby driver. I
rewrote my script in Python using pymongo and the issue disappeared.

[Kyle Banker]

I'm not sure how the old replica set connection would actually connect to any of the new servers unless:

* The servers somehow share the same hosts and ports.

Or

* The result of running db.isMaster() is returning hosts from the new server.

Can you share more details about the composition and host naming for the old and new replica sets? Feel free to anonymise the data.

[Justin Dossey]

Thanks for the response. 

My setup is as follows:

Old servers are a 2-member replica set with an arbiter.  The hosts are mongo-01 and mongo-02, the replica set is named "pod_one" and the database name is "foo".  Mongo-01 is the primary. db.isMaster() lists mongo-01 and mongo-02 as the hosts.

New servers are a 3-member replica set, no arbiter.  The hosts are mongo-03, mongo-04, and mongo-05.  Mongo-04 is the primary.  The database name is also "foo".  This replica set is called "pod_two".  db.isMaster() on the primary returns mongo-03, mongo-04, and mongo-05.

Could it be that the assignment of Thread.current[:manager] is being overwritten by the ReplSetConnection to the new database?  If I omit the connection to the new replica set, the code functions as it should, which leads me to believe that the bug is in repl_set_connection.rb.  In 1.6.2, the line I'm interested in is repl_set_connection.rb:153.

Justin Dossey

--
You received this message because you are subscribed to the Google
Groups "mongodb-user" group.
To post to this group, send email to mongod...@googlegroups.com
To unsubscribe from this group, send email to
mongodb-user...@googlegroups.com
See also the IRC channel -- freenode.net#mongodb

[Kyle Banker]

Thanks, Justin. I see the issue and will fix today or tomorrow:

https://jira.mongodb.org/browse/RUBY-444

Kyle

Justin Dossey

mongodb-user+unsubscribe@googlegroups.com

[Kyle Banker]

Just committed a fix. This will go out in the next Ruby driver release.