mongoexport with slaveOk

[PaulKE]

Hello, mongo people, 

I have a question regarding mongoexport with slaveOk option. 

Currently, i have a replica set with master, slave and arbiter in a shard.  

my reading service is currently reading from slave with 'slaveOk' option from my driver. 

But, when I tried export data based on a query with slaveOk, it seems to export from master 

this mongoexport command ran on 'mongos' and, asking to export from slave since our master is busy with 'write'. 

here is my shell script that runs on 'mongos':

 #!/bin/bash

query=$1       

fields=$2     

params="-d {ourdb} -c {collection-name} -slaveOk 1 -q "$query" -f "$fields" --csv -o download/mongoexport.csv"

echo $params

mongoexport $params

As you can see in above 'red' text, I turned on slaveOk option (according mongo db doc, it's default) to make sure. 

But, exported records are from master. for testing, i put slaveDelay to be one hour, and master & secondary data were not the same due to delay. 

So, i export exported csv is from secondary. but, actually, data were from primary version. 

Again, i am running on 'mongos'. 

Is there anything I did wrong? 

Thanks in advance. 

[PaulKE]

Actually, 

on mongos, reading with 'slaveOk' flag on v2.4.3 still reads from primary instead of secondary. 

So, my reading with slaveOk flag (as here: http://docs.mongodb.org/manual/core/read-operations/)

and my mongoexport with slaveOk DO NOT seem to read from secondary. 

My current driver (lua) does not have read preference options yet. 

But, mongoexport case does not use driver at all. I am running straight from shell. But, i don't know why I cannot read from secondary. 

Please help.. 

Thanks. 

[PaulKE]

Or is there way I can set this on Replication Level (not for current connection or cursor level). 

Ultimately, I would like to make a reading from secondary all the time. 

here are three solutions I would like to have:

1. set readPreference globally: here, i would set in mongo shell in 'primary', i will set like this: rs.setReadPreference('secondary'). Then, I would like to reading is done from always from secondary. 

2. if '1' cannot be done, I would like to set on my current driver level with 'slaveOk' flag on cursor options. But, as I described earlier, that did not seem to work for now. 

3. or if there is a way I can set for current connection at the driver level. again, i cannot seem to find this from my current driver (it's unfortunately not official lua (luamongo)). 

Any help would be appreciated. 

[Bernie Hackett]

mongoexport doesn't do replica set member discovery. You have to specify the host and port (using --host and --port) of the secondary that you want to use. "--slaveOk 1" just tells the mongod instance that it is OK to return documents to mongoexport, even though it is a secondary. Here's an example:

./mongoexport --host localhost --port 27018 --slaveOk 1 -d test -c test

connected to: localhost:27018

{ "_id" : { "$oid" : "5196c2b9e2417b41a7effc14" } }

exported 1 records

Also, "--slaveOk 1" appears to be the default, so all you have to do is add --host and --port parameters.

--
--
You received this message because you are subscribed to the Google
Groups "mongodb-user" group.
To post to this group, send email to mongod...@googlegroups.com
To unsubscribe from this group, send email to
mongodb-user...@googlegroups.com
See also the IRC channel -- freenode.net#mongodb
 
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[PaulKE]

If that is the case, it would be quite difficult to export data in sharded environment..? 

say, what we want is to get 'exported' data based on a 'query', but, I would not know where these data are among secondary nodes in a sharded environment. 

Ideally, what I wanted to run 'mongoexport' on a 'mongos' instance with query, but, do not affect 'write' performance on 'primary' nodes by running on all 'secondary' nodes. 

[Bernie Hackett]

Apologies, I didn't fully read your original post and missed the part about mongos. The reason slaveOk appears to not be working is that your only secondary has a slaveDelay applied. Applying a slaveDelay hides the secondary from the discovery mechanisms that clients use to discover replica set members (the same behavior as setting hidden to true). Since mongos does not see the secondary it can't use it for the query.

Here's the output of rs.conf() on my test machine:

repl0:PRIMARY> rs.conf()

{

"_id" : "repl0",

"version" : 6,

"members" : [

{

"_id" : 0,

"host" : "devbox:27017"

},

{

"_id" : 1,

"host" : "devbox:27018",

"priority" : 0,

"slaveDelay" : 3600

},

{

"_id" : 2,

"host" : "devbox:27019",

"arbiterOnly" : true

}

]

}

And here's the output of 'ismaster', which clients use for discovery and query routing:

repl0:PRIMARY> rs.isMaster()

{

"setName" : "repl0",

"ismaster" : true,

"secondary" : false,

"hosts" : [

"devbox:27017"

],

"arbiters" : [

"devbox:27019"

],

"primary" : "devbox:27017",

"me" : "devbox:27017",

"maxBsonObjectSize" : 16777216,

"maxMessageSizeBytes" : 48000000,

"localTime" : ISODate("2013-05-18T01:21:04.095Z"),

"ok" : 1

}

Notice the slaveDelayed secondary does not show up in the result set. If you had a secondary that was not slaveDelayed, mongos would use it for the query.

[PaulKE]

Yes, this was correct. Thanks a lot!