Disable write concern on Query Router ( Mongos )

[Seref Acet]

I have realised a problem is that i can't disable the write concern parameter on query router.

When i mongorestore the data via query router , every tuple requires an acknowledgements. It totally damages the write performance of MongoDB.

Normally , we can disable it by -w 0 parameter but it doesn't make any sense on query router.

Is there anybody to help that issue ?

Thanks,

 

[Will Berkeley]

By query router, I assume you mean mongos? What do you mean by disable the write concern parameter on mongos? Write concern is per-write, though it can usually be set to a default value per client/database/collection in the drivers. Write concern makes sense when using mongos because mongos should pass the write concern to the shards. mongorestore uses unacknowledged writes - what are you seeing that is telling you that writes from mongorestore require an acknowledgement?

-Will

[Seref Acet]

Hi Will, I have tried the explain my problem which is below.

Hi all,

I can't disable the write concern while i mongorestore with .bson file. It totally damages the insertion rate (insert/sec).

This is my sharding status.

mongos> sh.status()
--- Sharding Status --- 
  sharding version: {
    "_id" : 1,
    "version" : 4,
    "minCompatibleVersion" : 4,
    "currentVersion" : 5,
    "clusterId" : ObjectId("546bd498f0531e27d40544b2")
}
  shards:
    {  "_id" : "shard2",  "host" : "shard2/hadoop5.xx.com:27018" }
  databases:
    {  "_id" : "admin",  "partitioned" : false,  "primary" : "config" }
    {  "_id" : "deneme6",  "partitioned" : false,  "primary" : "shard2" }
    {  "_id" : "seref",  "partitioned" : true,  "primary" : "shard2" }

This is my shard.

shard2:PRIMARY> show tables;
me
oplog.rs
startup_log
system.indexes
system.replset
shard2:PRIMARY> db.me.find()
{ "_id" : ObjectId("546bd2869e14c01f13708886"), "host" : "hadoop5.xx.com" }
shard2:PRIMARY> db.system.replset.find()
{ "_id" : "shard2", "version" : 2, "members" : [ { "_id" : 0, "host" : "hadoop5.xx.com:27018" } ], "settings" : { "getLastErrorDefaults" : { "w" : 0 } } }
shard2:PRIMARY>

This is my mongorestore command.

mongorestore --db seref --collection can11 --noobjcheck --w 0 --noIndexRestore --host 10.0.0.9 --port 27017  /tmp/inventory_kwd.bson

Take attention to log below and notice the `writeConcern: { w: 1 }`.

2014-11-18T18:36:19.560-0500 [conn1] ReplicaSetMonitor::get shard2
2014-11-18T18:36:19.560-0500 [conn1] sending write batch to hadoop5.xx.com:27018: { insert: "can11", documents: [ { _id: ObjectId('54142eacf6365307dcba5f5b'), country: "PL", keyword: "żyje richard kiel", type: 3, depth: 5, avg_daily_views: 1374, avg_unique_users: 311 } ], writeConcern: { w: 1 }, ordered: true, metadata: { shardName: "shard2", shardVersion: [ Timestamp 0|0, ObjectId('000000000000000000000000') ], session: 0 } }
2014-11-18T18:36:19.560-0500 [conn1] write results received from hadoop5.xx.com:27018: { ok: 1, n: 1, lastOp: Timestamp 1416353779000|132, electionId: ObjectId('546bd2879e14c01f13708888') }

how can i disable it ?

18 Kasım 2014 Salı 17:54:09 UTC+2 tarihinde Will Berkeley yazdı:

[Seref Acet]

I totally need this answer.

After I have read all of the documentation , i have noticed that in MongoDB 2.6 version write concern is enforced while writing ( http://docs.mongodb.org/manual/release-notes/2.6-compatibility/#write-methods-incompatibility )

I must find a solution to insert nearly ~15 GB document efficiently (it is growing day by day exponentially ) .

While i'm inserting on one mongodb the insertion rate is = 12.000.000 row/ sec.

While i'm inserting on sharded cluster via mongos the insertion rate is = 300.000 row / sec.

I have only one .bson file which i obtained from Pig Script with HDFS . 

Finally , what is the best practise of restoring .bson file to sharded cluster ? 

19 Kasım 2014 Çarşamba 09:42:56 UTC+2 tarihinde Seref Acet yazdı: