How to create mongocxx::options::find::modifiers in C++11

[prosper...@gmail.com]

Windows 10

MS VS 2015

MongoDB 3.3.5

MongoDB C++11 Driver 3.0.1

bsoncxx::builder::stream::document find_modifiers;

find_modifiers << "$min" << open_document << concatenate(current_key_values_builder.view()) << close_document << "$maxScan" << 2;
find_options.modifiers(find_modifiers.view());

The format of the document is:

find_modifiers: {

    "$min" : {

        "employeenum" : 1,

        "check" : "B"

    },

    "$maxScan" : 2

}

I am not getting any results (unlike in MongoDB 2.x). The documentation for mongocxx::options::find::modifiers simply says "see also http://docs.mongodb.org/manual/reference/operator/query-modifier/" which are the cursor methods for the shell. At min() it states that "min() exists primarily to support the mongos process, and is a shell wrapper around the query modifier $min." But then at $min it states "The $min operator is deprecated. Use cursor.min() instead."

So, is the $min operator deprecated for C++11 also? Should I be creating my find::modifiers using a different method?

[David Golden]

Hello.

You said that this worked with MongoDB 2.x.  Does that mean that if you use the exact same code with the exact same version of the Mongo C++11 Driver but against a MongoDB server version 2.6 (e.g. 2.6.12), that the code did what you expected?  Or something else?

Some other thoughts:

• Do you have the same indexes on the version that worked and the version that didn't?
• Could you please try it with 3.2.8 rather than a development release of the server to ensure it's not a new bug in the server?
  
• Does it work in the shell?
• If you run an explain with the same query/modifiers what do you get?

Regards,

David

[prosper...@gmail.com]

No, sorry I was unclear. When I was working on this project in 2013, we were using the old C++ driver with:

std::auto_ptr< DBClientCursor > mongo::DBClientConnection::query

 

            cursor = client_connection->query( current_file_blueprint->DB_file_name(), 

                                               mongoDB_query,

                                               number_to_return,   // # to return

                                               number_to_skip,     // # to skip

                                              &fields_to_return);

with a complex query

{ query: { _id: { $ne: 0 }, $hint: { employeenum:  1, check:  1 }, sort: { employeenum:  1, check:  1 }, $min: { employeenum: 1, check: "B" }, $maxScan: 2 )

The reason I can't get this to work in the new driver probably isn't a bug with the new driver or MongoDB but more my understanding of how the "number_to_return" and "number_to_skip" of the old query relates to "mongocxx::options::find::limit" and "mongocxx::options::find::skip" respectively. At least that  is my current hypothesis that I am working on.

My question is only this: Am I coding the mongocxx::options::find::modifiers correctly since there is no documentation or examples? There are no member functions for setting min() or maxScan(), so this seems to be the only way.

If so, then that isn't the reason why I'm not able to replicate the old results and I'll go back to analyzing other possibilities (such as the limit/skip behavior).

[David Golden]

I've tried the kind of query you describe in a different driver (vs MongoDB 3.2.8) and it appears to work (though I had to change the hint to an index name rather than a document).  I'm going to keep digging on what's going on with the C++11 driver and will get back to you today, tomorrow (or worst case on Monday).

About "number_to_return" -- that controlled the batch size returned by the server, which could be the same as the limit for small limits.  Limit is actually enforced by the driver cursor client-side.

David

[prosper...@gmail.com]

David --

Thanks for the help. I believe I have solved the problem. There is apparently some subtle difference in $maxScan between the drivers (pure guess, it has to do with the natural order v. using the index -- I'll do some more experimenting but I have not yet re-installed the older driver/server). By modifying how I use $maxScan gave me the results I wanted. 

I thank you for the help! No need to do any more digging.

Mark

[prosper...@gmail.com]

To be exact, I changed MaxScan from 2 (which worked on the old driver) to 3. I'm unsure why the difference. 

[David Golden]

That's perplexing, but I'm glad to hear it's working for you now.

[prosper...@gmail.com]

Here is where the problem lies (using the MongoDb 3.3.5 shell only):

use TESTMAXSCAN

db.users.insert({name:"sue",age:26,status:"A"})
db.users.insert({name:"bob",age:42,status:"B"})
db.users.insert({name:"mark",age:29,status:"B"})

db.users.find()
{ "_id" : ObjectId("57891a9c52fec6668177681b"), "name" : "sue", "age" : 26, "status" : "A" }
{ "_id" : ObjectId("57891aad52fec6668177681c"), "name" : "bob", "age" : 42, "status" : "B" }
{ "_id" : ObjectId("57891abc52fec6668177681d"), "name" : "mark", "age" : 29, "status" : "B" }

// This is good (a document scan):
db.users.find().maxScan(2)
{ "_id" : ObjectId("57891a9c52fec6668177681b"), "name" : "sue", "age" : 26, "status" : "A" }
{ "_id" : ObjectId("57891aad52fec6668177681c"), "name" : "bob", "age" : 42, "status" : "B" }

db.users.createIndex({name:1})

// This is where I am confused with an index scan -- why only one?
db.users.find().maxScan(2).sort({name:1})
{ "_id" : ObjectId("57891aad52fec6668177681c"), "name" : "bob", "age" : 42, "status" : "B" }

The last statement should have printed 2 documents, right?

In MongoDB 2.2.0 shell (which I reinstalled for testing):

use TESTMAXSCAN

db.users.insert({name:"sue",age:26,status:"A"})
db.users.insert({name:"bob",age:42,status:"B"})
db.users.insert({name:"mark",age:29,status:"B"})

db.users.find()
{ "_id" : ObjectId("57891a9c52fec6668177681b"), "name" : "sue", "age" : 26, "status" : "A" }
{ "_id" : ObjectId("57891aad52fec6668177681c"), "name" : "bob", "age" : 42, "status" : "B" }
{ "_id" : ObjectId("57891abc52fec6668177681d"), "name" : "mark", "age" : 29, "status" : "B" }

db.users.find({$query:{},$maxScan:2})
{ "_id" : ObjectId("57891c05adc58efb84db213f"), "name" : "sue", "age" : 26, "status" : "A" }
{ "_id" : ObjectId("57891c06adc58efb84db2140"), "name" : "bob", "age" : 42, "status" : "B" }

b.users.createIndex({name:1})

// Notice it prints two documents
db.users.find({$query:{},$maxScan:2,$orderby:{name:1}})
{ "_id" : ObjectId("57891c06adc58efb84db2140"), "name" : "bob", "age" : 42, "status" : "B" }
{ "_id" : ObjectId("57891c09adc58efb84db2141"), "name" : "mark", "age" : 29, "status" : "B" }

Is this a bug or a change in maxScan that I don't understand?

[prosper...@gmail.com]

To add: the 2.2.0 shell is using the legacy storage engine and the 3.3.5 is using WiredTiger.

[prosper...@gmail.com]

FYI I've reported this as a bug. https://jira.mongodb.org/browse/SERVER-25111

[David Golden]

Thanks for doing the full diagnosis and reporting a SERVER ticket.