bsoncxx::to_json produces invalid json output

[Guy Shefer]

Hi all,

I'm trying to parse a 'view' retrieved from my db. the document has nested objects.

this is the output of bsoncxx::to_json(doc->view()) : 

{

"_id": {

"$oid": "57617d54016b8ed1ed76fde1"

},

"username": "anonymous",

"passwordHash": "blah blah blah doesn't matter",

"isBoundToDevice": true,

"deviceInfo": "{"

IDT1 ":" < default > ","

IDT2 ":" < default > ","

IDT3 ":"

"}",

"virtualDeviceId": "blah blah blah doesn't matter"

}

for some reason the nested "deviceInfo" object has additional parenthesis wrapping it. 

this is not valid json format according to RFC-4627 (checked with several online 'json validator' sites.

does anybody know if there is anything to do other than costly string manipulation for each document?

[Andrew Morrow]

Hi -

I'm looking at your example, and I don't see any parenthesis at all. What I do see that looks a little odd is that the open { after deviceInfo is wrapped in quotes. Is that the issue that you are having? Also, what version of the C++11 driver are you using?

Thanks,

Andrew

--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
 
For other MongoDB technical support options, see: https://docs.mongodb.org/manual/support/
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user...@googlegroups.com.
To post to this group, send email to mongod...@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/c424649f-b9b6-4487-ac3f-6a80f2938852%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Guy Shefer]

darn it, my bad. I meant quotations. the "deviceInfo" object value is surrounded with a pair of quotations and this appears to be incorrect.

[Andrew Morrow]

OK, thanks. In that case, I agree, it looks wrong and I suspect a bug of some sort. Can you tell me what the BSON type of the 'deviceInfo' field is? It appears to be a BSON Object, but I'd like to double check. Also, if there is any way you can get me a sanitized byte-array that reproduces this behavior, that would be the best case for debugging.

Thanks,

Andrew

To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/fcee100e-57eb-4361-b371-3e54ae01ac67%40googlegroups.com.

[Guy Shefer]

yes, it is an object.

I'll see how to get the byte-array (I'm new to mongodb), but the field in the document that caused the problem was inserted as:
doc << "deviceInfo" << "{"IDT1": "<default>","IDT2":"<default>","IDT3":""}"

and this caused the incorrect format upon calling the to_json methos on the retrieved doc's view.

[Andrew Morrow]

To me, that looks like you are putting a string value into deviceInfo. That string happens to look like an Object, but it isn't. I think it is likely then that the to_json method is not escaping quotes within printed string values.

I'm also puzzled about the syntax, as it appears that it is incorrectly quoted (i.e. it quotes {, leaving IDT1 as an unquoted token). I don't see how that could work.

To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/9e8f760c-e47b-4004-aab4-b158614ba58b%40googlegroups.com.

[Guy Shefer]

the actual assignment is with an std::string( "{"IDT1": "<default>","IDT2":"<default>","IDT3":""}") object.

it looks as if the output field value gets wrapped in parentheses needlessly and this messes up the pairing from that point- 

the beginning " of IDT1 gets grouped as the end " of the '{' brace

then the end  " of IDT1 gets grouped as the beginning " of the colon, and so on...

[Guy Shefer]

the prentheses inside the string object are '\"' in the actual code, of course...

[Andrew Morrow]

Hi -

It would be much better if you would post some actual code that compiles and runs and demonstrates the issue. Otherwise, we risk going around in circles. My suspicion is that there is a bug where we aren't properly escaping strings before printing them out in the BSON visitor that emits JSON. Since this is tricky, it would be very helpful if we could look at actual working code rather than discussing code snippets.

Also, just a recommendation, but if you are working with strings with embedded " (double quote) characters in C++11, you may find that using C++11 raw string literals lets you avoid some of the annoying quote escaping:

https://en.wikipedia.org/wiki/C%2B%2B11#New_string_literals

Thanks,

Andrew

To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/869cb07f-fae4-426c-99c2-9efec7e76699%40googlegroups.com.

[Guy Shefer]

Hi,

here is all the code needed to recreate the issue:

document docu{};
std::string obj_value = "{\"id1\":\"val1\", \"id2\":\"val2\"}";
docu << "obj_name" << obj_value;
std::string output = bsoncxx::to_json(docu.view());

this is the output, with the unnecessary parentheses:

{

    "_id" : {

        "$oid" : "5766593d016b8e7ed95def61"

    }, 

    "obj_name" : "{"id1":"val1", "id2":"val2"}"

}

[Andrew Morrow]

On Sun, Jun 19, 2016 at 4:45 AM, Guy Shefer <guy.s...@gmail.com> wrote:

Hi,

here is all the code needed to recreate the issue:

document docu{};
std::string obj_value = "{\"id1\":\"val1\", \"id2\":\"val2\"}";
docu << "obj_name" << obj_value;
std::string output = bsoncxx::to_json(docu.view());

Thanks, I ran this, and I confirm that I get incorrect output.

 

this is the output, with the unnecessary parentheses:

{

    "_id" : {

        "$oid" : "5766593d016b8e7ed95def61"

    }, 

    "obj_name" : "{"id1":"val1", "id2":"val2"}"

}

Here is what puzzles me though. You are saying "unnecessary parenthesis", but my view is that the parenthesis are not incorrect, as they are contained within the string. Instead, the issue is that the internal quote characters are not escaped as they should be per http://www.json.org/.

In particular, my position is that the output here should be:

{

    "_id" : {

        "$oid" : "5766593d016b8e7ed95def61"

    }, 

    "obj_name" : "{\"id1\":\"val1\", \"id2\":\"val2\"}"

}

Do you agree?

Thanks,

Andrew

 

To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/7577dc02-6427-47d4-98cb-8c353e703786%40googlegroups.com.

[Guy Shefer]

the thing is that the value of the "obj_name" key is also a json object...

So I guess what I'm missing here is if it's possible to take a document with embedding and convert it to a json object with nesting

my intent was to compose an embedded document similar to this example , taken from the MongoDB documentation

{
   title: "MongoDB: The Definitive Guide",
   author: [ "Kristina Chodorow", "Mike Dirolf" ],
   published_date: ISODate("2010-09-24"),
   pages: 216,
   language: "English",
   publisher: {
              name: "O'Reilly Media",
              founded: 1980,
              location: "CA"
            }
}

and have the to_json() methos output a valid json with the corresponding nesting,:

{

"title": "MongoDB: The Definitive Guide",

"author": ["Kristina Chodorow", "Mike Dirolf"],

"published_date": 1285286400,

"pages": "216",

"language": "English",

"publisher": {

"name": "O'Reilly Media",

"founded": 1980,

"location": "CA"

}

}

You received this message because you are subscribed to a topic in the Google Groups "mongodb-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mongodb-user/E48TJbBXHRk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mongodb-user...@googlegroups.com.

To post to this group, send email to mongod...@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.

To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/CAHX05qHHdAPGC72-71HPgO%3Dm6y_WZwMG%3DUXM-QNgTBvBgB1UjQ%40mail.gmail.com.

[javantamt]

Hey there,

there is a example which was very helpful to me.
If I got this right, you are going to put a document into a document.
It's possible. But using the streambuilder like that doesn't work. Those function do not interpret what is in between the "".
Using strembuilder you can to say:

bsoncxx::document::value MyfancyDocument = document{} << ... << open_document << "myKey" << "MyVal" << close_document << ... << finalize;

It's also possible to use a existing document. See the the link ;).

Maybe you are new to the whole driver.
Try to search for: "mongocxx bsoncxx new c++11 driver:" as topic in this usergroup. Maybe those topics will help you too ;).

all the best javantamt

[Andrew Morrow]

Hi -

Yes, javantamt is pointing you in the right direction. There is also definitely a bug here related to JSON escaping, so I've filed https://jira.mongodb.org/browse/CXX-941.

The quick answer is that (ignoring the above escaping bug) the driver is doing what you have asked. You asked it to add a new field, of type string, and then gave it some bytes. The fact that those bytes happen to contain JSON text is opaque to the driver.

If you have a std::string containing JSON text, you will first need to convert it to BSON via the bsoncxx::from_json method. You can then append it to the builder:

#include <cstdlib>

#include <iostream>

#include <bsoncxx/builder/stream/document.hpp>

#include <bsoncxx/json.hpp>

int main(int argc, char* argv[]) {

    std::string foobar = "{ \"foo\" : \"bar\" }";

    auto subdoc = bsoncxx::from_json(foobar);

    auto doc = bsoncxx::builder::stream::document{} << "outer" << subdoc << bsoncxx::builder::stream::finalize;

    std::cout << bsoncxx::to_json(doc) << std::endl;

    return EXIT_SUCCESS;

}

For me, this gives the following output:

{

    "outer" : {

        "foo" : "bar"

    }

}

Please have a look through the examples, as there should be several that build nested documents like this. I'm happy to answer any further questions you have about the C++11 BSON API.

Thanks,

Andrew

To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/be17a535-baa8-4fc0-a3da-c4a92b1e5d33%40googlegroups.com.

[Guy Shefer]

Hi Andrew,

yup, it looks like proper document embedding is what I need to read up on next :)

thanks for your assistance and patience!

Guy.

[Guy Shefer]

Hi javantamt,

You're right, I am new to the MongoDB driver- and to non-relational DBs in general.  So I appreciate the help! :)

it looks like that's what I was missing, thanks!!

[Andrew Morrow]

I'm happy to hear that we were able to get that cleared up, and also that your example pointed out an important bug. Please feel free to post in this forum with any further questions you may have about the new C++11 driver API.

Thanks,

Andrew

To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/704b5db9-fc52-4319-8339-91f247e088bf%40googlegroups.com.