Java Error loading BSON file with $ref

[PCimino]

Programatically created a an object with a reference. I can see and
export the data. In Java, I can load and reload the data from MongoDB.

Now I want to read it back in from a BSON file for unit tests.

The com.mongodb.DBCollection.validateKey() is throwing
java.lang.IllegalArgumentException: fields stored in the db can't
start with '$' (Bad Key: '$ref')

Objects look like this:

Parent:
{
"_id" : ObjectId("4f97f90b697eadc853dc0b4c"),
"_class" : "com.myorg.CustomerDetails",
"customerAddress" : {
"$ref" : "customerAddress",
"$id" : ObjectId("000000000000000000000011")
}
}

Reference Object:
{
"_id" : ObjectId("000000000000000000000011"),
"_class" : "com.myorg.CustomerAddress",
}

[Sid]

Mongo Object fields hold $ prefixed fields. This is for operations and
internal working.

You would need to remove these $ symbols before using the fields
within the DB since $ for fields would be ambiqous with operations and
Mongo would not be able to tell the difference.

[Scott Hernandez]

How are you building that object? Are you parsing it from a string? If
so, are you using the JSON class in the driver?

> --
> You received this message because you are subscribed to the Google Groups "mongodb-user" group.
> To post to this group, send email to mongod...@googlegroups.com.
> To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
>

[PCimino]

Remove them from where?

If I remove them from the BSON file, then what happens is the MongoDB
gets loaded, but the parent contains an instance of the child, all
attributes are null except the id and ref fields.

If I create the parent in Java, the child attribute has the @DBRef
annotation, store an instance of the child and then save from Java,
then MongoDB contains the objects shown above.

Yes I'm building from a string, I'll look into the driver being used,
I believe its the Mongo Java Driver. Or is there a JSON parsing class
in that package?

[PCimino]

Mongo Java Driver. The issue desn't appear to be the driver pushing/
pulling to/from MongoDB. Its the parsing of the BSON file.

db is a refenece to the Mongo database
br is a BufferedReader to the BSON file
[code]
while ((thisLine = br.readLine()) != null) {
Map<String, Object> objectInMap =
mapper.readValue(thisLine.getBytes(), new TypeReference<Map<String,
Object>>() {});
BasicDBObjectBuilder b = BasicDBObjectBuilder.start(objectInMap);
DBObject dbObject = b.get();
BasicDBObject dbObject = new BasicDBObject(objectInMap);
db.getCollection(collection).insert(dbObject);
}
[/code]

On Apr 26, 11:36 am, Scott Hernandez <scotthernan...@gmail.com> wrote:
> How are you building that object? Are you parsing it from a string? If
> so, are you using the JSON class in the driver?
>
>
>
>
>
>
>
> On Thu, Apr 26, 2012 at 5:58 AM, PCimino <pcim...@gmail.com> wrote:
> > Programatically created a an object with a reference. I can see and
> > export the data. In Java, I can load and reload the data from MongoDB.
>
> > Now I want to read it back in from a BSON file for unit tests.
>
> > The com.mongodb.DBCollection.validateKey() is throwing
> > java.lang.IllegalArgumentException: fields stored in the db can't
> > start with '$' (Bad Key: '$ref')
>
> > Objects look like this:
>
> > Parent:
> > {
> >     "_id" : ObjectId("4f97f90b697eadc853dc0b4c"),
> >     "_class" : "com.myorg.CustomerDetails",
> >           "customerAddress" : {
> >                "$ref" : "customerAddress",
> >                "$id" : ObjectId("000000000000000000000011")
> >           }
> > }
>
> > Reference Object:
> > {
> >         "_id" : ObjectId("000000000000000000000011"),
> >          "_class" : "com.myorg.CustomerAddress",
> > }
>
> > --

> > You received this message because you are subscribed to the Google Groups "mongodb-user" group.> To post to this group, send email tomongo...@googlegroups.com.> To unsubscribe from this group, send email tomongodb-use...@googlegroups.com.

[Scott Hernandez]

That isn't BSON, it is JSON, which is fairly different.

How are you generating the data you use the BufferedReader to read?
Where is it coming from?

> To post to this group, send email to mongod...@googlegroups.com.
> To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.

[PCimino]

Mongodump to a file, then bsondump to get the file into bson format.
The file basically looks like what I already posted, had to take out
the "Object()" class identifier t get things to work.

{ "_id" : "4f97f90b697eadc853dc0b4c", "_class" :
"com.myorg.CustomerDetails", "name" : "my parent", "customerAddress" :
{ "$ref" : "customerAddress", "$id" :
"000000000000000000000011")} }

{ "_id" : "000000000000000000000011", "_class" :
"com.myorg.CustomerAddress", "name" : "my child"}

> >> > -->> > You received this message because you are subscribed to the Google Groups "mongodb-user" group.> To post to this group, send emailtomo...@googlegroups.com.> To unsubscribe from this group, send emailtomongodb-...@googlegroups.com.

[Scott Hernandez]

If you have BSON files you can load their binary contents directly
into DBObjects in java. This will bypass the BSON->JSON (string) ->
??? -> JAVA (DBOjbect) conversion, which is causing the problems you
are seeing, and will be much faster as well.

Here is a snippet which you can work from:
InputStream in = new FileInputStream("somefile.bson");
DBDecoder decoder = new DefaultDBDecoder();
while(in.available() > 0) {
DBObject dbObj = decoder.decode( in, (DBCollection) null );
System.out.println(dbObj.toString());

> To post to this group, send email to mongod...@googlegroups.com.
> To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.

[PCimino]

Thanks. That does work.

Ideally we want files to be flat files so they can be edited directly
to create test data..

Seems like we might have to have fixtures to create test data in Mongo
and then export DBs when we want to reinitialize a new DB. Like to
eliminate the extra step if possible. We don't want to write code to
create test data to test code.

> >> >> > -->> > You received this message because you are subscribed to the Google Groups "mongodb-user" group.> To post to this group, sendemailto...@googlegroups.com.> To unsubscribe from this group, sendemailtomongod...@googlegroups.com.

[Scott Hernandez]

In that case you probably want to store them as json and use the
JSON.parse(...) to load them back. Please not that not all data will
round-trip via json to the same resulting data. Numbers for example
may change from int32/64/double depending on how they are parsed and
their values.

> To post to this group, send email to mongod...@googlegroups.com.
> To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.