Error when updating: can't append to array using string field name

[ChrisC]

So I'm new to Mongo and in the past few days I've been stumped by this
error many times:

exception 13048 can't append to array using string field name
[configs]

First off are there any docs that might help me understand what this
error means? I have searched around and from what I can gather it has
something to do with inserting collections into a document that has
indexes present? I'm really not sure but I've seen that mentioned.

I am currently trying to work out a way to migrate our relational data
to Mongo to hopefully be the core of our reporting infrastructure.
Right now my current method is to take each data point from the
relational DB and import them one by one into a Mongo document
structure. So I need to insert one document and then update that to
add each nested collection until we've built up our entire data
structure. I am able to update our initial document to add one nested
collection but once I get deeper than that I am running into the error
above. I have one ascending index on one of the fields in the top
level document (otherwise updates take way too long) but that's it. I
am doing the update using the Java driver using code like:

DBObject find = BasicDBObjectBuilder.start()
.add("data.legacyId", currentId)
.get();
DBObject up = BasicDBObjectBuilder.start()
.add("$push", new
BasicDBObject("data.configs", config))
.get();
profiles.update(find, up, true, false);

I am using Mongo v1.6.5. Any tips on how to add nested collections to
an existing document without getting the error above?

[Chris Carrier]

Just to provide some more info I can repeat this on the CLI:

> db.profiles.insert({"_id": 123, "createdById" : 3 , "legacyId" : 11420 , "otherId" : 1})
> db.profiles.update({'_id': 123}, {$push:{"newfield" : { "createdById" : 3 , "legacyId" : 11420 , "otherId": 11}}})
> db.profiles.update({'newfield.createdById': 3}, {$push:{"newfield.newnewfield" : { "createdById" : 3 , "legacyId" : 11420 , "otherId" : 11}}})
can't append to array using string field name [newnewfield]

I must be doing something stupid. Is this kind of functionality just
not supported by Mongo?

Thanks!
Chris

> --
> You received this message because you are subscribed to the Google Groups "mongodb-user" group.
> To post to this group, send email to mongod...@googlegroups.com.
> To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
>
>

[Scott Hernandez]

On Fri, Jan 28, 2011 at 3:12 PM, Chris Carrier <ctca...@gmail.com> wrote:
> Just to provide some more info I can repeat this on the CLI:
>
>> db.profiles.insert({"_id": 123,  "createdById" : 3 , "legacyId" : 11420 , "otherId" : 1})
>> db.profiles.update({'_id': 123}, {$push:{"newfield" : { "createdById" : 3 , "legacyId" : 11420 , "otherId": 11}}})
>> db.profiles.update({'newfield.createdById': 3}, {$push:{"newfield.newnewfield" : { "createdById" : 3 , "legacyId" : 11420 , "otherId" : 11}}})
> can't append to array using string field name [newnewfield]

> db.profiles.findOne()
{
"_id" : 123,
"createdById" : 3,
"legacyId" : 11420,
"newfield" : [
{
"createdById" : 3,
"legacyId" : 11420,
"otherId" : 11
}
],
"otherId" : 1
}

Yes, you need to use the positional operator:
>db.profiles.update({'newfield.createdById': 3}, {$push:{"newfield.$.newnewfield" : { "createdById" : 3 , "legacyId" : 11420 , "otherId" : 11}}})

> db.profiles.findOne()
{
"_id" : 123,
"createdById" : 3,
"legacyId" : 11420,
"newfield" : [
{
"createdById" : 3,
"legacyId" : 11420,
"newnewfield" : [
{
"createdById" : 3,
"legacyId" : 11420,
"otherId" : 11
}
],
"otherId" : 11
}
],
"otherId" : 1
}

I doubt this is what you want, but it is the closest to what you have
described with your updates.

Can you provide the finished document you want?

Maybe this:
>db.profiles.update({'newfield.createdById': 3}, {$push:{"newfield.$" : { "createdById" : 3 , "legacyId" : 11420 , "otherId" : 11}}})
or this:
>db.profiles.update({'newfield.createdById': 3}, {$push:{"newnewfield" : { "createdById" : 3 , "legacyId" : 11420 , "otherId" : 11}}})

[GVP]

@Scott has definitely identified the syntax problem. However, I think
there's a semantic problem here that's best illustrated by breaking
out that last update.

@Chris: you did the following.

> db.profiles.update({'newfield.createdById': 3}, {$push:{"newfield.newnewfield" : { "createdById" : 3 , "legacyId" : 11420 , "otherId" : 11}}})

That command says:
1. Find the following document: {'newfield.createdById': 3}
2. On that document perform the following update:
a. Push the json object into the newfield.newnewfield object.

The problem here is that last part.

In step 2 you push an object into "newfield". When you do this
"newfield" then becomes an array of objects.

The following query you use the $push command on
"newfield.newnewfield". But that's object notation. "newfield" isn't a
JSON object, "newfield" is an array of JSON objects. So when you say
"newfield.newnewfield" the DB doesn't know what's going on.

*But wait, how come my query works, isn't that wrong?*

Well, in a way, the query is kind of wrong, you're just say
"newfield.createdById" and we just explained that "newfield" was an
array, so how does this work?

The reason this works is that queries behave a little differently. The
query is smart enough to see that "newfield" is an array of objects so
it loops through the array searching each object for "createdById".

The query basically has some extra brains here.

So @scott's answer definitely works, but it does point to something
else you may want to consider on how you lay out your data.

Working with an array of objects can be a little painful. Don't get me
wrong, it works and we do have the positional operator, but you may
find that making sub-objects into actual objects is useful.

To take your example above, instead of this:

"newfield" : [
{
"createdById" : 3,
"legacyId" : 11420,
"otherId" : 11
}
],

Try the following:

"newfield" : { "11420" : { "createdById" : 3, "otherId" : 11 } }

Notice how I made the "legacyId" into a key?

By avoiding the arrays, I can now do things like
{ $set : {"newfields.11420.extra" : { more:"stuff" } } }

Basically it lets me drill in directly to the spot I'm looking for.

This is especially useful if you're deeply nested. Trying to update an
array of objects contains arrays of objects just becomes really hairy.

On Jan 28, 5:22 pm, Scott Hernandez <scotthernan...@gmail.com> wrote:

> >> For more options, visit this group athttp://groups.google.com/group/mongodb-user?hl=en.

[Chris Carrier]

Hey GVP thanks so much for the detailed response that really helped me
understand why my inserts weren't working. Your suggestion for the
data structure is interesting it hadn't occurred to me to use the id
as a key. I'm trying to wrap my mind around the details of how it
would work. My data is definitely pretty deeply nested so i need to
find some way to work with it. I see how the model you suggest makes
it easier to insert into deeply nested objects. How do I query
against the data though? With the list of subobjects I can do:

db.profiles.find({'newfield.createdById': 3})

But when each is its own document how could I do a query like that if
I want to search across all the newfields?

Thanks so much for the help,
Chris

[Keith Branton]

IMO GVP's suggestion is not particularly helpful because it makes it impossible to query across sub-objects, and because there is no practical way of indexing values in sub-objects when they are laid out like this. 

I'd stick with the format you were already using.

[Chris Carrier]

Then could you help me understand how to use the positional operator
to update subdocuments that are deeply nested? For instance from
Scott's suggestion I was able to insert subdocuments a couple levels
deep like:

db.profiles.update({'newfield.createdById': 3},
{$push:{"newfield.$.newnewfield" : { "createdById" : 3 , "legacyId" :
11420 , "otherId" : 11}}})

But now I want to insert one deeper but this doesn't work:

db.profiles.update({'newfield.newnewfield.createdById': 3},
{$push:{"newfield.newnewfield.$.deepercollection" : { "createdById" :

3 , "legacyId" : 11420 , "otherId" : 11}}})

I sense that I'm still misunderstanding the positional operator. But
the second case seems similar to the first. The first clause
identifies the document(s) i want updated and then the second says
where to put it. Is this possible?

Chris

[Keith Branton]

Sorry - you can't. I pretty much always replace documents rather than using modifiers for this very reason. If you need to query across objects you can't store them the way GVP suggested.

Virtual collections may help http://jira.mongodb.org/browse/SERVER-142 but it is not very clear exactly what it will offer - and from the number of votes I suspect everyone thinks it will fix everything with nested collections.

[Voltron]

I think I came upon the same problem yesterday. I opted to use nested
objects instead of arrays, to query across documents I did something
like this:


results = db.test_db['test_collection'].find({'newfield.newfield':
{ '$exists' : True } })

You could search for nested structures that way. I have not tested
anything more than 4 layers though

[Keith Branton]

@Voltron I think perhaps you are misunderstanding the problem. If you follow GVP's advice you may end up with:

"newfield" : { 

    "11420" :  { "createdById" : 3,  "otherId" : 11  } 

    "11421" :  { "createdById" : 3,  "otherId" : 16  } 

    "11422" :  { "createdById" : 4,  "otherId" : 18  } 

                 }

Now find a record with any newfield was createdById=3. Without knowing and testing every possible key (11420,11421,11422) you cannot. Nor can you index createdById unless you do it for every key.

Don't get me wrong - I'm sure there are many use cases where nested objects would work fine, but if you need to perform a query by a field inside one then you run into problems.

[Chris Carrier]

OK so then let me see if I understand the work-around here. If I want
to insert more than the 2 levels deep I can basically just query for
the documents I want to update, insert the necessary stuff in code,
and then call save to re-save the entire document? Is that basically
what you mean by 'replace the whole document'?

Thanks a bunch,
Chris

[Keith Branton]

Exactly. It's not perfect, and will not be as efficient as an atomic modifier, but it works.

I'd also recommend employing an also use an optimistic locking mechanism to protect against concurrent changes.

[Chris Carrier]

Great Keith thanks for all the help this really had me stumped for
awhile. I think I have a clear handle on it now.

Chris

[Keith Branton]

It would be great if mongo could support arrays of sub-objects better with update modifiers. I've made the following ticket: http://jira.mongodb.org/browse/SERVER-2476 in the hopes of getting a decent proposal in motion. Comments/criticisms/additions/votes welcome :)

[Chris Carrier]

I agree functionality like this would be awesome. I'm finding I have
to write some pretty gnarly code to get my data inserted in the right
spot. I'm hoping the benefits once I get my data loaded will be worth
it but the loading is much more complicated than I expected.

Chris

[GVP]

@Keith:

Thanks for following up, you did identify the issue with really deep-
nesting. It does become difficult to query against these deeper
objects is you're using the my "id-as-key" method.

Constructing the document as a whole and saving it as a whole is
probably the simplest method for something nested this deep.

I have successfully used the "ID as key" method on some systems, but
not for querying below that point.

@Chris: I'm going to go with Keith here and say that constructing and
saving the whole object is probably the way to go.