addToSet performance

[Manish Pandit]

My document has an array of documents, which has denormalized
documents. In other words, I am sticking whole documents inside the
arrays for faster reads, and I am using $addToSet to "push" any
additional documents to the array in the parent document. I was
wondering though - how does mongo find out the "uniqueness" among,
say, 30 attributes of the document? Will the performance suffer
linearly, i.e. $addToSet a 5000th document in the array will be
extremely slow compared to say, $addToSet when the array size is 10?.
I am not aware of any index to help with this, as I am not sure which
attributes are compared to check for uniqueness.

I am trying to avoid taking out the whole array in Java, manipulating
it, and sticking it back in via save() as that'd be network heavy
operation. $push wont work as I do not want any dupes in the array.

Appreciate the help I am getting through this forum!

-cheers,
Manish

[Kristina Chodorow]

Yes, it'll have to scan through all of the elements of the array.  Honestly, a larger concern is the time it'll take to append 5000 elements. MongoDB will have to keep moving the document as it gets bigger.  I guess you could make a temporary field 5000*(element size) big, then insert the document and unset the temporary field.

--
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To post to this group, send email to mongod...@googlegroups.com.
To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.

[Manish Pandit]

Thanks Kristina! I am not appending 5000 items at once, but I am doing
an $addToSet a 5000th element to an existing array with 4999 elements.
I understand from your response that it will indeed have to scan the
full array to make sure the 5000th element is not a dupe with the 4999
existing ones. Is that correct?

-cheers,
Manish

On Oct 21, 4:40 pm, Kristina Chodorow <krist...@10gen.com> wrote:
> Yes, it'll have to scan through all of the elements of the array.  Honestly,
> a larger concern is the time it'll take to append 5000 elements. MongoDB
> will have to keep moving the document as it gets bigger.  I guess you could
> make a temporary field 5000*(element size) big, then insert the document and
> unset the temporary field.
>

> On Thu, Oct 21, 2010 at 2:36 AM, Manish Pandit <pandit.man...@gmail.com>wrote:
>
>
>
>
>
>
>
> > My document has an array of documents, which has denormalized
> > documents. In other words, I am sticking whole documents inside the
> > arrays for faster reads, and I am using $addToSet to "push" any
> > additional documents to the array in the parent document. I was
> > wondering though - how does mongo find out the "uniqueness" among,
> > say, 30 attributes of the document?  Will the performance suffer
> > linearly, i.e. $addToSet a 5000th document in the array will be
> > extremely slow compared to say, $addToSet when the array size is 10?.
> > I am not aware of any index to help with this, as I am not sure which
> > attributes are compared to check for uniqueness.
>
> > I am trying to avoid taking out the whole array in Java, manipulating
> > it, and sticking it back in via save() as that'd be network heavy
> > operation. $push wont work as I do not want any dupes in the array.
>
> > Appreciate the help I am getting through this forum!
>
> > -cheers,
> > Manish
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "mongodb-user" group.
> > To post to this group, send email to mongod...@googlegroups.com.
> > To unsubscribe from this group, send email to

> > mongodb-user...@googlegroups.com<mongodb-user%2Bunsubscribe@google groups.com>
> > .

[Kristina Chodorow]

Yes.

To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.