I am going to combine documents from two separate collections.
I will do using mongodump and mongorestore. commands.
There may be a chance of having same auto generated Id in both
collections.
Will combine operation (using mongo restore) happen properly due to
same auto generated Id?
On Sun, May 13, 2012 at 2:31 AM, kumar <kumar.a...@gmail.com> wrote:
> Hi All,
> I am going to combine documents from two separate collections.
> I will do using mongodump and mongorestore. commands.
> There may be a chance of having same auto generated Id in both
> collections.
> Will combine operation (using mongo restore) happen properly due to
> same auto generated Id?
> Thanks.
> Kumar
> --
> You received this message because you are subscribed to the Google Groups
> "mongodb-user" group.
> To post to this group, send email to mongodb-user@googlegroups.com.
> To unsubscribe from this group, send email to
> mongodb-user+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/mongodb-user?hl=en.
> Best is to handle via application side logic instead of below commands.
> On Sun, May 13, 2012 at 2:31 AM, kumar <kumar.a...@gmail.com> wrote:
>> Hi All,
>> I am going to combine documents from two separate collections.
>> I will do using mongodump and mongorestore. commands.
>> There may be a chance of having same auto generated Id in both
>> collections.
>> Will combine operation (using mongo restore) happen properly due to
>> same auto generated Id?
>> Thanks.
>> Kumar
>> --
>> You received this message because you are subscribed to the Google Groups
>> "mongodb-user" group.
>> To post to this group, send email to mongodb-user@googlegroups.com.
>> To unsubscribe from this group, send email to
>> mongodb-user+unsubscribe@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/mongodb-user?hl=en.
If you are planning on having the field that may have duplicates be unique (have a unique index), then you need to decide how you want to handle that case. The most obvious way is not to have a unique index on that field, reconcile the differences etc. but it is really up to you. See this for example:
Alternatively you may want to have more control over what happens, in which case you might want to explore writing something yourself and using application code to reconcile your data before insertion (if you do look at the suggested merging collections link, be sure to take a look at Kristina's comment about batch insert).
Adam
On Saturday, May 12, 2012 10:14:43 PM UTC+1, Raxit Sheth <Mobile 4 Mumbai> wrote:
> On Sun, May 13, 2012 at 2:39 AM, Raxit Sheth <raxitsheth2...@gmail.com>wrote:
>> Best is to handle via application side logic instead of below commands.
>> On Sun, May 13, 2012 at 2:31 AM, kumar <kumar.a...@gmail.com> wrote:
>>> Hi All,
>>> I am going to combine documents from two separate collections. >>> I will do using mongodump and mongorestore. commands.
>>> There may be a chance of having same auto generated Id in both >>> collections. >>> Will combine operation (using mongo restore) happen properly due to >>> same auto generated Id?
>>> Thanks. >>> Kumar
>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "mongodb-user" group. >>> To post to this group, send email to mongodb-user@googlegroups.com. >>> To unsubscribe from this group, send email to >>> mongodb-user+unsubscribe@googlegroups.com. >>> For more options, visit this group at >>> http://groups.google.com/group/mongodb-user?hl=en.
Hi,
I have the same problem. In my case, i'd if a document has an _id
already present in the collection, i'd like it to be regenerated to
make sure all documents are inserted. Is there an option for that? if
not, how can I remove the _id fields of dumped documents?
thanks
jean
> If you are planning on having the field that may have duplicates be unique
> (have a unique index), then you need to decide how you want to handle that
> case. The most obvious way is not to have a unique index on that field,
> reconcile the differences etc. but it is really up to you. See this for
> example:
> Alternatively you may want to have more control over what happens, in which
> case you might want to explore writing something yourself and using
> application code to reconcile your data before insertion (if you do look at
> the suggested merging collections link, be sure to take a look at
> Kristina's comment about batch insert).
> Adam
> On Saturday, May 12, 2012 10:14:43 PM UTC+1, Raxit Sheth <Mobile 4 Mumbai>
> wrote:
> > On Sun, May 13, 2012 at 2:39 AM, Raxit Sheth <raxitsheth2...@gmail.com>wrote:
> >> Best is to handle via application side logic instead of below commands.
> >> On Sun, May 13, 2012 at 2:31 AM, kumar <kumar.a...@gmail.com> wrote:
> >>> Hi All,
> >>> I am going to combine documents from two separate collections.
> >>> I will do using mongodump and mongorestore. commands.
> >>> There may be a chance of having same auto generated Id in both
> >>> collections.
> >>> Will combine operation (using mongo restore) happen properly due to
> >>> same auto generated Id?
> >>> Thanks.
> >>> Kumar
> >>> --
> >>> You received this message because you are subscribed to the Google
> >>> Groups "mongodb-user" group.
> >>> To post to this group, send email to mongodb-user@googlegroups.com.
> >>> To unsubscribe from this group, send email to
> >>> mongodb-user+unsubscribe@googlegroups.com.
> >>> For more options, visit this group at
> >>>http://groups.google.com/group/mongodb-user?hl=en.
There is no automatic way to do this. There are several of approaches you could take though, for example:
You could restore the data to a new collection, identify the dupes, and insert them without the _id field specified, that would mean a new _id would be generated automatically. Then you can simply drop the dupes when you do your normal restore as I mentioned.
Or, you could write a script to iterate through the records check for duplicates against your target collection, if found, generate a new one, insert, otherwise insert as-is
Or, you could remove the unique index on the collection, restore all the data, update the dupes in place and re-create the index
It's all about finding the right approach to fit your needs.
On Wednesday, May 23, 2012 9:23:18 AM UTC+1, jeannot wrote:
> Hi, > I have the same problem. In my case, i'd if a document has an _id > already present in the collection, i'd like it to be regenerated to > make sure all documents are inserted. Is there an option for that? if > not, how can I remove the _id fields of dumped documents? > thanks > jean
> On 14 mai, 14:07, Adam C <ad...@10gen.com> wrote: > > If you are planning on having the field that may have duplicates be > unique > > (have a unique index), then you need to decide how you want to handle > that > > case. The most obvious way is not to have a unique index on that field, > > reconcile the differences etc. but it is really up to you. See this for > > example:
> > Alternatively you may want to have more control over what happens, in > which > > case you might want to explore writing something yourself and using > > application code to reconcile your data before insertion (if you do look > at > > the suggested merging collections link, be sure to take a look at > > Kristina's comment about batch insert).
> > Adam
> > On Saturday, May 12, 2012 10:14:43 PM UTC+1, Raxit Sheth <Mobile 4 > Mumbai> > > wrote:
> > > you may want to chk comment also on this blog
> > > On Sun, May 13, 2012 at 2:39 AM, Raxit Sheth <raxitsheth2...@gmail.com>wrote:
> > >> Best is to handle via application side logic instead of below > commands.
> > >> On Sun, May 13, 2012 at 2:31 AM, kumar <kumar.a...@gmail.com> wrote:
> > >>> Hi All,
> > >>> I am going to combine documents from two separate collections. > > >>> I will do using mongodump and mongorestore. commands.
> > >>> There may be a chance of having same auto generated Id in both > > >>> collections. > > >>> Will combine operation (using mongo restore) happen properly due to > > >>> same auto generated Id?
> > >>> Thanks. > > >>> Kumar
> > >>> -- > > >>> You received this message because you are subscribed to the Google > > >>> Groups "mongodb-user" group. > > >>> To post to this group, send email to mongodb-user@googlegroups.com. > > >>> To unsubscribe from this group, send email to > > >>> mongodb-user+unsubscribe@googlegroups.com. > > >>> For more options, visit this group at > > >>>http://groups.google.com/group/mongodb-user?hl=en.
> There is no automatic way to do this. There are several of approaches you
> could take though, for example:
> You could restore the data to a new collection, identify the dupes, and
> insert them without the _id field specified, that would mean a new _id
> would be generated automatically. Then you can simply drop the dupes when
> you do your normal restore as I mentioned.
> Or, you could write a script to iterate through the records check for
> duplicates against your target collection, if found, generate a new one,
> insert, otherwise insert as-is
> Or, you could remove the unique index on the collection, restore all the
> data, update the dupes in place and re-create the index
> It's all about finding the right approach to fit your needs.
> Adam
> On Wednesday, May 23, 2012 9:23:18 AM UTC+1, jeannot wrote:
> > Hi,
> > I have the same problem. In my case, i'd if a document has an _id
> > already present in the collection, i'd like it to be regenerated to
> > make sure all documents are inserted. Is there an option for that? if
> > not, how can I remove the _id fields of dumped documents?
> > thanks
> > jean
> > On 14 mai, 14:07, Adam C <ad...@10gen.com> wrote:
> > > If you are planning on having the field that may have duplicates be
> > unique
> > > (have a unique index), then you need to decide how you want to handle
> > that
> > > case. The most obvious way is not to have a unique index on that field,
> > > reconcile the differences etc. but it is really up to you. See this for
> > > example:
> > > Alternatively you may want to have more control over what happens, in
> > which
> > > case you might want to explore writing something yourself and using
> > > application code to reconcile your data before insertion (if you do look
> > at
> > > the suggested merging collections link, be sure to take a look at
> > > Kristina's comment about batch insert).
> > > > you may want to chk comment also on this blog
> > > > On Sun, May 13, 2012 at 2:39 AM, Raxit Sheth <raxitsheth2...@gmail.com>wrote:
> > > >> Best is to handle via application side logic instead of below
> > commands.
> > > >> On Sun, May 13, 2012 at 2:31 AM, kumar <kumar.a...@gmail.com> wrote:
> > > >>> Hi All,
> > > >>> I am going to combine documents from two separate collections.
> > > >>> I will do using mongodump and mongorestore. commands.
> > > >>> There may be a chance of having same auto generated Id in both
> > > >>> collections.
> > > >>> Will combine operation (using mongo restore) happen properly due to
> > > >>> same auto generated Id?
> > > >>> Thanks.
> > > >>> Kumar
> > > >>> --
> > > >>> You received this message because you are subscribed to the Google
> > > >>> Groups "mongodb-user" group.
> > > >>> To post to this group, send email to mongodb-user@googlegroups.com.
> > > >>> To unsubscribe from this group, send email to
> > > >>> mongodb-user+unsubscribe@googlegroups.com.
> > > >>> For more options, visit this group at
> > > >>>http://groups.google.com/group/mongodb-user?hl=en.
If you simply insert and there is existing document then you will get
an error and you can re-insert with a new _id value, or none, for that
document. To do this, and get back the error, you must do safe inserts
and handle the error case.
On Wed, May 30, 2012 at 7:39 AM, jeannot <jean.oll...@gmail.com> wrote:
> Hi Adam,
> Thanks for your answer, the first solution is ok for my problem
> jeannot
> On 23 mai, 11:46, Adam C <ad...@10gen.com> wrote:
>> There is no automatic way to do this. There are several of approaches you
>> could take though, for example:
>> You could restore the data to a new collection, identify the dupes, and
>> insert them without the _id field specified, that would mean a new _id
>> would be generated automatically. Then you can simply drop the dupes when
>> you do your normal restore as I mentioned.
>> Or, you could write a script to iterate through the records check for
>> duplicates against your target collection, if found, generate a new one,
>> insert, otherwise insert as-is
>> Or, you could remove the unique index on the collection, restore all the
>> data, update the dupes in place and re-create the index
>> It's all about finding the right approach to fit your needs.
>> Adam
>> On Wednesday, May 23, 2012 9:23:18 AM UTC+1, jeannot wrote:
>> > Hi,
>> > I have the same problem. In my case, i'd if a document has an _id
>> > already present in the collection, i'd like it to be regenerated to
>> > make sure all documents are inserted. Is there an option for that? if
>> > not, how can I remove the _id fields of dumped documents?
>> > thanks
>> > jean
>> > On 14 mai, 14:07, Adam C <ad...@10gen.com> wrote:
>> > > If you are planning on having the field that may have duplicates be
>> > unique
>> > > (have a unique index), then you need to decide how you want to handle
>> > that
>> > > case. The most obvious way is not to have a unique index on that field,
>> > > reconcile the differences etc. but it is really up to you. See this for
>> > > example:
>> > > Alternatively you may want to have more control over what happens, in
>> > which
>> > > case you might want to explore writing something yourself and using
>> > > application code to reconcile your data before insertion (if you do look
>> > at
>> > > the suggested merging collections link, be sure to take a look at
>> > > Kristina's comment about batch insert).
>> > > > you may want to chk comment also on this blog
>> > > > On Sun, May 13, 2012 at 2:39 AM, Raxit Sheth <raxitsheth2...@gmail.com>wrote:
>> > > >> Best is to handle via application side logic instead of below
>> > commands.
>> > > >> On Sun, May 13, 2012 at 2:31 AM, kumar <kumar.a...@gmail.com> wrote:
>> > > >>> Hi All,
>> > > >>> I am going to combine documents from two separate collections.
>> > > >>> I will do using mongodump and mongorestore. commands.
>> > > >>> There may be a chance of having same auto generated Id in both
>> > > >>> collections.
>> > > >>> Will combine operation (using mongo restore) happen properly due to
>> > > >>> same auto generated Id?
>> > > >>> Thanks.
>> > > >>> Kumar
>> > > >>> --
>> > > >>> You received this message because you are subscribed to the Google
>> > > >>> Groups "mongodb-user" group.
>> > > >>> To post to this group, send email to mongodb-user@googlegroups.com.
>> > > >>> To unsubscribe from this group, send email to
>> > > >>> mongodb-user+unsubscribe@googlegroups.com.
>> > > >>> For more options, visit this group at
>> > > >>>http://groups.google.com/group/mongodb-user?hl=en.
> --
> You received this message because you are subscribed to the Google
> Groups "mongodb-user" group.
> To post to this group, send email to mongodb-user@googlegroups.com
> To unsubscribe from this group, send email to
> mongodb-user+unsubscribe@googlegroups.com
> See also the IRC channel -- freenode.net#mongodb
