Mongorestore crashed. Changing KeyMax value to support larger index keys

[Astro]

It turns out that mongodb fails to index those documents where size of index key is larger than 1024 b.

1 .We modified mongo/db/structure/btree/btree.h (Line294)   2014 to 1048576 and compiled src.

2.Now while restoring data which was actually dumped from instace running with mongodb 2.4.9. It fails to restore few indexes into mongodb.

3.Error: Error creating index mydatabase.mycollections: 1 err: "assertion src/mongo/db/structure/btree/btree.cpp:579"
Aborted (core dumped).

4.  We tried to set  db.getSiblingDB('admin').runCommand( { setParameter: 1, failIndexKeyTooLong: false }  before mongorestore. But still mongorestore failed to create those indexes.

The KeyMax related discussion was also found here.

https://groups.google.com/forum/#!searchin/mongodb-user/index$20key$20too$20large/mongodb-user/Baz2XDH6T3c/fS56XOmHouoJ

[Dwight Merriman]

you can't set KeyMax that large -- that is larger than the btree bucket size (8KB).  there is likely index corruption due to the source code change.

i would suggest not going much about 2KB if you change that constant.

you probably need to find a workaround that is different, such as for example storing the value twice, perhaps: 

{ 

  long_form: ___,

  prefix: ___

}

where prefix is the first 1023 or so characters of the long_form, and then index prefix only, and you can do searches specifying both parameters, with prefix indexed.  this will perform well if the prefix is fairly unique, and poorly if the first 1KB or so is highly non-unique.

another option if you aren't doing ordered sequential would be something like:

{ 

  long_form: ___,

  hash: <md5_of_long_form>

}

and then index field hash.

i don't know the key limit with the wildtiger storage engine, you might want to look into that too.

[Tim Hawkins]

Yes, i recently ran into this too, restoring some data that had been dumped from a 2.2 machine as part of an upgrade.

It should not crash, that is a bug, but it should have also an option to either drop the record or truncate the key. Otherwise you end up with unrestorable backups and that is a no no.

If mongorestore had an option to create a json file of the data it could not insert becuase of either uniqueness or key length constraints, then there would be at least a way of fixing the data and recovering it using mongoimport.

--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
 
For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user...@googlegroups.com.
To post to this group, send email to mongod...@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/f89e0f1e-89c4-4301-a952-76bccd962cf2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Asya Kamsky]

Right, this change is discussed here:

http://docs.mongodb.org/manual/reference/limits/#Index-Key-Limit

And there is an option in 2.6 to emulate the old behavior:
failIndexKeyTooLong which is discussed at the end of this page:
http://docs.mongodb.org/manual/release-notes/2.6-compatibility/#enforce-index-key-length-limit

Asya

> https://groups.google.com/d/msgid/mongodb-user/CAEbsasTvAOtmSrMYpeWO1HZu976K0U7_%3D8nKGK%3DsjCg87BJjFg%40mail.gmail.com.

>
> For more options, visit https://groups.google.com/d/optout.

--
MongoDB World is back! June 1-2 in NYC. Use code ASYA for 25% off!

[Atish Andhare]

Hi Asya,

We tried to set failIndexKeyTooLong before restoring the dump. But still it fails.

You received this message because you are subscribed to a topic in the Google Groups "mongodb-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mongodb-user/yzT5oC8lSFs/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mongodb-user...@googlegroups.com.

To post to this group, send email to mongod...@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.

To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/CAOe6dJAjTa6rL1nJfFJn1FBBAigP%2BDqKGpxSLpq8WgEnf_s6iQ%40mail.gmail.com.

[Astro]

 
 
  We also have one more change in code to pass db.upgradeCheckAllDBs(). We modified /mongo/db/index/external_key_generator.cpp L206.
  The 1024 numeral was replaced by 1048576.

[Asya Kamsky]

You set it to false and it still failed with error about key being too long?

Can you please include the exact error you got?

To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/CA%2BMEUNt_d%2BrfkScgWfFx2%3DMQVRQ3cFs8nhz2mmt7Bp1FnYBONQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Astro]

Found following error messages:

mongorestore :  Error creating index mydatabase.mycollection: 1 err: "assertion src/mongo/db/structure/btree/btree.cpp:579"
Aborted (core dumped)

mongod logs: Btree::insert: key too large to index, failing mydatabase.somekey.$somekey_id_1_onemorekey_1_value_1 2037

On Tuesday, March 17, 2015 at 11:41:58 AM UTC+5:30, Asya Kamsky wrote:

You set it to false and it still failed with error about key being too long?

Can you please include the exact error you got?

Hi Asya,

We tried to set failIndexKeyTooLong before restoring the dump. But still it fails.

On 16 Mar 2015 22:30, "Asya Kamsky" <as...@mongodb.com> wrote:

Right, this change is discussed here:

http://docs.mongodb.org/manual/reference/limits/#Index-Key-Limit

And there is an option in 2.6 to emulate the old behavior:
failIndexKeyTooLong which is discussed at the end of this page:
http://docs.mongodb.org/manual/release-notes/2.6-compatibility/#enforce-index-key-length-limit

Asya


On Mon, Mar 16, 2015 at 12:33 PM, Tim Hawkins <tim.th...@gmail.com> wrote:
> Yes, i recently ran into this too, restoring some data that had been dumped
> from a 2.2 machine as part of an upgrade.
>
> It should not crash, that is a bug, but it should have also an option to
> either drop the record or truncate the key. Otherwise you end up with
> unrestorable backups and that is a no no.
>
> If mongorestore had an option to create a json file of the data it could not
> insert becuase of either uniqueness or key length constraints, then there
> would be at least a way of fixing the data and recovering it using
> mongoimport.
>
>
>
>>
>>

>> It turns out that mongodb fails to index those documents where size of
>> index key is larger than 1024 b.
>>
>> 1 .We modified mongo/db/structure/btree/btree.h (Line294)   2014 to
>> 1048576 and compiled src.
>>
>> 2.Now while restoring data which was actually dumped from instace running
>> with mongodb 2.4.9. It fails to restore few indexes into mongodb.
>>
>> 3.Error: Error creating index mydatabase.mycollections: 1 err: "assertion
>> src/mongo/db/structure/btree/btree.cpp:579"
>> Aborted (core dumped).
>>
>> 4.  We tried to set  db.getSiblingDB('admin').runCommand( { setParameter:
>> 1, failIndexKeyTooLong: false }  before mongorestore. But still mongorestore
>> failed to create those indexes.
>>
>> The KeyMax related discussion was also found here.
>>
>>
>> https://groups.google.com/forum/#!searchin/mongodb-user/index$20key$20too$20large/mongodb-user/Baz2XDH6T3c/fS56XOmHouoJ
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "mongodb-user"
>> group.
>>
>> For other MongoDB technical support options, see:
>> http://www.mongodb.org/about/support/.
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "mongodb-user" group.
>> To unsubscribe from this group and stop receiving emails from it, send an

>> email to mongodb-user+unsubscribe@googlegroups.com.

>> To post to this group, send email to mongod...@googlegroups.com.
>> Visit this group at http://groups.google.com/group/mongodb-user.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/mongodb-user/f89e0f1e-89c4-4301-a952-76bccd962cf2%40googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google Groups
> "mongodb-user"
> group.
>
> For other MongoDB technical support options, see:
> http://www.mongodb.org/about/support/.
> ---
> You received this message because you are subscribed to the Google Groups
> "mongodb-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to mongodb-user+unsubscribe@googlegroups.com.

> To post to this group, send email to mongod...@googlegroups.com.
> Visit this group at http://groups.google.com/group/mongodb-user.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/mongodb-user/CAEbsasTvAOtmSrMYpeWO1HZu976K0U7_%3D8nKGK%3DsjCg87BJjFg%40mail.gmail.com.
>
> For more options, visit https://groups.google.com/d/optout.



--
MongoDB World is back! June 1-2 in NYC. Use code ASYA for 25% off!

--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to a topic in the Google Groups "mongodb-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mongodb-user/yzT5oC8lSFs/unsubscribe.

To unsubscribe from this group and all its topics, send an email to mongodb-user+unsubscribe@googlegroups.com.

To post to this group, send email to mongod...@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/CAOe6dJAjTa6rL1nJfFJn1FBBAigP%2BDqKGpxSLpq8WgEnf_s6iQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
 
For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+unsubscribe@googlegroups.com.

To post to this group, send email to mongod...@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/CA%2BMEUNt_d%2BrfkScgWfFx2%3DMQVRQ3cFs8nhz2mmt7Bp1FnYBONQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Asya Kamsky]

And you are starting this mongod with the option
--failIndexKeyTooLong=false? Can you double check that please?

Asya

>>>> >> email to mongodb-user...@googlegroups.com.

>>>> >> To post to this group, send email to mongod...@googlegroups.com.
>>>> >> Visit this group at http://groups.google.com/group/mongodb-user.
>>>> >> To view this discussion on the web visit
>>>> >>
>>>> >> https://groups.google.com/d/msgid/mongodb-user/f89e0f1e-89c4-4301-a952-76bccd962cf2%40googlegroups.com.
>>>> >> For more options, visit https://groups.google.com/d/optout.
>>>> >
>>>> > --
>>>> > You received this message because you are subscribed to the Google
>>>> > Groups
>>>> > "mongodb-user"
>>>> > group.
>>>> >
>>>> > For other MongoDB technical support options, see:
>>>> > http://www.mongodb.org/about/support/.
>>>> > ---
>>>> > You received this message because you are subscribed to the Google
>>>> > Groups
>>>> > "mongodb-user" group.
>>>> > To unsubscribe from this group and stop receiving emails from it, send
>>>> > an

>>>> > email to mongodb-user...@googlegroups.com.

>>>> > To post to this group, send email to mongod...@googlegroups.com.
>>>> > Visit this group at http://groups.google.com/group/mongodb-user.
>>>> > To view this discussion on the web visit
>>>> >
>>>> > https://groups.google.com/d/msgid/mongodb-user/CAEbsasTvAOtmSrMYpeWO1HZu976K0U7_%3D8nKGK%3DsjCg87BJjFg%40mail.gmail.com.
>>>> >
>>>> > For more options, visit https://groups.google.com/d/optout.
>>>>
>>>>
>>>>
>>>> --
>>>> MongoDB World is back! June 1-2 in NYC. Use code ASYA for 25% off!
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "mongodb-user"
>>>> group.
>>>>
>>>> For other MongoDB technical support options, see:
>>>> http://www.mongodb.org/about/support/.
>>>> ---
>>>> You received this message because you are subscribed to a topic in the
>>>> Google Groups "mongodb-user" group.
>>>> To unsubscribe from this topic, visit
>>>> https://groups.google.com/d/topic/mongodb-user/yzT5oC8lSFs/unsubscribe.
>>>> To unsubscribe from this group and all its topics, send an email to

>>>> mongodb-user...@googlegroups.com.

>>>> To post to this group, send email to mongod...@googlegroups.com.
>>>> Visit this group at http://groups.google.com/group/mongodb-user.
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/mongodb-user/CAOe6dJAjTa6rL1nJfFJn1FBBAigP%2BDqKGpxSLpq8WgEnf_s6iQ%40mail.gmail.com.
>>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups
>>> "mongodb-user"
>>> group.
>>>
>>> For other MongoDB technical support options, see:
>>> http://www.mongodb.org/about/support/.
>>> ---
>>> You received this message because you are subscribed to the Google Groups
>>> "mongodb-user" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an

>>> email to mongodb-user...@googlegroups.com.

>>> To post to this group, send email to mongod...@googlegroups.com.
>>> Visit this group at http://groups.google.com/group/mongodb-user.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/mongodb-user/CA%2BMEUNt_d%2BrfkScgWfFx2%3DMQVRQ3cFs8nhz2mmt7Bp1FnYBONQ%40mail.gmail.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>>
>> --
>> MongoDB World is back! June 1-2 in NYC. Use code ASYA for 25% off!
>>

> --
> You received this message because you are subscribed to the Google Groups
> "mongodb-user"
> group.
>
> For other MongoDB technical support options, see:
> http://www.mongodb.org/about/support/.
> ---
> You received this message because you are subscribed to the Google Groups
> "mongodb-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to mongodb-user...@googlegroups.com.

> To post to this group, send email to mongod...@googlegroups.com.
> Visit this group at http://groups.google.com/group/mongodb-user.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/mongodb-user/4c240f3d-315d-4928-9b65-8063ac4f112f%40googlegroups.com.

[Atish Andhare]

No.
From documentation:

"Issue the following command to disable the index key length validation: for a running:program:mongod instance:

db.getSiblingDB('admin').runCommand( { setParameter: 1, failIndexKeyTooLong: false})

I am just doing it from mongo shell before restoring dump. It says I can either do it from mongo shell or
" mongod --setParameter failIndexKeyTooLong=false".
Done it from shell instead.

To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/CAOe6dJDDAQgRkf_BJVUDVCPXZTD9RJ_TY%2Bdm3VZ9RoJLEs6VRw%40mail.gmail.com.

[Asya Kamsky]

Okay, so you can test if it worked via:

db.getSiblingDB('admin').runCommand( { getParameter: 1, failIndexKeyTooLong: 1})

Do you get back "false" for its value?

If you do and you are still getting the error with key too long, you
may have encountered a bug - what is the exact build that you are
running?

Asya
P.S. I realize you said you made your own build, but to see if you
found a bug you will need to use one of our builds, otherwise we may
not be able to reproduce the problem to fix it.

> https://groups.google.com/d/msgid/mongodb-user/CA%2BMEUNupaFrApi2-8QBM%3D8V8ojZkQnfDvJc%3DT%2B_zCoPLHW0DSg%40mail.gmail.com.

[Astro]

Hi Asya,

Just to confirm this. Yes, the value was set to false.

Command in mongo shell:db.getSiblingDB('admin').runCommand( { getParameter: 1, failIndexKeyTooLong: 1}
Output: { "failIndexKeyTooLong" : false, "ok" : 1 }

The question is why it fails after setting this to false?

I think this can be related to btree bucket size Dwight Merriman mentioned above.

Our build consist of no other changes apart from:

1.  src/mongo/db/structure/btree/btree.h L294 . The KeyMax value was increased from 1024 to 1048576.
2.  src/mongo/db/index/external_key_generator.cpp L206 . The 1024 numeral was replaced by 1048576 to pass db.upgradeCheckAllDBs() test during upgrade.

[Asya Kamsky]

Which github branch are you building?

If it's master you may be running into some issue caused by the
unstable dev tree. If it's 3.0 branch then I would try restoring into
mongod that's packaged 3.0.1 with failIndexKeyTooLong set to false.
If it fails then we will need to open a bug against the server.

If that does not fail then I would recommend you use that version, as
you should be able to keep using keys that are too long like you did
in 2.4 with the same flag and you don't need to use a version that you
build yourself.

Asya

> --
> You received this message because you are subscribed to the Google Groups
> "mongodb-user"
> group.
>
> For other MongoDB technical support options, see:
> http://www.mongodb.org/about/support/.
> ---
> You received this message because you are subscribed to the Google Groups
> "mongodb-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to mongodb-user...@googlegroups.com.
> To post to this group, send email to mongod...@googlegroups.com.
> Visit this group at http://groups.google.com/group/mongodb-user.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/mongodb-user/83f1abd3-525f-446f-8830-75cc83fa8ac0%40googlegroups.com.

[Astro]

 Hi,

We downloaded source code from https://fastdl.mongodb.org/src/mongodb-src-r2.6.7.tar.gz?_ga=1.62651116.1797456972.1415870465.

[Asya Kamsky]

I know for a fact that this option is working fine and correctly for multiple current users of MongoDB.

Can you try packaged version (not one that you made changes in and rebuilt) and see if the problem still happens?

Asya

On Mon, Mar 23, 2015 at 1:29 AM, Astro <andhar...@gmail.com> wrote:

Hi Asya,

We downloaded source code from https://fastdl.mongodb.org/src/mongodb-src-r2.6.8.tar.gz?_ga=1.62651116.1797456972.1415870465.

--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
 
For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user...@googlegroups.com.
To post to this group, send email to mongod...@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.

To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/054421e4-4980-4411-a16b-a37f1dc056e7%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Atish Andhare]

We tried that earlier by using mongod that comes with packaged version 2.6.7 and that works well under these circumstances. No problem occurred.

You received this message because you are subscribed to a topic in the Google Groups "mongodb-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mongodb-user/yzT5oC8lSFs/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mongodb-user...@googlegroups.com.

To post to this group, send email to mongod...@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.

To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/CAOe6dJCkCbUJ1X6hG9MyECTBw3OyM5rrHAvg3qcnuBsoKTTOHg%40mail.gmail.com.

[Atish Andhare]

Solved..!! Have set KeyMax=2KB.

Thank you all!!