Account Options

  1. Sign in
The old Google Groups will be going away soon, but your browser is incompatible with the new version.
Google Groups Home
« Groups Home
mongodb-user
Home
+ new post
About this group
Join this group
mongodump big bug
Options
There are currently too many topics in this group that display first. To make this topic appear first, remove this option from another topic.
There was an error processing your request. Please try again.
flag
   9 messages - Collapse all  -  Translate all to Translated (View all originals)
The group you are posting to is a Usenet group. Messages posted to this group will make your email address visible to anyone on the Internet.
Your reply message has not been sent.
Your post was successful
 
From:
To:
Cc:
Followup To:
Add Cc | Add Followup-to | Edit Subject
Subject:
Validation:
For verification purposes please type the characters you see in the picture below or the numbers you hear by clicking the accessibility icon. Listen and type the numbers you hear
 
Thomas  
View profile  
 More options Sep 5 2012, 10:21 am
From: Thomas <thomas.rosenbl...@gmail.com>
Date: Wed, 5 Sep 2012 07:21:54 -0700 (PDT)
Local: Wed, Sep 5 2012 10:21 am
Subject: mongodump big bug
Print | Individual message | Show original | Report this message | Find messages by this author

Hello,
I have a big backup issue with mongo 2.2 under ubuntu 12.04.

When I backup my whole database (mongodump on the db) it badly exports some
of my collections (the metadata contains undefined values)
Exemple:

> db.testairq.stats()

{
"ns" : "test.testairq",
"count" : 0,
"size" : 0,
"storageSize" : 5713248256,
"numExtents" : 32,
"nindexes" : 1,
"lastExtentSize" : 961069056,
"paddingFactor" : 1.5299999998863023,
"systemFlags" : 0,
"userFlags" : 0,
"totalIndexSize" : 8176,
"indexSizes" : {
"_id.loc___id.time_1" : 8176

},
"ok" : 1
}

And the resulted metadata :
{options : { "create" : "testairq", undefined, undefined, undefined,
"autoIndexId" : false }, indexes:[{ "v" : 1, "key" : { "_id.loc" : "2d",
"_id.time" : 1 }, "unique" : true, "ns" : "test.testairq", "name" :
"_id.loc___id.time_1", "sparse" : true }]}

mongorestore failed then to restore with this message
Wed Sep  5 16:17:12 dump/test/testairq.bson
Wed Sep  5 16:17:12 going into namespace [test.testairq]
Wed Sep  5 16:17:12   Assertion failure b.empty() src/mongo/db/json.cpp 645
0xad2211 0x67fced 0xa7daa5 0x56aa20 0x569cd1 0x569cd1 0x56db4a 0xb3b078
0xb41712 0x553f92 0x7f3a3c43076d 0x553e09
 mongorestore(_ZN5mongo15printStackTraceERSo+0x21) [0xad2211]
 mongorestore(_ZN5mongo12verifyFailedEPKcS1_j+0xfd) [0x67fced]
 mongorestore(_ZN5mongo8fromjsonEPKcPi+0x585) [0xa7daa5]
 mongorestore(_ZN7Restore9drillDownEN5boost11filesystem210basic_pathISsNS1_1 1path_traitsEEEbbb+0xff0)
[0x56aa20]
 mongorestore(_ZN7Restore9drillDownEN5boost11filesystem210basic_pathISsNS1_1 1path_traitsEEEbbb+0x2a1)
[0x569cd1]
 mongorestore(_ZN7Restore9drillDownEN5boost11filesystem210basic_pathISsNS1_1 1path_traitsEEEbbb+0x2a1)
[0x569cd1]
 mongorestore(_ZN7Restore5doRunEv+0xdda) [0x56db4a]
 mongorestore(_ZN5mongo8BSONTool3runEv+0x448) [0xb3b078]
 mongorestore(_ZN5mongo4Tool4mainEiPPc+0x1712) [0xb41712]
 mongorestore(main+0x32) [0x553f92]
 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7f3a3c43076d]
 mongorestore(__gxx_personality_v0+0x421) [0x553e09]
assertion: 0 assertion src/mongo/db/json.cpp:645

Is this a known bug or do I have to open a new issue ?
It's really a big issue since my backup system is totally based on this
mechanism.

Thank you,

Thomas.


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Mathias Stearn  
View profile  
 More options Sep 5 2012, 5:03 pm
From: Mathias Stearn <math...@10gen.com>
Date: Wed, 5 Sep 2012 14:03:35 -0700 (PDT)
Local: Wed, Sep 5 2012 5:03 pm
Subject: Re: mongodump big bug
Print | Individual message | Show original | Report this message | Find messages by this author

What does db.system.namespaces.find() say?


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Thomas  
View profile  
 More options Sep 6 2012, 3:55 am
From: Thomas <thomas.rosenbl...@gmail.com>
Date: Thu, 6 Sep 2012 00:55:03 -0700 (PDT)
Local: Thurs, Sep 6 2012 3:55 am
Subject: Re: mongodump big bug
Print | Individual message | Show original | Report this message | Find messages by this author

I don't have any namespaces in the system collection ...

Le mercredi 5 septembre 2012 23:03:35 UTC+2, Mathias Stearn a écrit :


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Mathias Stearn  
View profile  
 More options Sep 7 2012, 1:18 pm
From: Mathias Stearn <math...@10gen.com>
Date: Fri, 7 Sep 2012 10:18:35 -0700 (PDT)
Local: Fri, Sep 7 2012 1:18 pm
Subject: Re: mongodump big bug
Print | Individual message | Show original | Report this message | Find messages by this author

Every database has a system.namespaces collection. Note that there is no
"systems" collection, the actual name of the collection is
"system.namespaces". Please try running db.system.namespaces.find()


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Thomas  
View profile  
 More options Sep 7 2012, 1:30 pm
From: Thomas <thomas.rosenbl...@gmail.com>
Date: Fri, 7 Sep 2012 10:30:55 -0700 (PDT)
Local: Fri, Sep 7 2012 1:30 pm
Subject: Re: mongodump big bug
Print | Individual message | Show original | Report this message | Find messages by this author

Indeed (i just thought that, because the completion on the console did not
work for namespaces) :)

Well for the collection message_queue i have this metadata genrated by
mongodump:
{options : { "create" : "message_queue", "capped" : true, "size" : 1024,
undefined }, indexes:[{ "v" : 1, "key" : { "time" : 1 }, "unique" : true,
"ns" : "nadb.message_queue", "sparse" : true, "name" : "time_1" }]}

And in the db.system.namespaces.find() : (for our message_queue part only)
{ "name" : "nadb.message_queue", "options" : { "create" : "message_queue",
"capped" : true, "size" : 1024, "max" : null } }
{ "name" : "nadb.message_queue.$time_1" }

db.message_queue.stats() produces :

> db.message_queue.stats()

{
"ns" : "nadb.message_queue",
"count" : 63,
"size" : 2796,
"avgObjSize" : 44.38095238095238,
"storageSize" : 4096,
"numExtents" : 1,
"nindexes" : 1,
"lastExtentSize" : 4096,
"paddingFactor" : 1,
"systemFlags" : 0,
"userFlags" : 0,
"totalIndexSize" : 8176,
"indexSizes" : {
"time_1" : 8176
},

"capped" : true,
"max" : 2147483647,
"ok" : 1

}

Thank you,

Thomas.

Le vendredi 7 septembre 2012 19:18:35 UTC+2, Mathias Stearn a écrit :


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Mathias Stearn  
View profile  
 More options Sep 27 2012, 4:27 pm
From: Mathias Stearn <math...@10gen.com>
Date: Thu, 27 Sep 2012 13:27:11 -0700 (PDT)
Local: Thurs, Sep 27 2012 4:27 pm
Subject: Re: mongodump big bug
Print | Individual message | Show original | Report this message | Find messages by this author

I've tracked this down to something that was recently fixed in master:
https://jira.mongodb.org/browse/SERVER-7104. I requested that it be
backported to 2.2.

For now, a possible workaround would be to manually edit the json files to
remove the offending undefineds.


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Alexey Gaziev  
View profile  
 More options Oct 29 2012, 2:25 am
From: Alexey Gaziev <alex.gaz...@gmail.com>
Date: Sun, 28 Oct 2012 23:25:48 -0700 (PDT)
Local: Mon, Oct 29 2012 2:25 am
Subject: Re: mongodump big bug
Print | Individual message | Show original | Report this message | Find messages by this author

Any updates? I have similar issue, can't restore dumped database with same
error type.


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Ketema Harris  
View profile  
 More options Nov 15 2012, 8:43 pm
From: Ketema Harris <ket...@gmail.com>
Date: Thu, 15 Nov 2012 17:43:21 -0800 (PST)
Local: Thurs, Nov 15 2012 8:43 pm
Subject: Re: mongodump big bug
Print | Individual message | Show original | Report this message | Find messages by this author

I found that if I manually edit the file manually to remove undefined, the
restore still fails with:

User Assertion: 15934:JSON object size didn't match file size


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Mark Fossen  
View profile  
 More options Nov 16 2012, 3:40 pm
From: Mark Fossen <m...@mover.io>
Date: Fri, 16 Nov 2012 12:40:11 -0800 (PST)
Local: Fri, Nov 16 2012 3:40 pm
Subject: Re: mongodump big bug
Print | Individual message | Show original | Report this message | Find messages by this author

Unfortunately this is a bug in the mongodump code. When a collection is
created with the command line it creates the collection with all the
options for a capped table but the values are null. you can see this
using db.system.namespaces.find():

{ "name" : "mydatabase.mycollection", "options" : { "create" : "api_apps",
"capped" : null, "size" : null, "max" : null } }

The offending table should have and entry like above. There are a few ways
to fix this. The easiest one I found is to do the following:
1. 'mongoexport -d mydatabase --collection mycollection > mycollection.json'
2. mongo cli 'db.mycolleciton.renameCollection("myoldcollection")
3. 'mongoimport -d mydatabase --collection mycollection --file
mycollection.json

The result namespace entry will be
{ "name" : "mydatabase.mycollection"}

Of course, on an active collection, you may not have this opportunity.
Really this is a band-aid anyway. The real fix should be in the mongodump
code when it creates the metadata file. We can see the offender
in https://github.com/mongodb/mongo/blob/master/src/mongo/tools/dump.cpp

string sns = db + ".system.namespaces";
cursor = conn( true ).query( sns.c_str() , Query() , 0 , 0 , 0 ,
QueryOption_SlaveOk | QueryOption_NoCursorTimeout );
while ( cursor->more() ) {
   BSONObj obj = cursor->nextSafe();
   const string name = obj.getField( "name" ).valuestr();                  
                       <-------------- line 197 name is null here.
   if (obj.hasField("options")) {
     collectionOptions[name] =
obj.getField("options").embeddedObject().getOwned();  <----------------line
199 null name is undefined.
    }

I am pretty sure a small fix here could resolve this issue going forward.
If I had the time or the chops I would fix it. Hopefully there is someone
who can take a look into this.

--
Mark


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
End of messages
« Back to Discussions « Newer topic

Create a group - Google Groups - Google Home - Terms of Service - Privacy Policy
©2013 Google