Can you use mongodump --dbpath against a remote machine?

[Ben McCann]

Hi,

I'd like to use mongodump with the --dbpath arg against a remote machine, but it doesn't seem possible. Am I mistaken that you can't use both --host and --dbpath?

Thanks,

Ben

[Stephen Steneker]

On Wednesday, 23 April 2014 17:30:44 UTC+10, Ben McCann wrote:

I'd like to use mongodump with the --dbpath arg against a remote machine, but it doesn't seem possible. Am I mistaken that you can't use both --host and --dbpath?

Hi Ben,

The --dbpath option accesses data files directly, so they have to be on a path visible to the local machine. You cannot use --dbpath in combination with the --host directive (which will attempt to connect to a remote mongod server) or if there is a running instance of mongod already accessing the db files.

Your best bet would probably be to login to the remote machine to do the mongodump. If you have enough free space you could optionally compress the dump files before transferring over the network, or rely on ssh compression.

If you don't have enough free space to do a local mongodump, you still have a few options:

 - start up a mongod so you can dump remotely

 - if there isn't a running mongod instance, you could also just copy over the data files rather than doing a mongodump & mongorestore. The mongodump output has the benefit of only dumping data (so the files will be smaller) .. however during the mongorestore phase all of the indexes will have to be rebuilt.

Regards,

Stephen

[Ben McCann]

Cool. Thanks. The error message if you try to use both dbpath and host is pretty confusing ("couldn't remove fs lock errno:9 Bad file descriptor"), so this clarifies it a lot.

I think I'm going to try another alternative which is to use the --out option to send to standard out and pipe it over the network which has the advantage of both avoiding the free space issue and also parallelizing the mongodump and data transfer.

Thanks,

Ben

--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
 
For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to a topic in the Google Groups "mongodb-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mongodb-user/nlQSjW6cdfY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mongodb-user...@googlegroups.com.
To post to this group, send email to mongod...@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.

To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/d04d68db-7e84-4319-a013-c9d3b5b458a2%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
about.me/benmccann

[Stephen Steneker]

On Thursday, 24 April 2014 07:44:17 UTC+10, Ben McCann wrote:

Cool. Thanks. The error message if you try to use both dbpath and host is pretty confusing ("couldn't remove fs lock errno:9 Bad file descriptor"), so this clarifies it a lot.

I think I'm going to try another alternative which is to use the --out option to send to standard out and pipe it over the network which has the advantage of both avoiding the free space issue and also parallelizing the mongodump and data transfer.

Hi Ben,

Thanks for the feedback and noting the --out option. The error message is definitely opaque! I'll raise a SERVER issue to detect the option conflict and add a more appropriate error message.

Regards,

Stephen

[Ben McCann]

Hmm, I'm having trouble doing the mongodump locally on the same machine as well. Any idea why this wouldn't work?

$ sudo service mongodb stop

mongodb stop/waiting

$ sudo mongodump -u admin -p mypassword --dbpath /var/lib/mongod --journal --out -

Wed Apr 23 22:05:36.322 dbexit: 

Wed Apr 23 22:05:36.322 [tools] shutdown: going to close listening sockets...

Wed Apr 23 22:05:36.322 [tools] shutdown: going to flush diaglog...

Wed Apr 23 22:05:36.322 [tools] shutdown: going to close sockets...

Wed Apr 23 22:05:36.322 [tools] shutdown: waiting for fs preallocator...

Wed Apr 23 22:05:36.322 [tools] shutdown: lock for final commit...

Wed Apr 23 22:05:36.322 [tools] shutdown: final commit...

Wed Apr 23 22:05:36.322 [tools] shutdown: closing all files...

Wed Apr 23 22:05:36.322 [tools] closeAllFiles() finished

Wed Apr 23 22:05:36.322 [tools] shutdown: removing fs lock...

Wed Apr 23 22:05:36.322 [tools] couldn't remove fs lock errno:9 Bad file descriptor

Wed Apr 23 22:05:36.322 dbexit: really exiting now

$ ls -la /var/lib/mongodb/mongod.lock 

-rwxr-xr-x 1 mongodb mongodb 0 Apr 23 22:04 /var/lib/mongodb/mongod.lock

--

You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
 
For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to a topic in the Google Groups "mongodb-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mongodb-user/nlQSjW6cdfY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mongodb-user...@googlegroups.com.
To post to this group, send email to mongod...@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.

To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/64c85763-ecde-42fe-8380-f23922b7fa2f%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
about.me/benmccann

[Asya Kamsky]

I'm not sure that you can use --out to stdout with a full dump (not that it explains the complete lack of an error message).

Can you try starting this with--out  to a location? (you can interrupt it if it starts up okay, but that will confirm if that's related to what's going on).

Really, --out - only makes sense if you are doing a dump of a single collection (as that goes into a single file) otherwise it's hard to see how that would work.   Though I thought if directed to gzip it *should* work ...

In fact, I'm getting this error when I try it:

You must specify database and collection to print to stdout

I'm afraid you must be using previous version which doesn't enforce this limitation correctly...

Asya

You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user...@googlegroups.com.

To post to this group, send email to mongod...@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.

To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/CAH3cagPMPyE0vRvpyvcKhznvJ3SSwUenjUN_kjbFbq0TPPbHFA%40mail.gmail.com.

[Ben McCann]

Thanks Asya. That's a helpful note. I realized later from reading the docs that --out leaves out the metadata, so it wouldn't be appropriate for me. I'm using 2.4, so that's interesting that perhaps 2.6 works differently in this scenario. I'm not exactly sure what my problem was before, but I've gotten past it now in any case.

Thanks,

Ben

To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/CAOe6dJBLL6enTQD9HjAiMvfjJUxtA5YKQs8RtfF6z6T9pV5x%3Dg%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
about.me/benmccann