Clone the db from a meteor.com deployed app for local testing?

[Oli Evans]

What's the best way to get a mongo db dump from a meteor.com deployed app?

I've got http://goto.meteor.com/ deployed with some interesting data. I'm making some changes and improvements. I'd like to test those changes against a copy of the live db before I deploy the new stuff to make sure it's all gonna work nice.

Anyone got any pointers? I'm trying stuff like: 

meteor mongo goto --url

meteor mongo goto

db.copyDatabase( '[remote Db Name]', '[Local Db Name]', '[remote db host]' );

But I think the password protection on meteor.com are hosting it. Is it even possible? I've gotta go give a talk to the London MongoDB User Group tomorrow: http://www.meetup.com/London-MongoDB-User-Group/events/107557862/

so it would be great to get some pointers before they realise I'm a fool ;-)

O!

[Mina Mikhail]

I had to do this recently for an app I'm running (http://pegleg.it). I had a few small issues with getting the dump initially but it's actually pretty straightforward. You'll find the instructions in the top answer here:

http://stackoverflow.com/questions/11024888/is-there-a-simple-way-to-export-the-data-from-a-meteor-deployed-app

The key trick not mentioned there is that you have to put PASSWORD in quotes when using mongodump. This will pull down the database to your local project in a dump/ folder. You can then use mongorestore to restore it to your local DB (http://docs.mongodb.org/manual/reference/mongorestore/).

Hope this helps and gets to you on time.

[Oli Evans]

That worked, many thanks. 

I just got a dump of my live app along to the intro to Faster Pussy Cat! Kill! Kill! thanks to pegleg.it, a great idea really well executed. When the inevitable and pointless "but can I use it for my production app" questions come up I'm gonna point them to pegleg.

O!

[Oli Evans]

One good turn deserves another. I got bored of copying the password around before it expires so now there is a script:

https://gist.github.com/olizilla/5209369

[Oli Evans]

Oh god. I need to drink more coffee, there is like 10 scripts on that SO answer already. I shall get back to my cave at once.

[Mina Mikhail]

Don't think any of them have that great header & documentation though, so I think you're leading the pack...

[Oli Evans]

Now asciified banners I can help with.

https://npmjs.org/package/asciify

[Mina Mikhail]

That's awesome! Will be using that everywhere now. Wonder if it would be preserved in an FB status...

[steeve]

As a side note, it is super easy to clone your db into mongohq.com from meteor.com

Just create db on mongohq and select import from external

Use 

  ~/meteor/meteor mongo -U yoapp  

to get your url.

c&p into mongohq web interface.

destroy if you want

click button & done

u can then backup to S3 or still use the mongo dump command 

having a copy in your S3 is nice tho

On Tuesday, March 19, 2013 5:48:13 AM UTC-4, Oli Evans wrote:

[Oli Evans]

@steeve, That is good to know. Thanks. Sorry I didn't reply, I got stuck in the gravity well of "fear of public speaking", for a intro to Meteor talk for the London MongoDb User group, but it's done now so I can now function as a normal human again.

Slides are here: http://the-medium-is-the-message.meteor.com/

@Mina, you might like to take a look, pegleg.it gets a shout out.

O!

[Stephen Cannon]

Nice presentation.  Is a meteor app too! What no kiwi on the graph!

--
You received this message because you are subscribed to a topic in the Google Groups "meteor-talk" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/meteor-talk/KvDWrqZpy-w/unsubscribe?hl=en.
To unsubscribe from this group and all its topics, send an email to meteor-talk...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
Thanks
Stephen Cannon

Mobile: 727.386.6298
stephen...@gmail.com

[steeve]

Oli,

Your script works awesome.  I used it almost daily to pull production databases for local dev and testing.

I was wondering if you would take a crack at a restore script that pushes to meteor.com.

I am not a script wizard otherwise I would do it.

I find myself regularly using your script to dump prod dbs for local dev testing and then manually using mongorestore to push to testing and staging installs so I can test dev changes in the wild prior to deploying in prod.

Doing it manually isn't that difficult but a script would be rad.

I pushed a prod db that i dumped locally into a staged meteor.com app by and posted my findings here.

See https://groups.google.com/forum/#!topic/meteor-talk/OCSPUV8PnlY

or

http://stackoverflow.com/questions/14687404/restore-a-mongodb-in-meteor-production-server/15865565#15865565

Thanks

Steve

[George Kellerman]

Oli/Steve,

I have been working on an extension of the script that redeploys an app instance to a new dev meteor instance and restores the dumped mongoDB into the dB for the dev instance.

I am using google auth and need to provide a new clientId and secret for the new instance. I am having an issue where I want to prompt the terminal for some user input using read -p 'clientId:' clientId ... but I am getting "read error: 0: Resource temporarily unavailable"

I debugged this issue back to where this is called in the original script:

     # Get the mongo url for your meteor app

      meteor mongo $METEOR_DOMAIN --url | tee "${TMP_FILE}"

if I use read before I get to that point the read prompt works, otherwise I get the cited error...

Any thoughts?

[Stephen Cannon]

George,

Wish I could help unfortunately I am not that familiar with this script.  

Thanks

Steve

--
You received this message because you are subscribed to a topic in the Google Groups "meteor-talk" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/meteor-talk/KvDWrqZpy-w/unsubscribe?hl=en.
To unsubscribe from this group and all its topics, send an email to meteor-talk...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Oli Evans]

Post some code if you can, it's hard to debug prose.

The `meteor mongo` line at: https://gist.github.com/olizilla/5209369#file-meteor-dump-sh-L56

has to let the meteor command prompt you for input, so you can enter your password, and also capture the output of the command for processing later.

I found a bunch of super complex ways to do it, but piping the output to tee, and storing the url in a text file was just way easier.

I have no idea why calling read after tee would cause an error, as I'd assume once that line is finished executing and the tmp file is written, stdin should be available.

[George Kellerman]

here's the Gist: https://gist.github.com/funkyeah/5744652

lines 48->56 are there because I need the read commands above the tee command I mentioned...

but yeah, crazy issue I just couldn't get past or make sense of.

[Ryan O'Toole]

Steeve,

I need to do exactly this, but my app is hosted on AWS not Meteor.com. When I run:

meteor mongo -U myapp

It outputs a connection string for:

mongodb://client:xxxx-xxxx-xxxx@production-db-b2.meteor.io:27017/meteor_meteor_com

Since my app isn't deployed on Meteor, I don't see that this URL would work. Is there a way to get the same connection URL for AWS? 

Ryan

[steeve]

Ryan

meteor mongo is helper script specific to applications deployed on meteor.com or on your dev env when you are running an app via meteor run.  It allows you to connect locally from the directory you are running in via meteor mongo 

If your app is on AWS you must have released it via bundle and running it as a node application?

If so you must have passed the mongo url as an environment variable?

http://docs.meteor.com/#deploying

So, you wouldn't use meteor mongo command but just use the regular mongo cmd line tools that ship with it.  

Or are you deployed on AWS as just a meteor app and you didn't bundle it and are just running it?

If so then you just go into app dir and mongocommand --host 127.0.0.1:3002 --db meteor

So, something like this to restore and use mongodump to dump

mongorestore --host 127.0.0.1:3002 --db meteor /home/user/mongodumps/dump/some_db
mongodump --host 127.0.0.1:3002 --db meteor

Though, hopefully you bundled and deployed on AWS.  Which if you did just connect to where ever you put your mongo installation, either on that machine, another machine or someplace like mongohq/lab.

[Ryan O'Toole]

Steeve,

My app is not bundled, I've just been developing and running it on an EC2 instance. After starting down the path you suggested, I found this MongoHQ FAQ page about importing from a localhost, which elaborates a bit further:

http://support.mongohq.com/importing-exporting/local-to-mongohq.html

The only gotcha was I also needed to install the MongoDB package for Ubuntu, in order to have the mongodump binary available:

# sudo apt-get install mongodb

After that, it all worked like a charm. Thanks for the pointers!

Ryan

[Alexey Komissarouk]

Oli - to your original question, you could always do 

> MONGO_URL=`meteor mongo yourapp.meteor.com` meteor.

I ended up submitting (and getting accepting) a patch to Meteor that deals with the tee issue.

https://github.com/AlexeyMK/meteor-download for a script that helps download prod db to local.

(Sorry to respond to an old thread, but it's probably worth it for folks googling for the question).