High level (parse server) database backup/restore

[Roman Kalisetskiy]

Hi Guys,

I'm seeing some strange behaviors with one of my containers.

At first, there is some inconsistence counting objects in classes,

for example, I have two objects in class Key but Parse dashboard shows me only one. Look at the screenshot below.

https://drive.google.com/a/artua.com/file/d/0B1iTxC4_xSVUYkgxeUhwZWVmUTQ

If I delete two, it shows me -1.

Next, it shows me 335MB of storage used:

https://drive.google.com/a/artua.com/file/d/0B1iTxC4_xSVUcnl1a0pCdHBqTDQ

But there is no so much data, if I download a backup, it is only 26MB unzipped.

I tried to recreate database by moving it to another container, then truncate then import back from backup container. But it seems do not help. It grows very quick regardless of real data size. I tried to remove class and recreate one, but still see more problems: parse dashboard said "can't delete class because it not empty" but shows no objects in this class. I removed it from actual Mongo DB, then I can delete and recreate one. 

I think some data corrupted in database, and I think the best way is to backup it "high level", from parse server side, then recreate database and import it. But seems we have no backup/restore tools for parse server.

Any other ideas why it is so and how to fix that?

Thanks!

[dev.no...@gmail.com]

Because your database instance is dedicated there is always a fixed 256 MB dedicated to the operations log. The operation is very important, if you want to roll the the database back to a point in time. We are releasing an automated version of this feature sometime this month. Also your backups are highly compressed using gzip without any indexes so obviously it will be significantly smaller than the actual size in the database. Also the storage is measured by the actual space allocated by the database engine not the size of the data in the database. So for example when you delete data unless compaction happens, that space is never released back to the operating system.

I have noticed this counter issue with the rocksdb engine when running MongoDB version 3.0.2 for my own test instance. It appears there is a possibility it can happen when the database crashes due to an out of memory exception. Typically Dropping the collection altogether and recreating and reloading the data fixes it. I will advice you upgrade to the version 3.4.4 which i noticed you have.

[Roman Kalisetskiy]

So do I understand right, it will grow to 256MB+actual data, then stop growing so much?

It's ok then.

And if I want to fix counters, I just need to repeat, import all data to another (test) database, then recreate this one, then import all data back to original database?

[dev.no...@gmail.com]

You can do it for just the class. We are releasing a tool this month which provides a one click functionality to restore any class or the entire database to a point in time. You can still do it with our CSV import tool. you do not have to truncate the entire database, it allows you to truncate the class first before import.

[dev.no...@gmail.com]

MongoDB point in time database restore is now available for the entire database as well as specific collections. It can be accessed under DB Actions > Backups & Restore