Backup metadata cleanup

[husayt]

We have lot's of temporary entities in Datastore generated by backup task over the time and want to clean them up.

Which ones are ok to clean?

_AE_Backup_Information_Kind_Files

_AE_Backup_Information

_AE_DatastoreAdmin_Operation

_AE_Backup_Information_Kind_Type_Info

_GAE_MR_TaskPayload

_GAE_MR_MapreduceState

_GAE_MR_ShardState

Maybe there is already in place for cleanup? I guess this will be useful to most here, as currently this multiGB  backup metadata accounts for most of our data storage usage.

Thanks

HG

[Zeehad (Cloud Platform Support)]

Hello,

You can delete any of these entities if there aren't any Backup or Mapreduce jobs on your task queue. 

First use the Datastore Admin to delete all but the following kinds:

_AE_DatastoreAdmin_Operation
_GAE_MR_MapreduceState
_GAE_MR_ShardState
_GAE_MR_TaskPayload

After the delete is finished, you can manually delete entities in these kinds via the Datastore Viewer. These kinds are used to keep track of the delete process itself, and while many of these entities are removed automatically upon successful completion, some are expected to remain.

I hope it helps. Cheers!

[husayt]

Thank you very much Zeehan for useful reply.

As backups happen everyday and I have 100s of namespaces, hence it is problematic to delete manually. I am trying to automate this. Maybe you can suggest something?

And what about these entities :

_AE_Backup_Information_Kind_Files

_AE_Backup_Information

_AE_Backup_Information_Kind_Type_Info

Thanks,

HG

[Zeehad (Cloud Platform Support)]

Deleting entities using Datastore Admin is currently the fastest way to bulk-delete data. The entities _AE_Backup_Information_Kind_Files,  _AE_Backup_Information and _AE_Backup_Information_Kind_Type_Info can also be deleted this way.