firestore backup function permission issue

[Florian Geier]

Hi Firebase Users,

our project uses firestore and I want to set up a firebase function to backup data to a GCP bucket.
I set up a function that uses the firestoreAdminClient exportDocuments function, basically the same as described here https://levelup.gitconnected.com/how-to-back-up-firestore-easily-and-automatically-eab6bf0d7e1f
but get a PERMISSION_DENIED error ("The caller does not have permission").

According to this https://firebase.google.com/docs/firestore/manage-data/export-import#default_service_account_permissions guide, I was assuming that the service account in our project would already have the required permissions (it says "If your Cloud Storage bucket is in the same project as your Cloud Firestore database, then the default service account has access to the bucket by default."

What do I have to do to be able to backup my firestore data? I already tried explicitly giving the service account the "Storage Admin" role Do I have to set up an admin SDK private key? Or is there a better way to backup my data?

Thank you and best regards,
Florian

[Philip Carina]

There are two ways that I have used. One allows you to backup individual collections and the other backs up all collections and both backup to a storage bucket.

https://firebase.google.com/docs/firestore/manage-data/export-import discusses these methods in detail.

You can do it from the google cloud console as talked about. You can not specify anything other than all collections.

If you use gcloud (I used a gcloud shell) then you can specify which collections you need in the export.

If you want to import select collections then you must use this method, as the google cloud console method will only allow an import of all collections.

I hope this helps a bit.

Philip

[Florian Geier]

Hi Philip,

thanks for your help. I saw that in the firebase docs, but I were wondering: how would you set it up as a recurring, scheduled backup job? Did you by any chance do that?

Best regards,
Florian

[Philip Carina]

I have not, however, using scheduled cloud functions, you should be able to achieve this.

https://firebase.google.com/docs/firestore/solutions/schedule-export#firebase-cli

This will probably be the easiest version to set up. You can also click on the GCP Console tab in the documentation and look at this method, but I would try the firebase version first as it will almost certainly be easier for you to get to grips with. You can see the output logs in the firebase cloud function log function as well which is an easy way to get feedback. Just remember, that it can take a minute or two for the logs to be visible in the cloud function logs. They are not updated instantly.

Philip

[Florian Geier]

Hi Philip,

the issue is, I tried that. I set up a scheduled cloud function, but when it tries to access the bucket, it fails with a PERMISSION_DENIED error although the service user running the function is in the same project as the bucket.
I'm trying to find out why this error occurs.

Florian