Recover deleted Logs | App Engine | Logs Storage | Default Bucket

[Br Secy Election.]

Hi All,

I created an App engine web app and made it available in June 2020, during June and July there were lots of hits and I added efficient logging in the code to track down issues.

I used to see the logs quite often, however today I realized that by default App Engine logs are stored in _Default (default bucket) and its Retention Period is 30 days only by default. 

So, all my old logs of June and July are lost and I am not able to access them any more.

I also learnt, one can change this retention period of _Default bucket at a cost, which I did now to 365 days. However, I really want to retrieve my old logs of past months, even if it comes at a cost.

Is there a way ? I can get those back... I tried looking at this : 

https://cloud.google.com/sdk/gcloud/reference/alpha/logging/buckets/undelete

but not much helpful as it is not working for default , always says wrong location.

Please advice / help. 

[Olu]

Hello, 

Thank you for reporting this. While there are a number of options available to retain logs that are yet to be deleted, for example, this article[1] shows which logs are configurable for Custom retention[2]. However, once the logs have been deleted, it becomes  difficult and seems impossible to restore.

The doc[3] you referenced indeed would not work for _Default Buckets since these cannot be deleted[4]. The "gcloud alpha logging buckets undelete" command would only undelete a bucket that can be and is already deleted. By using Custom logs Bucket and applying log sinks to the buckets, you would be able to manager better subsets of your logs to any logs bucket. Custom retention can be configured for Custom logs buckets as well. 

In your use case, however, the _Default buckets are used and the Data Access audit logs and other logs are only retained for 30 days and cannot be restored after deleted. 

 

[1]https://cloud.google.com/logging/quotas#logs_retention_periods 

[2]https://cloud.google.com/logging/docs/storage#gcloud

[3]https://cloud.google.com/sdk/gcloud/reference/alpha/logging/buckets/undelete

[4]https://cloud.google.com/logging/docs/storage#overview