vault dynamodb backup and restore

[oded priva]

Hi, 

I'm looking for a process to backup vault dynamodb backend to s3 and restore it in case of a disaster. 

my current design : 

1. backup process that go over all documents in dynamo and creates objects in S3

2. dynamo triggered process that for each modify/insert/remove entry in dynamo table update the s3 object. 

my questions : 

1. is this method suppose to work ? 

2. is there best practice for this kind of use case ( that seems to me, very common use case ) 

3. documents in dynamo has similar struct, ( Key, Path, Value ) except for _lock records that has some binary value - what should I do with those ? Can i skip them and the process will still work? 

Any suggestion would be great . 

Thanks 

[Jeff Mitchell]

Hi Oded,

I'm not familiar with Dynamo backup/restore procedures, but generally speaking backup of Vault data is best done by atomic backup methods of the underlying data store. For instance, if running on Consul, taking frequent Consul snapshots is a good strategy since the snapshot is atomic, so any restore of the data should be consistent (in and of itself, although if Vault was in the middle of operations when the snapshot was taken there might be some minor inconsistency inside Vault -- similar to a file system snapshot).

Best,

Jeff

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/hashicorp/vault/issues
IRC: #vault-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Vault" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vault-tool+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/vault-tool/2e7822c4-b675-416c-8365-075e8c8ed8c9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Josh Smift]

AWS announced DynamoDB on-demand backups yesterday at re:Invent. :^) (https://aws.amazon.com/blogs/aws/new-for-amazon-dynamodb-global-tables-and-on-demand-backup/) They haven't shown up for our account yet, but we're looking forward to it when they do.

[Jeff Mitchell]

Perfect :-D

On Thu, Nov 30, 2017 at 10:10 AM, Josh Smift <jbs.care...@gmail.com> wrote:

AWS announced DynamoDB on-demand backups yesterday at re:Invent. :^) (https://aws.amazon.com/blogs/aws/new-for-amazon-dynamodb-global-tables-and-on-demand-backup/) They haven't shown up for our account yet, but we're looking forward to it when they do.

--

This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/hashicorp/vault/issues
IRC: #vault-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Vault" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vault-tool+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/vault-tool/657e53cd-3343-44bc-8325-3efbd223157d%40googlegroups.com.