What happens when content is deleted?

[David Chandek-Stark]

We are considering implementing the "sync deletes" feature of the sync tool. However, we would like to be clear on exactly what happens when content is deleted this way.

- Does the Glacier copy remain for a period of time? If so, how long?

- If the deleted content was updated with "rename updates", are both the current and previous copies deleted?

Thanks,

David

[Bill Branan]

Hi David,

The Sync Deletes option on the SyncTool will seek to remove all files in DuraCloud which do not have a corresponding local file. It determines this by looking at each file in DuraCloud, and attempting to find a file with a matching path and name based on the set of content directories included in the tool. This means a few things:

- If you have content in a space, and your current run of the SyncTool does not include the content directory where those files originated, the files will be removed from DuraCloud

- If you've used the option to rename on update, all of those renamed files will be removed (since they don't have a corresponding local file with the same name) The SyncTool does not allow the Sync Deletes feature to be turned on at the same time as the rename updates feature for this reason.

I would be *very* wary about turning on Sync Deletes after you have already loaded content into a space, as you run the risk of that content being deleted if the SyncTool is not set up properly.

With regard to Glacier, there is no defined delay period for duplication. When content is added or removed from your primary store, it will immediately be placed on the queue for duplication. Depending on how large that queue is at the moment, it may take a little time before the duplication event is processed, but it may also happen immediately. I wouldn't depend on being able to get the data back from Glacier after you've deleted it from your primary.

I hope this helps.

Bill

--
You received this message because you are subscribed to the Google Groups "DuraCloud Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to duracloud-users+unsubscribe@googlegroups.com.
To post to this group, send email to duracloud-users@googlegroups.com.
Visit this group at https://groups.google.com/group/duracloud-users.
For more options, visit https://groups.google.com/d/optout.

[Dan Pritts]

Bill Branan

February 7, 2017 at 11:17 AM

I would be *very* wary about turning on Sync Deletes after you have already loaded content into a space, as you run the risk of that content being deleted if the SyncTool is not set up properly.

yes, yes, yes.

We had a scheduled outage of the campus NFS service a while ago.  That service holds our "master" that we push up to duracloud. 

We neglected to disable the sync tool, and when the master directory disappeared, it happily started deleting stuff.  It blasted a couple hundred thousand files before we (or maybe the duraspace folks) noticed. 

--
Dan Pritts

ICPSR Computing & Network Services

University of Michigan