Delayed data removal

[Alexander Obuhovich]

In-Portal offers these basic functions in every list of db records:

• view
• add
• edit
• delete

In this discussion we'll talk only about deleting.

Deleting might seem a trivial task, however it can create a lot of problems when:

• many records needs to be deleted at once (can result in out of memory problem)
• many related data (e.g. links in a categories, categories in a categories) exists, that also needs to be deleted along with deleted items

Instead of deleting records right now we need do:

• mark record as deleted
• don't show records marked as deleted
• safely delete records in a small portions via scheduled task

This not only creates an impression (to end user), that data was instantly deleted, but also allows to keep deleted data for later usage if scheduled task mentioned before will be disabled.

This is sort-of recycle bin in "Structure & Data" section, but it:

• keeps record where it was, just don't show it
• is system-wide

--
Best Regards,

http://www.in-portal.com
http://www.alex-time.com

[Dmitry A.]

Hi Alex,

Yes, I think we can definitely benefit from this so I support the idea.

May be it should be enabled only when needed - ie. large websites?

DA

[Phil]

interesting idea indeed. This new agent function looks like "empty trashbin after x days", it could be the right time to create a "real" trashbin feature, where we list all deleted items, and NOT item moved to this category...

Envoyé avec Sparrow

[Alexander Obuhovich]

Actually we implemented this idea several years ago in our issue tracker (one we use at Intechnic).

There it was done as a safety measure against accidental deletion. Then in each page with a list of items (only, when user had special permission) checkbox "Show Deleted" (unchecked by default) was displayed.

When enabled records, that were deleted before (in this list only) were displayed and even were available for editing.

Following this idea we can:

1. Create "Delete Button Behavior" setting with these values: "Delete immediately" / "Hide from list" / "Hide from list and delete later automatically"
2. Add "advanced:show_deleted" permission to each section and if user have it, then show "[ ] Show Deleted" checkbox above the lists. Clicking on a checkbox would toggle deleted record display.

Note that setting is called "Delete Button Behavior" and this means, that items won't be deleted instantly and will obey this setting only if user is deleting them via Front-End/Admin Console. If item is being deleted internally via direct call to kDBItem::Delete method, then of course it will be instantly deleted.

Delete Button Behavior in depth:

• Delete immediately - item is deleted instantly after "Delete" button in the list is pressed (as it works right now)
• Hide from list - will mark item as deleted and hide it from list (user with "advanced:show_deleted" in that list still will be able to see that item if "Show Deleted" checkbox is checked).
• Hide from list and delete later automatically - same as "Hide from list", but this time a scheduled task will delete records marked for deletion as it see fit.

What isn't obvious, but would be good to have too:

1. Restore of deleted items. When "Show Deleted" mode enabled item should be visible and by selecting it and pressing new "Restore" toolbar button (or "restore" link somewhere in that record) would revive it and all related records (e.g. images, custom data).
2. Prevent any operations on deleted item and force user to restore it first. Deleting item just to be able to modify it isn't good idea.
3. Mode, when you only see deleted items (not real + deleted as I've explained before). This is sort of "Recycle Bin" emulation for Windows. Converting "Show Deleted" checkbox into more complex control (like adding 2nd checkbox "Show Not Deleted" or even transforming both checkboxes into dropdown) isn't good idea.

[Dmitry A.]

Hi Alex,

Thanks for great outline!

I believe we are ready for at ask here. Would you please create one?

One additional clarification is needed. What would happen after Restore from Deleted with Data in some fields of Deleted Items (marked as deleted and restored later) that is dependent on the other records in the same or other tables? 

Good example would be Filename field which contains unique filename path generated automatically, so after we marked item as deleted next item can use the same filename automatically, but then in Restore original item will fail to have the same filename. In other words, how do we deal with data changes between time when Original Item marked as deleted and now (as of restore)?

DA

[Alexander Obuhovich]

Performing filename regeneration after restore is bad idea. That's why I believe, that we should include records "marked for deletion" in unique filename check.

This might seem odd at first, but would solve our restore problem completely.

[Dmitry A.]

Thanks for the inside and I agree with proposed solution.

Would you please create a new task for 5.3.0 for this?

DA