DIP status not updating in Storage Service database

[Julie S]

Hello,

In our production instance, a DIP that was deleted from our backend S3 storage is still appearing as "Uploaded" and with a package size (128.6MB) in the Archivematica Storage Service database. Since the package is no longer in the backend, clicking "Delete" in the Storage Service interface only returns a "Package deletion failed" error. The associated AIP is also deleted from storage and its status in the Storage Service is "Deleted."

Has anyone else encountered this issue or know more about what's causing it? We've noted the API option to directly update the database, but would like to avoid making changes directly to our production instance if possible.

Any help is appreciated. Thank you!

Julie

Scholars Portal

[Joseph Anderson]

In my experience, you would need to delete the file initially through the storage service interface otherwise it causes this kind of error. And I believe the only way to correct it is to directly edit the mysql table for that entry. 

-Joseph

[Sarah Romkey]

I think Joseph is correct. There are commands for reindexing the AIP and transfer indexes (https://archivematica.org/en/docs/archivematica-1.13/admin-manual/maintenance/maintenance/#elasticsearch) but they would need tweaking to work with stored DIPs, because the METS is stored in a different path in the DIP than it is in the AIP. If you have are a developer/know a developer who could look into this for you, this would be a place to look: https://github.com/artefactual/archivematica/blob/stable/1.13.x/src/dashboard/src/main/management/commands/rebuild_aip_index_from_storage_service.py#L52 (Thank you to our developer Radda for helping me locate that).

Cheers,

Sarah

Sarah Romkey, MAS,MLIS

Archivematica Program Manager

She/her
Artefactual Systems
604-527-2056

@archivematica / @accesstomemory

--
You received this message because you are subscribed to the Google Groups "archivematica" group.
To unsubscribe from this group and stop receiving emails from it, send an email to archivematic...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/archivematica/c87d7195-b2c8-437b-805b-0d6bba0fd856n%40googlegroups.com.

[Julie S]

Hi both, thanks for your responses! Our systems team will look into the commands you shared, Sarah.

We have been using the storage service interface to delete the packages, but the status in the interface is still not updating from "Uploaded" to "Deleted". For the most recent AIP/DIP set that had this issue, I deleted the AIP first and then the DIP. Would you know if the order of deletion might be causing the issue or if there is something else we should look into? Any suggestions are helpful if we can avoid this in future!

Best,

Julie

[Julie S]

Hello,

As an update, we've determined that the Status field in the database updates successfully after restarting an instance, however we were unable to identify which service/s is/are the root cause of the issue. Does anyone have any thoughts on which service/s we should examine first? As well, we haven't been able to determine how many packages across our production instances are affected by this issue (we provide hosted Archivematica instances as part of our service). Any suggestions for how to determine if database entries are updated or not would be much appreciated!

Thank you!

For reference, this is what we're seeing:

• A DIP that is deleted but still appearing in the database as Uploaded. It downloads as a 0B file
  

• The Download dialog that appears when both the Current Location link and Download action are selected
  

• The error that appears in the Storage Service if we try to Delete it again
  

• Log indicating the file is already deleted from storage:

INFO2022-11-23 08:55:21 locations.models.s3:s3:delete_path:193: S3 path to delete [package location] begins with /; removing from path prior to deletion WARNING 2022-11-23 08:55:21 locations.models.s3:s3:delete_path:206: No packages found in S3 at: [package location] ERROR 2022-11-23 08:55:21 locations.views:views:package_delete:235: Package deletion failed: No packages found in S3 at: [package location]

[Joseph Anderson]

I imagine that you could write some sort of script that would get the s3 key of every dip in your storage service and then use to s3 directly verify if the dip exists or not.
.