Error DIP upload from Archivematica

[kj ti]

I have an Atom 2.7.1 installation which is used by the Archivematica 1.13.2 DIP Upload.
When the job is running in Atom I get following errors message in the Atom sever:

info] [2023-05-08 06:49:55] Job 486 "qtSwordPluginWorker": Job started.
[info] [2023-05-08 06:49:55] Job 486 "qtSwordPluginWorker": Parameters: {"format":"http:\/\/purl.org\/net\/sword-types\/METSArchivematicaDIP","location":"file:\/\/\/xxxx041_1-75dbd31c-4eb2-49ce-be14-f31d2b4e736b","suggested_name":null,"objectId":"480","name":"qtSwordPluginWorker","id":"486"}
[info] [2023-05-08 06:49:56] Job 486 "qtSwordPluginWorker": A package was deposited by reference.
[info] [2023-05-08 06:49:56] Job 486 "qtSwordPluginWorker": Location: file:///xxxx041_1-75dbd31c-4eb2-49ce-be14-f31d2b4e736b
[info] [2023-05-08 06:49:56] Job 486 "qtSwordPluginWorker": Processing...
[info] [2023-05-08 06:49:56] Job 486 "qtSwordPluginWorker": Object slug: xxxx04
[info] [2023-05-08 06:49:56] Job 486 "qtSwordPluginWorker": Processing complete, deleting "/atomtmp/xxxx041_1-75dbd31c-4eb2-49ce-be14-f31d2b4e736b"
[err] [2023-05-08 06:49:56] Job 486 "qtSwordPluginWorker": Exception: File not found: /atomtmp/xxxx41_1-75dbd31c-4eb2-49ce-be14-f31d2b4e736b
[err] [2023-05-08 06:49:56] Job 486 "qtSwordPluginWorker": File: /usr/share/nginx/atom/lib/AccessToMemory/Path.php
[err] [2023-05-08 06:49:56] Job 486 "qtSwordPluginWorker": Line: 152

The uploaded data are not visable in Atom.
I find the errormessage a little bit strange. The file is deleted in the loggentry before the exception occours.

[Dan Gillean]

Hi there, 

I may be wrong, but at first glance, I think there are two separate things occurring here. First, I suspect you may be encountering a known issue that was reported via the forum recently, and which our Maintainers have added to their backlog for review. It was reported in the following user forum thread that currently after a DIP upload, a reindex is required to make the digital objects visible. See: 

• https://groups.google.com/g/ica-atom-users/c/i_FsjHwkNZw/m/p-OZ8nblAgAJ
  

As such, I suggest that for now as a workaround, you try running the search:populate task to see if this helps - and for good measure (to make sure you are seeing the most recent versions of your AtoM pages), I would also suggest clearing the application cache and restarting PHP-FPM: 

• Repopulate the search index: php symfony search:populate
• See: https://www.accesstomemory.org/docs/latest/admin-manual/maintenance/populate-search-index/
• Clear the application cache: php symfony cc
• See: https://www.accesstomemory.org/docs/latest/admin-manual/maintenance/clear-cache/
• Restart PHP-FPM: sudo systemctl restart php7.4-fpm
• See: https://www.accesstomemory.org/docs/latest/admin-manual/maintenance/troubleshooting/#restarting-php-fpm

Hopefully that will allow your DIP objects to show up in AtoM while we sort out the underlying issue! 

The second issue seems to involve cleanup after processing the DIP upload. If I am reading this right (keep in mind I am not a developer or system administrator!), it looks like these two lines point to the problem: 

[info] [2023-05-08 06:49:56] Job 486 "qtSwordPluginWorker": Location: file:///xxxx041_1-75dbd31c-4eb2-49ce-be14-f31d2b4e736b

...

[err] [2023-05-08 06:49:56] Job 486 "qtSwordPluginWorker": Exception: File not found: /atomtmp/xxxx41_1-75dbd31c-4eb2-49ce-be14-f31d2b4e736b

It looks to me like the cleanup activity is not looking in the same place the files are actually being deposited during processing. I am not yet sure if this is a configuration issue, or a bug that was introduced with the fix added in 2.7 for cleaning up DIPs after upload, as described in this issue ticket: 

• https://projects.artefactual.com/issues/13396
  

Once again, I will pass this thread on to our Maintainers for their review. Thanks for letting us know - I will report back if I hear anything helpful soon! 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

he / him

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/305b10d9-b352-4775-873c-d3ca69836124n%40googlegroups.com.

[안대진]

I got the same problem.

I also have an Atom 2.7.1 installation which is used by the Archivematica 1.13.2 DIP Upload.

I'm curious what caused the problem.

-- Logs begin at Thu 2023-02-16 13:42:09 UTC. --

Jun 06 06:35:34 oss-2023-1 php7.4[1116]: 2023-06-05 23:35:34 > Parameters: {"format":"http:\/\/purl.org\/net\/sword-types\/METSArchivematicaDIP","location":"file:\/\/\/daejin0606-869fcfc5-0226-4a64-a82d-57df627dd1f5","suggested_name":null,"objectId":"19882","name":"qtSwordPluginWorker","id":"19883"}

Jun 06 06:35:34 oss-2023-1 php7.4[1116]: 2023-06-05 23:35:34 > A package was deposited by reference.

Jun 06 06:35:34 oss-2023-1 php7.4[1116]: 2023-06-05 23:35:34 > Location: file:///daejin0606-869fcfc5-0226-4a64-a82d-57df627dd1f5

Jun 06 06:35:34 oss-2023-1 php7.4[1116]: 2023-06-05 23:35:34 > Processing...

Jun 06 06:35:34 oss-2023-1 php7.4[1116]: 2023-06-05 23:35:34 > Object slug: dip

Jun 06 06:35:34 oss-2023-1 php7.4[1116]: 2023-06-05 23:35:34 > Processing complete, deleting "/tmp/daejin0606-869fcfc5-0226-4a64-a82d-57df627dd1f5"

Jun 06 06:35:34 oss-2023-1 php7.4[1116]: 2023-06-05 23:35:34 > Exception: File not found: /tmp/daejin0606-869fcfc5-0226-4a64-a82d-57df627dd1f5

Jun 06 06:35:34 oss-2023-1 php7.4[1116]: 2023-06-05 23:35:34 > File: /usr/share/nginx/atom/lib/AccessToMemory/Path.php

Jun 06 06:35:34 oss-2023-1 php7.4[1116]: 2023-06-05 23:35:34 > Line: 152

Jun 06 06:35:34 oss-2023-1 php7.4[1116]: 2023-06-05 23:35:34 > Jobs completed: 94

Jun 06 07:01:25 oss-2023-1 php7.4[1116]: 2023-06-06 00:01:25 > Job started.

Jun 06 07:01:25 oss-2023-1 php7.4[1116]: 2023-06-06 00:01:25 > Parameters: {"objectId":"18484","reportType":"fileList","reportTypeLabel":"File","sortBy":"referenceCode","reportFormat":"html","includeThumbnails":true,"name":"arGenerateReportJob","id":"19884"}

Jun 06 07:01:25 oss-2023-1 php7.4[1116]: 2023-06-06 00:01:25 > Job finished.

Jun 06 07:01:25 oss-2023-1 php7.4[1116]: 2023-06-06 00:01:25 > Jobs completed: 95

2023년 5월 17일 수요일 오후 10시 16분 38초 UTC+9에 Dan Gillean님이 작성:

[Dan Gillean]

Hi both, 

Sorry for the delay as I have tried to track down further information on this. 

It turns out that this problem may be due to a change in 2.7 that renders the /tmp directory unsuitable as a default deposit directory for SWORD, since the new DIP cleanup activity that will run (to ensure that your disk space is used by keeping temporary copies of the DIP after upload completes) cannot access this directory. 

See the following documentation links: 

• The NOTE admonition in this section: https://www.accesstomemory.org/docs/2.7/user-manual/administer/settings/#dip-upload-settings
• The bulk of what you need to change is listed here: https://www.accesstomemory.org/docs/2.7/user-manual/administer/settings/#sword-directory

Please try implementing these changes and let me know if it helps! 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

he / him

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/c24f7324-eec0-4b55-b357-f414452bf320n%40googlegroups.com.

[안대진]

I'll try and let you know. 

Thank you.

2023년 6월 8일 목요일 오후 11시 17분 39초 UTC+9에 Dan Gillean님이 작성:

[Kevin Dusenberry]

Ran into this same issue today while testing the archivematica to atom transfer process. I've attempted the solution that you linked to and it is still not working.

[Dan Gillean]

Hi Kevin, 

Thanks for this report. I'm not currently well set up to test with Archivematica, and am not actually focusing on AtoM projects right now in my daily work, so: would you be willing to file a bug report in AtoM's new GitHub issue tracker, so our Maintainers can try to reproduce this issue and hopefully address it in a future release?

• https://github.com/artefactual/atom/issues/new/choose
  

Thanks in advance! 

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

he / him

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/388a4764-700e-4863-b024-2bae8e50e77en%40googlegroups.com.