AIP reingest error & issues
176 views
Skip to first unread message
Elie Abu Haydar
Jun 14, 2016, 7:16:37 AM6/14/16
to archivematica
Archivematica version 1.5.0 (upgraded from 1.4.1 which is in turn upgraded from 1.3.2)
1- Permission denied, archivematica user seems to need sufficient permissions to read and write to the following directory and sub directories ( sorry i dont remember the problematic permissions). Setting the below permissions solves the issue:
chmod 2770 /var/archivematica/sharedDirectory/watchedDirectories/system/reingestAIP
2- Reingest results in wrong aip file size displayed in archive storage tab. Very close to zero. This does not reflect the actual aip file size.
3- Reingest results in a wrong number of files displayed in the files columns in the archive storage tab.
For example, a sip with one file to be normalized ingested the first time would display 1 file as a label in the files column. Reingest this aip, it will display 2 files as a label. It will keep on adding the files on every reingest.
Elie Abu Haydar
Sarah Romkey
Jun 14, 2016, 2:37:36 PM6/14/16
to archiv...@googlegroups.com
Hello Elie,
Thanks so much for reporting these issues to the list.Regarding your upgrade from 1.3.2 to 1.4.1 to 1.5, did you upgrade elasticsearch during your upgrades?
Cheers,
Sarah
--
You received this message because you are subscribed to the Google Groups "archivematica" group.
To unsubscribe from this group and stop receiving emails from it, send an email to archivematic...@googlegroups.com.
To post to this group, send email to archiv...@googlegroups.com.
Visit this group at https://groups.google.com/group/archivematica.
For more options, visit https://groups.google.com/d/optout.
Elie Abu Haydar
Jun 15, 2016, 1:59:28 AM6/15/16
to archivematica
Hi Sarah,
Please find my comments below:
On Tuesday, June 14, 2016 at 9:37:36 PM UTC+3, Sarah Romkey wrote:
1. What error did you receive that prompted you to do this fix? Was it by chance: "Error re-ingesting package: Error from pipeline: Error in approve reingest API."Hello Elie,Thanks so much for reporting these issues to the list.
I faced several errors when i tried it, and i do remember one of them was "Error in approve reingest API", but they were due to wrong configuration from my side. (Wrong dashboard url, missing/wrong port, no configured whitelist ips etc). I can't really remember exactly what error was displayed for the "Permission denied" one but luckily the logs are still there:
DEBUG 2016-06-09 10:11:23 archivematica.dashboard:views:start_reingest:426: Reingest moving from /var/archivematica/sharedDirectory/tmp/tmpUIk2cu/similar_videos-fc428e3e-3894-4767-9c7e-7ba0be736903 to /var/archivematica/sharedDirectory/watchedDirectories/system/reingestAIP/
WARNING 2016-06-09 10:11:23 archivematica.dashboard:views:start_reingest:430: Unable to move reingested AIP to start reingest
Traceback (most recent call last):
File "/usr/share/archivematica/dashboard/components/api/views.py", line 427, in start_reingest
shutil.move(source, dest)
File "/usr/lib/python2.7/shutil.py", line 299, in move
copytree(src, real_dst, symlinks=True)
File "/usr/lib/python2.7/shutil.py", line 177, in copytree
os.makedirs(dst)
File "/usr/lib/python2.7/os.py", line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/var/archivematica/sharedDirectory/watchedDirectories/system/reingestAIP/similar_videos-fc428e3e-3894-4767-9c7e-7ba0be736903'
ERROR 2016-06-09 10:11:23 archivematica.common:storageService:request_reingest:374: Unable to reingest fc428e3e-3894-4767-9c7e-7ba0be736903
2. I haven't been able to reproduce this error. Did you download the AIP after reingest and confirm that the contents are actually there?
yes.
In case it becomes relevant, did you reingest with metadata and objects, or metadata only?
I tested with metadata only and confirmed the change in metadata as well as the recorded re-ingestion events in the mets file generated in the new bag.
3. This error I was able to reproduce- we believe it is an error in the elasticsearch index. I have created this issue ticket: https://projects.artefactual.com/issues/10022 .
great thanks
We're not sure yet if we will be doing a point release for bug fixes or just focus releasing 1.6 sooner. We'll keep in touch with the community as plans progress.Regarding your upgrade from 1.3.2 to 1.4.1 to 1.5, did you upgrade elasticsearch during your upgrades?
Yes, twice, one during 1.4.1 upgrade and one during 1.5. My current elasticsearch version is: 1.7.5
thanks a lot.
Sarah Romkey
Jun 15, 2016, 1:52:22 PM6/15/16
to archiv...@googlegroups.com
Hi Elie,
Thanks for this additional information- we have now filed three tickets in relation to the three issues you found:
Julie Allinson
Jun 16, 2016, 8:07:40 AM6/16/16
to archiv...@googlegroups.com
Hello,
Just to add another error to the pot. I have found that requesting a DIP reingest of an AIP that was extracted from a transferred zip file results in an error:Oxum error. Found 647 files and 4594929 bytes on disk; expected 647 files and 4595571 bytes.
Library & Archives
University of York
Heslington, York, YO10 5DD, UK
tel: ++44 (0) 1904 324083
skype: j.allinson | gtalk: julie.a...@york.ac.uk
University of York
Heslington, York, YO10 5DD, UK
tel: ++44 (0) 1904 324083
skype: j.allinson | gtalk: julie.a...@york.ac.uk
email disclaimer: http://www.york.ac.uk/docs/disclaimer/email.htm
Sarah Romkey
Jun 16, 2016, 10:49:09 AM6/16/16
to archiv...@googlegroups.com
Hi Julie,
Did you extract the contents and delete the zipped package when you first created the AIP, or did you extract the contents and not delete the zipped package?
Cheers,
Sarah
Julie Allinson
Jun 16, 2016, 11:02:03 AM6/16/16
to archiv...@googlegroups.com
Hi Sarah,
I set it up to extract and delete the package.Sarah Romkey
Jun 16, 2016, 11:56:24 AM6/16/16
to archiv...@googlegroups.com
Hi Julie,
I'm not able to reproduce this error. Are you using the stable release of 1.5? We fixed a bug regarding extracted packages near the end. Is there anything else notable about this AIP? If it's practical to do so, you can give me a copy of it off-list to test with.
Cheers,
Sarah
Sarah Romkey
Jun 17, 2016, 12:44:36 PM6/17/16
to archivematica
Hi everyone,
Julie provided a package to test with and I confirmed the same error although we're still not sure what's happening. Other AIPs containing zipped packages which were subsequently deleted are working. I also tried removing DS_Store files in case this is some kind of Mac thing, but that didn't clear the error either. Here is a public ticket for the issue:
Cheers,
Sarah
To unsubscribe from this group and stop receiving emails from it, send an email to archivematica+unsubscribe@googlegroups.com.
To post to this group, send email to archiv...@googlegroups.com.
Visit this group at https://groups.google.com/group/archivematica.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "archivematica" group.
To unsubscribe from this group and stop receiving emails from it, send an email to archivematica+unsubscribe@googlegroups.com.
To post to this group, send email to archiv...@googlegroups.com.
Visit this group at https://groups.google.com/group/archivematica.
For more options, visit https://groups.google.com/d/optout.
Technology Development Manager
Collections Services
Library & Archives
University of York
Heslington, York, YO10 5DD, UK
tel: ++44 (0) 1904 324083
skype: j.allinson | gtalk: julie.a...@york.ac.ukemail disclaimer: http://www.york.ac.uk/docs/disclaimer/email.htm
--
You received this message because you are subscribed to the Google Groups "archivematica" group.
To unsubscribe from this group and stop receiving emails from it, send an email to archivematica+unsubscribe@googlegroups.com.
To post to this group, send email to archiv...@googlegroups.com.
Visit this group at https://groups.google.com/group/archivematica.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "archivematica" group.
To unsubscribe from this group and stop receiving emails from it, send an email to archivematica+unsubscribe@googlegroups.com.
To post to this group, send email to archiv...@googlegroups.com.
Visit this group at https://groups.google.com/group/archivematica.
For more options, visit https://groups.google.com/d/optout.
Technology Development Manager
Collections Services
Library & Archives
University of York
Heslington, York, YO10 5DD, UK
tel: ++44 (0) 1904 324083
skype: j.allinson | gtalk: julie.a...@york.ac.ukemail disclaimer: http://www.york.ac.uk/docs/disclaimer/email.htm
--
You received this message because you are subscribed to the Google Groups "archivematica" group.
To unsubscribe from this group and stop receiving emails from it, send an email to archivematica+unsubscribe@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages