Bagit Validation Error

[Hilary Wang]

Hello All, 

I'm seeking some advice/help/guidance on a bagging validation error I keep running into while moving bags to a network server

1. I have a hard drive with bagged folders (files were bagged on the drive)

2. When I --validate --fast the bags via terminal on the hard drive I receive a valid message

"C:\User...>bagit.py --validate --fast "filepath to bag on hard drive" 2023-01-10  09:33:27,276 - INFO - "filepath" valid according to Payload-Oxum" 

3. (where the error occurs for some of the bags) When I then copy the bag from the hard drive to our network server via TerraCopy (Windows) or rsync (Mac) and go to validate the bag on the server I get a manifest unsafe error for specific files. Example:

C:\Users..>bagit.py --validate --fast "filepath to bag on server"
2023-01-12 09:27:39,229 - ERROR - "filepath to bag on server" is invalid: Path "path to the specific file in bag" in manifest "filepath to\manifest-sha256.txt" is unsafe

if I copy the bag from the hard drive locally on my computer the validation is successful. I'm not sure if the error is a result of our IT security parameters for the network server. Has anyone run into this? Or can clarify what 'unsafe' means?

Thanks so much!

Hilary Wang

Digital Archivist

Brown University 

[Andreas Romeyke]

Hi,

you could revalidate the bag with https://metacpan.org/pod/Archive::BagIt additionally

I assume there is a problem with non-portable filenames in your bag. The tool above is very detailed to detect such errors.

Hope this helps,

With best regards

Andreas

--
You received this message because you are subscribed to the Google Groups "Digital Curation" group.
To unsubscribe from this group and stop receiving emails from it, send an email to digital-curati...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/digital-curation/fcfb66d4-14b1-4e93-9903-d8722fdf833dn%40googlegroups.com.

[Chris Adams]

Bagit-python generates that error here:

https://github.com/LibraryOfCongress/bagit-python/blob/4ef98ac491ef6f0b4c926575cd116effb5762112/bagit.py#L572

The function which checks for dangerous filenames is here:

https://github.com/LibraryOfCongress/bagit-python/blob/4ef98ac491ef6f0b4c926575cd116effb5762112/bagit.py#L926-L943

Based on your description, I suspect your manifest file might have a hard-coded path using something like a drive letter or relative path reference (e.g. “..\otherfile”) which triggers those warnings.

Chris

[Hilary Wang]

Thanks so much Andreas and Chris!