Reingest feature error in 1.5 upgrade

[davihart]

We recently upgraded our archivematica demo VM from 1.4 to 1.5 with the .8 version storage service.  When we used the dashboard to try re-ingesting an existing AIP it came back with "An unknown error has occurred".  A similar attempt from the storage service came back "Whoops".  The storage service error log contained the following:

DEBUG     2016-06-27 12:43:25  locations.models.space:space:move_to_storage_service:221:  TO: src: var/archivematica/sharedDirectory/www/AIPsStore/54a2/241a/4a8c/4f90/9e0b/aede/d52f/4be2/battalion_1894-01-01-54a2241a-4a8c-4f90-9e0b-aeded52f4be2/data/objects/metadata/

DEBUG     2016-06-27 12:43:25  locations.models.space:space:move_to_storage_service:222:  TO: dst: 54a2/241a/4a8c/4f90/9e0b/aede/d52f/4be2/battalion_1894-01-01-54a2241a-4a8c-4f90-9e0b-aeded52f4be2/data/objects/metadata/

DEBUG     2016-06-27 12:43:25  locations.models.space:space:move_to_storage_service:223:  TO: staging: /var/archivematica/storage_service

INFO      2016-06-27 12:43:25  locations.models.space:space:move_rsync:375:  Rsyncing from /var/archivematica/sharedDirectory/www/AIPsStore/54a2/241a/4a8c/4f90/9e0b/aede/d52f/4be2/battalion_1894-01-01-54a2241a-4a8c-4f90-9e0b-aeded52f4be2/data/objects/metadata/ to /var/archivematica/storage_service/54a2/241a/4a8c/4f90/9e0b/aede/d52f/4be2/battalion_1894-01-01-54a2241a-4a8c-4f90-9e0b-aeded52f4be2/data/objects/metadata/

INFO      2016-06-27 12:43:25  locations.models.space:space:move_rsync:383:  rsync command: ['rsync', '-t', '-O', '--protect-args', '-vv', '--chmod=ugo+rw', '-r', '/var/archivematica/sharedDirectory/www/AIPsStore/54a2/241a/4a8c/4f90/9e0b/aede/d52f/4be2/battalion_1894-01-01-54a2241a-4a8c-4f90-9e0b-aeded52f4be2/data/objects/metadata/', '/var/archivematica/storage_service/54a2/241a/4a8c/4f90/9e0b/aede/d52f/4be2/battalion_1894-01-01-54a2241a-4a8c-4f90-9e0b-aeded52f4be2/data/objects/metadata/']

DEBUG     2016-06-27 12:43:25  locations.models.space:space:move_from_storage_service:290:  FROM: src: 54a2/241a/4a8c/4f90/9e0b/aede/d52f/4be2/battalion_1894-01-01-54a2241a-4a8c-4f90-9e0b-aeded52f4be2/data/objects/metadata/

DEBUG     2016-06-27 12:43:25  locations.models.space:space:move_from_storage_service:291:  FROM: dst: /var/archivematica/sharedDirectory/tmp/54a2/241a/4a8c/4f90/9e0b/aede/d52f/4be2/battalion_1894-01-01-54a2241a-4a8c-4f90-9e0b-aeded52f4be2/data/objects/metadata/

INFO      2016-06-27 12:43:25  locations.models.space:space:move_rsync:375:  Rsyncing from /var/archivematica/storage_service/54a2/241a/4a8c/4f90/9e0b/aede/d52f/4be2/battalion_1894-01-01-54a2241a-4a8c-4f90-9e0b-aeded52f4be2/data/objects/metadata// to /var/archivematica/sharedDirectory/tmp/54a2/241a/4a8c/4f90/9e0b/aede/d52f/4be2/battalion_1894-01-01-54a2241a-4a8c-4f90-9e0b-aeded52f4be2/data/objects/metadata/

INFO      2016-06-27 12:43:25  locations.models.space:space:move_rsync:383:  rsync command: ['rsync', '-t', '-O', '--protect-args', '-vv', '--chmod=ugo+rw', '-r', '/var/archivematica/storage_service/54a2/241a/4a8c/4f90/9e0b/aede/d52f/4be2/battalion_1894-01-01-54a2241a-4a8c-4f90-9e0b-aeded52f4be2/data/objects/metadata//', '/var/archivematica/sharedDirectory/tmp/54a2/241a/4a8c/4f90/9e0b/aede/d52f/4be2/battalion_1894-01-01-54a2241a-4a8c-4f90-9e0b-aeded52f4be2/data/objects/metadata/']

ERROR     2016-06-27 12:43:25  django.request.tastypie:resources:_handle_500:301:  Internal Server Error: /api/v2/file/54a2241a-4a8c-4f90-9e0b-aeded52f4be2/reingest/

Traceback (most recent call last):

  File "/usr/share/python/archivematica-storage-service/local/lib/python2.7/site-packages/tastypie/resources.py", line 220, in wrapper

    response = callback(request, *args, **kwargs)

  File "./locations/api/resources.py", line 91, in wrapper

    result = func(resource, request, bundle, **kwargs)

  File "./locations/api/resources.py", line 793, in reingest_request

    response = bundle.obj.start_reingest(pipeline, reingest_type)

  File "./locations/models/package.py", line 925, in start_reingest

    reingest_url = 'http://' + pipeline.remote_name + '/api/ingest/reingest'

TypeError: cannot concatenate 'str' and 'NoneType' objects

INFO      2016-06-27 12:43:40  locations.models.package:package:compress_package:572:  Compressing package with: ['tar', 'c', '', '-C', u'/var/archivematica/sharedDirectory/www/AIPsStore/54a2/241a/4a8c/4f90/9e0b/aede/d52f/4be2', '-f', u'/var/archivematica/storage_service/tmpkySlsh/battalion_1894-01-01-54a2241a-4a8c-4f90-9e0b-aeded52f4be2.tar', u'battalion_1894-01-01-54a2241a-4a8c-4f90-9e0b-aeded52f4be2'] to /var/archivematica/storage_service/tmpkySlsh/battalion_1894-01-01-54a2241a-4a8c-4f90-9e0b-aeded52f4be2.tar

DEBUG     2016-06-27 12:43:40  locations.models.package:package:compress_package:574:  Extract file RC: 2

Any ideas what happened?  Thanks

David Hart | Systems Administrator

Digital Initiatives | Texas A&M University Libraries

[Sarah Romkey]

Hi David,

I wonder if you are experiencing this permissions problem which has been reported- there is a workaround described in this ticket if you would like to try to resolve it that way:

https://projects.artefactual.com/issues/10031

Let us know if that does the trick.

Cheers,

Sarah

Sarah Romkey, MAS,MLIS

Archivematica Program Manager
Artefactual Systems
604-527-2056

@archivematica / @accesstomemory

--
You received this message because you are subscribed to the Google Groups "archivematica" group.
To unsubscribe from this group and stop receiving emails from it, send an email to archivematic...@googlegroups.com.
To post to this group, send email to archiv...@googlegroups.com.
Visit this group at https://groups.google.com/group/archivematica.
For more options, visit https://groups.google.com/d/optout.

[davihart]

It appears from the storage service error log that in the file package.py line 925 there is an attempt to concatenate the strings "http://", whatever is in pipeline.remote.name and '/api/injest/reingest'.  since two of those are strings and one is a variable apparently the variable was not assigned correctly and so has type "NoneType" which cannot be contactenated to the other two strings.  Can one of your developers look at this?

Thanks

[Holly Becker]

Hi David,

It looks like the storage service isn't configured to know where the pipeline is.  Since reingest requires both the storage service and Archivematica to initiate communication to the other, the storage service has to know the pipeline's hostname.

In the storage service, go to the Pipelines tab, select the problematic pipeline, and edit it.  The "Remote name" field should be the Archivematica pipeline's hostname. This could be an IP address (eg 127.0.0.1 or 198.199.119.37) or a hostname (eg localhost or sandbox.archivematica.org) and should include a port if any (eg localhost:8000).  If you're unsure, go to Archivematica and copy everything in the URL between the https:// and the first /

Cheers,
-Holly
Archivematica Developer

[davihart]

Thanks Holly that worked but now I am getting a new error

ERROR     2016-07-11 14:39:10  locations.models.package:package:start_reingest:946:  Error in approve reingest API.

in the storage-service log.  Any idea what this is?

On Monday, June 27, 2016 at 3:26:42 PM UTC-5, davihart wrote:

[Holly Becker]

Hi David,

Hard to say without more context.  What do the logs before & after that say in the storage-service log? Are there more details in the dashboard log?  Since reingest touches both the storage service and the dashboard, either place might have more information.  Also, where are the reingested AIP files? Have they made it to the dashboard, either /var/archivematica/sharedDirectory/tmp/ or /var/archivematica/sharedDirectory/watchedDirectories/system/reingestAIP/ ?

Cheers,
-Holly