Reingest problem

[Gessy Junior]

Hi,

I installed the 1.5.1 version of CentOS and I performed a tranfer and when I try reingest an AIP the following error occurs:

Error re-ingesting package: Error from pipeline: Error in approve reingest API.

looking on  dashboard.log:

ERROR     2016-09-12 15:24:20  archivematica.common:storageService:request_reingest:374:  Unable to reingest e670abe2-f25e-4336-abee-c70e6dd2f89a

Traceback (most recent call last):

  File "/usr/lib/archivematica/archivematicaCommon/storageService.py", line 372, in request_reingest

    response = api.file(package_uuid).reingest.post(api_request)

  File "/usr/lib/python2.7/archivematica/dashboard/lib/python2.7/site-packages/slumber/__init__.py", line 146, in post

    resp = self._request("POST", data=data, files=files, params=kwargs)

  File "/usr/lib/python2.7/archivematica/dashboard/lib/python2.7/site-packages/slumber/__init__.py", line 110, in _request

    raise exceptions.HttpServerError("Server Error %s: %s" % (resp.status_code, url), response=resp, content=resp.content)

HttpServerError: Server Error 502: https://tst03.lcc.ufmg.br:8001/api/v2/file/e670abe2-f25e-4336-abee-c70e6dd2f89a/reingest/

INFO      2016-09-12 15:24:20  archivematica.common:storageService:get_file_info:314:  Files returned: []

Has anyone had this error?

Thanks a lot

Gessy

[Holly Becker]

Hi Gessy,

There's a couple of things to check for problems with reingest.  Since it requires communication both directions between Archivematica and the Storage Service, both sides need to know the hostname/IP and API key of the other.

In Archivematica, under Administration -> REST API check that the Storage Service IP or hostname is listed.  This should be the same URL you used to set the location of the storage service.  You can also check that the user and API key are listed under General, but since you were able to start a Transfer I assume that's set up correctly.

In the Storage Service, under Pipelines, verify that the entry for your pipeline has remote name, API username and API key for the Archivematica pipeline populated.  The remote name should not include http:// - if the URL is http://archivematica.example.com, then you want archivematica.example.com

If that is correct, are there any logs in the storage service log at /var/log/archivematica/storage-service/storage_service_debug.log that look related?  Are there any logs in the dashboard log before the error log you posted?

Cheers,
-Holly

[Gessy Junior]

Hello Holly backer

Thank you for explanations

I checked the information and seem to be ok. I put the addresses in both IP and FQDN REST API and storage service Pipeline I was using 127.0.0.1 as they are on the same server but changed to the real IP and also the FQDN and the result was the same.

The log that appears is the following:

...

...

INFO      2016-09-27 04:18:31  locations.models.space:space:move_rsync:381:  Moving from /var/archivematica/storage_service/tmpxzXUXc/Test_13-805990e0-8ad3-4276-a7f7-aaa0ab13285a/data/objects/evelyn_s_photo-27a0470b-e79b-41e8-8fa8-081689eca4d2.tif to /var/archivematica/sharedDirectory/tmp/tmpxzXUXc/Test_13-805990e0-8ad3-4276-a7f7-aaa0ab13285a/data/objects/evelyn_s_photo-27a0470b-e79b-41e8-8fa8-081689eca4d2.tif

DEBUG     2016-09-27 04:18:31  locations.models.space:space:move_to_storage_service:225:  TO: src: var/archivematica/storage_service/tmpxzXUXc/Test_13-805990e0-8ad3-4276-a7f7-aaa0ab13285a/data/objects/evelyn_s_photo.jpg

DEBUG     2016-09-27 04:18:31  locations.models.space:space:move_to_storage_service:226:  TO: dst: tmpxzXUXc/Test_13-805990e0-8ad3-4276-a7f7-aaa0ab13285a/data/objects/evelyn_s_photo.jpg

DEBUG     2016-09-27 04:18:31  locations.models.space:space:move_to_storage_service:227:  TO: staging: /var/archivematica/storage_service

INFO      2016-09-27 04:18:31  locations.models.space:space:move_rsync:381:  Moving from /var/archivematica/storage_service/tmpxzXUXc/Test_13-805990e0-8ad3-4276-a7f7-aaa0ab13285a/data/objects/evelyn_s_photo.jpg to /var/archivematica/storage_service/tmpxzXUXc/Test_13-805990e0-8ad3-4276-a7f7-aaa0ab13285a/data/objects/evelyn_s_photo.jpg

DEBUG     2016-09-27 04:18:31  locations.models.space:space:move_from_storage_service:294:  FROM: src: tmpxzXUXc/Test_13-805990e0-8ad3-4276-a7f7-aaa0ab13285a/data/objects/evelyn_s_photo.jpg

DEBUG     2016-09-27 04:18:31  locations.models.space:space:move_from_storage_service:295:  FROM: dst: var/archivematica/sharedDirectory/tmp/tmpxzXUXc/Test_13-805990e0-8ad3-4276-a7f7-aaa0ab13285a/data/objects/evelyn_s_photo.jpg

INFO      2016-09-27 04:18:31  locations.models.space:space:move_rsync:381:  Moving from /var/archivematica/storage_service/tmpxzXUXc/Test_13-805990e0-8ad3-4276-a7f7-aaa0ab13285a/data/objects/evelyn_s_photo.jpg to /var/archivematica/sharedDirectory/tmp/tmpxzXUXc/Test_13-805990e0-8ad3-4276-a7f7-aaa0ab13285a/data/objects/evelyn_s_photo.jpg

DEBUG     2016-09-27 04:18:31  locations.models.package:package:start_reingest:979:  Reingest: URL: http://xxx.xxx.ufmg.br/api/ingest/reingest; params: {'username': u'storageuser', 'api_key': u'f186b630f5fed6c25eb19e5d71eaefea8d7d242d', 'name': u'tmpxzXUXc/Test_13-805990e0-8ad3-4276-a7f7-aaa0ab13285a', 'uuid': u'805990e0-8ad3-4276-a7f7-aaa0ab13285a'}

DEBUG     2016-09-27 04:18:31  locations.models.package:package:start_reingest:986:  Response: 405 

ERROR     2016-09-27 04:18:31  locations.models.package:package:start_reingest:993:  Error in approve reingest API.

On painel appers only the message Error re-ingesting package: Error from pipeline: Error in approve reingest API.

I'll reinstall from scratch because I'm probably missing something I'm not getting realize now ...

But Thanks for everything

Gessy

[Gessy Junior]

Hi,

Looking on browser's console i found the message:

Failed to clear temp storage: It was determined that certain files are unsafe for access within a Web application, or that too many calls are being made on file resources. SecurityError

Does this indicate the cause of the problem?

Thanks a lot!

Gessy

[Holly Becker]

Hi Gessy,

It's possible the browser console log is related, but I'm not sure what it indicates.  The logs you posted show a HTTP 405 error, which happens when the HTTP request type (GET, POST etc) isn't valid for that URL. This make me think there's a version mismatch between the storage service and the pipeline.  However, the only packaged versions for CentOS are Archivematica 1.5.1 and Storage Service 0.9.1, which work together.  It might help to restart the services to ensure the newest installed version is running.  Restart services here https://www.archivematica.org/en/docs/archivematica-1.5/admin-manual/installation/installation/ has instructions.

Cheers,
-Holly

[Kirsty Lee]

We're also encountering this issue. Was a resolution reached at all as this would greatly help us? Was a reinstallation necessary?

Kirsty

[Gessy Junior]

Kirsty Lee,

I reinstalled the whole environment and realize that rsync setup is needed to complete dip upload process... 

The metadata is passed to Atom's endpoint via REST API but the DIP package need rsync  to be transfered to the Atom's server.

After I  set that up, everything worked.