ATOM upload error

807 views
Skip to first unread message

BrzI Channel

unread,
Feb 8, 2019, 1:55:43 PM2/8/19
to archivematica
I am testing my new Archivematica/AtoM in an isolated environment.
My DIP upload is failing with these errors :

Module upload-qubit_v0.0

--url="http://xxx.xxx.xxx.xxx"  --email="ya...@yada.ca" --password="xxxxxxxxxxxxxx" --uuid="cbc8c620-b1f2-4841-9fee-f4a2f82fa3b6" --debug="True" --version="2" --rsync-command="" --rsync-target=""

Standard streams

Standard error (stderr)

upload-qubit_v0.0: ERROR     2019-02-07 23:58:24,489  archivematica.upload.qubit.log:64  [uploadDIP] Target: slug-test1
upload-qubit_v0.0: ERROR     2019-02-07 23:58:24,490  archivematica.upload.qubit.log:64  [uploadDIP] About to deposit to: http://xxx.xxx.xxx.xxx/sword/deposit/slug-test1
upload-qubit_v0.0: ERROR     2019-02-07 23:58:28,947  archivematica.upload.qubit.log:64  [uploadDIP] > Response code: 500
upload-qubit_v0.0: ERROR     2019-02-07 23:58:28,947  archivematica.upload.qubit.log:64  [uploadDIP] > Location: None
upload-qubit_v0.0: ERROR     2019-02-07 23:58:28,947  archivematica.upload.qubit.log:64  [uploadDIP] > Content received: <!DOCTYPE html>
<html>
  <head>
    <title>Error</title>
    <link rel="stylesheet" type="text/css" href="/plugins/arDominionPlugin/css/main.css"/>
  </head>
  <body class="yui-skin-sam admin error">

    <div id="wrapper" class="container">

      <section class="admin-message" id="error-404">

        <h2>
          <img src="/images/logo.png"/>
          Oops! An Error Occurred
        </h2>

        <p>
          Sorry, something went wrong.<br />
          The server returned a <code>500 Internal Server Error</code>.
        </p>

        <div class="tips">
          <p>
            Try again a little later or ask in the <a href="http://groups.google.ca/group/ica-atom-users">discussion group</a>.<br />
            <a href="javascript:history.go(-1)">Back to previous page.</a>
          </p>
        </div>

      </section>

  </body>
</html>

[uploadDIP] Response code not expected

What could be the problem ?

Thanks

Ashley Blewer

unread,
Feb 11, 2019, 12:29:48 PM2/11/19
to archiv...@googlegroups.com
Hi,

Here are some tips for debugging this issue:

- Make sure all of your configuration settings are set correctly in Archivematica
- "slug-test1" has to already exist in AtoM before uploading
- Check to make sure atom-worker is running on the AtoM machine
- You can look at past attempts at uploads in Archivematica's Access tab
- Check AtoM's logs for any error messages that might provide you with more clues as to the nature of the 500 error

Ashley

--
You received this message because you are subscribed to the Google Groups "archivematica" group.
To unsubscribe from this group and stop receiving emails from it, send an email to archivematic...@googlegroups.com.
To post to this group, send email to archiv...@googlegroups.com.
Visit this group at https://groups.google.com/group/archivematica.
For more options, visit https://groups.google.com/d/optout.


--
Ashley Blewer
AV Preservation Specialist
Artefactual Systems, Inc.

Dan Gillean

unread,
Feb 11, 2019, 12:34:51 PM2/11/19
to archiv...@googlegroups.com
On the AtoM side of things, I would also add: 
  • Make sure you have installed and properly configured the AtoM job scheduler - see the docs here
  • Make sure the job scheduler is running (the doc page linked above has commands to check the status, and restart the atom-worker if needed, for both Ubuntu 14.04 and 16.04 installations)
  • Make sure the qtSwordPlugin is enabled in AtoM in Admin > Plugins - see the docs here
  • If you are doing any arrangement in AM using the AtoM levels, you may also need to enable the arRestApiPlugin as well
Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056
@accesstomemory

BrzI Channel

unread,
Feb 11, 2019, 3:04:33 PM2/11/19
to archivematica
So I made these changes:
Expanded the privileges of the atom account include contributor, editor and translator on top of existing administrator and authenticated.
Made sure the target slug was published (from draft) and eliminated any warning messages.

Stopped getting the Upload DIP error when ingesting via the Archivematica dashboard. But have a newer issue now. Please see below.

innopac@ATOM221:/usr/share/nginx/atom$ sudo php symfony jobs:worker
[sudo] password for innopac:
2019-02-11 11:29:30 > New ability: arGenerateFindingAidJob
2019-02-11 11:29:30 > New ability: arInheritRightsJob
2019-02-11 11:29:30 > New ability: qtSwordPluginWorker
2019-02-11 11:29:30 > Running worker...
2019-02-11 11:29:30 > PID 2879
2019-02-11 11:55:48 > Job 89179 "qtSwordPluginWorker": A new job has started to being processed.
2019-02-11 11:55:48 > Job 89179 "qtSwordPluginWorker": A package was deposited by reference.
2019-02-11 11:55:48 > Job 89179 "qtSwordPluginWorker": Location: file:///Feb11_1-28e8cdf5-302e-45d5-8290-9ad80d58722d
2019-02-11 11:55:48 > Job 89179 "qtSwordPluginWorker": Processing...
2019-02-11 11:55:48 > Job 89179 "qtSwordPluginWorker": Object slug: goran-test1
2019-02-11 11:55:48 > Job 89179 "qtSwordPluginWorker": Exception: File/directory does not exist or is not readable: /tmp/Feb11_1-28e8cdf5-302e-45d5-8290-9ad80d58722d.
2019-02-11 11:55:48 > Job failed: File/directory does not exist or is not readable: /tmp/Feb11_1-28e8cdf5-302e-45d5-8290-9ad80d58722d.


I presumed the directory referred to was /usr/share/nginx/atom. So I created a new folder called tmp and assigned www:data as owner and group. Still getting this message . . .

What could be wrong ?

Dan Gillean

unread,
Feb 11, 2019, 4:10:15 PM2/11/19
to archiv...@googlegroups.com
Hi there, 

You're almost there! At this point, I think you have just encountered a known issue with a simple workaround. 

When first installed, AtoM does not include an uploads directory, where digital objects are stored. Normally, this directory is automatically created the first time a digital object is uploaded via AtoM's user interface. However, we have discovered that this automated process is not working when AtoM's first digital object is a DIP upload. See the following ticket: 
  • Manually create the uploads directory
  • In an installation that follows our recommended instructions, this should be at /usr/share/nginx/atom/uploads
  • Make sure the directory's permissions are correct (it should belong to the www-data user)
  • Restart the atom-worker and try your DIP upload again
Another user has outlined the steps involved in more detail in a previous post in the AtoM user forum, here: 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056
@accesstomemory

--

BrzI Channel

unread,
Feb 11, 2019, 4:35:24 PM2/11/19
to archivematica
Hi Dan,

the Atom instance is an exact replica of a fully functional production server. I built it for testing only. It already has an uploads folder (where I restored over 100Gb of data when building the replica).

It has two folders (r and tmp). All permission are set to www-data - as they should.

Because it is a replica I am running version 2.2.1 of Atom on Ubuntu 14.04...and I do not want to use rsync (at least not initially). Connecting to it from Archivematica 1.8.1 on Ubuntu 18.04.1

Since it is Atom 2.2.1 there is no need to do anything about the scheduler - documentation says it is enabled by default.

On another note.

I have also discovered that the atom-worker.conf upstart service starts correctly but it seems it does not have the correct permissions - hence the DIP upload errors in the archivematica dashboard. When I start the atom-worker manually by doing sudo php symphony jobs:worker then archivematica dashboard errors stop occurring - but the DIPs still do not get written to Atom.

Looks like permissions but where ?

Thanks

Dan Gillean

unread,
Feb 11, 2019, 5:43:51 PM2/11/19
to archiv...@googlegroups.com
Hi again, 

Having taken the most recent error message you've shared with us and done a bit more searching in the forum, I've found a few more things to investigate. Long story short: it looks like you may need to configure the rsync settings in Archivematica after all. The instructions definitely could stand to be clearer on the Archivematica side as it makes the rsync configuration sound optional, but I also don't see alternative configurations provided. After looking at a few threads in the forums with similar error messages, in each case the solution has been to configure the rsync target. 

Some relevant links for you are included below: 

First, here is the server configuration instructions in the latest Archivematica documentation: 
This post explains a bit about how DIP upload from Archivematica to AtoM works: 
From the sounds of this post, if your Archivematica and AtoM instances are on the same server, then you may be able to just set the rsync target to /tmp without having to configure SSH. Otherwise, if your AtoM installation is on a different server than Archivematica, then you will need to enable SSH between the servers and fully configure the rsync credentials. 

A little later in the same thread above, the user confirms that setting the rsync target to /tmp/ worked when AtoM and AM were on the same server. 

I would experiment with this and let us know if it helps?

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056
@accesstomemory

BrzI Channel

unread,
Feb 13, 2019, 1:08:18 PM2/13/19
to archivematica
ATOM and archivematica are on different machines.

Will configure rsync transport and report back...


BrzI Channel

unread,
Feb 13, 2019, 4:48:44 PM2/13/19
to archivematica
Hey Dan,

quick question...

Is it OK to specify rsync target to the same folder where ATOM has the tmp folder ? Will it confuse ATOM ?

Thanks

Dan Gillean

unread,
Feb 13, 2019, 4:56:45 PM2/13/19
to archiv...@googlegroups.com
Hi again, 

I confess I haven't personally spent much time configuring the connection between AM and Archivematica, so I will defer to Archivematica team members for a definitive answer. That said, I think your safest bet will be to follow the recommendations in the docs and the threads I sent, which seem to suggest a root tmp directory, rather than the tmp directory specifically in the uploads folder. 

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056
@accesstomemory

--

BrzI Channel

unread,
Feb 13, 2019, 5:02:07 PM2/13/19
to archivematica
OK...will do that.

BrzI Channel

unread,
Feb 13, 2019, 7:24:54 PM2/13/19
to archivematica
Here are the current log errors :

Module upload-qubit_v0.0

--url="http://142.232.42.47/index.php"  --email="xxxxxxx@xxxxxxx" --password="xxxxxxxxxxx" --uuid="31c5db82-8793-4f26-8456-0f6668e698ed" --debug="True" --version="2" --rsync-command="ssh -p 22 -l amatica" --rsync-target="ama...@142.232.42.47: /DIPUP"

upload-qubit_v0.0: ERROR     2019-02-14 00:17:50,734  archivematica.upload.qubit.log:64  [uploadDIP] Target: xxxxx-test1
upload-qubit_v0.0: ERROR     2019-02-14 00:17:50,735  archivematica.upload.qubit.log:64  [uploadDIP] rsync -e ssh -p 22 -l amatica --protect-args -rltz -P --chmod=ugo=rwX /var/archivematica/sharedDirectory/watchedDirectories/uploadDIP/Feb13-31c5db82-8793-4f26-8456-0f6668e698ed ama...@142.232.42.47: /DIPUP
upload-qubit_v0.0: ERROR     2019-02-14 00:17:50,735  archivematica.upload.qubit.log:64  [uploadDIP] Rsync output is being saved in /tmp/tmpL7RiBo
[uploadDIP] Rsync quit unexpectedly (exit 255), the upload script will be stopped here

Rsync target set to :
ama...@142.232.42.47: /DIPUP

Rsync command set to :
ssh -p 22 -l amatica

amatica user on atom is a member of www-data that has full access to
/DIPUP folder on atom


BrzI Channel

unread,
Feb 13, 2019, 7:33:07 PM2/13/19
to archivematica
I also did the following as a test

On my Archivematica server  :

innopac@AMATICA:~$ sudo -u amatica ssh ama...@142.232.42.47
sudo: unknown user: amatica
sudo: unable to initialize policy plugin

I do not have an amatica user on the archivematica server I only have it ont he Atom server. Should I create one ?

If I do the following :
innopac@AMATICA:~$ sudo -u innopac ssh ama...@142.232.42.47
The authenticity of host '142.232.42.47 (142.232.42.47)' can't be established.
ECDSA key fingerprint is SHA256:mJRuP2oJ7LT8WaypzE45XXiAKeIb48HzaLiuBXa8aQQ.
Are you sure you want to continue connecting (yes/no)?

So my connection works...




Karl Goetz

unread,
Feb 13, 2019, 7:35:27 PM2/13/19
to archiv...@googlegroups.com
Hi,

Can the ‘amatica’ user ssh to the atom server? Can you manually rsync files across as ‘amatica’?

karl.

--
You received this message because you are subscribed to the Google Groups "archivematica" group.
To unsubscribe from this group and stop receiving emails from it, send an email to archivematic...@googlegroups.com.
To post to this group, send email to archiv...@googlegroups.com.
Visit this group at https://groups.google.com/group/archivematica.
For more options, visit https://groups.google.com/d/optout.

-- 
Karl Goetz,  Senior Library Officer (Library Systems)
University of Tasmania, Private Bag 25, Hobart 7001
Available Tuesday, Wednesday, Thursday



University of Tasmania Electronic Communications Policy (December, 2014).
This email is confidential, and is for the intended recipient only. Access, disclosure, copying, distribution, or reliance on any of it by anyone outside the intended recipient organisation is prohibited and may be a criminal offence. Please delete if obtained in error and email confirmation to the sender. The views expressed in this email are not necessarily the views of the University of Tasmania, unless clearly intended otherwise.

Karl Goetz

unread,
Feb 13, 2019, 7:37:21 PM2/13/19
to archiv...@googlegroups.com
Hi,

Seeing the key prompt means your port is open but doesn’t explain much past that.

If you run ssh ama...@142.232.42.47 then accept the fingerprint (assuming its correct) are you able to log in?

Btw, Adding sudo is not giving you any value.

karl.

--
You received this message because you are subscribed to the Google Groups "archivematica" group.
To unsubscribe from this group and stop receiving emails from it, send an email to archivematic...@googlegroups.com.
To post to this group, send email to archiv...@googlegroups.com.
Visit this group at https://groups.google.com/group/archivematica.
For more options, visit https://groups.google.com/d/optout.
-- 
Karl Goetz,  Senior Library Officer (Library Systems)
University of Tasmania, Private Bag 25, Hobart 7001
Available Tuesday, Wednesday, Thursday

BrzI Channel

unread,
Feb 14, 2019, 12:39:01 PM2/14/19
to archivematica


If you run ssh ama...@142.232.42.47 then accept the fingerprint (assuming its correct) are you able to log in?


I get this :

innopac@AMATICA:~$ ssh ama...@142.232.42.47

The authenticity of host '142.232.42.47 (142.232.42.47)' can't be established.
ECDSA key fingerprint is SHA256:mJRuP2oJ7LT8WaypzE45XXiAKeIb48HzaLiuBXa8aQQ.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '142.232.42.47' (ECDSA) to the list of known hosts.
ama...@142.232.42.47's password:

... a lot of fluff here....

This account is restricted by rssh.
Allowed commands: rsync


If you believe this is in error, please contact your system administrator.

Connection to 142.232.42.47 closed.


BrzI Channel

unread,
Feb 14, 2019, 12:58:51 PM2/14/19
to archivematica
Yes, I was able to manually rsync copy a fiel from Archivematica to the the atom /DIPUP folder:

rsync test.txt ama...@142.232.42.47:/DIPUP
innopac@AMATICA:~/Desktop$

I was prompted for password - which I understand should not be the case.l

Should I rerun sudo passwd -l amatica on ATOM ?

Thanks

Dan Gillean

unread,
Feb 14, 2019, 2:12:11 PM2/14/19
to archiv...@googlegroups.com
Hi there, 

This is the last bit of suggestions I can offer until Archivematica team members review the thread, but some thoughts: 

First, in your earlier rsync configuration example, there was a space before /DIPUP in the target. I noticed that none of the examples in the docs include this, so that would be one small thing to check. 

When configuring SSH, make sure that you have opened the /etc/rssh.conf file and uncommented the allowrsync line, as per the instructions here

Finally, one of our sysadmins previously shared a simple SSH test, to see if you can create a new file in the target directory. It might be worth trying as well. See: 

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056
@accesstomemory

BrzI Channel

unread,
Feb 14, 2019, 4:06:07 PM2/14/19
to archivematica
 Hi Dan,

I had caught the space and fixed it. The rssh.conf file was already configured correctly.


The poster recommends this to test :
sudo -u archivematica ssh archiv...@192.168.0.103 "touch /tmp/preDipTest"

I interpret this to mean that there is a local account on the Archivematica server called archivematica. I do not have such an account. My understanng is that it is not needed for DIP upload. Am I right ?



Karl Goetz

unread,
Feb 14, 2019, 4:47:54 PM2/14/19
to archiv...@googlegroups.com
Hi,

Looks like your SSH key authorisation isn’t quite working.

The relevant part of the documentation is configuring dip upload and the key item is #5: 

Configure SSH keys to allow rsync to work for the Archivematica user, from the Archivematica server to the AtoM server.

https://www.archivematica.org/en/docs/archivematica-1.8/admin-manual/installation-setup/integrations/atom-setup/#configure-dip-upload

Archivematica user means the user archivematica runs as - could be a special user, could be www-data, could be sharing a normal user account (please don’t do that though).

First log in to your atom server as amatica and run this:

cat ~/.ssh/authorized_keys

If the file is missing or empty you will need to add keys. If it has content it may have the wrong permissions, check with

ls -lh ~/.ssh


To re copy your key from archivematica to atom run the following command *as the user archivematica runs as*.

ssh-copy-id  ama...@142.232.42.47

karl.

--
You received this message because you are subscribed to the Google Groups "archivematica" group.
To unsubscribe from this group and stop receiving emails from it, send an email to archivematic...@googlegroups.com.
To post to this group, send email to archiv...@googlegroups.com.
Visit this group at https://groups.google.com/group/archivematica.
For more options, visit https://groups.google.com/d/optout.
-- 
Karl Goetz,  Senior Library Officer (Library Systems)
University of Tasmania, Private Bag 25, Hobart 7001
Available Tuesday, Wednesday, Thursday

Dan Gillean

unread,
Feb 14, 2019, 4:52:51 PM2/14/19
to archiv...@googlegroups.com
Thank you Karl for all your helpful input on this thread! 

For the original poster: If you have followed any of Artefactual's installation instructions for Archivematica, then I believe an archivematica user is created in the process - this is the user that Archivematica itself will use when executing commands. 

The example test I shared is just to confirm that the user that Archivematica is configured to use can actually create files via SSH in the target location. If you have made changes during installation, then feel free to test with a different user. I suspect that Karl is on the right track here, and would suggest you double-check the SSH key configuration. 

Good luck! 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056
@accesstomemory

BrzI Channel

unread,
Feb 15, 2019, 5:10:17 PM2/15/19
to archivematica
Hi Dan,

I do not have an archivematica user either on the dashboard or on the storage service under "users".
The only thing I have is the base user created for generating the initial storage server key.


So therefore, I am presuming that I can only use one of the user accounts under port 80 or port 8000. If yes, which one ? From 80 or from 8000 ?

Thanks

BrzI Channel

unread,
Feb 15, 2019, 5:17:57 PM2/15/19
to archivematica

Archivematica user means the user archivematica runs as - could be a special user, could be www-data, could be sharing a normal user account (please don’t do that though).


How do I find out what user archivematica runs as ? I presume it is www-data as I had to give it access rights the to source transfer folder...

***************************************************

And this I find confusing :
To re copy your key from archivematica to atom run the following command *as the user archivematica runs as*.
ssh-copy-id  ama...@142.232.42.47

I thought the keys were generated on the Atom server. What am I then copying from archivematica to Atom then ?


BrzI Channel

unread,
Feb 15, 2019, 6:26:48 PM2/15/19
to archivematica
OK. Let me see if I have this ssh rsync understood correctly.

ATOM server :

Create amatica account and give it required permissions to the DIP upload folder.
Log in as that account and run the following command
ssh-keygen
Still on Atom upload the SSH copy to Archivematica by running :
ssh-copy-id -i ~/.ssh/id_rsa.pub user@amatica  (I believe I should create a new user on Archivematica for this command right ?)

On Archivematica server :

Test connection by running :
ssh-i ~/.ssh/id_rsa.pub amatica@atom

If I am not challenged for credentials then the connection is good.

Did I miss anything ?



Karl Goetz

unread,
Feb 18, 2019, 4:46:09 PM2/18/19
to archiv...@googlegroups.com
Hi,
You have some access bits the wrong way around.
I have replied to your email inline, you may need to ‘show more’ or similar in your mail client to see the reply.

On 16 Feb 2019, at 10:26 am, BrzI Channel <brzic...@gmail.com> wrote:

OK. Let me see if I have this ssh rsync understood correctly.

ATOM server :

Create amatica account and give it required permissions to the DIP upload folder.

That bit is correct.

The following bit is back to front: ssh-keygen should be run on your archivematica server as the system level user archivematica runs as. I believe that if you followed the guide that user is ‘archivematica’, otherwise based on what you said it may be www-data.

Log in as that account and run the following command
ssh-keygen
Still on Atom upload the SSH copy to Archivematica by running :
ssh-copy-id -i ~/.ssh/id_rsa.pub user@amatica  (I believe I should create a new user on Archivematica for this command right ?)


So you should be copying your key FROM archivematica@archivematica-server TO amatica@atom-server.

On Archivematica server :

Test connection by running :
ssh-i ~/.ssh/id_rsa.pub amatica@atom

If I am not challenged for credentials then the connection is good.

This bit is correct.

Karl.

-- 
Karl Goetz,  Senior Library Officer (Library Systems)
University of Tasmania, Private Bag 25, Hobart 7001
Available Tuesday, Wednesday, Thursday

Message has been deleted
Message has been deleted
Message has been deleted

BrzI Channel

unread,
Feb 20, 2019, 2:32:08 PM2/20/19
to archivematica
So here is what I did (142.232.42.47 is the atom server - AMATICA is the archivematica server)

innopac@AMATICA:~$ sudo -u archivematica ssh-keygen
[sudo] password for innopac:
Generating public/private rsa key pair.
Enter file in which to save the key (/var/lib/archivematica//.ssh/id_rsa):
Created directory '/var/lib/archivematica//.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /var/lib/archivematica//.ssh/id_rsa.
Your public key has been saved in /var/lib/archivematica//.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:d0C0QSS0PeJugW+qSqAcqdRe6KRlMivPbd0EwIQgt58 archivematica@AMATICA
The key's randomart image is:
+---[RSA 2048]----+
|o..+.  .o+*      |
|....o    = o     |
|  .  .  o =      |
|  o....o . o     |
|.* *E...S . .    |
|=.% .  o.o .     |
|++.o . o=        |
|.+ .. .+.        |
|  +oo..          |
+----[SHA256]-----+
innopac@AMATICA:~$ sudo ssh-copy-id -i /var/lib/archivematica//.ssh/id_rsa.pub archiv...@142.232.42.47
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/var/lib/archivematica//.ssh/id_rsa.pub"
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
archiv...@142.232.42.47's password:
Number of key(s) added: 1
Now try logging into the machine, with:   "ssh 'archiv...@142.232.42.47'"
and check to make sure that only the key(s) you wanted were added.
innopac@AMATICA:~$ ssh 'archiv...@142.232.42.47'
archiv...@142.232.42.47's password:
Welcome to Ubuntu 14.04.5 LTS (GNU/Linux 4.4.0-141-generic x86_64)
 * Documentation:  https://help.ubuntu.com/
  System information as of Wed Feb 20 10:41:35 PST 2019
  System load:  0.0                 Processes:          170
  Usage of /:   60.5% of 358.68GB   Users logged in:    0
  Memory usage: 7%                  IP address for em1: 142.232.42.47
  Swap usage:   0%
  Graph this data and manage this system at:
    https://landscape.canonical.com/
56 packages can be updated.
48 updates are security updates.
New release '16.04.5 LTS' available.
Run 'do-release-upgrade' to upgrade to it.
Your Hardware Enablement Stack (HWE) is supported until April 2019.
Last login: Wed Feb 20 10:16:48 2019 from 142.232.43.160
archivematica@ATOM221:~$

The authenticity of host '142.232.42.47 (142.232.42.47)' can't be established.
ECDSA key fingerprint is SHA256:mJRuP2oJ7LT8WaypzE45XXiAKeIb48HzaLiuBXa8aQQ.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '142.232.42.47' (ECDSA) to the list of known hosts.

Welcome to Ubuntu 14.04.5 LTS (GNU/Linux 4.4.0-141-generic x86_64)
 * Documentation:  https://help.ubuntu.com/
  System information as of Wed Feb 20 11:26:06 PST 2019
  System load:  0.01                Processes:          214
  Usage of /:   60.5% of 358.68GB   Users logged in:    1
  Memory usage: 9%                  IP address for em1: 142.232.42.47
  Swap usage:   0%
  Graph this data and manage this system at:
    https://landscape.canonical.com/
56 packages can be updated.
48 updates are security updates.
New release '16.04.5 LTS' available.
Run 'do-release-upgrade' to upgrade to it.
Your Hardware Enablement Stack (HWE) is supported until April 2019.
Last login: Wed Feb 20 11:26:06 2019 from 142.232.43.160 

Looks like it is not working. I can confirm that the authorized_keys file is copied over to atom

What am I doing wrong ?

BrzI Channel

unread,
Feb 20, 2019, 2:39:30 PM2/20/19
to archivematica
Or maybe I did not do anything wrong ? I was nto prompted for a password for archivematica (see below).

[sudo] password for innopac:

Welcome to Ubuntu 14.04.5 LTS (GNU/Linux 4.4.0-141-generic x86_64)
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Last login: Wed Feb 20 11:26:58 2019 from 142.232.43.160
archivematica@ATOM221:~$ 

Am I good.... ? If yes, what rsync command should I use ?

Thanks

BrzI Channel

unread,
Feb 20, 2019, 5:15:52 PM2/20/19
to archivematica
Following the recent changes I had this :
innopac@ATOM221:/usr/share/nginx/atom$ sudo php symfony jobs:worker
2019-02-20 14:00:39 > New ability: arGenerateFindingAidJob
2019-02-20 14:00:39 > New ability: arInheritRightsJob
2019-02-20 14:00:39 > New ability: qtSwordPluginWorker
2019-02-20 14:00:39 > Running worker...
2019-02-20 14:00:39 > PID 16021
2019-02-20 14:05:51 > Job 89187 "qtSwordPluginWorker": A new job has started to being processed.
2019-02-20 14:05:51 > Job 89187 "qtSwordPluginWorker": A package was deposited by reference.
2019-02-20 14:05:51 > Job 89187 "qtSwordPluginWorker": Location: file:///Feb20-f0db0769-1738-473b-975b-4836801a05ba
2019-02-20 14:05:51 > Job 89187 "qtSwordPluginWorker": Processing...
2019-02-20 14:05:51 > Job 89187 "qtSwordPluginWorker": Object slug: goran-test1
2019-02-20 14:05:51 > Job 89187 "qtSwordPluginWorker": Exception: File/directory does not exist or is not readable: /tmp/Feb20-f0db0769-1738-473b-975b-4836801a05ba.
2019-02-20 14:05:51 > Job failed: File/directory does not exist or is not readable: /tmp/Feb20-f0db0769-1738-473b-975b-4836801a05ba.

Question : is the /tmp folder on AtoM or on Archivematica ?

BrzI Channel

unread,
Feb 20, 2019, 5:49:40 PM2/20/19
to archivematica
After tweaking the rsync target I go this :

2019-02-20 14:45:42 > Job 89189 "qtSwordPluginWorker": A new job has started to being processed.
2019-02-20 14:45:42 > Job 89189 "qtSwordPluginWorker": A package was deposited by reference.
2019-02-20 14:45:42 > Job 89189 "qtSwordPluginWorker": Location: file:///Feb20-f0db0769-1738-473b-975b-4836801a05ba
2019-02-20 14:45:42 > Job 89189 "qtSwordPluginWorker": Processing...
2019-02-20 14:45:42 > Job 89189 "qtSwordPluginWorker": Object slug: goran-test1
2019-02-20 14:45:42 > Job 89189 "qtSwordPluginWorker": Exception: Level of description not found: Directory
2019-02-20 14:45:42 > Job failed: Level of description not found: Directory


Thanks

BrzI Channel

unread,
Feb 20, 2019, 6:57:46 PM2/20/19
to archivematica
In the Assets tab I see the following :

AIP DIP URL Upload date Upload status Actions
Feb20_2 http://142.232.42.47/index.php/sword/deposit/1960s-xxx-campus-and-buildings-3 Feb. 20, 2019, 11:52 p.m. Deposited asynchronously, AtoM is processing the DIP in the job queue Delete


So it looks like the rsync upload worked....but nothing is seen when I go to the actual DIP url (without the sword/deposit).

atom-worker still gives me :

Dan Gillean

unread,
Feb 21, 2019, 10:23:32 AM2/21/19
to archiv...@googlegroups.com
It looks like you are really close! 

I am out of my depth trying to troubleshoot Archivematica issues, but I have previously only seen this error when the SIP arrange panel is used in Archivematica using AtoM levels of description. See: 
By default, when SIP arrange is not used, Archivematica should pass the object to AtoM and attach it to an item level description. In your case, it looks like it is trying to find a level of description called "Directory." I'm not sure why that is, but if you are trying to fetch levels of description from AtoM, make sure that the arRestApiPlugin is enabled in AtoM (found in Admin > Plugins). If you are trying to attach the DIP to a new description in AtoM with a level of description of "Directory" using this method, then you might first need to create the "Directory" term in the Levels of description taxonomy in AtoM. For more information on managing terms in AtoM, see: 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056
@accesstomemory

BrzI Channel

unread,
Feb 21, 2019, 3:19:04 PM2/21/19
to archivematica


On Thursday, February 21, 2019 at 7:23:32 AM UTC-8, Dan Gillean wrote:
It looks like you are really close! 

I am out of my depth trying to troubleshoot Archivematica issues, but I have previously only seen this error when the SIP arrange panel is used in Archivematica using AtoM levels of description. See: 


Hi Dan,

I have a question : In the SIP arrange panel I can only see the folder but I cannot expand it to see the two objects inside it (two .jpeg files).

-Should I be able to see them ?
-Should there be another csv or xml file along with the two .jpegs ?

BrzI Channel

unread,
Feb 21, 2019, 4:31:18 PM2/21/19
to archivematica
After creating a Directory level of description the atom worker gives me this :

2019-02-21 12:32:24 > New ability: arGenerateFindingAidJob
2019-02-21 12:32:24 > New ability: arInheritRightsJob
2019-02-21 12:32:24 > New ability: qtSwordPluginWorker
2019-02-21 12:32:24 > Running worker...
2019-02-21 12:32:24 > PID 4245
2019-02-21 13:19:19 > Job 89205 "qtSwordPluginWorker": A new job has started to being processed.
2019-02-21 13:19:19 > Job 89205 "qtSwordPluginWorker": A package was deposited by reference.
2019-02-21 13:19:19 > Job 89205 "qtSwordPluginWorker": Location: file:///Feb21-4-4ecd3ff4-c733-4912-bd19-4d58f805d32d
2019-02-21 13:19:19 > Job 89205 "qtSwordPluginWorker": Processing...
2019-02-21 13:19:19 > Job 89205 "qtSwordPluginWorker": Object slug: goran-test1
2019-02-21 13:19:23 > Job 89205 "qtSwordPluginWorker": Job finished.

Guess this is good....but still no sight of the two added items.

In the archivematica dashboard the log says this : Is it an error because it says ERROR ?


Standard streams

Standard error (stderr)

upload-qubit_v0.0: ERROR     2019-02-21 21:19:18,438  archivematica.upload.qubit.log:64  [uploadDIP] Target: goran-test1
upload-qubit_v0.0: ERROR     2019-02-21 21:19:18,438  archivematica.upload.qubit.log:64  [uploadDIP] rsync -e ssh -p 22 -l archivematica --protect-args -rltz -P --chmod=ugo=rwX /var/archivematica/sharedDirectory/watchedDirectories/uploadDIP/Feb21-4-4ecd3ff4-c733-4912-bd19-4d58f805d32d archiv...@142.232.42.47:/tmp
upload-qubit_v0.0: ERROR     2019-02-21 21:19:18,439  archivematica.upload.qubit.log:64  [uploadDIP] Rsync output is being saved in /tmp/tmpslrkX0
upload-qubit_v0.0: ERROR     2019-02-21 21:19:19,463  archivematica.upload.qubit.log:64  [uploadDIP] Sending... 100 (ETA: 0:00:00)
upload-qubit_v0.0: ERROR     2019-02-21 21:19:19,465  archivematica.upload.qubit.log:64  [uploadDIP] About to deposit to: http://142.232.42.47/index.php/sword/deposit/goran-test1
upload-qubit_v0.0: ERROR     2019-02-21 21:19:19,615  archivematica.upload.qubit.log:64  [uploadDIP] > Response code: 302
upload-qubit_v0.0: ERROR     2019-02-21 21:19:19,615  archivematica.upload.qubit.log:64  [uploadDIP] > Location: /index.php/goran-test1
upload-qubit_v0.0: ERROR     2019-02-21 21:19:19,616  archivematica.upload.qubit.log:64  [uploadDIP] > Content received: <?xml version="1.0" encoding="utf-8" ?><entry xmlns="http://www.w3.org/2005/Atom"
       xmlns:sword="http://purl.org/net/sword/">

<title>goran-test1</title>
<id>89173 / 2019-02-06T10:32:55</id>
  <updated>2019-02-06T10:32:55</updated>
  <author>
    <name>archivematica</name>
  </author>
  <generator uri="http://142.232.42.47/index.php/" version="2.2.1">Qubit 2.2.1</generator>
  <content type="text/html" src="http://142.232.42.47/index.php/goran-test1" />
  <sword:noOp>false</sword:noOp>
  <sword:packaging>http://purl.org/net/sword-types/METSArchivematicaDIP</sword:packaging>
  <sword:userAgent>Archivematica</sword:userAgent>
</entry>
upload-qubit_v0.0: ERROR     2019-02-21 21:19:19,616  archivematica.upload.qubit.log:64  [uploadDIP] Deposited asynchronously, AtoM is processing the DIP in the job queue

Dan Gillean

unread,
Feb 21, 2019, 4:49:31 PM2/21/19
to archiv...@googlegroups.com
Hi there, 

I believe that before you start dragging content into the arrange panel, you should be able to see objects inside of directories on the left side of the SIP arrange page. The documentation does note that once you have moved some objects, they will be greyed out and inaccessible in the oirigin tab. 

Regarding your output - it looks like it is mostly working! I don't think it saying ERROR in this case is actually an issue - this is the stderr output and I think that prefix is added to nearly all logs captured. 

As for the objects not showing up in AtoM... I think it is time to turn our attention over to AtoM. 

First, to confirm: When you were installing AtoM, did you also install and configure the AtoM job scheduler as per the instructions on this page?
If yes, you might want to check the status of the atom-worker. If it is not running try restarting it - there are commands provided on the page linked above for both Ubuntu 14.04 and 16.04 installations. Additionally, if you are using 16.04, then you should be able to use the following command to see the log for the atom-worker: 
  • sudo journalctl -f -u atom-worker
Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056
@accesstomemory

--

Karl Goetz

unread,
Feb 21, 2019, 11:55:30 PM2/21/19
to archiv...@googlegroups.com
Hi,
I concur with Dan here, I think you’ve got things set up correctly on the Archivematica side - great job!

My only question around your AM configuration would be if you have imported the levels of description from your destination atom in to your archivematica.I assume you have, but I’m asking in case.

Are you putting things in backlog before uploading or moving them directly from ingest to transfer during processing?

I see from the message you posted a couple ago that atom is redirecting you from sword/deposit/goran-test1 to goran-test1. To confirm, are you putting in sword/deposit/goran-test1 or goran-test1 when performing the atom upload?

Karl.
-- 
Karl Goetz,  Senior Library Officer (Library Systems)
University of Tasmania, Private Bag 25, Hobart 7001
Available Tuesday, Wednesday, Thursday

BrzI Channel

unread,
Feb 22, 2019, 1:20:23 PM2/22/19
to archivematica


My only question around your AM configuration would be if you have imported the levels of description from your destination atom in to your archivematica.I assume you have, but I’m asking in case.

Yes, I fetched the levels of description from AtoM but the worker would not say "job finished" until I actually created a Directory level of decsription on AtoM.

Are you putting things in backlog before uploading or moving them directly from ingest to transfer during processing?

After the transfer I always went directly to ingest because that is what would get highlighted with the red circle. I have not to this point done anything in backlog - ever. Should I have ?
 
I see from the message you posted a couple ago that atom is redirecting you from sword/deposit/goran-test1 to goran-test1. To confirm, are you putting in sword/deposit/goran-test1 or goran-test1 when performing the atom upload?
 
I always wrote goran-test1 with the presumption that the system would automatically prepend sword/deposit.

BrzI Channel

unread,
Feb 22, 2019, 1:30:41 PM2/22/19
to archivematica
First, to confirm: When you were installing AtoM, did you also install and configure the AtoM job scheduler as per the instructions on this page?

Yes, I did. My /etc/default/gearman-job-server has this line enabled.
PARAMS="--listen=localhost --port=4730"
ufw is inactive
 
If yes, you might want to check the status of the atom-worker. If it is not running try restarting it - there are commands provided on the page linked above for both Ubuntu 14.04 and 16.04 installations. Additionally, if you are using 16.04, then you should be able to use the following command to see the log for the atom-worker: 
  • sudo journalctl -f -u atom-worker
I run Ubuntu 14.04 so I configured /etc/init/atom-worker.conf as per the instructions.

When I type sudo status atom-worker I get:
atom-worker stop/waiting

Note that not all four command listed on that page work with this upstart.

For all my testing I actually manually started the worker by going

php symfony jobs:worker

This is because it looks like the upstart is not quite working correctly.
I left this issue for later as my understanding is that multiple workers can coexist just fine.
My /usr/share/nginx/atom/log/atom-worker.log is totally empty.

Thanks
 

Dan Gillean

unread,
Feb 25, 2019, 1:17:38 PM2/25/19
to archiv...@googlegroups.com
Hi again, 

After the transfer I always went directly to ingest because that is what would get highlighted with the red circle. I have not to this point done anything in backlog - ever. Should I have ?

You will notice in the documentation that this is called "Arrange SIP from Backlog." This might also be part of the issues you are encountering. Archivematica's SIP arrange is looking for items that are added to Backlog first, so if you want to use the arrange function, I would suggest first sending your SIP to backlog, and then using SIP arrange from there. 

Note that not all four command listed on that page work with this upstart.

When running as the www-data user, you might need to preface these commands with sudo, but if these commands don't work, then there might be an issue with how the upstart script is configured, or with permissions? Might be worth investigating further if you continue to run into problems. In any case, before you start your next DIP test, I would recommend running sudo restart atom-worker from the root AtoM installation directory (generally /usr/share/nginx/atom) on your AtoM server. This should make sure that the job scheduler and its workers are still active and ready to receive a new job. Occasionally after long periods of inactivity, the workers will need to be restarted to avoid 500 errors. 

You could also try executing a task in AtoM that uses the job scheduler, to make sure it's working correctly, independently of Archivematica. For example, you could try one of the following, all of which use the job scheduler in AtoM: 
If you encounter a 500 error, then try restarting the atom-worker. If you still run into issues, then check the Nginx error logs, and see if anything has been added to the atom-worker logs. 

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056
@accesstomemory

--

BrzI Channel

unread,
Feb 26, 2019, 2:37:30 PM2/26/19
to archivematica
AtoM ver 2.2.1 Amatica ver 1.8.1

I tried again today....once with the upstart atom worker and once with the atom-worker explicitly started using :

sudo php symfony jobs:worker

In both cases, I saw a file called Feb26_e31eaf27-eb4c-4365-8eba-fbab71494d78 written to the /tmp folder. So that is good.

When I use the upstart worker I get a 500 error in the ingest phase (even though the file is written)

When I start the worker explicitly I get "job finished" message..

In access I see this :

So I see two issues :

1. The atom worker upstart does not seem very reliable...imho (lines 1 and 2)

2. Even though the file is written to the atom server the additional items never get written to the destination record.

Question:

In my Atom Server settings the path for the SWORD deposit directory is configured to /tmp.
Should it be configured to /usr/share/nginx/atom/uploads/tmp ?

Thanks

BrzI Channel

unread,
Feb 26, 2019, 2:53:11 PM2/26/19
to archivematica
I also notice that the logs in access show UTC time not my local time...where can I change that in Archivematica so that it shows local time :

Atom:
                  Local time: Tue 2019-02-26 11:50:26 PST
                  Universal time: Tue 2019-02-26 19:50:26 UTC
                  Timezone: America/Vancouver (PST, -0800)
                  NTP enabled: yes
                  NTP synchronized: yes
                  RTC in local TZ: no
                  DST active: no
 Last DST change: DST ended at
                  Sun 2018-11-04 01:59:59 PDT
                  Sun 2018-11-04 01:00:00 PST
 Next DST change: DST begins (the clock jumps one hour forward) at
                  Sun 2019-03-10 01:59:59 PST
                  Sun 2019-03-10 03:00:00 PDT

Archivematica
                  Local time: Tue 2019-02-26 11:51:11 PST
                  Universal time: Tue 2019-02-26 19:51:11 UTC
                  RTC time: Tue 2019-02-26 19:51:11
                  Time zone: America/Vancouver (PST, -0800)
                  System clock synchronized: yes
                  systemd-timesyncd.service active: yes
                  RTC in local TZ: no



Sara @ Artefactual

unread,
Feb 26, 2019, 3:06:10 PM2/26/19
to archivematica
Hi there,

We currently have an open documentation issue (#291) regarding setting the time zone in Archivematica - it needs to be changed in several places and there is currently no list that we can provide. Keep an eye on that issue - it will be updated once the information has been added to the documentation

Unfortunately, I don't think we (that is, Artefactual) can provide any more advice regarding your setup issues. We do offer paid support services that can be tailored to your needs - for more information, you can check out the Artefactual website.

Regards,
Sara Allain
Systems Archivist
Artefactual Systems, Inc.

BrzI Channel

unread,
Feb 26, 2019, 4:17:41 PM2/26/19
to archivematica
No errors in atom worker log.
innopac@ATOM221:~$ sudo journalctl -u atom-worker
-- Logs begin at Tue 2019-02-26 10:19:09 PST, end at Tue 2019-02-26 13:16:17 PST. --


NGINX access log has this line:
142.232.43.160 - xxxx...@xxxx.co [26/Feb/2019:13:11:11 -0800] "POST /index.php/sword/deposit/goran-test1 HTTP/1.1" 302 675 "-" "Archivematica"

NGINX error log has no erros.

BrzI Channel

unread,
Feb 26, 2019, 7:16:52 PM2/26/19
to archivematica
 It turns out I was not looking in the right place.

I have a test series-level record in AtoM with a single attached .jpg item-level file.

My test upload from Archivematica has two .jpg files which I was expecting to show up alongside the existing item.

I have two issues :

1. My atom worker upstart service keeps giving me a 500 error. Nothing ever happens until I explicitly start the worker. Once I do I start seeing content being populated in draft status on AtoM. So I have to figure out the issue with the atom worker.

2. I need to figure out how to organize the data that I submitting...I will enlist my archivist's help in figuring this one out...
   Here is my import data folder structure :
  

When I import it into AtoM it lists itself as a directory - not as an item....I suspect this should not be too difficult to sort out...

Thanks



Reply all
Reply to author
Forward
0 new messages