Relationships feature not working in new install

56 views
Skip to first unread message

Patrick O'Neill

unread,
Aug 28, 2022, 11:12:45 PM8/28/22
to AtoM Users
Hi, I am working on a new install of Atom, and having difficulty making the relationships feature work in Authority Records. This is a single-institution use of AtoM, we have entered a dozen Authority Records so far, but each time we try to establish a relationship (temporal, hierarchical etc) between the Authority Records, we hit a 500 internal server error. I have seen that there was a bug in this feature a few years ago (https://projects.artefactual.com/issues/13499, also https://projects.artefactual.com/issues/10682), but these don't seem to be exactly the same. Ours seems like a fairly simple process (create an authority record, save it, edit it to establish a relationship), and there is so little data so far in our install that it's hard to think there should be a problem. 
Has anyone else encountered this issue recently in a new install, or is there likely a feature of the install that we should look at?
Regards
Patrick

Dan Gillean

unread,
Aug 29, 2022, 9:07:16 AM8/29/22
to ICA-AtoM Users
Hi Patrick, 

The two issues you linked to could possibly be related to this - it's hard to know without seeing further information. In any case, those issues have been patched in the upcoming 2.7 release, but not in current releases. 

First, to clarify your current process where you said  you make a record, save it, then try to add a relationship: is that relationship to an existing second authority record, or are you trying to create the second record via the relationship? If the latter, then this is exactly the issue described in the bugs you referenced. Instead, make sure both records exist first, and then add a relationship between them. 

Beyond that, a good start would be knowing what error you are seeing in the web server logs. If you've followed our recommended installation instructions, this can be checked with the command on the following documentation page: 
Feel free to share the output you find. 

Also, please remind me about your installation - the full version number listed in Admin > Settings; whether you made any changes to the recommended installation docs (and if yes, what), whether your site has any local customizations (including custom theme plugins, etc), and any other information you think would be helpful, including any steps you've already tried (like running common maintenance tasks etc) to resolve the issue. Hopefully with that, we can offer further suggestions! 

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056
@accesstomemory
he / him


--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/e73398a2-2368-4d94-8f58-dd793de58a82n%40googlegroups.com.

Patrick O'Neill

unread,
Aug 30, 2022, 12:13:49 AM8/30/22
to AtoM Users
Thanks very  much, Dan.
We are running 2.6.4 - 184, with no mods. We have been attempting to create relationships between existing authority records. Here are the most common error messages we have been getting:

2022/08/29 11:10:30 [error] 56272#56272: *1014 FastCGI sent in stderr: "PHP message: No Gearman worker available that can handle the job arUpdateEsActorRelationsJob" while reading response header from upstream, client: 10.3.132.16, server: archives.catholic.net.au, request: "POST /index.php/hanly-james-1815-1895-dean/edit HTTP/1.1", upstream: "fastcgi://unix:/run/php7.4-fpm.atom.sock:", host: "archives.bne.catholic.net.au", referrer: "https://archives.bne.catholic.net.au/index.php/hanly-james-1815-1895-dean/edit"

 2022/08/24 09:34:50 [error] 56273#56273: *139 FastCGI sent in stderr: "PHP message: Empty module and/or action after parsing the URL "/bishop-quinn-141-jpg/digitalobject/delete" (/)" while reading response header from upstream, client: 10.3.132.16, server: archives.catholic.net.au, request: "GET /index.php/bishop-quinn-141-jpg/digitalobject/delete HTTP/1.1", upstream: "fastcgi://unix:/run/php7.4-fpm.atom.sock:", host: "archives.bne.catholic.net.au", referrer: "https://archives.bne.catholic.net.au/index.php/bishop-quinn-jpg/edit"

2022/08/09 15:02:12 [error] 941#941: *102 FastCGI sent in stderr: "PHP message: No Gearman worker available that can handle the job arUpdateEsActorRelationsJob" while reading response header from upstream, client: 10.3.132.16, server: archives.catholic.net.au, request: "POST /index.php/diocese-of-brisbane/edit HTTP/1.1", upstream: "fastcgi://unix:/run/php7.4-fpm.atom.sock:", host: "archives.bne.catholic.net.au", referrer: "https://archives.bne.catholic.net.au/index.php/diocese-of-brisbane/edit"

 2022/08/09 15:03:26 [error] 941#941: *119 FastCGI sent in stderr: "PHP message: No Gearman worker available that can handle the job arUpdateEsActorRelationsJob" while reading response header from upstream, client: 10.3.132.16, server: archives.catholic.net.au, request: "POST /index.php/hanly-james-dean-1815-1895/edit HTTP/1.1", upstream: "fastcgi://unix:/run/php7.4-fpm.atom.sock:", host: "archives.bne.catholic.net.au", referrer: "https://archives.bne.catholic.net.au/index.php/hanly-james-dean-1815-1895/edit"

2022/08/15 10:46:08 [error] 151944#151944: *348 FastCGI sent in stderr: "PHP message: No Gearman worker available that can handle the job arUpdateEsActorRelationsJob" while reading response header from upstream, client: 10.3.132.16, server: archives.catholic.net.au, request: "POST /index.php/diocese-of-brisbane/edit HTTP/1.1", upstream: "fastcgi://unix:/run/php7.4-fpm.atom.sock:", host: "archives.bne.catholic.net.au", referrer: "https://archives.bne.catholic.net.au/index.php/diocese-of-brisbane/edit"

The core error is "PHP message: No Gearman worker available that can handle the job arUpdateEsActorRelationsJob". Whilst we couldn't specifically find an error relating to "arUpdateEsActorRelationsJob", this link is closely related to what might be a solution: https://groups.google.com/g/ica-atom-users/c/LLQKhFlFVtI We have tried the following steps: clear the application cache, restart services, and populate the search. Thought this might be related to the fact that we have escaped the backslash as a special character, but I don't think so.

Cheers
Patrick

On Monday, 29 August 2022 at 11:07:16 pm UTC+10 Dan Gillean wrote:
Hi Patrick, 

The two issues you linked to could possibly be related to this - it's hard to know without seeing further information. In any case, those issues have been patched in the upcoming 2.7 release, but not in current releases. 

First, to clarify your current process where you said  you make a record, save it, then try to add a relationship: is that relationship to an existing second authority record, or are you trying to create the second record via the relationship? If the latter, then this is exactly the issue described in the bugs you referenced. Instead, make sure both records exist first, and then add a relationship between them. 

Beyond that, a good start would be knowing what error you are seeing in the web server logs. If you've followed our recommended installation instructions, this can be checked with the command on the following documentation page: 
Feel free to share the output you find. 

Also, please remind me about your installation - the full version number listed in Admin > Settings; whether you made any changes to the recommended installation docs (and if yes, what), whether your site has any local customizations (including custom theme plugins, etc), and any other information you think would be helpful, including any steps you've already tried (like running common maintenance tasks etc) to resolve the issue. Hopefully with that, we can offer further suggestions! 

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056
@accesstomemory
he / him


Dan Gillean

unread,
Aug 30, 2022, 9:21:05 AM8/30/22
to ICA-AtoM Users
Hi Patrick, 

Aha! Well hopefully this will be easy to resolve then. Anytime you see an error log entry that includes the message, "No Gearman worker available that can handle the job ..." take it as a clue to check the job scheduler. It sounds likely that this just needs to be restarted. 

AtoM includes a job scheduler to handle long-running tasks asynchronously in the background, so they don't time out in the web browser (which typically has a 1min timeout limit), and so you don't have to sit there waiting for them to complete before going to a different page. This is used for imports and exports, move operations, and a lot more - including quietly updating the search index of related entities in the background when you do things like add new relationships. It looks like the job scheduler's worker isn't running, so when AtoM tries to index your new authority record relationships, it can't. 

The installation instructions for the Gearman job scheduler we use in AtoM can be found here (these are linked from the main installation docs, but sometimes people miss it - we will be reorganizing the documentation for the 2.7 release): 
You can check the status of the job scheduler's atom-worker with: 
  • sudo systemctl status atom-worker
You can restart the atom-worker with: 
  • sudo systemctl restart atom-worker
However, one important note: The worker has some built in logic to retry operations if it fails - but to prevent this from allowing the worker to get caught in an endless loop of retrying and failing, there's a restart limit in place (added in the configuration file you create during installation) - if the worker attempts to restart more than 3 times in a 24 hour period, then the start limit is hit. You will then need to reset the fail-counter to be able to restart the worker. 

So: you might want to run the status command again after restarting. If the worker is not running, then try resetting the fail-counter and the restarting again, like so: 
  • sudo systemctl reset-failed atom-worker
  • sudo systemctl restart atom-worker
Once you have the atom-worker running again, test out an authority record relationship - let us know if that resolves the issue! And if not, please remember to provide some of the additional information about your installation environment requested in my first response to help us identify further possible suggestions. 

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056
@accesstomemory
he / him

Patrick O'Neill

unread,
Sep 5, 2022, 6:53:49 PM9/5/22
to AtoM Users
Hi Dan, your advice was spot on!
We received this info from our ICT department:

As per the advice from the vendor, I ran:
 sudo systemctl status atom-worker
 The result was:
Unit atom-worker.service could not be found.
 
So the atom-worker service was never installed for this feature to operate correctly. I have now installed the service as outlined in the following:
https://www.accesstomemory.org/en/docs/2.6/admin-manual/installation/asynchronous-jobs/

And now our relationships feature, locations and other features are running perfectly! :-)

Thanks for your help!
Patrick
On Tuesday, 30 August 2022 at 11:21:05 pm UTC+10 Dan Gillean wrote:
Hi Patrick, 

Aha! Well hopefully this will be easy to resolve then. Anytime you see an error log entry that includes the message, "No Gearman worker available that can handle the job ..." take it as a clue to check the job scheduler. It sounds likely that this just needs to be restarted. 

AtoM includes a job scheduler to handle long-running tasks asynchronously in the background, so they don't time out in the web browser (which typically has a 1min timeout limit), and so you don't have to sit there waiting for them to complete before going to a different page. This is used for imports and exports, move operations, and a lot more - including quietly updating the search index of related entities in the background when you do things like add new relationships. It looks like the job scheduler's worker isn't running, so when AtoM tries to index your new authority record relationships, it can't. 

The installation instructions for the Gearman job scheduler we use in AtoM can be found here (these are linked from the main installation docs, but sometimes people miss it - we will be reorganizing the documentation for the 2.7 release): 
You can check the status of the job scheduler's atom-worker with: 
  • sudo systemctl status atom-worker
You can restart the atom-worker with: 
  • sudo systemctl restart atom-worker
However, one important note: The worker has some built in logic to retry operations if it fails - but to prevent this from allowing the worker to get caught in an endless loop of retrying and failing, there's a restart limit in place (added in the configuration file you create during installation) - if the worker attempts to restart more than 3 times in a 24 hour period, then the start limit is hit. You will then need to reset the fail-counter to be able to restart the worker. 

So: you might want to run the status command again after restarting. If the worker is not running, then try resetting the fail-counter and the restarting again, like so: 
  • sudo systemctl reset-failed atom-worker
  • sudo systemctl restart atom-worker
Once you have the atom-worker running again, test out an authority record relationship - let us know if that resolves the issue! And if not, please remember to provide some of the additional information about your installation environment requested in my first response to help us identify further possible suggestions. 

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056
@accesstomemory
he / him

Reply all
Reply to author
Forward
0 new messages