problema con atom-worker.service

[ANTHONY ROLANDO DAZA ORTIZ]

Good afternoon.

I have an error with the atom-worker.service service, the one I mentioned cannot be started, it always gives me one or two different errors and no matter how much I have been guided by everyone else with this problem in the forum, I still cannot solve the problem. problem, I leave the errors that it gives me when trying to start the service.

These are the logs it gives me:
                                                                   
Apr 23 16:28:38 atom-VirtualBox systemd[1]: atom-worker.service: Main process exited, code=exited, status=1/FAILURE
Apr 23 16:28:38 atom-VirtualBox systemd[1]: atom-worker.service: Failed with result 'exit-code'.
Apr 23 16:29:08 atom-VirtualBox systemd[1]: atom-worker.service: Scheduled restart job, restart counter is at 3.
Apr 23 16:29:08 atom-VirtualBox systemd[1]: Stopped atom-worker.
Apr 23 16:29:08 atom-VirtualBox systemd[1]: atom-worker.service: Start request repeated too quickly.
Apr 23 16:29:08 atom-VirtualBox systemd[1]: atom-worker.service: Failed with result 'exit-code'.
Apr 23 16:29:08 atom-VirtualBox systemd[1]: Failed to start atom worker.

and this is one of the errors when trying to run: Configuration file "/usr/share/nginx/atom/vendor/symfony/lib/config/config/settings.yml, /usr/share/nginx/atom/apps/qubit/config /settings.yml" does not have a registered handler.

[Dan Gillean]

Hi Anthony, 

First, the job scheduler has some automated retry policies in place, but to prevent the worker from getting caught in an infinite loop if it encounters a problem that restarting cannot solve, we have also added a fail counter - meaning once 3 restarts have been attempted in 24 hours or less, then the restart limit has been met, and the fail counter  must be manually reset before restarting will work. From the job scheduler messages you posted, this looks to be the case (it says "restart counter is at 3" and then the restart fails). 

You can manually reset the counter with: 

• sudo systemctl reset-failed atom-worker
• sudo systemctl start atom-worker
  

See: 

• https://www.accesstomemory.org/docs/latest/admin-manual/maintenance/asynchronous-jobs/#atom-worker-start-rate-limit

As for the other message, and if that doesn't work...

Searching the user forum, I found one other instance with a Symfony message about no registered handler. In that case, the problem was that the server's disk space was completely full. Be sure to check how much disk space you have, and add more if needed. The other case also saw similar problems when attempting to clear the cache, and a manual cache clear helped: 

• rm -fr cache/*
  
• See: https://groups.google.com/g/ica-atom-users/c/Iw0glcWaIDM/m/Ba6YDMnLAAAJ

In general, for a medium-sized production instance, our docs recommend the following as a good starting point: 

Processor: 2 vCPUs @ 2.3GHz
Memory: 7GB
Disk space (processing): 50GB at a minimum for AtoM’s core stack plus more storage would be required for supporting any substantial number of digital objects.

See: https://www.accesstomemory.org/docs/latest/admin-manual/installation/requirements/#minimum-hardware-requirements

Good luck, and let us know how it goes!

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

he / him

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/1b6f6800-4aaa-455c-8aed-ee57164e21b2n%40googlegroups.com.

[ANTHONY ROLANDO DAZA ORTIZ]

Thank you very much, but even so the service continues to fail after running sudo systemctl reset-failed fdp-worker, and it still gives me the same error with the same logs, I also tried clearing cache as they did in the forum conversation that they gave me. You attached it and it doesn't work, I think the problem arose when I didn't notice that there was no more space in the VB, I already increased it but I don't know if you have it or do you know what else could happen?

[Dan Gillean]

Hi again, 

I will ask our developers if they have further suggestions, but just so I am clear: 

• Resetting the fail counter did not work
• You did try manually clearing the cache, as described in the linked thread and my last message, but  it also didn't help
• However, you DID discover that the disk space was full in your virtual machine

Correct?

A few more questions: 

• Did you try restarting any services after adding more disk? E.g. your webserver, PHP-FPM, etc?
• You checked and increased the disk space, but what about memory and CPU? Do you meet the recommended minimum requirements (e.g. 7GB RAM, at least 2vCPU)?
• You could always try looking for bottlenecks by using htop to monitor resources - see this section of the Troubleshooting docs
• Did you try clearing the cache, resetting the fail counter, and then restarting the job scheduler AFTER adding more disk?
• Is there any more information in the atom-worker logs? Per this documentation page, you can check with: 
• sudo journalctl -f -u atom-worker
  
• What is the full AtoM version number shown in Admin > Settings?
• Does your installation follow our recommended installation documentation (e.g. for 2.8 that means using Ubuntu 20.04, PHP 7.4, Nginx, MySQL 8.0, Elasticsearch 5.6, etc)?
• Does your installation have any local code changes or customizations, such as custom features or a custom theme plugin?

Hopefully with some more information, we can provide further suggestions. 

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

he / him

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/c104c506-a19b-4796-a4d2-6022b2f61d8bn%40googlegroups.com.

[ANTHONY ROLANDO DAZA ORTIZ]

Hello.

If you are correct, the only thing is that the error began to happen since the machine was full, that is, since I uploaded the question to the forum, I had already uploaded the disk in the virtual machine and made all of its recommendations with more than enough space.

Also clarify that the machine has 4vCPU, 16GB RAM and 300GB of storage.

Also answering the questions you give me, here are the logs that appear in sudo 'journalctl -f -u atom-worker':
Apr 24 12:45:42 atom-VirtualBox systemd[1]: Started atomworker.
Apr 24 12:45:42 atom-VirtualBox systemd[1276191]: atom-worker.service: Changing to the requested working directory failed: No such file or directory
Apr 24 12:45:42 atom-VirtualBox systemd[1276191]: atom-worker.service: Failed at step CHDIR spawning /usr/bin/php7.4: No such file or directory
Apr 24 12:45:42 atom-VirtualBox systemd[1]: atom-worker.service: Main process exited, code=exited, status=200/CHDIR
Apr 24 12:45:42 atom-VirtualBox systemd[1]: atom-worker.service: Failed with result 'exit-code'.
Apr 24 12:46:13 atom-VirtualBox systemd[1]: atom-worker.service: Scheduled restart job, restart counter is at 3.
Apr 24 12:46:13 atom-VirtualBox systemd[1]: Stopped atom worker.
Apr 24 12:46:13 atom-VirtualBox systemd[1]: atom-worker.service: Start request repeated too quickly.
Apr 24 12:46:13 atom-VirtualBox systemd[1]: atom-worker.service: Failed with result 'exit-code'.
Apr 24 12:46:13 atom-VirtualBox systemd[1]: Failed to start atom worker.

The version of AtoM used is 2.8.1 - 193 and we use all the dependencies that the documentation told us, we also have custom themes and finally we do not add more functionalities.

Thanks for your help.

[Dan Gillean]

Hi Anthony, 

The main error seems to be these lines: 

Apr 24 12:45:42 atom-VirtualBox systemd[1276191]: atom-worker.service: Changing to the requested working directory failed: No such file or directory
Apr 24 12:45:42 atom-VirtualBox systemd[1276191]: atom-worker.service: Failed at step CHDIR spawning /usr/bin/php7.4: No such file or directory

It is trying to change to /usr/bin/php7.4 and not finding a directory by that name, and throwing an error.  

Is it possible you are running a later version of Ubuntu? I am wondering if you have PHP installed, but it's not version 7.4...

There are a couple more things you can check - first, what do you get if you run the following?

• php --version

Next, let's make sure that your atom-worker configuration block is correct. Review the file created during installation at /usr/lib/systemd/system/atom-worker.service, and make sure it matches what is found in the docs: 

• https://www.accesstomemory.org/docs/latest/admin-manual/installation/ubuntu/#deployment-of-workers
  

So: if that looks correct, then I suspect that either you don't have PHP installed, or you have the wrong version of PHP installed. I found a decent guide online that covers how you can install additional PHP versions (it uses PHP 7.3 as an example, but you can change those commands to 7.4), and how you can configure which version will be used by default. See: 

• https://www.thecoachsmb.com/upgrade-downgrade-php-on-ubuntu/
  

Finally, make sure that you properly installed the PHP 7.4 extensions (and not for a different version), as defined in this section of the installation docs: 

• https://www.accesstomemory.org/docs/latest/admin-manual/installation/ubuntu/#php
  

After making sure you have (the right version of) PHP installed and configured, I think you will probably need to run some of the maintenance tasks again, such as: 

• clear cache
• restart PHP-FPM
• restart atom-worker (and if needed, reset the fail counter again)

There are instructions and links for all these tasks on our Troubleshooting page. 

• https://www.accesstomemory.org/docs/latest/admin-manual/maintenance/troubleshooting/
  

I hope this helps! 

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

he / him

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/de47944f-8782-45f8-82cf-3ca415ca154en%40googlegroups.com.