How to configure AtoM with gearman?

[Shrinivasan T]

Hello AtoM team,

I see the following para needs more details and example configurations.

As per https://www.accesstomemory.org/en/docs/2.2/admin-manual/installation/asynchronous-jobs/

An AtoM worker needs to know where the job server is running, which is defined in an application setting called gearman_job_server under config/app.yml. Whenever you change that setting, make sure that the Symfony cache is refreshed and the worker is restarted.

I installed gearman and atom in same server.
Followed this https://www.accesstomemory.org/en/docs/2.0/admin-manual/installation/linux/

In my config/app.yml file, I dont see any option for gearman_job_server

Give the configuration details for this and commands to refresh symfony cache.

Thanks.

Shrinivasan

[Dan Gillean]

Hi there,

It looks like you've found a place where we've forgotten to update our documentation! Sorry about that!

With AtoM 2.2 and newer versions, the configuration for the gearman server has gotten its own config file, as more operations in AtoM will be managed asynchronously (For example: Finding aid generation and PREMIS rights inheritance in 2.2; CSV search results and Clipboard results export in 2.3, etc)

You will now find the relevant configuration file in config/gearman.yml

Example in our GitHub repo: https://github.com/artefactual/atom/blob/stable/2.2.x/config/gearman.yml

You can clear the application cache with the following command:

• php symfony cc

(https://www.accesstomemory.org/docs/2.2/admin-manual/maintenance/clear-cache/)

Depending on your installation environment, you may want to resatart php5-fpm as well:

• sudo service php5-fpm restart

I will try to update the documentation for the 2.2 and 2.3 branches as soon as possible. Thanks for catching that, and sorry for the confusion!

Cheers,

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

--
You received this message because you are subscribed to the Google Groups "ICA-AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To post to this group, send email to ica-ato...@googlegroups.com.
Visit this group at https://groups.google.com/group/ica-atom-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/ff2f0362-8d66-4d74-938e-41da2bacc794%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Shrinivasan T]

Hi,

Thanks for the reply.

You will now find the relevant configuration file in config/gearman.yml

Example in our GitHub repo: https://github.com/artefactual/atom/blob/stable/2.2.x/config/gearman.yml

I have the same file in /usr/share/nginx/atom/config/gearman.yml

Is this means that we dont need to do anything to configure with gearman?

Update here once the documentations are fixed.

Shrini

[Dan Gillean]

Hi Shrini,

If you have followed our recommended installation instructions, then yes: you should be able to start using AtoM (with Gearman) without having to make any changes to the gearman.yml file.

I'll update the location of the config file in the documentation as soon as I can! :)

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

--

You received this message because you are subscribed to the Google Groups "ICA-AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To post to this group, send email to ica-ato...@googlegroups.com.
Visit this group at https://groups.google.com/group/ica-atom-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/36f6ffda-444d-414e-b871-fe5828253015%40googlegroups.com.

[Jim Adamson]

Hi Shrini, Dan,

This was a very timely thread for us and we now have this working under 2.2 :)

The only thing I had to adjust was the start on (started mysql) line in the sample upstart config file, since our MySql database is hosted remotely. I tried changing this to start on (started nginx) and start on (started php5-fpm) but after rebooting the server the atom-worker process hadn't started:

$status atom-worker

atom-worker stop/waiting

In the end I got it working by using start on (local-filesystems and net-device-up IFACE!=lo) straight out of the Upstart manual. We're using Ubuntu 12.04.5 LTS.

Jim

[Dan Gillean]

Thanks for sharing what worked for you Jim, very useful!

Cheers,

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

--

You received this message because you are subscribed to the Google Groups "ICA-AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To post to this group, send email to ica-ato...@googlegroups.com.
Visit this group at https://groups.google.com/group/ica-atom-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/e56061f1-c967-4f9c-bb78-e8c09c146ec1%40googlegroups.com.

[Shrinivasan T]

Hi Jim,

Thanks for the reply.

My /etc/init/atom-worker.conf

description "AtoM worker (gearmand) upstart service"

start on (started mysql)

start on (local-filesystems and net-device-up IFACE!=lo)

start on (started nginx)
start on (started php5-fpm)
stop on runlevel [016]

respawn
respawn limit 5 10

env LOCATION=/usr/share/nginx/atom
env LOGFILE=/usr/share/nginx/atom/log/atom-worker.log

setuid www-data
setgid www-data

script

  php \
    -d memory_limit=-1 \
    -d error_reporting="E_ALL" \
      ${LOCATION}/symfony tools:gearman-worker >> ${LOGFILE} 2>&1

end script
~             



results:
sudo status atom-worker
atom-worker stop/waiting


sudo restart atom-worker
restart: Unknown instance:

sudo start atom-worker
atom-worker start/running, process 15888

status atom-worker
atom-worker stop/waiting


/usr/share/nginx/atom/log/atom-worker.log

has many lines as   Task "tools:gearman-worker" is not defined. 


But, if I execute as "php symfony jobs:worker" in the folder /usr/share/nginx/atom
it gives as

2016-01-23 16:26:52 > New ability: arGenerateFindingAidJob
2016-01-23 16:26:52 > New ability: arInheritRightsJob
2016-01-23 16:26:52 > New ability: qtSwordPluginWorker
2016-01-23 16:26:52 > Running worker...
2016-01-23 16:26:52 > PID 15963

But it closes If I close the terminal.

Help me on how to make atom to run with gearman.


Thanks a lot.

[Jim Adamson]

Hi Shrini,

I was caught out by the "not defined" lines too. You need to edit atom-worker.conf:

Replace:

${LOCATION}/symfony tools:gearman-worker >> ${LOGFILE} 2>&1

With:

${LOCATION}/symfony jobs:worker >> ${LOGFILE} 2>&1

I think the sample atom-worker.conf in the documentation got updated shortly after you posted your original message.

Hope that helps,

Jim

--
You received this message because you are subscribed to a topic in the Google Groups "ICA-AtoM Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ica-atom-users/2mMKQGNvWdI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ica-atom-user...@googlegroups.com.

To post to this group, send email to ica-ato...@googlegroups.com.
Visit this group at https://groups.google.com/group/ica-atom-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/5702600d-f798-4208-ab1f-e69ffed60481%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Jim Adamson

Systems Administrator/Developer
Collections Services
Information Services

LFA/237 | Harry Fairhurst building | University of York | Heslington | York | YO10 5DD

+44 (0)1904 323859

My calendar

Library Footprints Knowledge Base & Enquiries

Email disclaimer: https://www.york.ac.uk/docs/disclaimer/email.htm

[Shrinivasan T]

Thanks Jim for the fix.

atom-worker is working fine now.

sudo status atom-worker
atom-worker start/running, process 20210


Thanks for the great help.