Atom worker problem

[Miguel Alfaro]

Hello!

When I install AtoM 2.1 gearman worker and I type "sudo php symfony tools:gearman", gearman-workers looks good, but when i install 2.2 if i type sudo symfony tools:gearman, it show me error and i need to type "php symfony jobs:worker" to activate the worker. 

In other other installation I send my Dip from archivematica(1.3) and i see the DIP in /tmp on the AtoM (2.1) side, but my worker do not take my DIP. Sword Plugin activate, job schulding activate, port 4730 ok.  what am I doing wrong?

thank you!

Miguel Alfaro

[Dan Gillean]

Hi Miguel,

Regarding the first part of your question (2.2), the job scheduler is being used differently in AtoM 2.2 and for more things, so it is possible that the old command won't work. You can find more information about getting the gearman worker going in 2.2 in this page of our admin manual:

• https://www.accesstomemory.org/docs/2.2/admin-manual/installation/asynchronous-jobs/

Regarding 2.1 and DIP upload....

First, based on what you've said, I am assuming you have already checked out the configuration steps listed in these Archivematica pages, yes?

• https://www.archivematica.org/docs/archivematica-1.3/admin-manual/installation/dashboard-config/#atom-dip-upload
• https://www.archivematica.org/docs/archivematica-1.3/user-manual/administer/dashboard-admin/#atom-dip-upload

The 2.1 documentation did not actually include instructions for setting up gearman (since it was only being used for Archivematica DIP upload), but you can find these in the 2.2 documentation and they will work for 2.1 as well... with one change:

• https://www.accesstomemory.org/docs/2.2/admin-manual/installation/asynchronous-jobs/

Note that in the sample upstart service block, it uses symfony jobs:worker in the 2.2 instructions - for 2.1, you would need to substitute this for symfony tools:gearman-worker

Then, remember - if you upgrade to 2.2, you will need to change this back to symfony jobs:worker

I would also suggest checking out the gearman error log to see if there is more useful information found in it about why the DIP upload is failing. If you follow the installation steps outlined in the link above, you will find this error log in:

/usr/share/nginx/atom/log/atom-worker.log

However, if you have NOT followed the above instructions to install gearman in 2.1, then you may be running into another issue - the default location for the gearman log file in AtoM's default atom-worker.conf file does not actually exist, unless you go and create the subdirectories yourself! This file says that logs will go to:

/var/log/nginx/atom/log/atom-worker.log

But, if you haven't created atom/log/ inside of the /var/log/nginx directory manually, or changed the default location of the log file, then your worker may fail!

Here is the default atom-worker.conf file location in AtoM 2.1 - you can find it in the init directory:

• https://github.com/artefactual/atom/blob/stable/2.1.x/init/atom-worker.conf#L10

If this is the case for you, then I would suggest either you change the default location in the atom-worker.conf file, or your go and create the necessary subdirectories in /var/log/nginx/ so that gearman can find the error log location outlined in the atom-worker.conf file.

I hope these tips give you a few leads to follow up on - let us know what you find, and how it goes!

Cheers,

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

--
You received this message because you are subscribed to the Google Groups "ICA-AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To post to this group, send email to ica-ato...@googlegroups.com.
Visit this group at http://groups.google.com/group/ica-atom-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/3e624d9d-18fd-417f-adf9-d0423b644d4e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Miguel Alfaro]

Hi Dan:

Thanks for the reply, but my problem was that when I install ATOM the uploads folder isn't created. I follow  these recommendations: https://groups.google.com/d/msg/ica-atom-users/yYAqctRTKaA/Iv92M5aGUkIJs.

Now it works perfectly.

Thanks

[Dan Gillean]

Oh!

Well, thank you for the update, Miguel. I'm glad to hear you've gotten it working!

Cheers,

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/f91dace9-a99c-4e03-8072-3cd52013c49d%40googlegroups.com.

[rus...@c7a.ca]

Old message, but it helped my installation today.

I followed the instructions on https://www.accesstomemory.org/en/docs/2.3/admin-manual/installation/asynchronous-jobs/#upstart-ubuntu-14-04 to create /etc/init/atom-worker.conf , but it didn't start because the log directory was missing.  It might be useful to add that to the documentation.

sudo mkdir -p /var/log/nginx/atom/log/

sudo chown www-data.www-data /var/log/nginx/atom/log/

I'm now able to upload from Archivematica to AtoM running on the same machine.

BTW: some suggestions for the /administration/dips/atom/ page would have been helpful as well.   Ended up figuring out that I can just use '/tmp' in the rsync target when on the same machine, and that would work fine.  The instructions presumed it would be a different machine, and I would be setting up ssh keys/etc.

We have much more to learn, and will likely have more questions, but we have an install of everything to learn with.

-- 

System Administration and software developer, Canadiana.org http://www.canadiana.ca

Connect with our DevOps team https://github.com/c7a 

Public notes about work: http://mcormond.blogspot.ca/search/label/Canadiana.org

[Dan Gillean]

Hi Russell, 

First of all, thank you for taking the time to look through existing forum threads before posting! I'm happy to hear that you've found a solution. 

Thanks as well for the feedback you've provided. I was under the impression that AtoM would automatically create any missing directories, but this might be incorrect, or dependent on filesystem permissions or other factors - I'm not a developer or a systems administrator, so I'm not sure. I'll share this thread with our team for feedback, and we'll try to add a note to the documentation to help other users avoid this issue. 

Thanks again! 

Regards, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-atom-users@googlegroups.com.
Visit this group at https://groups.google.com/group/ica-atom-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/3834f789-a968-4a95-a537-f61ba428b0d2%40googlegroups.com.

[Dan Gillean]

Hi again, 

One of our system administrators pointed out that the location where you created your log directory  -  /var/log/nginx/atom/log/ -  is actually different from the location outlined in those docs -  /usr/share/nginx/atom/log/, as defined in the upstart service conf block provided in the documentation. 

I'm wondering if the reason the log directory was not automatically created is because of this? As in, the /usr/share/nginx/atom directory is owned by www-data - but you had to explicitly change the permissions to grant access to /var/log/nginx/atom/log/ - perhaps this is why?

Regards, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

[Russell McOrmond]

On Tuesday, August 22, 2017 at 5:49:06 PM UTC-4, Dan Gillean wrote:

Hi again, 

One of our system administrators pointed out that the location where you created your log directory  -  /var/log/nginx/atom/log/ -  is actually different from the location outlined in those docs -  /usr/share/nginx/atom/log/, as defined in the upstart service conf block provided in the documentation. 

Apparently I was wrong about which documentation I used to create /etc/init/atom- worker.conf , as it was https://www.archivematica.org/en/docs/archivematica-1.6/admin-manual/installation/installation/#using-atom-2-x-with-archivematica .  This suggests I "copy the atom- worker.conf file from atom source to /etc/init/".


I started from https://www.accesstomemory.org/en/docs/2.3/admin-manual/installation/linux/ubuntu-trusty/ , and I used "Check out the code from our Git repository", which means I was using https://github.com/artefactual/atom/blob/stable/2.3.x/init/atom-worker.conf which doesn't match the "env LOGFILE=" line from https://www.accesstomemory.org/en/docs/2.3/admin-manual/installation/asynchronous-jobs/#upstart-ubuntu-14-04

For consistency I edited my /etc/init/atom-worker.conf to match the installation page, but it might be useful to update the git repository to match.

Thank you.

General question:

The technical team at Canadiana (I'm the lead sysadmin, and I'm working with our metadata architect and lead application developer) are doing an evaluation of Archivematica and AtoM, with the possibility of collaborating with Artefactual rather than continuing to develop and maintain our own software platform in the areas of overlap.  I am wondering who at Artefactual would be the right person to be in touch with to potentially speed up this evaluation process?   Thus far we planned to do this alone based on public documentation, but thought I would ask if there is a better way to speed us past the initial learning curve.

[Dan Gillean]

Hi Russell, 

Thanks for giving us more details - I agree that we should fix this inconsistency. Sorry it gave you trouble, and thanks for bringing it to our attention. I've filed a ticket here for one of our developers to take a look at: 

• https://projects.artefactual.com/issues/11452
  

Regarding Canadiana - please feel free to follow up with myself off-list! dan [at] artefactual [dot] com. Depending on the nature of your questions or requests, I can can connect you with other team members as needed. 

Regards, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

--

You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.

To post to this group, send email to ica-atom-users@googlegroups.com.

Visit this group at https://groups.google.com/group/ica-atom-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/bf17c27b-f68c-40f5-bdc8-dc7a10e58c9c%40googlegroups.com.

[Dan Gillean]

Hi again Russell, 

Just wanted to let you know that one of our developers has updated the location in our GitHub code repository atom-worker.conf file so it matches the location recommended in the documentation. We've made this change in both our stable/2.3.x branch, and our qa/2.4.x branch: 

• https://projects.artefactual.com/issues/11452
  
• https://github.com/artefactual/atom/blob/stable/2.3.x/init/atom-worker.conf
  
• https://github.com/artefactual/atom/blob/HEAD/init/atom-worker.conf
  

Thanks again for letting us know!

Regards, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

To post to this group, send email to ica-ato...@googlegroups.com.

Visit this group at https://groups.google.com/group/ica-atom-users.

[Russell McOrmond]

On Fri, Aug 25, 2017 at 11:47 AM, Dan Gillean <d...@artefactual.com> wrote:

Hi again Russell, 

Just wanted to let you know that one of our developers has updated the location in our GitHub code repository atom-worker.conf file so it matches the location recommended in the documentation. We've made this change in both our stable/2.3.x branch, and our qa/2.4.x branch: 

Thank you.  We have already upgraded our evaluation environment to qa/2.4.x as 2.4 will likely have been released before we are ready to move anything into production.  Better to learn with the latest.

Another documentation suggestion that might help other people:

I set up both Archivematica and AtoM on the same machine, which required unique domain names be used in the nginx configurations. While some things worked, the ability to do a re-ingest didn't work any more.   Took me a while to realise I needed to update the "Remote name" in the storage server pipeline to use the specific name rather than an IP address (I believe it was using 127.0.0.1, which ended up going to "atom" rather than "dashboard.conf")

While we will likely be running Archivematica and AtoM on different servers in production, I wouldn't be surprised if running on the same server is common for test and training environments.

[Dan Gillean]

Thanks for this, Russell. I've alerted our Archivematica team about this, so hopefully they can add some further clarification to the docs in the near future. 

Regards, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.

To post to this group, send email to ica-atom-users@googlegroups.com.

Visit this group at https://groups.google.com/group/ica-atom-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/CANYAYGhYS%3DGK6_QKqmsGehA95mrSp%3DR6wkJWhG%2BnBBsQdhF4kg%40mail.gmail.com.