Pedido de Suporte

[miguelmonteiro@scml]

Good Afternoon.

The AtoM system has been in use in the Historical Archives of Santa Casa da Misericórdia de Lisboa since 2012 and 320,536 descriptive records have already been introduced, some of which with digital objects associated with the records (around 3,720 thumbnails).

 

We are using version 2.4. of the software and, for some time now, we've been experiencing several problems related to a set of jobs that the AtoM no longer performs.

In other words, these tasks remain indefinitely in the list of jobs to be executed, without actually being completed. These AtoM jobs are very important work tools for the describing historical documentation work that we carry out daily. We refer, in concrete, to three types of tasks:

1- "Move" descriptive records from one hierarchical level to another.

2- "Explore reports".

3- "Update publication status"

 

1/ Job MOVE DESCRIPTIVE RECORDS. this task is not being performed by the system. This task is paramount to place descriptive records already made in another open source system (Arqbase) and which are still to be integrated in the hierarchy of the respective document series.

 

2/ Job "EXPLORE REPORTS" of archival units descending from a higher level of description. Inability to respond by the application (with reports ordered to be executed since september 2019 and not yet concluded). We have choose higher levels with few dependent records, so the amount or "weight" of data to process is negligible. This task is also essential to correct any errors in the hierarchy of records

 

3/ Job: "UPDATE PUBLICATION STATUS" of the descriptive records (upper and lower) from "preliminary" to "published", and so on, and to identify the descriptions that can be consulted by researchers, those that have confidential elements and should not be accessed.

 

We send you two Pdf files from the "manage jobs" list, in which you can see that most of the tasks requested (several days ago) are not yet completed.

In the atom system we have the gearman installed and the atom-worker service avaiable, but for some reason some tasks end with success, but another tasks never ends.

We have a server with atom application and they have 8Gbs and 4 vCores.

Can we get help?

Greetings.

Miguel

[Dan Gillean]

Hi Miguel,

Occasionally with very large tasks, the job scheduler can get caught in a loop - I think it happens when the available system memory has been exhausted by part of the task. Because you have so many jobs that are backed up at this point, unfortunately I think the best thing to do will be to clear the entire jobs queue. Once you have everything working as expected, you can then re-launch the jobs you still need. 

To clear all jobs currently in the queue, run: 

• php symfony jobs:clear

At this time, there is no way to clear ONLY a stalled job. We would love to add this ability in the future, but it will likely require analysis and development support from our community to be able to implement it. If your institution might be interested in sponsoring this work, please feel free to contact me off-list. 

Now, before we restart the job scheduler, in the 2.5 release we included a new atom-worker configuration block, that should also prevent the job scheduler from dying as often. See: 

• https://www.accesstomemory.org/docs/latest/admin-manual/installation/asynchronous-jobs/#systemd-ubuntu-16-04-18-04 

Unfortunately, I'm not sure that this will work in Ubuntu 14.04, since Ubuntu moved from Upstart to Systemd as the service daemon. You could try researching what is possible - the most important variables we added to the config block are: 

• StartLimitIntervalSec=24h
• StartLimitBurst=3
  
• Restart=on-failure
• RestartSec=30
  

Otherwise, once the atom-worker queue is fully cleared, you can try restarting the job scheduler, and restarting a new job. I recommend that you just try running one at a time for now, to confirm that things are working as expected. 

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/6cf12389-10f7-4d7e-bb9d-74cb37ce6c35%40googlegroups.com.