Speeding up normalization
31 views
Skip to first unread message
Roberto Greiner
Sep 17, 2025, 8:56:18 AMSep 17
to archivematica
Hi,
About normalization when a transfer task is being run in Archivematica.
I'm currently importing a relatively large amount of files to Archivematica and Atom (all are PDF's, about 8.000 files, ~450GB), and noticed a bottleneck. When normalization is running, it is done using 'gs', but only one process is run at a time, and gs is not distributing the taska among multiple CPUs. I have a server whit 12/24 CPU and only one is being used when normalization is running, taking a long time to complete the normalization processes.
Is there any way to make this step take advantage of multiple CPU's? I did a search and found no reference to doing this. It would speed up the import jobs considerably.
Thank you,
Roberto Greiner
About normalization when a transfer task is being run in Archivematica.
I'm currently importing a relatively large amount of files to Archivematica and Atom (all are PDF's, about 8.000 files, ~450GB), and noticed a bottleneck. When normalization is running, it is done using 'gs', but only one process is run at a time, and gs is not distributing the taska among multiple CPUs. I have a server whit 12/24 CPU and only one is being used when normalization is running, taking a long time to complete the normalization processes.
Is there any way to make this step take advantage of multiple CPU's? I did a search and found no reference to doing this. It would speed up the import jobs considerably.
Thank you,
Roberto Greiner
--
-----------------------------------------------------
Marcos Roberto Greiner
Os otimistas acham que estamos no melhor dos mundos
Os pessimistas tem medo de que isto seja verdade
James Branch Cabell
-----------------------------------------------------
Marcos Roberto Greiner
Os otimistas acham que estamos no melhor dos mundos
Os pessimistas tem medo de que isto seja verdade
James Branch Cabell
-----------------------------------------------------
Santiago Rodríguez Collazo
Sep 18, 2025, 3:14:57 AMSep 18
to archiv...@googlegroups.com
Hi Roberto
Deploying more MCPClients ( as explained in https://www.archivematica.org/en/docs/archivematica-1.17/admin-manual/installation-setup/customization/scaling-archivematica/#deploy-multiple-mcpclients ) will help engage all the available CPUs
/santi
--
You received this message because you are subscribed to the Google Groups "archivematica" group.
To unsubscribe from this group and stop receiving emails from it, send an email to archivematic...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/archivematica/CAJgdVFsojrbkCbMhQZ%2B94s%3DuMggyr7Ao%3D3vR-_G%2Birr7%3Du2zTQ%40mail.gmail.com.
Roberto Greiner
Sep 18, 2025, 6:58:01 AMSep 18
to archiv...@googlegroups.com
Great!
I will read the document and implement it as soon as possible.
Tks!
To view this discussion visit https://groups.google.com/d/msgid/archivematica/CAMpnCy8Lg8FoHSVZeQoOaO9v50qML7k_Dv1F6ubS0YDcCwF%3DwQ%40mail.gmail.com.
Reply all
Reply to author
Forward
0 new messages