Too many files for job Change object and directory filenames

14 views

Skip to first unread message

Sietse Rispens

unread,

Sep 17, 2026, 12:03:30 PM (9 days ago) Sep 17

to archivematica

Hi everyone! I have a question about required resources for transferring a package with very many files. We got a failed transfer for a package with 82,177 files.

I understood from the faq question 7 (FAQ | Documentation (Archivematica 1.17.1) | Archivematica: open-source digital preservation system) that it should be possible to handle 100,000 files.

Our error occurred in the job "Change object and directory filenames". We did not get an error message from the job itself, which reported exit=0; code=success. However, the reported STDOUT in the log is incomplete, only a first part of the files that were processed are listed, the number differs per run with the same input.

Gearman reports an error: archivematica.mcp.server.jobs.tasks:gearman_backend:update_task_results:238: Gearman task batch cf5b4105-4b3e-44a4-9af0-b26b152ec6f3 failed to execute

We have reproduced the problem with a test package containing 50,000 files (successful ingestion) and with 100,000 files (failed transfer). This test set has no file name conflicts or other issues. The problem really seems to be the large number of files.