Normalization errors with ""An error occurred in the current transaction. You can't execute queries until the end of the 'atomic' block.""

[arth...@gmail.com]

Hi all,

I'm struggling with a possible nasty bug and, strange as it is, what I describe below is actually happening in an installation of AM 1.11.2 in RHEL7.8.

During a routine check I discovered a series of Normalization Failures (Dashboard interface, Administration/Failures menu).
Investigating I found that in the case of packages with more than (I believe) 128 TIFF files, normalization is done correctly for the first 128 files, thus producing 128 normalized files. For all remaining files the normalization fails with this type of message
"An error occurred in the current transaction. You can't execute queries until the end of the 'atomic' block."
This happens both for the generation of the normalizations of the AIPs and for the DIPs.
For example, if an incoming package has 300 TIFF files, this is what is produced:
AIP: 428 TIFF files, the original 300 and only 128 TIFF normalizations.
DIP: Only 128 JPEG normalizations (and only these are then loaded into AtoM! The other are lost!).

Unfortunately I have no way to recover the MCPServer and MCPClient logs because they have already been deleted from the system.
I randomly verified a good number of Transfer/Ingest and all of them follow this "bug model".

Googling I found this which appears to be related to a Django problem in the most recent versions.
However, the coincidence regarding the maximum number of normalizations carried out seems strange to me: 128.
Is it perhaps some parameterization that I missed?

I thank in advance anyone who can give me useful information for the solution of this problem.

Arthy

[370136847]

您好！您的邮件我已收到！谢谢！

[Sean Kalynuk]

Hi Arthy,

 

Are you running more than one MCP Client?

 

My guess is you have run into the bug I reported here:

 

https://github.com/archivematica/Issues/issues/752

 

The clue to me here is that 128 is the default batch size, so one batch completes, but others fail.

 

--

Sean

 

From: archiv...@googlegroups.com <archiv...@googlegroups.com> on behalf of arth...@gmail.com <arth...@gmail.com>
Date: Tuesday, December 28, 2021 at 5:50 AM
To: archivematica <archiv...@googlegroups.com>
Subject: [archivematica] Normalization errors with ""An error occurred in the current transaction. You can't execute queries until the end of the 'atomic' block.""

Caution: This message was sent from outside the University of Manitoba.

--
You received this message because you are subscribed to the Google Groups "archivematica" group.
To unsubscribe from this group and stop receiving emails from it, send an email to archivematic...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/archivematica/a2fd4a8a-913e-49a9-8dd5-f8635e11f440n%40googlegroups.com.

[arth...@gmail.com]

Hi Sean,

I think so too and in fact it is related to a completely similar problem already raised by Jorik van Kemenade and by me too in #1111 wich is in Refining Status.

I didn't realize that #752 is unfortunately still in the Open state.

arthy

[Sean Kalynuk]

Hi Arthy,

 

After reading #1111, I’ll also share a separate issue I reported related to database update timing issues and how Ingest can be started too early if you have thousands of files.

 

https://github.com/archivematica/Issues/issues/759

To view this discussion on the web visit https://groups.google.com/d/msgid/archivematica/df4d950c-7dbb-4709-88d5-a7bea4efec9cn%40googlegroups.com.