Database errors after crash

[Roman Belenya]

Hi everyone,

In our group we use XNAT version 1.7.6, build 1694. It runs in a virtual machine with the xnat-docker-compose installation.

Yesterday we encountered a problem where the disk space ran out. The virtual machine with all docker containers crashed. We deleted everything from data/build and data/cache/DELETED directories to free up some space. When I restarted the containers, there were some errors in the database logs. Also, the behaviour of the web application is strange.

Here are examples of errors from the log:

xnat-db_1      | 2021-12-16T19:01:51.471862661Z ERROR:  current transaction is aborted, commands ignored until end of transaction block

xnat-db_1      | 2021-12-16T19:01:51.477757699Z STATEMENT:  ALTER TABLE ACTIVEMQ_ACKS ADD PRIMARY KEY (CONTAINER, CLIENT_ID, SUB_NAME, PRIORITY)
xnat-db_1      | 2021-12-16T19:02:15.692233194Z ERROR:  could not read block 3104 in file "base/16385/24327": read only 0 of 8192 bytes

I am attaching the output of docker-compose logs xnat-db and a few screenshots from the web app. After the crash, I can't access any projects, even though I'm an admin. I can still access individual experiments from the recent activity tab. Also, on the subject page, experiment names are displayed incorrectly.

Does anyone have any suggestions on how we can fix this?

Thanks and best wishes,

Roman