Question about "StorageAccessOnFind" : "Answers", "Never", "Always"

[Stephen Douglas Scotti]

I just upgraded to the Osimis 23.2.0 image, Orthanc 1.11.3.

I was playing around with the StorageAccessOnFind setting.  I've always had that set to "Answers", but I changed it to "Never" just to see what would happen and I notice that some of the series that I have with PDF's in them do not show up when I change the setting to "Never".

The series in question are of modality type "DOC", and they have encapsulated PDF's as instances.  The thumbnails have a slashed - eye icon with the "Never" setting, and and when I drag the icon to the viewport nothing happens.  When I change the setting back to "Answers" it works fine again.

I can dig into it a little more, but just wondering why that might be happening ?  Do I need to perhaps define some MainDicomtags and then run the HouseKeeper in order for those PDF's to show up with "Never" setting.  I presume that I would get faster performance if it worked that way ?

Stephen D. Scotti

[Stephen Douglas Scotti]

I think they are actually of "OT" type, and the extra tags seem to be mostly these:

0042,0011 - EncapsulatedDocument:

0042,0012 - MIMETypeOfEncapsulatedDoc

/sds

[Stephen Douglas Scotti]

Follow up question about implementing the HouseKeeper with an instance that did not have it before.

I enabled the housekeeper and added the recommended set of ExtraMainDicomTags and tried restarting.  It looks like the Housekeeper by default uses the default location for the OrthancStorage:

/var/lib/orthanc/db:Z

On this page in the book:  https://book.orthanc-server.com/users/docker-osimis.html#default-configuration

it mentions:

Orthanc and each plugin might have some default settings that might eventually be different from the defaults included in the Orthanc executable or the plugin library.

Orthanc non-standard defaults:

{

"StorageDirectory" : "/var/lib/orthanc/db",

"RemoteAccessAllowed": true,

"AuthenticationEnabled": true,

 "HttpsCACertificates" : "/etc/ssl/certs/ca-certificates.crt",

"Plugins" : ["/usr/share/orthanc/plugins/"]

}

I actually had something different previously, so the Housekeeper failed to find the resources to reconstruct because it is apparently hard coded to look there and not in the location specified in the orthanc.json ?

I changed my docker compose and my configuration to map it to the default location in the Docker Container and I think it 'works' and I also no longer see that issue with the PDF's not displaying in the Stone Viewer.  I am presuming that one of those tags maybe fixes it ?

W0310 16:55:10.063446 PluginsManager.cpp:157] Starting Housekeeper worker thread
W0310 16:55:10.084835 PluginsManager.cpp:157] Housekeeper: your storage might still contain some dicom-as-json files -> will perform housekeeping
W0310 16:55:10.085980 PluginsManager.cpp:157] Housekeeper: the DB configuration has changed since last run, will reprocess the whole DB !
W0310 16:55:10.156017 ServerToolbox.cpp:259] Reconstructing resource 11818585-0511df9d-a82b2632-c0956624-dc0f3e76
W0310 16:55:16.582057 ServerToolbox.cpp:259] Reconstructing resource 6e5a8118-f48298d2-397c6d1f-f8dd022f-5fb02d2d
W0310 16:55:21.820040 ServerToolbox.cpp:259] Reconstructing resource 3f01062a-4ca86d1a-24e6c4c2-4da48650-2f948c25
W0310 16:55:26.840274 ServerToolbox.cpp:259] Reconstructing resource a393b54b-8be40ca9-66790f82-a19697f9-eda8c6da


Although I am a little hesitant to deploy it just yet on a production version.  Just wondering if there is a way to specify the location for the Storage Directory for the HouseKeeper, or does it require it to be the default location ?  I didn't really see source code for the Housekeeper Plug-in in the repo, like maybe an environment variable or mapping from the orthanc.json config ?

    "ExtraMainDicomTags" : {
    "Instance" : [
        "Rows",
        "Columns",
        "ImageType",
        "SOPClassUID",
        "ContentDate",
        "ContentTime",
        "FrameOfReferenceUID",
        "PixelSpacing",
        "SpecificCharacterSet",
        "BitsAllocated",
        "BitsStored"
    ],
    "Series" : [
        "TimezoneOffsetFromUTC",
        "PerformedProcedureStepStartDate",
        "PerformedProcedureStepStartTime"
    ],
    "Study": [
        "TimezoneOffsetFromUTC"
    ],
    "Patient": []
    },

. . . .

. . . .

/sds

[Stephen Douglas Scotti]

I was able to upgrade and run the Housekeeper on one of my populated development instances with a 208 studies on it.  It did not take that long really, like maybe 90 minutes with a 5 second throttle delay.

For the most part that seems to have worked, but with a few warnings and errors, e.g.

W0312 07:48:56.842954 PluginsManager.cpp:157] Housekeeper: your storage might still contain some dicom-as-json files -> will perform housekeeping
W0312 07:48:56.842974 PluginsManager.cpp:157] Housekeeper: Study main dicom tags have changed, -> will perform housekeeping
W0312 07:48:56.842983 PluginsManager.cpp:157] Housekeeper: Series main dicom tags have changed, -> will perform housekeeping
W0312 07:48:56.842990 PluginsManager.cpp:157] Housekeeper: Instance main dicom tags have changed, -> will perform housekeeping
W0312 07:48:56.842997 PluginsManager.cpp:157] Housekeeper: the DB configuration has changed since last run, will reprocess the whole DB !
W0312 07:48:57.132879 ServerToolbox.cpp:259] Reconstructing resource c88c15f5-6c2eddaf-7fc3cf72-149b63e5-d3553f9e
W0312 07:49:05.379419 ServerToolbox.cpp:259] Reconstructing resource 4376185a-280d89fd-0848ec74-fc4769fa-1d0ed6c7
W0312 07:49:13.092274 ServerToolbox.cpp:259] Reconstructing resource 4ae8d3ad-749abf08-6c30b089-c3a7c09b-5be8d54f
W0312 07:49:37.649298 ServerToolbox.cpp:259] Reconstructing resource 3c961f6f-3fc282fb-5261da2a-1d8196ac-efacde45
W0312 07:49:42.950616 ServerToolbox.cpp:259] Reconstructing resource f94ee238-3ca423bf-9ffdff8b-ea26a8f5-6db5631f
W0312 07:49:49.467824 ServerToolbox.cpp:259] Reconstructing resource c8cf78c5-c6cee467-27a2b379-16d88500-a5162b82
W0312 07:49:54.668209 ServerToolbox.cpp:259] Reconstructing resource ac56a206-39789ffc-ec096792-ac465dac-0e3c416a
E0312 07:50:37.068147 PluginsManager.cpp:153] PostgreSQL error: ERROR:  could not serialize access due to read/write dependencies among transactions
DETAIL:  Reason code: Canceled on identification as a pivot, during commit attempt.
HINT:  The transaction might succeed if retried.

E0312 07:50:37.072046 ServerContext.cpp:290] Cannot serialize the jobs engine: Error with the database engine

W0312 07:50:39.153579 ServerToolbox.cpp:259] Reconstructing resource 9dd82f92-c9e9c114-452c23ea-adce9e1b-5f740dd9

. . .

W0312 08:35:06.985734 ServerContext.cpp:2438] W002: study has been stored with another version of Main Dicom Tags list, you should POST to /studies/035d979a-b747accf-5c6062c1-bf8fb5e0-e857b2a4/reconstruct to update the list of tags saved in DB.  Some MainDicomTags might be missing from this answer.
W0312 08:35:06.996774 ServerContext.cpp:2438] W002: study has been stored with another version of Main Dicom Tags list, you should POST to /studies/046dc1bf-8952cdfd-50a7d6df-a846ea9b-c40714bf/reconstruct to update the list of tags saved in DB.  Some MainDicomTags might be missing from this answer.

..

W0312 08:43:28.383196 ServerToolbox.cpp:259] Reconstructing resource f55579a9-f511738e-254c1142-dde9981d-88e6be33
W0312 08:43:33.717292 ServerToolbox.cpp:259] Reconstructing resource 74170bd9-44bcf3f4-836302b5-14950e5e-74c70ace
W0312 08:43:49.013968 ServerToolbox.cpp:259] Reconstructing resource bc17714d-4ffebb59-6b2e623a-132bb248-75ea8226
W0312 08:43:56.782957 ServerToolbox.cpp:259] Reconstructing resource e0099f12-302dce0c-5882e930-ab12a514-919efdd2
W0312 08:44:34.507525 PluginsManager.cpp:157] Housekeeper: finished processing all changes

I will have to run through the studies and view them also with the Stone Viewer, but it seems like everything updated and there does not appear to be an issue in the Stone Viewer with PDF's.

"StorageAccessOnFind" : "Never",

I suspect that maybe that was related to "SOPClassUID" now being available in the DB ?

Just wondering if I need to worry about the erros and warnings that I got.  There are quite a few with the you should POST to /studies/ ..., but I'm wondering if that is being done automatically now because I have the most recent version of Orthanc now, 1.11.3, and that supposed does a reconstruction after a modification ?

Stephen D. Scotti, MD

[Alain Mazy]

Hi Stephen,

Trying to answer a few questions together.

The Housekeeper plugin does not need to know what your storage directory is.  You should even be able to use it with an object-storage plugin.

Concerning the message "study has been stored with another version of Main Dicom Tags list, you should POST to /studies/046dc1bf-8952cdfd-50a7d6df-a846ea9b-c40714bf/reconstruct to update the list of tags saved in DB.  Some MainDicomTags might be missing from this answer.": This should not happen anymore once you've run the Housekeeper on all studies (the Housekeeper actually calls the /reconstruct route for you).  You may try to call /reconstruct manually on this specific study and check if the error happens again.

Concerning the message "PostgreSQL error: ERROR:  could not serialize access due to read/write dependencies among transactions": This is actually not an error but a warning.  The transaction has very likely been retried right after (I added a TODO to clarify that log).

HTH,

Alain.

--
You received this message because you are subscribed to the Google Groups "Orthanc Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to orthanc-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/orthanc-users/33007478-744e-4107-a333-221e4ba807f6n%40googlegroups.com.