Replication verification from filesystem to Postgres

[i...@srgmri.co.nz]

Hi,

I am attempting to migrate my filesystem based DICOM store and index into Postgres. I am using the supplied ImportDicomFiles python script to walk through the directories. However a number are failing for some reason or another, I would say roughly half.

Is there any way of just re-sending the failed studies? I think the reason they are failing is the sheer volume I am trying to transmit.

Is there any way of verifying that all the instances on the file system have made it into the database?

Regards,
Ben Ganley

[Sébastien Jodogne]

Hello,

The first part of your question is answered in the following FAQ:

https://orthanc.chu.ulg.ac.be/book/faq/orthanc-storage.html

"Orthanc also associates each incoming DICOM file with a JSON file that summarizes all its DICOM tags, which speeds up subsequent processing by avoiding a costly DICOM parsing." This explains why your "OrthancStorange" folder contains twice more files than the number of instances that are stored by your Orthanc server.

You can certainly verify that all the instances have been properly replicated by creating a script (e.g. in Python) that makes use of the REST API of Orthanc:

https://orthanc.chu.ulg.ac.be/book/users/rest.html

In a nutshell, the script would iterate over all the DICOM instances of your "Source" Orthanc server, and check whether the "Target" server contains each of them. You could also test whether their MD5 hash matches (cf. the "http://localhost:8042/instances/{...}/attachments/dicom/md5" URI).

HTH,

Sébastien-

[i...@srgmri.co.nz]

Ahh okay, so when the ImportDicomFiles scripts finished running I checked the totals and saw that exactly half of the studies succeeded. This makes sense as above you have said that half the files are JSON.

Perhaps it is worth noting this in https://orthanc.chu.ulg.ac.be/book/users/replication.html#replication
in the direct access to the filesystem section? So people expect half to fail in this instance?
Or perhaps modify the ImportDicomFiles to run
import json
try:
json.loads(data)
except ValueError:
//Post this file as it is not JSON.
This would add a lot of overhead to this script but there may be a better way of doing it.

[Sébastien Jodogne]

Thanks for this interesting suggestion! I have just updated the Orthanc Book accordingly:

https://orthanc.chu.ulg.ac.be/book/users/replication.html#direct-access-to-the-filesystem

I will not modify the default ImportDicomFile.py to keep it as straightforward and generic as possible. Obviously, it is quite easy to fine-tune it to serve one's needs.

Also, don't forget that a standalone GitHub repository is opened to store such user contributions, feel free to issue pull requests therein:

https://github.com/jodogne/OrthancContributed

Regards,

Sébastien-