C-Find is slow

[Jalal Al-Mussawi]

Testing c-find queries on orthanc and dcm4chee using

same dataset

same database (postgresql)

same resources

same dicom included fields

i get 350-450ms loading about 100 studies on orthanc

i get 30-40ms loading the same dataset on dcm4chee

using weasis rs for loading images directly from api (CT with 1200 images) 

i have to wait 10-12 seconds!! on orthanc

barely one second on dcm4chee 

same thing loading studies directly from dicom viewer (RADIANT)

i wait almost 7-10 seconds to finish c-find on orthanc

one second on dcm4chee

[Jalal Al-Mussawi]

my orthanc configuration

{
    "Name": "sPACS",
    "StorageDirectory": "C:\\sPACS",
    "IndexDirectory": "C:\\sPACS",
    "StorageCompression": false,
    "MaximumStorageSize": 0,
    "MaximumStorageCacheSize": 1000,
    "Plugins": [
        "C:\\Program Files\\Orthanc Server\\Plugins"
    ],
    "ConcurrentJobs": 0,
    "HttpServerEnabled": true,
    "OrthancExplorerEnabled": true,
    "HttpPort": 8081,
    "HttpCompressionEnabled": true,
    "WebDavEnabled": false,
    "WebDavDeleteAllowed": false,
    "WebDavUploadAllowed": true,
    "DicomAet": "SPACS",
    "DicomCheckCalledAet": true,
    "DicomPort": 104,
    "RemoteAccessAllowed": true,
    "AuthenticationEnabled": true,
    "RegisteredUsers": {
        "user": "password"
        },
    "DicomAlwaysAllowEcho": true,
    "DicomAlwaysAllowStore": true,
    "DicomAlwaysAllowFind": false,
    "DicomAlwaysAllowFindWorklist": false,
    "DicomAlwaysAllowGet": false,
    "DicomAlwaysAllowMove": false,
    "DicomCheckModalityHost": false,
    "DicomModalities": {
        "untrusted": {
            "AET": "RADIANT",
            "Port": 11112,
            "Host": "127.0.0.1",
            "Manufacturer": "Generic",
            "AllowEcho": true,
            "AllowFind": true,
            "AllowFindWorklist": false,
            "AllowGet": true,
            "AllowMove": false,
            "AllowStore": true,
            "AllowStorageCommitment": true,
            "AllowTranscoding": true,
            "UseDicomTls": false,
            "LocalAet": "SPACS",
            "Timeout": 60
        }
    },
    "DicomThreadsCount": 8,
    "LimitFindResults": 100,
    "LimitFindInstances": 0,
    "SaveJobs": false,
    "StorageAccessOnFind": "Never",
    "ExtraMainDicomTags": {
        "Instance": [
            "0008,0016",
            "0008,0201",
            "0028,0010",
            "0028,0011",
            "0028,0100",
            "0040,0244",
            "0040,0245",
            "0040,0275"
        ],
        "Series": [
            "0008,0201",
            "0040,0244",
            "0040,0245",
            "0040,0275"
        ],
        "Study": [
            "0008,0201",
            "0010,0021",
            "0008,1040",
            "0008,1050",
            "0008,1060"
        ],
        "Patient": [
        ]

[Bartosz Kaczyński]

I have similar experience but on Linux platform and comparing to Conquest DICOM running also on Linux.

I've running Orthanc on Ubuntu 20.04 using standalone Debian repository with PostgreSQL plugin with disabled EnableStorage option according to scalability recommendations. My setup also includes installation OS and containing database files on separate SAS SSD disks, DICOM files on large SAS HDDs.

On both Orthanc and Conquest I've similar datasets with more than 2 TB DICOM files. On Horos client without input constraints (patient name, study description), just last 3 months I receive responds of available studies from Conquest in less than 2 seconds.  In opposite of Conquest querying Orthanc with the same data takes more than 30 seconds.

Is there any field to improve this time? I haven't tested option PostgreSQL plugin with EnableStorage option set to true or MySQL plugin at all.

I noticed in Orthanc logs (-verbose flag) that during large queries every single attachment is scanned (instead of querying directly database?):

[...]

I0725 23:54:55.466161 main.cpp:224] No limit on the number of C-FIND results at the Patient, Study and Series levels
I0725 23:54:55.466173 main.cpp:234] No limit on the number of C-FIND results at the Instance level
I0725 23:54:55.466530 OrthancFindRequestHandler.cpp:653] (dicom) DICOM C-Find request at level: Study
I0725 23:54:55.466564 OrthancFindRequestHandler.cpp:659] (dicom)   (0008,0005)  SpecificCharacterSet = ISO_IR 100
I0725 23:54:55.466580 OrthancFindRequestHandler.cpp:659] (dicom)   (0008,0020)  StudyDate = *** POTENTIAL PHI ***
I0725 23:54:55.466593 OrthancFindRequestHandler.cpp:659] (dicom)   (0008,0030)  StudyTime =
I0725 23:54:55.466606 OrthancFindRequestHandler.cpp:659] (dicom)   (0008,0050)  AccessionNumber =
I0725 23:54:55.466620 OrthancFindRequestHandler.cpp:659] (dicom)   (0008,0052)  QueryRetrieveLevel = STUDY
I0725 23:54:55.466633 OrthancFindRequestHandler.cpp:659] (dicom)   (0008,0061)  ModalitiesInStudy =
I0725 23:54:55.466646 OrthancFindRequestHandler.cpp:659] (dicom)   (0008,0080)  InstitutionName =
I0725 23:54:55.466658 OrthancFindRequestHandler.cpp:659] (dicom)   (0008,0090)  ReferringPhysicianName =
I0725 23:54:55.466712 OrthancFindRequestHandler.cpp:659] (dicom)   (0008,1030)  StudyDescription =
I0725 23:54:55.466726 OrthancFindRequestHandler.cpp:659] (dicom)   (0008,1050)  PerformingPhysicianName =
I0725 23:54:55.466738 OrthancFindRequestHandler.cpp:659] (dicom)   (0010,0010)  PatientName =
I0725 23:54:55.466751 OrthancFindRequestHandler.cpp:659] (dicom)   (0010,0020)  PatientID =
I0725 23:54:55.466763 OrthancFindRequestHandler.cpp:659] (dicom)   (0010,0030)  PatientBirthDate =
I0725 23:54:55.466775 OrthancFindRequestHandler.cpp:659] (dicom)   (0020,000d)  StudyInstanceUID =
I0725 23:54:55.466787 OrthancFindRequestHandler.cpp:659] (dicom)   (0020,0010)  StudyID =
I0725 23:54:55.466800 OrthancFindRequestHandler.cpp:659] (dicom)   (0020,1208)  NumberOfStudyRelatedInstances =
I0725 23:54:55.466812 OrthancFindRequestHandler.cpp:659] (dicom)   (0032,4000)  RETIRED_StudyComments =
I0725 23:54:55.466824 OrthancFindRequestHandler.cpp:659] (dicom)   (4008,0212)  RETIRED_InterpretationStatusID =
I0725 23:54:55.804233 ServerContext.cpp:1376] Number of candidate resources after fast DB filtering on main DICOM tags: 4507
I0725 23:54:55.816137 FilesystemStorage.cpp:176] Reading attachment "0f0df7c7-7fa5-4d5e-a89f-ee0717aa3c10" of "DICOM" content type (range from 0 to 36474)
I0725 23:54:55.836059 FilesystemStorage.cpp:176] Reading attachment "65ab2bf4-a7f6-463f-8b05-a82fe12ed217" of "DICOM" content type (range from 0 to 1716)
I0725 23:54:55.850097 FilesystemStorage.cpp:176] Reading attachment "c6392261-2c9b-4f49-a4e5-68a375693a66" of "DICOM" content type (range from 0 to 36468)
I0725 23:54:55.859142 FilesystemStorage.cpp:176] Reading attachment "e213eb01-54d2-46c4-ac2c-631515a5755e" of "DICOM" content type (range from 0 to 1716)
I0725 23:54:55.872066 FilesystemStorage.cpp:176] Reading attachment "2fcd68bf-7132-4ba3-8067-8ca2571420f0" of "DICOM" content type (range from 0 to 1730)
I0725 23:54:55.879136 FilesystemStorage.cpp:176] Reading attachment "85635742-6697-46b5-bbe2-d400513e2460" of "DICOM" content type (range from 0 to 36506)
I0725 23:54:55.887633 FilesystemStorage.cpp:176] Reading attachment "ab290b34-1d29-4d90-aa42-e47ad4ba1fa7" of "DICOM" content type (range from 0 to 1664)
I0725 23:54:55.900652 FilesystemStorage.cpp:176] Reading attachment "18ae46c8-7326-4b6a-a50b-30aedb10d84d" of "DICOM" content type (range from 0 to 1758)
I0725 23:54:55.907874 FilesystemStorage.cpp:176] Reading attachment "7129b9c8-e272-475a-8097-fb11d22467a3" of "DICOM" content type (range from 0 to 1706)
I0725 23:54:55.914963 FilesystemStorage.cpp:176] Reading attachment "399a7535-2ef2-4d1a-b1ee-453b852c7504" of "DICOM" content type (range from 0 to 1702)
I0725 23:54:55.925303 FilesystemStorage.cpp:176] Reading attachment "11de19d9-05b3-4811-bccc-38b30777ecfa" of "DICOM" content type (range from 0 to 1730)

[...]

My settings below:

orthanc.json (rest is default

{
    "Name" : "MY_ORTHANC",
    "StorageDirectory" : "/dicom",
    "IndexDirectory" : "/dicom",
    "Plugins" : [
        "/usr/share/orthanc/plugins/"
    ],
    "ConcurrentJobs" : 4,
    "WebDavEnabled" : false,
    "DicomAet" : "MY_ORTHANC,
    "DicomPort" : 4006,
    "DefaultEncoding" : "Utf8",
    "RemoteAccessAllowed" : true,
    "AuthenticationEnabled" : true,
    "DicomThreadsCount" : 8,
    "OverwriteInstances" : true,
    "DicomScpTimeout" : 400,
    "DicomScuTimeout" : 400,
}

postgresql.json

{
    "PostgreSQL" : {
        "EnableIndex" : true,
        "EnableStorage" : false,
        "Host" : "localhost",
        "Port" : 5432,
        "Database" : "orthanc_db",
        "Username" : "orthanc_user",
        "Password" : "my_password",
        "Lock" : true,
        "EnableSsl" : false,
        "MaximumConnectionRetries" : 10,
        "ConnectionRetryInterval" : 5,
        "IndexConnectionsCount" : 1
    }
}

[Bartosz Kaczyński]

Sorry, I discovered StorageAccessOnFind should be set to Never to prevent reading attachments on storage.

The log is different and time for response is slightly faster: from ~30 seconds with default option Always to ~25 seconds with almost 5k studies.

[Jalal Al-Mussawi]

i followed every single word in the scalability of orthanc article but still having the loading time problem,

im thinking this is not configuration related issue but orthanc issue or limitation,

i spent weeks trying and testing, they should put "orthanc is ment for small clinics" somewhere in the documentation

[Alain Mazy]

Hi Bartos and Jalal,

At least for Bartos, your client seems to request tags that are not in DB.  Have you tried setting the ExtraMainDicomTags to avoid accessing the Storage ?

If setting AccessOnFind to "Never" does not help (as mentioned by Jalal here), please provide sample queries and data that could help me reproduce the issues.

Best regards,

Alain.

--
You received this message because you are subscribed to the Google Groups "Orthanc Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to orthanc-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/orthanc-users/5e905541-99fa-436d-bf58-9afe24e2021dn%40googlegroups.com.

[Jalal Al-Mussawi]

thanks for responding

this is a piece of the verbose log file when trying c-find from radiant viewer

I0802 18:04:50.218049 CommandDispatcher.cpp:333] (dicom) Association Received from AET RADIANT on IP 172.16.30.165
I0802 18:04:50.233617 main.cpp:317] Incoming connection from AET RADIANT on IP 172.16.30.165, calling AET DCM4CHEE
I0802 18:04:50.233617 CommandDispatcher.cpp:664] (dicom) Association Acknowledged (Max Send PDV: 16372) to AET RADIANT on IP 172.16.30.165
I0802 18:04:50.233617 main.cpp:352] Incoming Find request from AET RADIANT on IP 172.16.30.165, calling AET DCM4CHEE
I0802 18:04:50.233617 main.cpp:228] Maximum 100 results for C-FIND queries at the Patient, Study and Series levels
I0802 18:04:50.233617 main.cpp:238] Maximum 2000 instances will be returned for C-FIND queries at the Instance level
I0802 18:04:50.233617 OrthancFindRequestHandler.cpp:395] (dicom) DICOM C-Find request at level: Study
I0802 18:04:50.233617 OrthancFindRequestHandler.cpp:401] (dicom)   (0008,0020)  StudyDate =
I0802 18:04:50.233617 OrthancFindRequestHandler.cpp:401] (dicom)   (0008,0030)  StudyTime =
I0802 18:04:50.233617 OrthancFindRequestHandler.cpp:401] (dicom)   (0008,0050)  AccessionNumber =
I0802 18:04:50.233617 OrthancFindRequestHandler.cpp:401] (dicom)   (0008,0052)  QueryRetrieveLevel = STUDY
I0802 18:04:50.233617 OrthancFindRequestHandler.cpp:401] (dicom)   (0008,0056)  InstanceAvailability =
I0802 18:04:50.233617 OrthancFindRequestHandler.cpp:401] (dicom)   (0008,0061)  ModalitiesInStudy =
I0802 18:04:50.233617 OrthancFindRequestHandler.cpp:401] (dicom)   (0008,0080)  InstitutionName =
I0802 18:04:50.233617 OrthancFindRequestHandler.cpp:401] (dicom)   (0008,0090)  ReferringPhysicianName =
I0802 18:04:50.233617 OrthancFindRequestHandler.cpp:401] (dicom)   (0008,1030)  StudyDescription =
I0802 18:04:50.233617 OrthancFindRequestHandler.cpp:401] (dicom)   (0008,1040)  InstitutionalDepartmentName =
I0802 18:04:50.233617 OrthancFindRequestHandler.cpp:401] (dicom)   (0008,1050)  PerformingPhysicianName =
I0802 18:04:50.233617 OrthancFindRequestHandler.cpp:401] (dicom)   (0008,1060)  NameOfPhysiciansReadingStudy =
I0802 18:04:50.233617 OrthancFindRequestHandler.cpp:401] (dicom)   (0010,0010)  PatientName =
I0802 18:04:50.233617 OrthancFindRequestHandler.cpp:401] (dicom)   (0010,0020)  PatientID =
I0802 18:04:50.233617 OrthancFindRequestHandler.cpp:401] (dicom)   (0010,0030)  PatientBirthDate =
I0802 18:04:50.233617 OrthancFindRequestHandler.cpp:401] (dicom)   (0010,0040)  PatientSex =
I0802 18:04:50.233617 OrthancFindRequestHandler.cpp:401] (dicom)   (0020,000d)  StudyInstanceUID =
I0802 18:04:50.233617 OrthancFindRequestHandler.cpp:401] (dicom)   (0020,0010)  StudyID =
I0802 18:04:50.233617 OrthancFindRequestHandler.cpp:401] (dicom)   (0020,1208)  NumberOfStudyRelatedInstances =
I0802 18:04:50.671128 ServerContext.cpp:1426] Number of candidate resources after fast DB filtering on main DICOM tags: 101
I0802 18:05:11.140538 ServerContext.cpp:1576] Number of matching resources: 100
W0802 18:05:11.171790 FindScp.cpp:342] Too many results for an incoming C-FIND query
I0802 18:05:11.187415 CommandDispatcher.cpp:929] (dicom) Finishing association with AET RADIANT on IP 172.16.30.165: DUL Peer Requested Release
I0802 18:05:11.187415 CommandDispatcher.cpp:937] (dicom) Association Release with AET RADIANT on IP 172.16.30.165

please note the timing it takes about 20 seconds

[M.R. Dijkhuizen]

Hi All,

I had the same problems with my Orthanc installation. 

To fix this I tweaked the postgres.conf file and added the items below.

Also check the postgres logs for any errors.

This config is based on a Windows Based postgres installation. If memory serves me right the Linux install also need to change te pagesize which is 4 KB by default

https://www.percona.com/blog/why-linux-hugepages-are-super-important-for-database-servers-a-case-with-postgresql/

Specs of the machine.

4 Cores 

16 GB Ram

The postgres DB is about 100 GB

Orthanc is hosting about 28000 studies 

Database storage is hosted on SSD/NVME storage. 

I also added "IndexConnectionsCount" : 75, to the orhanc postgresql.json 

Please note that the max_connections = 100 in the postgres.conf needs to be higher then the IndexConnectionsCount in the postgresql.json.

Postgres.conf

--------

listen_addresses = '*'        # what IP address(es) to listen on;
                    # comma-separated list of addresses;
                    # defaults to 'localhost'; use '*' for all
                    # (change requires restart)
port = 5432                # (change requires restart)
superuser_reserved_connections = 3
max_connections = 100
shared_buffers = 6GB
effective_cache_size = 22GB
maintenance_work_mem = 1GB
checkpoint_completion_target = 0.9

#wal_buffers = 16MB
wal_compression = on
wal_buffers = -1
default_statistics_target = 100
random_page_cost = 4
work_mem = 32MB
min_wal_size = 2GB
max_wal_size = 8GB
max_worker_processes = 4
max_parallel_workers_per_gather = 2
max_parallel_workers = 4
max_parallel_maintenance_workers = 2
log_temp_files = 0            # log temporary files equal or larger
                    # than the specified size in kilobytes;
                    # -1 disables, 0 logs all temp files

log_autovacuum_min_duration = 0    # log autovacuum activity;
                    # -1 disables, 0 logs all actions and
                    # their durations, > 0 logs only
                    # actions running at least this number
                    # of milliseconds.
log_checkpoints = on
autovacuum = on            # Enable autovacuum subprocess?  'on'
                    # requires track_counts to also be on.
autovacuum_vacuum_cost_limit = 3000    # default vacuum cost limit for
                    # autovacuum, -1 means use
                    # vacuum_cost_limit
autovacuum_max_workers = 5        # max number of autovacuum subprocesses
                    # (change requires restart)
                   
track_counts = on
---------------------------------------------------------------------------------------------                    

With kind regards,

Martin 

[Bartosz Kaczyński]

Thx Martin for your tips.

I've a question. Does IndexConnectionsCount set to higher than 1 value matters in terms of performance if there is only one Orthanc instance connected to PostgreSQL db?

I'will check your DB configuration but I also done some tweaks in postgresql.conf without improving respond time.

Dear Alain,

I haven't add any ExtraMainDicomTags yet but I have also a question related with it

I've checked Orthanc log during querying from the HOROS viewer (any date without any filters like PatientName like in the screen below)

here is a log from Orthanc (attached full as file):

# Dicom-Data-Set
# Used TransferSyntax: Little Endian Explicit
(0008,0005) CS [ISO_IR 100]                             #  10, 1 SpecificCharacterSet
(0008,0020) DA (no value available)                     #   0, 0 StudyDate
(0008,0030) TM (no value available)                     #   0, 0 StudyTime
(0008,0050) SH (no value available)                     #   0, 0 AccessionNumber
(0008,0052) CS [STUDY]                                  #   6, 1 QueryRetrieveLevel
(0008,0061) CS (no value available)                     #   0, 0 ModalitiesInStudy
(0008,0080) LO (no value available)                     #   0, 0 InstitutionName
(0008,0090) PN (no value available)                     #   0, 0 ReferringPhysicianName
(0008,1030) LO (no value available)                     #   0, 0 StudyDescription
(0008,1050) PN (no value available)                     #   0, 0 PerformingPhysicianName
(0010,0010) PN (no value available)                     #   0, 0 PatientName
(0010,0020) LO (no value available)                     #   0, 0 PatientID
(0010,0030) DA (no value available)                     #   0, 0 PatientBirthDate
(0020,000d) UI (no value available)                     #   0, 0 StudyInstanceUID
(0020,0010) SH (no value available)                     #   0, 0 StudyID
(0020,1208) IS (no value available)                     #   0, 0 NumberOfStudyRelatedInstances
(0032,4000) LT (no value available)                     #   0, 0 RETIRED_StudyComments
(4008,0212) CS (no value available)                     #   0, 0 RETIRED_InterpretationStatusID

so to improve response time I've to add all (or some) of this DICOM tags to ExtraMainDicomTags, then reconstruct the database and C-FIND should be faster? How to check which of the tags are already stored in postgres Orthanc db? I searched in maindicomtags relation but it's hard to identify each of them.

Regards

[Dijkhuizen, M.R.]

Hi Bartosz,

For me it definitely made a big change setting the IndexConnectionsCount higher than one. .

Please make sure you don't set your IndexConnectionsCount to the same value as the max_connections in de postgres.conf. In my case I leave 25 connections for worker / admin processes. 

Like I said in the previous post. Also check your postgres logs for errors. 

With kind regards,

Martin

You received this message because you are subscribed to a topic in the Google Groups "Orthanc Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/orthanc-users/9KQYHfKvdiQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to orthanc-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/orthanc-users/1dce64aa-6f9d-4545-a6c8-f882a93bf509n%40googlegroups.com.

[Bartosz Kaczyński]

Thank you, I'll definitely test it.

Regarding to my question to Alain: I found the answer in documentation.

[Bartosz Kaczyński]

Looking at predefined list of Orthanc DICOM tags I think they are pretty reasonable and I am not sure which I can add to improve response C-Find time. Do you have any recommendations?

[walid...@gmail.com]

I have a new installation of Orthanc but don't know why I'm getting W001 warning, this is my verbose from radiant

I0804 09:10:21.792910 main.cpp:352] Incoming Find request from AET RADIANT on IP 127.0.0.1, calling AET RADBACKUP
I0804 09:10:21.808602 main.cpp:228] Maximum 300 results for C-FIND queries at the Patient, Study and Series levels
I0804 09:10:21.808602 main.cpp:234] No limit on the number of C-FIND results at the Instance level
I0804 09:10:21.808602 OrthancFindRequestHandler.cpp:342] (dicom) C-FIND: Allowing SCU request from unknown modality with AET: RADIANT
I0804 09:10:21.808602 OrthancFindRequestHandler.cpp:395] (dicom) DICOM C-Find request at level: Study
I0804 09:10:21.808602 OrthancFindRequestHandler.cpp:401] (dicom)   (0008,0020)  StudyDate = *** POTENTIAL PHI ***
I0804 09:10:21.808602 OrthancFindRequestHandler.cpp:401] (dicom)   (0008,0030)  StudyTime =
I0804 09:10:21.808602 OrthancFindRequestHandler.cpp:401] (dicom)   (0008,0050)  AccessionNumber =
I0804 09:10:21.808602 OrthancFindRequestHandler.cpp:401] (dicom)   (0008,0052)  QueryRetrieveLevel = STUDY
I0804 09:10:21.808602 OrthancFindRequestHandler.cpp:401] (dicom)   (0008,0056)  InstanceAvailability =
I0804 09:10:21.808602 OrthancFindRequestHandler.cpp:401] (dicom)   (0008,0061)  ModalitiesInStudy =
I0804 09:10:21.808602 OrthancFindRequestHandler.cpp:401] (dicom)   (0008,0080)  InstitutionName =
I0804 09:10:21.808602 OrthancFindRequestHandler.cpp:401] (dicom)   (0008,0090)  ReferringPhysicianName =
I0804 09:10:21.808602 OrthancFindRequestHandler.cpp:401] (dicom)   (0008,1030)  StudyDescription =
I0804 09:10:21.808602 OrthancFindRequestHandler.cpp:401] (dicom)   (0008,1040)  InstitutionalDepartmentName =
I0804 09:10:21.808602 OrthancFindRequestHandler.cpp:401] (dicom)   (0008,1050)  PerformingPhysicianName =
I0804 09:10:21.808602 OrthancFindRequestHandler.cpp:401] (dicom)   (0008,1060)  NameOfPhysiciansReadingStudy =
I0804 09:10:21.808602 OrthancFindRequestHandler.cpp:401] (dicom)   (0010,0010)  PatientName =
I0804 09:10:21.808602 OrthancFindRequestHandler.cpp:401] (dicom)   (0010,0020)  PatientID =
I0804 09:10:21.808602 OrthancFindRequestHandler.cpp:401] (dicom)   (0010,0030)  PatientBirthDate =
I0804 09:10:21.808602 OrthancFindRequestHandler.cpp:401] (dicom)   (0010,0040)  PatientSex =
I0804 09:10:21.808602 OrthancFindRequestHandler.cpp:401] (dicom)   (0020,000d)  StudyInstanceUID =
I0804 09:10:21.808602 OrthancFindRequestHandler.cpp:401] (dicom)   (0020,0010)  StudyID =
I0804 09:10:21.808602 OrthancFindRequestHandler.cpp:401] (dicom)   (0020,1208)  NumberOfStudyRelatedInstances =
I0804 09:10:21.808602 ServerContext.cpp:1426] Number of candidate resources after fast DB filtering on main DICOM tags: 3
I0804 09:10:21.824166 StorageCache.cpp:125] Read start of attachment "75345394-d981-42e7-abca-d8eed17c236b" with content type 1 from cache
I0804 09:10:21.824166 StorageCache.cpp:125] Read start of attachment "1bb4d21e-3139-46eb-b644-870e9c0ccf3d" with content type 1 from cache
I0804 09:10:21.839789 StorageCache.cpp:125] Read start of attachment "a6fd9b5a-a5b0-427b-850e-587abbde5c42" with content type 1 from cache
W0804 09:10:21.839789 ServerContext.cpp:2441] W001: Accessing Dicom tags from storage when accessing study : 0008,1040;0008,1050;0008,1060
I0804 09:10:21.855415 ServerContext.cpp:1576] Number of matching resources: 3
I0804 09:10:21.855415 CommandDispatcher.cpp:929] (dicom) Finishing association with AET RADIANT on IP 127.0.0.1: DUL Peer Requested Release
I0804 09:10:21.855415 CommandDispatcher.cpp:937] (dicom) Association Release with AET RADIANT on IP 127.0.0.1

I thought dicom 0008,1040;0008,1050;0008,1060 were part of predefined dicom tags in orthanc what can I do to stop it from accessing the storage for this tags? would adding them to the extradicom tags help? thanks.

[Alain Mazy]

I thought dicom 0008,1040;0008,1050;0008,1060 were part of predefined dicom tags (NO) in orthanc what can I do to stop it from accessing the storage for this tags? would adding them to the extradicom tags help?  YES

All info is in the doc

To view this discussion on the web visit https://groups.google.com/d/msgid/orthanc-users/042021f2-3143-48bc-99f8-c7bc32b0e39cn%40googlegroups.com.

[walid...@gmail.com]

I've read the doc, and put the tags as follows. Housekeeper plugin is enabled but I still get W001 warning for the tags. Is there anything wrong with the setup?

"ExtraMainDicomTags" : {
    "Instance" : [
      "Rows",
      "Columns",
      "ImageType",
      "SOPClassUID",
      "ContentDate",
      "ContentTime",
      "FrameOfReferenceUID",
      "PixelSpacing",
      "SpecificCharacterSet",
      "BitsAllocated"
    ],
    "Series" : [],
    "Study": [
        "NameOfPhysiciansReadingStudy",
        "PerformingPhysicianName",
        "InstitutionalDepartmentName"
    ],
    "Patient": []
  },

[Bartosz Kaczyński]

You have to check which tags are requested by your DICOM Viewer during C-FIND.

In my case during querying by Horos I see in Orthanc logs following messages:

I0808 07:35:26.666134 OrthancFindRequestHandler.cpp:653] (dicom) DICOM C-Find request at level: Study
I0808 07:35:26.666197 OrthancFindRequestHandler.cpp:659] (dicom)   (0008,0005)  SpecificCharacterSet = ISO_IR 100
I0808 07:35:26.666224 OrthancFindRequestHandler.cpp:659] (dicom)   (0008,0020)  StudyDate = *** POTENTIAL PHI ***
I0808 07:35:26.666235 OrthancFindRequestHandler.cpp:659] (dicom)   (0008,0030)  StudyTime =
I0808 07:35:26.666245 OrthancFindRequestHandler.cpp:659] (dicom)   (0008,0050)  AccessionNumber =
I0808 07:35:26.666257 OrthancFindRequestHandler.cpp:659] (dicom)   (0008,0052)  QueryRetrieveLevel = STUDY
I0808 07:35:26.666267 OrthancFindRequestHandler.cpp:659] (dicom)   (0008,0061)  ModalitiesInStudy =
I0808 07:35:26.666277 OrthancFindRequestHandler.cpp:659] (dicom)   (0008,0080)  InstitutionName =
I0808 07:35:26.666287 OrthancFindRequestHandler.cpp:659] (dicom)   (0008,0090)  ReferringPhysicianName =
I0808 07:35:26.666297 OrthancFindRequestHandler.cpp:659] (dicom)   (0008,1030)  StudyDescription =
I0808 07:35:26.666307 OrthancFindRequestHandler.cpp:659] (dicom)   (0008,1050)  PerformingPhysicianName =
I0808 07:35:26.666317 OrthancFindRequestHandler.cpp:659] (dicom)   (0010,0010)  PatientName =
I0808 07:35:26.666343 OrthancFindRequestHandler.cpp:659] (dicom)   (0010,0020)  PatientID =
I0808 07:35:26.666354 OrthancFindRequestHandler.cpp:659] (dicom)   (0010,0030)  PatientBirthDate =
I0808 07:35:26.666365 OrthancFindRequestHandler.cpp:659] (dicom)   (0020,000d)  StudyInstanceUID =
I0808 07:35:26.666375 OrthancFindRequestHandler.cpp:659] (dicom)   (0020,0010)  StudyID =
I0808 07:35:26.666386 OrthancFindRequestHandler.cpp:659] (dicom)   (0020,1208)  NumberOfStudyRelatedInstances =
I0808 07:35:26.666396 OrthancFindRequestHandler.cpp:659] (dicom)   (0032,4000)  RETIRED_StudyComments =

I0808 07:35:26.666407 OrthancFindRequestHandler.cpp:659] (dicom)   (4008,0212)  RETIRED_InterpretationStatusID = 

I've added those tags to appropriate section in by configuration file (I am not sure where to put some of the tags, some of them I had to comment out):

"ExtraMainDicomTags" : {
    "Instance" : [
        "0008,0005"
    ],
    "Series" : [
        "0008,1050"
    ],
    "Study": [
        "0008,0052",
        //"0008,0061",
        //"0020,1208",
        "0032,4000",
        "4008,0212"
    ],
    "Patient": []
}

I've reconstructed all studies, but during C-FIND I still receive messages:

W0809 13:53:28.313838 ServerContext.cpp:2424] W001: Accessing Dicom tags from storage when accessing study : 0008,0005;0008,1050

even that those tags are added to configuration file.

Is that error mentioned here or I put those tags or wrong level (maybe shot be in more than one)?

[Alain Mazy]

Since the C-Find is performed at Study level, Orthanc will only look at Study and Patient tags -> it won't find the 0008,1050 in DB.

Adding 0008,0005 at Study level does not really make sense and it has anyway a special behaviour in C-Find -> you might try not to add it in the ExtraMainDicomTags.  Please report the result of your tests.

Alain.

To view this discussion on the web visit https://groups.google.com/d/msgid/orthanc-users/1e2485c2-f019-442d-b322-d245fe53d5f1n%40googlegroups.com.

[Bartosz Kaczyński]

Thank you, Alain.

I added (0008,0005) (which is SpecificCharacterSet) on Instance level because I saw that example in documentation :) So I'll remove it, add 0008,1050 on Study level, then perform reconstruction and give feedback here.

[Bartosz Kaczyński]

Hi, for testing purpose I've migrate my environment to docker container using Osimis images.

In first approach I did what is described above: added PerformingPhysicianName (0008,1050) on the study level, removed completely SpecificCharacterSet (0008,0005), reconstructed studies and the result is that I receive in logs:

orthanc_1  | I0810 11:13:27.544363 FilesystemStorage.cpp:176] Reading attachment "4bee76c1-05f5-439f-afa1-64fc253248fb" of "DICOM" content type (range from 0 to 98308)
orthanc_1  | W0810 11:13:27.618801 ServerContext.cpp:2441] W001: Accessing Dicom tags from storage when accessing study : 0008,0005

In second approach I've added anyway SpecificCharacterSet (0008,0005) tag on study level, reconstructed. Logs like above didn't appear during C-Find.

In both cases response time has not changed. Tested on Orthanc version 1.11.1.

There are no missing tags in DB during C-Find, StorageAccessOnFind is set on Never, but C-Find is still slow (about 2 minutes waiting for 1200 of MRI studies). Is it possible to be related with the confirmed bug?

[Alain Mazy]

  Hi Bartosz,

No, this does not seem related to the confirmed bug, you would have warnings in the logs.

First of all, I must admit that Orthanc has not been designed to have the quickest searches.  The searches could be improved a lot from the code itself but that would likely require a lot of work.

We had a very interesting chat with Rob (also active on this forum) about this last week.  He's been observing the same kind of performances as you when he started to work with lots of studies with lots of instances (around 100k studies with thousands of instances) while he did not have the same issue with the same amount of studies with only a few instances.  

Once you reach 100k studies with 1000 instances, the MainDicomTags table can grow around 2 billion rows and Orthanc is performing at least one request for each study found in this huge table and the execution time can pile up very quickly.

Also note that this can be improved on PG server side as well although I'm unable to provide good recommandations (check the recommendations from Martin earlier in this thread).

I just performed a quick test on one of my customer installations.  Orthanc is running in Docker and the PG server is running on another VM (ther server is provided by the hospital IT so I don't know how it is configured).

Here are the stats: 250k Studies, 107 million instances.  Searching for studies takes around 1.5s for 100 studies in the results (it is very important to limit the tools/find and c-find results to 100-200 results in order to limit the execution time).

I know that Rob is prototyping a plugin to speed up tools/find thanks to extra tables whose structure is more suited for efficient search.  Hopefully, I'll be able to reuse his work and include it e.g in an "optimized postgresql plugin".

Best regards,

Alain

To view this discussion on the web visit https://groups.google.com/d/msgid/orthanc-users/a4c62970-98b4-4718-be10-130b35881263n%40googlegroups.com.

[Bartosz Kaczyński]

Thank you for clarifying and for doing test. I'll lookup some another performance tuning at PG site.

Regards,

BK