Issues with XNAT DQR

[Jonika Tannous]

We have been putting together a pipeline that will use the XNAT DQR to connect to our PACS system and import dicoms. Out of 326 attempted sequences 18 (5.3%) failed with both the API and the UI. We noticed that it was individual series within the sequences that may have been causing the problem, and as soon as XNAT hits one series in a sequence that fails to import, it does not import the rest. We found this by adjusting our call to import one series at a time. After the implementation of individual calls and cross-referencing with what we can query in our Synedra/PACS system, we found that the vast majority of series that failed to import had duplicate series descriptions but unique series ids and UUIDs. We found that their series IDs were not in our PACS system (XNAT was essentially attempting to import something that didn't exist in PACS). Once we account for that, only 2 (0.6%) sequences fail and they wind up with an "archive pending status" that doesn't go away (of note, the subject field in prearchive becomes blank for these sequences). Image below. My specific questions are:
1. Do you know what could be causing XNAT to pick up on these duplicate series that don't exist in our PACS system?

2. What could be causing archive pending to hang on an import and the subject field to become blank?

[Rick Herrick]

Just to clarify: you're getting failures when you have a scenario like this (UIDs dumbed way down for convenience):

Series UID	Series number	Series description
1.1	1	Series A
1.2	2	Series B
1.3	3	Series B

I'm not sure what you're referring to when you mention series UUIDs...

Presuming this is correct...

You said the "series IDs" (do you mean the series instance UIDs?) XNAT is using to request the data are not in your PACS system. What do those IDs look like? XNAT uses a combination of study instance UID and series instance UID to request each series, so I'm unclear on how the series instance UID wouldn't be on your PACS, since that's where XNAT is getting the ID from in the first place...

--

Rick Herrick

Senior Software Developer

rickh...@flywheel.io

------ Original Message ------

From "Jonika Tannous" <jonika.d...@gmail.com>

To "xnat_discussion" <xnat_di...@googlegroups.com>

Date 9/12/2022 3:09:33 PM

Subject [XNAT Discussion] Issues with XNAT DQR

--
You received this message because you are subscribed to the Google Groups "xnat_discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to xnat_discussi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/xnat_discussion/533d0911-798f-41a3-ae59-45b816322eb0n%40googlegroups.com.

[Jonika Tannous]

Thanks for your response.

Yes - the table you made is correct. To expand upon it, we see in Synedra (which we use to interface with PACS) that series numbers 1 and 2 are there but there is no series 3. Series 3 will then inevitably fail in XNAT queue history with a status of "failed" while the rest are successfully imported. I also am struggling to figure out where XNAT is pulling that info from if we can't see it in Synedra. My only thought is if a series was started and then deleted/not completed and still has metadata somewhere in PACS that Synedra excludes but XNAT still tries to import. I was wondering if others had the same issues.

The scans that go into perpetual "archive pending" seem to be a different problem. No ideas here about what causes them to fail as their Synedra files are pretty unremarkable. 
Jonika

[Rick Herrick]

Basically DQR gets data from PACS in a two-step process:

• Search for studies/series based on some set of criteria (e.g. study description, accession number, etc.)
• Request that the PACS send one or more series by study and series instance UID

I can definitely see it being an issue if the PACS responds with images that are then no longer available when XNAT requests them. That said, XNAT should *probably* handle that properly and still archive the data that did actually arrive, but I can also see the flip side: the PACS told XNAT this data was available and XNAT expects it to be part of the session to be archived, so should it really proceed with archiving the session, given that the data is quite possibly incomplete?

Also, you said "as soon as XNAT hits one series in a sequence that fails to import, it does not import the rest.” Do you mean that the data just stays in the prearchive? That seems reasonable given that XNAT expects more data to arrive because the PACS told it that data was available and XNAT requested it.

How are the DQR requests from XNAT to your PACS initiated? One way to circumvent this problem would be to stop requesting series that are subsequently deleted from the PACs.

Regarding sessions that are in archive pending state, we’d need info from the XNAT server logs contemporaneous with that occurring. There are a number of reasons that can occur, but it's usually related to something in the DICOM data, issues with write permissions, or other issues with the processing environment (mind you, I’m not saying XNAT shouldn’t deal with those issues properly, just that these can be very difficult issues to diagnose because they can’t be recreated reliably outside of the environment in which they’re happening!).

Rick Herrick

Senior Software Developer

rickh...@flywheel.io

------ Original Message ------

From "Jonika Tannous" <jonika.d...@gmail.com>

To "xnat_discussion" <xnat_di...@googlegroups.com>

Date 9/12/2022 4:22:48 PM

Subject Re: [XNAT Discussion] Issues with XNAT DQR

To view this discussion on the web visit https://groups.google.com/d/msgid/xnat_discussion/75b7bd2a-9159-4085-accc-e686f5094c34n%40googlegroups.com.

[elijah....@gmail.com]

Hey Rick,

I'm working with Jonika. We tested some pulls in real time and found these log entries. Not sure if we can find a clue here.

Dqr.log

2022-09-15 13:10:36,821 [Thread-7348] WARN  org.nrg.xnatx.dqr.dicom.command.cecho.dcm4che.tool.Dcm4cheToolCEchoSCU - An error occurred trying to check the connection to AE PACSARCHIVE:104 on host 10.4.1.132

java.net.ConnectException: Connection refused (Connection refused)

        at java.net.PlainSocketImpl.socketConnect(Native Method)

        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)

        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)

        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)

        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)

        at java.net.Socket.connect(Socket.java:607)

        at org.dcm4che2.net.NetworkConnection.connect(NetworkConnection.java:569)

        at org.dcm4che2.net.NetworkApplicationEntity.connect(NetworkApplicationEntity.java:963)

        at org.dcm4che2.net.NetworkApplicationEntity.connect(NetworkApplicationEntity.java:843)

        at org.dcm4che2.tool.dcmecho.DcmEcho.open(DcmEcho.java:656)

        at org.nrg.xnatx.dqr.dicom.command.cecho.dcm4che.tool.Dcm4cheToolCEchoSCU.canConnect(Dcm4cheToolCEchoSCU.java:63)

        at org.nrg.xnatx.dqr.services.impl.basic.BasicDicomQueryRetrieveService.canConnect(BasicDicomQueryRetrieveService.java:111)

        at org.nrg.xnatx.dqr.events.PacsDequeueThread.runTask(PacsDequeueThread.java:86)

        at org.nrg.xnat.task.AbstractXnatRunnable.run(AbstractXnatRunnable.java:33)

        at java.lang.Thread.run(Thread.java:748)

2022-09-15 13:10:36,821 [Thread-7348] WARN  org.nrg.xnatx.dqr.events.PacsThreads - Tried to decrement thread count for PACS 4 but that's not in the thread table

 

 

Prearchive.log

2022-09-15 13:09:35,339 [DefaultMessageListenerContainer-3] ERROR org.nrg.xnat.helpers.prearchive.PrearcUtils - /data/prearchive/PRO00025034/20220814_231159640/CT_ANGIOGRAM_HEAD_W_WO_CONTRAST_357069642.xml is empty.

 

 

 

Received.log

2022-09-15 13:22:35,554 - PACSARCHIVE@/10.7.1.243:60261:/data/prearchive/PRO00025034/20220915_132235519/CT_ANGIOGRAM_HEAD_W_WO_CONTRAST_357069642/SCANS/455/DICOM/1.2.840.114350.2.430.2.798268.2.357069642.1-455-2-3xut9.dcm

2022-09-15 13:22:35,587 - PACSARCHIVE@/10.7.1.243:60261:/data/prearchive/PRO00025034/20220915_132235519/CT_ANGIOGRAM_HEAD_W_WO_CONTRAST_357069642/SCANS/455/DICOM/1.2.840.114350.2.430.2.798268.2.357069642.1-455-3-3xuta.dcm

2022-09-15 13:22:35,607 - PACSARCHIVE@/10.7.1.243:60261:/data/prearchive/PRO00025034/20220915_132235519/CT_ANGIOGRAM_HEAD_W_WO_CONTRAST_357069642/SCANS/455/DICOM/1.2.840.114350.2.430.2.798268.2.357069642.1-455-4-3xutb.dcm

2022-09-15 13:22:35,666 - PACSARCHIVE@/10.7.1.243:60261:/data/prearchive/PRO00025034/20220915_132235519/CT_ANGIOGRAM_HEAD_W_WO_CONTRAST_357069642/SCANS/455/DICOM/1.2.840.114350.2.430.2.798268.2.357069642.1-455-6-3xutd.dcm

2022-09-15 13:22:35,740 - PACSARCHIVE@/10.7.1.243:60261:/data/prearchive/PRO00025034/20220915_132235519/CT_ANGIOGRAM_HEAD_W_WO_CONTRAST_357069642/SCANS/455/DICOM/1.2.840.114350.2.430.2.798268.2.357069642.1-455-5-3xutc.dcm

2022-09-15 13:22:35,817 - PACSARCHIVE@/10.7.1.243:60261:/data/prearchive/PRO00025034/20220915_132235519/CT_ANGIOGRAM_HEAD_W_WO_CONTRAST_357069642/SCANS/455/DICOM/1.2.840.114350.2.430.2.798268.2.357069642.1-455-9-3xutg.dcm

2022-09-15 13:22:35,857 - PACSARCHIVE@/10.7.1.243:60261:/data/prearchive/PRO00025034/20220915_132235519/CT_ANGIOGRAM_HEAD_W_WO_CONTRAST_357069642/SCANS/455/DICOM/1.2.840.114350.2.430.2.798268.2.357069642.1-455-8-3xutf.dcm

2022-09-15 13:22:35,929 - PACSARCHIVE@/10.7.1.243:60261:/data/prearchive/PRO00025034/20220915_132235519/CT_ANGIOGRAM_HEAD_W_WO_CONTRAST_357069642/SCANS/455/DICOM/1.2.840.114350.2.430.2.798268.2.357069642.1-455-1-3xusn.dcm

2022-09-15 13:22:35,949 - PACSARCHIVE@/10.7.1.243:60261:/data/prearchive/PRO00025034/20220915_132235519/CT_ANGIOGRAM_HEAD_W_WO_CONTRAST_357069642/SCANS/455/DICOM/1.2.840.114350.2.430.2.798268.2.357069642.1-455-7-3xute.dcm

2022-09-15 13:22:35,993 - PACSARCHIVE@/10.7.1.243:60261:/data/prearchive/PRO00025034/20220915_132235519/CT_ANGIOGRAM_HEAD_W_WO_CONTRAST_357069642/SCANS/455/DICOM/1.2.840.114350.2.430.2.798268.2.357069642.1-455-11-3xuti.dcm

2022-09-15 13:22:36,026 - PACSARCHIVE@/10.7.1.243:60261:/data/prearchive/PRO00025034/20220915_132235519/CT_ANGIOGRAM_HEAD_W_WO_CONTRAST_357069642/SCANS/455/DICOM/1.2.840.114350.2.430.2.798268.2.357069642.1-455-12-4hnfh.dcm

2022-09-15 13:22:36,046 - PACSARCHIVE@/10.7.1.243:60261:/data/prearchive/PRO00025034/20220915_132235519/CT_ANGIOGRAM_HEAD_W_WO_CONTRAST_357069642/SCANS/455/DICOM/1.2.840.114350.2.430.2.798268.2.357069642.1-455-13-4hnfi.dcm

2022-09-15 13:22:36,108 - PACSARCHIVE@/10.7.1.243:60261:/data/prearchive/PRO00025034/20220915_132235519/CT_ANGIOGRAM_HEAD_W_WO_CONTRAST_357069642/SCANS/455/DICOM/1.2.840.114350.2.430.2.798268.2.357069642.1-455-14-4hnfj.dcm

2022-09-15 13:22:36,152 - PACSARCHIVE@/10.7.1.243:60261:/data/prearchive/PRO00025034/20220915_132235519/CT_ANGIOGRAM_HEAD_W_WO_CONTRAST_357069642/SCANS/455/DICOM/1.2.840.114350.2.430.2.798268.2.357069642.1-455-16-4hnfl.dcm

2022-09-15 13:22:36,173 - PACSARCHIVE@/10.7.1.243:60261:/data/prearchive/PRO00025034/20220915_132235519/CT_ANGIOGRAM_HEAD_W_WO_CONTRAST_357069642/SCANS/455/DICOM/1.2.840.114350.2.430.2.798268.2.357069642.1-455-15-4hnfk.dcm

2022-09-15 13:22:36,197 - PACSARCHIVE@/10.7.1.243:60261:/data/prearchive/PRO00025034/20220915_132235519/CT_ANGIOGRAM_HEAD_W_WO_CONTRAST_357069642/SCANS/455/DICOM/1.2.840.114350.2.430.2.798268.2.357069642.1-455-10-3xuth.dcm

 

 

Dicom.log

2022-09-15 13:09:35,214 [DefaultMessageListenerContainer-3] WARN  org.nrg.dcm.xnat.DICOMSessionBuilder - Session builder not implemented for SOP class(es) [1.2.840.10008.5.1.4.1.1.11.1] or modality(ies) [PR], trying default session builder factory