Web vs API query/retrieve with Orthanc Docker

[John Roberts]

     I've been working with Docker installations of Orthanc and have problems when I attempt to run a query against one of the Dicom nodes setup in my configuration file.

     In the web interface, I can successfully run an Echo Test on the remote node.  However, the query/retrieve always returns empty results.

I thought the issue might be a network problem, so I installed dcmtk within the container.  From inside the container, I can successfully run a findscu against the same remote node and see results.

I then tried the Orthanc restful API and ran a query.  In that case, the query returned results.

So, why might the web interface fail while an internal findscu and an external API query succeed?

Thanks,
John.

[Sébastien Jodogne]

Hello,

The REST API of Orthanc and DICOM C-Move are conceptually very different. Please carefully read the section "Understanding DICOM" of the Orthanc Book, especially the topic about Query/Retrieve:

https://orthanc.chu.ulg.ac.be/book/dicom-guide.html#c-move-query-retrieve

In a nutshell, it is not sufficient for your Orthanc server to connect to your DICOM node (as for C-Echo, C-Move or C-Find). Your DICOM node also has to know about your Orthanc server for C-Move to work properly (i.e. in the reverse direction). This explains why the Echo and findscu work, but not Query/Retrieve.

Also check out the troubleshooting guide:

https://orthanc.chu.ulg.ac.be/book/faq/dicom.html

HTH,

Sébastien-

[John Roberts]

Hi Sebastien,

Thanks for the feedback.  I had read through the Orthanc DICOM documentation already.

Let me see if I can clarify my questions with a few more questions.

Is there an API command that will print the current working version?  I checked "system", but it only indicates that I'm running a mainline version.  I'm using the following docker image, about 8 weeks old:

jodogne/orthanc-plugins             luapostgres         179aaa0cf531        8 weeks ago

Is the Query/Retrieve aspect of the web interface currently working?  Assuming I had everything set up correctly?  I ask because with large open source freeware projects, I accepts that some features are broken on occasion with upgrades.  I think Orthanc is a brilliant piece of work and very useful.  I can live without this feature on the web interface if it's currently broken.

Second, while I generally understand the distinctions between Echo, Move and Find, I'm still confused on what aspects are implemented "behind the scenes" when I click the Query button on the Query/Retrieve page.  I guess I was assuming that it was first executing a Find to generate the list of possible studies on the remote node.  I assumed that the Move part came later and would only be executed when I clicked on a particular "found" study to have it transferred to the Orthanc.

So, in my case, when I click Query on the web interface, I see nothing - no results at all, a blank page labeled "2 of 3".  Naturally, I thought there might be a network issue (or the feature is currently broken?).  So I installed dcmtk to run their findscu command, which I assume executes a C-Find operation.  findscu works - it returns results from the same machine running Orthanc.  What's more, I can make findscu pretend to be the Orthanc AET that the remote node expects.

Now, does the web interface execute something different from a c-find during the initial query process?  If so, that would help me understand why the two queries have different results.

To add to my confusion, I find that the Orthanc rest API works fine when generating a query.  That is, I carefully construct the URL, set my query parameters and call Orthanc directly.  That query works!  I get results.  I had assumed the web interface would similarly generate the same sort of query to retrieve the initial results.  In fact, when I looked at the javascript for the web interface, it seemed like it was constructing the URL to call the rest API. 

So, I have a working query if I construct the URL myself, but I don't get any results when I let the web interface do it.  Why would using the rest API to generate the query produce results, but clicking the Query button on the web interface with the same inputs produce no results? 

Thanks,
John.

[Chico Venancio]

What is the actual query being sent by Orthanc Explorer to the api?

I've seen that behavior because of the wildcard used by Orthanc Explorer, an asterisk *. I have found modalities that require an empty string instead.

Chico Venancio
CEO e Diretor de Criação
VM TECH - (98)8800-2743

[John Roberts]

Good point.  Asterisks seem to work for me,

• I find I can use an asterisk with findscu and the remote node. 
  
• findscu -v -P --call REMOTEAET -aet ORTHANC RemoteIPAddress RemotePort -k "(0008,0052)=PATIENT" -k "PatientID=*"
  
• With the rest API, I've been sending
• curl -d '{ "Level" : "Patient", "Query" : { "PatientID" : "*" }}' -X POST -k https://localhost:8042/modalities/REMOTEAET/query

I tried deleting the asterisk from the web interface - no change.  I tried putting two single quotes to mimic an empty string '', but still get a blank query. 

I'm curious what Level is being used by the web interface to construct the query.  I dug around a bit in the javascript, but I haven't seen yet where it specified either a Patient or Study root based query.

John.

[Chico Venancio]

Orthanc Explorer uses a study level query. See query.js lines 77-90 https://bitbucket.org/sjodogne/orthanc/src/58a0ee0b4be1cca8dc6cefae984d16cfdfb13bce/OrthancExplorer/query-retrieve.js?at=default&fileviewer=file-view-default#query-retrieve.js-77

Does your modality not accept a Study level query?

ps: use the dev console (ctrl + shift + I in Chrome, Firefox, Safari) to see what is actually sent/received from Orthanc Explorer

Chico Venancio 

(98) 9 8800 2743

CEO - VM TECH 

[John Roberts]

Thanks for finding that and pointing out the dev console.  I can see from the data header submission that the query starts with the default in the code:

{ 'Level' : 'Study', 

  'Query' : { 'AccessionNumber' : '*', 

              'PatientBirthDate' : '*', 

              'PatientID' : '*', 

              'PatientName' : '*', 

              'PatientSex' : '*', 

              'SpecificCharacterSet' : 'ISO_IR 192',  // UTF-8 

              'StudyDate' : $('#qr-date').val(), 

              'StudyDescription' : '*' } 

}

and then modifies the query with the values from the web page.

To answer your question, yes, the remote node I'm querying does accept Study level queries.  However, I have been testing slightly different queries with my findscu and rest API queries than what the web is doing.

Specifically, I have not added the character set limitation to my queries.  I'm curious if that makes a difference here.

Unfortunately, I will have to wait until tomorrow since the remote node was taken down for the night.

[Alain Mazy]

Hi John,

Just a remark about the same kind of problems we had:  Some PACS (we've seen it with an AGFA PACS but I have seen it with another PACS in the past) does not accept to return PatientBirthDate at the Study Level.  In this case, the PACS won't return any results.

The Orthanc Web interface always requests the PatientBirthDate.

I would suggest you to make C-Find queries with findscu and generate the same requests as Orthanc would (including all fields that you have found in the js code).  findscu should fail as well.  Then, remove the fields one by one and you'll determine the one that is not supported by your modality.  If it happens that your modality does not accepth the PatientBirthDate as well, then, we might ask Sebastien to remove it from the queries issued from the web interface (or we may submit a patch).

--

Alain Mazy / CTO 
a...@osimis.io / +32 494 31 67 27

	OSIMIS S.A.  Rue bois Saint-Jean 15/1BE-4102 Seraing  www.osimis.io

[John Roberts]

I had a chance to test queries with findscu this morning against a remote scanner and against a remote PACS node at our institution.

The good news is that PatientBirthDate is allowed in the query.  Most of the parameters in the default Orthanc Study based search are permitted on our system.

The problem key was not the SpecificCharacterSet as I wondered, but a missing StudyInstanceUID request (or filter).

Apparently, on our system, Study level queries must request or filter the initial StudyInstanceUID.  I don't know enough about DICOM to understand why, though I might speculate that the service must break down the response for a Study level query into something unique to each Study and StudyInstanceUID may be the only unique thing available if AccessionNumber is not defined.

I may try hacking my local Orthanc javascript to add the missing key and see what happens.

[John Roberts]

Ah, I think the javascript may not reside in text form in the Docker container, or at least I can't find it.

[John Roberts]

There it is, baked into the orthanc executable.

[Chico Venancio]

Yes, it is all baked in. You could write a plugin to change the Javascript files,  but I just put nginx in front and serve js and html from it.

Chico Venancio

[John Roberts]

While I wasn't able to edit the javascript classes directly, I could pause the execution with the browser developer console, reset the query, and experiment with the query contents.

I made a few interesting observations:

• findscu queries
• require StudyInstanceUID
• With PatientBirthDate as a parameter
• Works if undefined (-k PatientBirthDate)
  
• Fails if defined (-k PatientBirthDate=* or -k PatientBirthDate="*")
  
• /modalities/NODE/query
• PatientBirthDate
• works if initially empty string ("")
• does NOT work if initially a wildcard ("*")
• Does NOT require StudyInstanceUID in order to work

What I think is probably important here is how the initial values given to query parameters (ex. a wildcard "*") are being handled.  On the remote node, I'm guessing the initial value types are constrained by their VR.  Someone more knowledgeable of DICOM can speak up here.

findscu and the Orthanc back-end must convert the human readable query into whatever is expected by the remote node.  That may mean that PatientBirthDate can only be initialized as NULL or some string or literal resembling a date.  The literal '*' asterisk character might be unacceptable to some PACS nodes.

I don't know why the rest API can get away with not adding StudyInstanceUID to the query (perhaps it does in the background), or conversely whey findscu seems to require it when Orthanc does not.

John.

On Friday, October 14, 2016 at 9:05:03 AM UTC-6, John Roberts wrote:

[Sébastien Jodogne]

Hello,

To summarize, depending upon the manufacturer of the remote modality, the "*" might have to be replaced with the empty string "". Other patches might be required as well.

We have therefore decided to implement a Lua hook in order to be able to dynamically patch outgoing C-Find SCU requests. This mechanism allows to make the query/retrieve mechanism more flexible, while avoiding any loss of compatibility with Orthanc <= 1.1.0. Here is a sample Lua callback that would replace "*" by "" for any query/retrieve issued by Orthanc (including Orthanc Explorer):

function OutgoingFindRequestFilter(query, modality)

   PrintRecursive(query)

   for key, value in pairs(query) do

      if value == '*' then

         query[key] = ''

      end

   end

   return query

end

Note that you have access to the AET of the queried modality, in order to implement modality-specific patches. This new feature will be shortly available in Orthanc 1.2.0.

We would obviously like to gather information about manufacturers for which such patches are necessary. This would allow us to make the patches persistent, to the benefit of the community. Please share your experience!

HTH,

Sébastien-

[John Roberts]

Returning to a long lost topic, the Lua hook and function you posted solves the problem for me with our Philips PACS systems.

Thanks,
John.

[John Roberts]

Now that I have the hook implemented, I'm getting farther than I did previously using the web interface to retrieve images from our PACS system.

I notice that if I drill down to the series level, I can successfully pull down individual series.

However, if I remain on the 2/4 page and click the "Retrieve all" link, nothing happens.

Drilling down into the javascript, I can't tell for sure, but I think maybe there's a difference in how details are stored in the pageData variable for later construction of the query URI. 

On page 2/4 when pulling a whole study is an option:

    ChangePage('query-retrieve-4', {
                  'query' : pageData.uuid,
                  'answer' : answerId,
                  'server' : pageData.server
                });

While on page 3/4 where series are handled,

   ChangePage('query-retrieve-4', {
                      'query' : queryUuid,
                      'study' : pageData.uuid,
                      'answer' : answerId,
                      'server' : pageData.server
                    });

The query is subsequently constructed as:
    var uri = '../queries/' + pageData.query + '/answers/' + pageData.answer + '/retrieve';

I can understand how Orthanc uses its own query IDs and I can see how those unique local IDs are addressed in the series move case, but it almost looks like the StudyInstanceUID is accidentally used in the study move case rather than whatever Orthanc uid was associated with the query that pulled the original StudyInstanceUID.

Does that make sense?  Or are StudyInstanceUID used as the Orthanc query IDs at the study level?

Thanks,
John.

[John Roberts]

Having some time to dig into this a bit more in the Firefox developer's console, I'm not sure this is a problem with the query variables that I once thought. 

The issue may be with the construction of the click event.  As near as I can tell, the arrow to download the full study on page 2 should be set to generate an event but does not, at least in my Firefox javascript console.  I've tried placing a breakpoint to trigger on the event but can't capture anything. 

I've also tried increasing the notifications in the Orthanc log, but that doesn't help much since this is probably a javascript problem occurring on the browser side.

The construction of the click event at the series level (if you drill down from page 2/study to page 3/series) is somewhat different, leading me to wonder whether this is a javascript problem rather than a malformed query as I suspected the last time I looked at this.