Send Queue?

[Simon Rascovsky]

Hi all. I think Orthanc as a lightweight DICOM router/preprocessor can be an excellent choice. However, one feature would greatly increase it's reliability is some sort of send queue functionality, in case transfers are interrupted. In it's simplest form, this queue should:

• Move failed move instances (series?/studies?) to the queue
• Retry the transfer after a configurable delay
• Have the possibility of removing items from the queue if needed. (in case the transfer is permanently impossible)

I can think of 3 approaches to this:

• Develop a plugin that extends core Orthanc functionality, perhaps keeping such a queue in the DB.
• Develop a Lua script: If the SendToModality function returns true on successful transfer, it may be possible.
• Develop an external script that polls the changes API and performs all the logic, including the transfer requests. (How can we know through the API if the transfer was successful?)

Any thoughts on what the best/simplest approach should be?

The only mention of queuing mechanisms as far as I could find was here, briefly mentioning the need for it but no specific mechanism is advocated.

I would be very interested in hearing the group's thoughts on this.

Cheers!

Simon

[Sébastien Jodogne]

Dear Simon,

Thanks for your feedback!

Currently, the internals of Orthanc already feature a scheduler (class "ServerScheduler") that handles a queue of jobs (class "ServerJob"), each job consisting of a set of commands to be executed (interface "IServerCommand"). Almost all Lua scripts and REST commands (except for anonymization) already go through this scheduler, but the full refactoring is still WIP. A job can be internally canceled, and it is planned to expose this "job API" through REST. Also note that the scheduler is currently single-threaded (i.e. at most 1 command is executed at a given time).

However, when a job fails (which is the case as soon as 1 of its child commands fails), the job is simply dropped. It is not possible to restart it, contrarily to your request. I have added your idea to our roadmap:
https://trello.com/c/sB0hyTid

This task will most probably have to wait for the 1.0.0 release: My top-priority for the Orthanc core is currently to implement a query/retrieve client (more precisely, to expose C-MOVE SCU through REST).

So, to answer your question, I think that the most direct way to add a "replay" of interrupted transfers in Orthanc would be to develop a plugin. This plugin would manage a queue consisting of pairs (InstanceID, TargetModality). It would have a thread that would continuously monitor this queue, doing a "OrthancPluginRestApiPost()" for each DICOM instance to be sent on the URI "/modalities/.../store", and would monitor the status code of the function. In the case of an error, the pair would be added to a second "error" queue. The content of this error queue could be re-added to the primary queue by a newly introduced REST callback.

Sébastien-

[Simon Rascovsky]

Thank you Sébastien for your thoughts. So am I correct in understanding that when I invoke a "OrthancPluginRestApiPost()" on the URI "/modalities/.../store", the return status code represents the success of the transfer, and not of the request to transfer?

[Sébastien Jodogne]

Yes, the 500 HTTP status will be raised if the transfer fails (not just the request to transfer), and the 200 status will be returned if the transfer is successful.

This is implemented at the following line:

https://code.google.com/p/orthanc/source/browse/OrthancServer/OrthancRestApi/OrthancRestModalities.cpp?nam

You will notice there the call to "SubmitAndWait()", that asks the scheduler to wait for the completion of the submitted job.

Sébastien-

[Simon Rascovsky]

Thank you, this is actually quite helpful.