Transfers accelerator plugin - Error

[Ric Smi]

Dear All,

I've been using Autorouting on OnStoredInstance for some time and it works.  I wanted to try Transfer accelerator plugin.  My Lua script is given below

PEER = 'MyPeer'

function OnStableSeries(seriesId, tags, metadata)

if (metadata['ModifiedFrom'] == nil and 

    metadata['AnonymizedFrom'] == nil) then

 

-- The tags to be replaced

local replace = {}

replace['StationName'] = 'ABC'

   

-- Modify the instance

local command = {}

command['Replace'] = replace

-- Modify the entire series in one single call

local modifiedId = RestApiPost('/series/' .. seriesId .. '/modify', DumpJson(command, true))

local transfer = {}

transfer['Resources'] = {}

transfer['Resources'][1] = {}

transfer['Resources'][1]['Level'] = 'Series'

transfer['Resources'][1]['ID'] = modifiedId

transfer['Compression'] = 'gzip'

transfer['Peer'] = PEER

   

local job = ParseJson(RestApiPost('/transfers/send', DumpJson(transfer, true)))

RestApiDelete('/series/' .. seriesId)

RestApiDelete('/series/' .. modifiedId)

end

end

My transfer.json looks like below...

  "Transfers" : {

    "Threads" : 8,             // Number of worker threads for one transfer

    "BucketSize" : 4096,       // Optimal size for a bucket (in KB)

    "CacheSize" : 128,         // Size of the memory cache to process DICOM files (in MB)

    "MaxPushTransactions" : 8, // Maximum number of simultaneous receptions in push mode

    "MaxHttpRetries" : 4       // Maximum number of HTTP retries for one bucket

  }

However, I have not been able to run this plugin successfully.  Please note I'm using 1.5.6 64-bit under Windows.

Any idea what could be wrong here?

Many thanks!

[Alain Mazy]

Have you checked your logs ? (http://book.orthanc-server.com/faq/log.html)

--

Alain Mazy / Software Developer
a...@osimis.io / +32 494 31 67 27

	OSIMIS S.A.  Quai Banning 6BE-4000 Liège  www.osimis.io

[Ric Smi]

Thank you for getting back to me...

I have not run the verbose but on regular log there is nothing.

I will be running the verbose to see if I get any information and will report back if I find any.

Thank you!

[Ric Smi]

When I ran with Verbose, I get following...

W0318 13:54:08.179306 main.cpp:1436] Orthanc version: 1.5.6

W0318 13:54:08.179306 OrthancConfiguration.cpp:61] Reading the configuration from: "Configuration.json"

I0318 13:54:08.179306 Toolbox.cpp:1370] Using locale: "" for case-insensitive comparison of strings

I0318 13:54:08.179306 Enumerations.cpp:2217] Default encoding for DICOM was changed to: Latin1

I0318 13:54:08.179306 FromDcmtkBridge.cpp:233] Using DCTMK version: 364

I0318 13:54:08.179306 FromDcmtkBridge.cpp:241] Loading the embedded dictionaries

I0318 13:54:08.226166 FromDcmtkBridge.cpp:253] Loading the embedded dictionary of private tags

I0318 13:54:08.257418 FromDcmtkBridge.cpp:2175] Registering JPEG Lossless codecs in DCMTK

I0318 13:54:08.257418 FromDcmtkBridge.cpp:2180] Registering JPEG codecs in DCMTK

W0318 13:54:08.257418 main.cpp:700] Loading plugin(s) from: D:\temp_Orthanc\Plugins

I0318 13:54:08.257418 PluginsManager.cpp:288] Scanning folder D:\temp_Orthanc\Plugins for plugins

I0318 13:54:08.273043 PluginsManager.cpp:311] Found a shared library: "D:\temp_Orthanc\Plugins\iconv.dll"

E0318 13:54:08.273043 SharedLibrary.cpp:60] LoadLibrary(D:\temp_Orthanc\Plugins\iconv.dll) failed: Error 193

E0318 13:54:08.273043 main.cpp:1462] Uncaught exception, stopping now: [Error while using a shared library (plugin)] (code 25)

W0318 13:54:08.273043 main.cpp:1495] Orthanc has stopped

[Alain Mazy]

This might look like a 32bits/64bits issue (the exe and the dll must have the same target architecture).

BTW, I realized that we do not provide the DLL yet for Windows in our installer.  I'll update it....

[Ric Smi]

I downloaded the latest 32 bit windows file and ran with verbose with following errors...

W0319 09:27:37.278281 main.cpp:1436] Orthanc version: 1.5.6
W0319 09:27:37.278281 OrthancConfiguration.cpp:61] Reading the configuration from: "Configuration.json"
I0319 09:27:37.293910 Toolbox.cpp:1370] Using locale: "" for case-insensitive comparison of strings
I0319 09:27:37.293910 Enumerations.cpp:2217] Default encoding for DICOM was changed to: Latin1
I0319 09:27:37.293910 FromDcmtkBridge.cpp:233] Using DCTMK version: 364
I0319 09:27:37.293910 FromDcmtkBridge.cpp:241] Loading the embedded dictionaries
I0319 09:27:37.340774 FromDcmtkBridge.cpp:253] Loading the embedded dictionary of private tags
I0319 09:27:37.356402 FromDcmtkBridge.cpp:2175] Registering JPEG Lossless codecs in DCMTK
I0319 09:27:37.356402 FromDcmtkBridge.cpp:2180] Registering JPEG codecs in DCMTK
W0319 09:27:37.372052 main.cpp:700] Loading plugin(s) from: D:\temp_Orthanc\Plugins
I0319 09:27:37.372052 PluginsManager.cpp:288] Scanning folder D:\temp_Orthanc\Plugins for plugins
I0319 09:27:37.372052 PluginsManager.cpp:311] Found a shared library: "D:\temp_Orthanc\Plugins\iconv.dll"
I0319 09:27:37.387660 PluginsManager.cpp:311] Found a shared library: "D:\temp_Orthanc\Plugins\libcairo-2.dll"
E0319 09:27:37.403279 SharedLibrary.cpp:60] LoadLibrary(D:\temp_Orthanc\Plugins\libcairo-2.dll) failed: Error 126
E0319 09:27:37.403279 main.cpp:1462] Uncaught exception, stopping now: [Error while using a shared library (plugin)] (code 25)
W0319 09:27:37.403279 main.cpp:1495] Orthanc has stopped

[Alain Mazy]

In your "Plugins" setting, you've probably provided a folder.  Orthanc is therefore trying to load every .dll file from that folder considering they're all Orthanc plugins which is obviously not the case for libcairo-2.dll ...

Instead of a folder, you should explicitly list all Orthanc plugin DLLs you which to load.

[Ric Smi]

Okay, it this time Orthanc started with verbose.  The log, Lua script, Jobs image, and Error image attached.

From the log it appears missing series.  However, the series is there.  Could there be timing issue?

Thank you

[Alain Mazy]

Hi Ric,

instead of 

  transfer['Resources'][1]['ID'] = m

you should have:

  transfer['Resources'][1]['ID'] = m['ID']

since m is actually a json object with multiple fields

[Ric Smi]

Thank you Alain for pointing out potential issue.  However, fixing the Lua script did not provide the desired result.

Kindly find Verbose log, and error images.

Many thanks for the kind support!!!

[Ric Smi]

Kindly guide me where to look for more info to get Transfers accelerator working?

Many thanks!

[Alain Mazy]

Just a reminder: it is normal in a forum that you don't get a response as soon as you expect.  As already stated, we have a life, we have a job, sometimes, we even have to sleep and eat.

Have you tried transferring the data without the lua scripts that modifies the data ?

Have you checked the logs on the other Orthanc (the one that shall receive the data) ?

On Fri, Mar 22, 2019 at 2:38 PM Ric Smi <smir...@gmail.com> wrote:

Kindly guide me where to look for more info to get Transfers accelerator working?

Many thanks!

[Ric Smi]

Dear Alain, I truly appreciate your support - many thanks!! 

Also, I totally understand the nature of this form hence my deep appreciation to you and to all others who contribute in this form.  So, I apologize if I sound pushy.

I did try transferring the data without modifying the data however using lua script. And it did work.

So, the question remains how do I modify the data in OnStableSeries and push it to remote Orthanc?

Many thanks!

[Alain Mazy]

Hi Ric,

The problem is actually that you delete the series while it is still being sent (because the transfer is asynchronous: the RestApiPost returns immediately while the job is still executing).

 

local job = ParseJson(RestApiPost('/transfers/send', DumpJson(transfer, true)))

RestApiDelete('/series/' .. seriesId)

RestApiDelete('/series/' .. modifiedId)

Right now, I don't know of any workaround for your problem if you want to stick to Lua and the transfers plugin.  

With an external script, you could monitor the transfer job status and delete the series only once the job is complete.
If using DicomWeb or Orthanc peers, you could force the synchronous mode but this is not available with the transfers plugin right now.

Best regards,

Alain.

[Alain Mazy]

Hi Ric,

Note that there are 3 new job related Lua callbacks I was not aware of (http://book.orthanc-server.com/users/lua.html#callbacks-to-react-to-events).  So you can actually enable a lua callback on job completion.

However, one problem is that, when a transfer job completes, you don't know what series it relates to.

So, once your job is submitted, you could save the jobId-seriesId mapping somewhere (could be in files in /tmp).

Once your job complete, the OnJobCompleted callback would be called, you can therefore retrieve the seriesId from the mapping you've saved and delete the series.

Best regards,

Alain

[Salim Kanoun]

Hi there,

Just a small question about the new Orthanc Transfers plugin,

I had several HTTP Timout,

In the Transfers Doc the value of HTTP timout shown is 120 (http://book.orthanc-server.com/plugins/transfers.html)

in the Default Orthanc.json the HTTP Timout is 10 which seems to me very low for external communication.

Isn't there a default settings to update to be compliant for Transfers plugin ?

Is there a recommanded value for "regular" internet connection ?

Best regards,

Salim

[Ric Smi]

Dear Alain,

Thank you for the response and a suggestion to try.  Let me dive into this and I revert back to you with findings.

Best regards,

[Ric Smi]

Dear Alain,

I did following to test...

OnStableStudy() - modify the tags

OnStableSeries() - push to remote Orthanc via Transfers accelerator and save seriesId in some temp text file

OnJobSuccess() - remove temp text file and delete series based on saved seriesId.

I did the test on a CT study where too many series were there and the transfer did not happened completely.  I got errors.

Many thanks!

[Alain Mazy]

OnStableSeries will happen before OnStableStudy so you'll start pushing data and they will be modified while they are sent.

[Ric Smi]

Sorry, there was a typo in previous post.  Correct order is given below.

OnStoredInstance() - modify the tags

OnStableSeries() - push to remote Orthanc via Transfers accelerator and save seriesId in some temp text file for later process

OnJobSuccess() - remove temp text file and delete series based on saved seriesId.

Many thanks!

[Alain Mazy]

Hi Ric,

I think you scripts have changed quite a lot since this thread has started so, that would be nice if you could post your lua scripts again and the logs from both Orthanc (in verbose mode).

Best regards,

Alain.

[Sébastien Jodogne]

Hi Salim,

This is actually the same "HttpTimeout" option. It is shared by the Orthanc core and the plugins.

Indeed, if you use external communications, 10 seconds is most probably too short. The 120 seconds setting should be fine on a regular Internet connection.

Sébastien-