DSI::datashield.results - could this be added to DSI?

5 views
Skip to first unread message

stuart....@arjuna.com

unread,
Dec 15, 2020, 5:06:49 AM12/15/20
to obiba-users
Hi, my understanding is that if DSI::datashield.aggregate(...) fails, DSI::datashield.errors() can be used to obtained the individual error messages, if any, from the servers. Given that only one server may have failed, would it be possible to add a method to DSI datashield.results() to permit obtaining the results from the subset of the servers which didn't fail?
Stuart

Yannick Marcon

unread,
Dec 15, 2020, 7:45:31 AM12/15/20
to obiba...@googlegroups.com
Hi Stuart,

Thanks for the suggestion, I will have a look, in addition to the ability of running long tasks based on the preliminary work of last week's hackathon.

Best,
Yannick

On Tue, Dec 15, 2020 at 11:06 AM stuart....@arjuna.com <stuart....@arjuna.com> wrote:
Hi, my understanding is that if DSI::datashield.aggregate(...) fails, DSI::datashield.errors() can be used to obtained the individual error messages, if any, from the servers. Given that only one server may have failed, would it be possible to add a method to DSI datashield.results() to permit obtaining the results from the subset of the servers which didn't fail?
Stuart

--
You received this message because you are subscribed to the Google Groups "obiba-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to obiba-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/obiba-users/6461ee05-2161-4400-aec4-632f7a5d99ddn%40googlegroups.com.

Yannick Marcon

unread,
Jan 4, 2021, 9:57:36 AM1/4/21
to obiba...@googlegroups.com
Hi Stuart,

There is a DSI option "datashiel.errors.stop" which allows to alter the behaviour of DSI when errors are encountered: instead of stopping when one or more errors have been raised during a datashield function call (default behaviour), a simple warning can be issued. Then when "datashiel.errors.stop" option is FALSE, the datashield.errors() function can be used to identify the failing servers (if there are any) and the returned value of datashield.aggregate() contains the valid results for the servers that did not fail.

This option is already available in the latest release of DSI.

Cheers
Yannick

Stuart Wheater

unread,
Jan 5, 2021, 5:17:57 AM1/5/21
to obiba...@googlegroups.com
Hi Yannick,
    Thanks for letting me know about "datashield.errors.stop", I take it I need to install 'master' from GitHub. Is there a version I should have?

You have made a number of interesting changes to DSI and opalr recently. I wonder if you would like a slot on Thursdays Technical Meeting to explain them?

Stuart
Dr Stuart Wheater
Postal Address: Arjuna Technologies Limited,
Urban Sciences Building, Newcastle Helix, Bath Lane, Newcastle upon Tyne, NE4 5TG United Kingdom
VAT Registration No: 764173128. Company registered in England
Registration No: 4497081



On Mon, 4 Jan 2021 at 14:57, Yannick Marcon <yannick...@obiba.org> wrote:
Hi Stuart,

There is a DSI option "datashiel.errors.stop" which allows to alter the behaviour of DSI when errors are encountered: instead of stopping when one or more errors have been raised during a datashield function call (default behaviour), a simple warning can be issued. Then when "datashiel.errors.stop" option is FALSE, the datashield.errors() function can be used to identify the failing servers (if there are any) and the returned value of datashield.aggregate() contains the valid results for the servers that did not fail.

This option is already available in the latest release of DSI.

Cheers
Yannick

On Tue, Dec 15, 2020 at 1:45 PM Yannick Marcon <yannick...@obiba.org> wrote:
Hi Stuart,

Thanks for the suggestion, I will have a look, in addition to the ability of running long tasks based on the preliminary work of last week's hackathon.

Best,
Yannick

On Tue, Dec 15, 2020 at 11:06 AM stuart....@arjuna.com <stuart....@arjuna.com> wrote:
Hi, my understanding is that if DSI::datashield.aggregate(...) fails, DSI::datashield.errors() can be used to obtained the individual error messages, if any, from the servers. Given that only one server may have failed, would it be possible to add a method to DSI datashield.results() to permit obtaining the results from the subset of the servers which didn't fail?
Stuart

--
You received this message because you are subscribed to the Google Groups "obiba-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to obiba-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/obiba-users/6461ee05-2161-4400-aec4-632f7a5d99ddn%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "obiba-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to obiba-users...@googlegroups.com.

Yannick Marcon

unread,
Jan 5, 2021, 5:41:04 AM1/5/21
to obiba...@googlegroups.com
Hi,

The "datashield.errors.stop" option is already available since DSI 1.1.0. 

DSI recent development has introduced long time running tasks support. We will talk about that on Thursday.

Best
Yannick




Stuart Wheater

unread,
Jan 5, 2021, 5:45:44 AM1/5/21
to obiba...@googlegroups.com
Yannick,
    Sorry I missed it was an actual option, I was looking for a method which would set the option.

Stuart
Dr Stuart Wheater
Postal Address: Arjuna Technologies Limited,
Urban Sciences Building, Newcastle Helix, Bath Lane, Newcastle upon Tyne, NE4 5TG United Kingdom
VAT Registration No: 764173128. Company registered in England
Registration No: 4497081


Yannick Marcon

unread,
Jan 5, 2021, 6:00:04 AM1/5/21
to obiba...@googlegroups.com
Reply all
Reply to author
Forward
0 new messages