Handling exceptions in upstream futures with dataflow

[Arindam Mukherjee]

If I have three futures f1, f2 and f3 and I run the following:

Concurrent.dataflow(f1, f2) do |v1, v2|
  f3.execute
end

What is expected to happen if f2 throws an exception? I noticed that f3 runs once both f1 and f2 complete, irrespective of whether either of them was rejected.

How do I ensure that f3 is run only if f1 and f2 are both fulfilled?

Arindam

[Arindam Mukherjee]

Here is a working example:

require 'concurrent'

f1 = Concurrent::Future.new {
  puts "Executing f1"
  sleep rand * 6
  raise StandardError.new("Raising exception during execution of f1")
  puts "done executing f1"
  rand
}

f2 = Concurrent::Future.new {
  puts "Executing f2"
  sleep rand * 6
  raise StandardError.new("Raising exception during execution of f2")
  puts "done executing f2"
  rand
}

f3 = Concurrent::Future.new {
  puts "Executing f3"
  sleep rand * 3
  puts "done executing f3"
  rand
}

f4 = Concurrent::Future.new {
  puts "Executing f4"
  sleep rand * 3
  puts "done executing f4"
  rand
}

p = Concurrent.dataflow(f1, f2) {
  f3.execute
}

Concurrent.dataflow(f3) {
  f4.execute
}

f1.execute
f2.execute

sleep 10

The graph of dependencies is

f1
 |
 v
f3 --> f4
 ^
 |
f2

Since both f1 and f2 fail, I would have expected f3 and f4 to not be run at all. However both are run.

Thanks,
Arindam

[Chris Seaton]

I’m afraid I don’t think I considered exceptions when I wrote this dataflow stuff, so what you see might be what you get.

If you have an idea of how it should be improved to do what you think makes sense, please open an issue and we can implement that.

Chris

--
You received this message because you are subscribed to the Google Groups "Concurrent Ruby" group.
To unsubscribe from this group and stop receiving emails from it, send an email to concurrent-ru...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Arindam Mukherjee]

I'll try to summarize what I think would be sensible behavior. I think the current behavior is to start a future once the predecessor futures are all either fulfilled or rejected. This is reasonable behavior, but the dependent future should ideally have a means of figuring out that there was a failure in one or more of its predecessors. I didn't see a way to do that.

Arindam

[Petr Chalupa]

Did you have a look at Promises, it might be a better fit.

[Arindam Mukherjee]

I had some trouble integrating Promises, but maybe I will go back and try again.

[Arindam Mukherjee]

On Saturday, 30 July 2016 00:18:06 UTC+5:30, Petr Chalupa wrote:

Did you have a look at Promises, it might be a better fit.

I am looking at Promises for an alternative implementation. Can I use Concurrent::Promise.all? to simulate dataflow type dependency?

The problem is that if I have three promises p1, p2 and p3, and I run this:

Concurrent::Promise.all?(p1, p2, p3).then do

   stuff

end

Then p1, p2 and p3 run serially in that order. Is there a way to ensure they can run concurrently?

Regards,

Arindam

[Petr Chalupa]

That might be because of an inefficient implementation of all? (It will be replaced in next version). Please try zip instead. Or new promises from edge, the API is in a process of renaming though.

To unsubscribe from this group and stop receiving emails from it, send an email to concurrent-ruby+unsubscribe@googlegroups.com.