What's wrong with an API which non-deterministically returns either A or B(Set(A, ...))?

[Simon Ochsenreither]

... which seems pretty much what the exception handling behavior of the parallel collections does.

If exceptions of type A occur, either an exception of type A or an exception of type B, wrapping multiple exceptions of As, is observed.
Imho, this behavior is incredibly broken and so unintuitive, that even people writing tests don't handle it correctly. See files/run/t5375.scala.

Concerning “non-deterministic”:
How many exceptions are observed before the operation is aborted depends on the machine, the available cores and hyper-threading, the operating system, the threadpool implementation and configuration, the size of the collection and the runtime itself.

Is it just me or does this feel horribly wrong?

[Aleksandar Prokopec]

Any parallel collection operation is non-deterministic in terms of its execution order.
It depends on the operator it is instantiated with if that nondeterminism is visible to the user or not.
A most notable example of this is probably `find` which may find different elements depending on the execution schedule.

However, philosophy aside, I cannot disagree with you that the CompositeException may not have been the best idea. Perhaps we should change that.

What would be the solution you suggest?

Cheers,
Alex

On 4/12/13 2:31 PM, Simon Ochsenreither wrote:

... which seems pretty much what the exception handling behavior of the parallel collections does.

If exceptions of type A occur, either an exception of type A or an exception of type B, wrapping multiple exceptions of As, is observed.
Imho, this behavior is incredibly broken and so unintuitive, that even people writing tests don't handle it correctly. See files/run/t5375.scala.

Concerning �non-deterministic�:

How many exceptions are observed before the operation is aborted depends on the machine, the available cores and hyper-threading, the operating system, the threadpool implementation and configuration, the size of the collection and the runtime itself.

Is it just me or does this feel horribly wrong?

--
You received this message because you are subscribed to the Google Groups "scala-internals" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-interna...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
�
�

-- 
Aleksandar Prokopec,
Doctoral Assistant
LAMP, IC, EPFL
http://people.epfl.ch/aleksandar.prokopec

[Rex Kerr]

Why not _always_ return CompositeException even if you only get one?  It's not like actually getting one frees the programmer from the burden of having to handle an arbitrary number.

On Fri, Apr 12, 2013 at 1:03 PM, Aleksandar Prokopec <aleksanda...@epfl.ch> wrote:

Any parallel collection operation is non-deterministic in terms of its execution order.
It depends on the operator it is instantiated with if that nondeterminism is visible to the user or not.
A most notable example of this is probably `find` which may find different elements depending on the execution schedule.

However, philosophy aside, I cannot disagree with you that the CompositeException may not have been the best idea. Perhaps we should change that.

What would be the solution you suggest?

Cheers,
Alex

On 4/12/13 2:31 PM, Simon Ochsenreither wrote:

... which seems pretty much what the exception handling behavior of the parallel collections does.

If exceptions of type A occur, either an exception of type A or an exception of type B, wrapping multiple exceptions of As, is observed.
Imho, this behavior is incredibly broken and so unintuitive, that even people writing tests don't handle it correctly. See files/run/t5375.scala.

Concerning “non-deterministic”:

How many exceptions are observed before the operation is aborted depends on the machine, the available cores and hyper-threading, the operating system, the threadpool implementation and configuration, the size of the collection and the runtime itself.

Is it just me or does this feel horribly wrong?
--
You received this message because you are subscribed to the Google Groups "scala-internals" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-interna...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

-- 
Aleksandar Prokopec,
Doctoral Assistant
LAMP, IC, EPFL
http://people.epfl.ch/aleksandar.prokopec

--

[Simon Ochsenreither]

I suggest returning some ParallelExecutionException in all cases. (Or have a look at the name/type Java chose for their implementation.) That way, the name doesn't imply whether it wraps one or multiple exceptions.

But returning multiple, structurally different exceptions non-deterministically is not acceptable, as demonstrated by the immediate test failure as soon as the test was run on non-HotSpot runtime.

[Som Snytt]

> as demonstrated by the immediate test failure

Philosophy aside, tests often operate on a narrowly prescribed domain (as I just rediscovered in a different context).

If a test did not fail on another JVM, you might expect that the tests are inadequate.

I like the green bar as much as the next guy, but only after a failure.  "You mean my test wasn't run?  You mean the failure happened on some other thread and wasn't reported?  You mean I wasn't testing what I thought I was testing?" and so on.

On Fri, Apr 12, 2013 at 10:18 AM, Simon Ochsenreither <simon.och...@gmail.com> wrote:

I suggest returning some ParallelExecutionException in all cases. (Or have a look at the name/type Java chose for their implementation.) That way, the name doesn't imply whether it wraps one or multiple exceptions.

But returning multiple, structurally different exceptions non-deterministically is not acceptable, as demonstrated by the immediate test failure as soon as the test was run on non-HotSpot runtime.

[Aleksandar Prokopec]

Makes sense to me.
On the subject of handling them, seems to me that the programmer would always have to write this:

def handle(e: Exception) = e match {
  case ... =>
  case ... =>
}
try {
  xs.par.find(_ < 0)
} catch {
  case CompositeException(exs) => for (e <- exs) handle(e)
}

Maybe a custom `catch` variant could simplify exception handling - something along the lines of:

try {
  xs.par.find(_ < 0)
} catchAll {
  case ... =>
  case ... =>
}

so that if a single, non-parallel-collection-related exception is thrown, it is handled as a single exception, and if a composite coming from a parallel collection is thrown, all the sub exceptions are handled.
Seems more concise to me.

What do you think?

On 4/12/13 7:18 PM, Simon Ochsenreither wrote:

I suggest returning some ParallelExecutionException in all cases. (Or have a look at the name/type Java chose for their implementation.) That way, the name doesn't imply whether it wraps one or multiple exceptions.

But returning multiple, structurally different exceptions non-deterministically is not acceptable, as demonstrated by the immediate test failure as soon as the test was run on non-HotSpot runtime.

[Simon Ochsenreither]

I think approaches which depend on people having to correctly duplicate their error handling, once for an application-specific exception and once for a generic exception wrapper, are too error-prone.
Comparing that to traditional API design, this would be like having a return type of

/** Returns an A if one element in list conforming to predicate is found
  * and List[A] if more than one element is found. */
def findAll[A](list: List[A], predicate: A => Boolean): A with List[A]

instead of

/** Returns a List[A] of all elements in list conforming to predicate. */
def findAll[A](list: List[A], predicate: A => Boolean): List[A]

[Simon Ochsenreither]

Philosophy aside, tests often operate on a narrowly prescribed domain (as I just rediscovered in a different context).

The point is that the test failed, because it was based on certain timing assumption which are heavily dependent on external factors.
Of course I could just simply fix the test, but the fact that this innocently looking piece of code went into the test-suite without anyone realizing that it was broken means that the API is not right.
 

If a test did not fail on another JVM, you might expect that the tests are inadequate.

Yes, but I would be happier if it was the JVM's fault and not Scala's.

[Rex Kerr]

What is wrong with just giving the CompositeException object a partial function creation method like so:

try { xs.par.find(_ < 0) }

catch CompositeException.handler {

  case npe: NullPointerException => println("Argh, Java got me!")

  case _ => println("Um, no idea, man.")
}

 

  --Rex

--

[Aleksandar Prokopec]

This looks good to me.

Cheers,
Alex

[Simon Ochsenreither]

After thinking about this, I propose just throwing the "first" exception which occurs.
This seems to be what C# and Java both do, too.

Even consistently returning CompositeThrowable doesn't make much sense (because we have fail-fast behaviour and don't wait until all tasks have finished or have thrown an exception).
Therefore, there is no useful semantic in having a CompositeThrowable which returns "some" exceptions. Either we gather and return all the exceptions or we just return one, instead of wrapping a non-deterministic number of exceptions into a completely unrelated wrapper type.

Considering that changing the parallel collection semantics in such a profound way is out of question, I propose the second option.

As soon as we are targeting Java > 7 we can use addSurpressed to add further exceptions which also occurred.

[Simon Ochsenreither]

https://issues.scala-lang.org/browse/SI-7474

[Simon Ochsenreither]

https://github.com/scala/scala/pull/2563