FaultTolerance TCK tests results

[Matthieu Brouillard]

Hi list,

are there already implementations that pass the fault tolerance 1.0 TCK tests? If yes, is it without modifications?

I saw https://github.com/eclipse/microprofile-fault-tolerance/pull/193 poping up and have also difficulties with some other tests.

For example BulkheadAsynchRetryTest#testBulkheadMethodAsynchronousRetry55 expects 20 iterations, but I think it might be 10.

Thanks for hints.

Matthieu Brouillard

[Emily Jiang]

Open Liberty (https://openliberty.io/) passed all of the tests without any modification. Please join this week's FT hangout to discuss your difficulties. Please see the MicroProfile calendar for the meeting info. In this week's call, I will go through all outstanding PRs and issues. By the way, I am having issues with the PR you mentioned.

Emily

[John D. Ament]

Hi Matthieu,

I can tell you first hand, the TCK tests much more than is specified.  It leaves an implementor very confused when it comes down to it.  I believe where you're struggling is that @Bulkhead + @Asynchronous should not result in an asynchronous execution, but instead that submits executions to a threadpool.  Apache Safeguard doesn't pass 100% of the TCK yet (two functional areas missing), but if you want to review its at https://github.com/apache/geronimo-safeguard

Granted, both Open Liberty and Safeguard are based on Failsafe, and I'm using AsyncFailsafe to execute anything async, I do treat this combination as not async - https://github.com/apache/geronimo-safeguard/blob/master/safeguard-impl/src/main/java/org/apache/safeguard/impl/executionPlans/ExecutionPlanFactory.java#L190-L193

John

[Matthieu Brouillard]

Hi,

Emily & John, thanks already for your first answers.

>>> I can tell you first hand, the TCK tests much more than is specified

I saw that and it is very good. It is a huge & complicated job to write such a TCK, thank you for that also.

You're right John the "bulkhead + asynchronous" part is the one that causes me the most troubles.

If I come back to my example 'BulkheadAsynchRetryTest#testBulkheadMethodAsynchronousRetry55', here the tests are failing because my implementation throws:

org.eclipse.microprofile.faulttolerance.exceptions.BulkheadException: cannot acquire a slot in bulkhead waiting queue

Reading the spec and so on, I do not see anything that says it should not be the case. In this particular example 20 calls are performed against something that can absorb only 10 (5+5).

I just want to be sure that the tck tests do not rely on some intrinsic & hidden behavior expected by the implementations. The tests are in some parts a bit difficult to understand & to follow (which is "normal" when it comes to asynchronous) and thus I want to be sure to spend my time correctly by not trying to understand a problem in my implementation that could be caused by an unexpected/partially wrong expectation of the test itself.

@Emily: I'll try to join tomorrow for the FT meeting.

Matthieu

[John D. Ament]

Matthieu,

Ok, so if that's your issue, it's likely due to the order of operations.  This is also not specified behavior, but retry should be an outer call to the bulkhead.  That means that even though you have a max queue of 10 entries, the retries should drain quick enough that it eventually does complete.  If you look at this code block, you'll see that the callable behavior is described as invoking the bulkhead.

I personally don't know how useful a test like that is, or how real life it represents.

John

[Matthieu Brouillard]

Ok I got it for this test.

Thanks.