Lagom with Java: Circuit breaker for a managed service

[pankhur...@knoldus.in]

Hi,

My question is regarding implementation of custom circuit breaker for a managed service in lagom. 

Suppose I have the following path call in my API

public interface DemoService extends Service {

ServiceCall<NotUsed, DemoResponsePOJO> getDemoResponse(String param1, String param2, Optional<String> param3, Optional<String> param4);

default Descriptor descriptor() {

   Descriptor descriptor = named("demo").withCalls(

         pathCall("/version1/demo/:param1/:param2?param3&param4", this::getDemoResponse)

   ).withAutoAcl(true);

   return descriptor;

}

}

In my IMPL (DemoServiceImpl.java), I am first doing some error handling for the parameters (param1 through param4). 

If the format of any of the parameters is wrong, I'm returning a 400 status code, i.e., Bad parameter. 

If the format of all the parameters is correct, I'm hitting an external Solr search client from the IMPL, retrieving the Solr search response, and then mapping it to the expected API's response (DemoResponsePOJO) 

Now in general, in case of unmanaged services, the Circuit breaker flow goes as - whenever the third party client is not giving a response for a threshhold number of times (due to bad connection or system overload), the circuit breaker trips and for the specified reset-timeout duration, our API returns a CircuitBreakerOpenException for any request. 

Problem: In my case, I want to use the circuit breaker to help manage the load on my Solr search client. But I'm observing an unwanted behaviour, that is, if I give some bad parameters (I tried running a number of negative test cases consecutively), the circuit breaker trips. My understanding is that even though the search client is not even hit once, my API is giving an invalid response (400), and since the circuit breaker is not concerned with the IMPL, it doesn't care whether the error is in 400 series, or in 500 series. 

Now, I have two questions: 

1. Can we customise the circuit breaker to trip only when the error response is in 500 series? 

2. If not, how do I solve my problem?

Thanks for reading through my query. Looking forward to your response. 

Pankhurie

[Tim Moore]

Hi Pankhurie,

Circuit breakers are a client side concern, not something that runs on the server side. What's happening is that your client is getting back too many error responses, and decides to stop sending requests to the server.

If your client is routinely getting 400 error responses back from the server, that usually indicates either that the client has a bug in it, or that it is not sufficiently validating parameters before making the request. In your case, you're intentionally sending erroneous requests. For testing purposes, you might want to disable the circuit breaker on the client side.

Check the docs for details: https://www.lagomframework.com/documentation/1.3.x/java/ServiceClients.html#Circuit-Breakers

The circuit breaker trips on all failures, and Lagom currently considers all HTTP status codes >= 400 && < 600 to be failures, so it isn't possible to customize the error range, but I'm not sure that's actually what you really want to do.

Cheers,

Tim

--
You received this message because you are subscribed to the Google Groups "Lagom Framework Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lagom-framework+unsubscribe@googlegroups.com.
To post to this group, send email to lagom-framework@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lagom-framework/a55f2677-4ab8-4cbb-a478-fe7475e30238%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Tim Moore

Senior Engineer, Lagom, Lightbend, Inc.

tim....@lightbend.com
+61 420 981 589
Skype: timothy.m.moore