retries?

[John Langley]

If the host I'm talking to with dispatch fails to respond, because for example I'm debugging it and I have it at break point, will the dispatch library send multiple requests, i.e. will it retry?

If so, is there a parameter or configuration I can set to disable this feature?

Thanks in advance, I LOVE how concise my dispatch based code can be!

-- Langley

[Nathan Hamblen]

On 01/24/2012 12:03 PM, John Langley wrote:

If the host I'm talking to with dispatch fails to respond, because for example I'm debugging it and I have it at break point, will the dispatch library send multiple requests, i.e. will it retry?

Yes, the underlying Apache http client will retry by default in certain cases:
http://hc.apache.org/httpcomponents-client-ga/tutorial/html/fundamentals.html#d4e279

If so, is there a parameter or configuration I can set to disable this feature?

Yep!

http://stackoverflow.com/questions/2285799/java-apache-httpclient-how-to-disable-retry

You just need to configure the client in your instance of dispatch.Http. Overriding make_client is usually the best way to do that.

https://github.com/dispatch/dispatch/blob/master/http/src/main/scala/Http.scala#L41

Thanks in advance, I LOVE how concise my dispatch based code can be!

Thank you!

Nathan

[Roberto Tyley]

In Dispatch-Reboot, now that it's using sonatype/async-http-client, what are the retry options? In my particular case, I'd like to know if there is a way in the library to automatically do retries when HTTP 500 error codes are encountered. In the async-http-client library the only mention of retries I can see is in NettyResponseFuture, which seems to operate retries on Network errors, but not HTTP errors.

As I write this, it occurs to me that it is perhaps only reasonable that only calling code can know how to handle HTTP errors, so I guess I should be expecting to do this work myself - I'm merely checking to see if I can be lazy.

best regards,

Roberto

[Nathan Hamblen]

On 10/08/2012 05:25 AM, Roberto Tyley wrote:
>
> As I write this, it occurs to me that it is perhaps only reasonable that
> only calling code can know how to handle HTTP errors, so I guess I
> should be expecting to do this work myself

That's probably true, but doing this asynchronously is a fairly
interesting problem. That is, if I want a Promise[String] that
represents the response after n retries, without blocking at any point.

I'm interested in solving this too, actually a harder variation of the
problem where the service in question requires exponential backoff.
We're using Dispatch to publish to Google Cloud Messaging, and for the
moment just dropping the occasional failed attempts.

I think it can be built outside of Dispatch and then, when there's a
good implementation, provided in the library as a response promise
transformer.

[Roberto Tyley]

On Monday, October 8, 2012 3:52:29 PM UTC+1, n8han wrote:

doing this asynchronously is a fairly
interesting problem. That is, if I want a Promise[String] that
represents the response after n retries, without blocking at any point.

Yup, at the risk of embarrassing myself with some pretty terrible code, here is my nasty first attempt:

https://gist.github.com/3853105

 

I'm interested in solving this too, actually a harder variation of the
problem where the service in question requires exponential backoff.

I use Thread.sleep(5000 / remainingRetries) in my example, but obviously this is a form of blocking. I'd love to see what improvements you might suggest.

We're using Dispatch to publish to Google Cloud Messaging, and for the
moment just dropping the occasional failed attempts.

Unfortunately failed attempts are quite high impact for us, aborting a critical process :(

Roberto

 

[Nathan Hamblen]

Roberto, I've made some progress on a SleepPromise:
https://github.com/dispatch/reboot/blob/retry/core/src/main/scala/sleep.scala

It's about where it needs to be. The remaining step is to make a retry interface on top of it, one that doesn't assume the conditions that require a retry.

That should be interesting but before I can do that there is some more refactoring required; I believe that Promise will be source of retries (and PromiseSleep) so it will need to hold a reference back to its HttpExecutor. This is not a huge deal but it conflicts with the idea of the Promise object as a source of non-http utility promises (Promise.apply, Promise.all).

[Nathan Hamblen]

This came out well, I merged it to master.

You can now do retries with Either, Option, or some other type that you want to define a success predicate for. It looks like this:

```
import dispatch._

def myRemoteCall: Promise[Either[String, MyObject]]

val p: Promise[Either[String, MyObject]] =
  retry.Directly(5)(myRemoteCall)
```

Anything that wasn't success is discarded, you just get the final Left or None if all attempts failed. In the dispatch.retry package there are Directly, Pause, and (exponential) Backoff retriers. None of them block internally, Pause and Backoff use a HashedWheelTimer that is now configured in HttpExecutor.

Since this was a good chunk of new work I took the opportunity to make a few changes to the library structure, some of which are breaking. Most people won't be affected by any of them. I'll document that later, before the release.

There's also some daemon thread factory joojoo in master which, unfortunately, isn't sufficient to allow the jvm to terminate without an explicit Http.shutdown. If anybody wants to try to fix this, go for it. Otherwise I'll remove that code before the release since it's not providing better behavior.

Nathan

[Sourav Chandra]

Hi,

I am trying to use dispatch retry package but it seems it does not work with asynchronius handlers onSuccess and onFailure.

Even though retry is defined, output is available after 1st run only.

As per my understanding output whether error or a valid http response must always only be available after all retry attempts are finished OR if success condition returns true

The code snippet is below:

object DispatchTest {

  type ResponseEither = Either[Throwable, Response]

  type PromiseEither = Promise[Either[Throwable, Response]]

  def retry(f: => PromiseEither)

  (implicit retryConfig: Retry): PromiseEither = {

    val failablePromise = f

    Pause(retryConfig.max, retryConfig.delay)(failablePromise){

      new Success(retryConfig.buildSuccessCondition())

    }

    failablePromise

  }

  

  case class Retry(

      successCodes: List[Int] = List(200),

      max: Int = 3,

      delay: Duration = Duration(100, TimeUnit.MILLISECONDS)

  ) {

    private[DispatchUtils] def buildSuccessCondition() = (e: ResponseEither) => {

      e match {

        case Right(r) => println("retry on success ");successCodes.contains(r.getStatusCode())

        case Left(t) => println("retry on failure");false

      }

    }

  }

 def main(args: Array[String]) {

    val svc = url("http://api1.hostip.info/country1.php")

    val http = Http()

    val p1 = http(svc)

    val p2 = p1.either

    implicit val r = Retry()

    val p3 = retry(p2)

    p3 onSuccess {

      case Left(l) => {

        println("left of onSuccess")

        println(l)

      }

      case Right(r) => {

        println("right of onSuccess")

        println(r.getStatusCode())

        println(r.getResponseBody())

//        println(p1().getStatusCode())

      } 

    }

    p3 onFailure {

      case e: Any => {

        println("onFailure")

        println(e)

      } 

    }  

  }

}

console output >

retry on failure  --> only 1 time

left of onSuccess --> no retry and output is available

java.net.ConnectException: http://api1.hostip.info/country1.php

Could you please let me know how to overcome this?

Thanks,

Sourav