Where to put dispatch.Http.shutdown() in case of nested Http calls
193 views
Skip to first unread message
Normen Müller
Feb 15, 2014, 2:25:33 PM2/15/14
to dispatc...@googlegroups.com
Hi,
I'm having the problem that my command line client is not terminating having (nested) Http calls.
In the following I explain my problem via code snippets. Example2 solves the non-terminating problem of Example1 (by explicitly calling `Http.shutdown()`), but I don't know how to solve Example3. Please note, all Http requests are only sample requests to the GitHub API. They do not really make sense. They serve just as sample calls.
Also I would like to get feedback on Example2, say, if this is the "proper" way to do it.
Any help is highly appreciated... currently I'm really stuck, Best, /nm
P.S. In order that I can grep a better understanding, please don't suggest to use Akka actors or something similar. First I would like to get these snippets solved with "basic" Scala and Dispatch. Thank again, in advance!
import dispatch._
import scala.util.{ Failure, Success }
import com.typesafe.scalalogging.slf4j.Logging
object Main extends App with Logging {
import scala.concurrent.ExecutionContext.Implicits.global
// All request are taken from `http://developer.github.com/guides/getting-started/`
// I'm using `net.databinder.dispatch" %% "dispatch-core" % "0.11.0"` and
// `net.databinder.dispatch" %% "dispatch-json4s-native" % "0.11.0`
val h = host("api.github.com").secure
def example1() = {
logger.debug("Example1")
// `outer1` and `outer2` are independent request
// Request `outer1`
Http(h / "users" / "defunkt" OK as.json4s.Json) onComplete {
case Success(json) => logger.debug(s"Example 1 Outer 1 Output: ${json.toString()}")
case Failure(error) => logger.debug(s"Example 1 Outer 1 Error: ${error.toString()}")
}
// Request `outer2`
Http(h / "repos" / "twbs" / "bootstrap" OK as.json4s.Json) onComplete {
case Success(json) => logger.debug(s"Example 1 Outer 2 Output: ${json.toString()}")
case Failure(error) => logger.debug(s"Example 1 Outer 2 Error: ${error.toString()}")
}
// >>> program not terminating <<<
}
def example2() = {
logger.debug("Example2")
// again, `outer1` and `outer2` are independent request
// Request `outer1`
val outer1 = Http(h / "users" / "defunkt" OK as.json4s.Json)
outer1 onComplete {
case Success(json) => logger.debug(s"Example 2 Outer 1 Output: ${json.toString()}")
case Failure(error) => logger.debug(s"Example 2 Outer 1 Error: ${error.toString()}")
}
// Request `outer2`
val outer2 = Http(h / "repos" / "twbs" / "bootstrap" OK as.json4s.Json)
outer2 onComplete {
case Success(json) => logger.debug(s"Example 2 Outer 2 Output: ${json.toString()}")
case Failure(error) => logger.debug(s"Example 2 Outer 2 Error: ${error.toString()}")
}
Future.sequence(outer1 :: outer2 :: Nil) onComplete { case _ => Http.shutdown() }
// >>> program terminating <<<
}
def example3() = {
logger.debug("Example3")
// again, `outer1` and `outer2` are independent request
// but this time each of them has dependent requests
// which are independt of each other, though
// Request `outer1`
val outer1 = Http(h / "users" / "defunkt" OK as.json4s.Json)
outer1 onComplete {
case Success(json) =>
logger.debug(s"Example 3 Outer 1 Output: ${json.toString()}")
// `inner11`
Http(h / "users" / "technoweenie" / "repos" OK as.json4s.Json) onComplete {
case Success(json) => logger.debug(s"Example 3 Inner11 Output: ${json.toString()}")
case Failure(error) => logger.debug(s"Example 3 Inner11 Error: ${error.toString()}")
}
// `inner12`
Http(h / "orgs" / "mozilla" / "repos" OK as.json4s.Json) onComplete {
case Success(json) => logger.debug(s"Example 3 Inner12 Output: ${json.toString()}")
case Failure(error) => logger.debug(s"Example 3 Inner12 Error: ${error.toString()}")
}
case Failure(error) => logger.debug(s"Example 3 Outer 1 Error: ${error.toString()}")
}
// Request `outer2`
val outer2 = Http(h / "repos" / "twbs" / "bootstrap" OK as.json4s.Json)
outer2 onComplete {
case Success(json) =>
logger.debug(s"Example 3 Outer 2 Output: ${json.toString()}")
// `inner21`
Http(h / "repos" / "rails" / "rails" / "issues" OK as.json4s.Json) onComplete {
case Success(json) => logger.debug(s"Example 3 Inner21 Output: ${json.toString()}")
case Failure(error) => logger.debug(s"Example 3 Inner21 Error: ${error.toString()}")
}
// `inner22`
Http(h / "users" / "normenmueller" OK as.json4s.Json) onComplete {
case Success(json) => logger.debug(s"Example 3 Inner22 Output: ${json.toString()}")
case Failure(error) => logger.debug(s"Example 3 Inner22 Error: ${error.toString()}")
}
case Failure(error) => logger.debug(s"Example 3 Outer 2 Error: ${error.toString()}")
}
Future.sequence(outer1 :: outer2 :: Nil) onComplete { case _ => Http.shutdown() }
// >>> program terminating, but inner HTTP calls are canceled <<<
// >>> and an exception is thrown <<<
}
//example1()
//example2()
example3()
}
Nathan Hamblen
Feb 17, 2014, 11:54:39 PM2/17/14
to dispatc...@googlegroups.com
On 02/15/2014 02:25 PM, Normen Müller wrote:
> In the following I explain my problem via code snippets. Example2 solves
> the non-terminating problem of Example1 (by explicitly calling
> `Http.shutdown()`), but I don't know how to solve Example3.
That should be fairly straightforward, you can flatMap the inner futures
> In the following I explain my problem via code snippets. Example2 solves
> the non-terminating problem of Example1 (by explicitly calling
> `Http.shutdown()`), but I don't know how to solve Example3.
to the outer futures so that the result is flat Future. Then you can
call Future.sequence( ... ) onComplete against these flat outer futures
and the callback should not fire until the inner ones have completed as
well.
As far as the inexplicit terminating behavior goes, all I can say is I
made it work as well as I could. If you look back through these mailing
list archives and release notes you can see the story there. Things
behave differently depending on whether you're running inside sbt or
not, because they had to.
Nathan
Reply all
Reply to author
Forward
0 new messages