Akka HTTP Future response too late - scala

Question

I want to ban a URL from varnish. What I do first is collect all the healthy IP's from consul.
private def nodeInfo:Future[List[NodeInfo]] = {
val request = HttpRequest(method = HttpMethods.GET, uri = consulUrl)
Future.successful {
Http().singleRequest(request).flatMap(response =>
response.status match {
case OK => Unmarshal(response).to[List[NodeInfo]]
case _ =>
response.entity.toStrict(5.seconds).flatMap { entity =>
val body = entity.data.decodeString("UTF-8")
log.warning(errorMessage(request, response, body))
Future.failed(new IOException(errorMessage(response, body)))
}
})
}.flatMap(value => value)
}
This works as expected.
With the help of for comprehension I want to loop through all of them.
def banFromCache(toBanUrl:String): Future[String] = {
for {
nodes <- nodeInfo
result <- loopNodes(nodes, toBanUrl)
} yield result
}
With a foreach loop I send HttpRequest and get HttpResponses for each. But caused by the Future functionality the result yield is done before the request are completed.
private def loopNodes(nodes:List[NodeInfo], toBanUrl:String):Future[String] = Future {
val banResult = new ListBuffer[String]
nodes.foreach(node => {
banAllHealthy(node, toBanUrl).onComplete {
case Failure(err) =>
banResult += node.Node.Address + " " + err.getMessage
log.error("Request failed: " + node.Node.Address + " " + err.getMessage)
case Success(res) =>
banResult += node.Node.Address + " " + res.toString
log.info("Request success: " + node.Node.Address + " " + res.toString)
}
})
banResult.toList.toString()
}
private def banAllHealthy(nodeInfo:NodeInfo, toBanUrl: String):Future[HttpResponse] = {
def request(): Future[HttpResponse] =
Http().singleRequest(HttpRequest(method = HttpMethods.GET, uri = "http://localhost:9000/healthcheck"))
//Http().singleRequest(HttpRequest(method = HttpMethods.GET, uri = "http://" + nodeInfo.Node.Address + "/" + toBanUrl))
val responseFuture: Future[HttpResponse] = request()
responseFuture
}
The route is quite simple here:
} ~ pathPrefix("ban") {
pathPrefix(Segment) { banpath =>
pathEndOrSingleSlash {
get {
complete(banFromCache(banpath).map(_.asJson))
}
}
}
Is there a way to show all the responses at once ?

Answer 1

To accumulate the result strings, stay within the context of a Future:
private def loopNodes(nodes: List[NodeInfo], toBanUrl: String): Future[String] = {
val futures: List[Future[String]] = nodes.map { node =>
banAllHealthy(node, toBanUrl)
.map(res => s"${node.Node.Address} ${res}")
.recover { case err => s"${node.Node.Address} ${err.getMessage}" }
}
Future.reduceLeft(futures)(_ + "\n" + _)
}

Answer 2

If you put the callbacks to handle the result of each future you will lost the control of the execution. With the foreach, each future lives in its own parallel execution. The parent future yields because it does not to wait for anything. I would suggest to not use the ListBuffer and use a more immutable style. Anyway, try to build the whole computation as just one future which encapsulates the whole computation:
private def loopNodes(nodes:List[NodeInfo], toBanUrl:String):Future[String] = {
nodes.map { node =>
// Creates a tuple to store current node and http result
// (Node, result, HttpResult)
(node, "", banAllHealthy(node, toBanUrl))
}.foldLeft(Future(""))((str, b) =>
b match {
case (node, str ,response) => {
// Each response will be transformed to string
(response map (result => str + " " + node.Node.Address + " " + "Success"))
// In case of node is not available its suppose that the HttpClient will raise an execption
.recover {
case err: Throwable =>
str + " " + node.Node.Address + " " + "Error " + err.getMessage
case _ =>
str + " " + node.Node.Address + " " + "Unknown Error"
}
}
})
}