Extracting and accumulate result from multiple HTTP request - scala

Question

I met a problem that I cannot solve these days. I'm doing multiple http requests and in each response, it should have a Array[DTAnnotation]. I want to accumulate all the resulting list into one (this is not the problem here). My problem is that I cannot return the results from a WSResponse. What I try :
import mymodel.{DTFeatures, DTResponse, DTRequest, DTAnnotations}
def checkForSpike
(
awsKey : String,
collection : JSONCollection,
record : Record,
features : Array[DTFeatures]
) : Unit = {
val url = config.getString("url").getOrElse
val holder = WS.url(url)
val complexHolder =
holder.withHeaders(("Content-Type","application/json"))
// excepting result is List[Array[DTAnnotations]]
val test : List[Array[DTAnnotations]] =
for(feature <- features) yield {
val dtFeature = Json.stringify(Json.toJson(DTRequest(feature)))
val futureResponse = complexHolder.post(dtFeature)
Logger.info("Make the HTTP POST Request in " + (t1 - t0) + " msecs")
futureResponse.map { response =>
Logger.info("Get response in " + (System.currentTimeMillis - t1))
if(response.status == 200) {
response.json.validate[DTResponse].map { dtResponse =>
// I want to return this
dtResponse.annotations
}.recoverTotal { _ =>
Logger.debug("invalid json")
}
} else {
Logger.debug(Json.prettyPrint(Json.obj("status" -> response.status, "body" -> response.body)))
}
}
Await.result(futureResponse, 10.seconds)
}
}
Because the response is a Future, I try to add a Await to get the annotations but I have one error at the typing phase :
[error] found : Array[play.api.libs.ws.WSResponse]
[error] required: List[Array[DTAnnotation]]
How I can fix this ? Thank you !

Answer 1

There are a number of errors that avoid this to work. I add a version that compiles with your expected type and if you have questions I will answer in the comments.
def checkForSpike
(
awsKey: String,
collection: JSONCollection,
record: Record,
features: Array[DTFeatures]
): Unit = {
val url = config.getString("url").getOrElse
val holder = WS.url(url)
val complexHolder =
holder.withHeaders(("Content-Type", "application/json"))
// excepting result is List[Array[DTAnnotations]]
val test: List[Array[DTAnnotations]] =
for (feature <- features.toList) yield {
val dtFeature = Json.stringify(Json.toJson(DTRequest(feature)))
val futureResponse = complexHolder.post(dtFeature)
val futureAnnotations: Future[Array[DTAnnotations]] = futureResponse.map { response =>
if (response.status == 200) {
response.json.validate[DTResponse].map { dtResponse =>
// I want to return this
dtResponse.annotations
}.recoverTotal { _ =>
Logger.debug("invalid json")
??? // An Array response should be expected, maybe empty
}
} else {
Logger.debug(Json.prettyPrint(Json.obj("status" -> response.status, "body" -> response.body)))
??? // An Array response should be expected, maybe empty
}
}
Await.result(futureAnnotations, 10.seconds)
}
}
Issues:
Features must be converted to List if you expect a list to be
returned by the for comprehension
The map on future response returns
another future and this value should be used in the Await
To make sure the type of futureAnnotations is correct in all the branches the type should be valid