Calling multiple web services - scala

I'm using Play 2.5 with Scala, I created a class that will call multiple times external web services.
External web services are called on some condition and get a simple response of ok or nok. If it is ok then I should update internal objects status and if nok I do nothing for now.
Here is my class, it takes a list of list of String as paramters and return a Future list of object to be handled in the controller.
def callWSAndGetResponse(listOfList: List[List[String]]): Future[List[MyObject]] = {
val res = listOfList map { listOfIds =>
listOfIds map { id =>
val foundObj = allMyObject.find(obj => obj.id == id)
if(foundObj.isDefined) {
foundObj.get.urls map { url =>
val futureReponse: Future[WSResponse] = ws.url(url).get()
futureResponse map { response =>
(response.json \ "response").as[String]
}
}
}
}
}
// if responses are ok create a list of MyObject to return for example.
}
val res is of type list of list but I would like it to be just a simple list of response.
1) How to simplify and correct my code in order to get just a list of response, for later check if ok or not ?
2) How to check if responses are ok or have failed ?

It looks like you want this block to return a List[Future[Json]], from then you can use "List[Future] to Future[List] disregarding failed futures" or similar.
To do this you should use map and flatMap (rather than isDefined/get):
val res = listOfList map {
_.map { id =>
allMyObject
.find(obj => obj.id == id)
.map {
_.flatMap {url =>
val futureReponse: Future[WSResponse] = ws.url(url).get()
futureResponse map { response =>
(response.json \ "response").as[String]
}
}
}
}
}
if blocks assign to AnyVal rather than to a specific type (which will cause you issues here):
scala> if (true) 23
23: AnyVal
scala> if (false) 23
(): AnyVal
Using map keeps you in the Option monad:
scala> (None: Option[Int]).map(_ * 2)
None: Option[Int]

Related

Evaluating a Future Boolean asynchronously using Scala and Play

I have a method that returns a Future[Boolean] in a Play controller and i want to evaluate that using async but i can't seem to get it to compile.
The following will work:
def health = Action {
logger.info("Endpoint method: health")
val isHealthy = healthCheckService.checkDynamo()
val b: Boolean = Await.result(isHealthy, scala.concurrent.duration.Duration(5, "seconds"))
Ok(Json.toJson(HealthCheckResponse(b.toString)))
}
But i don't think i want that Await in there. So i'm trying things like this with no success:
def health =
Action.async {
Future {
logger.info("Endpoint method: health")
healthCheckService.checkDynamo() match {
case Future.successful(true) => Ok(Json.toJson("false"))
case false => Ok(Json.toJson("true"))
}
val r = healthCheckService.checkDynamo() match {
case true => Ok(Json.toJson("false"))
case false => Ok(Json.toJson("true"))
}
}
}
I can't even get those to compile to test them out.
Any suggestions?
Try this:
def health = Action.async {
healthCheckService.checkDynamo().map {
case true => Ok(Json.toJson("false"))
case false => Ok(Json.toJson("true"))
}
}
Let Play handle the awaiting for you under the hood. That is, Action.async accepts a Future, which checkDynamo() already returns. All you have to do is map it to the appropriate result.
With Futures you have to use combinators like map and flatMap to express the final value. For example:
Action.async {
healthCheckService.checkDynamo()
.map { result => // boolean
HealthCheckResponse(result.toString)
}
.map(Json.toJson(_))
.map(Ok(_))
}
(You can merge maps above to one map and construct the final Ok value there; it is more or less a matter of taste)
If you have, say, two async calls which you want to execute and return a result based on their results, you can use flatMap, which could be easily expressed using a for comprehension:
Action.async {
for {
result1 <- someService.someCall()
result2 <- anotherService.anotherCall(result1.someProperty)
finalResult = SomeFinalResultType(result1, result2)
} yield Ok(Json.toJson(finalResult))
}
If you are not familiar with futures, you might want to read some tutorial which explains their nature, how to combine them and how to get useful results from them, like this one: http://hello-scala.com/920-scala-futures.html

Scala Play: how to wait until future is complete before OK result is returned to frontend

In my playframework application I want to wait until my future is completed and the return it to the view.
my code looks like:
def getContentComponentUsageSearch: Action[AnyContent] = Action.async { implicit request =>
println(request.body.asJson)
request.body.asJson.map(_.validate[StepIds] match {
case JsSuccess(stepIds, _) =>
println("VALIDE SUCCESS -------------------------------")
val fList: List[Seq[Future[ProcessTemplatesModel]]] = List() :+ stepIds.s.map(s => {
processTemplateDTO.getProcessStepTemplate(s.processStep_id).flatMap(stepTemplate => {
processTemplateDTO.getProcessTemplate(stepTemplate.get.processTemplate_id.get).map(a => {
a.get
})
})
})
fList.map(u => {
val a: Seq[Future[ProcessTemplatesModel]] = u
Future.sequence(a).map(s => {
println(s)
})
})
Future.successful(Ok(Json.obj("id" -> "")))
case JsError(_) =>
println("NOT VALID -------------------------------")
Future.successful(BadRequest("Process Template not create client"))
case _ => Future.successful(BadRequest("Process Template create client"))
}).getOrElse(Future.successful(BadRequest("Process Template create client")))
}
the pirntln(s) is printing the finished stuff. But how can I wait until it is complete and return it then to the view?
thanks in advance
UPDATE:
also tried this:
val process = for {
fList: List[Seq[Future[ProcessTemplatesModel]]] <- List() :+ stepIds.s.map(s => {
processTemplateDTO.getProcessStepTemplate(s.processStep_id).flatMap(stepTemplate => {
processTemplateDTO.getProcessTemplate(stepTemplate.get.processTemplate_id.get).map(a => {
a.get
})
})
})
} yield (fList)
process.map({ case (fList) =>
Ok(Json.obj(
"processTemplate" -> fList
))
})
but then I got this:
UPDATE:
My problem is that the futures in fList do not complete before an OK result is returned
The code in the question didn't seem compilable, so here is an untested very rough sketch, that hopefully provides enough inspiration for further search of the correct solution:
def getContentComponentUsageSearch: = Action.async { implicit req =>
req.body.asJson.map(_.validate[StepIds] match {
case JsSuccess(stepIds, _) => {
// Create list of futures
val listFuts: List[Future[ProcessTemplatesModel]] = (stepIds.s.map(s => {
processTemplateDTO.
getProcessStepTemplate(s.processStep_id).
flatMap{ stepTemplate =>
processTemplateDTO.
getProcessTemplate(stepTemplate.get.processTemplate_id.get).
map(_.get)
}
})).toList
// Sequence all the futures into a single future of list
val futList = Future.sequence(listFuts)
// Flat map this single future to the OK result
for {
listPTMs <- futList
} yield {
// Apparently some debug output?
listPTMs foreach printl
Ok(Json.obj("id" -> ""))
}
}
case JsError(_) => {
println("NOT VALID -------------------------------")
Future.successful(BadRequest("Process Template not create client"))
}
case _ => Future.successful(BadRequest("Process Template create client"))
}).getOrElse(Future.successful(BadRequest("Process Template create client")))
}
If I understood your question correctly, what you wanted was to make sure that all futures in the list complete before you return the OK. Therefore I have first created a List[Future[...]]:
val listFuts: List[Future[ProcessTemplatesModel]] = // ...
Then I've combined all the futures into a single future of list, which completes only when every element has completed:
// Sequence all the futures into a single future of list
val futList = Future.sequence(listFuts)
Then I've used a for-comprehension to make sure that the listPTMs finishes computation before the OK is returned:
// Flat map this single future to the OK result
for {
listPTMs <- futList
} yield {
// Apparently some debug output?
listPTMs foreach printl
Ok(Json.obj("id" -> ""))
}
The for-yield (equivalent to map here) is what establishes the finish-this-before-doing-that behavior, so that listPTMs is fully evaluated before OK is constructed.
In order to wait until a Future is complete, it is most common to do one of two things:
Use a for-comprehension, which does a bunch of mapping and flatmapping behind the scenes before doing anything in the yield section (see Andrey's comment for a more detailed explanation). A simplified example:
def index: Action[AnyContent] = Action.async {
val future1 = Future(1)
val future2 = Future(2)
for {
f1 <- future1
f2 <- future2
} yield {
println(s"$f1 + $f2 = ${f1 + f2}") // prints 3
Ok(views.html.index("Home"))
}
}
Map inside a Future:
def index: Action[AnyContent] = Action.async {
val future1 = Future(1)
future1.map{
f1 =>
println(s"$f1")
Ok(views.html.index("Home"))
}
}
If there are multiple Futures:
def index: Action[AnyContent] = Action.async {
val future1 = Future(1)
val future2 = Future(2)
future1.flatMap{
f1 =>
future2.map {
f2 =>
println(s"$f1 + $f2 = ${f1 + f2}")
Ok(views.html.index("Home"))
}
}
}
}
When you have multiple Futures though, the argument for for-yield comprehensions gets much stronger as it gets easier to read. Also, you are probably aware but if you work with futures you may need to following imports:
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global

Combining/chaining futures in scala play framework async action

I'm a scala newbie trying to write a Rest Api using play framework. I have the following 3 data access methods
getDataDict: (dsType:String, name:String) => Future[Option[DatasetDictionary]]
getDatasetData: (DatasetDictionary) => Future[List[DatasetData]]
getMetadata: (DatasetDictionary) => Future[List[Metadata]]
I need to use these 3 methods to get the result of my async action method.
def index(dstype:String, name:String, metadata:Option[Boolean]) = Action.async{
/*
1. val result = getDataDict(type, name)
2. If result is Some(d) call getDatasetData
3.1 if metadata = Some(true)
call getMetadata function
return Ok((dict, result, metadata))
3.2 if metadata is None or Some(false)
return Ok(result)
4. If result is None
return BadRequest("Dataset not found")
*/
}
I got the steps 1 and 2 working as follows
def index1(dsType:String, dsName: String, metadata:Option[Boolean]) = Action.async {
getDataDict(dsType, dsName) flatMap {
case Some(x) => getDatasetData(x) map (x => Ok(Json.toJson(x)))
case None => Future.successful(BadRequest("Dataset not found"))
}
}
I'm stuck at how to get the metadata part working.
First of all, it is not very clear (d, result, x) what you really want to return. Hopefully I guessed it correctly:
def index(dstype:String, name:String, metadata:Option[Boolean]) = Action.async {
getDataDict(dstype, name) flatMap {
case Some(datasetDictionary) =>
getDatasetData(datasetDictionary) flatMap { datasetDataList =>
if (metadata == Some(true)) {
getMetadata(datasetDictionary) map { metadataList =>
Ok(Json.toJson((datasetDictionary, datasetDataList, metadataList)))
}
} else {
Future.successful(Ok(Json.toJson(datasetDataList)))
}
}
case None => Future.successful(BadRequest("Dataset not found"))
}
}

How to add state to a Play Enumerator?

To feed Play response I want to pass Enumerator to result's feed method. I need to pass a state from produce/consume iteration step to the next step (or to keep state). Here
http://engineering.klout.com/2013/01/iteratees-in-big-data-at-klout/
I have found an example but am not sure it is thread-safe:
def pagingEnumerator(url:String):Enumerator[JsValue]={
var maybeNextUrl = Some(url) //Next url to fetch
Enumerator.fromCallback[JsValue] ( retriever = {
val maybeResponsePromise =
maybeNextUrl map { nextUrl=>
WS.url(nextUrl).get.map { reponse =>
val json = response.json
maybeNextUrl = (json \ "next_url").asOpt[String]
val code = response.status //Potential error handling here
json
}
}
maybeResponsePromise match {
case Some(responsePromise) => responsePromise map Some.apply
case None => PlayPromise pure None
}
})
}
What is you way to add a state to Play Enumerator? Is this example thread-safe?
(in the above example old Play API related to Promise/Future is used; let's neglect this fact as far as it doesn't influence the issue itself)
You should use Enumerator.unfoldM:
Enumerator.unfoldM(Some(firstURL)) { maybeNextUrl =>
maybeNextUrl map { nextUrl =>
WS.url(nextUrl).get.map { response =>
val json = response.json
val newUrl = (json \ "next_url").asOpt[String]
val code = response.status //Potential error handling here
Some((newUrl, json))
}
} getOrElse (Future.successful(None))
}

scala, play, futures: combining results from multiple futures

I am using:
Scala 2.10
Play 2.1
Currently, I am using the Future class from scala.concurrent._, but I'm open to trying another API.
I am having trouble combining the results of multiple futures into a single List[(String, String)].
The following Controller method successfully returns the results of a single Future to an HTML template:
def test = Action { implicit request =>
queryForm.bindFromRequest.fold(
formWithErrors => Ok("Error!"),
query => {
Async {
getSearchResponse(query, 0).map { response =>
Ok(views.html.form(queryForm,
getAuthors(response.body, List[(String, String)]())))
}
}
})
}
The method getSearchResult(String, Int) performs a web service API call and returns a Future[play.api.libs.ws.Response]. The method getAuthors(String, List[(String, String)]) returns a List[(String, String)] to the HTML template.
Now, I am trying to call getSearchResult(String, Int) in a for loop to get several Response bodies. The following should give an idea of what I'm trying to do, but I get a compile-time error:
def test = Action { implicit request =>
queryForm.bindFromRequest.fold(
formWithErrors => Ok("Error!"),
query => {
Async {
val authors = for (i <- 0 to 100; if i % 10 == 0) yield {
getSearchResponse(query, i)
}.map { response =>
getAuthors(response.body, List[(String, String)]())
}
Ok(views.html.form(queryForm, authors))
}
})
}
type mismatch; found : scala.collection.immutable.IndexedSeq[scala.concurrent.Future[List[(String, String)]]] required: List[(String, String)]
How can I map the responses of several Future objects to a single Result?
Create a Future parametrized by a List or other Collection of the Result type.
From here:
In Play 1 you can do this:
F.Promise<List<WS.HttpResponse>> promises = F.Promise.waitAll(remoteCall1, remoteCall2, remoteCall3);
// where remoteCall1..3 are promises
List<WS.HttpResponse> httpResponses = await(promises); // request gets suspended here
In Play 2 less direct:
val httpResponses = for {
result1 <- remoteCall1
result2 <- remoteCall2
} yield List(result1, result2)