package p1
import scala.util.Failure
import scala.util.Success
import scala.concurrent.Await
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.duration._
object modCheck extends App {
def getStudentRoolNo(name: String) = Future {
println("getStudentRoolNo")
name match {
case "name1" => 1
case "name2" => 2
case _ => throw new Exception("No doesnt exist")
}
}
def getRank(roolNo: Int) = Future {
println("getRank")
Thread.sleep(500)
roolNo match {
case 1 => "1"
case 2 => "2"
case _ => throw new Exception("No roolNo exist")
}
}
def getDetails(roolNo: Int) = Future {
println("getDetails")
roolNo match {
case 1 => "details 1"
case 2 => "Details 2"
case _ => throw new Exception("No details exist")
}
}
def getStudentRecord(name: String) = {
for {
rollNo <- getStudentRoolNo(name)
rank <- getRank(rollNo)
details <- getDetails(rollNo)
} yield (rank + details)
}
getStudentRecord("name1").onComplete {
case Success(ground) => println(s"got my Details $ground")
case Failure(ex) => println("Exception!" + ex)
}
Thread.sleep(2000)
}
I want to execute functions getrank and getDetails in parallel in below code(once the getStudentRollNo is returned). How can I achieve this?
I Tried below way, It seems it still executing in sequentially
Please let me know , How to execute in parallel
Future starts computation when it's created.
for (a <- x; b <- y) yield ??? is desugared to x.flatMap(a => y.map(b => ???))
flatMap() and map() execute it's argument after a Future is completed.
getDetails() can start before completion of getRank() by separating creation of Future and flatMap() invocation.
for {
rollNo <- getStudentRoolNo(name)
rankFuture = getRank(rollNo)
detailsFuture = getDetails(rollNo)
rank <- rankFuture
details <- detailsFuture
} yield (rank + details)
As you have probable guessed, your current code does not make calls to getRank and getDetails in parallel, since its inside for-comprehension. Its a syntactic sugar for map operation. To achieve parallelism you need to create two futures outside for-comprehension.
val student = getStudentRollNo(name)
val detailsFuture = student map {s => getRank(rollNo) }
val rankFuture = student map {s => getDetails(rollNo) }
for {
rank <- rankFuture
details <- detailsFuture
} yield (rank + details)
You can use a zip in your for-comprehension to run two Futures in parallel:
def getStudentRecord(name: String) = {
for {
rollNo <- getStudentRoolNo(name)
rankRollNo <- getRank(rollNo) zip getDetails(rollNo)
} yield rankRollNo
}
In the above getRank and getDetails are run simultaneously and the result is a string tuple. If you want to yield a single string, you will need to separate rankRollNo into its separate components:
yield rankRollNo._1 + rankRollNo._2
Related
I'm a beginner in Scala.
Please let me know if there is a more concise part in the code below.
To supplement, I'd like to call each Future method synchronously.
◆getUser method:
def getUser: Option[User] = {
Await.ready(
twitterService.getUser(configService.getString(TWITTER_USERNAME_CONF)),
Duration.Inf)
.value
.flatMap(x => Option(x.getOrElse(null)))
}
◆ process method:
def process : Unit =
for {
user <- getUser
} yield {
Await.ready(
twitterService.delete(user.id, configService.getString(TWITTER_SEARCH_KEYWORD)),
Duration.Inf)
.value
.foreach {
case Success(tweets) => tweets.foreach(tweet => println(s"Delete Successfully!!. $tweet"))
case Failure(exception) => println(s"Failed Delete.... Exception:[$exception]")
}
}
I made some assumptions on user and tweet data types but I would rewrite that to:
def maybeDeleteUser(userName: String, maybeUser: Option[User]): Future[String] =
maybeUser match {
case Some(user) =>
twitterService.delete(user.id, configService.getString(TWITTER_SEARCH_KEYWORD)).map {
case Failure(exception) => s"Failed Delete.... Exception:[${exception.getMessage}]"
case Success(tweets) => tweets.map(tweet => s"Delete Successfully!!. $tweet").mkString(System.lineSeparator())
}
case _ => Future.successful(s"Failed to find user $userName")
}
def getStatusLogMessage: Future[String] = {
val userName = configService.getString(TWITTER_USERNAME_CONF)
for {
maybeUser <- twitterService.getUser(configService.getString(TWITTER_USERNAME_CONF))
statusLogMessage <- maybeDeleteUser(userName, maybeUser)
} yield statusLogMessage
}
def process: Unit = {
val message = Await.result(getStatusLogMessage, Duration.Inf)
println(message)
}
That way your side effect, i.e. println is isolated and other methods can be unit tested. If you need to block the execution, do it only at the end and use map and flatMap to chain Futures if you need to order the execution of those. Also be careful with Duration.Inf, if you really need to block, then you'd want to have some defined timeout.
I have this Action which should return a Future[Result] but I am unable to code it using for comprehension. This is the first time I am using for comprehension so I am also not sure if this is how I should use for.
Also, would someone comment on whether the usage of for is correct?
def verifyUser(token:String) = Action.async{
implicit request => { //the function takes a token
val tokenFutureOption:Future[Option[UserToken]] = userTokenRepo.find(UserTokenKey(UUID.fromString(token))) //checkc if the token exists in the db (db returns a Future)
for(tokenOption<- tokenFutureOption) yield { //resolve the future
tokenOption match {
case Some(userToken) =>{//token exists
val userOptionFuture = userRepo.findUser(userToken.loginInfo)//find user to which the token belongs. Another db request which returns a Future
for(userOption <- userOptionFuture) yield {//resolve future
userOption match {
case Some(user) =>{//user exists
val newInternalProfile = user.profile.internalProfileDetails.get.copy(confirmed=true) //modify user's profile
val newProfile = UserProfile(Some(newInternalProfile),user.profile.externalProfileDetails)
val confirmedUser = user.copy(profile=newProfile)
val userOptionFuture :Future[Option[User]] = userRepo.updateUser(confirmedUser) //update profile with new value. Another db operation with returns a Future
for(userOption <- userOptionFuture) yield {//resolve future
userTokenRepo.remove(UserTokenKey(UUID.fromString(token)))//remove the token
// Ok("user verified") //I WANT TO RETURN SUCCESS RESPONSE HERE BUT CODE DOESN'T COMPILE IF I UNCOMMENT THIS
}
}
case None =>{ //user doesn't exist
// Ok("user verified") //I WANT TO RETURN FAILURE RESPONSE HERE BUT CODE DOESN'T COMPILE IF I UNCOMMENT THIS
}
}
}
}
case None =>{//INVALID TOKEN RECEIVED
Redirect("http://localhost:9000/home"+";signup=error")//TODOM - pick from config //I CAN RETURN Redirect (WHICH IS OF SAME TYPE AS OK I.E. RESULT) BUT WHY AM I NOT ABLE TO USE OK ABOVE
}
}
}//THIS IS THE END OF FIRST FOR LOOP. HOW DO I HANDLE FAILURES IN THE FUTURE?
}
}
You are using Future, Option which are Monads and the idea behind them being helpful to sequence the computations just like Functors but also with capability to specify what happens next. .flatMap is what Monads have allow that which can be used as for yield for readability.
So in your example you can compose the api using sequence of operations.
object Api {
final case class UserTokenKey(uuid: UUID)
final case class UserToken(loginInfo: String)
object userTokenRepo {
def find(u: UserTokenKey) = {
Future.successful(Some(UserToken(loginInfo = "foundLoginInfo")))
}
def remove(u: UserTokenKey) = {
Future.successful(Some(UserToken(loginInfo = "")))
}
}
final case class User(profile: String)
object userRepo {
def findUser(u: String) = {
Future.successful(Some(User("profile")))
}
def updateUser(u: User) = {
Future.successful(Some(User("updated profile")))
}
}
def verifyUser(token: String)(implicit executionContext: ExecutionContext) = {
val user: Future[Option[UserToken]] = for {
tokenMaybe: Option[UserToken] <- userTokenRepo.find(UserTokenKey(UUID.fromString(token)))
userMaybe: Option[User] <-
tokenMaybe match {
case Some(t) => userRepo.findUser(t.loginInfo)
case _ => Future.successful(Option.empty[User])
}
updatedUserMaybe: Option[User] <-
userMaybe match {
case Some(u) => userRepo.updateUser(u)
case _ => Future.successful(Option.empty[User])
}
removedUserMaybe: Option[UserToken] <- userTokenRepo.remove(UserTokenKey(UUID.fromString(token)))
} yield removedUserMaybe
user
}
def httpLayer(request: String) = {
import scala.concurrent.ExecutionContext.Implicits.global
import play.api.mvc.Results
verifyUser(request).onComplete {
case Success(Some(user)) =>
println("user verified")
Results.Ok("user verified")
case Success(None) =>
println("user does not exist")
Results.Ok("user does not exist")
case Failure(f) =>
println("unknown error")
Results.Ok("unknown error")
}
}
}
Now you can test your api as below
def main(args: Array[String]): Unit = {
import Api._
httpLayer(UUID.randomUUID().toString) // user verified
}
Note1: you might want to use api to respond Future[Either[Error, User]] to better handle errors and use Error to determine what to respond back to the http consumers.
Note2: Since you are working with two monads Future[Option[a]], you can combine them using Monad Transformation OptionT[OuterMonad, Value Type] in scalaz or cats mtl library. Which gives a single monad OptionT.
def verifyUserV2(tokenString: String)(implicit executionContext: ExecutionContext) = {
import scalaz._
import Scalaz._
// import cats.implicits._
// import cats.data.OptionT
val userStack = for {
token <- OptionT(userTokenRepo.find(UserTokenKey(UUID.fromString(tokenString))))
user <- OptionT(userRepo.findUser(token.loginInfo))
updatedUser <- OptionT(userRepo.updateUser(user))
removedUser <- OptionT(userTokenRepo.remove(UserTokenKey(UUID.fromString(tokenString))))
} yield removedUser
userStack.run
//cats unpack stack
// userStack.value
}
Useful reads:
Futures - map vs flatmap
Using Either to process failures in Scala code
In my playframework application I want to wait until my future is completed and the return it to the view.
my code looks like:
def getContentComponentUsageSearch: Action[AnyContent] = Action.async { implicit request =>
println(request.body.asJson)
request.body.asJson.map(_.validate[StepIds] match {
case JsSuccess(stepIds, _) =>
println("VALIDE SUCCESS -------------------------------")
val fList: List[Seq[Future[ProcessTemplatesModel]]] = List() :+ stepIds.s.map(s => {
processTemplateDTO.getProcessStepTemplate(s.processStep_id).flatMap(stepTemplate => {
processTemplateDTO.getProcessTemplate(stepTemplate.get.processTemplate_id.get).map(a => {
a.get
})
})
})
fList.map(u => {
val a: Seq[Future[ProcessTemplatesModel]] = u
Future.sequence(a).map(s => {
println(s)
})
})
Future.successful(Ok(Json.obj("id" -> "")))
case JsError(_) =>
println("NOT VALID -------------------------------")
Future.successful(BadRequest("Process Template not create client"))
case _ => Future.successful(BadRequest("Process Template create client"))
}).getOrElse(Future.successful(BadRequest("Process Template create client")))
}
the pirntln(s) is printing the finished stuff. But how can I wait until it is complete and return it then to the view?
thanks in advance
UPDATE:
also tried this:
val process = for {
fList: List[Seq[Future[ProcessTemplatesModel]]] <- List() :+ stepIds.s.map(s => {
processTemplateDTO.getProcessStepTemplate(s.processStep_id).flatMap(stepTemplate => {
processTemplateDTO.getProcessTemplate(stepTemplate.get.processTemplate_id.get).map(a => {
a.get
})
})
})
} yield (fList)
process.map({ case (fList) =>
Ok(Json.obj(
"processTemplate" -> fList
))
})
but then I got this:
UPDATE:
My problem is that the futures in fList do not complete before an OK result is returned
The code in the question didn't seem compilable, so here is an untested very rough sketch, that hopefully provides enough inspiration for further search of the correct solution:
def getContentComponentUsageSearch: = Action.async { implicit req =>
req.body.asJson.map(_.validate[StepIds] match {
case JsSuccess(stepIds, _) => {
// Create list of futures
val listFuts: List[Future[ProcessTemplatesModel]] = (stepIds.s.map(s => {
processTemplateDTO.
getProcessStepTemplate(s.processStep_id).
flatMap{ stepTemplate =>
processTemplateDTO.
getProcessTemplate(stepTemplate.get.processTemplate_id.get).
map(_.get)
}
})).toList
// Sequence all the futures into a single future of list
val futList = Future.sequence(listFuts)
// Flat map this single future to the OK result
for {
listPTMs <- futList
} yield {
// Apparently some debug output?
listPTMs foreach printl
Ok(Json.obj("id" -> ""))
}
}
case JsError(_) => {
println("NOT VALID -------------------------------")
Future.successful(BadRequest("Process Template not create client"))
}
case _ => Future.successful(BadRequest("Process Template create client"))
}).getOrElse(Future.successful(BadRequest("Process Template create client")))
}
If I understood your question correctly, what you wanted was to make sure that all futures in the list complete before you return the OK. Therefore I have first created a List[Future[...]]:
val listFuts: List[Future[ProcessTemplatesModel]] = // ...
Then I've combined all the futures into a single future of list, which completes only when every element has completed:
// Sequence all the futures into a single future of list
val futList = Future.sequence(listFuts)
Then I've used a for-comprehension to make sure that the listPTMs finishes computation before the OK is returned:
// Flat map this single future to the OK result
for {
listPTMs <- futList
} yield {
// Apparently some debug output?
listPTMs foreach printl
Ok(Json.obj("id" -> ""))
}
The for-yield (equivalent to map here) is what establishes the finish-this-before-doing-that behavior, so that listPTMs is fully evaluated before OK is constructed.
In order to wait until a Future is complete, it is most common to do one of two things:
Use a for-comprehension, which does a bunch of mapping and flatmapping behind the scenes before doing anything in the yield section (see Andrey's comment for a more detailed explanation). A simplified example:
def index: Action[AnyContent] = Action.async {
val future1 = Future(1)
val future2 = Future(2)
for {
f1 <- future1
f2 <- future2
} yield {
println(s"$f1 + $f2 = ${f1 + f2}") // prints 3
Ok(views.html.index("Home"))
}
}
Map inside a Future:
def index: Action[AnyContent] = Action.async {
val future1 = Future(1)
future1.map{
f1 =>
println(s"$f1")
Ok(views.html.index("Home"))
}
}
If there are multiple Futures:
def index: Action[AnyContent] = Action.async {
val future1 = Future(1)
val future2 = Future(2)
future1.flatMap{
f1 =>
future2.map {
f2 =>
println(s"$f1 + $f2 = ${f1 + f2}")
Ok(views.html.index("Home"))
}
}
}
}
When you have multiple Futures though, the argument for for-yield comprehensions gets much stronger as it gets easier to read. Also, you are probably aware but if you work with futures you may need to following imports:
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global
I've got some code in my play framework app that parses a JSON request and uses it to update a user's data. The problem is that I need to return a Future[Result], but my userDAO.update function returns a Future[Int] so I have nested futures.
I've resorted to using Await which isn't very good. How can I rewrite this code to avoid the nested future?
def patchCurrentUser() = Action.async { request =>
Future {
request.body.asJson
}.map {
case Some(rawJson) => Json.fromJson[User](rawJson).map { newUser =>
val currentUserId = 1
logger.info(s"Retrieving users own profile for user ID $currentUserId")
val futureResult: Future[Result] = userDAO.findById(currentUserId).flatMap {
case Some(currentUser) =>
val mergedUser = currentUser.copy(
firstName = newUser.firstName // ... and the other fields
)
userDAO.update(mergedUser).map(_ => Ok("OK"))
case _ => Future { Status(404) }
}
import scala.concurrent.duration._
// this is bad. How can I get rid of this?
Await.result(futureResult, 1 seconds)
}.getOrElse(Status(400))
case _ => Status(400)
}
}
Update:
Sod's law: Just after posting this I worked it out:
Future {
request.body.asJson
}.flatMap {
case Some(rawJson) => Json.fromJson[User](rawJson).map { newUser =>
val currentUserId = 1
userDAO.findById(currentUserId).flatMap {
case Some(currentUser) =>
val updatedUser = currentUser.copy(
firstName = newUser.firstName
)
userDAO.update(updatedUser).map(_ => Ok("OK"))
case _ => Future { Status(404) }
}
}.getOrElse(Future(Status(400)))
case _ => Future(Status(400))
}
But, is there a more elegant way? It seems like I'm peppering Future() around quite liberally which seems like a code smell.
Use flatMap instead of map.
flatMap[A, B](f: A => Future[B])
map[A, B](f: A => B)
More elegant way is to use for comprehension
Using For comprehension Code looks like this
for {
jsonOpt <- Future (request.body.asJson)
result <- jsonOpt match {
case Some(json) =>
json.validate[User] match {
case JsSuccess(newUser, _ ) =>
for {
currentUser <- userDAO.findById(1)
_ <- userDAO.update(currentUser.copy(firstName = newUser.firstName))
} yield Ok("ok")
case JsError(_) => Future(Status(400))
}
case None => Future(Status(400))
}
} yield result
As #pamu said it might clear your code a bit if you would use a for comprehension.
Another interesting approach (and more pure in terms of Functional Programming) would be to use monad transformers (normally a type similar to Future[Option[T]] screams monad transformer).
You should take a look at libraries like cats (and or scalaz). I'll try to give a small "pseudo code" example using cats (because I don't have play framework locally):
import cats.data.OptionT
import cats.instances.future._
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.Future
def convertJsonToUser(json: Json): Future[Option[User]] = Json.fromJson[User](json)
def convertBodyToJson(request: Request): Future[Option[Json]] = Future {request.body.asJson}
def updateUser(user: User): Future[HttpResult] = Future {
// update user
Ok("ok")
}
def myFunction: Future[HttpResult] = {
val resultOpt: OptionT[Future, HttpResult] = for {
json <- OptionT(convertBodyToJson(request))
user <- OptionT(convertJsonToUser(json))
result <- OptionT.lift(updateUser(user))
} yield result
result.getOrElseF(Future {Status(400)})
}
As you can see, in this case the monad transformers allow to treat a type like Future[Option[T]] as a single "short-circuiting" type (e.g. the for comprehension will stop if you have either a failed future, or a future containing a None).
Consider the following code inside a Play Framework controller:
val firstFuture = function1(id)
val secondFuture = function2(id)
val resultFuture = for {
first <- firstFuture
second <- secondFuture(_.get)
result <- function3(first, second)
} yield Ok(s"Processed $id")
resultFuture.map(result => result).recover { case t => InternalServerError(s"Error organizing files: $t.getMessage")}
Here are some details about the functions:
function1 returns Future[List]
function2 returns Future[Option[Person]]
function1 and function2 can run in parallel, but function3 needs the results for both.
Given this information, I have some questions:
Although the application is such that this code is very unlikely to be called with an improper id, I would like to handle this possibility. Basically, I would like to return NotFound if function2 returns None, but I can't figure out how to do that.
Will the recover call handle an Exception thrown any step of the way?
Is there a more elegant or idiomatic way to write this code?
Perhaps using collect, and then you can recover the NoSuchElementException--which yes, will recover a failure from any step of the way. resultFuture will either be successful with the mapped Result, or failed with the first exception that was thrown.
val firstFuture = function1(id)
val secondFuture = function2(id)
val resultFuture = for {
first <- firstFuture
second <- secondFuture.collect(case Some(x) => x)
result <- function3(first, second)
} yield Ok(s"Processed $id")
resultFuture.map(result => result)
.recover { case java.util.NoSuchElementException => NotFound }
.recover { case t => InternalServerError(s"Error organizing files: $t.getMessage")}
I would go with Scalaz OptionT. Maybe when you have only one function Future[Optipn[T]] it's overkill, but when you'll start adding more functions it will become super useful
import scala.concurrent.ExecutionContext.Implicits.global
import scalaz.OptionT
import scalaz.OptionT._
import scalaz.std.scalaFuture._
// Wrap 'some' result into OptionT
private def someOptionT[T](t: Future[T]): OptionT[Future, T] =
optionT[Future](t.map(Some.apply))
val firstFuture = function1(id)
val secondFuture = function2(id)
val action = for {
list <- someOptionT(firstFuture)
person <- optionT(secondFuture)
result = function3(list, person)
} yield result
action.run.map {
case None => NotFound
case Some(result) => Ok(s"Processed $id")
} recover {
case NonFatal(err) => InternalServerError(s"Error organizing files: ${err.getMessage}")
}