I have outlined the problem below. Method calculate represents a calculation that is to run subCalc1 and subCalc2 asynchroniously, and mainCalc that will be passed results from the last two 'sub' calculations. Significantly, before starting these calculations a preliminary calculation isCalcNecessary is to return a Boolean. If true, the calculation will continue and eventually return Future[Some[Result]]. If preliminary calculation returns false, it should return Future[None], as to illustrate that calculation was not necessary. This small algorithm should be maximally asynchronious.
def isCalcNecessary:Future[Boolean] = ...
def subCalc1(param:Param):Future[SubResult1] = ...
def subCalc2(param:Param):Future[SubResult2] = ...
def mainCalc(subResult1:SubResult1, subResult2:SubResult2):Future[Result] = .
def calcute(param:Param):Future[Option[Result]] = for {
necessary <- isCalcNecessary(param)
if necessary // this illustration fails at runtime if 'necessary' is false
subResult1 <- subCalc1(param)
subResult2 <- subCalc2(param)
result <- mainCalc(subResult1, subResult2)
} yield Some(result)
The above illustration fails at runtime with (NoSuchElementException: Future.filter predicate is not satisfied (Future.scala:312)), if the if necessary conditional is not satisfied.
How would you write this algorithm?
isCalcNecessary(param).flatMap { necessary =>
if (necessary)
for {
subResult1 <- subCalc1(param)
subResult2 <- subCalc2(param)
result <- mainCalc(subResult1, subResult2)
} yield Some(result)
else
future(None)
}
Another option would be to nest for-comprehensions
def calculate(param:Param):Future[Option[Result]] = for {
necessary <- isCalcNecessary(param)
endResult <- if (necessary) for {
subResult1 <- subCalc1(param)
subResult2 <- subCalc2(param)
result <- mainCalc(subResult1, subResult2)
} yield Some(result)
else future(None)
} yield endResult
Related
I want to make first call in seq to next ones but then next 10 calls parallel to each other using Scala's futures. I dont want to program map flatmaps or for inside for comprehensions as they'll add on to unreadibility. Is it possible with one for comprehension.
reference code:
val fcaAndFplAccountDetails = getFcaAndFplAccount(fplId)
def orgDetailsIO(orgId:Int) = getOrgDetails(orgId)
def unbilledTxns(orgid: Int) = getUnbilledTxns(orgId)
for {
accDetails <- fcaAndFplAccountDetails
orgDetails <- orgDetailsIO(accDetails.orgId)
unbilledTxns <- unbilledTxnsIO(accDetails.orgId)
... other calls that depend on org id
} yield { ... further computations }
Can the calls after fcaAndFplAccountDetails be parallelized within the comprehension?
Perhaps follow a generator with a value definition using equals symbol for the calls that should execute in parallel
for {
accDetails <- fcaAndFplAccountDetails
orgDetailsF = orgDetailsIO(accDetails.orgId)
unbilledTxnsF = unbilledTxnsIO(accDetails.orgId)
orgDetails <- orgDetailsF
unbilledTxns <- unbilledTxnsF
} yield { ... further computations }
or convey the idea with Applicative
import cats.implicits._
fcaAndFplAccountDetails.flatMap { accDetails =>
(
orgDetailsIO(accDetails.orgId),
unbilledTxnsIO(accDetails.orgId)
).mapN((orgDetails, unbilledTxns) => {
... further computations
}
}
There must be dozens of options but one would be:
val fcaAndFplAccountDetails : Future[Int] = ???
def orgDetailsIO(orgId:Int) : Future[_] = ???
def unbilledTxns(orgid: Int) : Future[_] = ???
for {
accDetails <- fcaAndFplAccountDetails
(orgDetails, unbilledTxns) <- Future.sequence(List(orgDetailsIO(accDetails), unbilledTxns(accDetails))).map(el => (el(0), el(1)))
} yield ()
You could use Future.traverse too
Say I have the following operations that must proceed in order:
Get blog post
Post analytics
Forward blog post
In code it may look like this:
val blogPostFut: Future[BlogPost] = blogService.getPost(postId)
val afterAnalytics: Future[BlogPost] = blogPostFut.flatMap(blogPost =>
val ignoredResponse: Future[Analytics] = analyticsService.sendAnalytics(blogPost)
ignoredResponse.map(_ => blogPost) // <-- THIS BOTHERS ME
)
val finalValue: Future[ForwardResult] = afterAnalytics.flatMap(blogPost =>
forwardService.forward(blogPost)
)
I am bothered that, in order to ensure proper ordering of execution, I have to pass forward blogPost within ignoredResponse in order to ensure it is available for step 3.
I'd love if I could do something like this:
blogPostFut.magicalFlatMap(analyticsService.sendAnalytics)
Where magicalFlatMap might be implemented like so:
// pseudocode
def magicalFlatMap[A,B](f: A => Future[B]): Future[A] = f().map(_ => this.value)
Does magicalFlatMap exist in either the Scala stdlib or in Cats? Is it possible to map a Future for side effects while automatically retaining the value of the original Future and strict ordering of operations?
magicalFlatMap seems to be cats.FlatMap#flatTap
https://github.com/typelevel/cats/blob/master/core/src/main/scala/cats/FlatMap.scala#L150
Try Future.andThen for side-effects
for {
blogPost <- blogService.getPost(postId).andThen { case Success(post) => analyticsService.sendAnalytics(post) }
finalValue <- forwardService.forward(blogPost)
} yield {
finalValue
}
Here is a dummy example
val result = for {
v1 <- Future(1)
v2 <- Future(v1 + 2).andThen { case Success(v) => println(v) }
v3 <- Future(v1 + v2)
} yield {
v3
}
result.foreach(println)
which should output
3
4
We could also do
for {
blogPost <- blogService.getPost(postId)
_ <- analyticsService.sendAnalytics(blogPost)
finalValue <- forwardService.forward(blogPost)
} yield {
finalValue
}
however in this case failure in analyticsService.sendAnalytics(blogPost) would short-circuit the whole for-comprehension which might not be desirable.
Consider the below code snippet:
In this, I am trying to get values from future using 'For - yield' comprehension. Now in yield method, I need to do a check which makes the call to a function fallbackResult which returns a future and hence return type of getData becomes 'Future[Future[Option[Int]]]' rather than 'Future[Option[Int]]'. How could I make this with a better way? (I did use Map & FlatMap's, but the code little ugly due to nesting Maps and FlatMaps)
def getData(): Future[Future[Option[Int]]] = {
/* These are two future vectors. Ignore the Objects */
val substanceTableF: Future[Vector[OverviewPageTableRowModel]] = getSubstanceTable(substanceIds, propertyId, dataRange)
val mixtureTableF: Future[Vector[OverviewPageTableRowModel]] = getMixtureTableForSubstanceCombination(substanceIds, propertyId, dataRange)
/* I have put for yeild to get values from futures.*/
for {
substanceTable <- substanceTableF
mixtureTable <- mixtureTableF
} yield {
if(substanceTable.isEmpty && mixtureTable.isEmpty) {
val resultF = fallbackResult()
resultF.map(result => {Some(result)})
} else {
Future.successful(Some(100))
}
}
}
private def fallbackResult(): Future[Int] = {
// This method returns future of int
}
There are a lot of ways to handle this, but the key thing is to move your yield logic into your for comprehension. One way to do this is as follows:
for {
substanceTable <- substanceTableF
mixtureTable <- mixtureTableF
result <- (substanceTable.headOption orElse mixtureTable.headOption)
.map(_ => Future.successful(Some(100)))
.getOrElse(fallbackResult)
} yield result
Id put the code inside the for-coprehension:
for {
substanceTable <- substanceTableF
mixtureTable <- mixtureTableF
result <- {
if (substanceTable.isEmpty && mixtureTable.isEmpty)
fallbackResult()
else
Future.successful(Some(100))
}
} yield result
I'm trying some code which inspects this slides about Free Monad in Scala, and made a small project with some slightly changed code.
The project is here: https://github.com/freewind/free-the-monads
Everything seems good at first, the code is clean and beautiful:
def insertAndGet() = for {
_ <- Script.insert("name", "Freewind")
value <- Script.get("name")
} yield value
def insertAndDelete() = for {
_ <- Script.insert("name", "Freewind")
_ <- Script.delete("name")
value <- Script.get("name")
} yield value
def insertAndUpdateAndDelete() = for {
_ <- Script.insert("name", "Freewind1")
oriValue <- Script.update("name", "Freewind2")
_ <- Script.delete("name")
finalValue <- Script.get("name")
} yield (oriValue, finalValue)
But when my logic is complex, e.g. there is some Script[Option[_]], and I need to check the option value to decide to do something, I can't use the for-comprehension any more, the code is like:
private def isLongName(name: String): Script[Boolean] = for {
size <- Script.getLongNameConfig
} yield size.exists(name.length > _)
def upcaseLongName(key: String): Script[Option[String]] = {
Script.get(key) flatMap {
case Some(n) => for {
isLong <- isLongName(n)
} yield isLong match {
case true => Some(n.toUpperCase)
case false => Some(n)
}
case _ => Script.pure(None)
}
}
I found the Free Monad approach is really interesting and cool, but I'm not familiar with scalaz, and just beginning to learn Monad things, not sure how to improve it.
Is there any way to make it better?
PS: You can just clone the project https://github.com/freewind/free-the-monads and try it yourself
This is a good use case for the OptionT monad transformer:
import scalaz.OptionT, scalaz.syntax.monad._
def upcaseLongName(key: String): OptionT[Script, String] = for {
n <- OptionT.optionT(Script.get(key))
isLong <- isLongName(n).liftM[OptionT]
} yield if (isLong) n.toUpperCase else n
Here OptionT.optionT converts a Script[Option[String]] into an OptionT[Script, String], and .liftM[OptionT] raises a Script[Boolean] into the same monad.
Now instead of this:
println(upcaseLongName("name1").runWith(interpreter))
You'd write this:
println(upcaseLongName("name1").run.runWith(interpreter))
You could also have upcaseLongName return an Script[Option[String]] directly by calling run there, but if there's any chance you'll need to be composing it with other option-y script-y things it's probably best to have it return OptionT[Script, String].
I am asking a very basic question which confused me recently.
I want to write a Scala For expression to do something like the following:
for (i <- expr1) {
if (i.method) {
for (j <- i) {
if (j.method) {
doSomething()
} else {
doSomethingElseA()
}
}
} else {
doSomethingElseB()
}
}
The problem is that, in the multiple generators For expression, I don't know where can I put each for expression body.
for {i <- expr1
if(i.method) // where can I write the else logic ?
j <- i
if (j.method)
} doSomething()
How can I rewrite the code in Scala Style?
The first code you wrote is perfectly valid, so there's no need to rewrite it. Elsewhere you said you wanted to know how to do it Scala-style. There isn't really a "Scala-style", but I'll assume a more functional style and tack that.
for (i <- expr1) {
if (i.method) {
for (j <- i) {
if (j.method) {
doSomething()
} else {
doSomethingElseA()
}
}
} else {
doSomethingElseB()
}
}
The first concern is that this returns no value. All it does is side effects, which are to be avoided as well. So the first change would be like this:
val result = for (i <- expr1) yield {
if (i.method) {
for (j <- i) yield {
if (j.method) {
returnSomething()
// etc
Now, there's a big difference between
for (i <- expr1; j <- i) yield ...
and
for (i <- expr1) yield for (j <- i) yield ...
They return different things, and there are times you want the later, not the former. I'll assume you want the former, though. Now, before we proceed, let's fix the code. It is ugly, difficult to follow and uninformative. Let's refactor it by extracting methods.
def resultOrB(j) = if (j.method) returnSomething else returnSomethingElseB
def nonCResults(i) = for (j <- i) yield resultOrB(j)
def resultOrC(i) = if (i.method) nonCResults(i) else returnSomethingC
val result = for (i <- expr1) yield resultOrC(i)
It is already much cleaner, but it isn't returning quite what we expect. Let's look at the difference:
trait Element
object Unrecognized extends Element
case class Letter(c: Char) extends Element
case class Punct(c: Char) extends Element
val expr1 = "This is a silly example." split "\\b"
def wordOrPunct(j: Char) = if (j.isLetter) Letter(j.toLower) else Punct(j)
def validElements(i: String) = for (j <- i) yield wordOrPunct(j)
def classifyElements(i: String) = if (i.nonEmpty) validElements(i) else Unrecognized
val result = for (i <- expr1) yield classifyElements(i)
The type of result there is Array[AnyRef], while using multiple generators would yield Array[Element]. The easy part of the fix is this:
val result = for {
i <- expr1
element <- classifyElements(i)
} yield element
But that alone won't work, because classifyElements itself returns AnyRef, and we want it returning a collection. Now, validElements return a collection, so that is not a problem. We only need to fix the else part. Since validElements is returning an IndexedSeq, let's return that on the else part as well. The final result is:
trait Element
object Unrecognized extends Element
case class Letter(c: Char) extends Element
case class Punct(c: Char) extends Element
val expr1 = "This is a silly example." split "\\b"
def wordOrPunct(j: Char) = if (j.isLetter) Letter(j.toLower) else Punct(j)
def validElements(i: String) = for (j <- i) yield wordOrPunct(j)
def classifyElements(i: String) = if (i.nonEmpty) validElements(i) else IndexedSeq(Unrecognized)
val result = for {
i <- expr1
element <- classifyElements(i)
} yield element
That does exactly the same combination of loops and conditions as you presented, but it is much more readable and easy to change.
About Yield
I think it is important to note one thing about the problem presented. Let's simplify it:
for (i <- expr1) {
for (j <- i) {
doSomething
}
}
Now, that is implemented with foreach (see here, or other similar questions and answer). That means the code above does exactly the same thing as this code:
for {
i <- expr1
j <- i
} doSomething
Exactly the same thing. That is not true at all when one is using yield. The following expressions do not yield the same result:
for (i <- expr1) yield for (j <- i) yield j
for (i <- expr1; j <- i) yield j
The first snippet will be implemented through two map calls, while the second snippet will use one flatMap and one map.
So, it is only in the context of yield that it even makes any sense to worry about nesting for loops or using multiple generators. And, in fact, generators stands for the fact that something is being generated, which is only true of true for-comprehensions (the ones yielding something).
The part
for (j <- i) {
if (j.method) {
doSomething(j)
} else {
doSomethingElse(j)
}
}
can be rewritten as
for(j <- i; e = Either.cond(j.method, j, j)) {
e.fold(doSomething _, doSomethingElse _)
}
(of course you can use a yield instead if your do.. methods return something)
Here it is not so terrible useful, but if you have a deeper nested structure, it could...
import scalaz._; import Scalaz._
val lhs = (_ : List[X]) collect { case j if j.methodJ => doSomething(j) }
val rhs = (_ : List[X]) map doSomethingElse
lhs <-: (expr1 partition methodI) :-> rhs
You can not. The for(expr; if) construct just filter the element that must be handled in the loop.
If the order isn't important for the calls to doSomething() and doSomethingElse() then you can rearrange the code like this.
val (tmp, no1) = expr1.partition(_.method)
val (yes, no2) = tmp.partition(_.method)
yes.foreach(doSomething())
no1.foreach(doSomethingElse())
no2.foreach(doSomethingElse())
To answer your original question, I think that for comprehensions can be quite nice for specific use cases, and your example doesn't fit nicely.
The conditions specified in a Scala for operation act to filter the elements from the generators. Elements not satisfying the conditions are discarded and are not presented to the yield / code block.
What this means is that if you want to perform alternate operations based on a conditional expression, the test needs to be deferred to the yield / code block.
Also be aware that the for operation is relatively expensive to compute (currently) so perhaps a simpler iterative approach might be more appropriate, perhaps something like:
expr1 foreach {i =>
if (i.method) {
i foreach {j =>
if (j.method)
doSomething()
else
doSomethingElseA()
}
}
else
doSomethingElseB()
}
Update:
If you must use a for comprehension and you can live with some restrictions, this might work:
for (i <- expr1; j <- i) {
if (i.method) {if (j.method) doSomething() else doSomethingElseA()} else doSomethingElseB()
}