I want to make first call in seq to next ones but then next 10 calls parallel to each other using Scala's futures. I dont want to program map flatmaps or for inside for comprehensions as they'll add on to unreadibility. Is it possible with one for comprehension.
reference code:
val fcaAndFplAccountDetails = getFcaAndFplAccount(fplId)
def orgDetailsIO(orgId:Int) = getOrgDetails(orgId)
def unbilledTxns(orgid: Int) = getUnbilledTxns(orgId)
for {
accDetails <- fcaAndFplAccountDetails
orgDetails <- orgDetailsIO(accDetails.orgId)
unbilledTxns <- unbilledTxnsIO(accDetails.orgId)
... other calls that depend on org id
} yield { ... further computations }
Can the calls after fcaAndFplAccountDetails be parallelized within the comprehension?
Perhaps follow a generator with a value definition using equals symbol for the calls that should execute in parallel
for {
accDetails <- fcaAndFplAccountDetails
orgDetailsF = orgDetailsIO(accDetails.orgId)
unbilledTxnsF = unbilledTxnsIO(accDetails.orgId)
orgDetails <- orgDetailsF
unbilledTxns <- unbilledTxnsF
} yield { ... further computations }
or convey the idea with Applicative
import cats.implicits._
fcaAndFplAccountDetails.flatMap { accDetails =>
(
orgDetailsIO(accDetails.orgId),
unbilledTxnsIO(accDetails.orgId)
).mapN((orgDetails, unbilledTxns) => {
... further computations
}
}
There must be dozens of options but one would be:
val fcaAndFplAccountDetails : Future[Int] = ???
def orgDetailsIO(orgId:Int) : Future[_] = ???
def unbilledTxns(orgid: Int) : Future[_] = ???
for {
accDetails <- fcaAndFplAccountDetails
(orgDetails, unbilledTxns) <- Future.sequence(List(orgDetailsIO(accDetails), unbilledTxns(accDetails))).map(el => (el(0), el(1)))
} yield ()
You could use Future.traverse too
Related
I have the following algebra in Scala (I am using the Tagless Final Pattern):
trait ShoppingCarts[F[_]] {
def create(id: String): F[Unit]
def find(id: String): F[Option[ShoppingCart]]
def add(sc: ShoppingCart, product: Product): F[ShoppingCart]
}
Using the above algebra, I created the following program:
def createAndToCart[F[_] : Monad : ShoppingCarts](product: Product, cartId: String): F[Option[ShoppingCart]] =
for {
_ <- ShoppingCarts[F].create(cartId)
maybeSc <- ShoppingCarts[F].find(cartId)
maybeNewScF = maybeSc.map(sc => ShoppingCarts[F].add(sc, product))
maybeNewSc <- maybeNewScF match {
case Some(d) => d.map(s1 => Option.apply(s1))
case _ => Monad[F].pure(Option.empty[ShoppingCart])
}
} yield maybeNewSc
I don't like the code very much in the for-comprehension construct that converts an Option[F[ShoppingCart]] into an F[Option[ShoppingCart]]. I know for sure that I can do better, but I don't know how to improve it.
I am using Cats.
You are looking for traverse and sequence. What these functions do is "switch" the order of effects, so they are able to change G[F[A]] to F[G[A]] if G has an instance of Applicative. Option has such an instance in scope, so you'd be able to use it.
traverse takes additional mapping function, but if you just want to "switch" effect, so sequence will be way to go:
for {
_ <- ShoppingCarts[F].create(cartId)
maybeSc <- ShoppingCarts[F].find(cartId)
maybeNewSc <- maybeSc.map(sc => ShoppingCarts[F].add(sc, product)).sequence //here you need to use sequence
} yield maybeNewSc
or you can merge map and sequence into one step with traverse:
for {
_ <- ShoppingCarts[F].create(cartId)
maybeSc <- ShoppingCarts[F].find(cartId)
maybeNewSc <- maybeSc.traverse(sc => ShoppingCarts[F].add(sc, product))
} yield maybeNewSc
You can read more on sequence and traverse in cats docs.
Another thing I would suggest you is checking out are monad transformers since they're making dealing with nested monads stacks like F[Option[A]] a lot easier.
I am trying to do some handson with scala basic operations and got stuck here in the following sample code
def insuranceRateQuote(a: Int, tickets:Int) : Either[Exception, Double] = {
// ... something
Right(Double)
}
def parseInsuranceQuoteFromWebForm(age: String, numOfTickets: String) : Either[Exception, Double]= {
try{
val a = Try(age.toInt)
val tickets = Try(numOfTickets.toInt)
for{
aa <- a
t <- tickets
} yield insuranceRateQuote(aa,t) // ERROR HERE
} catch {
case _ => Left(new Exception)}
}
The Error I am getting is that it says found Try[Either[Exception,Double]]
I am not getting why it is wrapper under Try of Either
PS - This must not be the perfect way to do in scala so feel free to post your sample code :)
The key to understand is that for-comprehensions might transform what is inside the wrapper but will not change the wrapper itself. The reason is because for-comprehension de-sugar to map/flatMap calls on the wrapper determined in the first step of the chain. For example consider the following snippet
val result: Try[Int] = Try(41).map(v => v + 1)
// result: scala.util.Try[Int] = Success(42)
Note how we transformed the value inside the Try wrapper from 41 to 42 however the wrapper remained unchanged. Alternatively we could express the same thing using a for-comprehension
val result: Try[Int] = for { v <- Try(41) } yield v + 1
// result: scala.util.Try[Int] = Success(42)
Note how the effect is exactly the same. Now consider the following for comprehension which chains multiple steps
val result: Try[Int] =
for {
a <- Try(41) // first step determines the wrapper for all the other steps
b <- Try(1)
} yield a + b
// result: scala.util.Try[Int] = Success(42)
This expands to
val result: Try[Int] =
Try(41).flatMap { (a: Int) =>
Try(1).map { (b: Int) => a + b }
}
// result: scala.util.Try[Int] = Success(42)
where again we see the result is the same, namely, a value transformed inside the wrapper but wrapper remained untransformed.
Finally consider
val result: Try[Either[Exception, Int]] =
for {
a <- Try(41) // first step still determines the top-level wrapper
b <- Try(1)
} yield Right(a + b) // here we wrap inside `Either`
// result: scala.util.Try[Either[Exception,Int]] = Success(Right(42))
The principle remains the same - we did wrap a + b inside Either however this does not affect the top-level outer wrapper which is still Try.
Mario Galic's answer already explains the problem with your code, but I'd fix it differently.
Two points:
Either[Exception, A] (or rather, Either[Throwable, A]) is kind of equivalent to Try[A], with Left taking the role of Failure and Right the role of Success.
The outer try/catch is not useful because the exceptions should be captured by working in Try.
So you probably want something like
def insuranceRateQuote(a: Int, tickets:Int) : Try[Double] = {
// ... something
Success(someDouble)
}
def parseInsuranceQuoteFromWebForm(age: String, numOfTickets: String): Try[Double] = {
val a = Try(age.toInt)
val tickets = Try(numOfTickets.toInt)
for{
aa <- a
t <- tickets
q <- insuranceRateQuote(aa,t)
} yield q
}
A bit unfortunately, this does a useless map(q => q) if you figure out what the comprehension does, so you can write it more directly as
a.flatMap(aa => tickets.flatMap(t => insuranceRateQuote(aa,t)))
I wrote this simple program in my attempt to learn how Cats Writer works
import cats.data.Writer
import cats.syntax.applicative._
import cats.syntax.writer._
import cats.instances.vector._
object WriterTest extends App {
type Logged2[A] = Writer[Vector[String], A]
Vector("started the program").tell
val output1 = calculate1(10)
val foo = new Foo()
val output2 = foo.calculate2(20)
val (log, sum) = (output1 + output2).pure[Logged2].run
println(log)
println(sum)
def calculate1(x : Int) : Int = {
Vector("came inside calculate1").tell
val output = 10 + x
Vector(s"Calculated value ${output}").tell
output
}
}
class Foo {
def calculate2(x: Int) : Int = {
Vector("came inside calculate 2").tell
val output = 10 + x
Vector(s"calculated ${output}").tell
output
}
}
The program works and the output is
> run-main WriterTest
[info] Compiling 1 Scala source to /Users/Cats/target/scala-2.11/classes...
[info] Running WriterTest
Vector()
50
[success] Total time: 1 s, completed Jan 21, 2017 8:14:19 AM
But why is the vector empty? Shouldn't it contain all the strings on which I used the "tell" method?
When you call tell on your Vectors, each time you create a Writer[Vector[String], Unit]. However, you never actually do anything with your Writers, you just discard them. Further, you call pure to create your final Writer, which simply creates a Writer with an empty Vector. You have to combine the writers together in a chain that carries your value and message around.
type Logged[A] = Writer[Vector[String], A]
val (log, sum) = (for {
_ <- Vector("started the program").tell
output1 <- calculate1(10)
foo = new Foo()
output2 <- foo.calculate2(20)
} yield output1 + output2).run
def calculate1(x: Int): Logged[Int] = for {
_ <- Vector("came inside calculate1").tell
output = 10 + x
_ <- Vector(s"Calculated value ${output}").tell
} yield output
class Foo {
def calculate2(x: Int): Logged[Int] = for {
_ <- Vector("came inside calculate2").tell
output = 10 + x
_ <- Vector(s"calculated ${output}").tell
} yield output
}
Note the use of for notation. The definition of calculate1 is really
def calculate1(x: Int): Logged[Int] = Vector("came inside calculate1").tell.flatMap { _ =>
val output = 10 + x
Vector(s"calculated ${output}").tell.map { _ => output }
}
flatMap is the monadic bind operation, which means it understands how to take two monadic values (in this case Writer) and join them together to get a new one. In this case, it makes a Writer containing the concatenation of the logs and the value of the one on the right.
Note how there are no side effects. There is no global state by which Writer can remember all your calls to tell. You instead make many Writers and join them together with flatMap to get one big one at the end.
The problem with your example code is that you're not using the result of the tell method.
If you take a look at its signature, you'll see this:
final class WriterIdSyntax[A](val a: A) extends AnyVal {
def tell: Writer[A, Unit] = Writer(a, ())
}
it is clear that tell returns a Writer[A, Unit] result which is immediately discarded because you didn't assign it to a value.
The proper way to use a Writer (and any monad in Scala) is through its flatMap method. It would look similar to this:
println(
Vector("started the program").tell.flatMap { _ =>
15.pure[Logged2].flatMap { i =>
Writer(Vector("ended program"), i)
}
}
)
The code above, when executed will give you this:
WriterT((Vector(started the program, ended program),15))
As you can see, both messages and the int are stored in the result.
Now this is a bit ugly, and Scala actually provides a better way to do this: for-comprehensions. For-comprehension are a bit of syntactic sugar that allows us to write the same code in this way:
println(
for {
_ <- Vector("started the program").tell
i <- 15.pure[Logged2]
_ <- Vector("ended program").tell
} yield i
)
Now going back to your example, what I would recommend is for you to change the return type of compute1 and compute2 to be Writer[Vector[String], Int] and then try to make your application compile using what I wrote above.
I think I have understanding of how future composition works but I am confused how to invoke the next future on chunk of response from first future.
Say the first future returns a list of integer and list is huge. I want to apply some function to that list with 2 elements at a time. How do I do that?
This example summarizes my dilemma:
val a = Future(List(1,2,3,4,5,6))
def f(a: List[Int]) = Future(a map (_ + 2))
val res = for {
list <- a
chunked <- list.grouped(2).toList
} yield f(chunked)
<console>:14: error: type mismatch;
found : List[scala.concurrent.Future[List[Int]]]
required: scala.concurrent.Future[?]
chunked <- list.grouped(2).toList
^
The return type has to be Future[?] so I can fix it by moving second future to yield part:
val res = for {
list <- a
} yield {
val temp = for {
chunked <- list.grouped(2).toList
} yield f(chunked)
Future.sequence(temp)
}
I feel it loses its elegance now, since it becomes nested (see two for comprehensions instead of one in the first approach). Is there a better way to achieve the same?
Consider
a.map { _.grouped(2).toList }.flatMap { Future.traverse(_)(f) }
Or, if you are set on only using for comprehension for some reason, here is how, without "cheating" :)
for {
b <- a
c <- Future.traverse(b.grouped(2).toList)(f)
} yield c
Edit in response to the comment It's not really that hard to add more processing to your chunked list if needed:
for {
b <- a
chunks = b.grouped(2).toList
processedChunks = processChunks(chunks)
c <- Future.traverse(processedChunks)
} yield c
Or, without for comprehension:
a
.map { _.grouped(2).toList }
.map(processChunks)
.flatMap { Future.traverse(_)(f) }
You cannot mix Future with List in a for-comprehension. All involved objects have to be of the same type. Also, in your working example, your result value res is of type Future[Future[List[List[Int]]]], which is probably not what you want.
import scala.concurrent._
import scala.concurrent.ExecutionContext.Implicits.global
a: scala.concurrent.Future[List[Int]] = scala.concurrent.impl.Promise$DefaultPromise#3bd3cdc8
f: (a: List[Int])scala.concurrent.Future[List[Int]]
scala> val b: Future[List[List[Int]]] = a.map(list => list.grouped(2).toList)
b: scala.concurrent.Future[List[List[Int]]] = scala.concurrent.impl.Promise$DefaultPromise#74db196c
scala> val res: Future[List[List[Int]]] = b.flatMap(lists => Future.sequence(lists.map(f)))
res: scala.concurrent.Future[List[List[Int]]] = scala.concurrent.impl.Promise$DefaultPromise#28f9873c
With for-comprehension
for {
b ← a.map(list ⇒ list.grouped( 2 ).toList)
res ← Future.sequence(b.map(f))
} yield res
I'm trying some code which inspects this slides about Free Monad in Scala, and made a small project with some slightly changed code.
The project is here: https://github.com/freewind/free-the-monads
Everything seems good at first, the code is clean and beautiful:
def insertAndGet() = for {
_ <- Script.insert("name", "Freewind")
value <- Script.get("name")
} yield value
def insertAndDelete() = for {
_ <- Script.insert("name", "Freewind")
_ <- Script.delete("name")
value <- Script.get("name")
} yield value
def insertAndUpdateAndDelete() = for {
_ <- Script.insert("name", "Freewind1")
oriValue <- Script.update("name", "Freewind2")
_ <- Script.delete("name")
finalValue <- Script.get("name")
} yield (oriValue, finalValue)
But when my logic is complex, e.g. there is some Script[Option[_]], and I need to check the option value to decide to do something, I can't use the for-comprehension any more, the code is like:
private def isLongName(name: String): Script[Boolean] = for {
size <- Script.getLongNameConfig
} yield size.exists(name.length > _)
def upcaseLongName(key: String): Script[Option[String]] = {
Script.get(key) flatMap {
case Some(n) => for {
isLong <- isLongName(n)
} yield isLong match {
case true => Some(n.toUpperCase)
case false => Some(n)
}
case _ => Script.pure(None)
}
}
I found the Free Monad approach is really interesting and cool, but I'm not familiar with scalaz, and just beginning to learn Monad things, not sure how to improve it.
Is there any way to make it better?
PS: You can just clone the project https://github.com/freewind/free-the-monads and try it yourself
This is a good use case for the OptionT monad transformer:
import scalaz.OptionT, scalaz.syntax.monad._
def upcaseLongName(key: String): OptionT[Script, String] = for {
n <- OptionT.optionT(Script.get(key))
isLong <- isLongName(n).liftM[OptionT]
} yield if (isLong) n.toUpperCase else n
Here OptionT.optionT converts a Script[Option[String]] into an OptionT[Script, String], and .liftM[OptionT] raises a Script[Boolean] into the same monad.
Now instead of this:
println(upcaseLongName("name1").runWith(interpreter))
You'd write this:
println(upcaseLongName("name1").run.runWith(interpreter))
You could also have upcaseLongName return an Script[Option[String]] directly by calling run there, but if there's any chance you'll need to be composing it with other option-y script-y things it's probably best to have it return OptionT[Script, String].