I have a method load which is relatively expensive to call. In order to allow some kind of exception handling during loading it returns a Try. I need now an implementation for the loadAll method which basically delegates to load for each of the given keys. Here is my first approach, however I don't know if this is best practices with respect to the work with Try. Is there a better way to achieve the following?
def load(key: K): Try[V] // very expensive
def loadAll(keys: Traversable[K]): Try[Map[K, V]] = {
// remove duplicate keys
val ukeys = Set.empty ++ keys
val result: Iterable[Try[(K, V)]] = ukeys map { key =>
load(key) match {
case Success(value) => Success(key, value)
case Failure(exception) => Failure(exception)
}
}
Try(result.map { _.get }.toMap)
}
You can do this using a fold to iterate over the keys, and a for comprehension to combine the Try instances:
def toSet(keys: Traversable[K]): Try[Map[K, V]] = {
keys.toSet.foldLeft( Try( Map.empty[K, V] ) ){ case (tryMap, key) =>
for ( m <- tryMap; v <- load( key ) ) yield m.updated( key, v )
}
}
It's a one-liner in vanilla Scala (assuming you've already got Try in scope):
def loadAll(keys: Traversable[K]) = Try{ keys.toSet.map((k: K) => (k,load(k).get)).toMap }
If you're interested in a scalaz solution, this is a general operation, available via Traversable functors, called sequence. Needed instances for Try reside in scalaz-contrib. Here's how it might look like:
Welcome to Scala version 2.10.1 (OpenJDK 64-Bit Server VM, Java 1.7.0_21).
Type in expressions to have them evaluated.
Type :help for more information.
scala> import scalaz._, Scalaz._, scalaz.contrib.std.utilTry._
import scalaz._
import Scalaz._
import scalaz.contrib.std.utilTry._
scala> import scala.util.Try
import scala.util.Try
scala> val result: Iterable[Try[(Int, String)]] = Iterable.empty
result: Iterable[scala.util.Try[(Int, String)]] = List()
scala> result.toList.sequence.map(_.toMap)
res0: scala.util.Try[scala.collection.immutable.Map[Int,String]] = Success(Map())
By the way, there's a paper "The essence of the Iterator pattern", describing/deriving traverse (and sequence, as it's special case). There's a great summary of this paper by Eric Torreborre here.
Related
That may be a dumb question, but starting with ZIO, I cannot manage to convert a Seq[ZIO] to ZIO[Seq]:
def translate(keys: Seq[String], locales: Seq[Locale]):RIO[Translator, Seq[Translation]] = {
for {
service <- ZIO.environment[Translator]
} yield {
// service.translate produce a zio.Task[Translation]
keys.map(k => service.translate(k, locales)
}
}
Required: RIO[Translator, Seq[Translation]]
Found : ZIO[Translator, Nothing, Seq[zio.Task[Translation]]
I tried flatMap, flatten, collectAll and merge but I was not able to get the expected result with anyone.
How can I transform a Seq[ZIO[_, _, B]] to a ZIO[_, _, Seq[B]] ?
Thanks
Edit: It seems that ZIO.foreach is the best option, however I still have it wrapped inside another ZIO due to the for comprehension.
Because for loops translate to flatMap except for the last line which is a map, you want to add the foreach call within the for-loop.
def translate(keys: Seq[String], locales: Seq[Locale]): RIO[Translator, Seq[Translation]] = {
for {
translator <- ZIO.environment[Translator]
translations <- ZIO.foreach(keys)(translator.translate(_, locales))
} yield translations
}
If I got you right you can do it using traverse function from cats:
import cats.instances.list._
import cats.syntax.traverse._
import zio.{RIO, Task, ZIO}
import zio.interop.catz._
import java.util.Locale
case class Translation()
trait Translator {
def translate(k: String, locales: Seq[Locale]): Task[Translation]
}
def translate(keys: Seq[String], locales: Seq[Locale]): RIO[Translator, Seq[Translation]] = {
val translator: Translator = ???
for {
service <- ZIO.effect(translator)
result <- keys.toList.traverse(k => service.translate(k, locales))
} yield result
}
For map List[ZIO[_, _, B]] to ZIO[_, _, List[B]] you can use sequence function and I would advice to use cats library for that.
import zio.ZIO
import zio.interop.catz._
import cats.syntax.traverse._
import cats.instances.list._
def ziosSequence[B](seqZIO: Seq[ZIO[Any, Throwable, B]]): ZIO[Any, Throwable, Seq[B]] =
seqZIO.toList.sequence.map(_.toSeq)
the sequence signature is:
def sequence[G[_]: Applicative, A](fga: F[G[A]]): G[F[A]] =
traverse(fga)(ga => ga)
Here we see what function do what we need.
it requires Applicative instance for G (G is ZIO in your case), and we just import it using import zio.interop.catz._
Also, to make list is able to call sequence we need import Traverse instance for List:
by import cats.instances.list._
Unfortunetaly we can not do the same tricks with Seq because we need Traverse instance for sequence, and we should convert Seq to List back and forth before and after sequence.
useful links:
typelevel documentation cats.Traverse
typelevel documentation cats.Applicative
To "exchange" List and ZIO you could dance this way:
def dance(x: List[ZIO[Any,Throwable,Int]]): ZIO[Any, Throwable, List[Int]] =
x.map ( a => a.map(x=> List(x)))
.fold ( ZIO.succeed( List[Int]()) )
((x, y) => x.map(a => y.map(b => a ++ b ) )
.flatten
)
How can I convert Option[Future[T]] to Future[Option[T]] in scala?
I want to use it in:
val customerAddresses = for {
a <- addressDAO.insert(ca.address) // Future[Address]
ia <- ca.invoiceAddress.map(addressDAO.insert) // Option[Future[Address]]
} yield (a, ia) // Invalid value have to be two futures
Here signature insert method
def insert(address: Address): Future[Address]
ca is a CustomerData
case class CustomerData(address: Address, invoiceAddress: Option[Address])
import scala.concurrent.Future
import scala.concurrent.ExecutionContext
def f[A](x: Option[Future[A]])(implicit ec: ExecutionContext): Future[Option[A]] =
x match {
case Some(f) => f.map(Some(_))
case None => Future.successful(None)
}
Examples:
scala> f[Int](Some(Future.successful(42)))
res3: scala.concurrent.Future[Option[Int]] = Success(Some(42))
scala> f[Int](None)
res4: scala.concurrent.Future[Option[Int]] = scala.concurrent.impl.Promise$KeptPromise#c88a337
If you have cats as a dependency in your application, the most beautiful way would be to use traverse
import cats._
import cats.implicits._
val customerAddresses = for {
a <- addressDAO.insert(ca.address) // Future[Address]
ia <- ca.invoiceAddress.traverse(addressDAO.insert) // Future[Option[Address]]
} yield (a, ia)
The standard library does provide the methods to use Future.sequence on an Option, unfortunately you have to plumb them together.
Either as a quick method:
def swap[M](x: Option[Future[M]]): Future[Option[M]] =
Future.sequence(Option.option2Iterable(x)).map(_.headOption)
Note I found the implicit Option.option2Iterable was already in scope for me. So you may not need to provide it, reducing the code down to Future.sequence(x).map(_.headOption)
Or you may prefer an extension method:
implicit class OptionSwitch[A](f: Option[Future[A]]) {
import scala.concurrent.Future
def switch: Future[Option[A]] = Future.sequence(Option.option2Iterable(f))
.map(_.headOption)
}
val myOpt = Option(Future(3))
myOpt.switch
Here is another solution:
def swap[T](o: Option[Future[T]]): Future[Option[T]] =
o.map(_.map(Some(_))).getOrElse(Future.successful(None))
The trick is to convert Option[Future[T]] into Option[Future[Option[T]]] which is easy, and then extract the value from that Option.
When you have a list (or any TraversableOnce) of futures and want a single future for computing the whole list, you use Future.sequence or Future.traverse. You can think of an Option like a list of 1 or 0 elements but since is technically not a list you have to go for a little conversion in this case. Anyway, this is a code that does it normally:
val optionFuture:Option[Future[String]] = ???
val futureOption:Future[Option[String]] = Future.sequence(optionFuture.toIterable).map(_.headOption)
In you example use better Future.traverse:
val customerAddresses = for {
a <- addressDAO.insert(ca.address) // Future[Address]
ia <- Future.traverse(ca.invoiceAddress.toIterable)(addressDAO.insert).map(_.headOption) // Future[Option[Address]]
} yield CustomerData(a, ia) // Types OK
val customerAddresses = for {
a <- addressDAO.insert(ca.address) // Future[Address]
ia <- ca.invoiceAddress.map(x => addressDAO.insert(x).map(_.map(k => Some(k))).getOrElse(Future.successful(None)))
} yield (a, ia)
You can do it simply using Dumonad
import io.github.dumonad.dumonad.Implicits._
futureOfOption.dummed
Given a Option[Future[Option[Int]]]:
scala> val x: Option[Future[Option[Int]]] = Some ( Future ( Some ( 10 ) ) )
x: Option[scala.concurrent.Future[Option[Int]]] =
Some(scala.concurrent.impl.Promise$DefaultPromise#446a1e84)
I want Future[Option[Int]].
I can pattern match (or use Option#getOrElse):
scala> x match {
| case Some(f) => f
| case None => Future { None }
| }
res6: scala.concurrent.Future[Option[Int]] =
scala.concurrent.impl.Promise$DefaultPromise#446a1e84
scala> res6.value
res7: Option[scala.util.Try[Option[Int]]] = Some(Success(Some(10)))
But, is there a higher-order function that will do the job?
I thought of using sequence, but I don't have an outer type of List:
> :t sequence
sequence :: Monad m => [m a] -> m [a]
Haskell's sequence isn't as generic as it could be, or as generic as Scalaz's (and I'm assuming you're okay with a Scalaz solution since you mention sequence).
Scalaz's sequence (and Haskell's sequenceA in Data.Traversable) only requires that the outer type constructor have a Traverse instance—it doesn't necessarily have to be a list. Option has a Traverse instance, so sequence will work just fine here:
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.Future
import scalaz._, Scalaz._
def collapse(x: Option[Future[Option[Int]]]): Future[Option[Int]] =
x.sequence.map(_.flatten)
Scalaz also provides an orZero extension method for Option, which would allow you just to write x.orZero, since the zero of Future[Option[Int]] is Future(None).
I'd actually probably use x.getOrElse(Future.successful(None)), though, in this case—it's slightly (probably irrelevantly) more performant, but more importantly it's as clear and almost as concise as the Scalaz options.
I am using scalaz validation, and have some code to validate products.
def validateProduct(product: Option[Product]): ValidationNel[String, Product] = ???
Given a list of products, I want to get a single validation containing the whole list as a successful value or a list of validation errors. It seems like some kind of fold should do it, but I am not sure what the combination function should be.
def validateProducts(products: Seq[Option[Product]]): ValidationNel[String, Seq[Product]] = {
val listOfValidations: Seq[ValidationNel[String, Product]] = products.map(validateProduct _)
val validatedList:ValidationNel[Seq[String], Seq[Product]] = ??? // what to do here?
???
}
Any help is appreciated
If instead of a ValidationNel[List[String], List[Product]] you want a ValidationNel[String, List[Product]] (i.e., all the failures in the same list), you can just use traverse:
val result: ValidationNel[String, List[Product]] =
products.toList.traverseU(validateProduct)
Note that I've converted the Seq to a List as there are no type class instances for raw Seq, and I'm using traverseU rather than traverse as Scala's type inference doesn't quite work for non-trivial type constructors like ValidationNel
You can use fold with applicative
import scalaz.syntax.validation._
import scalaz.syntax.applicative._
case class Product(name: String)
val allGood = Seq(
Product("a").successNel[String],
Product("b").successNel[String]
)
val aggregated: ValidationNel[String, Seq[Product]] =
allGood.foldLeft(Seq.empty[Product].successNel[String]) {
case (acc , v) => (acc |#| v)(_ :+ _)
}
println(aggregated)
I've been working with the Scala Akka library and have come across a bit of a problem. As the title says, I need to convert Map[A, Future[B]] to Future[Map[A,B]]. I know that one can use Future.sequence for Iterables like Lists, but that doesn't work in this case.
I was wondering: is there a clean way in Scala to make this conversion?
See if this works for you:
val map = Map("a" -> future{1}, "b" -> future{2}, "c" -> future{3})
val fut = Future.sequence(map.map(entry => entry._2.map(i => (entry._1, i)))).map(_.toMap)
The idea is to map the map to an Iterable for a Tuple of the key of the map and the result of the future tied to that key. From there you can sequence that Iterable and then once you have the aggregate Future, map it and convert that Iterable of Tuples to a map via toMap.
Now, an alternative to this approach is to try and do something similar to what the sequence function is doing, with a couple of tweaks. You could write a sequenceMap function like so:
def sequenceMap[A, B](in: Map[B, Future[A]])(implicit executor: ExecutionContext): Future[Map[B, A]] = {
val mb = new MapBuilder[B,A, Map[B,A]](Map())
in.foldLeft(Promise.successful(mb).future) {
(fr, fa) => for (r <- fr; a <- fa._2.asInstanceOf[Future[A]]) yield (r += ((fa._1, a)))
} map (_.result)
}
And then use it in an example like this:
val map = Map("a" -> future{1}, "b" -> future{2}, "c" -> future{3})
val fut = sequenceMap(map)
fut onComplete{
case Success(m) => println(m)
case Failure(ex) => ex.printStackTrace()
}
This might be slightly more efficient than the first example as it creates less intermediate collections and has less hits to the ExecutionContext.
I think the most succinct we can be with core Scala 2.12.x is
val futureMap = Map("a" -> future{1}, "b" -> future{2}, "c" -> future{3})
Future.traverse(futureMap.toList) { case (k, fv) => fv.map(k -> _) } map(_.toMap)
Update: You can actually get the nice .sequence syntax in Scalaz 7 without too much fuss:
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.{ Future, future }
import scalaz._, Scalaz.{ ToTraverseOps => _, _ }
import scalaz.contrib.std._
val m = Map("a" -> future(1), "b" -> future(2), "c" -> future(3))
And then:
scala> m.sequence.onSuccess { case result => println(result) }
Map(a -> 1, b -> 2, c -> 3)
In principle it shouldn't be necessary to hide ToTraverseOps like this, but for now it does the trick. See the rest of my answer below for more details about the Traverse type class, dependencies, etc.
As copumpkin notes in a comment above, Scalaz contains a Traverse type class with an instance for Map[A, _] that is one of the puzzle pieces here. The other piece is the Applicative instance for Future, which isn't in Scalaz 7 (which is still cross-built against pre-Future 2.9), but is in scalaz-contrib.
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.Future
import scalaz._, Scalaz._
import scalaz.contrib.std._
def sequence[A, B](m: Map[A, Future[B]]): Future[Map[A, B]] = {
type M[X] = Map[A, X]
(m: M[Future[B]]).sequence
}
Or:
def sequence[A, B](m: Map[A, Future[B]]): Future[Map[A, B]] =
Traverse[({ type L[X] = Map[A, X] })#L] sequence m
Or:
def sequence[A, B](m: Map[A, Future[B]]): Future[Map[A, B]] =
TraverseOpsUnapply(m).sequence
In a perfect world you'd be able to write m.sequence, but the TraverseOps machinery that should make this syntax possible isn't currently able to tell how to go from a particular Map instance to the appropriate Traverse instance.
This also works, where the idea is to use the sequence result (of the map's values) to fire a promise that says you can start retrieving values from your map. mapValues gives you a non-strict view of your map, so the value.get.get is only applied when you retrieve the value. That's right, you get to keep your map! Free ad for the puzzlers in that link.
import concurrent._
import concurrent.duration._
import scala.util._
import ExecutionContext.Implicits.global
object Test extends App {
def calc(i: Int) = { Thread sleep i * 1000L ; i }
val m = Map("a" -> future{calc(1)}, "b" -> future{calc(2)}, "c" -> future{calc(3)})
val m2 = m mapValues (_.value.get.get)
val k = Future sequence m.values
val p = Promise[Map[String,Int]]
k onFailure { case t: Throwable => p failure t }
k onSuccess { case _ => p success m2 }
val res = Await.result(p.future, Duration.Inf)
Console println res
}
Here's the REPL where you see it force the m2 map by printing all its values:
scala> val m2 = m mapValues (_.value.get.get)
m2: scala.collection.immutable.Map[String,Int] = Map(a -> 1, b -> 2, c -> 3)
This shows the same thing with futures that are still in the future:
scala> val m2 = m mapValues (_.value.get.get)
java.util.NoSuchElementException: None.get
Just make a new future which waits for all futures in the map values , then builds a map to return.
I would try to avoid using overengineered Scalaz based super-functional solutions (unless your project is already heavily Scalaz based and has tons of "computationally sophisticated" code; no offense on the "overengineered" remark):
// the map you have
val foo: Map[A, Future[B]] = ???
// get a Seq[Future[...]] so that we can run Future.sequence on it
val bar: Seq[Future[(A, B)]] = foo.map { case (k, v) => v.map(k -> _) }
// here you go; convert back `toMap` once it completes
Future.sequence(bar).onComplete { data =>
// do something with data.toMap
}
However, it should be safe to assume that your map values are somehow generated from the map keys, which initially reside in a Seq such as List, and that the part of code that builds the initial Map is under your control as opposed to being sent from elsewhere. So I would personally take an even simpler/cleaner approach instead by not starting out with Map[A, Future[B]] in the first place.
def fetchAgeFromDb(name: String): Future[Int] = ???
// no foo needed anymore
// no Map at all before the future completes
val bar = personNames.map { name => fetchAgeFromDb(name).map(name -> _) }
// just as above
Future.sequence(bar).onComplete { data =>
// do something with data.toMap
}
Is this solution acceptable :
without an execution context this should works ...
def removeMapFuture[A, B](in: Future[Map[A, Future[B]]]) = {
in.flatMap { k =>
Future.sequence(k.map(l =>
l._2.map(l._1 -> _)
)).map {
p => p.toMap
}
}
}