How can I convert Option[Future[T]] to Future[Option[T]] in scala?
I want to use it in:
val customerAddresses = for {
a <- addressDAO.insert(ca.address) // Future[Address]
ia <- ca.invoiceAddress.map(addressDAO.insert) // Option[Future[Address]]
} yield (a, ia) // Invalid value have to be two futures
Here signature insert method
def insert(address: Address): Future[Address]
ca is a CustomerData
case class CustomerData(address: Address, invoiceAddress: Option[Address])
import scala.concurrent.Future
import scala.concurrent.ExecutionContext
def f[A](x: Option[Future[A]])(implicit ec: ExecutionContext): Future[Option[A]] =
x match {
case Some(f) => f.map(Some(_))
case None => Future.successful(None)
}
Examples:
scala> f[Int](Some(Future.successful(42)))
res3: scala.concurrent.Future[Option[Int]] = Success(Some(42))
scala> f[Int](None)
res4: scala.concurrent.Future[Option[Int]] = scala.concurrent.impl.Promise$KeptPromise#c88a337
If you have cats as a dependency in your application, the most beautiful way would be to use traverse
import cats._
import cats.implicits._
val customerAddresses = for {
a <- addressDAO.insert(ca.address) // Future[Address]
ia <- ca.invoiceAddress.traverse(addressDAO.insert) // Future[Option[Address]]
} yield (a, ia)
The standard library does provide the methods to use Future.sequence on an Option, unfortunately you have to plumb them together.
Either as a quick method:
def swap[M](x: Option[Future[M]]): Future[Option[M]] =
Future.sequence(Option.option2Iterable(x)).map(_.headOption)
Note I found the implicit Option.option2Iterable was already in scope for me. So you may not need to provide it, reducing the code down to Future.sequence(x).map(_.headOption)
Or you may prefer an extension method:
implicit class OptionSwitch[A](f: Option[Future[A]]) {
import scala.concurrent.Future
def switch: Future[Option[A]] = Future.sequence(Option.option2Iterable(f))
.map(_.headOption)
}
val myOpt = Option(Future(3))
myOpt.switch
Here is another solution:
def swap[T](o: Option[Future[T]]): Future[Option[T]] =
o.map(_.map(Some(_))).getOrElse(Future.successful(None))
The trick is to convert Option[Future[T]] into Option[Future[Option[T]]] which is easy, and then extract the value from that Option.
When you have a list (or any TraversableOnce) of futures and want a single future for computing the whole list, you use Future.sequence or Future.traverse. You can think of an Option like a list of 1 or 0 elements but since is technically not a list you have to go for a little conversion in this case. Anyway, this is a code that does it normally:
val optionFuture:Option[Future[String]] = ???
val futureOption:Future[Option[String]] = Future.sequence(optionFuture.toIterable).map(_.headOption)
In you example use better Future.traverse:
val customerAddresses = for {
a <- addressDAO.insert(ca.address) // Future[Address]
ia <- Future.traverse(ca.invoiceAddress.toIterable)(addressDAO.insert).map(_.headOption) // Future[Option[Address]]
} yield CustomerData(a, ia) // Types OK
val customerAddresses = for {
a <- addressDAO.insert(ca.address) // Future[Address]
ia <- ca.invoiceAddress.map(x => addressDAO.insert(x).map(_.map(k => Some(k))).getOrElse(Future.successful(None)))
} yield (a, ia)
You can do it simply using Dumonad
import io.github.dumonad.dumonad.Implicits._
futureOfOption.dummed
Related
That may be a dumb question, but starting with ZIO, I cannot manage to convert a Seq[ZIO] to ZIO[Seq]:
def translate(keys: Seq[String], locales: Seq[Locale]):RIO[Translator, Seq[Translation]] = {
for {
service <- ZIO.environment[Translator]
} yield {
// service.translate produce a zio.Task[Translation]
keys.map(k => service.translate(k, locales)
}
}
Required: RIO[Translator, Seq[Translation]]
Found : ZIO[Translator, Nothing, Seq[zio.Task[Translation]]
I tried flatMap, flatten, collectAll and merge but I was not able to get the expected result with anyone.
How can I transform a Seq[ZIO[_, _, B]] to a ZIO[_, _, Seq[B]] ?
Thanks
Edit: It seems that ZIO.foreach is the best option, however I still have it wrapped inside another ZIO due to the for comprehension.
Because for loops translate to flatMap except for the last line which is a map, you want to add the foreach call within the for-loop.
def translate(keys: Seq[String], locales: Seq[Locale]): RIO[Translator, Seq[Translation]] = {
for {
translator <- ZIO.environment[Translator]
translations <- ZIO.foreach(keys)(translator.translate(_, locales))
} yield translations
}
If I got you right you can do it using traverse function from cats:
import cats.instances.list._
import cats.syntax.traverse._
import zio.{RIO, Task, ZIO}
import zio.interop.catz._
import java.util.Locale
case class Translation()
trait Translator {
def translate(k: String, locales: Seq[Locale]): Task[Translation]
}
def translate(keys: Seq[String], locales: Seq[Locale]): RIO[Translator, Seq[Translation]] = {
val translator: Translator = ???
for {
service <- ZIO.effect(translator)
result <- keys.toList.traverse(k => service.translate(k, locales))
} yield result
}
For map List[ZIO[_, _, B]] to ZIO[_, _, List[B]] you can use sequence function and I would advice to use cats library for that.
import zio.ZIO
import zio.interop.catz._
import cats.syntax.traverse._
import cats.instances.list._
def ziosSequence[B](seqZIO: Seq[ZIO[Any, Throwable, B]]): ZIO[Any, Throwable, Seq[B]] =
seqZIO.toList.sequence.map(_.toSeq)
the sequence signature is:
def sequence[G[_]: Applicative, A](fga: F[G[A]]): G[F[A]] =
traverse(fga)(ga => ga)
Here we see what function do what we need.
it requires Applicative instance for G (G is ZIO in your case), and we just import it using import zio.interop.catz._
Also, to make list is able to call sequence we need import Traverse instance for List:
by import cats.instances.list._
Unfortunetaly we can not do the same tricks with Seq because we need Traverse instance for sequence, and we should convert Seq to List back and forth before and after sequence.
useful links:
typelevel documentation cats.Traverse
typelevel documentation cats.Applicative
To "exchange" List and ZIO you could dance this way:
def dance(x: List[ZIO[Any,Throwable,Int]]): ZIO[Any, Throwable, List[Int]] =
x.map ( a => a.map(x=> List(x)))
.fold ( ZIO.succeed( List[Int]()) )
((x, y) => x.map(a => y.map(b => a ++ b ) )
.flatten
)
I have a method load which is relatively expensive to call. In order to allow some kind of exception handling during loading it returns a Try. I need now an implementation for the loadAll method which basically delegates to load for each of the given keys. Here is my first approach, however I don't know if this is best practices with respect to the work with Try. Is there a better way to achieve the following?
def load(key: K): Try[V] // very expensive
def loadAll(keys: Traversable[K]): Try[Map[K, V]] = {
// remove duplicate keys
val ukeys = Set.empty ++ keys
val result: Iterable[Try[(K, V)]] = ukeys map { key =>
load(key) match {
case Success(value) => Success(key, value)
case Failure(exception) => Failure(exception)
}
}
Try(result.map { _.get }.toMap)
}
You can do this using a fold to iterate over the keys, and a for comprehension to combine the Try instances:
def toSet(keys: Traversable[K]): Try[Map[K, V]] = {
keys.toSet.foldLeft( Try( Map.empty[K, V] ) ){ case (tryMap, key) =>
for ( m <- tryMap; v <- load( key ) ) yield m.updated( key, v )
}
}
It's a one-liner in vanilla Scala (assuming you've already got Try in scope):
def loadAll(keys: Traversable[K]) = Try{ keys.toSet.map((k: K) => (k,load(k).get)).toMap }
If you're interested in a scalaz solution, this is a general operation, available via Traversable functors, called sequence. Needed instances for Try reside in scalaz-contrib. Here's how it might look like:
Welcome to Scala version 2.10.1 (OpenJDK 64-Bit Server VM, Java 1.7.0_21).
Type in expressions to have them evaluated.
Type :help for more information.
scala> import scalaz._, Scalaz._, scalaz.contrib.std.utilTry._
import scalaz._
import Scalaz._
import scalaz.contrib.std.utilTry._
scala> import scala.util.Try
import scala.util.Try
scala> val result: Iterable[Try[(Int, String)]] = Iterable.empty
result: Iterable[scala.util.Try[(Int, String)]] = List()
scala> result.toList.sequence.map(_.toMap)
res0: scala.util.Try[scala.collection.immutable.Map[Int,String]] = Success(Map())
By the way, there's a paper "The essence of the Iterator pattern", describing/deriving traverse (and sequence, as it's special case). There's a great summary of this paper by Eric Torreborre here.
I have a list of string ids representing DB records. I'd like to load them from the DB asynchronously, then upload each record to a remote server asynchronously, then when all are done uploading, make a record of the ids of the records that were uploaded.
Since I'm on Scala 2.9.2, I'm using Twitter's core-util Future implementation, but it should work exactly like the 2.10 futures in terms of Monadic transformations.
The general concept is this:
def fetch(id: String): Future[Option[Record]]
def upload(record: Record): Future[String]
def notifyUploaded(ids: Seq[String]): Unit
val ids: Seq[String] = ....
I'm trying to do this via a for comprehension but the fact that fetch returns a Future of Option makes it obscure and the code doesn't compile:
for {
id <- ids
maybeRecord <- fetch(id)
record <- maybeRecord
uploadedId <- upload(record)
} yield uploadedId
Compiling this results in the following error:
scala: type mismatch;
found : com.twitter.util.Future[String]
required: Option[?]
uploadedId <- upload(record)
^
What am I missing? why does the compiler expect uploadedId to be an Option? is there any pretty way I could work around this?
Consider the signature of the flatMap (or bind) function:
trait Monad[M[_]] {
def flatMap[A](a : M[A], f : A => M[B]) : M[B]
....
In your case, you're trying to use flatMap on an Option, giving it an f that generates a Future. But as in the signature above, f should be generating something in the same monad that it's been called on.
Scala isn't necessarily terribly helpful in this regard, since it's pretty good at converting things around (to Seqs, for example) in such a way that you get the impression that you can chain arbitrary flatMap calls together, regardless of the container.
What you possibly want is a 'Monad transformer', which gives you some ability to compose monads. Debasish Ghosh has a post on using Scalaz monad transformers here.
You cannot mix all different types in one for comprehension, I figured out that you might mix Seq and Option and result would be either Seq or Option depending on what is first. It is not possible to mix Future and Seq or Option. If you want to use for-comprehension you would have to cascade them few. In such cases it might be nicer with map/flatMap. I implemented your question in both ways and added types to few intermediate results so that you see the mess that is being created while working with all that different types.
object TestClass {
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent._
import scala.concurrent.duration._
case class Record(id: String)
def fetch(id: String): Future[Option[Record]] = Future {
Thread.sleep(1000);
Some(Record(id))
}
def upload(record: Record): Future[String] = Future {
Thread.sleep(3000);
record.id + "_uploaded"
}
def notifyUploaded(ids: Seq[String]): Unit = println("notified" + ids)
val ids: Seq[String] = Seq("a", "b", "c")
def main(args: Array[String]): Unit = {
forComprehensionImpl()
mapAndFlatMapImpl()
}
def forComprehensionImpl() = {
val result: Seq[Future[Option[Future[String]]]] = for {
id <- ids
} yield {
for {
maybeRecord <- fetch(id)
} yield {
for {
record <- maybeRecord
} yield {
for {
uploadedId <- upload(record)
} yield uploadedId
}
}
}
val result2: Future[Seq[Option[Future[String]]]] = Future.sequence(result)
val result3: Future[Unit] = result2.flatMap { x: Seq[Option[Future[String]]] =>
Future.sequence(x.flatten).map(notifyUploaded)
}
Await.result(result3, Duration.Inf)
}
def mapAndFlatMapImpl() = {
val res: Seq[Future[Iterable[String]]] = ids.map { id =>
fetch(id).flatMap { maybeRecord =>
val res1: Option[Future[Seq[String]]] = maybeRecord.map { record =>
upload(record) map (Seq(_))
}
res1 match {
case Some(a) => a
case None => Future(Seq())
}
}
}
val res3: Future[Unit] = Future.sequence(res) map (a => notifyUploaded(a.flatten))
Await.result(res3, Duration.Inf)
}
}
Consider:
def xs(c: String): Option[List[Long]] = ...
val ys: Stream[Long] = ...
Now I'd write a method something like:
def method(oc: Option[String]): Option[Long] = for {
c <- oc
list <- xs(c)
} yield{
for {
first <- ys.find(list contains _)
} yield first
}
but of course this doesn't compile, since the inferred type is Option[Option[Long]].
Is there a way in terms of scala syntax and standard library to get an Option[Long]? I know I can pattern match, but the question if it can be done using for comprehensions only just arised.
Thanks to tenshi for the answer, that does the job, however I just encountered another example of my problem:
class T
class U
class A(t: String)(implicit x: T)
def getU(a: A): Option[U] = ...
def getU_2(oc: Option[String]): Option[U] = for{
c <- oc
} yield{
implicit val someImplicit: T = new T
val a = A(c)
getU(a)
}
I can add a in the for as: a <- Some(A(c)) but what about the implicit? Should that imply a design change in my code?
Why are you using 2 nested for comprehensions? Shouldn't one do the job?
def method(oc: Option[String]): Option[Long] =
for {
c <- oc
list <- xs(c)
first <- ys.find(list contains _)
} yield first
Update
About your second example. You can define implicit elsewhere and import it or define it in the beginning of the method, but I guess you want to make it's scope as narrow as possible. In this case you can use block directly in the for comprehension:
def getU_2(oc: Option[String]): Option[U] = for {
c <- oc
a <- {
implicit val someImplicit: T = new T
getU(new A(c))
}
} yield a
or (probably the simplest) provide implicit parameter explicitly:
def getU_2(oc: Option[String]): Option[U] = for {
c <- oc
a <- getU(new A(c)(new T))
} yield a
Is there any way to create a PartialFunction except through the case statement?
I'm curious, because I'd like to express the following (scala pseudo ahead!)...
val bi = BigInt(_)
if (bi.isValidInt) bi.intValue
... as a partial function, and doing
val toInt : PartialFunction[String, Int] = {
case s if BigInt(s).isValidInt => BigInt(s).intValue
}
seems redundant since I create a BigInt twice.
Not sure I understand the question. But here's my attempt: Why not create an extractor?
object ValidBigInt {
def unapply(s: String): Option[Int] = {
val bi = BigInt(s)
if (bi.isValidInt) Some(bi.intValue) else None
}
}
val toInt: PartialFunction[String, Int] = {
case ValidBigInt(i) => i
}
The other option is (and that may answer the question as to whether one can create PartialFunction other than with a case literal):
val toInt = new PartialFunction[String, Int] {
def isDefinedAt(s: String) = BigInt(s).isValidInt
def apply(s: String) = BigInt(s).intValue
}
However since the idea of a partial function is that it's only partially defined, in the end you will still do redundant things -- you need to create a big int to test whether it's valid, and then in the function application you create the big int again...
I saw a project at Github that tried to come around this by somewhat caching the results from isDefinedAt. If you go down to the benchmarks, you'll see that it turned out to be slower than the default Scala implementation :)
So if you want to get around the double nature of isDefinedAt versus apply, you should just go straight for a (full) function that provides an Option[Int] as result.
I think you're looking for lift/unlift. lift takes a partial function and turns it into a function that returns an Option. Unlift takes a function with one argument that returns an option, and returns a partial function.
import scala.util.control.Exception._
scala> def fn(s: String) = catching(classOf[NumberFormatException]) opt {BigInt(s)}
fn: (s: String)Option[scala.math.BigInt]
scala> val fnPf = Function.unlift(fn)
fnPf: PartialFunction[String,scala.math.BigInt] = <function1>
scala> val fn = fnPf.lift
fn: String => Option[scala.math.BigInt] = <function1>
Closely related, you also want to look at this answer for information about cond and condOpt:
scala> import PartialFunction._
import PartialFunction._
scala> cond("abc") { case "def" => true }
res0: Boolean = false
scala> condOpt("abc") { case x if x.length == 3 => x + x }
res1: Option[java.lang.String] = Some(abcabc)
You can write out a PartialFunction "longhand" if you'd like:
object pf extends PartialFunction[Int,String] {
def isDefinedAt(in: Int) = in % 2 == 0
def apply(in: Int) = {
if (in % 2 == 0)
"even"
else
throw new MatchError(in + " is odd")
}
Okay, I got this
import java.lang.NumberFormatException
import scala.util.control.Exception._
val toInt: PartialFunction[String, Int] = {
catching(classOf[NumberFormatException]) opt BigInt(_) match {
case Some(bi) if bi.isValidInt => bi.intValue
}
}
How about this?
val toInt: PartialFunction[String, Int] = (s: String) => BigInt(s) match {
case bi if bi.isValidInt => bi.intValue
}