Scala Future, flatMap that works on Either - scala

Is there really a way to transform an object of type Future[Either[Future[T1], Future[T2]]] to and object of type Either[Future[T1], Future[T2]] ??
Maybe something like flatMap that works on Either....
I'm trying to make this code work (I have similar code that implements wrapped chain-of actions, but it doesn't involve future. It works, much simpler). The code below is based on that, with necessary modification to make it work for situation that involves futures.
case class WebServResp(msg: String)
case class WebStatus(code: Int)
type InnerActionOutType = Either[Future[Option[WebServResp]], Future[WebStatus]]
type InnerActionSig = Future[Option[WebServResp]] => Either[Future[Option[WebServResp]], Future[WebStatus]]
val chainOfActions: InnerActionSig = Seq(
{prevRespOptFut =>
println("in action 1: " + prevRespOptFut)
//dont care about prev result
Left(Future.successful(Some(WebServResp("result from 1"))))
},
{prevRespOptFut =>
println("in action 2: " + prevFutopt)
prevRespOptFut.map {prevRespOpt =>
//i know prevResp contains instance of WebServResp. so i skip the opt-matching
val prevWebServResp = prevRespOpt.get
Left(Some(prevWebServResp.msg + " & " + " additional result from 2"))
}
//But the outcome of the map above is: Future[Left(...)]
//What I want is Left(Future[...])
}
)
type WrappedActionSig = InnerActionOutType => InnerActionOutType
val wrappedChainOfActions = chainOfActions.map {innerAction =>
val wrappedAction: WrappedActionSig = {respFromPrevWrappedAction =>
respFromPrevWrappedAction match {
case Left(wsRespOptFut) => {
innerAction(wsRespOptFut)
}
case Right(wsStatusFut) => {
respFromPrevWrappedAction
}
}
}
wrappedAction
}
wrappedChainOfActions.fold(identity[WrappedActionIOType] _) ((l, r) => l andThen r).apply(Left(None))
UPDATE UPDATE UPDATE
Based on comments from Didier below ( Scala Future, flatMap that works on Either )... here's a code that works:
//API
case class WebRespString(str: String)
case class WebStatus(code: Int, str: String)
type InnerActionOutType = Either[Future[Option[WebRespString]], Future[WebStatus]]
type InnerActionSig = Future[Option[WebRespString]] => InnerActionOutType
type WrappedActionSig = InnerActionOutType => InnerActionOutType
def executeChainOfActions(chainOfActions: Seq[InnerActionSig]): Future[WebStatus] = {
val wrappedChainOfActions : Seq[WrappedActionSig] = chainOfActions.map {innerAction =>
val wrappedAction: WrappedActionSig = {respFromPrevWrappedAction =>
respFromPrevWrappedAction match {
case Left(wsRespOptFut) => {
innerAction(wsRespOptFut) }
case Right(wsStatusFut) => {
respFromPrevWrappedAction
}
}
}
wrappedAction
}
val finalResultPossibilities = wrappedChainOfActions.fold(identity[InnerActionOutType] _) ((l, r) => l andThen r).apply(Left(Future.successful(None)))
finalResultPossibilities match {
case Left(webRespStringOptFut) => webRespStringOptFut.map {webRespStringOpt => WebStatus(200, webRespStringOpt.get.str)}
case Right(webStatusFut) => webStatusFut
}
}
//API-USER
executeChainOfActions(Seq(
{prevRespOptFut =>
println("in action 1: " + prevRespOptFut)
//dont care about prev result
Left(Future.successful(Some(WebRespString("result from 1"))))
},
{prevRespOptFut =>
println("in action 2: " + prevRespOptFut)
Left(prevRespOptFut.map {prevRespOpt =>
val prevWebRespString = prevRespOpt.get
Some(WebRespString(prevWebRespString.str + " & " + " additional result from 2"))
})
}
)).map {webStatus =>
println(webStatus.code + ":" + webStatus.str)
}
executeChainOfActions(Seq(
{prevRespOptFut =>
println("in action 1: " + prevRespOptFut)
//Let's short-circuit here
Right(Future.successful(WebStatus(404, "resource non-existent")))
},
{prevRespOptFut =>
println("in action 2: " + prevRespOptFut)
Left(prevRespOptFut.map {prevRespOpt =>
val prevWebRespString = prevRespOpt.get
Some(WebRespString(prevWebRespString.str + " & " + " additional result from 2"))
})
}
)).map {webStatus =>
println(webStatus.code + ":" + webStatus.str)
}
Thanks,
Raka

The type Future[Either[Future[T1], Future[T2]]] means that sometimes later (that's future) one gets an Either, so at that time, one will know which way the calculation will go, and whether one will, still later, gets a T1 or a T2.
So the knowledge of which branch will be chosen (Left or Right) will come later. The type Either[Future[T1], Future[T2] means that one has that knowledge now (don't know what the result will be, but knows already what type it will be). The only way to get out of the Future is to wait.
No magic here, the only way for later to become now is to wait, which is done with result on the Future, and not recommended. `
What you can do instead is say that you are not too interested in knowing which branch is taken, as long has it has not completed, so Future[Either[T1, T2]] is good enough. That is easy. Say you have the Either and you would rather not look not but wait for the actual result :
def asFuture[T1, T2](
either: Either[Future[T1], Future[T2]])(
implicit ec: ExecutionContext)
: Future[Either[T1, T2] = either match {
case Left(ft1) => ft1 map {t1 => Left(t1)}
case Right(ft2) => ft2 map {t2 => Right(t2)}
}
You don't have the Either yet, but a future on that, so just flatMap
f.flatMap(asFuture) : Future[Either[T1, T2]]
(will need an ExecutionContext implicitly available)

It seems like you don't actually need the "failure" case of the Either to be a Future? In which case we can use scalaz (note that the "success" case of an either should be on the right):
import scalaz._
import scalaz.Scalaz._
def futureEitherFutureToFuture[A, B](f: Future[Either[A, Future[B]]])(
implicit ec: ExecutionContext): Future[Either[A, B]] =
f.flatMap(_.sequence)
But it's probably best to always keep the Future on the outside in your API, and to flatMap in your code rather than in the client. (Here it's part of the foldLeftM):
case class WebServResp(msg: String)
case class WebStatus(code: Int)
type OWSR = Option[WebServResp]
type InnerActionOutType = Future[Either[WebStatus, OWSR]]
type InnerActionSig = OWSR => InnerActionOutType
def executeChain(chain: List[InnerActionSig]): InnerActionOutType =
chain.foldLeftM(None: OWSR) {
(prevResp, action) => action(prevResp)
}
//if you want that same API
def executeChainOfActions(chainOfActions: Seq[InnerActionSig]) =
executeChain(chainOfActions.toList).map {
case Left(webStatus) => webStatus
case Right(webRespStringOpt) => WebStatus(200, webRespStringOpt.get.str)
}
(If you need "recovery" type actions, so you really need OWSR to be an Either, then you should still make InnerActionOutType a Future[Either[...]], and you can use .traverse or .sequence in your actions as necessary. If you have an example of an "error-recovery" type action, I can put an example of that here)

Related

Scala return variable type after future is complete Add Comment Collapse

I've got a problem with returning a list after handling futures in scala. My code looks like this:
def getElements(arrayOfIds: Future[Seq[Int]]): Future[Seq[Element]] = {
var elementArray: Seq[Element] = Seq()
arrayOfIds.map {
ids => ids.map(id => dto.getElementById(id).map {
case Some(element) => elementArray = elementArray :+ element
case None => println("Element not found")
})
}
arrayOfIds.onComplete(_ => elementArray)
}
I'd like to do something like .onComplete, however the return type is
Unit and I'd like to return a Future[Seq[Whatever]]. Is there clean way to handle futures like this? Thanks!
Please provide the type of function dto.getElementById. If it is Int => Future[Option[Element]], then:
def getElements(arrayOfIds: Future[Seq[Int]]): Future[Seq[Element]] = {
val allElements: Future[Seq[Option[Element]]] = arrayOfIds.flatMap( ids =>
Future.sequence(ids.map(dto.getElementById))
)
allElements.map(_.flatMap{
case None => println();None
case some => some
})
}
Without logging, it would be:
arrayOfIds.flatMap( ids => Future.traverse(ids.map(dto.getElementById))(_.flatten))
Instead of assigning the result to a mutable variable, return it from the continuation of the Future. You can use flatMap to extract only the Element results which actually contain a value:
def getElements(arrayOfIds: Future[Seq[Int]]): Future[Seq[Element]] = {
arrayOfIds.flatMap(id => Future.fold(id.map(getElementById))(Seq.empty[Element])(_ ++ _))
}

Unpacking tuple directly into class in scala

Scala gives the ability to unpack a tuple into multiple local variables when performing various operations, for example if I have some data
val infos = Array(("Matt", "Awesome"), ("Matt's Brother", "Just OK"))
then instead of doing something ugly like
infos.map{ person_info => person_info._1 + " is " + person_info._2 }
I can choose the much more elegant
infos.map{ case (person, status) => person + " is " + status }
One thing I've often wondered about is how to directly unpack the tuple into, say, the arguments to be used in a class constructor. I'm imagining something like this:
case class PersonInfo(person: String, status: String)
infos.map{ case (p: PersonInfo) => p.person + " is " + p.status }
or even better if PersonInfo has methods:
infos.map{ case (p: PersonInfo) => p.verboseStatus() }
But of course this doesn't work. Apologies if this has already been asked -- I haven't been able to find a direct answer -- is there a way to do this?
I believe you can get to the methods at least in Scala 2.11.x, also, if you haven't heard of it, you should checkout The Neophyte's Guide to Scala Part 1: Extractors.
The whole 16 part series is fantastic, but part 1 deals with case classes, pattern matching and extractors, which is what I think you are after.
Also, I get that java.lang.String complaint in IntelliJ as well, it defaults to that for reasons that are not entirely clear to me, I was able to work around it by explicitly setting the type in the typical "postfix style" i.e. _: String. There must be some way to work around that though.
object Demo {
case class Person(name: String, status: String) {
def verboseStatus() = s"The status of $name is $status"
}
val peeps = Array(("Matt", "Alive"), ("Soraya", "Dead"))
peeps.map {
case p # (_ :String, _ :String) => Person.tupled(p).verboseStatus()
}
}
UPDATE:
So after seeing a few of the other answers, I was curious if there was any performance differences between them. So I set up, what I think might be a reasonable test using an Array of 1,000,000 random string tuples and each implementation is run 100 times:
import scala.util.Random
object Demo extends App {
//Utility Code
def randomTuple(): (String, String) = {
val random = new Random
(random.nextString(5), random.nextString(5))
}
def timer[R](code: => R)(implicit runs: Int): Unit = {
var total = 0L
(1 to runs).foreach { i =>
val t0 = System.currentTimeMillis()
code
val t1 = System.currentTimeMillis()
total += (t1 - t0)
}
println(s"Time to perform code block ${total / runs}ms\n")
}
//Setup
case class Person(name: String, status: String) {
def verboseStatus() = s"$name is $status"
}
object PersonInfoU {
def unapply(x: (String, String)) = Some(Person(x._1, x._2))
}
val infos = Array.fill[(String, String)](1000000)(randomTuple)
//Timer
implicit val runs: Int = 100
println("Using two map operations")
timer {
infos.map(Person.tupled).map(_.verboseStatus)
}
println("Pattern matching and calling tupled")
timer {
infos.map {
case p # (_: String, _: String) => Person.tupled(p).verboseStatus()
}
}
println("Another pattern matching without tupled")
timer {
infos.map {
case (name, status) => Person(name, status).verboseStatus()
}
}
println("Using unapply in a companion object that takes a tuple parameter")
timer {
infos.map { case PersonInfoU(p) => p.name + " is " + p.status }
}
}
/*Results
Using two map operations
Time to perform code block 208ms
Pattern matching and calling tupled
Time to perform code block 130ms
Another pattern matching without tupled
Time to perform code block 130ms
WINNER
Using unapply in a companion object that takes a tuple parameter
Time to perform code block 69ms
*/
Assuming my test is sound, it seems the unapply in a companion-ish object was ~2x faster than the pattern matching, and pattern matching another ~1.5x faster than two maps. Each implementation probably has its use cases/limitations.
I'd appreciate if anyone sees anything glaringly dumb in my testing strategy to let me know about it (and sorry about that var). Thanks!
The extractor for a case class takes an instance of the case class and returns a tuple of its fields. You can write an extractor which does the opposite:
object PersonInfoU {
def unapply(x: (String, String)) = Some(PersonInfo(x._1, x._2))
}
infos.map { case PersonInfoU(p) => p.person + " is " + p.status }
You can use tuppled for case class
val infos = Array(("Matt", "Awesome"), ("Matt's Brother", "Just OK"))
infos.map(PersonInfo.tupled)
scala> infos: Array[(String, String)] = Array((Matt,Awesome), (Matt's Brother,Just OK))
scala> res1: Array[PersonInfo] = Array(PersonInfo(Matt,Awesome), PersonInfo(Matt's Brother,Just OK))
and then you can use PersonInfo how you need
You mean like this (scala 2.11.8):
scala> :paste
// Entering paste mode (ctrl-D to finish)
case class PersonInfo(p: String)
Seq(PersonInfo("foo")) map {
case p# PersonInfo(info) => s"info=$info / ${p.p}"
}
// Exiting paste mode, now interpreting.
defined class PersonInfo
res4: Seq[String] = List(info=foo / foo)
Methods won't be possible by the way.
Several answers can be combined to produce a final, unified approach:
val infos = Array(("Matt", "Awesome"), ("Matt's Brother", "Just OK"))
object Person{
case class Info(name: String, status: String){
def verboseStatus() = name + " is " + status
}
def unapply(x: (String, String)) = Some(Info(x._1, x._2))
}
infos.map{ case Person(p) => p.verboseStatus }
Of course in this small case it's overkill, but for more complex use cases this is the basic skeleton.

Making a convenient logging function for Try[T], but got stuck because of the type system

I wonder if this can be done using Scala's type system.
Basically, I want to make a logging method which accepts a result of type Try[T] and prints out a message that differs a bit depending whether result is a Success or a Failure
For example, the signature might look like
def logTry[T](what: Try[T], block: T => String): Unit
and can be used as such:
val size: Try[(Int, Int)] = Try(getSizeAndTimestampFromDatabase())
logTry(size, e => "size is " + e._2 + " kb")
which will output
size is 13 kb if size is Success(x: Int), e.g., Success(13)
or
error: size is (not available) if size if of type Failure(t: Throwable)
The tricky part is that we need to be able to access the object within the Try for printing to the screen if it's a Success, or print a chosen default string (e.g. "(not available)") if it's a Failure as the placeholder. Also, it must work with a very general type T or Any that can range from a simple scalar value to an instance of a class, etc.
The use case for this eccentric function is that it will be very handy for logging a Try object in an informative way, without cluttering the code with map / recover or a match statement.
This is the skeleton that I have came up with, but of course the hard part hasn't been figured out yet.
def logTry[T](what: Try[T], block: T => String): Unit = {
what match {
case Success(res) => println(block(res))
case Failure(t) => println(???) // how to do this
}
}
Here the solution, which not forces you to explicitly supply types:
implicit class LogTry[T](attempt: Try[T]) {
def log[E](comp: T => E, block: String => String, error: Throwable => String = _ => "(N/A)") =
println(block(attempt map (comp andThen (_.toString)) recover { case ex => error(ex) } get))
}
use it as
size.log(_._2, size => f"size is $size kb")
or
size.log(_._2, size => f"size is $size kb", err => f"(not available because $err)")
This is what I would do (well, if I didn't use implicits to add the method to Try):
def logTry[T](t: Try[T])(f: Try[T] => String) { println(f(t)) }
val size: Try[Integer] = Try(getSizeFromDatabase())
logTry(size) {
case Success(e) => s"size is $e kb"
case Failure(e) => s"error: size is (${e.getMessage}})"
}
Edit: after understanding your request
def logTry(t: Try[Any], f: Any => String) {
t match {
case Success(e) => println(f(e))
case Failure(e) => println(f(new Object {
override def toString = "(not available)"
}))
}
}
val size: Try[Integer] = Try(getSizeFromDatabase())
logTry(size, s => s"Size is $s kb")
I don't know of a way to achieve for a generic type T, but I don't think that's a good idea anyway: it's possible for the client to make special use of the methods of T when printing, but you want it to always print "not available" where it was supposed to print the information from T, and that's not so simple.
Another option is to create a special formatter (think "size is ${...} kb") so the logger would know how to replace it, but I'm not sure if this is what you want.

Composing futures - how to get another variable associated with the result of a list of futures

I'm a bit new to future composition so I haven't figured out all the common patterns yet.
I have a list of futures but I need to associate a name with the futures when they are created so I can somehow reconcile the list.
EG if I create a list of futures like this, how can I get x to be associated with the future's result?
val requestsForMaster = shardNames.map { x ⇒
sentinel ? Request("SENTINEL", "get-master-addr-by-name", x)
}
I would do something like this to get the futures into a sequence
val mastersConfig = Future.sequence(requestsForMaster)
mastersConfig.onSuccess {
case x: List[Some[List[Some[ByteString]]]] ⇒
self ! x.map {
case Some(List(Some(host: ByteString), Some(port: ByteString))) ⇒
println("Host/port: " + host.utf8String + ":" + port.utf8String)
Shard("name", host.utf8String, port.utf8String.toInt, None)
}
}
But when I go to create the Shard object, the name (x) isn't available and I need it in there.
Any idea as to how I can compose these to get the name in there?
Edit:
Here is the solution I used:
val requestsForMaster = shardNames.map { x ⇒
(sentinel ? Request("SENTINEL", "get-master-addr-by-name", x)).map(y ⇒ (x, y))
}
val mastersConfig = Future.sequence(requestsForMaster)
mastersConfig.onSuccess {
case x: (List[(String, Some[List[Some[ByteString]]])]) ⇒
self ! x.map {
case (name, Some(List(Some(host: ByteString), Some(port: ByteString)))) ⇒
println("Name Host:port: " + name + " " + host.utf8String + ":" + port.utf8String)
Shard("name", host.utf8String, port.utf8String.toInt, None)
}
}
If I understand correctly, it sounds like you want to use map on the request futures to pair each response with the shard's name:
val requestsForMaster: List[Future[(String, Some[List[Some[ByteString]])] =
shardNames.map { x =>
val result = sentinel ? Request("SENTINEL", "get-master-addr-by-name", x)
result.map(x -> _)
}
val mastersConfig = Future.sequence(requestsForMaster)
mastersConfig.onSuccess {
case results: List[(String, Some[List[Some[ByteString]]])] =>
self ! results.map {
case (x, Some(List(Some(host: ByteString), Some(port: ByteString)))) =>
// Create the shard object.
}
}
More generally, if you have a Future[V] and a K, you can create a Future[(K, V)] by writing fv.map(k -> _).

Scala: how to traverse stream/iterator collecting results into several different collections

I'm going through log file that is too big to fit into memory and collecting 2 type of expressions, what is better functional alternative to my iterative snippet below?
def streamData(file: File, errorPat: Regex, loginPat: Regex): List[(String, String)]={
val lines : Iterator[String] = io.Source.fromFile(file).getLines()
val logins: mutable.Map[String, String] = new mutable.HashMap[String, String]()
val errors: mutable.ListBuffer[(String, String)] = mutable.ListBuffer.empty
for (line <- lines){
line match {
case errorPat(date,ip)=> errors.append((ip,date))
case loginPat(date,user,ip,id) =>logins.put(ip, id)
case _ => ""
}
}
errors.toList.map(line => (logins.getOrElse(line._1,"none") + " " + line._1,line._2))
}
Here is a possible solution:
def streamData(file: File, errorPat: Regex, loginPat: Regex): List[(String,String)] = {
val lines = Source.fromFile(file).getLines
val (err, log) = lines.collect {
case errorPat(inf, ip) => (Some((ip, inf)), None)
case loginPat(_, _, ip, id) => (None, Some((ip, id)))
}.toList.unzip
val ip2id = log.flatten.toMap
err.collect{ case Some((ip,inf)) => (ip2id.getOrElse(ip,"none") + "" + ip, inf) }
}
Corrections:
1) removed unnecessary types declarations
2) tuple deconstruction instead of ulgy ._1
3) left fold instead of mutable accumulators
4) used more convenient operator-like methods :+ and +
def streamData(file: File, errorPat: Regex, loginPat: Regex): List[(String, String)] = {
val lines = io.Source.fromFile(file).getLines()
val (logins, errors) =
((Map.empty[String, String], Seq.empty[(String, String)]) /: lines) {
case ((loginsAcc, errorsAcc), next) =>
next match {
case errorPat(date, ip) => (loginsAcc, errorsAcc :+ (ip -> date))
case loginPat(date, user, ip, id) => (loginsAcc + (ip -> id) , errorsAcc)
case _ => (loginsAcc, errorsAcc)
}
}
// more concise equivalent for
// errors.toList.map { case (ip, date) => (logins.getOrElse(ip, "none") + " " + ip) -> date }
for ((ip, date) <- errors.toList)
yield (logins.getOrElse(ip, "none") + " " + ip) -> date
}
I have a few suggestions:
Instead of a pair/tuple, it's often better to use your own class. It gives meaningful names to both the type and its fields, which makes the code much more readable.
Split the code into small parts. In particular, try to decouple pieces of code that don't need to be tied together. This makes your code easier to understand, more robust, less prone to errors and easier to test. In your case it'd be good to separate producing your input (lines of a log file) and consuming it to produce a result. For example, you'd be able to make automatic tests for your function without having to store sample data in a file.
As an example and exercise, I tried to make a solution based on Scalaz iteratees. It's a bit longer (includes some auxiliary code for IteratorEnumerator) and perhaps it's a bit overkill for the task, but perhaps someone will find it helpful.
import java.io._;
import scala.util.matching.Regex
import scalaz._
import scalaz.IterV._
object MyApp extends App {
// A type for the result. Having names keeps things
// clearer and shorter.
type LogResult = List[(String,String)]
// Represents a state of our computation. Not only it
// gives a name to the data, we can also put here
// functions that modify the state. This nicely
// separates what we're computing and how.
sealed case class State(
logins: Map[String,String],
errors: Seq[(String,String)]
) {
def this() = {
this(Map.empty[String,String], Seq.empty[(String,String)])
}
def addError(date: String, ip: String): State =
State(logins, errors :+ (ip -> date));
def addLogin(ip: String, id: String): State =
State(logins + (ip -> id), errors);
// Produce the final result from accumulated data.
def result: LogResult =
for ((ip, date) <- errors.toList)
yield (logins.getOrElse(ip, "none") + " " + ip) -> date
}
// An iteratee that consumes lines of our input. Based
// on the given regular expressions, it produces an
// iteratee that parses the input and uses State to
// compute the result.
def logIteratee(errorPat: Regex, loginPat: Regex):
IterV[String,List[(String,String)]] = {
// Consumes a signle line.
def consume(line: String, state: State): State =
line match {
case errorPat(date, ip) => state.addError(date, ip);
case loginPat(date, user, ip, id) => state.addLogin(ip, id);
case _ => state
}
// The core of the iteratee. Every time we consume a
// line, we update our state. When done, compute the
// final result.
def step(state: State)(s: Input[String]): IterV[String, LogResult] =
s(el = line => Cont(step(consume(line, state))),
empty = Cont(step(state)),
eof = Done(state.result, EOF[String]))
// Return the iterate waiting for its first input.
Cont(step(new State()));
}
// Converts an iterator into an enumerator. This
// should be more likely moved to Scalaz.
// Adapted from scalaz.ExampleIteratee
implicit val IteratorEnumerator = new Enumerator[Iterator] {
#annotation.tailrec def apply[E, A](e: Iterator[E], i: IterV[E, A]): IterV[E, A] = {
val next: Option[(Iterator[E], IterV[E, A])] =
if (e.hasNext) {
val x = e.next();
i.fold(done = (_, _) => None, cont = k => Some((e, k(El(x)))))
} else
None;
next match {
case None => i
case Some((es, is)) => apply(es, is)
}
}
}
// main ---------------------------------------------------
{
// Read a file as an iterator of lines:
// val lines: Iterator[String] =
// io.Source.fromFile("test.log").getLines();
// Create our testing iterator:
val lines: Iterator[String] = Seq(
"Error: 2012/03 1.2.3.4",
"Login: 2012/03 user 1.2.3.4 Joe",
"Error: 2012/03 1.2.3.5",
"Error: 2012/04 1.2.3.4"
).iterator;
// Create an iteratee.
val iter = logIteratee("Error: (\\S+) (\\S+)".r,
"Login: (\\S+) (\\S+) (\\S+) (\\S+)".r);
// Run the the iteratee against the input
// (the enumerator is implicit)
println(iter(lines).run);
}
}