Scala App doesn't run, just finishes with exit code 0 - scala

Full application below - why does this not run with the menu ready for user inputs? Simply returns "Process finished with exit code 0" - I am unsure what I am doing wrong here, the menu should be displayed ready for user input to then return the data from the specified file.
import scala.util.{Try, Success, Failure, Using}
case object MyApp5 {
case class State(name : String
,code : Int
,parties : Array[(String,Int)])
val mapdata = readFile("filename.txt")
def readFile(filename: String): Try[List[State]] = {
val dataRE = "([^(]+) \\((\\d+)\\),(.+)".r
val pVotes = "([^:]+):(\\d+)".r
Using(io.Source.fromFile(filename)) {
_.getLines()
.toList
.collect{ case dataRE(name, code, votes) =>
State(name.trim
,code.toInt
,votes.split(",")
.collect{case pVotes(p,v) => (p,v.toInt)})
}
}
}
class Menu(states: List[State]) {
def apply(key: String): Boolean = {
val (_, op, continue) = lookup(key)
op()
continue
}
private val lookup: Map[String,(String,()=>Unit,Boolean)] =
Map("?" -> ("show this menu", menu _, true)
,"menu" -> ("show this menu", menu _, true)
,"all" -> ("display all voting data", all _, true)
,"st" -> ("vote totals by state", stVotes _, true)
,"x" -> ("exit", done _, false)
,"quit" -> ("exit", done _, false)
).withDefaultValue(("",unknown _, true))
private def done(): Unit = println("bye")
private def unknown(): Unit = println("unknown selection ('?' for main menu)")
private def menu(): Unit =
lookup.keys.toVector.sorted
.map(k => s"$k\t: ${lookup(k)._1}")
.foreach(println)
private def all(): Unit =
states.sortBy(_.name) //alphabetical
.foreach{ st =>
println(st.name) //state name
st.parties
.sortBy(-_._2) //votes in decreasing order
.map{case (p,v) => f"\t$p%-12s:$v%9d"}
.foreach(println)
}
private def stVotes(): Unit =
states.map(st => (st.name, st.parties.map(_._2).sum))
.sortBy(-_._2) //votes in decreasing order
.map{case (state,total) => f"$state%-9s:$total%8d"}
.foreach(println)
}
def main(args: Array[String]): Unit =
args.headOption.map(readFile) match {
case None =>
println(s"usage: ${this.productPrefix} <data_file>")
case Some(Failure(exc)) =>
println(s"Error reading data file: $exc")
case Some(Success(stateData)) =>
val menu = new Menu(stateData)
menu("menu")
Iterator.continually(menu(io.StdIn.readLine(">> ").toLowerCase))
.dropWhile(identity)
.next()
}
}
This is a copy and paste from my scala case object MyApp5 (with the exception of "filename.txt" where I have put the absolute path - what am I missing?

Hard-coding the data file into the program would be a very unfortunate design decision. That way you wouldn't be able to run the program on different data sets, e.g. the results from different elections.
But ... oh well.
def main(args: Array[String]): Unit =
readFile("filename.txt") match {
case Failure(exc) =>
println(s"Error reading data file: $exc")
case Success(stateData) =>
val menu = new Menu(stateData)
menu("menu")
Iterator.continually(menu(io.StdIn.readLine(">> ").toLowerCase))
.dropWhile(identity)
.next()
}
I really don't see any point or advantage in doing it this way.

Related

Why does the stream never get triggered?

I have the following stream, that never reach the map after flatMapConcat.
private def stream[A](ref: ActorRef[ServerHealthStreamer])(implicit system: ActorSystem[A])
: KillSwitch = {
implicit val materializer = ActorMaterializer()
implicit val dispatcher = materializer.executionContext
system.log.info("=============> Start KafkaDetectorStream <=============")
val addr = system
.settings
.config
.getConfig("kafka")
.getString("servers")
val sink: Sink[ServerHealthEvent, NotUsed] =
ActorSink.actorRefWithAck[ServerHealthEvent, ServerHealthStreamer, Ack](
ref = ref,
onCompleteMessage = Complete,
onFailureMessage = Fail.apply,
messageAdapter = Message.apply,
onInitMessage = Init.apply,
ackMessage = Ack)
Source.tick(1.seconds, 5.seconds, NotUsed)
.flatMapConcat(_ => Source.fromFuture(health(addr)))
.map {
case true =>
KafkaActiveConfirmed
case false =>
KafkaInactiveConfirmed
}
.viaMat(KillSwitches.single)(Keep.right)
.to(sink)
.run()
}
private def health(server: String)(implicit executor: ExecutionContext): Future[Boolean] = {
val props = new Properties
props.put(AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG, server)
props.put(AdminClientConfig.CONNECTIONS_MAX_IDLE_MS_CONFIG, "10000")
props.put(AdminClientConfig.REQUEST_TIMEOUT_MS_CONFIG, "5000")
Future {
AdminClient
.create(props)
.listTopics()
.names()
.get()
}
.map(_ => true)
.recover {
case _: Throwable => false
}
}
What I mean is, that this part:
.map {
case true =>
KafkaActiveConfirmed
case false =>
KafkaInactiveConfirmed
}
never gets executed and I do not know the reason. The method health executes as expected.
Try to add .log between flatMapConcat and map to see emited element. log can else log errors and stream cancelation.
https://doc.akka.io/docs/akka/current/stream/operators/Source-or-Flow/log.html
Note, .log using implicit logger
And your .flatMapConcat(_ => Source.fromFuture(health(addr))) seams triky,
try .mapAsyncUnordered(1)(_ => health(addr))

Scala resolve multiple Futures and get a Map(String, AnyRef)

I am currently trying to resolve multiple futures at once but as some of them may fail, I don't want to get a failure on all if one of them fails, instead, end up with a Map(String, AnyRef) (meaning a Map with the future name and the response converted to what a need).
Currently I have the following:
val fsResp = channelList.map {
channelRef => channelRef.ask(ReportStatus).mapTo[EventMessage]
}
Future.sequence(fsResp).onComplete{
case Success(resp: Seq[EventMessage]) =>
resp.foreach { event => Supervisor.foreach(_ ! event) }
val channels = loadConfiguredComponents()
.collect {
case ("processor" | "external", components) => components.map {
case (name, config: Channel) =>
(name, serializeDetails(config, resp.find(_.channel == ChannelName(name))))
}
}.flatten.toMap
val event = EventMessage(...)
Supervisor.foreach(_ ! event)
case Failure(exception) => originalSender ! replayError(exception.getMessage)
}
But this fails if any of those fails. So How can I end up with a Map(channelRef.path.name, event() | exception) ?
Thanks!
You can use fallbackTo in order to avoid a Failure. In this example I change Future[T] to Future[Option[T]] in order to fallback to None, and then remove None elements.
import scala.concurrent.ExecutionContext.Implicits.global
def method(value:Int) = { Thread.sleep(2000); println(value); value }
println("start")
val eventualNone = Future.successful(None)
val futures = List(Future(method(1)), Future(method(2)), Future(method(3)), Future(throw new RuntimeException))
val withoutFailures = futures.map(_.map(Option.apply).fallbackTo(eventualNone))
Future.sequence(withoutFailures).map(_.flatten).onComplete {
case Success(values) => println(values)
case Failure(ex:Throwable) => println("FAIL")
}
Thread.sleep(5000)
output
start
1
3
2
List(1, 2, 3)
Can be changed to Either[Throwable, T] instead of Option[T] if you want to know what fails.
This code always be Success (regarding the Future result), so you need to inspect your values in order to know if all futures fail.
To capture successful/failed values from the list of Futures, you can first apply map/recover to each of them, then use Future.sequence to transform the result list into a Future of List[Either[Throwable,EventMessage]], as shown in the following trivialized example:
import scala.concurrent.{Future, Await}
import scala.concurrent.duration._
import scala.concurrent.ExecutionContext.Implicits.global
case class EventMessage(id: Int, msg: String)
val fsResp = List(
Future{EventMessage(1, "M1")}, Future{throw new Throwable()}, Future{EventMessage(3, "M3")}
)
val f = Future.sequence(
fsResp.map( _.map{ resp =>
// Do stuff with `resp`, add item to `Map()`, etc ...
Right(resp)
}.
recover{ case e: Throwable =>
// Log `exception` info, etc ...
Left(e)
} )
)
Await.result(f, Duration.Inf)
// f: scala.concurrent.Future[List[Product with Serializable with scala.util.
// Either[Throwable,EventMessage]]] = Future(Success(List(
// Right(EventMessage(1,M1)), Left(java.lang.Throwable), Right(EventMessage(3,M3))
// )))

Why does my Akka data stream stops processing a huge file (~250,000 lines of strings) but works for small file?

My stream works for smaller file of 1000 lines but stops when I test it on a large file ~12MB and ~250,000 lines? I tried applying backpressure with a buffer and throttling it and still same thing...
Here is my data streamer:
class UserDataStreaming(usersFile: File) {
implicit val system = ActorSystemContainer.getInstance().getSystem
implicit val materializer = ActorSystemContainer.getInstance().getMaterializer
def startStreaming() = {
val graph = RunnableGraph.fromGraph(GraphDSL.create() {
implicit builder =>
val usersSource = builder.add(Source.fromIterator(() => usersDataLines)).out
val stringToUserFlowShape: FlowShape[String, User] = builder.add(csvToUser)
val averageAgeFlowShape: FlowShape[User, (String, Int, Int)] = builder.add(averageUserAgeFlow)
val averageAgeSink = builder.add(Sink.foreach(averageUserAgeSink)).in
usersSource ~> stringToUserFlowShape ~> averageAgeFlowShape ~> averageAgeSink
ClosedShape
})
graph.run()
}
val usersDataLines = scala.io.Source.fromFile(usersFile, "ISO-8859-1").getLines().drop(1)
val csvToUser = Flow[String].map(_.split(";").map(_.trim)).map(csvLinesArrayToUser)
def csvLinesArrayToUser(line: Array[String]) = User(line(0), line(1), line(2))
def averageUserAgeSink[usersSource](source: usersSource) {
source match {
case (age: String, count: Int, totalAge: Int) => println(s"age = $age; Average reader age is: ${Try(totalAge/count).getOrElse(0)} count = $count and total age = $totalAge")
case bad => println(s"Bad case: $bad")
}
}
def averageUserAgeFlow = Flow[User].fold(("", 0, 0)) {
(nums: (String, Int, Int), user: User) =>
var counter: Option[Int] = None
var totalAge: Option[Int] = None
val ageInt = Try(user.age.substring(1, user.age.length-1).toInt)
if (ageInt.isSuccess) {
counter = Some(nums._2 + 1)
totalAge = Some(nums._3 + ageInt.get)
}
else {
counter = Some(nums._2 + 0)
totalAge = Some(nums._3 + 0)
}
//println(counter.get)
(user.age, counter.get, totalAge.get)
}
}
Here is my Main:
object Main {
def main(args: Array[String]): Unit = {
implicit val system = ActorSystemContainer.getInstance().getSystem
implicit val materializer = ActorSystemContainer.getInstance().getMaterializer
val usersFile = new File("data/BX-Users.csv")
println(usersFile.length())
val userDataStreamer = new UserDataStreaming(usersFile)
userDataStreamer.startStreaming()
}
It´s possible that there may be any error related to one row of your csv file. In that case, the stream materializes and stops. Try to define your flows like that:
FlowFlowShape[String, User].map {
case (user) => try {
csvToUser(user)
}
}.withAttributes(ActorAttributes.supervisionStrategy {
case ex: Throwable =>
log.error("Error parsing row event: {}", ex)
Supervision.Resume
}
In this case the possible exception is captured and the stream ignores the error and continues.
If you use Supervision.Stop, the stream stops.

How to download a HTTP resource to a file with Akka Streams and HTTP?

Over the past few days I have been trying to figure out the best way to download a HTTP resource to a file using Akka Streams and HTTP.
Initially I started with the Future-Based Variant and that looked something like this:
def downloadViaFutures(uri: Uri, file: File): Future[Long] = {
val request = Get(uri)
val responseFuture = Http().singleRequest(request)
responseFuture.flatMap { response =>
val source = response.entity.dataBytes
source.runWith(FileIO.toFile(file))
}
}
That was kind of okay but once I learnt more about pure Akka Streams I wanted to try and use the Flow-Based Variant to create a stream starting from a Source[HttpRequest]. At first this completely stumped me until I stumbled upon the flatMapConcat flow transformation. This ended up a little more verbose:
def responseOrFail[T](in: (Try[HttpResponse], T)): (HttpResponse, T) = in match {
case (responseTry, context) => (responseTry.get, context)
}
def responseToByteSource[T](in: (HttpResponse, T)): Source[ByteString, Any] = in match {
case (response, _) => response.entity.dataBytes
}
def downloadViaFlow(uri: Uri, file: File): Future[Long] = {
val request = Get(uri)
val source = Source.single((request, ()))
val requestResponseFlow = Http().superPool[Unit]()
source.
via(requestResponseFlow).
map(responseOrFail).
flatMapConcat(responseToByteSource).
runWith(FileIO.toFile(file))
}
Then I wanted to get a little tricky and use the Content-Disposition header.
Going back to the Future-Based Variant:
def destinationFile(downloadDir: File, response: HttpResponse): File = {
val fileName = response.header[ContentDisposition].get.value
val file = new File(downloadDir, fileName)
file.createNewFile()
file
}
def downloadViaFutures2(uri: Uri, downloadDir: File): Future[Long] = {
val request = Get(uri)
val responseFuture = Http().singleRequest(request)
responseFuture.flatMap { response =>
val file = destinationFile(downloadDir, response)
val source = response.entity.dataBytes
source.runWith(FileIO.toFile(file))
}
}
But now I have no idea how to do this with the Future-Based Variant. This is as far as I got:
def responseToByteSourceWithDest[T](in: (HttpResponse, T), downloadDir: File): Source[(ByteString, File), Any] = in match {
case (response, _) =>
val source = responseToByteSource(in)
val file = destinationFile(downloadDir, response)
source.map((_, file))
}
def downloadViaFlow2(uri: Uri, downloadDir: File): Future[Long] = {
val request = Get(uri)
val source = Source.single((request, ()))
val requestResponseFlow = Http().superPool[Unit]()
val sourceWithDest: Source[(ByteString, File), Unit] = source.
via(requestResponseFlow).
map(responseOrFail).
flatMapConcat(responseToByteSourceWithDest(_, downloadDir))
sourceWithDest.runWith(???)
}
So now I have a Source that will emit one or more (ByteString, File) elements for each File (I say each File since there is no reason the original Source has to be a single HttpRequest).
Is there anyway to take these and route them to a dynamic Sink?
I'm thinking something like flatMapConcat, such as:
def runWithMap[T, Mat2](f: T => Graph[SinkShape[Out], Mat2])(implicit materializer: Materializer): Mat2 = ???
So that I could complete downloadViaFlow2 with:
def destToSink(destination: File): Sink[(ByteString, File), Future[Long]] = {
val sink = FileIO.toFile(destination, true)
Flow[(ByteString, File)].map(_._1).toMat(sink)(Keep.right)
}
sourceWithDest.runWithMap {
case (_, file) => destToSink(file)
}
The solution does not require a flatMapConcat. If you don't need any return values from the file writing then you can use Sink.foreach:
def writeFile(downloadDir : File)(httpResponse : HttpResponse) : Future[Long] = {
val file = destinationFile(downloadDir, httpResponse)
httpResponse.entity.dataBytes.runWith(FileIO.toFile(file))
}
def downloadViaFlow2(uri: Uri, downloadDir: File) : Future[Unit] = {
val request = HttpRequest(uri=uri)
val source = Source.single((request, ()))
val requestResponseFlow = Http().superPool[Unit]()
source.via(requestResponseFlow)
.map(responseOrFail)
.map(_._1)
.runWith(Sink.foreach(writeFile(downloadDir)))
}
Note that the Sink.foreach creates Futures from the writeFile function. Therefore there's not much back-pressure involved. The writeFile could be slowed down by the hard drive but the stream would keep generating Futures. To control this you can use Flow.mapAsyncUnordered (or Flow.mapAsync) :
val parallelism = 10
source.via(requestResponseFlow)
.map(responseOrFail)
.map(_._1)
.mapAsyncUnordered(parallelism)(writeFile(downloadDir))
.runWith(Sink.ignore)
If you want to accumulate the Long values for a total count you need to combine with a Sink.fold:
source.via(requestResponseFlow)
.map(responseOrFail)
.map(_._1)
.mapAsyncUnordered(parallelism)(writeFile(downloadDir))
.runWith(Sink.fold(0L)(_ + _))
The fold will keep a running sum and emit the final value when the source of requests has dried up.
Using the play Web Services client injected in ws, remmebering to import scala.concurrent.duration._:
def downloadFromUrl(url: String)(ws: WSClient): Future[Try[File]] = {
val file = File.createTempFile("my-prefix", new File("/tmp"))
file.deleteOnExit()
val futureResponse: Future[WSResponse] =
ws.url(url).withMethod("GET").withRequestTimeout(5 minutes).stream()
futureResponse.flatMap { res =>
res.status match {
case 200 =>
val outputStream = java.nio.file.Files.newOutputStream(file.toPath)
val sink = Sink.foreach[ByteString] { bytes => outputStream.write(bytes.toArray) }
res.bodyAsSource.runWith(sink).andThen {
case result =>
outputStream.close()
result.get
} map (_ => Success(file))
case other => Future(Failure[File](new Exception("HTTP Failure, response code " + other + " : " + res.statusText)))
}
}
}

Scala: List[Future] to Future[List] disregarding failed futures

I'm looking for a way to convert an arbitrary length list of Futures to a Future of List. I'm using Playframework, so ultimately, what I really want is a Future[Result], but to make things simpler, let's just say Future[List[Int]] The normal way to do this would be to use Future.sequence(...) but there's a twist... The list I'm given usually has around 10-20 futures in it, and it's not uncommon for one of those futures to fail (they are making external web service requests).
Instead of having to retry all of them in the event that one of them fails, I'd like to be able to get at the ones that succeeded and return those.
For example, doing the following doesn't work:
import scala.concurrent._
import scala.concurrent.ExecutionContext.Implicits.global
import scala.util.Success
import scala.util.Failure
val listOfFutures = Future.successful(1) :: Future.failed(new Exception("Failure")) ::
Future.successful(3) :: Nil
val futureOfList = Future.sequence(listOfFutures)
futureOfList onComplete {
case Success(x) => println("Success!!! " + x)
case Failure(ex) => println("Failed !!! " + ex)
}
scala> Failed !!! java.lang.Exception: Failure
Instead of getting the only the exception, I'd like to be able to pull the 1 and 3 out of there. I tried using Future.fold, but that apparently just calls Future.sequence behind the scenes.
The trick is to first make sure that none of the futures has failed. .recover is your friend here, you can combine it with map to convert all the Future[T] results to Future[Try[T]]] instances, all of which are certain to be successful futures.
note: You can use Option or Either as well here, but Try is the cleanest way if you specifically want to trap exceptions
def futureToFutureTry[T](f: Future[T]): Future[Try[T]] =
f.map(Success(_)).recover { case x => Failure(x)}
val listOfFutures = ...
val listOfFutureTrys = listOfFutures.map(futureToFutureTry(_))
Then use Future.sequence as before, to give you a Future[List[Try[T]]]
val futureListOfTrys = Future.sequence(listOfFutureTrys)
Then filter:
val futureListOfSuccesses = futureListOfTrys.map(_.filter(_.isSuccess))
You can even pull out the specific failures, if you need them:
val futureListOfFailures = futureListOfTrys.map(_.filter(_.isFailure))
Scala 2.12 has an improvement on Future.transform that lends itself in an anwser with less codes.
val futures = Seq(Future{1},Future{throw new Exception})
// instead of `map` and `recover`, use `transform`
val seq = Future.sequence(futures.map(_.transform(Success(_))))
val successes = seq.map(_.collect{case Success(x)=>x})
successes
//res1: Future[Seq[Int]] = Future(Success(List(1)))
val failures = seq.map(_.collect{case Failure(x)=>x})
failures
//res2: Future[Seq[Throwable]] = Future(Success(List(java.lang.Exception)))
I tried Kevin's answer, and I ran into a glitch on my version of Scala (2.11.5)... I corrected that, and wrote a few additional tests if anyone is interested... here is my version >
implicit class FutureCompanionOps(val f: Future.type) extends AnyVal {
/** Given a list of futures `fs`, returns the future holding the list of Try's of the futures from `fs`.
* The returned future is completed only once all of the futures in `fs` have been completed.
*/
def allAsTrys[T](fItems: /* future items */ List[Future[T]]): Future[List[Try[T]]] = {
val listOfFutureTrys: List[Future[Try[T]]] = fItems.map(futureToFutureTry)
Future.sequence(listOfFutureTrys)
}
def futureToFutureTry[T](f: Future[T]): Future[Try[T]] = {
f.map(Success(_)) .recover({case x => Failure(x)})
}
def allFailedAsTrys[T](fItems: /* future items */ List[Future[T]]): Future[List[Try[T]]] = {
allAsTrys(fItems).map(_.filter(_.isFailure))
}
def allSucceededAsTrys[T](fItems: /* future items */ List[Future[T]]): Future[List[Try[T]]] = {
allAsTrys(fItems).map(_.filter(_.isSuccess))
}
}
// Tests...
// allAsTrys tests
//
test("futureToFutureTry returns Success if no exception") {
val future = Future.futureToFutureTry(Future{"mouse"})
Thread.sleep(0, 100)
val futureValue = future.value
assert(futureValue == Some(Success(Success("mouse"))))
}
test("futureToFutureTry returns Failure if exception thrown") {
val future = Future.futureToFutureTry(Future{throw new IllegalStateException("bad news")})
Thread.sleep(5) // need to sleep a LOT longer to get Exception from failure case... interesting.....
val futureValue = future.value
assertResult(true) {
futureValue match {
case Some(Success(Failure(error: IllegalStateException))) => true
}
}
}
test("Future.allAsTrys returns Nil given Nil list as input") {
val future = Future.allAsTrys(Nil)
assert ( Await.result(future, 100 nanosecond).isEmpty )
}
test("Future.allAsTrys returns successful item even if preceded by failing item") {
val future1 = Future{throw new IllegalStateException("bad news")}
var future2 = Future{"dog"}
val futureListOfTrys = Future.allAsTrys(List(future1,future2))
val listOfTrys = Await.result(futureListOfTrys, 10 milli)
System.out.println("successItem:" + listOfTrys);
assert(listOfTrys(0).failed.get.getMessage.contains("bad news"))
assert(listOfTrys(1) == Success("dog"))
}
test("Future.allAsTrys returns successful item even if followed by failing item") {
var future1 = Future{"dog"}
val future2 = Future{throw new IllegalStateException("bad news")}
val futureListOfTrys = Future.allAsTrys(List(future1,future2))
val listOfTrys = Await.result(futureListOfTrys, 10 milli)
System.out.println("successItem:" + listOfTrys);
assert(listOfTrys(1).failed.get.getMessage.contains("bad news"))
assert(listOfTrys(0) == Success("dog"))
}
test("Future.allFailedAsTrys returns the failed item and only that item") {
var future1 = Future{"dog"}
val future2 = Future{throw new IllegalStateException("bad news")}
val futureListOfTrys = Future.allFailedAsTrys(List(future1,future2))
val listOfTrys = Await.result(futureListOfTrys, 10 milli)
assert(listOfTrys(0).failed.get.getMessage.contains("bad news"))
assert(listOfTrys.size == 1)
}
test("Future.allSucceededAsTrys returns the succeeded item and only that item") {
var future1 = Future{"dog"}
val future2 = Future{throw new IllegalStateException("bad news")}
val futureListOfTrys = Future.allSucceededAsTrys(List(future1,future2))
val listOfTrys = Await.result(futureListOfTrys, 10 milli)
assert(listOfTrys(0) == Success("dog"))
assert(listOfTrys.size == 1)
}
I just came across this question and have another solution to offer:
def allSuccessful[A, M[X] <: TraversableOnce[X]](in: M[Future[A]])
(implicit cbf: CanBuildFrom[M[Future[A]], A, M[A]],
executor: ExecutionContext): Future[M[A]] = {
in.foldLeft(Future.successful(cbf(in))) {
(fr, fa) ⇒ (for (r ← fr; a ← fa) yield r += a) fallbackTo fr
} map (_.result())
}
The idea here is that within the fold you are waiting for the next element in the list to complete (using the for-comprehension syntax) and if the next one fails you just fallback to what you already have.
You can easily wraps future result with option and then flatten the list:
def futureToFutureOption[T](f: Future[T]): Future[Option[T]] =
f.map(Some(_)).recover {
case e => None
}
val listOfFutureOptions = listOfFutures.map(futureToFutureOption(_))
val futureListOfOptions = Future.sequence(listOfFutureOptions)
val futureListOfSuccesses = futureListOfOptions.flatten
You can also collect successful and unsuccessful results in different lists:
def safeSequence[A](futures: List[Future[A]]): Future[(List[Throwable], List[A])] = {
futures.foldLeft(Future.successful((List.empty[Throwable], List.empty[A]))) { (flist, future) =>
flist.flatMap { case (elist, alist) =>
future
.map { success => (elist, alist :+ success) }
.recover { case error: Throwable => (elist :+ error, alist) }
}
}
}
If you need to keep failed futures for some reason, e.g., logging or conditional processing, this works with Scala 2.12+. You can find working code here.
val f1 = Future(1)
val f2 = Future(2)
val ff = Future.failed(new Exception())
val futures: Seq[Future[Either[Throwable, Int]]] =
Seq(f1, f2, ff).map(_.transform(f => Success(f.toEither)))
val sum = Future
.sequence(futures)
.map { eithers =>
val (failures, successes) = eithers.partitionMap(identity)
val fsum = failures.map(_ => 100).sum
val ssum = successes.sum
fsum + ssum
}
assert(Await.result(sum, 1.second) == 103)