I am new to Scala and am using the method def process however getting build errors like "method wasSuccessful in class IOResult is deprecated (since 2.6.0): status is always set to Success(Done)
case Success((sftpResult, updateList)) if sftpResult.wasSuccessful =>". My question is how would I then implement this in my code? I apologise if the question seems stupid.
override def process(changedAfter: Option[ZonedDateTime], changedBefore: Option[ZonedDateTime])
(implicit fc: FlowContext): Future[(IOResult, Seq[Operation.Value])] = {
val now = nowUtc
val accountUpdatesFromDeletions: Source[AccountUpdate, NotUsed] =
dbService
.getDeleteActions(before = now)
.map(deletionEventToAccountUpdate)
val result = getAccountUpdates(changedAfter, changedBefore)
.merge(getErrorUpdates)
.merge(accountUpdatesFromDeletions)
.mapAsync(4)(writer.writePSVString)
.viaMat(creditBureauService.sendUpdate)(Keep.right)
.mapAsync(4)(au =>
for {
_ <- dbService.performUpdate(au)
_ <- performActionsDelete(now, au)
} yield au.operation
)
.toMat(Sink.seq)(Keep.both)
.withAttributes(ActorAttributes.supervisionStrategy(decider))
.run()
tupleFutureToFutureTuple(result) andThen {
case Success((sftpResult, updateList)) if sftpResult.wasSuccessful =>
val total = updateList.size
val deleted = updateList.count(_ == Operation.DELETED)
val updated = updateList.count(_ == Operation.UPDATED)
val inserted = updateList.count(_ == Operation.INSERTED)
log.info(s"SUCCESS! Uploaded $total accounts to Equifax.")
log.info(s"There were $deleted deletions, " +
s"$updated updates and " +
s"$inserted insertions to the database")
monitor.gauge("upload.process.batch.successful.total", total)
monitor.gauge("upload.process.batch.successful.deleted", deleted)
monitor.gauge("upload.process.batch.successful.updated", updated)
monitor.gauge("upload.process.batch.successful.inserted", inserted)
case Success((sftpResult, _)) if !sftpResult.wasSuccessful =>
val sw = new StringWriter
sftpResult.getError.printStackTrace(new PrintWriter(sw))
log.error("Failed to send data: " + sw.toString)
monitor.gauge("upload.process.batch.sftp.failed", 1)
case Failure(e: Throwable) =>
val sw = new StringWriter
e.printStackTrace(new PrintWriter(sw))
log.error(sw.toString)
monitor.gauge("upload.process.batch.failed", 1)
}
}
Both wasSuccessful and getError have been deprecated since 2.6. Since IOResult is in general wrapped in a Future which by itself already tells whether the I/O operation is a Success or Failure, the status parameter of a successfully returned IOResult is in essence redundant, thus is now always set to Success[Done].
Given that, you could simplify/refactor your case matching snippet to something like below:
tupleFutureToFutureTuple(result) andThen {
case Success((sftpResult, updateList)) =>
val total = updateList.size
val deleted = updateList.count(_ == Operation.DELETED)
val updated = updateList.count(_ == Operation.UPDATED)
val inserted = updateList.count(_ == Operation.INSERTED)
log.info(s"SUCCESS! Uploaded $total accounts to Equifax.")
log.info(s"There were $deleted deletions, " +
s"$updated updates and " +
s"$inserted insertions to the database")
monitor.gauge("upload.process.batch.successful.total", total)
monitor.gauge("upload.process.batch.successful.deleted", deleted)
monitor.gauge("upload.process.batch.successful.updated", updated)
monitor.gauge("upload.process.batch.successful.inserted", inserted)
case Failure(e: Throwable) =>
val sw = new StringWriter
e.printStackTrace(new PrintWriter(sw))
log.error(sw.toString)
monitor.gauge("upload.process.batch.failed", 1)
}
Related
I have an Akka server who is asking the mallet file (some output) from an actor.
However in mallet actor code, several steps are done.
In which files are taken, modified, new files are created and saved in resource directory couple of times.
I need to run the actor sequentially so I am using map in calling future.
However the job is getting NullPointerException as there is no file for next future.
And as soon as I am stopping the server. all the files are getting generated in resources directory.
I need the files in resources directory as soon as individual future is completed. Please suggest
Below is the code of my server
lazy val routes: Route = apiRoutes
Configuration.parser.parse(args,Configuration.ConfigurationOptions()) match {
case Some(config) =>
val serverBinding: Future[Http.ServerBinding] = Http().bindAndHandle(routes, config.interface, config.port)
serverBinding.onComplete {
case Success(bound) =>
println(s"Server online at http://${bound.localAddress.getHostString}:${bound.localAddress.getPort}/")
case Failure(e) =>
log.error("Server could not start: ", e)
system.terminate()
}
case None =>
system.terminate()
}
Await.result(system.whenTerminated, Duration.Inf)
}
Below is the code of receive function.
def receive: Receive = {
case GetMalletOutput(malletFile) => createMalletResult(malletFile).pipeTo(sender())
}
def createMalletResult(malletFile: String): Future[MalletModel] = {
//sample malletResult
val topics = Array(Topic("1", "2").toJson)
var mM: Future[MalletModel] = Future{MalletModel("contentID", topics)}
//first Future to save file in resource
def saveFile(malletFile: String): Future[String] = Future {
val res = MalletResult(malletFile)
val converted = res.Score.parseJson.convertTo[MalletRepo]
val fileName = converted.ContentId
val fileTemp = new File("src/main/resources/new_corpus/" + fileName)
val output = new BufferedWriter(new FileWriter("src/main/resources/new_corpus/" + fileName))
output.write(converted.ContentText)
//output.close()
malletFile
}
//Second Future to used the resource file and create new one
def t2v(malletFile: String): Future[String] = Future{
val tmpDir = "src/main/resources/"
logger.debug(tmpDir.toString)
logger.debug("t2v Started")
Text2Vectors.main(("--input " + tmpDir + "new_corpus/ --keep-sequence --remove-stopwords " + "--output " + tmpDir + "new_corpus.mallet --use-pipe-from " + tmpDir + "corpus.mallet").split(" "))
logger.debug("text2Vector Completed")
malletFile
}
//another future to take file from resource and save in the new file back in resource
def infer(malletFile: String): Future[String] = Future {
val tmpDir = "src/main/resources/"
val tmpDirNew = "src/main/resources/inferResult/"
logger.debug("infer started")
InferTopics.main(("--input " + tmpDir + "new_corpus.mallet --inferencer " + tmpDir + "inferencer " + "--output-doc-topics " + tmpDirNew + "doc-topics-new.txt --num-iterations 1000").split(" "))
logger.debug("infer Completed")
malletFile
}
//final future to return the requested output using the saved future
def response(malletFile: String): Future[MalletModel] = Future{
logger.debug("response Started")
val lines = Source.fromResource("src/main/resources/inferResult/doc-topics-new.txt")
.getLines
.toList
.drop(1) match {
case Nil => List.empty
case x :: xs => x.split(" ").drop(2).mkString(" ") :: xs
}
logger.debug("response On")
val result = MalletResult(malletFile)
val convert = result.Score.parseJson.convertTo[MalletRepo]
val contentID = convert.ContentId
val inFile = lines.mkString(" ")
val a = inFile.split(" ").zipWithIndex.collect { case (v, i) if (i % 2 == 0) =>
(v, i)
}.map(_._1)
val b = inFile.split(" ").zipWithIndex.collect { case (v, i) if (i % 2 != 0) =>
(v, i)
}.map(_._1)
val paired = a.zip(b) // [(s,t),(s,t)]
val topics = paired.map(x => Topic(x._2, x._1).toJson)
logger.debug("validating")
logger.debug("mallet results...")
logger.debug("response Done")
MalletModel(contentID, topics)
}
//calling one future after another to run future sequntially
val result: Future[MalletModel] =
saveFile(malletFile).flatMap(malletFile =>
t2v(malletFile).flatMap(mf =>
infer(mf).flatMap(mf =>
response(mf))))
result
}
}
It seems I have got the answer myself.
The problem was I was trying to write the file in resource directory and reading the file back. to resolve the issue I wrote the file in tmpDir and read it back with bufferreader instead of source. and bam it worked.
I am creating a user registration module. On submission (using JSON), I want to check if the JSON parsed correctly. If there is a problem in JSON, I want to return error. If the JSON is correct, I want to check if the user already exists (look at firstname). Data is in MongoDB. I am using ReactiveMongoPlugin 0.10. I will use 'one' method which returns Future[Option[BSONDocument]]. How do I wait for this Future to finish before the Action completes?
Approach 1 - use Action and try to handle result of Future myself. Code doesn't compile and dont know how to wait for the Future to finish
def registrationRequest = Action(parse.json) { request => {
Logger.debug("received message:" + request)
Logger.debug("received message:" + request.body)
val jr:JsResult[User2] = request.body.validate[User2]
Logger.debug( "jr is "+jr)
jr match {
case s:JsSuccess[User2] => {
val user = s.get
Logger.debug("opening database connection")
val driver = new MongoDriver()
val connection = driver.connection(List("localhost"))
val db = connection.db("website-db")
val collection = db.collection[BSONCollection]("users")
// the data from client is a JSON of type {user:{firstname:"name"}}. I have created code to parse the JSON
val query = BSONDocument("user"-> BSONDocument("firstname"->user.firstname))
Logger.debug("query is:"+query)
val result = collection.find(query).one
I want to now wait for result and return either an Ok(Json.toJson(ack)) or BadRequest(Json.toJson(ack)). How do i do that? I have written following code but I am stuck at two points (a) will the code wait for future to finish (b) onComplete returns Unit but Play's Action requires play.api.mvc.Result. How do I do that?
//I guess data would be Success or Failure
result onComplete ( data =>
data match {
//If Success, value would be Some or None
case Success(value) => {
value match {
case None => { //no record. Can add
Logger.debug("No record from mongo: Can add")
val ack = Acknowledgment (1, "Welcome " + user.firstName + " " + user.lastName)
Logger.debug ("success ack Json:" + Json.toJson (ack) )
Ok (Json.toJson (ack) )
}
case Some(x) => { //duplicae record
Logger.debug("error from Mongo. Duplicate:"+x)
val ack = Acknowledgment(0,"duplicate: "+x.toString())
Logger.debug("fail ack:"+Json.toJson(ack))
BadRequest(Json.toJson(ack))
}
}
}
case Failure (e)=> {
Logger.debug("error from Mongo."+e)
val ack = Acknowledgment(0,"MongoDB Error: "+e.toString())
Logger.debug("fail ack:"+Json.toJson(ack))
BadRequest(Json.toJson(ack))
}
}) //onComplete returns Unit. Action needs play.api.mvc.Result
case f:JsError => {
Logger.debug("error: "+JsError.toFlatJson(f))
val ack = Acknowledgment(0,JsError.toFlatJson(f).toString())
Logger.debug("fail ack:"+Json.toJson(ack))
BadRequest(Json.toJson(ack))
}
}
}
Approach 2 - I read that I should use Action.async but I am unable to fit the pieces together. The 2nd approach I followed was to use Action.Async but the code didnt compile because it expects Future[SimpleResult]
def registrationRequest = Action.async(parse.json) { request => {
Logger.debug("received message:" + request)
Logger.debug("received message:" + request.body)
val jr:JsResult[User2] = request.body.validate[User2]
Logger.debug( "jr is "+jr)
jr match {
case s:JsSuccess[User2] => {
val user = s.get
Logger.debug("opening database connection")
val driver = new MongoDriver()
val connection = driver.connection(List("localhost"))
val db = connection.db("website-db")
val collection = db.collection[BSONCollection]("users")
val query = BSONDocument("user"-> BSONDocument("firstname"->user.firstName))
Logger.debug("query is:"+query)
//result is of type Future[Option[BSONDocument]]
val result = collection.find(query).one
result.map(option => option match {
case None => {
//no record. Can add
Logger.debug("No record from mongo: Can add")
val ack = Acknowledgment(1, "Welcome " + user.firstName + " " + user.lastName)
Logger.debug("success ack Json:" + Json.toJson(ack))
Ok(Json.toJson(ack))
}
case Some(x) => {
//duplicae record
Logger.debug("error from Mongo. Duplicate:" + x)
val ack = Acknowledgment(0, "duplicate: " + x.toString())
Logger.debug("fail ack:" + Json.toJson(ack))
BadRequest(Json.toJson(ack))
}
}
)
}
case f:JsError => {
Logger.debug("error: "+JsError.toFlatJson(f))
val ack = Acknowledgment(0,JsError.toFlatJson(f).toString())
Logger.debug("fail ack:"+Json.toJson(ack))
BadRequest(Json.toJson(ack)) //Action.async expect scala.concurrent.Future[play.api.mvc.SimpleResult]
}
}
}
The solution is to use Action.async which returns Future[SimpleResult]. Inside the code use map and flatMap on Future to return Future[SimpleResult].
I'm trying to write some basic queries with Slick for SQLite database
Here is my code:
class MigrationLog(name: String) {
val migrationEvents = TableQuery[MigrationEventTable]
lazy val db: Future[SQLiteDriver.backend.DatabaseDef] = {
val db = Database.forURL(s"jdbc:sqlite:$name.db", driver = "org.sqlite.JDBC")
val setup = DBIO.seq(migrationEvents.schema.create)
val createFuture = for {
tables <- db.run(MTable.getTables)
createResult <- if (tables.length == 0) db.run(setup) else Future.successful()
} yield createResult
createFuture.map(_ => db)
}
val addEvent: (String, String) => Future[String] = (aggregateId, eventType) => {
val id = java.util.UUID.randomUUID().toString
val command = DBIO.seq(migrationEvents += (id, aggregateId, None, eventType, "CREATED", System.currentTimeMillis, None))
db.flatMap(_.run(command).map(_ => id))
}
val eventSubmitted: (String, String) => Future[Unit] = (id, batchId) => {
val q = for { e <- migrationEvents if e.id === id } yield (e.batchId, e.status, e.updatedAt)
val updateAction = q.update(Some(batchId), "SUBMITTED", Some(System.currentTimeMillis))
db.map(_.run(updateAction))
}
val eventMigrationCompleted: (String, String, String) => Future[Unit] = (batchId, id, status) => {
val q = for { e <- migrationEvents if e.batchId === batchId && e.id === id} yield (e.status, e.updatedAt)
val updateAction = q.update(status, Some(System.currentTimeMillis))
db.map(_.run(updateAction))
}
val allEvents = () => {
db.flatMap(_.run(migrationEvents.result))
}
}
Here is how I'm using it:
val migrationLog = MigrationLog("test")
val events = for {
id <- migrationLog.addEvent("aggregateUserId", "userAccessControl")
_ <- migrationLog.eventSubmitted(id, "batchID_generated_from_idam")
_ <- migrationLog.eventMigrationCompleted("batchID_generated_from_idam", id, "Successful")
events <- migrationLog.allEvents()
} yield events
events.map(_.foreach(event => event match {
case (id, aggregateId, batchId, eventType, status, submitted, updatedAt) => println(s"$id $aggregateId $batchId $eventType $status $submitted $updatedAt")
}))
The idea is to add event first, then update it with batchId (which also updates status) and then update the status when the job is done. events should contain events with status Successful.
What happens is that after running this code it prints events with status SUBMITTED. If I wait a while and do the same allEvents query or just go and check the db from command line using sqlite3 then it's updated correctly.
I'm properly waiting for futures to be resolved before starting the next operation, auto-commit should be enabled by default.
Am I missing something?
Turns out the problem was with db.map(_.run(updateAction)) which returns Future[Future[Int]] which means that the command was not finished by the time I tried to run another query.
Replacing it with db.flatMap(_.run(updateAction)) solved the issue.
I moved from Casbah to Reactive Mongo and from that moment I couldn't make work the test of my actor.
I have a dao for the persistence layer and tests for that tier. All the tests passed. So, the only thing that comes to my mind its a problem of synchronization.
" UserActor " should {
val socketActorProbe = new TestProbe(system)
val peyiProbe = new TestProbe(system)
val identifyId = 1
val emailCsr = "csr#gmail.com"
val emailPeyi = "peyi#gmail.com"
val point = new Point[LatLng](new LatLng(-31.4314041, -64.1670626))
" test preStart() " in new WithApplication {
db.createDB(id1, id2, id3)
val userActorRefCsr = TestActorRef[UserActor](Props(classOf[UserActor], emailCsr, socketActorProbe.ref))
val csr = userActorRefCsr.underlyingActor
val userActorRef = TestActorRef[UserActor](Props(classOf[UserActor], emailPeyi, socketActorProbe.ref))
val peyi = userActorRef.underlyingActor
peyi.receive(ActorIdentity(identifyId, Option(userActorRefCsr)))
db.clearDB()
}
Actor class.
class UserActor(email: String, upstream: ActorRef) extends Actor {
import UserActor._
val identifyId = 1
val usersFromDB = ReactiveMongoFactory.db.collection[BSONCollection]("users")
val userDao = new UserDao(usersFromDB)
val meFuture = userDao.findMeByEmail(email)
var friends: Map[String, ActorRef] = Map()
override def preStart() = {
meFuture onComplete { result =>
val emailsFriends: List[String] = userDao.getMyFriendsEmail(result.get.get)
println(emailsFriends)
for (email <- emailsFriends) {
println("sending msg to " + email)
context.actorSelection("/user/" + email) ! Identify(identifyId)
}
}
}
private def giveMyFriend(email: String): Option[ActorRef] = {
for(friend <- friends){
if (friend._1 == email) new Some(friend._2)
}
None
}
def active(another: ActorRef): Actor.Receive = {
case Terminated(`another`) => context.stop(self)
}
def receive = {
case ActorIdentity(`identifyId`, Some(actorRef)) =>
meFuture onComplete { result =>
println(" ... subscribing ... " + result.get.get.basicProfile.email)
actorRef ! Subscribe(result.get.get.basicProfile.email.get)
context.watch(actorRef)
context.become(active(actorRef))
}
case Subscribe(email) =>
friends += (email -> sender)
context watch sender
case Terminated(user) => {
for(friend <- friends){
if (friend._2 == user ) friends -= friend._1 //removing by key
}
}
case UserMoved(email, point) =>
upstream ! UserPosition(email, System.currentTimeMillis(), point.coordinates)
}
}
Im receiving the following output.
The exception is thrown in the following lines of code.
def findMeByEmail(email: String): Future[Option[User]] = {
val query = BSONDocument("email" -> email)
println( " .... finding user ..... email: " + email )
val cursor = users.find(query).cursor[BSONDocument]
val userFuture = cursor.headOption.map(
doc => Some(userReader.read(doc.get))
)
userFuture
}
If I run the test for that method, it's all ok.
describe("get my friends emails") {
it("returns a list of emails") {
val futureUser = userDao.findMeByEmail("csr#gmail.com")
ScalaFutures.whenReady(futureUser) { result =>
val friends = userDao.getMyFriendsEmail(result.get)
assert(friends.length == 2)
}
}
}
Basically, Im trying to look my friends (Other actor) and then register them in a map to have a reference. I couldn't find any good example which shows tests using Reactive Mongo with Actors.
I hope somebody can help me to understand whats going on here. Thanks in advance.
I'm looking for a way to convert an arbitrary length list of Futures to a Future of List. I'm using Playframework, so ultimately, what I really want is a Future[Result], but to make things simpler, let's just say Future[List[Int]] The normal way to do this would be to use Future.sequence(...) but there's a twist... The list I'm given usually has around 10-20 futures in it, and it's not uncommon for one of those futures to fail (they are making external web service requests).
Instead of having to retry all of them in the event that one of them fails, I'd like to be able to get at the ones that succeeded and return those.
For example, doing the following doesn't work:
import scala.concurrent._
import scala.concurrent.ExecutionContext.Implicits.global
import scala.util.Success
import scala.util.Failure
val listOfFutures = Future.successful(1) :: Future.failed(new Exception("Failure")) ::
Future.successful(3) :: Nil
val futureOfList = Future.sequence(listOfFutures)
futureOfList onComplete {
case Success(x) => println("Success!!! " + x)
case Failure(ex) => println("Failed !!! " + ex)
}
scala> Failed !!! java.lang.Exception: Failure
Instead of getting the only the exception, I'd like to be able to pull the 1 and 3 out of there. I tried using Future.fold, but that apparently just calls Future.sequence behind the scenes.
The trick is to first make sure that none of the futures has failed. .recover is your friend here, you can combine it with map to convert all the Future[T] results to Future[Try[T]]] instances, all of which are certain to be successful futures.
note: You can use Option or Either as well here, but Try is the cleanest way if you specifically want to trap exceptions
def futureToFutureTry[T](f: Future[T]): Future[Try[T]] =
f.map(Success(_)).recover { case x => Failure(x)}
val listOfFutures = ...
val listOfFutureTrys = listOfFutures.map(futureToFutureTry(_))
Then use Future.sequence as before, to give you a Future[List[Try[T]]]
val futureListOfTrys = Future.sequence(listOfFutureTrys)
Then filter:
val futureListOfSuccesses = futureListOfTrys.map(_.filter(_.isSuccess))
You can even pull out the specific failures, if you need them:
val futureListOfFailures = futureListOfTrys.map(_.filter(_.isFailure))
Scala 2.12 has an improvement on Future.transform that lends itself in an anwser with less codes.
val futures = Seq(Future{1},Future{throw new Exception})
// instead of `map` and `recover`, use `transform`
val seq = Future.sequence(futures.map(_.transform(Success(_))))
val successes = seq.map(_.collect{case Success(x)=>x})
successes
//res1: Future[Seq[Int]] = Future(Success(List(1)))
val failures = seq.map(_.collect{case Failure(x)=>x})
failures
//res2: Future[Seq[Throwable]] = Future(Success(List(java.lang.Exception)))
I tried Kevin's answer, and I ran into a glitch on my version of Scala (2.11.5)... I corrected that, and wrote a few additional tests if anyone is interested... here is my version >
implicit class FutureCompanionOps(val f: Future.type) extends AnyVal {
/** Given a list of futures `fs`, returns the future holding the list of Try's of the futures from `fs`.
* The returned future is completed only once all of the futures in `fs` have been completed.
*/
def allAsTrys[T](fItems: /* future items */ List[Future[T]]): Future[List[Try[T]]] = {
val listOfFutureTrys: List[Future[Try[T]]] = fItems.map(futureToFutureTry)
Future.sequence(listOfFutureTrys)
}
def futureToFutureTry[T](f: Future[T]): Future[Try[T]] = {
f.map(Success(_)) .recover({case x => Failure(x)})
}
def allFailedAsTrys[T](fItems: /* future items */ List[Future[T]]): Future[List[Try[T]]] = {
allAsTrys(fItems).map(_.filter(_.isFailure))
}
def allSucceededAsTrys[T](fItems: /* future items */ List[Future[T]]): Future[List[Try[T]]] = {
allAsTrys(fItems).map(_.filter(_.isSuccess))
}
}
// Tests...
// allAsTrys tests
//
test("futureToFutureTry returns Success if no exception") {
val future = Future.futureToFutureTry(Future{"mouse"})
Thread.sleep(0, 100)
val futureValue = future.value
assert(futureValue == Some(Success(Success("mouse"))))
}
test("futureToFutureTry returns Failure if exception thrown") {
val future = Future.futureToFutureTry(Future{throw new IllegalStateException("bad news")})
Thread.sleep(5) // need to sleep a LOT longer to get Exception from failure case... interesting.....
val futureValue = future.value
assertResult(true) {
futureValue match {
case Some(Success(Failure(error: IllegalStateException))) => true
}
}
}
test("Future.allAsTrys returns Nil given Nil list as input") {
val future = Future.allAsTrys(Nil)
assert ( Await.result(future, 100 nanosecond).isEmpty )
}
test("Future.allAsTrys returns successful item even if preceded by failing item") {
val future1 = Future{throw new IllegalStateException("bad news")}
var future2 = Future{"dog"}
val futureListOfTrys = Future.allAsTrys(List(future1,future2))
val listOfTrys = Await.result(futureListOfTrys, 10 milli)
System.out.println("successItem:" + listOfTrys);
assert(listOfTrys(0).failed.get.getMessage.contains("bad news"))
assert(listOfTrys(1) == Success("dog"))
}
test("Future.allAsTrys returns successful item even if followed by failing item") {
var future1 = Future{"dog"}
val future2 = Future{throw new IllegalStateException("bad news")}
val futureListOfTrys = Future.allAsTrys(List(future1,future2))
val listOfTrys = Await.result(futureListOfTrys, 10 milli)
System.out.println("successItem:" + listOfTrys);
assert(listOfTrys(1).failed.get.getMessage.contains("bad news"))
assert(listOfTrys(0) == Success("dog"))
}
test("Future.allFailedAsTrys returns the failed item and only that item") {
var future1 = Future{"dog"}
val future2 = Future{throw new IllegalStateException("bad news")}
val futureListOfTrys = Future.allFailedAsTrys(List(future1,future2))
val listOfTrys = Await.result(futureListOfTrys, 10 milli)
assert(listOfTrys(0).failed.get.getMessage.contains("bad news"))
assert(listOfTrys.size == 1)
}
test("Future.allSucceededAsTrys returns the succeeded item and only that item") {
var future1 = Future{"dog"}
val future2 = Future{throw new IllegalStateException("bad news")}
val futureListOfTrys = Future.allSucceededAsTrys(List(future1,future2))
val listOfTrys = Await.result(futureListOfTrys, 10 milli)
assert(listOfTrys(0) == Success("dog"))
assert(listOfTrys.size == 1)
}
I just came across this question and have another solution to offer:
def allSuccessful[A, M[X] <: TraversableOnce[X]](in: M[Future[A]])
(implicit cbf: CanBuildFrom[M[Future[A]], A, M[A]],
executor: ExecutionContext): Future[M[A]] = {
in.foldLeft(Future.successful(cbf(in))) {
(fr, fa) ⇒ (for (r ← fr; a ← fa) yield r += a) fallbackTo fr
} map (_.result())
}
The idea here is that within the fold you are waiting for the next element in the list to complete (using the for-comprehension syntax) and if the next one fails you just fallback to what you already have.
You can easily wraps future result with option and then flatten the list:
def futureToFutureOption[T](f: Future[T]): Future[Option[T]] =
f.map(Some(_)).recover {
case e => None
}
val listOfFutureOptions = listOfFutures.map(futureToFutureOption(_))
val futureListOfOptions = Future.sequence(listOfFutureOptions)
val futureListOfSuccesses = futureListOfOptions.flatten
You can also collect successful and unsuccessful results in different lists:
def safeSequence[A](futures: List[Future[A]]): Future[(List[Throwable], List[A])] = {
futures.foldLeft(Future.successful((List.empty[Throwable], List.empty[A]))) { (flist, future) =>
flist.flatMap { case (elist, alist) =>
future
.map { success => (elist, alist :+ success) }
.recover { case error: Throwable => (elist :+ error, alist) }
}
}
}
If you need to keep failed futures for some reason, e.g., logging or conditional processing, this works with Scala 2.12+. You can find working code here.
val f1 = Future(1)
val f2 = Future(2)
val ff = Future.failed(new Exception())
val futures: Seq[Future[Either[Throwable, Int]]] =
Seq(f1, f2, ff).map(_.transform(f => Success(f.toEither)))
val sum = Future
.sequence(futures)
.map { eithers =>
val (failures, successes) = eithers.partitionMap(identity)
val fsum = failures.map(_ => 100).sum
val ssum = successes.sum
fsum + ssum
}
assert(Await.result(sum, 1.second) == 103)