What I want to do is to use Future to open a thread to handle an async task that could be called frequently.
But in the async task. I also call two Future function to get the information from different data source.
I write a program to simulate this situation.
import scala.concurrent.{ExecutionContext, Future}
import scala.util.{Success, Failure}
import scala.util.control.Breaks
import ExecutionContext.Implicits.global
object AsyncTest {
def main(args: Array[String]) {
try {
println("Run...")
aSyncTask
} catch {
case e => e.printStackTrace()
}
Thread.sleep(999999)
}
//dataSource1
def getInfo1 : Future[String] = Future{
"A"
}
//dataSource2 let it wait 3 seconds...
def getInfo2 : Future[String] = Future{
Thread.sleep(3000)
"B"
}
def aSyncTask = Future{
getInfo1
getInfo2
var result1 : String = null
var result2 : String = null
getInfo1.onComplete{
case Success(value) => result1 = value
case Failure(t) => println {
"An error has occured: " + t.getMessage
}
}
getInfo2.onComplete{
case Success(value) => result2 = value
case Failure(t) => println {
"An error has occured: " + t.getMessage
}
}
/*I want to wait both Future function completed then
I can do some business logic */
Breaks.breakable{
while (true) {
if (result1 != null && result2 != null)
Breaks.break
}
}
println("----------------------")
println("result1:"+result1)
println("result2:"+result2)
println("----------------------")
}
}
After I compiled and executed this program, it output nothing. just wait.
Run...
I expected that I could see output :
Run...
----------------------
result1:A
result2:B
----------------------
So, I added some code in while loop for debugging .
Breaks.breakable{
while (true) {
println(result1)
println(result2)
if (result1 != null && result2 != null)
Breaks.break
}
}
Then it output :
Run...
A
null
A
null
A
null
A
null
A
null
(After 3 seconds...)
A
B
----------------------
result1:A
result2:B
----------------------
What's going on?? I just add two println to see the two variables.
Why the program could be executed as I expected when I just print it?
Future was created to be composable, so I will take a risk and assume that you wanted something like this:
// Important to initialize them outside of for comprehension
val (f1, f2) = (getInfo1, getInfo2)
val ab: Future[(String, String)] =
for {
r1 <- f1
r2 <- f2
} yield (r1, r2) // Do whatever you want to do with r1 and r2
println(Await.result(ab, Duration(10000, MILLISECONDS)))
Related
I am struggling to understand the for comprehension and exception handling in Scala.
If the first statement in a for comprehension fails , recover is not able to catch the exception.
Code where recover catches the exception successfully(Exception thrown in 2nd statement):
import scala.util.{Success, Try}
object ExceptionThrownIn2ndStatement {
def failTryUnit(x: Unit): Try[Int] = {
println(x)
val a = 1 / 0
new Success(a)
}
def main(args: Array[String]): Unit = {
(for {
var0 <- Try(println("Zeroth function"))
varFailure <- failTryUnit(var0) //exception thrown here
var1 <- Try(println("first function", varFailure))
} yield var1) recover { case e =>
println("Exception caught", e) //exception caught here
}
}
}
Output :
Zeroth function
()
(Exception caught,java.lang.ArithmeticException: / by zero)
Code where recover does NOT catch the exception successfully :
import scala.util.{Success, Try}
object ExceptionThrownIn1stStatement {
def failTryUnit(x: Unit): Try[Int] = {
println(x)
val a = 1 / 0
new Success(a)
}
def main(args: Array[String]): Unit = {
(for {
varFailure <- failTryUnit({}) //exception thrown here
var0 <- Try(println("zeroth function", varFailure))
var1 <- Try(println("first function", var0))
} yield var1) recover { case e =>
println("Exception caught",e) //Exception does not get caught here
}
}
}
Output:
()
Exception in thread "main" java.lang.ArithmeticException: / by zero
at ExceptionThrownIn1stStatement$.failTryUnit(ExceptionThrownIn1stStatement.scala:6)
at ExceptionThrownIn1stStatement$.main(ExceptionThrownIn1stStatement.scala:12)
at ExceptionThrownIn1stStatement.main(ExceptionThrownIn1stStatement.scala)
Edit : I understand that this is not the way recover is supposed to be used.
I am just confused as to why this happens. Please help me understand this. I am new to Scala.
Here is a shorter example that demonstrates the same behavior:
Success(42).flatMap(x => { assert(false); Success(x + 58) })
vs.
{ assert(false); Success(42) }.flatMap(x => Success(x + 58))
The first one will return a Failure with a caught error. The second one will crash with an AssertionError.
The first returns a Failure because that's the semantics of Trys flatMap - it catches all exceptions that occur during the execution of the function passed to it.
The second one crashes immediately, because the very first statement is an assert(false), so you never get to the point where you construct a Try in the first place, the AssertionError is thrown before the first Success constructor is invoked. It wouldn't matter whether you append more recovers on it or not - no Try will ever be instantiated in this program.
Here is what you would have to do to catch the exception occurring during the very first calculation (42):
Try { assert(false); 42 }.flatMap(x => Success(x + 58))
In your code, that would be
def failTryUnit(x: Unit): Try[Int] = Try {
println(x)
1 / 0
}
Trying to execute a function in a given time frame, but if computation fails by TimeOut get a partial result instead of an empty exception.
The attached code solves it.
The timedRun function is from Computation with time limit
Any better approach?.
package ga
object Ga extends App {
//this is the ugly...
var bestResult = "best result";
try {
val result = timedRun(150)(bestEffort())
} catch {
case e: Exception =>
print ("timed at = ")
}
println(bestResult)
//dummy function
def bestEffort(): String = {
var res = 0
for (i <- 0 until 100000) {
res = i
bestResult = s" $res"
}
" " + res
}
//This is the elegant part from stackoverflow gruenewa
#throws(classOf[java.util.concurrent.TimeoutException])
def timedRun[F](timeout: Long)(f: => F): F = {
import java.util.concurrent.{ Callable, FutureTask, TimeUnit }
val task = new FutureTask(new Callable[F]() {
def call() = f
})
new Thread(task).start()
task.get(timeout, TimeUnit.MILLISECONDS)
}
}
I would introduce a small intermediate class for more explicitly communicating the partial results between threads. That way you don't have to modify non-local state in any surprising ways. Then you can also just catch the exception within the timedRun method:
class Result[A](var result: A)
val result = timedRun(150)("best result")(bestEffort)
println(result)
//dummy function
def bestEffort(r: Result[String]): Unit = {
var res = 0
for (i <- 0 until 100000) {
res = i
r.result = s" $res"
}
r.result = " " + res
}
def timedRun[A](timeout: Long)(initial: A)(f: Result[A] => _): A = {
import java.util.concurrent.{ Callable, FutureTask, TimeUnit }
val result = new Result(initial)
val task = new FutureTask(new Callable[A]() {
def call() = { f(result); result.result }
})
new Thread(task).start()
try {
task.get(timeout, TimeUnit.MILLISECONDS)
} catch {
case e: java.util.concurrent.TimeoutException => result.result
}
}
It's admittedly a bit awkward since you don't usually have the "return value" of a function passed in as a parameter. But I think it's the least-radical modification of your code that makes sense. You could also consider modeling your computation as something that returns a Stream or Iterator of partial results, and then essentially do .takeWhile(notTimedOut).last. But how feasible that is really depends on the actual computation.
First, you need to use one of the solution to recover after the future timed out which are unfortunately not built-in in Scala:
See: Scala Futures - built in timeout?
For example:
def withTimeout[T](fut:Future[T])(implicit ec:ExecutionContext, after:Duration) = {
val prom = Promise[T]()
val timeout = TimeoutScheduler.scheduleTimeout(prom, after)
val combinedFut = Future.firstCompletedOf(List(fut, prom.future))
fut onComplete{case result => timeout.cancel()}
combinedFut
}
Then it is easy:
var bestResult = "best result"
val expensiveFunction = Future {
var res = 0
for (i <- 0 until 10000) {
Thread.sleep(10)
res = i
bestResult = s" $res"
}
" " + res
}
val timeoutFuture = withTimeout(expensiveFunction) recover {
case _: TimeoutException => bestResult
}
println(Await.result(timeoutFuture, 1 seconds))
How to run a transactionally statement in Slick 3.1.x, and capture the result in a Future (without the use of Await)?
This works (but uses Await)
val action = db.run((for {
_ <- table1.filter(_.id1 === id).delete
_ <- table2.filter(_.id2=== id).delete
} yield ()).transactionally)
val result = Await.result(action, Duration.Inf)
However this does not print anything:
val future = db.run((for {
_ <- table1.filter(_.id1 === id).delete
_ <- table2.filter(_.id2=== id).delete
} yield ()).transactionally)
future.map { result => println("result:"+result) }
UPDATE
This is the real code taken from the program that doesn't work. It prints "1" but it never prints "2"
case class UserRole (sk: Int, name: String)
class UserRoleDB(tag: Tag) extends Table[UserRole](tag, "user_roles") {
def sk = column[Int]("sk", O.PrimaryKey)
def name = column[String]("name")
def * = (sk, name) <> ((UserRole.apply _).tupled, UserRole.unapply)
}
class Test extends Controller {
def index = Action.async { request =>
val db = Database.forConfig("db1")
val userRoles = TableQuery[UserRoleDB]
val ur = UserRole(1002,"aaa")
try {
val action = (for {
userRole2 <- userRoles += ur
} yield (userRole2)).transactionally
val future = db.run(action)
println(1)
// val result = Await.result(future, Duration.Inf)
future.map { result => {
println(2)
Ok("Finished OK")
}
}
}
finally db.close
}
}
Coming from the other question you asked: You are opening and then immediately closing the db connection in the finally clause. Therefore your async db operation runs against a closed db connection. That's also why it works by using Await since that blocks the execution of db.close until you received the result set.
So how to fix this?
Either you move db.close into future.map or better you let play-slick handle db connections for you.
Side note
You should close your other question and update this thread accordingly instead.
Your second example is fine. My guess is that you are either running it in standalone program or in test - and it simply finishes before future has a chance to be executed.
Try to add some sleep after your code in your second sample and you'll see it is getting printed. This is definitely not something (this sleep) you would do in your actual code but it will show you it works as it should.
package p1
import scala.util.Failure
import scala.util.Success
import scala.concurrent.Await
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.duration._
object modCheck extends App {
def getStudentRoolNo(name: String) = Future {
println("getStudentRoolNo")
name match {
case "name1" => 1
case "name2" => 2
case _ => throw new Exception("No doesnt exist")
}
}
def getRank(roolNo: Int) = Future {
println("getRank")
Thread.sleep(500)
roolNo match {
case 1 => "1"
case 2 => "2"
case _ => throw new Exception("No roolNo exist")
}
}
def getDetails(roolNo: Int) = Future {
println("getDetails")
roolNo match {
case 1 => "details 1"
case 2 => "Details 2"
case _ => throw new Exception("No details exist")
}
}
def getStudentRecord(name: String) = {
for {
rollNo <- getStudentRoolNo(name)
rank <- getRank(rollNo)
details <- getDetails(rollNo)
} yield (rank + details)
}
getStudentRecord("name1").onComplete {
case Success(ground) => println(s"got my Details $ground")
case Failure(ex) => println("Exception!" + ex)
}
Thread.sleep(2000)
}
I want to execute functions getrank and getDetails in parallel in below code(once the getStudentRollNo is returned). How can I achieve this?
I Tried below way, It seems it still executing in sequentially
Please let me know , How to execute in parallel
Future starts computation when it's created.
for (a <- x; b <- y) yield ??? is desugared to x.flatMap(a => y.map(b => ???))
flatMap() and map() execute it's argument after a Future is completed.
getDetails() can start before completion of getRank() by separating creation of Future and flatMap() invocation.
for {
rollNo <- getStudentRoolNo(name)
rankFuture = getRank(rollNo)
detailsFuture = getDetails(rollNo)
rank <- rankFuture
details <- detailsFuture
} yield (rank + details)
As you have probable guessed, your current code does not make calls to getRank and getDetails in parallel, since its inside for-comprehension. Its a syntactic sugar for map operation. To achieve parallelism you need to create two futures outside for-comprehension.
val student = getStudentRollNo(name)
val detailsFuture = student map {s => getRank(rollNo) }
val rankFuture = student map {s => getDetails(rollNo) }
for {
rank <- rankFuture
details <- detailsFuture
} yield (rank + details)
You can use a zip in your for-comprehension to run two Futures in parallel:
def getStudentRecord(name: String) = {
for {
rollNo <- getStudentRoolNo(name)
rankRollNo <- getRank(rollNo) zip getDetails(rollNo)
} yield rankRollNo
}
In the above getRank and getDetails are run simultaneously and the result is a string tuple. If you want to yield a single string, you will need to separate rankRollNo into its separate components:
yield rankRollNo._1 + rankRollNo._2
In my current method, I am trying to make a series of calls and if any of them fail, I want to be able to continue running the remainder (while capturing the Exception that was thrown). I am having a hard time figuring this out in Scala.
So in this example, I want to kick off each of these calls - RunA, RunB and RunC but if RunB throws an exception, I want to print that and continue kicking off RunC after that.
var result = Try {
new RunA()
new RunB()
new RunC()
} catch {
case e: Throwable => e.printStackTrace()
false
}
Outside of having them all individually wrapped in a Try/Catch, I am sure there are better ways to do this which is why I am hoping someone can help with this.
I looked at the 'Ignoring' exception but it appears to completely ignore the exception which I want to atleast log.
Thanks!
First, don't mix try { ... } catch { ... } up with scala.util.Try{ ... }.
You can
import scala.util._
val runA = Try{ new RunA }
val runB = Try{ new RunB }
val runC = Try{ new RunC }
and then deal with the exceptions as you see fit. For instance, if you want to print and continue, you could deal with the try statements right there:
def getOrPrint[A](f: => A): Option[A] = Try{ f } match {
case Success(x) => Some(x)
case Failure(e) => e.printStackTrace; None
}
getOrPrint{ new RunA }
...
There can be more elegant ways for such things with scalaz (e.g. read an article here for some inspiration: http://johnkurkowski.com/posts/accumulating-multiple-failures-in-a-ValidationNEL/), but with "only" Scala you can do something like this:
import scala.reflect.ClassTag
import scala.util.{Try, Success, Failure}
def tryAndLog[T: ClassTag] = Try {
implicitly[ClassTag[T]].runtimeClass.newInstance.asInstanceOf[T] // new instance
} match {
case Success(_) => true
case Failure(ex) => ex.printStackTrace ; false
}
def tryRunAll = {
val A = tryAndLog[RunA]
val B = tryAndLog[RunB]
val C = tryAndLog[RunC]
A && B && C // returns true if all invocations succeeded, false otherwise
}
You are mixing scala.util.Try with try {} catch {} which are different concepts. Try wraps function into Success(result) or Failure(error) class, and try-catch is like Java try-catch. I suggest you something like this:
class RunA
class RunB
class RunC
class Result(a: RunA, b: RunB, c: RunC)
implicit class LogFailure[T](t: Try[T]) {
def logFailure: Try[T] = t match {
case scala.util.Failure(err) => err.printStackTrace(); t
case _ => t
}
}
val tryA= Try(new RunA())
val tryB= Try(new RunB())
val tryC = Try(new RunC())
val result: Try[Result] = for {
a <- tryA.logFailure
b <- tryB.logFailure
c <- tryC.logFailure
} yield {
// do smth with a, b, c
new Result(a, b, c)
}
If A, B, C will be successful you'll get Success(Result) if one of them failure you'll get Failure with first exception, however all of them will be logged (printed stack trace)