I am writing a recursive retry function in scala where I want to know if there was a runtime error with the future creation. If there is then future instantiation is retried.
Imagine I have a database query function:
dbLookup(userId : UserId) : Option[UserName] = ???
The retry would look something of the form:
retry[T](f : () => Future[Option[T]],
notifyFailure : (t : Throwable) => Unit,
n : Int) : Future[Option[T]] = {
if(n <= 0) { Future{None} }
else {
val fut = f()
if(f.resultsInException) { //I don't know how to write this
notifyFailure(f.exception)
retry(f, notifyFailure, n-1) //try again
}
else {
f.originalValueAsFuture
}
}
}
How can this future functionality be implemented and allow for tail recursion optimization?
This function could be used to retry the database 10 times for a user if the execution context keeps throwing an exception when I try to create a Future:
val userNameAfterRetries =
retry(() => Future{dbLookup("1234")},
(t) => system error (s"future creation error : $t"),
10)
Note: this is sort of possible with Future.fallbackTo, but unfortunately fallbackTo takes in a Future[T] rather than a () => Future[T]. This is important because using fallbackTo would result in retrying at least 1 extra times even if the first attempt was successful.
Thank you in advance for your consideration and response.
How about this?
def retry[T](f: () => Future[Option[T]],
notifyFailure: Throwable => Unit,
n: Int)(implicit ec : ExecutionContext): Future[Option[T]] = {
if (n <= 0) Future.failed(new RuntimeException("Exceeded number of allowed retries"))
else f().recoverWith { case originalError => notifyFailure(originalError); retry(f, notifyFailure, n - 1) }
}
Update on tail recursion: The nature of Future is asynchronous so unless you want to await for result, I do not quite see it being possible to make it #tailrec because you will have to use recursion in callback.
Also practical note: if you know it is always ~10 retries, I would not be afraid of recursion.
Related
I have a method, which may throw an Exception depends on passed value:
private def transform(in: Int): Future[Boolean] = in match {
case i if i < 0 => Future.successful(true)
case i if i > 0 => Future.successful(false)
case i if i == 0 => throw new IllegalStateException()
}
And second method, which apply above method to each element of List in parallel way.
def massTransform(ints: List[Int])(implicit ex: ExecutionContext):
Future[List[Boolean]] = {
Future.traverse(ints){
i => transform(i).recover {
case e: IllegalStateException => false
}
}
}
I expected, recover will capture the IllegalStateException and return Future(false). But my code fails with IllegalStateException
The problem is that you are throwing outside of a Future. You need to wrap your exception in a Future, otherwise what happens is that the method itself throws instead of returning a failed Future. You can simplify your method as follows:
def transform(in: Int): Future[Boolean] =
if (in == 0) Future.failed(new IllegalStateException)
else Future.successful(in < 0)
You can play around with this code here on Scastie.
This looks like a simplification of your actual logic. If by any chance that's not (or not completely) the case, I recommend removing the unnecessary Future nesting as follows:
def transform(in: Int): Boolean =
if (in == 0) throw new IllegalStateException else in < 0
def massTransform(
ints: List[Int]
)(implicit ex: ExecutionContext): Future[List[Boolean]] =
Future(ints.map { i =>
try transform(i)
catch { case e: IllegalStateException => false }
})
Notice how now the transform method throws and exception handling is done in the Future constructor.
You can play around with this second version on Scastie as well.
transform is implemented in incorrectly, should be:
case i if i == 0 => Future.failed(new IllegalStateException())
I guess this is a simplified example, because this code does not really parallelise the computation btw, unless transform is in reality doing IO or quite expensive computation.
In the code below I have two Play for Scala functions, the first one catches an exception (this works fine) and in the second one I'm trying to rewrite it using Try.
I have two problems with Try: (1) when the number is negative the method doesn't fail, (2) I need to wrap all the responses with Future.successful.
How to fix this code?
class Test extends Controller {
def test1 = Action.async { request =>
val future = isPositive(-1)
future.map { result =>
Ok("OK, it's positive")
}
.recover {
case e => Ok(e.getMessage)
}
}
def isPositive(i: Int) = Future {
if (i<0)
throw new Exception ( "Number is negative" )
else
i
}
def test2 = Action.async { request =>
isPositiveWithTry(-1) match {
case Success(s) => Future.successful(Ok("OK, it's positive (Try succeded)"))
case Failure(f) => Future.successful(Ok(f.getMessage + " (Try failed)"))
}
}
def isPositiveWithTry(i: Int) : Try[Future[Int]] = Try {
isPositive(i)
}
}
In isPositive method exceptions are already caught by Future
def isPositive(i: Int) = Future {
if (i<0)
throw new Exception ( "Number is negative" )
else
i
}
In the below code
def isPositiveWithTry(i: Int) : Try[Future[Int]] = Try {
isPositive(i)
}
isPositive already catches all expections and Try is always a success.
So, when i is negative. Exception raised are handled by future and try gets a success value, resultant Try is a success. So you get successful Try with a failed Future inside.
Understanding using Grenade example
Assume throwing the exception as blowing up a Grenade.
Assume Future and Try as two layers. When grenade is blasted inside the double layer of Try[Future] i.e Try is around Future and grenade is gone off in the Future.
Now Future withstands the blast and becomes a failed value. As Future already took the damage caused by the damage of exception(grenade). Try will be a success but the value inside the Try is a failed future value. That failed future value is nothing but the exception raised.
Try is redundant when you are using Future
You can refactor your code to below one
Get rid of isPositiveWithTry. This method is not needed.
def isPositive(i: Int) = Future {
if (i<0)
throw new Exception ( "Number is negative" )
else
i
}
def test2 = Action.async { request =>
isPositive(-1).flatMap { _ =>
Future.successful(Ok("OK, it's positive (Try succeded)"))
}.recoverWith {
case f: Throwable => Future.successful(Ok(f.getMessage + " (Try failed)"))
}
}
Again test2 can also be written as
def test2 = Action.async { request =>
isPositive(-1).map { _ =>
Ok("OK, it's positive (Try succeded)")
}.recover {
case f: Throwable => Ok(f.getMessage + " (Try failed)")
}
}
In case isPositive returns Try
def isPositive(i: Int) = Try {
if (i<0)
throw new Exception ( "Number is negative" )
else
i
}
Now test2 will look like
def test2 = Action.async { request =>
isPositive(-1) match {
case Success(s) => Future.successful(Ok("OK, it's positive (Try succeded)"))
case Failure(f) => Future.successful(Ok(f.getMessage + " (Try failed)"))
}
}
Couple points:
1) You need to rewrite your isPositive such that it does not surround itself via a Future. The Future is catching the exception.
def isPositive(i: Int) ={
if (i<0)
throw new Exception ( "Number is negative" )
else
i
}
2) If you have a Try and you want a Future, then you can use the method on the companion object of Future, Future.fromTry. That will take a Try and turn it into the correct state of a Future.
def returnFuture[A](x: A): Future[A] = {
val xFuture = Future { x } // suppose an API call that returns a future
xFuture.flatMap(x => {
println(x) // logging the value of x
xFuture
})
}
This is the way I'm currently doing it. To provide more context:
This function is being called inside an API when a request is made and I'd like the log message to be printed just before the value computed in the request is returned. Which is why, the following is not a good solution for me:
def returnFuture[A](x: A): Future[A] = {
val xFuture = Future { x } // suppose an API call that returns a future
xFuture.map(x => {
println(x) // logging the value of x
})
xFuture
}
Logging is a side-effect, meaning that you don't want the operation to fail if the logging fails for any reason (e.g. a call to toString throwing NPE).
Future#andThen is perfect for this use case. From the docs:
Applies the side-effecting function to the result of this future, and returns a new future with the result of this future.
This method allows one to enforce that the callbacks are executed in a specified order.
Note that if one of the chained andThen callbacks throws an exception, that exception is not propagated to the subsequent andThen callbacks. Instead, the subsequent andThen callbacks are given the original value of this future.
Your example becomes:
def returnFuture[A](x: A): Future[A] = {
Future { x } // suppose an API call that returns a future
.andThen { case Success(v) => println(v) }
}
You can use onComplete callback:
def returnFuture[A](x: A): Future[A] = {
val f = Future { x }
f.onComplete(println)
f
}
A map will work too:
def returnFuture[A](x: A): Future[A] = {
Future { x }.map { v =>
println(v)
v
}
}
Keep in mind that the whole point of using Futures is that you are trying to avoid blocking and that you don't control exactly when the Future will be executed. So, if you want more detailed logs while keeping the asynchronous nature of a Future, do something like this:
def doSomething(param: String): String = {
// log something here
val result = param.toUpperCase
// log something else here
result
}
def asFuture(param: String) = Future {
doSomething(param)
}
In other words, if this is an option, add logs to the x operation instead.
I am trying to create a try clause analogue which repeats code block if exception occurred inside this code block.
def retry(attempts: Int)(func: Unit => Unit) {
var attempt = 0
while (attempt < attempts) {
attempt += 1
try {
func()
} catch {
case _: Throwable =>
}
}
throw new Exception()
}
I expect that it can be used like this
retry(10) { // I would like to pass this block as function
val res = someNotReliableOp(); // <- exception may occur here
print(res)
}
But it doesn't work:
Operations.scala:27: error: type mismatch;
found : Unit
required: Unit => Unit
print(res)
^
one error found
What is the most concise way to pass custom block to my function?
You just need to change your method definition a tiny bit:
def retry(attempts: Int)(func: => Unit)
Unit => Unit means: a function that takes a parameter with a type of Unit and evaluates to Unit.
=> Unit means: a function that takes no parameters and evaluates to Unit. This is called call by name.
Consider
def retry(attempts: Int)(func: => Unit) {
for {
i <- Stream range (0, attempts)
v = Try (func()) toOption
if (v == None)
} ()
}
The for comprehension will invoke func up to an attempts number of times and will stop streaming if the invocation to func succeeds, namely if Try(func) toOption does not deliver None.
For each iteration in the for comprehension, the do-nothing () function is called.
If interested in details on each failure for func, consider replacing the Try in the comprehension with
v = Try(func()) match {
case Success(x) => Some(())
case Failure(x) => println(x) ; None
}
which preserves the semantics initially suggested, yet it extracts information on each failed attempt.
I'd like to run my future call n times, for example 5. Future "execution" will take some time and I want to call new one only when previous was completed. Something like:
def fun(times: Int): Future[AnyRef] = {
def _fun(times: Int) = {
createFuture()
}
(1 to times).foldLeft(_fun)((a,b) => {
println(s"Fun: $b of $times")
a.flatMap(_ => _fun)
})
}
So I want to call "_fun" function n times one by one. "createFuture()" will take some time, so "_fun" shouldn't be called again before previous future was completed. Also, I want to create a non-blocking solution. Currently, this code executes without waiting for previous future to end.
Any ideas how to make it work?
Thanks for your answers!
Without understanding what exactly you want the final future to return (I'm going to just return the result of the last completed future), you could try something like this:
def fun(times: Int): Future[AnyRef] = {
val prom = Promise[AnyRef]()
def _fun(t: Int) {
val fut = createFuture()
fut onComplete {
case Failure(ex) => prom.failure(ex)
case Success(result) if t >= times => prom.success(result)
case Success(result) => _fun(t + 1)
}
}
_fun(1)
prom.future
}
This is a sort of recursive solution that will chain the futures together on completion, stopping the chaining when the max number of times has been reached. This code is not perfect, but certainly conveys one possible solution for making sure the successive futures do not fire until the previous future completed successfully.
I think it will be nicer if you make it recursive using flatMap.
Let's imagine you have your createFuture defined as:
def createFuture() = Future( println("Create future"))
We can create a function to compose the result of createFuture with:
def compose(f: () => Future[Unit])(b: Future[Unit]) = b.flatMap(_ => f())
And then you can define fun as:
def fun(n : Int) = {
def nTimes(n : Int, f : Future[Unit] => Future[Unit], acc : Future[Unit]) = if (n == 0) acc else nTimes(n-1,f,f(acc))
nTimes(n,compose(createFuture),Future())
}