How do I perform a fold operation on a sequence of Failures - scala

We have some legacy code in our codebase that is eventually going to be refactored to use Validated and Either from Cats library. This is because Validated does not use fail-fast mechanics. The unrefactored code uses fail-fast mechanics of Try monad.
Since the refactoring hasn't happened yet, I am doing a kludgy hack to get around the fact that the Try monad is fail-fast. I am having trouble implementing it however.
I basically have a list of type Try[T] that is guaranteed to all be Failures.
I am trying to aggregate all of the error messages of all the Failures into a single Failure.
Here is the function I am refactoring:
private def extractTry[T](xs: IndexedSeq[Try[T]]): Try[IndexedSeq[T]] = {
val failures = xs.collect { case Failure(ex) => Failure(ex) }
if (failures.size > 0) failures.head
else Success(xs.map(_.get))
}
Instead of failures.head in the second line of the method, I want to aggregate all the Failures.
So something like
if (failures.size > 0) failures.foldLeft(Failure(new IllegalArgumentException(""))){case (Failure(acc), Failure(e)) => Failure(new IllegalArgumentException(acc.getMessage + e.getMessage))}
The only thing I don't like about this implementation is that I would like each step of fold not to use IllegalArgumentException, but to use the new element's exception type. So the idea is to keep the exception type of the last element in failures, and not to use an arbitrary exception type.
We are planning to eventually use Either[Throwable, T] in place of Try and will probably run into the exact same problem there when we try to aggregate errors. We want to keep the exception type and not assign an arbitrary one like IllegalArgumentException. So this problem is going to have to be solved sooner or later, and I would prefer that it be sooner.
Does anyone have any suggestions? Any help would be appreciated.

Ideally, we would follow suggestion by #Luis. Until then consider perhaps something like so
sealed trait OverallResult[+T]
case class OverallError(accumulatedMessage: String, finalErrorCode: Int) extends OverallResult[Nothing]
case class OverallSuccess[T](xs: IndexedSeq[T]) extends OverallResult[T]
object OverallResult {
/**
* Aggregating over a chain of Failures, it will only keep the exception type of the last Failure.
* This is just a heuristic to decide on the error code. Depending on the exception type, we use
* a different error code. So NoSuchElementException is 404 and IllegalArgumentException is 400.
*/
def apply[T](xs: IndexedSeq[Try[T]]): OverallResult[T] = {
val failures = xs.collect { case Failure(ex) => ex }
if (failures.nonEmpty) {
val accMessage = failures.map(_.getMessage).mkString("[", ",", "]")
OverallError(accMessage, errorCode(failures.last))
}
else OverallSuccess(xs.map(_.get))
}
private def errorCode(ex: Throwable): Int = ex match {
case _: NoSuchElementException => 404
case _: IllegalArgumentException => 400
case e => throw new RuntimeException("Unexpected exception. Fix ASAP!", e)
}
}
OverallResult(Vector(Try(throw new NoSuchElementException("boom")), Try(throw new IllegalArgumentException("crash"))))
OverallResult(Vector(Try(42), Try(11)))
which outputs
res0: OverallResult[Nothing] = OverallError([boom,crash],400)
res1: OverallResult[Int] = OverallSuccess(Vector(42, 11))
Note explicit documentation of the heuristic mentioned in the comments:
/**
* Aggregating over a chain of Failures, it will only keep the exception type of the last Failure.
* This is just a heuristic to decide on the error code. Depending on the exception type, we use
* a different error code. So NoSuchElementException is 404 and IllegalArgumentException is 400.
*/
Error accumulation is simulated with
failures.map(_.getMessage).mkString("[", ",", "]")
and overall status code decided with
errorCode(failures.last)
Now clients of extractTry need to be refactored to pattern match on OverallResult ADT, and finalErrorCode instead of exceptions, but lower level codebase should remain unaffected.

Mario wrote up a response that I think deserves being the accepted answer because of its thoroughness. It was while reading his answer that I stumbled upon another solution that requires less code change, but still gets the job done.
The answer is to pattern match on the exception type, which seems very obvious in retrospect.
if (failures.size > 0) failures.foldLeft(Failure(new IllegalArgumentException(""))){case (Failure(acc), Failure(e)) =>
val message = acc.getMessage + e.getMessage
e match {
case ex: IllegalArgumentException => Failure(new IllegalArgumentException(message))
case ex: NoSuchElementException => Failure(new NoSuchElementException(message))
}
}

Related

Is there a way to chain `Try` as a monad while also not catching exceptions raised along the chain?

The Scala Try construct together with its flatMap does not work as I would expect or want it to. The TL;DR is that I want to do a series of operations that can fail in two ways: either by raising an exception, which should be promoted and caught higher up in the call stack, or by returning Failure, as the failure must logically be handled in different parts of the program.
I would expect something like this to do the trick:
def firstStepSucceeds(): Try[Int] = Try {
1
}
def secondStepThrows(input: Int) = {
throw new Exception("Exception thrown in second step")
}
// I expect this to propagate the exception thrown in secondStepThrows
firstStepSucceeds() flatMap (secondStepThrows _)
(Full Scastie with example)
However, in this case, the flatMap() call actually implicitly catches the uncaught exception thrown by secondStepThrows, which is not what I want (which is why I left out the Try block). Is there a way to get the same behaviour without the implicit exception-catching?
What happens in a Try should stay in a Try. Most Scala programmers would be very surprised if a function returning a Try also sometimes threw an exception.
The typical pattern if you want to handle exceptions in different places is to differentiate by the type of the exception. So
val partiallyRecoveredTry = originalTry.recover{
case _: SecondStepException => "second step had an exception"
}
// Further up the call stack
partiallyRecoveredTry.getOrElse("first step had an exception")
Try.flatMap() did not caught exceptions implicitely, it is the essence of Try.
When you use it, it is very explicit, and that's the goal.
I don't really understand what you want, but is something like that is possible for you ?
try {
val first = firstStepSucceeds()
val second = first.map(secondStepThrows).get
val third = secondStepFails(second)
// ...
}
catch {
case e: Exception => ???
}
I did some further experimentation, and what I ended up with was this reimplementation of Try as (the now right-biased and hence monadic) Either:
object CatchAll {
def apply[SomeType](block: => SomeType) = try { Right(block) }
catch { case e: Throwable => Left(e) }
}
def firstStepSucceeds() = CatchAll {
1
}
def firstStepFails() = CatchAll {
throw new Exception("First step failed")
}
def secondStepSucceeds(input: Int) = CatchAll {
input + 1
}
def secondStepFails(input: Int) = CatchAll {
throw new Exception("Second step failed in try block!")
}
def secondStepThrows(input: Int) = {
throw new Exception("Second step failed unexpectedly!")
}
firstStepSucceeds() flatMap (secondStepSucceeds _)
firstStepFails() flatMap (secondStepSucceeds _)
firstStepSucceeds() flatMap (secondStepFails _)
// This now throws an exception as expected
//firstStepSucceeds() flatMap (secondStepThrows _)

Best practices on error handling in Scala using Try

I have handled exception as follows:
def calculate(input: Option[Double]): Try[String] =
Try {
input match {
case (Some(value)) => value.toString
case (None) => throw new IllegalArgumentException("No value found")
}
}
And in client code:
val result = calculate(....)
result match {
case Success(i) => println(i)
case Failure(s) => throw s // or log(s) to log the issue and continue
}
Is it good enough practice or much better can be done for clean and elegant code base?
Try usually used to cover parts which might throw an error, like in cases if you are using some Java libs, which can throw an exception. But, if you would like to return possible error and force client to handle it, Either[A, B] is much better option, at least because you can specify more precise error type for Left[A] and safely to pattern match over your's A type, instead of do possibly incorrect pattern matching against some Throwable, like you would do for Failure(t).
So, in your case possible solution would look like:
sealed trait CalculationError
case class Error1(cause: String) extends CalculationError
def calculate(input: Option[Double]): Either[CalculationError, String] =
input match {
case (Some(value)) => Right(value.toString)
case (None) => Left(Error1("No value found"))
}
}
val result = calculate(....)
result match {
case Right(i) => println(i)
case Left(Error1(s)) => println(s)
}
This is safer approach, because you can later add another type of error , say case class Error2(cause: String) extends CalculationError and on client pattern matching code part, compile will show a warn message that you missed handling of new error: Match is not exhaustive. In case of Failure(t) compile won't be able suggest such warning, so it's easier to make mistake on error handling side.
Hope this helps!

Is it wrong to give the unit back?

I have the following function:
private def constraintToJson(req: => Request[IO])
: EitherT[IO, Throwable, Unit]
= {
val err: EitherT[IO, Throwable, Unit] = EitherT.fromEither[IO](Left(new Exception("Not JSON format request.")))
req.contentType match {
case Some(s) =>
if (s != `Content-Type`(MediaType.`application/json`))
err
else
EitherT.fromEither[IO](Right(()))
case None =>
err
}
}
The question is, it is wrong to return a Unit if it is right or there is another choice?
I think that returning Unit (wrapped into Either/EitherT) might be OK if this is a kind of the final step of your computation and it actually does not produce any output. In other cases (most probably including yours) you should return some value for the successful case so that you can chain it further. So the main question is: how constraintToJson is supposed to be used? The obvious suspect for your case is returning Request or its body if the Content-Type matches JSON because this is most probably the data that will be used by the next step.
Simplified Question
Let me abstract your problem first by removing IO.
The question is whether a method with a signature like the following is useful:
def validate[A](a: Request[A]): Either[Error,()] =
if(isOk(a.someProperty)) Left(()) else Right("invalid")
Noting that Either[A,()] =:= Option[A], this method checks for the presence of any Error in the value of the request, and returns this Error, if detected.
What could be done with such a method?
Check and exit
We could write a program that checks a Request:
val someRequest: Request[X] = getRequestFrom("somewhere")
validate(someRequest).toLeft.foreach{error =>
println("invalid request: " + error
System.exit(-1)
}
System.exit(0)
In this case validate has a useful signature.
Check and go on
We could also validate a request, and then go on to execute it:
val someRequest: Request[X] = getRequestFrom("somewhere")
validate(someRequest) match {
case Left(error) =>
println("invalid request: " + error)
System.exit(-1)
case Right(_) =>
println("executing request: " + someRequest.execute)
System.exit(0)
}
While this code works perfectly well, it throws away an argument (_) in the second match clause. This is an indicator of some bad design. The code within the case Right(_) takes the value someRequest from outside, and treats it as a valid request because of the way you wrote your code.
If we would call someReuqest.execute within the Left case, we would probably end up with a runtime exception when executing an invalid Request.
Depending on our desired level of rigor, this can be a code smell.
We can improve the code in the following way.
Using a Non-Unit return type
We can circumvent returning Unit by simply returning the checked argument.
This appears to be a bit redundant, and we shall later see how to turn the returned value into something useful.
But let's first look at this option.
def validateTyped[A](a: Request[A]): Either[Error,Request[A]] =
if(isOk(a.someProperty)) Left(a) else Right("invalid")
Then we can write the Check and go on code as
val someRequest: Request[X] = getRequestFrom("somewhere")
validate(someRequestTyped) match {
case Left(error) =>
println("invalid request: " + error)
System.exit(-1)
case Right(validatedRequest) =>
//we now execute the validated request
println("executing request: " + validatedRequest.execute)
System.exit(0)
}
Now this improves the code a bit.
We don't throw away a returned value anymore in the second match clause.
We can execute the validatedRequest, because we've read in the documentation of validateRequest that a returned Right value is well formed, and it may be executed without error.
Looks good, right?
But we can do better.
Prevent execution of malformed requests
We can still improve it further by totally preventing executing a malformed Request by changing our Request type.
case class Request[A](query: String){
def validate: Either[Error,ValidRequest[A]] = ???
}
case class ValidRequest[A](query: String){
def execute: A
}
With this code, it is not possible to call Request.execute anymore, as a validation now becomes mandatory.
Otherwise the code will not compile.
We also note that the Right value returned by Request.validate is now needed.
It has become the only source of obtaining a ValidRequest.
Now it is impossible to call execute a Request that isn't validated, yet:
val someRequest: Request[X] = getRequestFrom("somewhere")
someRequest.validate match {
case Left(error) =>
println("invalid request: " + error)
System.exit(-1)
case Right(validatedRequest) =>
println("executing request: " + validatedRequest.execute)
System.exit(0)
}
Conclusion
In this way we have turned a strange looking method that returned Unit into a method that returns a meaningful and needed value.
In addition it has become impossible to execute a Request that has not been validated successfully.

Jedis in scala and handling errors

I am trying to find the best way to handle jedis commands from scala. I am trying to implement a finally block, and prevent the java exceptions from bubbling up to my caller.
Does the following code make sense, and is it the best I can do performance wise, if I want to ensure that I handle exceptions when redis may be down temporarily? This trait would be extended by an object, and I'd call objectname.del(key). I feel like I'm combining too many concepts (Either, Option, Try, feels like there should be a cleaner way)
trait MyExperiment {
implicit class TryOps[T](val t: Try[T]) {
def eventually[Ignore](effect: => Ignore): Try[T] = {
val ignoring = (_: Any) => { effect; t }
t transform (ignoring, ignoring)
}
}
val jpool:JedisPool = initialize()
// init the pool at object creation
private def initialize(): JedisPool =
{
val poolConfig = new JedisPoolConfig()
poolConfig.setMaxIdle(10)
poolConfig.setMinIdle(2)
poolConfig.setTestWhileIdle(true)
poolConfig.setTestOnBorrow(true)
poolConfig.setTestOnReturn(true)
poolConfig.setNumTestsPerEvictionRun(10)
new JedisPool( poolConfig , "localhost" )
}
// get a resource from pool. This can throw an error if redis is
// down
def getFromPool: Either[Throwable,Jedis] =
Try(jpool.getResource) match {
case Failure(m) => Left(m)
case Success(m) => Right(m)
}
// return an object to pool
// i believe this may also throw an error if redis is down?
def returnToPool(cache:Jedis): Unit =
Try(jpool.returnResource(cache))
// execute a command -- "del" in this case, (wrapped by
// the two methods above)
def del(key: String) : Option[Long] = {
getFromPool match {
case Left(m) => None
case Right(m) => Try(m.del(key)) eventually returnToPool(m) match {
case Success(r) => Option(r)
case Failure(r) => None
}
}
}
}
Not an exact answer, but I moved on after doing some performance testing. Using the standard java-ish exception blocks ended up being much faster at high iterations (at 10,000 iterations, it was about 2.5x faster than the (bad) code above). That also cleaned up my code, although it's more verbose.
So the answer I arrived at is to use the Java-style exception blocks which provide for the finally construct. I believe it should be significantly faster, as long as exceptions are a very rare occurance.

Try with exception logging

Scala's Try is very useful.
I'd like to use that pattern, but log all exceptions.
How can I do this?
Define the following helper:
import scala.util.{Try, Failure}
def LogTry[A](computation: => A): Try[A] = {
Try(computation) recoverWith {
case e: Throwable =>
log(e)
Failure(e)
}
}
Then you can use it as you would use Try, but any exception will be logged through log(e).
Starting Scala 2.13, the chaining operation tap can be used to apply a side effect (in this case some logging) on any value while returning the original value:
import util.chaining._
val x = Try("aa".toInt).tap(_.failed.foreach(println))
// java.lang.NumberFormatException: For input string: "aa"
// x: Try[Int] = Failure(java.lang.NumberFormatException: For input string: "aa")
Or an equivalent pattern matching version:
val x = Try("aa".toInt).tap { case Failure(e) => println(e) case _ => }
// java.lang.NumberFormatException: For input string: "aa"
// x: Try[Int] = Failure(java.lang.NumberFormatException: For input string: "aa")
The tap chaining operation applies a side effect (in this case println or some logging) on a value (in this case a Try) while returning the original unmodified value on which tap is applied (the Try):
def tap[U](f: (A) => U): A
You can tweak it even further using implicit class
def someMethod[A](f: => A): Try[A] = Try(f)
implicit class LogTry[A](res: Try[A]) {
def log() = res match {
case Success(s) => println("Success :) " + s); res
case Failure(f) => println("Failure :( " + f); res
}
}
Now you can call someMethod and on its result call log like this:
scala> someMethod(1/0).log
Failure :( java.lang.ArithmeticException: / by zero
and
scala> someMethod(1).log
Success :) 1
Of course println method inside implicit class can be substituted with any logging you want.
You used the term "exceptions" which is ambiguous. (java.lang.)Throwable is the root of anything that can be placed behind the throw term. java.lang.Exception is one of the two descendants of Throwable (the other being java.lang.Error). Further making this ambiguous is java.lang.RuntimeException, a descendant of Exception, which is probably where you mostly want to spend your logging time (unless you are doing lower level application framework or hardware driver implementations).
Assuming you are wanting to log literally ALL instances of Throwable, then you would need something like this (NOT RECOMMENDED):
def logAtThrowable(f: => A): Try[A] =
try
Try(f) match {
case failure # Failure(throwable) =>
log(s"Failure: {throwable.getMessage}")
failure
case success # _ =>
//uncomment out the next line if you want to also log Success-es
//log(s"Success: {throwable.getMessage}")
success
}
catch throwable: Throwable => {
//!NonFatal pathway
log(s"Failure: {throwable.getMessage}")
throw throwable
}
The external try/catch is required to capture all the Throwable instances which are filtered away by scala.util.control.NonFatal within the Try's try/catch block.
That said...there is a Java/JVM rule: you should never define a catch clause at the resolution of Throwable (again, unless you are doing lower level application framework or hardware driver implementations).
Following the intention of this rule, you would need to narrow the Throwable to you only emitted logging at the finer grained level, say something more refined, like java.lang.RuntimeException. If so, the code would look like this (recommended):
def logAtRuntimeException(f: => A): Try[A] =
Try(f) match {
case failure # Failure(throwable) =>
throwable match {
case runtimeException: RuntimeException =>
log(s"Failure: {runtimeException.getMessage}")
}
failure
case success # _ =>
success
}
In both code snippets above, you will notice that I used match as opposed to .recoverWith. This is to facilitate easily adding a rethrow that works. It turns out that all the methods on Try are themselves also wrapped with try/catch blocks. This means that if you want to log the Throwable and then rethrow it, if you are using one of the Try methods like recoverWith, the rethrow is immediately recaught and placed into a Failure thereby completely undermining the value of the intentional rethrow. By using match, the rethrow is guaranteed to succeed as it remains outside any of the Try methods.
If you would like to see more of the rabbit holes around this particular area, I created a blog post of my own exploration.