While dealing with error-handling in Scala, I came to the point where I asked myself, whether Trys in a for-comprehension make sense.
Please regard the unit test given below. This test shows two approaches:
The first approach (with call) embeds the methods fooA and fooB, which return regular Strings, into a Try construct.
The second approach (tryCall) uses a for-comprehension that uses methods tryFooA and tryFooB, which return Try[String] each.
For what reason should one prefer the for-comprehension variant with tryCall over the call-variant?
test("Stackoverflow post: Try and for-comprehensions.") {
val iae = new IllegalArgumentException("IAE")
val rt = new RuntimeException("RT")
import scala.util.{Try, Success, Failure}
def fooA(x1: Int) : String = {
println("fooA")
if (x1 == 1) "x1 is 1" else throw iae
}
def fooB(x2: Int) : String = {
println("fooB")
if (x2 == 1) "x2 is 1" else throw rt
}
def tryFooA(x1: Int) : Try[String] = {
Try {
println("tryFooA")
if (x1 == 1) "x1 is 1" else throw iae
}
}
def tryFooB(x2: Int) : Try[String] = {
Try {
println("tryFooB")
if (x2 == 1) "x2 is 1" else throw rt
}
}
def call( x1: Int, x2: Int ) : Try[String] = {
val res: Try[String] = Try{
val a = fooA(x1)
val b = fooB(x2)
a + " " + b
}
res
}
def tryCall( x1: Int, x2: Int ): Try[String] = {
for {
a <- tryFooA(x1)
b <- tryFooB(x2)
} yield (a + " " + b)
}
assert( call(0,0) === tryCall(0,0))
assert( call(0,1) === tryCall(0,1))
assert( call(1,0) === tryCall(1,0))
assert( call(1,1) === tryCall(1,1))
}
The purpose of Try is to let the compiler help you (and anyone else who uses your code) handle and reason about errors more responsibly.
In your examples, the behavior of call and tryCall are more or less identical, and if the fooA, etc. methods were not part of any public API, there'd be little reason to prefer one over the other.
The advantage of Try is that it lets you compose operations that can fail in a clear, concise way. The signatures of tryFooA and tryFooB are upfront about the fact that they may result in (recoverable) failure, and the compiler makes sure that anyone who calls those methods must deal with that possibility. The signatures of fooA and fooB aren't self-documenting in this way, and the compiler can't provide any assurances that they'll be called responsibly.
These are good reasons to prefer the tryFoo signatures. The downside is that callers have to deal with the extra overhead of using map, flatMap, etc., instead of working with the results (if they exist) directly. The for-comprehension syntax is an attempt to minimize this overhead—you get the safety provided by Try with only a little extra syntactic overhead.
Related
I am learning Scala and today came across the Fail Slow mechanism using Scalaz ValidationNel however it is really difficult to understand how to use it. I am reading these blogs: Blog1 , I am reading this StackOverflow post too: StackOverflow but it is really difficult to understand for non functional programmer. Can somebody provide a simple example on how to accumulate errors in ValidationNel in Scala? It will be really helpful to have a description about the example too.
Using the example from the blog you've linked
val sumV: ValidationNEL[String, Int] = for {
a <- 42.successNel[String]
b <- "Boo".failNel[Int]
c <- "Wah wah".failNel[Int] // by defn of flatMap, can't get here
} yield a + b + c
What this is doing is using a flatMap to chain together various operations. 42.successNel[String], for example, creates a Success, and "Boo".failNel[Int] creates a failure. The way flatMap works here is to continue on to the next operations only on a success. So this is a "fail fast" operation - it gathers the first failure into your error case and stops.
If you want to "fail slow" - ie. gather all possible failures, you need to use a different method. This is where Applicative comes in.
val yes = 3.14.successNel[String]
val doh = "Error".failNel[Double]
def addTwo(x: Double, y: Double) = x + y
(yes |#| yes)(addTwo) // Success(6.28)
(doh |#| doh)(addTwo) // Failure(NonEmptyList(Error, Error))
(a |#| b)(someFunctionOfTwoArgsHere) - What this is saying is "Perform the 'a' operation, and perform the 'b' operation, and if both are successful, perform someFunctionOfTwoArgsHere(a,b). Otherwise, take any failures and combine them. So if a fails, but b succeeds, you get a Validation failure with the result of a failing. If a AND b fails, you get a Validation failure with the results of both a and b failing.
Previous answer is good but I understand you come from OOP paradigm so let me put another example comparing both paradigms.
Common code:
val a = "1"
val b = "aaa"
val c = "bbb"
def isAllDigits(x: String) = x forall Character.isDigit
def appendError(x: String, errors: mutable.Buffer[String]) = errors += s"$x is not number"
type Errors = NonEmptyList[String]
// disjunction \/ for fail fast
def toDigitFailFast(x: String): Errors \/ Int = {
if (isAllDigits(x)) {
x.toInt.right
} else {
s"$x is not number".wrapNel.left
}
}
// validation nel (non empty list) for fail slow
def toDigitFS(x: String): ValidationNel[String, Int] = {
if (x forall Character.isDigit) {
x.toInt.successNel
} else {
s"$x is not number".failureNel
}
}
Code for fail fast imperative:
// fail fast imperative programming
println("---\nFail Fast imperative")
val failFastErrors = mutable.Buffer.empty[String]
if(isAllDigits(a)) {
if(isAllDigits(b)) {
if(isAllDigits(c)) {
val total = a.toInt + b.toInt + c.toInt
println(s"Total = ${total}!!")
} else {
appendError(c, failFastErrors)
}
} else {
appendError(b, failFastErrors)
}
} else {
appendError(a, failFastErrors)
}
if(failFastErrors.nonEmpty) {
println("Errors:")
for(error <- failFastErrors) {
println(error)
}
}
Code for fail fast functional (with disjunction /):
val resultFunc = for {
x <- toDigitFailFast(a)
y <- toDigitFailFast(b)
z <- toDigitFailFast(c)
} yield (x + y + z)
resultFunc match {
case \/-(total) => println(s"Total = $total")
case -\/(errors) =>
println("Errors:")
errors.foreach(println)
}
Output on fail fast (only tells you first error):
Fail Fast imperative
Errors:
aaa is not number
Fail Fast functional
Errors:
aaa is not number
Now the fail slow code for imperative:
// fail slow imperative programming
println("---\nFail Slow imperative")
val failSlowErrors = mutable.Buffer.empty[String]
if(!isAllDigits(a)) {
appendError(a, failSlowErrors)
}
if(!isAllDigits(b)) {
appendError(b, failSlowErrors)
}
if(!isAllDigits(c)) {
appendError(c, failSlowErrors)
}
if(failSlowErrors.isEmpty) {
val total = a.toInt + b.toInt + c.toInt
println(s"Total = ${total}!!")
} else {
println("Errors:")
for(error <- failSlowErrors) {
println(error)
}
}
And the functional version (fail slow):
// fail slow functional programming
println("---\nFail Slow functional")
val resultFuncSlow =
(toDigitFS(a) |#| toDigitFS(b) |#| toDigitFS(c)) { _ + _ + _ }
resultFuncSlow match {
case Success(result) => println(result)
case Failure(errors) =>
println("Errors:")
errors.foreach(println)
}
And the output with both errors:
Fail Slow imperative
Errors:
aaa is not number
bbb is not number
---
Fail Slow functional
Errors:
aaa is not number
bbb is not number
val LIST = scala.collection.mutable.MutableList[String]()
val filterF = new Function[Path, Boolean] {
def apply(x: Path): Boolean = {
println("looking into " + x)
val flag = if (x.toString.split("/").last.split("_").last.toLong < System.currentTimeMillis) {
println("considered " + x)
LIST += x.toString
return true
} else {
println("NOT considered " + x)
return false
}
return flag
}
}
I am trying to update the external variable LIST inside the function filterF. But the problem is that after the println("looking into "+x)
line the rest of the code is unreachable.
val flag = if (x.toString.split("/").last.split("_").last.toLong < System.currentTimeMillis) {
println("considered " + x)
LIST += x.toString
return true
} else {
println("NOT considered " + x)
return false
}
return flag
I can't understand why this code is unreachable. Is there some character in the code that is actually reason for this?
Do not use return
When you use return control of execution will leave the function and all code after the return statement will not be reachable
code after return will be unreachable
def foo: Int = {
return 1
2 + 3 //unreachable
}
In case of if expression
def bar: Int = {
if (true) {
return 1
} else {
return 2
}
1 + 2 //unreachable
}
In Scala return statement is optional and not recommended as its not consider functional coding practice.
The value of last expression in the code block is the return value in Scala. So don't worry about explicit return just leave it to the program
val flag = if (x.toString.split("/").last.split("_").last.toLong < System.currentTimeMillis) {
println("considered " + x)
LIST += x.toString
true //removed return
} else {
println("NOT considered " + x)
false // removed return
}
Halting the program executing by throwing an exception or by returning value or by explicitly calling exit is not functional way of doing things. Unfortunately Scala does allow it. But if you want to be a good citizen of functional world. You better avoid it.
Avoid mutable collections
Use mutable collections if you have a strong need for it. There are advantages of immutable collections
1) They are thread safe.
2) Bug free (no surprises by accidental mutations and no blocking).
3) Referential transparency.
4) Reasonable performance.
Use immutable list instead of mutable list.
Use Scala lambda notation and Syntactic sugar
Syntactic sugar is there for a reason. Syntactic sugar reduces the boilerplate code. So that your code looks clear, cleaner and better. Helps in code maintainability. Code remains bug free for longer time.
Instead of Function1 trait use lambda notation.
scala> val f = new Function1[String, String] { def apply(str: String): String = str}
f: String => String = <function1>
scala> val f = {str: String => str}
f: String => String = <function1>
So your code becomes
val paths = List[Path]() //assume you have Paths in it.
val filter = {path: Path => path.toString.split("/").last.split("_").last.toLong < System.currentTimeMillis }
val resultList = paths.filter(filter)
This is caused the flag is val not def, but your statement is using the return to return true or false. the return keywords is only for method not for function.
The correct way maybe like:
val flag = if (x.toString.split("/").last.split("_").last.toLong < System.currentTimeMillis) {
println("considered " + x)
LIST += x.toString
true
}
else {
println("NOT considered " + x)
false
}
Much like this question:
Functional code for looping with early exit
Say the code is
def findFirst[T](objects: List[T]):T = {
for (obj <- objects) {
if (expensiveFunc(obj) != null) return /*???*/ Some(obj)
}
None
}
How to yield a single element from a for loop like this in scala?
I do not want to use find, as proposed in the original question, i am curious about if and how it could be implemented using the for loop.
* UPDATE *
First, thanks for all the comments, but i guess i was not clear in the question. I am shooting for something like this:
val seven = for {
x <- 1 to 10
if x == 7
} return x
And that does not compile. The two errors are:
- return outside method definition
- method main has return statement; needs result type
I know find() would be better in this case, i am just learning and exploring the language. And in a more complex case with several iterators, i think finding with for can actually be usefull.
Thanks commenters, i'll start a bounty to make up for the bad posing of the question :)
If you want to use a for loop, which uses a nicer syntax than chained invocations of .find, .filter, etc., there is a neat trick. Instead of iterating over strict collections like list, iterate over lazy ones like iterators or streams. If you're starting with a strict collection, make it lazy with, e.g. .toIterator.
Let's see an example.
First let's define a "noisy" int, that will show us when it is invoked
def noisyInt(i : Int) = () => { println("Getting %d!".format(i)); i }
Now let's fill a list with some of these:
val l = List(1, 2, 3, 4).map(noisyInt)
We want to look for the first element which is even.
val r1 = for(e <- l; val v = e() ; if v % 2 == 0) yield v
The above line results in:
Getting 1!
Getting 2!
Getting 3!
Getting 4!
r1: List[Int] = List(2, 4)
...meaning that all elements were accessed. That makes sense, given that the resulting list contains all even numbers. Let's iterate over an iterator this time:
val r2 = (for(e <- l.toIterator; val v = e() ; if v % 2 == 0) yield v)
This results in:
Getting 1!
Getting 2!
r2: Iterator[Int] = non-empty iterator
Notice that the loop was executed only up to the point were it could figure out whether the result was an empty or non-empty iterator.
To get the first result, you can now simply call r2.next.
If you want a result of an Option type, use:
if(r2.hasNext) Some(r2.next) else None
Edit Your second example in this encoding is just:
val seven = (for {
x <- (1 to 10).toIterator
if x == 7
} yield x).next
...of course, you should be sure that there is always at least a solution if you're going to use .next. Alternatively, use headOption, defined for all Traversables, to get an Option[Int].
You can turn your list into a stream, so that any filters that the for-loop contains are only evaluated on-demand. However, yielding from the stream will always return a stream, and what you want is I suppose an option, so, as a final step you can check whether the resulting stream has at least one element, and return its head as a option. The headOption function does exactly that.
def findFirst[T](objects: List[T], expensiveFunc: T => Boolean): Option[T] =
(for (obj <- objects.toStream if expensiveFunc(obj)) yield obj).headOption
Why not do exactly what you sketched above, that is, return from the loop early? If you are interested in what Scala actually does under the hood, run your code with -print. Scala desugares the loop into a foreach and then uses an exception to leave the foreach prematurely.
So what you are trying to do is to break out a loop after your condition is satisfied. Answer here might be what you are looking for. How do I break out of a loop in Scala?.
Overall, for comprehension in Scala is translated into map, flatmap and filter operations. So it will not be possible to break out of these functions unless you throw an exception.
If you are wondering, this is how find is implemented in LineerSeqOptimized.scala; which List inherits
override /*IterableLike*/
def find(p: A => Boolean): Option[A] = {
var these = this
while (!these.isEmpty) {
if (p(these.head)) return Some(these.head)
these = these.tail
}
None
}
This is a horrible hack. But it would get you the result you wished for.
Idiomatically you'd use a Stream or View and just compute the parts you need.
def findFirst[T](objects: List[T]): T = {
def expensiveFunc(o : T) = // unclear what should be returned here
case class MissusedException(val data: T) extends Exception
try {
(for (obj <- objects) {
if (expensiveFunc(obj) != null) throw new MissusedException(obj)
})
objects.head // T must be returned from loop, dummy
} catch {
case MissusedException(obj) => obj
}
}
Why not something like
object Main {
def main(args: Array[String]): Unit = {
val seven = (for (
x <- 1 to 10
if x == 7
) yield x).headOption
}
}
Variable seven will be an Option holding Some(value) if value satisfies condition
I hope to help you.
I think ... no 'return' impl.
object TakeWhileLoop extends App {
println("first non-null: " + func(Seq(null, null, "x", "y", "z")))
def func[T](seq: Seq[T]): T = if (seq.isEmpty) null.asInstanceOf[T] else
seq(seq.takeWhile(_ == null).size)
}
object OptionLoop extends App {
println("first non-null: " + func(Seq(null, null, "x", "y", "z")))
def func[T](seq: Seq[T], index: Int = 0): T = if (seq.isEmpty) null.asInstanceOf[T] else
Option(seq(index)) getOrElse func(seq, index + 1)
}
object WhileLoop extends App {
println("first non-null: " + func(Seq(null, null, "x", "y", "z")))
def func[T](seq: Seq[T]): T = if (seq.isEmpty) null.asInstanceOf[T] else {
var i = 0
def obj = seq(i)
while (obj == null)
i += 1
obj
}
}
objects iterator filter { obj => (expensiveFunc(obj) != null } next
The trick is to get some lazy evaluated view on the colelction, either an iterator or a Stream, or objects.view. The filter will only execute as far as needed.
Suppose I would like to code the following logic in Scala
val xdir = System.getProperty("XDir")
if (xdir == null)
error("No XDir") // log the error and exit
val ydir = System.getProperty("YDir")
if (ydir == null)
error("No YDir")
if (!new File(xdir).isDirectory)
error("XDir is not a directory")
if (!new File(ydir).isDirectory)
error("YDir is not a directory")
if (!new File(xdir).exists)
error("XDir does not exis")
if (!new File(ydir).exists)
error("YDir does not exist")
...
(and so on)
What is the best way to code this chain of validations in Scala?
Here's some useful things:
def sysValue(prop: String) = Option(System.getProperty(prop)) //returns Option[String]
def trySysValue(prop: String) = //returns Either[String, String]
sysValue(prop) map Right getOrElse Left("Absent property: " + prop)
Then you can use monadic composition of Either through its right-projection
val batch = //batch is Either[String, (File, File)]
for {
x <- trySysValue("XDir")).right
xf <- dir(x).right
y <- trySysValue("YDir").right
yf <- dir(y).right
}
yield (xf, yf)
Where:
def dir(s: String) = { //returns Either[String, File]
val f = new File(s)
if (!f.exists()) Left("Does not exist: " + f)
else if (!f.isDir()) Left("Is not a directory: " + f)
else Right(f)
}
The left-hand-side of the Either will be an error message. This monadic composition is fail fast. You can achieve composition which will accumulate all failures (for example, if neither XDir nor YDir exist, you would see both messages) using scalaz Validation. In that case, the code would look like this:
def trySysValue(prop: String) = //returns Validation[String, String]
sysValue(prop) map Success getOrElse ("Absent property: " + prop).fail
def dir(s: String) = {
val f = new File(s)
if (!f.exists())("Does not exist: " + f).fail
else if (!f.isDir()) ("Is not a directory: " + f).fail
else f.success
}
val batch = //batch is ValidationNEL[String, (File, File)]
(trySysValue("XDir")) flatMap dir).liftFailNel <|*|> (trySysValue("YDir")) flatMap dir).liftFailNel
something like:
val batch = for{
a <- safe(doA, "A failed") either
b <- safe(doB, "B failed") either
c <- safe(doC, "C failed") either
} yield(a,b,c)
batch fold( error(_), doSuccess(_) )
Where safe performs a, you guessed it, safe (try/catch) operation that takes a failure (Left outcome) message and returns an Either RightProjection (which allows you to do above batch operation while threading through the point-of-failure error message)
class Catching[T](f: => T) {
def either(msg: String) = {
try { Right(f).right } catch { Left(msg).right }
}
}
def safe[T](f: => T) = new Catching(f)
Can add an option method to Catching class as well, along with logging if you want to log particular error types.
See Jason Zaugg's solution for right biasing Either and this thread from scala-debate on the subject as well. No consensus as yet, but most scala "heavies" seem to be in favor.
One limitation of this approach is that if you attempt to add conditionals (if a = b) to the for{} block, it won't compile (since default Either filter method returns Option). The workaround is to implement filter and withFilter, returning Either, something I have yet to figure out/do (if someone has already done so, please post)
Yes you can use validation without scalaz, see here for a self containt implementation :
http://applicative-errors-scala.googlecode.com/svn/artifacts/0.6/chunk-xhtml/apa.html
HTH
Disclaimer: Before someone says it: yes, I know it's bad style and not encouraged. I'm just doing this to play with Scala and try to learn more about how the type inference system works and how to tweak control flow. I don't intend to use this code in practice.
So: suppose I'm in a rather lengthy function, with lots of successive checks at the beginning, which, if they fail, are all supposed to cause the function to return some other value (not throw), and otherwise return the normal value. I cannot use return in the body of a Function. But can I simulate it? A bit like break is simulated in scala.util.control.Breaks?
I have come up with this:
object TestMain {
case class EarlyReturnThrowable[T](val thrower: EarlyReturn[T], val value: T) extends ControlThrowable
class EarlyReturn[T] {
def earlyReturn(value: T): Nothing = throw new EarlyReturnThrowable[T](this, value)
}
def withEarlyReturn[U](work: EarlyReturn[U] => U): U = {
val myThrower = new EarlyReturn[U]
try work(myThrower)
catch {
case EarlyReturnThrowable(`myThrower`, value) => value.asInstanceOf[U]
}
}
def main(args: Array[String]) {
val g = withEarlyReturn[Int] { block =>
if (!someCondition)
block.earlyReturn(4)
val foo = precomputeSomething
if (!someOtherCondition(foo))
block.earlyReturn(5)
val bar = normalize(foo)
if (!checkBar(bar))
block.earlyReturn(6)
val baz = bazify(bar)
if (!baz.isOK)
block.earlyReturn(7)
// now the actual, interesting part of the computation happens here
// and I would like to keep it non-nested as it is here
foo + bar + baz + 42 // just a dummy here, but in practice this is longer
}
println(g)
}
}
My checks here are obviously dummy, but the main point is that I'd like to avoid something like this, where the actually interesting code ends up being way too nested for my taste:
if (!someCondition) 4 else {
val foo = precomputeSomething
if (!someOtherCondition(foo)) 5 else {
val bar = normalize(foo)
if (!checkBar(bar)) 6 else {
val baz = bazify(bar)
if (!baz.isOK) 7 else {
// actual computation
foo + bar + baz + 42
}
}
}
}
My solution works fine here, and I can return early with 4 as return value if I want. Trouble is, I have to explicitly write the type parameter [Int] — which is a bit of a pain. Is there any way I can get around this?
It's a bit unrelated to your main question, but I think, a more effective approach (that doesn't require throwing an exception) to implement return would involve continuations:
def earlyReturn[T](ret: T): Any #cpsParam[Any, Any] = shift((k: Any => Any) => ret)
def withEarlyReturn[T](f: => T #cpsParam[T, T]): T = reset(f)
def cpsunit: Unit #cps[Any] = ()
def compute(bool: Boolean) = {
val g = withEarlyReturn {
val a = 1
if(bool) earlyReturn(4) else cpsunit
val b = 1
earlyReturn2(4, bool)
val c = 1
if(bool) earlyReturn(4) else cpsunit
a + b + c + 42
}
println(g)
}
The only problem here, is that you have to explicitly use cpsunit.
EDIT1: Yes, earlyReturn(4, cond = !checkOK) can be implemented, but it won't be that general and elegant:
def earlyReturn2[T](ret: T, cond: => Boolean): Any #cpsParam[Any, Any] =
shift((k: Any => Any) => if(cond) ret else k())
k in the snippet above represents the rest of the computation. Depending on the value of cond, we either return the value, or continue the computation.
EDIT2: Any chance we might get rid of cpsunit? The problem here is that shift inside the if statement is not allowed without else. The compiler refuses to convert Unit to Unit #cps[Unit].
I think a custom exception is the right instinct here.