Suppose I would like to code the following logic in Scala
val xdir = System.getProperty("XDir")
if (xdir == null)
error("No XDir") // log the error and exit
val ydir = System.getProperty("YDir")
if (ydir == null)
error("No YDir")
if (!new File(xdir).isDirectory)
error("XDir is not a directory")
if (!new File(ydir).isDirectory)
error("YDir is not a directory")
if (!new File(xdir).exists)
error("XDir does not exis")
if (!new File(ydir).exists)
error("YDir does not exist")
...
(and so on)
What is the best way to code this chain of validations in Scala?
Here's some useful things:
def sysValue(prop: String) = Option(System.getProperty(prop)) //returns Option[String]
def trySysValue(prop: String) = //returns Either[String, String]
sysValue(prop) map Right getOrElse Left("Absent property: " + prop)
Then you can use monadic composition of Either through its right-projection
val batch = //batch is Either[String, (File, File)]
for {
x <- trySysValue("XDir")).right
xf <- dir(x).right
y <- trySysValue("YDir").right
yf <- dir(y).right
}
yield (xf, yf)
Where:
def dir(s: String) = { //returns Either[String, File]
val f = new File(s)
if (!f.exists()) Left("Does not exist: " + f)
else if (!f.isDir()) Left("Is not a directory: " + f)
else Right(f)
}
The left-hand-side of the Either will be an error message. This monadic composition is fail fast. You can achieve composition which will accumulate all failures (for example, if neither XDir nor YDir exist, you would see both messages) using scalaz Validation. In that case, the code would look like this:
def trySysValue(prop: String) = //returns Validation[String, String]
sysValue(prop) map Success getOrElse ("Absent property: " + prop).fail
def dir(s: String) = {
val f = new File(s)
if (!f.exists())("Does not exist: " + f).fail
else if (!f.isDir()) ("Is not a directory: " + f).fail
else f.success
}
val batch = //batch is ValidationNEL[String, (File, File)]
(trySysValue("XDir")) flatMap dir).liftFailNel <|*|> (trySysValue("YDir")) flatMap dir).liftFailNel
something like:
val batch = for{
a <- safe(doA, "A failed") either
b <- safe(doB, "B failed") either
c <- safe(doC, "C failed") either
} yield(a,b,c)
batch fold( error(_), doSuccess(_) )
Where safe performs a, you guessed it, safe (try/catch) operation that takes a failure (Left outcome) message and returns an Either RightProjection (which allows you to do above batch operation while threading through the point-of-failure error message)
class Catching[T](f: => T) {
def either(msg: String) = {
try { Right(f).right } catch { Left(msg).right }
}
}
def safe[T](f: => T) = new Catching(f)
Can add an option method to Catching class as well, along with logging if you want to log particular error types.
See Jason Zaugg's solution for right biasing Either and this thread from scala-debate on the subject as well. No consensus as yet, but most scala "heavies" seem to be in favor.
One limitation of this approach is that if you attempt to add conditionals (if a = b) to the for{} block, it won't compile (since default Either filter method returns Option). The workaround is to implement filter and withFilter, returning Either, something I have yet to figure out/do (if someone has already done so, please post)
Yes you can use validation without scalaz, see here for a self containt implementation :
http://applicative-errors-scala.googlecode.com/svn/artifacts/0.6/chunk-xhtml/apa.html
HTH
Related
I'm following the tutorial from Alvin Alexander to use Loan Pattern
Here is the code what I use -
val year = 2016
val nationalData = {
val source = io.Source.fromFile(s"resources/Babynames/names/yob$year.txt")
// names is iterator of String, split() gives the array
//.toArray & toSeq is a slow process compare to .toSet // .toSeq gives Stream Closed error
val names = source.getLines().filter(_.nonEmpty).map(_.split(",")(0)).toSet
source.close()
names
// println(names.mkString(","))
}
println("Names " + nationalData)
val info = for (stateFile <- new java.io.File("resources/Babynames/namesbystate").list(); if stateFile.endsWith(".TXT")) yield {
val source = io.Source.fromFile("resources/Babynames/namesbystate/" + stateFile)
val names = source.getLines().filter(_.nonEmpty).map(_.split(",")).
filter(a => a(2).toInt == year).map(a => a(3)).toArray // .toSet
source.close()
(stateFile.take(2), names)
}
println(info(0)._2.size + " names from state "+ info(0)._1)
println(info(1)._2.size + " names from state "+ info(1)._1)
for ((state, sname) <- info) {
println("State: " +state + " Coverage of name in "+ year+" "+ sname.count(n => nationalData.contains(n)).toDouble / nationalData.size) // Set doesn't have length method
}
This is how I applied readTextFile, readTextFileWithTry on the above code to learn/experiment Loan Pattern in the above code
def using[A <: { def close(): Unit }, B](resource: A)(f: A => B): B =
try {
f(resource)
} finally {
resource.close()
}
def readTextFile(filename: String): Option[List[String]] = {
try {
val lines = using(fromFile(filename)) { source =>
(for (line <- source.getLines) yield line).toList
}
Some(lines)
} catch {
case e: Exception => None
}
}
def readTextFileWithTry(filename: String): Try[List[String]] = {
Try {
val lines = using(fromFile(filename)) { source =>
(for (line <- source.getLines) yield line).toList
}
lines
}
}
val year = 2016
val data = readTextFile(s"resources/Babynames/names/yob$year.txt") match {
case Some(lines) =>
val n = lines.filter(_.nonEmpty).map(_.split(",")(0)).toSet
println(n)
case None => println("couldn't read file")
}
val data1 = readTextFileWithTry("resources/Babynames/namesbystate")
data1 match {
case Success(lines) => {
val info = for (stateFile <- data1; if stateFile.endsWith(".TXT")) yield {
val source = fromFile("resources/Babynames/namesbystate/" + stateFile)
val names = source.getLines().filter(_.nonEmpty).map(_.split(",")).
filter(a => a(2).toInt == year).map(a => a(3)).toArray // .toSet
(stateFile.take(2), names)
println(names)
}
}
But in the second case, readTextFileWithTry, I am getting the following error -
Failed, message is: java.io.FileNotFoundException: resources\Babynames\namesbystate (Access is denied)
I guess the reason for the failure is from SO what I understand -
I am trying to open the same file on each iteration of the for loop
Apart from that, I have few concerns regarding how I use -
Is it the good way to use? Can some help me how can I use the TRY on multiple occasions?
I tried to change the return type of readTextFileWithTry like Option[A] or Set/Map or Scala Collection to apply higher-order functions later on that. but not able to succeed. Not sure that is a good practice or not.
How can I use higher-order functions in Success case, as there are multiple operations and in Success case the code blocks get bigger? I can't use any field outside of Success case.
Can someone help me to understand?
I think that you problem has nothing to do with "I am trying to open the same file on each iteration of the for loop" and it is actually the same as in the accepted answer
Unfortunately you didn't provide stack trace so it is not clear on which line this happens. I would guess that the falling call is
val data1 = readTextFileWithTry("resources/Babynames/namesbystate")
And looking at your first code sample:
val info = for (stateFile <- new java.io.File("resources/Babynames/namesbystate").list(); if stateFile.endsWith(".TXT")) yield {
it looks like the path "resources/Babynames/namesbystate" points to a directory. But in your second example you are trying to read it as a file and this is the reason for the error. It comes from the fact that your readTextFileWithTry is not a valid substitute for java.io.File.list call. And File.list doesn't need a wrapper because it doesn't use any intermediate closeable/disposable entity.
P.S. it might make more sense to use File.list(FilenameFilter filter) instead of if stateFile.endsWith(".TXT"))
I wrote this simple program in my attempt to learn how Cats Writer works
import cats.data.Writer
import cats.syntax.applicative._
import cats.syntax.writer._
import cats.instances.vector._
object WriterTest extends App {
type Logged2[A] = Writer[Vector[String], A]
Vector("started the program").tell
val output1 = calculate1(10)
val foo = new Foo()
val output2 = foo.calculate2(20)
val (log, sum) = (output1 + output2).pure[Logged2].run
println(log)
println(sum)
def calculate1(x : Int) : Int = {
Vector("came inside calculate1").tell
val output = 10 + x
Vector(s"Calculated value ${output}").tell
output
}
}
class Foo {
def calculate2(x: Int) : Int = {
Vector("came inside calculate 2").tell
val output = 10 + x
Vector(s"calculated ${output}").tell
output
}
}
The program works and the output is
> run-main WriterTest
[info] Compiling 1 Scala source to /Users/Cats/target/scala-2.11/classes...
[info] Running WriterTest
Vector()
50
[success] Total time: 1 s, completed Jan 21, 2017 8:14:19 AM
But why is the vector empty? Shouldn't it contain all the strings on which I used the "tell" method?
When you call tell on your Vectors, each time you create a Writer[Vector[String], Unit]. However, you never actually do anything with your Writers, you just discard them. Further, you call pure to create your final Writer, which simply creates a Writer with an empty Vector. You have to combine the writers together in a chain that carries your value and message around.
type Logged[A] = Writer[Vector[String], A]
val (log, sum) = (for {
_ <- Vector("started the program").tell
output1 <- calculate1(10)
foo = new Foo()
output2 <- foo.calculate2(20)
} yield output1 + output2).run
def calculate1(x: Int): Logged[Int] = for {
_ <- Vector("came inside calculate1").tell
output = 10 + x
_ <- Vector(s"Calculated value ${output}").tell
} yield output
class Foo {
def calculate2(x: Int): Logged[Int] = for {
_ <- Vector("came inside calculate2").tell
output = 10 + x
_ <- Vector(s"calculated ${output}").tell
} yield output
}
Note the use of for notation. The definition of calculate1 is really
def calculate1(x: Int): Logged[Int] = Vector("came inside calculate1").tell.flatMap { _ =>
val output = 10 + x
Vector(s"calculated ${output}").tell.map { _ => output }
}
flatMap is the monadic bind operation, which means it understands how to take two monadic values (in this case Writer) and join them together to get a new one. In this case, it makes a Writer containing the concatenation of the logs and the value of the one on the right.
Note how there are no side effects. There is no global state by which Writer can remember all your calls to tell. You instead make many Writers and join them together with flatMap to get one big one at the end.
The problem with your example code is that you're not using the result of the tell method.
If you take a look at its signature, you'll see this:
final class WriterIdSyntax[A](val a: A) extends AnyVal {
def tell: Writer[A, Unit] = Writer(a, ())
}
it is clear that tell returns a Writer[A, Unit] result which is immediately discarded because you didn't assign it to a value.
The proper way to use a Writer (and any monad in Scala) is through its flatMap method. It would look similar to this:
println(
Vector("started the program").tell.flatMap { _ =>
15.pure[Logged2].flatMap { i =>
Writer(Vector("ended program"), i)
}
}
)
The code above, when executed will give you this:
WriterT((Vector(started the program, ended program),15))
As you can see, both messages and the int are stored in the result.
Now this is a bit ugly, and Scala actually provides a better way to do this: for-comprehensions. For-comprehension are a bit of syntactic sugar that allows us to write the same code in this way:
println(
for {
_ <- Vector("started the program").tell
i <- 15.pure[Logged2]
_ <- Vector("ended program").tell
} yield i
)
Now going back to your example, what I would recommend is for you to change the return type of compute1 and compute2 to be Writer[Vector[String], Int] and then try to make your application compile using what I wrote above.
While dealing with error-handling in Scala, I came to the point where I asked myself, whether Trys in a for-comprehension make sense.
Please regard the unit test given below. This test shows two approaches:
The first approach (with call) embeds the methods fooA and fooB, which return regular Strings, into a Try construct.
The second approach (tryCall) uses a for-comprehension that uses methods tryFooA and tryFooB, which return Try[String] each.
For what reason should one prefer the for-comprehension variant with tryCall over the call-variant?
test("Stackoverflow post: Try and for-comprehensions.") {
val iae = new IllegalArgumentException("IAE")
val rt = new RuntimeException("RT")
import scala.util.{Try, Success, Failure}
def fooA(x1: Int) : String = {
println("fooA")
if (x1 == 1) "x1 is 1" else throw iae
}
def fooB(x2: Int) : String = {
println("fooB")
if (x2 == 1) "x2 is 1" else throw rt
}
def tryFooA(x1: Int) : Try[String] = {
Try {
println("tryFooA")
if (x1 == 1) "x1 is 1" else throw iae
}
}
def tryFooB(x2: Int) : Try[String] = {
Try {
println("tryFooB")
if (x2 == 1) "x2 is 1" else throw rt
}
}
def call( x1: Int, x2: Int ) : Try[String] = {
val res: Try[String] = Try{
val a = fooA(x1)
val b = fooB(x2)
a + " " + b
}
res
}
def tryCall( x1: Int, x2: Int ): Try[String] = {
for {
a <- tryFooA(x1)
b <- tryFooB(x2)
} yield (a + " " + b)
}
assert( call(0,0) === tryCall(0,0))
assert( call(0,1) === tryCall(0,1))
assert( call(1,0) === tryCall(1,0))
assert( call(1,1) === tryCall(1,1))
}
The purpose of Try is to let the compiler help you (and anyone else who uses your code) handle and reason about errors more responsibly.
In your examples, the behavior of call and tryCall are more or less identical, and if the fooA, etc. methods were not part of any public API, there'd be little reason to prefer one over the other.
The advantage of Try is that it lets you compose operations that can fail in a clear, concise way. The signatures of tryFooA and tryFooB are upfront about the fact that they may result in (recoverable) failure, and the compiler makes sure that anyone who calls those methods must deal with that possibility. The signatures of fooA and fooB aren't self-documenting in this way, and the compiler can't provide any assurances that they'll be called responsibly.
These are good reasons to prefer the tryFoo signatures. The downside is that callers have to deal with the extra overhead of using map, flatMap, etc., instead of working with the results (if they exist) directly. The for-comprehension syntax is an attempt to minimize this overhead—you get the safety provided by Try with only a little extra syntactic overhead.
Much like this question:
Functional code for looping with early exit
Say the code is
def findFirst[T](objects: List[T]):T = {
for (obj <- objects) {
if (expensiveFunc(obj) != null) return /*???*/ Some(obj)
}
None
}
How to yield a single element from a for loop like this in scala?
I do not want to use find, as proposed in the original question, i am curious about if and how it could be implemented using the for loop.
* UPDATE *
First, thanks for all the comments, but i guess i was not clear in the question. I am shooting for something like this:
val seven = for {
x <- 1 to 10
if x == 7
} return x
And that does not compile. The two errors are:
- return outside method definition
- method main has return statement; needs result type
I know find() would be better in this case, i am just learning and exploring the language. And in a more complex case with several iterators, i think finding with for can actually be usefull.
Thanks commenters, i'll start a bounty to make up for the bad posing of the question :)
If you want to use a for loop, which uses a nicer syntax than chained invocations of .find, .filter, etc., there is a neat trick. Instead of iterating over strict collections like list, iterate over lazy ones like iterators or streams. If you're starting with a strict collection, make it lazy with, e.g. .toIterator.
Let's see an example.
First let's define a "noisy" int, that will show us when it is invoked
def noisyInt(i : Int) = () => { println("Getting %d!".format(i)); i }
Now let's fill a list with some of these:
val l = List(1, 2, 3, 4).map(noisyInt)
We want to look for the first element which is even.
val r1 = for(e <- l; val v = e() ; if v % 2 == 0) yield v
The above line results in:
Getting 1!
Getting 2!
Getting 3!
Getting 4!
r1: List[Int] = List(2, 4)
...meaning that all elements were accessed. That makes sense, given that the resulting list contains all even numbers. Let's iterate over an iterator this time:
val r2 = (for(e <- l.toIterator; val v = e() ; if v % 2 == 0) yield v)
This results in:
Getting 1!
Getting 2!
r2: Iterator[Int] = non-empty iterator
Notice that the loop was executed only up to the point were it could figure out whether the result was an empty or non-empty iterator.
To get the first result, you can now simply call r2.next.
If you want a result of an Option type, use:
if(r2.hasNext) Some(r2.next) else None
Edit Your second example in this encoding is just:
val seven = (for {
x <- (1 to 10).toIterator
if x == 7
} yield x).next
...of course, you should be sure that there is always at least a solution if you're going to use .next. Alternatively, use headOption, defined for all Traversables, to get an Option[Int].
You can turn your list into a stream, so that any filters that the for-loop contains are only evaluated on-demand. However, yielding from the stream will always return a stream, and what you want is I suppose an option, so, as a final step you can check whether the resulting stream has at least one element, and return its head as a option. The headOption function does exactly that.
def findFirst[T](objects: List[T], expensiveFunc: T => Boolean): Option[T] =
(for (obj <- objects.toStream if expensiveFunc(obj)) yield obj).headOption
Why not do exactly what you sketched above, that is, return from the loop early? If you are interested in what Scala actually does under the hood, run your code with -print. Scala desugares the loop into a foreach and then uses an exception to leave the foreach prematurely.
So what you are trying to do is to break out a loop after your condition is satisfied. Answer here might be what you are looking for. How do I break out of a loop in Scala?.
Overall, for comprehension in Scala is translated into map, flatmap and filter operations. So it will not be possible to break out of these functions unless you throw an exception.
If you are wondering, this is how find is implemented in LineerSeqOptimized.scala; which List inherits
override /*IterableLike*/
def find(p: A => Boolean): Option[A] = {
var these = this
while (!these.isEmpty) {
if (p(these.head)) return Some(these.head)
these = these.tail
}
None
}
This is a horrible hack. But it would get you the result you wished for.
Idiomatically you'd use a Stream or View and just compute the parts you need.
def findFirst[T](objects: List[T]): T = {
def expensiveFunc(o : T) = // unclear what should be returned here
case class MissusedException(val data: T) extends Exception
try {
(for (obj <- objects) {
if (expensiveFunc(obj) != null) throw new MissusedException(obj)
})
objects.head // T must be returned from loop, dummy
} catch {
case MissusedException(obj) => obj
}
}
Why not something like
object Main {
def main(args: Array[String]): Unit = {
val seven = (for (
x <- 1 to 10
if x == 7
) yield x).headOption
}
}
Variable seven will be an Option holding Some(value) if value satisfies condition
I hope to help you.
I think ... no 'return' impl.
object TakeWhileLoop extends App {
println("first non-null: " + func(Seq(null, null, "x", "y", "z")))
def func[T](seq: Seq[T]): T = if (seq.isEmpty) null.asInstanceOf[T] else
seq(seq.takeWhile(_ == null).size)
}
object OptionLoop extends App {
println("first non-null: " + func(Seq(null, null, "x", "y", "z")))
def func[T](seq: Seq[T], index: Int = 0): T = if (seq.isEmpty) null.asInstanceOf[T] else
Option(seq(index)) getOrElse func(seq, index + 1)
}
object WhileLoop extends App {
println("first non-null: " + func(Seq(null, null, "x", "y", "z")))
def func[T](seq: Seq[T]): T = if (seq.isEmpty) null.asInstanceOf[T] else {
var i = 0
def obj = seq(i)
while (obj == null)
i += 1
obj
}
}
objects iterator filter { obj => (expensiveFunc(obj) != null } next
The trick is to get some lazy evaluated view on the colelction, either an iterator or a Stream, or objects.view. The filter will only execute as far as needed.
Option monad is a great expressive way to deal with something-or-nothing things in Scala. But what if one needs to log a message when "nothing" occurs? According to the Scala API documentation,
The Either type is often used as an
alternative to scala.Option where Left
represents failure (by convention) and
Right is akin to Some.
However, I had no luck to find best practices using Either or good real-world examples involving Either for processing failures. Finally I've come up with the following code for my own project:
def logs: Array[String] = {
def props: Option[Map[String, Any]] = configAdmin.map{ ca =>
val config = ca.getConfiguration(PID, null)
config.properties getOrElse immutable.Map.empty
}
def checkType(any: Any): Option[Array[String]] = any match {
case a: Array[String] => Some(a)
case _ => None
}
def lookup: Either[(Symbol, String), Array[String]] =
for {val properties <- props.toRight('warning -> "ConfigurationAdmin service not bound").right
val logsParam <- properties.get("logs").toRight('debug -> "'logs' not defined in the configuration").right
val array <- checkType(logsParam).toRight('warning -> "unknown type of 'logs' confguration parameter").right}
yield array
lookup.fold(failure => { failure match {
case ('warning, msg) => log(LogService.WARNING, msg)
case ('debug, msg) => log(LogService.DEBUG, msg)
case _ =>
}; new Array[String](0) }, success => success)
}
(Please note this is a snippet from a real project, so it will not compile on its own)
I'd be grateful to know how you are using Either in your code and/or better ideas on refactoring the above code.
Either is used to return one of possible two meaningful results, unlike Option which is used to return a single meaningful result or nothing.
An easy to understand example is given below (circulated on the Scala mailing list a while back):
def throwableToLeft[T](block: => T): Either[java.lang.Throwable, T] =
try {
Right(block)
} catch {
case ex => Left(ex)
}
As the function name implies, if the execution of "block" is successful, it will return "Right(<result>)". Otherwise, if a Throwable is thrown, it will return "Left(<throwable>)". Use pattern matching to process the result:
var s = "hello"
throwableToLeft { s.toUpperCase } match {
case Right(s) => println(s)
case Left(e) => e.printStackTrace
}
// prints "HELLO"
s = null
throwableToLeft { s.toUpperCase } match {
case Right(s) => println(s)
case Left(e) => e.printStackTrace
}
// prints NullPointerException stack trace
Hope that helps.
Scalaz library has something alike Either named Validation. It is more idiomatic than Either for use as "get either a valid result or a failure".
Validation also allows to accumulate errors.
Edit: "alike" Either is complettly false, because Validation is an applicative functor, and scalaz Either, named \/ (pronounced "disjonction" or "either"), is a monad.
The fact that Validation can accumalate errors is because of that nature. On the other hand, / has a "stop early" nature, stopping at the first -\/ (read it "left", or "error") it encounters. There is a perfect explanation here: http://typelevel.org/blog/2014/02/21/error-handling.html
See: http://scalaz.googlecode.com/svn/continuous/latest/browse.sxr/scalaz/example/ExampleValidation.scala.html
As requested by the comment, copy/paste of the above link (some lines removed):
// Extracting success or failure values
val s: Validation[String, Int] = 1.success
val f: Validation[String, Int] = "error".fail
// It is recommended to use fold rather than pattern matching:
val result: String = s.fold(e => "got error: " + e, s => "got success: " + s.toString)
s match {
case Success(a) => "success"
case Failure(e) => "fail"
}
// Validation is a Monad, and can be used in for comprehensions.
val k1 = for {
i <- s
j <- s
} yield i + j
k1.toOption assert_≟ Some(2)
// The first failing sub-computation fails the entire computation.
val k2 = for {
i <- f
j <- f
} yield i + j
k2.fail.toOption assert_≟ Some("error")
// Validation is also an Applicative Functor, if the type of the error side of the validation is a Semigroup.
// A number of computations are tried. If the all success, a function can combine them into a Success. If any
// of them fails, the individual errors are accumulated.
// Use the NonEmptyList semigroup to accumulate errors using the Validation Applicative Functor.
val k4 = (fNel <**> fNel){ _ + _ }
k4.fail.toOption assert_≟ some(nel1("error", "error"))
The snippet you posted seems very contrived. You use Either in a situation where:
It's not enough to just know the data isn't available.
You need to return one of two distinct types.
Turning an exception into a Left is, indeed, a common use case. Over try/catch, it has the advantage of keeping the code together, which makes sense if the exception is an expected result. The most common way of handling Either is pattern matching:
result match {
case Right(res) => ...
case Left(res) => ...
}
Another interesting way of handling Either is when it appears in a collection. When doing a map over a collection, throwing an exception might not be viable, and you may want to return some information other than "not possible". Using an Either enables you to do that without overburdening the algorithm:
val list = (
library
\\ "books"
map (book =>
if (book \ "author" isEmpty)
Left(book)
else
Right((book \ "author" toList) map (_ text))
)
)
Here we get a list of all authors in the library, plus a list of books without an author. So we can then further process it accordingly:
val authorCount = (
(Map[String,Int]() /: (list filter (_ isRight) map (_.right.get)))
((map, author) => map + (author -> (map.getOrElse(author, 0) + 1)))
toList
)
val problemBooks = list flatMap (_.left.toSeq) // thanks to Azarov for this variation
So, basic Either usage goes like that. It's not a particularly useful class, but if it were you'd have seen it before. On the other hand, it's not useless either.
Cats has a nice way to create an Either from exception-throwing code:
val either: Either[NumberFormatException, Int] =
Either.catchOnly[NumberFormatException]("abc".toInt)
// either: Either[NumberFormatException,Int] = Left(java.lang.NumberFormatException: For input string: "abc")
in https://typelevel.org/cats/datatypes/either.html#working-with-exception-y-code