scalaz, read and map the lines of a file - scala

Question

The following code to read and map the lines of a file works ok:
def readLines(fileName: String) = scala.io.Source.fromFile(fileName).getLines
def toInt(line: String) = line.toInt
val numbers: Iterator[Int] = readLines("/tmp/file.txt").map(toInt).map(_ * 2)
println(numbers.toList)
I get an iterator of Ints if the executing goes well. But the program throws an exception if the file is not found, or some line contains letters.
How can I transform the program to use scalaz monads and get a Disjunction[Exception, List[Int]]?
I tried this on scalaz 7.2.6, but it does not compile:
import scalaz.Scalaz._
import scalaz._
def readLines(fileName: String): Disjunction[Any, List[String]] =
try { scala.io.Source.fromFile(fileName).getLines.toList.right }
catch { case e: java.io.IOException => e.left}
def toInt(line: String): Disjunction[Any, Int] =
try { line.toInt.right }
catch { case e: NumberFormatException => e.left}
val numbers: Disjunction[Any, Int] = for {
lines: List[String] <- readLines("/tmp/file.txt")
line: String <- lines
n: Int <- toInt(line)
} yield (n * 2)
it fails to compile with these errors:
Error:(89, 37) could not find implicit value for parameter M: scalaz.Monoid[Any]
lines: List[String] <- readLines("/tmp/file.txt")
Error:(89, 37) not enough arguments for method filter: (implicit M: scalaz.Monoid[Any])scalaz.\/[Any,List[String]].
Unspecified value parameter M.
lines: List[String] <- readLines("/tmp/file.txt")
Error:(91, 20) could not find implicit value for parameter M: scalaz.Monoid[Any]
n: Int <- toInt(line)
Error:(91, 20) not enough arguments for method filter: (implicit M: scalaz.Monoid[Any])scalaz.\/[Any,Int].
Unspecified value parameter M.
n: Int <- toInt(line)
I don't understand the errors. what is the problem?
and how to improve this code, so that it does not read all the file into memory, but it reads and maps each line at a time?
Update: Answer from Filippo
import scalaz._
def readLines(fileName: String) = \/.fromTryCatchThrowable[List[String], Exception] {
scala.io.Source.fromFile(fileName).getLines.toList
}
def toInt(line: String) = \/.fromTryCatchThrowable[Int, NumberFormatException](line.toInt)
type λ[+A] = Exception \/ A
val numbers = for {
line: String <- ListT[λ, String](readLines("/tmp/file.txt"))
n: Int <- ListT[λ, Int](toInt(line).map(List(_)))
} yield n * 2
println(numbers)

Answer 1

To answer the second part of your question, I would simply use the Iterator out of the fromFile method:
val lines: Iterator[String] = scala.io.Source.fromFile(fileName).getLines
If you want to use toInt to convert String to Int:
import scala.util.Try
def toInt(line: String): Iterator[Int] =
Try(line.toInt).map(Iterator(_)).getOrElse(Iterator.empty)
Then numbers could look like:
val numbers = readLines("/tmp/file.txt").flatMap(toInt).map(_ * 2)
EDIT
Due the presence of all these try and catch, if you want to keep using that monadic-for I would suggest to check a scalaz helper like .fromTryCatchThrowable on Disjunction:
import scalaz._, Scalaz._
def readLines(fileName: String): Disjunction[Exception, List[String]] =
Disjunction.fromTryCatchThrowable(scala.io.Source.fromFile(fileName).getLines.toList)
def toInt(line: String): Disjunction[Exception, Int] =
Disjunction.fromTryCatchThrowable(line.toInt)
Now we also have Exception instead of Any as the left type.
val numbers = for {
lines: List[String] <- readLines("/tmp/file.txt")
line: String <- lines // The problem is here
n: Int <- toInt(line)
} yield n * 2
The problem with this monadic-for is that the first and third line are using the Disjunction context but the second one uses the List monad. Using a monad transformer like ListT or DisjunctionT here is possible but probably overkill.
EDIT - to reply the comment
As mentioned, if we want a single monadic-for comprehension, we need a monad transformer, in this case ListT. The Disjunction has two type parameters while a Monad M[_] obviously only one. We need to handle this "extra type parameter", for instance using type lambda:
def readLines(fileName: String) = \/.fromTryCatchThrowable[List[String], Exception] {
fromFile(fileName).getLines.toList
}
val listTLines = ListT[({type λ[+a] = Exception \/ a})#λ, String](readLines("/tmp/file.txt"))
What is the type of listTLines? The ListT transformer: ListT[\/[Exception, +?], String]
The last step in the original for-comprehension was toInt:
def toInt(line: String) = \/.fromTryCatchThrowable[Int, NumberFormatException](line.toInt)
val listTNumber = ListT[\/[Exception, +?], Int](toInt("line"))
What is the type of listTNumber? It doesn't even compile, because the toInt return an Int and not a List[Int]. We need a ListT to join that for-comprehension, one trick could be changing listTNumber to:
val listTNumber = ListT[\/[Exception, +?], Int](toInt("line").map(List(_)))
Now we have both steps:
val numbers = for {
line: String <- ListT[\/[Exception, +?], String](readLines("/tmp/file.txt"))
n: Int <- ListT[\/[Exception, +?], Int](toInt(line).map(List(_)))
} yield n * 2
scala> numbers.run.getOrElse(List.empty) foreach println
2
20
200
If you are wondering why all this unwrapping:
scala> val unwrap1 = numbers.run
unwrap1: scalaz.\/[Exception,List[Int]] = \/-(List(2, 20, 200))
scala> val unwrap2 = unwrap1.getOrElse(List())
unwrap2: List[Int] = List(2, 20, 200)
scala> unwrap2 foreach println
2
20
200
(assuming that the sample file contains the lines: 1, 10, 100)
EDIT - comment about compilation issues
The code above compiles thanks to the Kind Projector plugin:
addCompilerPlugin("org.spire-math" % "kind-projector_2.11" % "0.5.2")
With Kind Projector we can have anonymous types like:
Either[Int, +?] // equivalent to: type R[+A] = Either[Int, A]
Instead of:
type IntOrA[A] = Either[Int, A]
// or
({type L[A] = Either[Int, A]})#L

Answer 2

First, the compiler alerts that you´re using for comprehensions mixing types. Your code is transformed by the compiler as that :
readLines("/tmp/file.txt") flatMap { lines => lines } map { line => toInt(line) }
The definition of flatMap is:
def flatMap[A,B](ma: F[A])(f: A => F[B]): F[B]
In your case F is the \/, and this flatMap { lines => lines } is wrong. The compiler alerts with a message like this "List[Nothing] required: scalaz.\/[Any,Int]" because treats list as one function with no parameters and List[Nothing] as result type. Change your code like that:
import scalaz.Scalaz._
import scalaz._
def readLines(fileName: String): Disjunction[Any, List[String]] =
try { scala.io.Source.fromFile(fileName).getLines.toList.right }
catch { case e: java.io.IOException => e.left}
def toInt(line: List[String]): Disjunction[Any, List[Int]] =
try { (line map { _ toInt }).right }
catch { case e: NumberFormatException => e.left}
val numbers = for {
lines <- readLines("/tmp/file.txt")
n <- toInt(lines)
} yield (n map (_ * 2))
That works.
For read line by line maybe FileInputStream can be easier:
fis = new FileInputStream("/tmp/file.txt");
reader = new BufferedReader(new InputStreamReader(fis));
String line = reader.readLine();
while(line != null){
System.out.println(line);
line = reader.readLine();
}
Or you can test the readline function from Source class.