Related
The following code to read and map the lines of a file works ok:
def readLines(fileName: String) = scala.io.Source.fromFile(fileName).getLines
def toInt(line: String) = line.toInt
val numbers: Iterator[Int] = readLines("/tmp/file.txt").map(toInt).map(_ * 2)
println(numbers.toList)
I get an iterator of Ints if the executing goes well. But the program throws an exception if the file is not found, or some line contains letters.
How can I transform the program to use scalaz monads and get a Disjunction[Exception, List[Int]]?
I tried this on scalaz 7.2.6, but it does not compile:
import scalaz.Scalaz._
import scalaz._
def readLines(fileName: String): Disjunction[Any, List[String]] =
try { scala.io.Source.fromFile(fileName).getLines.toList.right }
catch { case e: java.io.IOException => e.left}
def toInt(line: String): Disjunction[Any, Int] =
try { line.toInt.right }
catch { case e: NumberFormatException => e.left}
val numbers: Disjunction[Any, Int] = for {
lines: List[String] <- readLines("/tmp/file.txt")
line: String <- lines
n: Int <- toInt(line)
} yield (n * 2)
it fails to compile with these errors:
Error:(89, 37) could not find implicit value for parameter M: scalaz.Monoid[Any]
lines: List[String] <- readLines("/tmp/file.txt")
Error:(89, 37) not enough arguments for method filter: (implicit M: scalaz.Monoid[Any])scalaz.\/[Any,List[String]].
Unspecified value parameter M.
lines: List[String] <- readLines("/tmp/file.txt")
Error:(91, 20) could not find implicit value for parameter M: scalaz.Monoid[Any]
n: Int <- toInt(line)
Error:(91, 20) not enough arguments for method filter: (implicit M: scalaz.Monoid[Any])scalaz.\/[Any,Int].
Unspecified value parameter M.
n: Int <- toInt(line)
I don't understand the errors. what is the problem?
and how to improve this code, so that it does not read all the file into memory, but it reads and maps each line at a time?
Update: Answer from Filippo
import scalaz._
def readLines(fileName: String) = \/.fromTryCatchThrowable[List[String], Exception] {
scala.io.Source.fromFile(fileName).getLines.toList
}
def toInt(line: String) = \/.fromTryCatchThrowable[Int, NumberFormatException](line.toInt)
type λ[+A] = Exception \/ A
val numbers = for {
line: String <- ListT[λ, String](readLines("/tmp/file.txt"))
n: Int <- ListT[λ, Int](toInt(line).map(List(_)))
} yield n * 2
println(numbers)
To answer the second part of your question, I would simply use the Iterator out of the fromFile method:
val lines: Iterator[String] = scala.io.Source.fromFile(fileName).getLines
If you want to use toInt to convert String to Int:
import scala.util.Try
def toInt(line: String): Iterator[Int] =
Try(line.toInt).map(Iterator(_)).getOrElse(Iterator.empty)
Then numbers could look like:
val numbers = readLines("/tmp/file.txt").flatMap(toInt).map(_ * 2)
EDIT
Due the presence of all these try and catch, if you want to keep using that monadic-for I would suggest to check a scalaz helper like .fromTryCatchThrowable on Disjunction:
import scalaz._, Scalaz._
def readLines(fileName: String): Disjunction[Exception, List[String]] =
Disjunction.fromTryCatchThrowable(scala.io.Source.fromFile(fileName).getLines.toList)
def toInt(line: String): Disjunction[Exception, Int] =
Disjunction.fromTryCatchThrowable(line.toInt)
Now we also have Exception instead of Any as the left type.
val numbers = for {
lines: List[String] <- readLines("/tmp/file.txt")
line: String <- lines // The problem is here
n: Int <- toInt(line)
} yield n * 2
The problem with this monadic-for is that the first and third line are using the Disjunction context but the second one uses the List monad. Using a monad transformer like ListT or DisjunctionT here is possible but probably overkill.
EDIT - to reply the comment
As mentioned, if we want a single monadic-for comprehension, we need a monad transformer, in this case ListT. The Disjunction has two type parameters while a Monad M[_] obviously only one. We need to handle this "extra type parameter", for instance using type lambda:
def readLines(fileName: String) = \/.fromTryCatchThrowable[List[String], Exception] {
fromFile(fileName).getLines.toList
}
val listTLines = ListT[({type λ[+a] = Exception \/ a})#λ, String](readLines("/tmp/file.txt"))
What is the type of listTLines? The ListT transformer: ListT[\/[Exception, +?], String]
The last step in the original for-comprehension was toInt:
def toInt(line: String) = \/.fromTryCatchThrowable[Int, NumberFormatException](line.toInt)
val listTNumber = ListT[\/[Exception, +?], Int](toInt("line"))
What is the type of listTNumber? It doesn't even compile, because the toInt return an Int and not a List[Int]. We need a ListT to join that for-comprehension, one trick could be changing listTNumber to:
val listTNumber = ListT[\/[Exception, +?], Int](toInt("line").map(List(_)))
Now we have both steps:
val numbers = for {
line: String <- ListT[\/[Exception, +?], String](readLines("/tmp/file.txt"))
n: Int <- ListT[\/[Exception, +?], Int](toInt(line).map(List(_)))
} yield n * 2
scala> numbers.run.getOrElse(List.empty) foreach println
2
20
200
If you are wondering why all this unwrapping:
scala> val unwrap1 = numbers.run
unwrap1: scalaz.\/[Exception,List[Int]] = \/-(List(2, 20, 200))
scala> val unwrap2 = unwrap1.getOrElse(List())
unwrap2: List[Int] = List(2, 20, 200)
scala> unwrap2 foreach println
2
20
200
(assuming that the sample file contains the lines: 1, 10, 100)
EDIT - comment about compilation issues
The code above compiles thanks to the Kind Projector plugin:
addCompilerPlugin("org.spire-math" % "kind-projector_2.11" % "0.5.2")
With Kind Projector we can have anonymous types like:
Either[Int, +?] // equivalent to: type R[+A] = Either[Int, A]
Instead of:
type IntOrA[A] = Either[Int, A]
// or
({type L[A] = Either[Int, A]})#L
First, the compiler alerts that you´re using for comprehensions mixing types. Your code is transformed by the compiler as that :
readLines("/tmp/file.txt") flatMap { lines => lines } map { line => toInt(line) }
The definition of flatMap is:
def flatMap[A,B](ma: F[A])(f: A => F[B]): F[B]
In your case F is the \/, and this flatMap { lines => lines } is wrong. The compiler alerts with a message like this "List[Nothing] required: scalaz.\/[Any,Int]" because treats list as one function with no parameters and List[Nothing] as result type. Change your code like that:
import scalaz.Scalaz._
import scalaz._
def readLines(fileName: String): Disjunction[Any, List[String]] =
try { scala.io.Source.fromFile(fileName).getLines.toList.right }
catch { case e: java.io.IOException => e.left}
def toInt(line: List[String]): Disjunction[Any, List[Int]] =
try { (line map { _ toInt }).right }
catch { case e: NumberFormatException => e.left}
val numbers = for {
lines <- readLines("/tmp/file.txt")
n <- toInt(lines)
} yield (n map (_ * 2))
That works.
For read line by line maybe FileInputStream can be easier:
fis = new FileInputStream("/tmp/file.txt");
reader = new BufferedReader(new InputStreamReader(fis));
String line = reader.readLine();
while(line != null){
System.out.println(line);
line = reader.readLine();
}
Or you can test the readline function from Source class.
I'm working examples from the book Learning Scala and one of the questions asks:
How would you add a “sum” method on all tuples, which returns the sum
of all numeric values in a tuple? For example, ('a', "hi", 2.5, 1,
true).sum should return 3.5.
My code:
implicit class PimpedProduct(val p: Product) {
def sum = p.productIterator.filter(_.isInstanceOf[Number]).sum
}
The problem I'm running into is how to convert Any to Numeric[Double]? I could do a match on each Numeric type but that sucks. I read this, which seemed helpful but not quite enough.
You can use java.lang.Number for matching and conversion to a double:
implicit class TupleSum(val p: Product) {
def sum = {
p.productIterator.collect {
case x: java.lang.Number => x.doubleValue
}.sum
}
}
It seems it is not possible to check runtime if some type class exists for given type, as explained in Scala: checking if an object is Numeric answer.
As the other answers show, this is pretty simple if the only thing one sees as a "numeric value" are instances of Number. If you instead want to rely on implicit conversions to Numeric, it gets more complicated. Using the runtime type to look up implicits is something between insane and impossible, but what we can do, is, use macros to look up the implicits at compile time.
The approach this macro follows is to determine the arity of the Tuple and then generate code for accessing its elments (i.e. x._1, x._2, ...). Then it type checks these expressions to determine their static type. Finally it uses the determined type to try to look up an implicit, if this succeeds in generates code accordingly, otherwise it just ignores that value.
I had to dig around a bit in the reflection API to get to this nice result. I hope this is now the definitive version...
So here is the macro:
import scala.language.experimental.macros
import scala.language.implicitConversions
import scala.reflect.macros.blackbox.Context
class FancySum(elements: Traversable[Double]) {
def sum = elements.sum
}
object FancySum {
implicit def toFancySum(product: Product): FancySum = macro toFancySumImpl
def toFancySumImpl(c: Context)(product: c.Expr[Product]): c.Tree = {
import c.universe._
// Search for Tuple amongst base classes and extract arity
val tuple = "scala.Tuple([0-9]+)".r
val arity = product.actualType.baseClasses.map(_.fullName).collectFirst {
case tuple(c) => c.toInt
} match {
case Some(c) => c
case None => c.abort(c.enclosingPosition, "Not a tupel.")
}
val result = for {
// for all entries in the tuple
accessor <- (1 to arity).toList.map(i => {q"""
${product.tree}.${TermName("_" + i)}
"""})
// get the type of that entry
tpe = c.Expr[Any](c.typecheck(accessor, silent = true)).actualType
// Find suitable implicit and generate code to convert to Double
num = c.typecheck(q"""
import ${c.prefix}._
implicitly[Numeric[$tpe]].toDouble($accessor)
""", silent = true)
r <- num match {
case EmptyTree => None // if it doesn't typecheck ignore the entry
case _ => Some(num)
}
} yield r
q"new FancySum($result)"
}
}
And a small test program:
object FancySumApp extends App {
import FancySum.toFancySum
val x= 1
val foo = (x, "asd", 3)
println(foo.sum)
println((0.5, List(), 3, BigInt(2), 10: Any).sum)
// 5.5, as the type of 10 is forced to Any
println((1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
1, 1, 1, 1, 1, 1, 1, 1).sum)
}
Note: if you want to compile it, you have to do it in two stages: the macro first and then the example. Pasting it into REPL step by step works as well.
(Written for Scala 2.11)
Here's an approach that avoids the runtime type checks. It almost certainly isn't quite what you want, but since you're trying to learn about implicits you might still find it useful...
trait ToDouble[T] {
def toDouble(x: T): Double
}
trait LowPriorityToDoubleImplicits {
implicit def defaultToDouble[T]: ToDouble[T] = new ToDouble[T] {
def toDouble(x: T) = 0.0
}
}
object ToDoubleImplicits extends LowPriorityToDoubleImplicits {
implicit def numericToDouble[T](implicit num: Numeric[T]) = new ToDouble[T] {
def toDouble(x: T) = num.toDouble(x)
}
}
implicit class ProductWrapper2[T1, T2](x: Product2[T1, T2])(
implicit ev1: ToDouble[T1], ev2: ToDouble[T2]) {
def sum = ev1.toDouble(x._1) + ev2.toDouble(x._2)
}
implicit class ProductWrapper3[T1, T2, T3](x: Product3[T1, T2, T3])(
implicit ev1: ToDouble[T1], ev2: ToDouble[T2], ev3: ToDouble[T3]) {
def sum = ev1.toDouble(x._1) +
ev2.toDouble(x._2) +
ev3.toDouble(x._3)
}
implicit class ProductWrapper4[T1, T2, T3, T4](x: Product4[T1, T2, T3, T4])(
implicit ev1: ToDouble[T1], ev2: ToDouble[T2], ev3: ToDouble[T3], ev4: ToDouble[T4]) {
def sum = ev1.toDouble(x._1) +
ev2.toDouble(x._2) +
ev3.toDouble(x._3) +
ev4.toDouble(x._4)
}
import ToDoubleImplicits._
(1, "asdf").sum
//1.0
(true, 1.0, BigInt("99999999999999999999999999999999999").sum
//1.0E35
('*', -42, 10.0f, -10L).sum
//0.0
Try this:
implicit class PimpedProduct(val p: Product) {
def sum = p.productIterator.filter(_.isInstanceOf[Number]).map(_.toString.toDouble).sum
}
I'm trying to construct multiple cross products of traversables of different (but each homogeneous) types. The desired return type is a traversable of a tuple with the type matching the types in the input traversables. For example:
List(1, 2, 3) cross Seq("a", "b") cross Set(0.5, 7.3)
This should give a Traversable[(Int, String, Double)] with all possible combinations from the three sources. The case of combining only two sources was nicely answered here. The given idea is:
implicit class Crossable[X](xs: Traversable[X]) {
def cross[A](ys: Traversable[A]) = for { x <- xs; y <- ys } yield (x, y)
}
The comments there briefly mention the problem of more sources, but I'm looking to find a solution that does not depend on either shapeless or scalaz (on the other hand, I don't mind having some boilerplate to scale up to Tuple22). What I would like to do is something like the following:
implicit class Crossable[X](xs: Traversable[X]) {
def cross[A](ys: Traversable[A]) = for { x <- xs; y <- ys } yield (x, y)
def cross[A,B](ys: Traversable[(A,B)]) = // ... extend all Tuple2's in ys with x in xs to Tuple3's
def cross[A,B,C](ys: Traversable[(A,B,C)]) = // ...
// ...
}
This obviously does not work due to type erasure (and, unfortunately, would probably require to use parenthesis in the example above, because cross would be right associative).
My question is: Is it somehow possible to exploit Scala 2.10's reflection features to solve the problem? In general, matching both A and X to the various tuple types (and their type parameters, which seems challenging) and merging them to larger tuples should provide a solution satisfying the associative law, right?
I had a go at it and came up with this:
trait Crosser[A,B,C] {
def cross( as: Traversable[A], bs: Traversable[B] ): Traversable[C]
}
trait LowPriorityCrosserImplicits {
private type T[X] = Traversable[X]
implicit def crosser2[A,B] = new Crosser[A,B,(A,B)] {
def cross( as: T[A], bs: T[B] ): T[(A,B)] = for { a <- as; b <- bs } yield (a, b)
}
}
object Crosser extends LowPriorityCrosserImplicits {
private type T[X] = Traversable[X]
implicit def crosser3[A,B,C] = new Crosser[(A,B),C,(A,B,C)] {
def cross( abs: T[(A,B)], cs: T[C] ): T[(A,B,C)] = for { (a,b) <- abs; c <- cs } yield (a, b, c)
}
implicit def crosser4[A,B,C,D] = new Crosser[(A,B,C),D,(A,B,C,D)] {
def cross( abcs: T[(A,B,C)], ds: T[D] ): T[(A,B,C,D)] = for { (a,b,c) <- abcs; d <- ds } yield (a, b, c, d)
}
// and so on ...
}
implicit class Crossable[A](xs: Traversable[A]) {
def cross[B,C](ys: Traversable[B])(implicit crosser: Crosser[A,B,C]): Traversable[C] = crosser.cross( xs, ys )
}
The main idea is to defer the work to a type class (Crosser) and implement all the different arities simply by specialising for Traversables of tuples with the corresponding arity minus one.
Some test in the REPL:
scala> List(1, 2, 3) cross Seq("a", "b") cross Set(0.5, 7.3)
res10: Traversable[(Int, String, Double)] = List((1,a,0.5), (1,a,7.3), (1,b,0.5), (1,b,7.3), (2,a,0.5), (2,a,7.3), (2,b,0.5), (2,b,7.3), (3,a,0.5), (3,a,7.3), (3,b,0.5), (3,b,7.3))
I am trying to rewrite some java math classes into Scala, but am having an odd problem.
class Polynomials[#specialized T](val coefficients:List[T]) {
def +(operand:Polynomials[T]):Polynomials[T] = {
return new Polynomials[T](coefficients =
(operand.coefficients, this.coefficients).zipped.map(_ + _))
}
}
My problem may be similar to this question: How do I make a class generic for all Numeric Types?, but when I remove the #specialized I get the same error.
type mismatch; found : T required: String
The second underscore in the map function is highlighted for the error, but I don't think that is the problem.
What I want to do is have:
Polynomial(1, 2, 3) + Polynomial(2, 3, 4) return Polynomial(3, 5, 7)
And Polynomial(1, 2, 3, 5) + Polynomial(2, 3, 4) return Polynomial(3, 5, 7, 5)
For the second one I may have to pad the shorter list with zero elements in order to get this to work, but that is my goal on this function.
So, how can I get this function to compile, so I can test it?
List is not specialized, so there's not much point making the class specialized. Only Array is specialized.
class Poly[T](val coef: List[T]) {
def +(op: Poly[T])(implicit adder: (T,T) => T) =
new Poly(Poly.combine(coef, op.coef, adder))
}
object Poly {
def combine[A](a: List[A], b: List[A], f: (A,A) => A, part: List[A] = Nil): List[A] = {
a match {
case Nil => if (b.isEmpty) part.reverse else combine(b,a,f,part)
case x :: xs => b match {
case Nil => part.reverse ::: a
case y :: ys => combine(xs, ys, f, f(x,y) :: part)
}
}
}
}
Now we can
implicit val stringAdd = (s: String, t: String) => (s+t)
scala> val p = new Poly(List("red","blue"))
p: Poly[String] = Poly#555214b9
scala> val q = new Poly(List("fish","cat","dog"))
q: Poly[String] = Poly#20f5498f
scala> val r = p+q; r.coef
r: Poly[String] = Poly#180f471e
res0: List[String] = List(redfish, bluecat, dog)
You could also ask the class provide the adder rather than the + method, or you could subclass Function2 so that you don't pollute things with implicit addition functions.
When I look at Scala libraries I see code like this. Why put test [A] .
def test[A](block : Int => Unit) : Unit = {
block(10)
}
test { u =>
println(u)
}
This is just as valid I suppose. It runs the same way.
def test(block : Int => Unit) : Unit = {
block(10)
}
I've just been curious what the reasoning(or design pattern) is behind it. Thanks.
The type parameter A makes no sense here because it is not used.
def test[A](block: Int => A): A = block(10)
Here A specifies the return type.
When there a generic type next to the function, it means that the function is a generic function.
The following is a very simple example:
// generic functions which returns type of `A`
def test1[A](x: A) = x
def test2[A](x: => A) = { println("Hello"); x }
val x1 = test1(1)
// x1: Int = 1
val x2 = test1("Hello World")
// x2: java.lang.String = Hello World
val x3 = test2(123.4)
// Hello
// x3: Double = 123.4
val x4 = test2("Test2")
// Hello
// x4: java.lang.String = Test2
As you can see, the return type of test1 and test2 are determined by the type of their arguments.
The following is another use case.
// We could implement `map` function ourself.
// We don't care about what type of List contains,
// so we make it a generic function.
def map[A, B](xs: List[A], f: A => B): List[B] = {
var result: List[B] = Nil
for (i <- xs) {
result ++= List(f(i))
}
result
}
// Now use can map any type of List to another List.
map(List("1", "2", "3"), (x: String) => x.toInt)
//res1: List[Int] = List(1, 2, 3)