I'm kinda new to Scala trying it out while reading Beggining Scala by David Pollack.
He defines a simple recursive function that loads all strings from the file:
def allStrings(expr: => String): List[String] = expr match {
case null => Nil
case w => w :: allStrings(expr)
}
It's elegant and awesome except that it had thrown a StackOverflow exception when I tried to load a huge dictionary file.
Now as far as I understand Scala supports tail recursion, so that function call couldn't possibly overflow the stack, probably compiler doesn't recognize it? So after some googling I tried #tailrec annotation to help the compiler out, but it said
error: could not optimize #tailrec annotated method: it contains a recursive call not in tail position
def allStrings(expr: => String): List[String] =
Am I understanding tail recursion wrong? How do I fix this code?
Scala can only optimise this if the last call is a call to the method itself.
Well, the last call is not to allStrings, it's actually to the :: (cons) method.
A way to make this tail recursive is to add an accumulator parameter, for example:
def allStrings(expr: => String, acc: List[String] = Nil): List[String] =
expr match {
case null => acc
case w => allStrings(expr, w :: acc)
}
To prevent the accumulator leaking into the API, you can define the tail recursive method as a nested method:
def allStrings(expr: => String) = {
def iter(expr: => String, acc: List[String]): List[String] =
expr match {
case null => acc
case w => iter(expr, w :: acc)
}
iter(expr, Nil)
}
It's not tail recursive (and can't ever be) because the final operation is not a recursive call to allStrings, it's a call to the :: method.
The safest way to resolve this is with a nested method that uses an accumulator:
def allStrings(expr: => String) = {
#tailrec
def inner(expr: => String, acc: List[String]): List[String] = expr match {
case null => acc
case w => inner(expr, w :: acc)
}
inner(expr, Nil)
}
In this particular case, you could also lift the accumulator to a parameter on allStrings, give it a default value of Nil, and avoid the need for an inner method. But that's not always possible, and it can't be called nicely from Java code if you're concerned about interop.
Related
As homework i have to write a function that will remove duplicates from a list. It should be recursive and with pattern matching. I am not allowed to use list functions like head,tail,contains,etc... .
For sorted lists i came up with this solution:
def remove(u:List[Int]):List[Int] = {
u match { case Nil => u
case hd::hd2::tl => if(hd == hd2) remove(hd2::tl) else hd :: remove(hd2::tl)
case hd::tl => hd :: remove(tl)
}
}
How can i do it for unsorted lists?
I won't do your homework for you, but hope, this will help.
You want to make your function tail-recursive. That means that the recursive call appears in the very last position of the function, so that the jvm can clear up the previous call from the stack before invoking it (it makes it execute very much like a loop, without requiring additional space on stack). In your original solution it is statements like this, that make it not tail-recursive: hd :: remove(tl): you have to invoke the recursive call, and then prepend hd to its result. The then part breaks the idea of tail recursion, because jvm has to remember on stack the place to return to after the recursive call is finished.
This is typically avoided by carrying the final result of the function through the recursion as an argument:
def remove(u: List[Int], result: List[Int] = Nil): List[Int] = u match {
case Nil => result
case a :: b :: tail if a == b => remove(b :: tail, result)
case head :: tail => remove(tail, head :: result)
}
(Note, that both recursive invocations here are in the tail position - there is nothing left to do after the call returns, so the previous entry can be cleared from the stack prior to invoking the recursion).
You need another recursive function - contains - that tells whether a given element is contained in a list. Once you have that, just replace the second case clause above with something like
case head :: tail if contains(head, result) => remove(tail, result)
and your job is done!
If you want to preserve the original order of elements of the list, you will need to reverse it afterwards (replace case Nil => result with case Nil => result.reverse) ... If you are not allowed to use .reverse here too, that's going to be another nice exercise for you. How do you reverse a list (tail-)recursively?
I would probably go about sorting the list firstly so that we can apply the O(n) complexity pattern matching method but in order to keep the order you will need to index the list so that you can recover the order later, this can be done with the zipWithIndex method. And also since the data type changes from List[Int] to List[(Int, Int)], you will need to define another recursive remove function inside the original one:
def remove(u: List[Int]): List[Int] = {
val sortedU = u.zipWithIndex.sortBy{ case (x, y) => x}
def removeRec(su: List[(Int, Int)]): List[(Int, Int)] = {
su match {
case Nil => su
case hd :: Nil => su
case hd :: hd2 :: tail => {
if (hd._1 == hd2._1) removeRec(hd2 :: tail)
else hd :: removeRec(hd2 :: tail)
}
}
}
removeRec(sortedU).sortBy{case (x, y) => y}.map{ case (x, y) => x}
}
val lst = List(1,2,3,1,3,3)
remove(lst)
// res51: List[Int] = List(2, 1, 3)
Note: This follows OP's pattern and is not tail-recursive. And if you want a tail recursive version, you can follow #Dima's nice explanation.
I have recently started reading the book Functional Programming in Scala by Paul Chiusano and Rúnar Bjarnason, as a means to learn FP. I want to learn it because it will open my head a bit, twist my way of thinking and also hopefully make me a better programmer overall, or so I hope.
In their book, Chp. 3, they define a basic singly-linked-list type as follows:
package fpinscala.datastructures
sealed trait List[+A]
case object Nil extends List[Nothing]
case class Cons[+A](head: A, tail: List[A]) extends List[A]
object List {
def sum(ints: List[Int]): Int = ints match {
case Nil => 0
case Cons(x,xs) => x + sum(xs)
}
def product(ds: List[Double]): Double = ds match {
case Nil => 1.0
case Cons(0.0, _) => 0.0
case Cons(x,xs) => x * product(xs)
}
def apply[A](as: A*): List[A] =
if (as.isEmpty) Nil
else Cons(as.head, apply(as.tail: _*))
}
I'm now working on implementing the tail method, which shall work similarly to the tail method defined in Scala libraries. I guess that the idea here, is to define a tail method inside the List object, what they call a companion method, and then call it normally in another file (like a Main file).
So far, I have this:
def tail[A](ls: List[A]): List[A] = ls match {
case Nil => Nil
case Cons(x,xs) => xs
}
Then I created a Main file in another folder:
package fpinscala.datastructures
object Main {
def main(args:Array[String]):Unit = {
println("Hello, Scala !! ")
val example = Cons(1, Cons(2, Cons(3, Nil)))
val example2 = List(1,2,3)
val example3 = Nil
val total = List.tail(example)
val total2 = List.tail(example3)
println(total2)
}
}
This works and gives me:
Hello, Scala !!
Cons(2,Cons(3,Nil))
My question is:
Is this the correct way to write the tail method, possibly, as the authors intended? And is this package structure correct? Because it feels very wrong to me, although I just followed the authors package.
I also don't know if I should have used a specific type instead of writing a polymorphic method (is this the name?)...
Bear with me, for I am a newbie in the art of FP.
In the default Scala list implementation, attempting to take the tail of an empty list would throw an UnsupportedOperationException. So you might want something more like
def tail[A](ls: List[A]): List[A] = ls match {
case Nil => throw new UnsupportedOperationException()
case Cons(x,xs) => xs
}
Also, qantik's answer where he suggests using the :: operator would work with Scala's default list implementation, but since there isn't a :: method defined on this custom list implementation it won't work.
Finally, you may want to consider defining tail so that instead of doing
val list = List(1, 2, 3)
val restOfList = tail(list).
you could instead do
val list = List(1, 2, 3)
val restOfList = list.tail
That would require defining the method on the List trait, as opposed to in the List object.
Looks okay. What about?
def tail[A](xs: List[A]): List[A] = xs match {
case Nil => Nil
case head :: xxs => xxs
}
I want to update a sequence in Scala, I have this code :
def update(userId: Long): Either[String, Int] = {
Logins.findByUserId(userId) map {
logins: Login => update(login.id,
Seq(NamedParameter("random_date", "prefix-" + logins.randomDate)))
} match {
case sequence : Seq(Nil, Int) => sequence.foldLeft(Right(_) + Right(_))
case _ => Left("error.logins.update")
}
}
Where findByUserId returns a Seq[Logins] and update returns Either[String, Int] where Int is the number of updated rows,
and String would be the description of the error.
What I want to achieve is to return an String if while updating the list an error happenes or an Int with the total number of updated rows.
The code is not working, I think I should do something different in the match, I don't know how I can check if every element in the Seq of Eithers is a Right value.
If you are open to using Scalaz or Cats you can use traverse. An example using Scalaz :
import scalaz.std.either._
import scalaz.std.list._
import scalaz.syntax.traverse._
val logins = Seq(1, 2, 3)
val updateRight: Int => Either[String, Int] = Right(_)
val updateLeft: Int => Either[String, Int] = _ => Left("kaboom")
logins.toList.traverseU(updateLeft).map(_.sum) // Left(kaboom)
logins.toList.traverseU(updateRight).map(_.sum) // Right(6)
Traversing over the logins gives us a Either[String, List[Int]], if we get the sum of the List we get the wanted Either[String, Int].
We use toList because there is no Traverse instance for Seq.
traverse is a combination of map and sequence.
We use traverseU instead of traverse because it infers some of the types for us (otherwise we should have introduced a type alias or a type lambda).
Because we imported scalaz.std.either._ we can use map directly without using a right projection (.right.map).
You shouldn't really use a fold if you want to exit early. A better solution would be to recursively iterate over the list, updating and counting successes, then return the error when you encounter one.
Here's a little example function that shows the technique. You would probably want to modify this to do the update on each login instead of just counting.
val noErrors = List[Either[String,Int]](Right(10), Right(12))
val hasError = List[Either[String,Int]](Right(10), Left("oops"), Right(12))
def checkList(l: List[Either[String,Int]], goodCount: Int): Either[String, Int] = {
l match {
case Left(err) :: xs =>
Left(err)
case Right(_) :: xs =>
checkList(xs, (goodCount + 1))
case Nil =>
Right(goodCount)
}
}
val r1 = checkList(noErrors, 0)
val r2 = checkList(hasError, 0)
// r1: Either[String,Int] = Right(2)
// r2: Either[String,Int] = Left(oops)
You want to stop as soon as an update fails, don't you?
That means that you want to be doing your matching inside the map, not outside. Try is actually a more suitable construct for this purpose, than Either. Something like this, perhaps:
def update(userId: Long): Either[String, Int] = Try {
Logins.findByUserId(userId) map { login =>
update(login.id, whatever) match {
case Right(x) => x
case Left(s) => throw new Exception(s)
}
}.sum
}
.map { n => Right(n) }
.recover { case ex => Left(ex.getMessage) }
BTW, a not-too-widely-known fact about scala is that putting a return statement inside a lambda, actually returns from the enclosing method. So, another, somewhat shorter way to write this would be like this:
def update(userId: Long): Either[String, Int] =
Logins.findByUserId(userId).foldLeft(Right(0)) { (sum,login) =>
update(login.id, whatever) match {
case Right(x) => Right(sum.right + x)
case error#Left(s) => return error
}
}
Also, why in the world does findUserById return a sequence???
I've defined an ordering for Seq[Seq[T]] such that it's a normal lexicographic ordering except all items (sub-sequences) are reversed first (so that C,B,A comes before A,B,C but after A,B,A):
implicit def ReverseListOrdering[T: Ordering] = new Ordering[Seq[T]] {
override def compare(xs1: Seq[T], xs2: Seq[T]) =
doCompare(xs1.reverse, xs2.reverse)
private def doCompare(xs1: Seq[T], xs2: Seq[T]): Int = (xs1, xs2) match {
case (Nil, Nil) => 0
case (x :: _, Nil) => 1
case (Nil, x :: _) => -1
case (x :: xs, y :: ys) =>
val a = implicitly[Ordering[T]].compare(x, y)
if (a != 0) a else doCompare(xs, ys)
}
}
This used to be defined on List[List[T]] at first but I later realized I want it for all Seq[Seq[T]]; this is why I initially left in the Nils in the pattern matching block, while failing to realize Nil never matches e.g. an empty Array.
Later I tried to run this block of code:
// the Seq[String] declarations are needed; otherwise sample` will be Array[Object] for some reason
val sample = List(
List("Estonia"): Seq[String],
Array("Tallinn", "Estonia"): Seq[String],
List("Tallinn", "Harju", "Estonia"): Seq[String])
println(sample.sorted)
This compiles just fine but results in the following error at runtime:
scala.MatchError: (WrappedArray(Estonia, Tallinn),List(Estonia)) (of class scala.Tuple2)
— while I understand the cause of the error perfectly well, what I fail to understand (or at least accept) is, if the Ordering is successfully defined on all Seq[Seq[T]], yet a superficially valid but obviously incompatible (in terms of how doCompare is defined) Seq[Seq[T]] (i.e. Array[List[String] | Array[String]]) attempt to use that ordering results in no static type errors or even warnings whatsoever.
Is this caused by the fact that the pattern matching code is not statically verified to cover all possible "instances" of Seq[Seq[T]] and that it only handles the List case? If yes, what are the currently available workarounds to achieving type safety in such cases? Is Scalaz something to be looked at yet again for a decent solution?
P.S. I'm aware I could easily do away with a solution that works for all Seq[Seq[T]] by not using pattern matching and resorting to head and tail with an if-else block (or case statements if guards, which is only superficially nicer), but I'm keen to learn how to make the most of out Scala's type capabilities (AFAIK F# and Haskell catch these errors for breakfast); not to mention pattern matching is by far more elegant and readable.
This might be closer:
scala> def cmp[A, B[_] <: Seq[_]](xs: B[A], ys: B[A]) = (xs, ys) match { case (Nil, Nil) => true }
<console>:7: error: pattern type is incompatible with expected type;
found : scala.collection.immutable.Nil.type
required: B[?A1] where type ?A1 (this is a GADT skolem)
def cmp[A, B[_] <: Seq[_]](xs: B[A], ys: B[A]) = (xs, ys) match { case (Nil, Nil) => true }
^
There is a known syndrome when the type parameter is on the class instead of the method.
It has surfaced on the ML recently. Here.
Comparing to:
scala> (null: Seq[_]) match { case _: Nil.type => true }
scala.MatchError: null
... 33 elided
scala> (null: List[_]) match { case _: Nil.type => true }
<console>:8: warning: match may not be exhaustive.
It would fail on the following input: List(_)
(null: List[_]) match { case _: Nil.type => true }
^
scala.MatchError: null
... 33 elided
scala> (null: List[_]) match { case Nil => true }
<console>:8: warning: match may not be exhaustive.
It would fail on the following input: List(_)
(null: List[_]) match { case Nil => true }
^
scala.MatchError: null
... 33 elided
Sorry to be lazy, but off to bed.
Scala compiler produces non-exhaustive match warnings only for sealed types, which Seq isn't. AFAIK, there is no way to force it to check, like there is for tail recursion.
(AFAIK F# and Haskell catch these errors for breakfast)
Haskell doesn't have an equivalent to Seq; the code you used to have with List is the one which is equivalent to Haskell (modulo laziness), and Scala does catch the error in this case. I don't know F# well, but looking at http://msdn.microsoft.com/en-us/library/dd547125.aspx it seems that pattern matching a general IEnumerable<T> isn't supported.
Use code:
implicit def ReverseListOrdering[T: Ordering] = new Ordering[Seq[T]] {
override def compare(xs1: Seq[T], xs2: Seq[T]) =
doCompare(xs1.reverse, xs2.reverse)
private def doCompare(xs1: Seq[T], xs2: Seq[T]): Int = (xs1, xs2) match {
case (Seq(), Seq()) => 0
case (x +: _, Seq()) => 1
case (Seq(), x +: _) => -1
case (x +: xs, y +: ys) =>
val a = implicitly[Ordering[T]].compare(x, y)
if (a != 0) a else doCompare(xs, ys)
}
}
As I mention is comment Nil is not the same type as Seq(). For example WrappedArray is not a List. And when you use x :: xs - it is matched as List. Array("Tallinn", "Estonia") is converted to WrappedArray. Always use +: in pattern matching when you use Seq
I'm working through the scala labs stuff and I'm building out a function that will, in the end, return something like this:
tails(List(1,2,3,4)) = List(List(1,2,3,4), List(2,3,4), List(3,4), List(4), List())
I got this working by using two functions and using some recursion on the second one.
def tails[T](l: List[T]): List[List[T]] = {
if ( l.length > 1 )trailUtil(List() ::: List(l))
else List() ::: List(l);
}
def trailUtil[T](l:List[List[T]]) : List[List[T]] = {
if ( l.last.length == 0)l
else trailUtil(l :+ l.last.init);
}
This is all good a great but it's bugging me that I need two functions to do this. I tried switching: trailUtil(List() ::: List(l)) for an anonymous function but I got this error type mismatch; found :List[List[T]] required:Int from the IDE.
val ret : List[List[T]] = (ll:List[List[T]]) => {
if ( ll.last.length == 0) ll else ret(ll :+ ll.last.init)
}
ret(List() ::: List(1))
Could someone please point me to what I am doing wrong, or a better way of doing this that would be great.
(I did look at this SO post but the different type are just not working for me):
What about this:
def tails[T](l: List[T]): List[List[T]] =
l match {
case h :: tail => l :: tails(tail)
case Nil => List(Nil)
}
And a little bit less idiomatic version:
def tails[T](input: List[T]): List[List[T]] =
if(input.isEmpty)
List(List())
else
input :: tails(input.tail)
BTW try to avoid List.length, it runs in O(n) time.
UPDATE: as suggested by tenshi, tail-recursive solution:
#tailrec def tails[T](l: List[T], init: List[List[T]] = Nil): List[List[T]] =
l match {
case h :: tail => tails(tail, l :: init)
case Nil => init
}
You actually can define def inside another def. It allows to define function that actually has name which can be referenced and used for recursion. Here is how tails can be implemented:
def tails[T](l: List[T]) = {
#annotation.tailrec
def ret(ll: List[List[T]]): List[List[T]] =
if (ll.last.isEmpty) ll
else ret(ll :+ ll.last.tail)
ret(l :: Nil)
}
This implementation is also tail-recursive. I added #annotation.tailrec annotation in order to ensure that it really is (code will not compile if it's not).
You can also use build-in function tails (see ScalaDoc):
List(1,2,3,4).tails.toList
tails returns Iterator, so you need to convert it to list (like I did), if you want it. Also result will contain one extra empty in the end (in my example result would be List(List(1, 2, 3, 4), List(2, 3, 4), List(3, 4), List(4), List())), so you need deal with it.
What you are doing wrong is this:
val ret : List[List[T]]
So ret is a list of list of T. Then you do this:
ret(ll :+ ll.last.init)
That mean you are calling the method apply on a list of list of T. The apply method for lists take an Int parameter, and returns an element with that index. For example:
scala> List("first", "second", "third")(2)
res0: java.lang.String = third
I assume you wanted to write val ret: List[List[T]] => List[List[T]], that is, a function that takes a List[List[T]] and returns a List[List[T]]. You'd have other problems then, because val is referring to itself in its definition. To get around that, you could replace it with a lazy val:
def tails[T](l: List[T]): List[List[T]] = {
lazy val ret : List[List[T]] => List[List[T]] = { (ll:List[List[T]]) =>
if ( ll.last.length == 0) ll
else ret(ll :+ ll.last.init)
}
if ( l.length > 1 )ret(List() ::: List(l))
else List() ::: List(l);
}
But, of course, the easy solution is to put one def inside the other, like tenshi suggested.
You can also use folding:
val l = List(1,2,3,4)
l.foldLeft(List[List[Int]](l))( (outerList,element) => {
println(outerList)
outerList.head.tail :: outerList
})
The first parameter list is your start value/accumulator. The second function is the modifier. Typically, it modifies the start value, which is then passed to every element in the list. I included a println so you can see the accumulator as the list is iterated over.