I got a following code:
trait Stream[+A] {
def uncons: Option[(A, Stream[A])]
def foldRight[B](z: => B)(f: (A, => B) => B): B = {
uncons.map(t => {
f(t._1, t._2.foldRight(z)(f))
}).getOrElse(z)
}
def exists(p: A => Boolean) =
foldRight(false)((x, acc) => acc || p(x))
def forAll(p: A => Boolean) =
foldRight(true)((x, acc) => p(x) && acc)
}
object Stream {
def cons[A](h: => A, t: => Stream[A]): Stream[A] =
new Stream[A] {
lazy val uncons = Some((h, t))
}
}
Then I create a Stream in a lazy manner and invoke exists method to check what stream elements were evaluated:
println(Stream.cons({println("5"); 1}, Stream.cons({println("6"); 2}, Stream.cons({println("7"); 3}, Stream.cons({println("8"); 4}, Stream.empty)))).exists(_ == 1))
And what I see is:
5
6
7
8
true
So all the elements were evaluated in spite of only first one would be enough. I seem to understand why exists acts the way it does.
Then I run the following code:
println(Stream.cons({println("13"); 1}, Stream.cons({println("14"); 2}, Stream.cons({println("15"); 3}, Stream.cons({println("16"); 4}, Stream.empty)))).forAll(_ < 2))
and see the following:
13
14
false
So as far as forAll comes across a non-satisfying value it terminates the traversal.
But why forAll acts that way? What's the crucial difference between it and exists?
There are two things to consider :
the type of acc
the order of p(x) in the boolean expression.
Laziness
If you change the type of acc to B, you won't be able to fail-fast (or short-circuit) in either of your methods. You must know it since your code extensively uses laziness, but a variable of type => B will get evaluated only when its value is required i.e. used in some expression. In this case, acc is the future of the result computed over the stream. This future will happen only if you try looking at it. Thus, to prevent the whole stream to be evaluated, you must prevent this future to be looked at.
Short-circuiting in boolean expressions
This is where the order of p(x) matters. In the expression a && b, if a is false then we know the whole conjunction is also false, thus Scala won't try evaluating b because it's pointless.
Combining the two
Now what happens if one of your operands is a lazy expression ? Well, if you have lazyA || b, Scala will read the expression from left to right and evaluate lazyA. In your case, lazyA represents the accumulation of the next element and the rest of the stream. Thus, lazyA expands to a0 :: lazyA1, which expands to a0 :: a1 :: lazyA2. You will therefore end up computing the whole stream just for computing the left part of your boolean binop.
Now, if you have a && lazyB, this expands to a && (b0 :: b1 :: lazyB2). As you see here, as soon as a or bi is false, this will return without evaluating the right part of the statement. This is what happens in your forAll.
How to fix it
The good news is that the fix is very easy : just swap the order of p(x) and acc : as soon as p(x) is true, the disjunction will return without evaluating acc, stopping the computation.
def exists(p: A => Boolean) = foldRight(false)((x, acc) => p(x) || acc)
Output :
5
true
Related
I was just playing around with a implementation of foldlLeft in Scala
def foldLeft[A,B] (as: List[A], z: B) (f: (B, A) => B): B = as match {
case Nil => z
case Cons(x, xs) => foldLeft(xs, f(z,x)) (f)
}
In this implementation, f(z,x) is in the recursive call given as parameter for z, but I was wondering how this actually works?
When the recursive call happens, does foldLeft() receive the value of the execution of f(z,b) or does it receive the function call the way it is written, and then execute when needed?
Example:
if we call foldLeft() with the following values
def foldLeft[A,B] ([1,2,3], 0) (f: (x, y) => x + y): B = as match {
case Nil => z
case Cons(x, xs) => foldLeft([2,3], f(0,1)) (f)
}
Will the next execution of foldLeft() like this, where z is equal to the value of f()?
def foldLeft[A,B] ([2,3], 1) (f: (x, y) => x + y): B = as match {
case Nil => z
case Cons(x, xs) => foldLeft([2,3], f(1,2)) (f)
}
or does it work like this, where foldLeft() receives the call itself?
def foldLeft[A,B] ([2,3], f(0,1)) (f: (x, y) => x + y): B = as match {
case Nil => z
case Cons(x, xs) => foldLeft([3], f(f(1,0),2)) (f)
}
The question is essentially about when the values that a tail recursive function are evaluated?
Scala, like almost every mainstream language, is a strict language with an eager evaluation strategy and pass-by-value argument passing semantics.
This means that all arguments to a method or function call will be fully evaluated before being passed into the method or function.
However, what I wrote is actually not quite true: that is only the default.
There are two ways to deviate from the default:
The lazy modifier makes a value, well, lazy.
by-name parameters are passed by name, not by value.
#1 is not applicable here. #2 needs to be explicitly declared in the parameter list, which is not the case here.
Therefore, we can conclude that, yes, f(z, x) will be fully evaluated before the recursive call to foldLeft.
This question already has answers here:
Scala - Combine two lists in an alternating fashion
(4 answers)
Closed 3 years ago.
The elements of the resulting list should alternate between the elements of the arguments. Assume that the two arguments have the same length.
USE RECURSION
My code as follows
val finalString = new ListBuffer[Int]
val buff2= new ListBuffer[Int]
def alternate(xs:List[Int], ys:List[Int]):List[Int] = {
while (xs.nonEmpty) {
finalString += xs.head + ys.head
alternate(xs.tail,ys.tail)
}
return finalString.toList
}
EXPECTED RESULT:
alternate ( List (1 , 3, 5) , List (2 , 4, 6)) = List (1 , 2, 3, 4, 6)
As far for the output, I don't get any output. The program is still running and cannot be executed.
Are there any Scala experts?
There are a few problems with the recursive solutions suggested so far (including yours, which would actually work, if you replace while with if): appending to end of list is a linear operation, making the whole thing quadratic (taking a .length of a list too, as well ас accessing elements by index), don't do that; also, if the lists are long, a recursion may require a lot of space on the stack, you should be using tail-recursion whenever possible.
Here is a solution that is free of both those problems: it builds the output backwards, by prepending elements to the list (constant time operation) rather than appending them, and reverses the result at the end. It is also tail-recursive: the recursive call is the last operation in the function, which allows the compiler to convert it into a loop, so that it will only use a single stack frame for execution regardless of the size of the lists.
#tailrec
def alternate(
a: List[Int],
b: List[Int],
result: List[Int] = Nil
): List[Int] = (a,b) match {
case (Nil, _) | (_, Nil) => result.reversed
case (ah :: at, bh :: bt) => alternate(at, bt, bh :: ah :: result)
}
(if the lists are of different lengths, the whole thing stops when the shortest one ends, and whatever is left in the longer one is thrown out. You may want to modify the first case (split it into two, perhaps) if you desire a different behavior).
BTW, your own solution is actually better than most suggested here: it is actually tail recursive (or rather can be made one if you add else after your if, which is now while), and appending to ListBuffer isn't actually as bad as to a List. But using mutable state is generally considered "code smell" in scala, and should be avoided (that's one of the main ideas behind using recursion instead of loops in the first place).
Condition xs.nonEmpty is true always so you have infinite while loop.
Maybe you meant if instead of while.
A more Scala-ish approach would be something like:
def alternate(xs: List[Int], ys: List[Int]): List[Int] = {
xs.zip(ys).flatMap{case (x, y) => List(x, y)}
}
alternate(List(1,3,5), List(2,4,6))
// List(1, 2, 3, 4, 5, 6)
A recursive solution using match
def alternate[T](a: List[T], b: List[T]): List[T] =
(a, b) match {
case (h1::t1, h2::t2) =>
h1 +: h2 +: alternate(t1, t2)
case _ =>
a ++ b
}
This could be more efficient at the cost of clarity.
Update
This is the more efficient solution:
def alternate[T](a: List[T], b: List[T]): List[T] = {
#annotation.tailrec
def loop(a: List[T], b: List[T], res: List[T]): List[T] =
(a, b) match {
case (h1 :: t1, h2 :: t2) =>
loop(t1, t2, h2 +: h1 +: res)
case _ =>
a ++ b ++ res.reverse
}
loop(a, b, Nil)
}
This retains the original function signature but uses an inner function that is an efficient, tail-recursive implementation of the algorithm.
You're accessing variables from outside the method, which is bad. I would suggest something like the following:
object Main extends App {
val l1 = List(1, 3, 5)
val l2 = List(2, 4, 6)
def alternate[A](l1: List[A], l2: List[A]): List[A] = {
if (l1.isEmpty || l2.isEmpty) List()
else List(l1.head,l2.head) ++ alternate(l1.tail, l2.tail)
}
println(alternate(l1, l2))
}
Still recursive but without accessing state from outside the method.
Assuming both lists are of the same length, you can use a ListBuffer to build up the alternating list. alternate is a pure function:
import scala.collection.mutable.ListBuffer
object Alternate extends App {
def alternate[T](xs: List[T], ys: List[T]): List[T] = {
val buffer = new ListBuffer[T]
for ((x, y) <- xs.zip(ys)) {
buffer += x
buffer += y
}
buffer.toList
}
alternate(List(1, 3, 5), List(2, 4, 6)).foreach(println)
}
I have the following code for sorting arrays which I want to verify in Stainless (previously known as Leon):
import stainless.lang._
import stainless.collection._
object QuickSort {
def isSorted(list: List[BigInt]): Boolean = list match {
case Cons(x, xs # Cons(y, _)) => x <= y && isSorted(xs)
case _ => true
}
def quickSort(list: List[BigInt]): List[BigInt] = (list match {
case Nil() => Nil[BigInt]()
case Cons(x, xs) => par(x, Nil(), Nil(), xs)
}) ensuring { res => isSorted(res) }
def par(x: BigInt, l: List[BigInt], r: List[BigInt], ls: List[BigInt]): List[BigInt] = {
require(l.forall(_ <= x) && r.forall(_ >= x))
ls match {
case Nil() => quickSort(l) ++ Cons(x, quickSort(r))
case Cons(x2, xs2) => if (x2 <= x) par(x, Cons(x2, l), r, xs2) else par(x, l, Cons(x2, r), xs2)
}
} ensuring {res => isSorted(res)}
}
I have plenty of directions to go from here (as it doesn't succeed to verify) however it seems to me that the verification should succeed with the hints provided and I want to know why it doesn't. I explain myself:
Apparently for verifying par function I need to proof that the two cases imply the isSorted postcondition separately. Now as the second case contains a recursive call then is evident that it implies the postcondition. For the first case of par, we have that the left and right subarrays are sorted and the precondition tells me that all the elements are sorted with respect to the pivot.
This last bit should imply in my opinion that the concatenating list is sorted as well. So why does it not verify? How could instruct Stainless to verify it? Do I need to add hints on the length and size to facilitate the task to Stainless?
Edit:
def concatIsSorted(l1 : List[BigInt],l2 : List[BigInt],pivot : BigInt) : Boolean = {
require(isSorted(l1) && isSorted(l2) && l1.forall(_ <= pivot) && l2.forall(_ >= pivot))
isSorted(l1 ++ Cons(pivot,l2)) because{
l1 match{
case Nil() => isSorted(Cons(pivot,l2))
case Cons(h,Nil()) => h <= pivot && isSorted(Cons(pivot,l2))
case Cons(h,t) => h <= t.head && concatIsSorted(t,l2,pivot)
}
}
}.holds
Since this is looks like a homework question, I will try to guide you towards the solution without giving it away.
First note that the program verifies if you replace the Nil() case in par with case Nil() => Nil(). This shows that the verifier is not able to prove that the result of quickSort(l) ++ Cons(x, quickSort(r)) is sorted (but it manage to do it for Nil()!).
When --debug=verification is not sufficient to understand why the verifier is not able to prove you think it should, the way to proceed is to introduce extra functions where you can precisely state your expectations. For instance if you define:
def plusplus(l: List[BigInt], r: List[BigInt]): List[BigInt] = l ++ r
And annotate it with your what you expect the verifier to prove, that is
Assuming l and r sorted and l < r (for the appropriate definition of <)
The result of l ++ r is sorted
You will see that the verifier is not able to prove this property, meaning you need to guide the verification further with addition addition auxiliary functions, pre and postcondition.
Note that this example is taken from Dependent Types for Program Termination Verification, reading the paper might help you here.
To find prime factors of a number I was using this piece of code :
def primeFactors(num: Long): List[Long] = {
val exists = (2L to math.sqrt(num).toLong).find(num % _ == 0)
exists match {
case Some(d) => d :: primeFactors(num/d)
case None => List(num)
}
}
but this I found a cool and more functional approach to solve this using this code:
def factors(n: Long): List[Long] = (2 to math.sqrt(n).toInt)
.find(n % _ == 0).fold(List(n)) ( i => i.toLong :: factors(n / i))
Earlier I was using foldLeft or fold simply to get sum of a list or other simple calculations, but here I can't seem to understand how fold is working and how this is breaking out of the recursive function.Can somebody plz explain how fold functionality is working here.
Option's fold
If you look at the signature of Option's fold function, it takes two parameters:
def fold[B](ifEmpty: => B)(f: A => B): B
What it does is, it applies f on the value of Option if it is not empty. If Option is empty, it simply returns output of ifEmpty (this is termination condition for recursion).
So in your case, i => i.toLong :: factors(n / i) represents f which will be evaluated if Option is not empty. While List(n) is termination condition.
fold used for collection / iterators
The other fold that you are taking about for getting sum of collection, comes from TraversableOnce and it has signature like:
def foldLeft[B](z: B)(op: (B, A) => B): B
Here, z is starting value (suppose incase of sum it's 0) and op is associative binary operator which is applied on z and each value of collection from left to right.
So both folds differ in their implementation.
It's a function to find the third largest of a collection of integers. I'm calling it like this:
val lineStream = thirdLargest(Source.fromFile("10m.txt").getLines.toIterable
val intStream = lineStream map { s => Integer.parseInt(s) }
thirdLargest(intStream)
The file 10m.txt contains 10 million lines with a random integer on each one. The thirdLargest function below should not be keeping any of the integers after it has tested them, and yet it causes the JVM to run out of memory (after about 90 seconds in my case).
def thirdLargest(numbers: Iterable[Int]): Option[Int] = {
def top3of4(top3: List[Int], fourth: Int) = top3 match {
case List(a, b, c) =>
if (fourth > c) List(b, c, fourth)
else if (fourth > b) List(b, fourth, c)
else if (fourth > a) List(fourth, b, c)
else top3
}
#tailrec
def find(top3: List[Int], rest: Iterable[Int]): Int = (top3, rest) match {
case (List(a, b, c), Nil) => a
case (top3, d #:: rest) => find(top3of4(top3, d), rest)
}
numbers match {
case a #:: b #:: c #:: rest => Some(find(List[Int](a, b, c).sorted, rest))
case _ => None
}
}
The OOM error has nothing to do with the way you read the file. It is totally fine and even recommended to use Source.getLines here. The problem is elsewhere.
Many people are being confused by the nature of Scala Stream concept. In fact this is not something you would want to use just to iterate over things. It is lazy indeed however it doesn't discard previous results – they're being memoized so there's no need to recalculate them again on the next use (which never happens in your case but that's where your memory goes). See also this answer.
Consider using foldLeft. Here's a working (but intentionally simplified) example for illustration purposes:
val lines = Source.fromFile("10m.txt").getLines()
print(lines.map(_.toInt).foldLeft(-1 :: -1 :: -1 :: Nil) { (best3, next) =>
(next :: best3).sorted.reverse.take(3)
})