Proof the concatenation of ordered list is an ordered list in Stainless - scala

I have the following code for sorting arrays which I want to verify in Stainless (previously known as Leon):
import stainless.lang._
import stainless.collection._
object QuickSort {
def isSorted(list: List[BigInt]): Boolean = list match {
case Cons(x, xs # Cons(y, _)) => x <= y && isSorted(xs)
case _ => true
}
def quickSort(list: List[BigInt]): List[BigInt] = (list match {
case Nil() => Nil[BigInt]()
case Cons(x, xs) => par(x, Nil(), Nil(), xs)
}) ensuring { res => isSorted(res) }
def par(x: BigInt, l: List[BigInt], r: List[BigInt], ls: List[BigInt]): List[BigInt] = {
require(l.forall(_ <= x) && r.forall(_ >= x))
ls match {
case Nil() => quickSort(l) ++ Cons(x, quickSort(r))
case Cons(x2, xs2) => if (x2 <= x) par(x, Cons(x2, l), r, xs2) else par(x, l, Cons(x2, r), xs2)
}
} ensuring {res => isSorted(res)}
}
I have plenty of directions to go from here (as it doesn't succeed to verify) however it seems to me that the verification should succeed with the hints provided and I want to know why it doesn't. I explain myself:
Apparently for verifying par function I need to proof that the two cases imply the isSorted postcondition separately. Now as the second case contains a recursive call then is evident that it implies the postcondition. For the first case of par, we have that the left and right subarrays are sorted and the precondition tells me that all the elements are sorted with respect to the pivot.
This last bit should imply in my opinion that the concatenating list is sorted as well. So why does it not verify? How could instruct Stainless to verify it? Do I need to add hints on the length and size to facilitate the task to Stainless?
Edit:
def concatIsSorted(l1 : List[BigInt],l2 : List[BigInt],pivot : BigInt) : Boolean = {
require(isSorted(l1) && isSorted(l2) && l1.forall(_ <= pivot) && l2.forall(_ >= pivot))
isSorted(l1 ++ Cons(pivot,l2)) because{
l1 match{
case Nil() => isSorted(Cons(pivot,l2))
case Cons(h,Nil()) => h <= pivot && isSorted(Cons(pivot,l2))
case Cons(h,t) => h <= t.head && concatIsSorted(t,l2,pivot)
}
}
}.holds

Since this is looks like a homework question, I will try to guide you towards the solution without giving it away.
First note that the program verifies if you replace the Nil() case in par with case Nil() => Nil(). This shows that the verifier is not able to prove that the result of quickSort(l) ++ Cons(x, quickSort(r)) is sorted (but it manage to do it for Nil()!).
When --debug=verification is not sufficient to understand why the verifier is not able to prove you think it should, the way to proceed is to introduce extra functions where you can precisely state your expectations. For instance if you define:
def plusplus(l: List[BigInt], r: List[BigInt]): List[BigInt] = l ++ r
And annotate it with your what you expect the verifier to prove, that is
Assuming l and r sorted and l < r (for the appropriate definition of <)
The result of l ++ r is sorted
You will see that the verifier is not able to prove this property, meaning you need to guide the verification further with addition addition auxiliary functions, pre and postcondition.
Note that this example is taken from Dependent Types for Program Termination Verification, reading the paper might help you here.

Related

disjoint Pattern-Matching

i have a question about disjoint Matching pattern. The matching pattern is disjoint when each case does not step on other cases.
def func(list: List[Int]): Int = list match {
case Nil => 0
case x::t if (x < func(t)) => x
case x::t => func(t)
}
My question: is "if" statement also counted to check if these cases disjoint is?
so if we have a patching pattern like this it means the last case also include the second case and it would not be disjoint anyway . But if i change the last case to
case x::t if (x >= func(t)) => func(t)
would the matching pattern considered disjoint ?
Since x < func(t) = !(x >= func(t)), yes, these patterns are all disjoint. The compiler doesn't use predicate disjointness in anyway; this will have no concrete implications.

Scala for expressions in tail recursive form

I have a bit of a problem trying to come up with a valid way to convert a for - expression N queens solution to a tail recursive form and still preserve the idiomatic nature achieved by using the for syntax. Any ideas are more than welcome.
def place(boardSize: Int, n: Int): Solutions = n match {
case 0 => List(Nil)
case _ =>
for {
queens <- place(boardSize, n - 1)
y <- 1 to boardSize
queen = (n, y)
if (isSafe(queen, queens))
} yield queen :: queens
}
def isSafe(queen: Queen, others: List[Queen]) = {...}
What you're writing basically corresponds to what's called Depth-First Search (DFS).
Although a recursive implementation of DFS is easily written, it is not tail-recursive. Here's a proposal for a tail-recursive one. Note that I did not test this code, but it should at least give you an idea of how to proceed.
def solve(): List[List[Int]] = {
#tailrec def solver(fringe: List[List[Int]], solutions: List[List[Int]]): List[List[Int]] = fringe match {
case Nil => solutions
case potentialSol :: fringeTail =>
if(potentialSol.length == n) // We found a solution
solver(fringe.tail, potentialSol.reverse :: solutions)
else { // Keep looking
val unused = (1 to n).toList filterNot potentialSol.contains
val children = for(u <- unused ; partial = u :: fringe.head if isValid(partial)) yield partial
solver(children ++ fringe.tail, solutions)
}
}
solver((1 to n).toList.map(List(_)), Nil).map(_.reverse)
}
If you're concerned about performances, note that this solution is very poor because it uses slow operations on immutable data structure, and because on the JVM you're better off using iteration where performance matters. This will start failing quite rapidly as n increases. Algorithmically, there are far better ways to solve NQueens than using DFS.

Scala Stream function evaluation

I got a following code:
trait Stream[+A] {
def uncons: Option[(A, Stream[A])]
def foldRight[B](z: => B)(f: (A, => B) => B): B = {
uncons.map(t => {
f(t._1, t._2.foldRight(z)(f))
}).getOrElse(z)
}
def exists(p: A => Boolean) =
foldRight(false)((x, acc) => acc || p(x))
def forAll(p: A => Boolean) =
foldRight(true)((x, acc) => p(x) && acc)
}
object Stream {
def cons[A](h: => A, t: => Stream[A]): Stream[A] =
new Stream[A] {
lazy val uncons = Some((h, t))
}
}
Then I create a Stream in a lazy manner and invoke exists method to check what stream elements were evaluated:
println(Stream.cons({println("5"); 1}, Stream.cons({println("6"); 2}, Stream.cons({println("7"); 3}, Stream.cons({println("8"); 4}, Stream.empty)))).exists(_ == 1))
And what I see is:
5
6
7
8
true
So all the elements were evaluated in spite of only first one would be enough. I seem to understand why exists acts the way it does.
Then I run the following code:
println(Stream.cons({println("13"); 1}, Stream.cons({println("14"); 2}, Stream.cons({println("15"); 3}, Stream.cons({println("16"); 4}, Stream.empty)))).forAll(_ < 2))
and see the following:
13
14
false
So as far as forAll comes across a non-satisfying value it terminates the traversal.
But why forAll acts that way? What's the crucial difference between it and exists?
There are two things to consider :
the type of acc
the order of p(x) in the boolean expression.
Laziness
If you change the type of acc to B, you won't be able to fail-fast (or short-circuit) in either of your methods. You must know it since your code extensively uses laziness, but a variable of type => B will get evaluated only when its value is required i.e. used in some expression. In this case, acc is the future of the result computed over the stream. This future will happen only if you try looking at it. Thus, to prevent the whole stream to be evaluated, you must prevent this future to be looked at.
Short-circuiting in boolean expressions
This is where the order of p(x) matters. In the expression a && b, if a is false then we know the whole conjunction is also false, thus Scala won't try evaluating b because it's pointless.
Combining the two
Now what happens if one of your operands is a lazy expression ? Well, if you have lazyA || b, Scala will read the expression from left to right and evaluate lazyA. In your case, lazyA represents the accumulation of the next element and the rest of the stream. Thus, lazyA expands to a0 :: lazyA1, which expands to a0 :: a1 :: lazyA2. You will therefore end up computing the whole stream just for computing the left part of your boolean binop.
Now, if you have a && lazyB, this expands to a && (b0 :: b1 :: lazyB2). As you see here, as soon as a or bi is false, this will return without evaluating the right part of the statement. This is what happens in your forAll.
How to fix it
The good news is that the fix is very easy : just swap the order of p(x) and acc : as soon as p(x) is true, the disjunction will return without evaluating acc, stopping the computation.
def exists(p: A => Boolean) = foldRight(false)((x, acc) => p(x) || acc)
Output :
5
true

Why does this function run out of memory?

It's a function to find the third largest of a collection of integers. I'm calling it like this:
val lineStream = thirdLargest(Source.fromFile("10m.txt").getLines.toIterable
val intStream = lineStream map { s => Integer.parseInt(s) }
thirdLargest(intStream)
The file 10m.txt contains 10 million lines with a random integer on each one. The thirdLargest function below should not be keeping any of the integers after it has tested them, and yet it causes the JVM to run out of memory (after about 90 seconds in my case).
def thirdLargest(numbers: Iterable[Int]): Option[Int] = {
def top3of4(top3: List[Int], fourth: Int) = top3 match {
case List(a, b, c) =>
if (fourth > c) List(b, c, fourth)
else if (fourth > b) List(b, fourth, c)
else if (fourth > a) List(fourth, b, c)
else top3
}
#tailrec
def find(top3: List[Int], rest: Iterable[Int]): Int = (top3, rest) match {
case (List(a, b, c), Nil) => a
case (top3, d #:: rest) => find(top3of4(top3, d), rest)
}
numbers match {
case a #:: b #:: c #:: rest => Some(find(List[Int](a, b, c).sorted, rest))
case _ => None
}
}
The OOM error has nothing to do with the way you read the file. It is totally fine and even recommended to use Source.getLines here. The problem is elsewhere.
Many people are being confused by the nature of Scala Stream concept. In fact this is not something you would want to use just to iterate over things. It is lazy indeed however it doesn't discard previous results – they're being memoized so there's no need to recalculate them again on the next use (which never happens in your case but that's where your memory goes). See also this answer.
Consider using foldLeft. Here's a working (but intentionally simplified) example for illustration purposes:
val lines = Source.fromFile("10m.txt").getLines()
print(lines.map(_.toInt).foldLeft(-1 :: -1 :: -1 :: Nil) { (best3, next) =>
(next :: best3).sorted.reverse.take(3)
})

Stable identifier required during pattern matching? (Scala)

Trying to produce a list of tuples showing prime factor multiplicity... the idea is to match each integer in a sorted list against the first value in a tuple, using the second value to count. Could probably do it more easily with takeWhile, but meh. Unfortunately my solution won't compile:
def primeFactorMultiplicity (primeFactors: List[Int]) = {
primeFactors.foldRight (List[(Int, Int)]()) ((a, b) => (a, b) match {
case (_, Nil) => (a, 1) :: b
case (b.head._1, _) => (a, b.head._2 + 1) :: b.tail
case _ => (a, 1) :: b
})
}
It says "error: stable identifier required, but b.head._1 found." But changing the second case line to the following works fine:
case (i, _) if (i == b.head._1) => (a, b.head._2 + 1) :: b.tail
Why is this, and why can't the compiler cope if there is such a simple fix?
A variable in a pattern captures the value in that position; it does not do a comparison. If the syntax worked at all, the result would be to put the value of a into b.head._1, overwriting the current value. The purpose of this is to let you use a pattern to pull something out of a complex structure.
b.head._1 is not a valid name for the result of the (x, y) tuple extractor
Try this instead:
case (x, _) if x == b.head._1 => (a, b.head._2 + 1) :: b.tail