I want to test my code which has a few tail recursive functions.
I couldn't mock the tail recursive functions because they need to declared either Final or Private.
Most mocking frameworks don't support mocking such methods, the ones that support isn't working as expected.
Is this possible at all?
Can some one provide me their ideas to mock tail recursive functions?
I tried mocking using Mockito Framework version 3.0.0. My Test suite is extended with the Mockito-Sugar trait.
Though mockito documentation suggests that final methods can be mocked, It results in failure for me.
I tried using scala-mock. I faced different problems and didn't work out.
One way to fix this is to wrap the recursive code in an outer function. For example:
def factorial(n: Int): Int = {
#annotation.tailrec
def loop(i: Int, res: Int): Int =
if (i <= 1) {
res
} else {
loop(i-1, i*res)
}
loop(n-1, n)
}
Using this pattern the factorial method does not need to be final or private so it can be overridden for testing.
Another advantage of this pattern is that the accumulator value res does not have to be exposed in the main function interface. This pattern also allows special cases to be handled outside the main recursive code, making the inner code simpler and potentially faster.
Related
In scala, the following 2 functions serve exactly the same purpose:
#tailrec
final def fn(str: String): Option[String] = {
Option(str).filter(_.nonEmpty).flatMap { v =>
fn(v.drop(1))
}
}
#tailrec
final def fn2(str: String): Option[String] = {
Option(str).filter(_.nonEmpty) match {
case None => None
case Some(v) => fn2(v.drop(1))
}
}
However #tailrec only works in second case, in the first case it will generate the following error:
Error: could not optimize #tailrec annotated method fn: it contains a
recursive call not in tail position
Option(str).filter(_.nonEmpty).flatMap { v =>
Why this error was given? And why these 2 codes generate different kinds JVM bytecode
For fn to be tail-recursive, the recursive call must be the last action in the function. If you pass fn to another function such as flatMap then the other function is free to perform other actions after calling fn and therefore the compiler cannot be sure that it is tail recursive.
In some cases the compiler could detect that calling fn is the last action in the other function, but not in the general case. And this would rely on a specific implementation of that other function so the tailrec annotation might become invalid if that other function were changed, which is an undesirable dependency.
Specifically for the last question:
And why these 2 codes generate different kinds JVM bytecode
Because on JVM there's no guarantee that the JAR containing Option class at runtime is the same as was seen at compile-time. This is good, because otherwise even minor versions of libraries (including standard Java and Scala libraries) would be incompatible, and you'd need all dependencies to be using the same minor version of their common dependencies.
If that class doesn't have a suitable flatMap method, you'll get AbstractMethodError, but otherwise semantics of Scala require that its flatMap method must be called. So the compiler has to emit bytecode to actually call the method.
Kotlin works around this by using inline functions and Scala 3 will support them too, but I don't know if it'll use them for such cases.
Consider the following:
List('a', 'b').flatMap(List(_,'g')) //res0: List[Char] = List(a, g, b, g)
I seems pretty obvious that flatMap() is doing some internal post-processing in order to achieve that result. How else would List('a','g') get combined with List('b','g')?
A mutable Set's retain method is implemented as follows:
def retain(p: A => Boolean): Unit =
for (elem <- this.toList) // SI-7269 toList avoids ConcurrentModificationException
if (!p(elem)) this -= elem
But if I implement my own method that doesn't make a copy for iterating, nothing blows up.
def dumbRetain[A](self: mutable.Set[A], p: A => Boolean): Unit =
for (elem <- self)
if (!p(elem)) self -= elem
dumbRetain(mutable.HashSet(1,2,3,4,5,6), Set(2,4,6))
// everything is ok
I see that SI-7269's test case uses the JavaConversions wrapper around a java Set/Map, and it seems like the issue arises from the underlying java collection.
I know there will never be a java collection passed to my algorithm, so can I use dumbRetain without worrying about the ConcurrentModificationException? Or is this "coincidental behavior" that I shouldn't rely on?
edit to clarify, I would be using dumbRetain as an implementation detail in an algorithm which would be in full control of what it passes to dumbRetain. And this would be run in a single-threaded context.
This seems to rely on the specific implementation of mutable.HashSet, and there is nothing in the API that guarantees that it would work for all other implementations of mutable.Set, even if we exclude all wrappers for the Java collections.
The for-loop
for (elem <- self) {
...
}
is desugared into foreach, which for mutable.HashSet is implemented as follows:
override def foreach[U](f: A => U) {
var i = 0
val len = table.length
while (i < len) {
val curEntry = table(i)
if (curEntry ne null) f(entryToElem(curEntry))
i += 1
}
}
Essentially, it simply loops through the Array of the underlying FlatHashTable, and invokes the passed function f on every element. The whole foreach simply does not have any lines which could throw anything, it doesn't check for concurrent [footnote-1] modifications at all.
A ConcurrentModificationException seems to be the less troubling case: at least, your program fails fast, and even returns a detailed stack trace that points to the line in which the problem occurred. It would be actually much worse if it simply deteriorated into undefined behavior without throwing anything. This would be the worst case. However, this worst case shouldn't occur for collections from the standard library: Throw ConcurrentModificationException exception's in scala collections? #188
Quote:
In scala/scala#5295 (merged in to 2.12.x) I made sure that removing the element last returned from an iterator would not cause a problem for the iterator.
So, as long as you clearly state in the documentation that only the collections from standard library are supported, you will most likely not have any problems using it in your own code. But if you use it in a public interface, this would be an invitation for a bug analogous to "SI-7269" quoted in your question.
[footnote-1] "concurrent" as in "ConcurrentModificationException", not as in "concurrently executed threads".
EDIT: I've tried to choose less ambiguous formulations. Great Thanks #Dima for the feedback and the numerous suggestions.
Yeah, you can do it, as long as you are sure this is the scala's native HashSet implementation, not a wrapper around java ... and with understanding, that this is not thread-safe, and should never be used concurrently (the original HashSet.retain is that way too as well as the other mutators).
Better yet, just use immutable Set.filter, unless you actually have real hard evidence (not just intuition) demonstrating that your specific case absolutely requires mutable container.
Here is a minimal code that raise the compilation error 'Recursive call not in tail position'. However, I'm using an #inline and the recursive call is in tail position. The reason why I'm using this #inline is that I have the code pf the original reccall duplicated twice.
import scala.annotation._
object Test {
#tailrec private def test(i: Int): Int = {
#inline def reccall(i: Int): Int = test(i-1)
i match {
case 0 => 0
case i => reccall(i)
}
}
}
I've looked at the answers Recursive call not in tail position #tailrec why does this method not compile with 'contains a recursive call not in tail position'? but they do not apply to my case. Using Scala 2.12
It appears, the way #inline is implemented is that it still passes the parameter via stack. The jump is eliminated, by inserting the code inline, but the stack is still used for the arguments. This makes it impossible to be in a tail position, because the stack needs to be cleaned up after the call is completed.
Besides, annotating a function with #inline does not guarantee that the optimizer will inline it, just that it will "try especially hard".
Well, the mechanism of how tail recursion is actualized in JVM is explained in following way:
Scala, in the case of tail recursion, can eliminate the creation of a
new stack frame and just re-use the current stack frame. The stack
never gets any deeper, no matter how many times the recursive call is
made.
So in your case it cannot reuse the current stack frame belonging to the test method since it MUST create a new stack frame for the reccall method anyway.
Recursive call is implicit in this case, made from another method. So I believe you cannot really have tail recursion implemented for such case.
You may just remove the reccall method altogether and write case i => test(i-1) and then compiler will not complain.
NOTE: also I believe #inline has nothing to do here and is not essential in this example, since if I remove it - compiler still complains the same reason.
The issue here is that #inline is strictly advisory: it doesn't guarantee that the compiler will inline the function. Since #tailrec only works if it's absolutely guaranteed that the tail-calls can be eliminated, this means that using #tailrec has to assume no inlining.
Our case classes have several require statements to throw exceptions when they receive malformed input. This is normally pretty helpful, but can make writing property-based tests a pain, since we have to write generators that satisfy all the requirements rather than simply using the built-in generators. Is there an easy way to set Scala to ignore require statements during testing?
A much better approach would be to avoid require entirely, and instead write smart constructors that return Either (or Validation if you're scalaz-inclined.) Better still would be to create newtype wrappers (i.e. case class MyWrapper private (i: Int) extends AnyVal) for your input types, where those wrappers come with smart constructor such that only valid values can be constructed. When you write generators for these bespoke types, you can ensure that the invariants you need are preserved.
Rather than look for a way around the type system (by using require to throw exceptions at runtime) let the type system work for you. This will make not only writing your property-based tests much easier, but the entirety of your system will benefit.
No - require is a very simple method, as you can see from the source:
def require(requirement: Boolean) {
if (!requirement)
throw new IllegalArgumentException("requirement failed")
}
#inline final def require(requirement: Boolean, message: => Any) {
if (!requirement)
throw new IllegalArgumentException("requirement failed: "+ message)
}
I'd suggest providing a separate way to construct your class that only the tests use. For example, if you scope the constructor to the package:
package foo
final case class Foo private[foo] (...)
object Foo {
def apply(...): Foo = /* put assertions in here */
}
Then you could put your scalacheck generators in the same package and let them use the private constructor.
Or if you really want to get weird with it, you could write your own require that determines whether to skip the checks based on some global state or an implicit parameter. But this seems unwise.
AFAIK, Iterator.map is lazy while Vector.map is eager, basically because they are different types of monads.
I would like to know if there is any chance of having a EagerTry and LazyTry that behave just like the current Try, but with the latter (LazyTry) delaying the execution of the closure passed until the result is needed (if it is needed).
Please note that declaring stuff as lazy doesn't work quite well in Scala, in particular it works for a given scope. An alternative exists when passing parameters (parameters by name). The question is how to achieve lazy behaviour when returning (lazy) values to an outer scope. Option is basically a collection of length 0 or 1, this would be an equivalent case for lazy collections (Iterator, Sequence) but limited to length 0 or 1 (like Option and Either). I'm particularly interested in Try, i.e. using LazyTry exactly as Try would be used. I guess this should be analogous in other cases (Option and Either).
Please note that we already have EagerTry, as the current standard Try is eager. Unfortunately, the class is sealed, therefore, to have eager and lazy versions of the same class we would need to define the three of them and implement two of them (as opposed to defining and implementing one). The point is returning a Try without other software layers worrying about the time of execution of that code, i.e. abstraction.
Yes, it wouldn't be hard to write LazyTry. One possible approach:
sealed class LazyTry[A](block: => A) {
// the only place block is used
private lazy val underlying: Try[A] = Try(block)
def get = underlying.get
def isSuccess = underlying.isSuccess
...
}
object LazyTry {
def apply[A](block: => A): LazyTry[A] = new LazyTry[A](block)
...
}
Note that you don't have LazySuccess and LazyFailure, since you don't know which class to use before running block.