Scala test dependent methods used to calculate vals are executed only once - scala

I am new to scala, and I'm trying figure out the best way to test the following process.
I have a class that gets a list of numbers from constructor parameter. The class supports various operations on the list, some operations may depend on the output of other operations. But every option should only perform calculations on demand and should be done at most once. No calculations should be done in the constructor.
Example class definition .
InputList: List[Int] .
x: returns a vector with the square of all elements in InputList .
y: returns the sum of all elements in x .
z: returns the square root of y .
As for class implementation, I think I was able to come up with a fitting solution but now I can't figure out how can I test the calculations of the dependent tree of operations are executed only once.
Class Implementation Approach #1:
class Operations(nums: List[Int]) {
lazy val x: List[Int] = nums.map(n => n*n)
lazy val y: Int = x.sum
lazy val z: Double = scala.math.sqrt(y)
}
This was my first approach which I'm confident will do the job but could not figure out how to properly test it so I decided to add some helper methods to confirm they are being called just ones
Class Implementation Approach #2:
class Ops(nums: List[Int]) {
def square(numbers: List[Int]): List[Int] = {
println("calling square function")
numbers.map(n => n*n)
}
def sum(numbers: List[Int]): Int = {
println("calling sum method")
numbers.sum
}
def sqrt(num: Int): Double = {
println("calling sqrt method")
scala.math.sqrt(num)
}
lazy val x: Vector[Double] = square(nums)
lazy val y: Double = sum(x)
lazy val z: Double = sqrt(y)
}
I can now confirm each dependent method of each method is called just once whenever necessary.
Now how can I write tests for these processes. I've seen a few posts about mockito and looked at the documentation but was not able to find what I was looking for. I looked at the following:
Shows how to test whether a function is called once but then how to test whether other depend functions where called?
http://www.scalatest.org/user_guide/testing_with_mock_objects#mockito
Mockito: How to verify a method was called only once with exact parameters ignoring calls to other methods?
Seems promising but I can't figure out the syntax:
https://github.com/mockito/mockito-scala
Example Tests I'd like to perform
var listoperations:Ops = new Ops(List(2,4,4))
listoperations.y // confirms 36 is return, confirms square and sum methods were called just once
listoperations.x // confirms List(4,16,16) and confirms square method was not called
listoperations.z // confirms 6 is returned and sqrt method called once and square and sum methods were not called.

Ok, lets leave the pre-mature optimisation argument for another time.
Mocks are meant to be used to stub/verify interactions with dependencies of your code (aka other classes), not to check internals of it, so in order to achieve what you want you'd need something like this
class Ops {
def square(numbers: List[Int]): List[Int] = numbers.map(n => n*n)
def sum(numbers: List[Int]): Int = numbers.sum
def sqrt(num: Int): Double = scala.math.sqrt(num)
}
class Operations(nums: List[Int])(implicit ops: Ops) {
lazy val x: List[Int] = ops.square(nums)
lazy val y: Int = ops.sum(x)
lazy val z: Double = ops.sqrt(y)
}
import org.mockito.{ ArgumentMatchersSugar, IdiomaticMockito}
class IdiomaticMockitoTest extends AnyWordSpec with IdiomaticMockito with ArgumentMatchersSugar
"operations" should {
"be memoised" in {
implicit val opsMock = spy(new Ops)
val testObj = new Operations(List(2, 4, 4))
testObj.x shouldBe List(4, 16, 16)
testObj.y shouldBe 36
testObj.y shouldBe 36 //call it again just for the sake of the argument
testObj.z shouldBe 6 //sqrt(36)
testObj.z shouldBe 6 //sqrt(36), call it again just for the sake of the argument
opsMock.sum(*) wasCalled once
opsMock.sqrt(*) wasCalled once
}
}
}
Hope it makes sense, you mentioned you're new to scala, so I didn't wanna go too crazy with implicits so this is a very basic example in which the API of your original Operations class is the same, but it extracts out the heavy lifting to a third party that can be mocked so you can verify the interactions.

As you mentioned the Mockito is the way to go, here is an example:
class NumberOPSTest extends FunSuite with Matchers with Mockito {
test("testSum") {
val listoperations = smartMock[NumberOPS]
when(listoperations.sum(any)).thenCallRealMethod()
listoperations.sum(List(2, 4, 4)) shouldEqual 10
verify(listoperations, never()).sqrt(any)
}
}

Related

Why do each new instance of case classes evaluate lazy vals again in Scala?

From what I have understood, scala treats val definitions as values.
So, any instance of a case class with same parameters should be equal.
But,
case class A(a: Int) {
lazy val k = {
println("k")
1
}
val a1 = A(5)
println(a1.k)
Output:
k
res1: Int = 1
println(a1.k)
Output:
res2: Int = 1
val a2 = A(5)
println(a1.k)
Output:
k
res3: Int = 1
I was expecting that for println(a2.k), it should not print k.
Since this is not the required behavior, how should I implement this so that for all instances of a case class with same parameters, it should only execute a lazy val definition only once. Do I need some memoization technique or Scala can handle this on its own?
I am very new to Scala and functional programming so please excuse me if you find the question trivial.
Assuming you're not overriding equals or doing something ill-advised like making the constructor args vars, it is the case that two case class instantiations with same constructor arguments will be equal. However, this does not mean that two case class instantiations with same constructor arguments will point to the same object in memory:
case class A(a: Int)
A(5) == A(5) // true, same as `A(5).equals(A(5))`
A(5) eq A(5) // false
If you want the constructor to always return the same object in memory, then you'll need to handle this yourself. Maybe use some sort of factory:
case class A private (a: Int) {
lazy val k = {
println("k")
1
}
}
object A {
private[this] val cache = collection.mutable.Map[Int, A]()
def build(a: Int) = {
cache.getOrElseUpdate(a, A(a))
}
}
val x = A.build(5)
x.k // prints k
val y = A.build(5)
y.k // doesn't print anything
x == y // true
x eq y // true
If, instead, you don't care about the constructor returning the same object, but you just care about the re-evaluation of k, you can just cache that part:
case class A(a: Int) {
lazy val k = A.kCache.getOrElseUpdate(a, {
println("k")
1
})
}
object A {
private[A] val kCache = collection.mutable.Map[Int, Int]()
}
A(5).k // prints k
A(5).k // doesn't print anything
The trivial answer is "this is what the language does according to the spec". That's the correct, but not very satisfying answer. It's more interesting why it does this.
It might be clearer that it has to do this with a different example:
case class A[B](b: B) {
lazy val k = {
println(b)
1
}
}
When you're constructing two A's, you can't know whether they are equal, because you haven't defined what it means for them to be equal (or what it means for B's to be equal). And you can't statically intitialize k either, as it depends on the passed in B.
If this has to print twice, it would be entirely intuitive if that would only be the case if k depends on b, but not if it doesn't depend on b.
When you ask
how should I implement this so that for all instances of a case class with same parameters, it should only execute a lazy val definition only once
that's a trickier question than it sounds. You make "the same parameters" sound like something that can be known at compile time without further information. It's not, you can only know it at runtime.
And if you only know that at runtime, that means you have to keep all past uses of the instance A[B] alive. This is a built in memory leak - no wonder Scala has no built-in way to do this.
If you really want this - and think long and hard about the memory leak - construct a Map[B, A[B]], and try to get a cached instance from that map, and if it doesn't exist, construct one and put it in the map.
I believe case classes only consider the arguments to their constructor (not any auxiliary constructor) to be part of their equality concept. Consider when you use a case class in a match statement, unapply only gives you access (by default) to the constructor parameters.
Consider anything in the body of case classes as "extra" or "side effect" stuffs. I consider it a good tactic to make case classes as near-empty as possible and put any custom logic in a companion object. Eg:
case class Foo(a:Int)
object Foo {
def apply(s: String) = Foo(s.toInt)
}
In addition to dhg answer, I should say, I'm not aware of functional language that does full constructor memoizing by default. You should understand that such memoizing means that all constructed instances should stick in memory, which is not always desirable.
Manual caching is not that hard, consider this simple code
import scala.collection.mutable
class Doubler private(a: Int) {
lazy val double = {
println("calculated")
a * 2
}
}
object Doubler{
val cache = mutable.WeakHashMap.empty[Int, Doubler]
def apply(a: Int): Doubler = cache.getOrElseUpdate(a, new Doubler(a))
}
Doubler(1).double //calculated
Doubler(5).double //calculated
Doubler(1).double //most probably not calculated

Unable to override implicits in scala

I am trying to learn scalaz and still new to scala (been using for a few months now). I really like the type-classes scalaz provides and trying to document the different use-cases for different features in scalaz. Right now I am having issues with the way implicts work and the way I want to do things.
Here is the code I would like to work
import scalaz._
import Scalaz._
object WhatIfIWantfFuzzyMatching extends App {
implicit object evensEquals extends Equal[Int] {
override def equal(left: Int, right: Int): Boolean = {
val leftMod = left % 2
val rightMod = right % 2
leftMod == rightMod
}
}
val even = 2
val odd = 3
assert(even =/= odd, "Shouldn't have matched!")
val evenMultTwo = even * 2
assert(even === evenMultTwo, "Both are even, so should have matched")
}
The idea is simple. For some parts of the code, I want the provided Equal[Int] from scalaz. In other parts of the code, I would like to override the Equal[Int] provided with a less strict one.
Right now I am hitting an issue where scala isn't able to figure out which implicit to use:
ambiguous implicit values:
both object evensEquals in object WhatIfIWantfFuzzyMatching of type com.gopivotal.scalaz_examples.equal.WhatIfIWantfFuzzyMatching.evensEquals.type
and value intInstance in trait AnyValInstances of type => scalaz.Monoid[Int] with scalaz.Enum[Int] with scalaz.Show[Int]
match expected type scalaz.Equal[Int]
assert(even =/= odd, "Shouldn't have matched!")
^
Looking at other threads on here, I see people say to just change the type so there isn't a conflict, or to only import when needed, but in the case of scalaz's === and mixing and matching different equals methods, I am not sure how to get that to work with the compiler.
Any thoughts?
EDIT:
Here is a working example that lets you switch between implementations (thanks #alexey-romanov)
object WhatIfIWantToSwitchBack extends App {
// so what if I want to switch back to the other Equals?
object modEqualsInt extends Equal[Int] {
override def equal(left: Int, right: Int): Boolean = {
val leftMod = left % 2
val rightMod = right % 2
leftMod == rightMod
}
}
implicit var intInstance: Equal[Int] = Scalaz.intInstance
assert(2 =/= 4)
intInstance = modEqualsInt
assert(2 === 4)
intInstance = Scalaz.intInstance
assert(2 =/= 4)
}
You can just use the same name intInstance (but this means you'll lose the Monoid, Enum and Show instances).

Call a function with arguments from a list

Is there a way to call a function with arguments from a list? The equivalent in Python is sum(*args).
// Scala
def sum(x: Int, y: Int) = x + y
val args = List(1, 4)
sum.???(args) // equivalent to sum(1, 4)
sum(args: _*) wouldn't work here.
Don't offer change the declaration of the function anyhow. I'm acquainted with a function with repeated parameters def sum(args: Int*).
Well, you can write
sum(args(0), args(1))
But I assume you want this to work for any list length? Then you would go for fold or reduce:
args.reduce(sum) // args must be non empty!
(0 /: args)(sum) // aka args.foldLeft(0)(sum)
These methods assume a pair-wise reduction of the list. For example, foldLeft[B](init: B)(fun: (B, A) => B): B reduces a list of elements of type A to a single element of type B. In this example, A = B = Int. It starts with the initial value init. Since you want to sum, the sum of an empty list would be zero. It then calls the function with the current accumulator (the running sum) and each successive element of the list.
So it's like
var result = 0
result = sum(result, 1)
result = sum(result, 4)
...
The reduce method assumes that the list is non-empty and requires that the element type doesn't change (the function must map from two Ints to an Int).
I wouldn't recommend it for most uses since it's a bit complicated and hard to read, bypasses compile-time checks, etc., but if you know what you're doing and need to do this, you can use reflection. This should work with any arbitary parameter types. For example, here's how you might call a constructor with arguments from a list:
import scala.reflect.runtime.universe
class MyClass(
val field1: String,
val field2: Int,
val field3: Double)
// Get our runtime mirror
val runtimeMirror = universe.runtimeMirror(getClass.getClassLoader)
// Get the MyClass class symbol
val classSymbol = universe.typeOf[MyClass].typeSymbol.asClass
// Get a class mirror for the MyClass class
val myClassMirror = runtimeMirror.reflectClass(classSymbol)
// Get a MyClass constructor representation
val myClassCtor = universe.typeOf[MyClass].decl(universe.termNames.CONSTRUCTOR).asMethod
// Get an invokable version of the constructor
val myClassInvokableCtor = myClassMirror.reflectConstructor(myClassCtor)
val myArgs: List[Any] = List("one", 2, 3.0)
val myInstance = myClassInvokableCtor(myArgs: _*).asInstanceOf[MyClass]

How to write this three-liner as a one-liner?

I like the way, you can write one-liner-methods in Scala, e.g. with List(1, 2, 3).foreach(..).map(..).
But there is a certain situation, that sometimes comes up when writing Scala code, where things get a bit ugly. Example:
def foo(a: A): Int = {
// do something with 'a' which results in an integer
// e.g. 'val result = a.calculateImportantThings
// clean up object 'a'
// e.g. 'a.cleanUp'
// Return the result of the previous calculation
return result
}
In this situation we have to return a result, but can not return it directly after the calculation is done, because we have to do some clean up before returning.
I always have to write a three-liner. Is there also a possibility to write a one-liner to do this (without changing the class of A, because this may be a external library which can not be changed) ?
There are clearly side-effects involved here (otherwise the order of invocation of calculateImportantThings and cleanUp wouldn't matter) so you would be well advised to reconsider your design.
However, if that's not an option you could try something like,
scala> class A { def cleanUp {} ; def calculateImportantThings = 23 }
defined class A
scala> val a = new A
a: A = A#927eadd
scala> (a.calculateImportantThings, a.cleanUp)._1
res2: Int = 23
The tuple value (a, b) is equivalent to the application Tuple2(a, b) and the Scala specification guarantees that its arguments will be evaluated left to right, which is what you want here.
This is a perfect use-case for try/finally:
try a.calculateImportantThings finally a.cleanUp
This works because try/catch/finally is an expression in scala, meaning it returns a value, and even better, you get the cleanup whether or not the calculation throws an exception.
Example:
scala> val x = try 42 finally println("complete")
complete
x: Int = 42
There is, in fact, a Haskell operator for just such an occasion:
(<*) :: Applicative f => f a -> f b -> f a
For example:
ghci> getLine <* putStrLn "Thanks for the input!"
asdf
Thanks for the input!
"asdf"
All that remains then is to discover the same operator in scalaz, since scalaz usually replicates everything that Haskell has. You can wrap values in Identity, since Scala doesn't have IO to classify effects. The result would look something like this:
import scalaz._
import Scalaz._
def foo(a: A): Int =
(a.calculateImportantThings.pure[Identity] <* a.cleanup.pure[Identity]).value
This is rather obnoxious, though, since we have to explicitly wrap the side-effecting computations in Identity. Well the truth is, scalaz does some magic that implicitly converts to and from the Identity container, so you can just write:
def foo(a: A): Int = Identity(a.calculateImportantThings) <* a.cleanup()
You do need to hint to the compiler somehow that the leftmost thing is in the Identity monad. The above was the shortest way I could think of. Another possibility is to use Identity() *> foo <* bar, which will invoke the effects of foo and bar in that order, and then produce the value of foo.
To return to the ghci example:
scala> import scalaz._; import Scalaz._
import scalaz._
import Scalaz._
scala> val x : String = Identity(readLine) <* println("Thanks for the input!")
<< input asdf and press enter >>
Thanks for the input!
x: String = asdf
Maybe you want to use a kestrel combinator? It is defined as follows:
Kxy = x
So you call it with the value you want to return and some side-effecting operation you want to execute.
You could implement it as follows:
def kestrel[A](x: A)(f: A => Unit): A = { f(x); x }
... and use it in this way:
kestrel(result)(result => a.cleanUp)
More information can be found here: debasish gosh blog.
[UPDATE] As Yaroslav correctly points out, this is not the best application of the kestrel combinator. But it should be no problem to define a similar combinator using a function without arguments, so instead:
f: A => Unit
someone could use:
f: () => Unit
class Test {
def cleanUp() {}
def getResult = 1
}
def autoCleanup[A <: Test, T](a: A)(x: => T) = {
try { x } finally { a.cleanUp }
}
def foo[A <: Test](a:A): Int = autoCleanup(a) { a.getResult }
foo(new Test)
You can take a look at scala-arm project for type class based solution.
Starting Scala 2.13, the chaining operation tap can be used to apply a side effect (in this case the cleanup of A) on any value while returning the original value untouched:
def tap[U](f: (A) => U): A
import util.chaining._
// class A { def cleanUp { println("clean up") } ; def calculateImportantThings = 23 }
// val a = new A
val x = a.calculateImportantThings.tap(_ => a.cleanUp)
// clean up
// x: Int = 23
In this case tap is a bit abused since we don't even use the value it's applied on (a.calculateImportantThings (23)) to perform the side effect (a.cleanUp).

JUnit Theories and Scala

I'm looking for a way to test my Scala code with multiple inputs. Comming from Java/JUnit, I immediately thought of #RunWith(Theories.class).
Where I'm stuck is the usage of #DataPoints and the absence of static members/methods in Scala. So is there a way to write the following code in Scala?
#RunWith(classOf[Theories])
class ScalaTheory {
#DataPoints
val numbers = Array(1, 2, 3)
#Theory
def shouldMultiplyByTwo(number : Int) = {
// Given
val testObject = ObjectUnderTest
// When
val result = testObject.run(number)
// Then
assertTrue(result == number * 2)
}
}
I'm neither fixed on JUnit nor Theories so if there is something Scala-specific for this use case, I'm happy to use that.
To make this work, you have to do two things: use a method[see edit below], not a value, and secondly, define your #DataPoints in a companion object. The following should work:
object ScalaTheory {
#DataPoints
def numbers() = Array(1, 2, 3) // note def not val
}
#RunWith(classOf[Theories])
class ScalaTheory {
#Theory
def shouldMultiplyByTwo(number : Int) = {
// Given
val testObject = ObjectUnderTest
// When
val result = testObject.run(number)
// Then
assertTrue(result == number * 2)
}
}
When you define methods or fields in a companion object in Scala, you get a static forwarder in the class. Decompiling using JAD:
#Theories
public static final int[] numbers()
{
return ScalaTheory$.MODULE$.numbers();
}
So this takes care of the static problem. However, when we use a val numbers = ..., the annotation isn't carried over to the field, but it is for methods. So using def works.
As the others have said, if you're developing from scratch, it may be worth starting in a Scala framework like scalatest. The tool integration with scalatest is improving (i.e maven, Eclipse, Intellij), but it's not the level of JUnit, so evaluate it for your project before you start.
EDIT: In fact, after this discussion on scala-user, you can use val, but you need to tell the scala compiler to apply the DataPoints annotation to the static forwarder:
object ScalaTheory {
#(DataPoints #scala.annotation.target.getter)
val numbers = Array(1, 2, 3)
}
The getter annotation says that the #DataPoints annotation should be applied to the accessor method for the numbers field, that is the numbers() method which is created by the compiler. See package target.
I expect that what you want is possible with every Scala testing framework. I'm only familiar with Specs2, so here's my shot:
class DataPoints extends Specification {
val objectUnderTest: Int => Int = _ + 2
val testCases = 1 :: 2 :: 3 :: 4 :: Nil
def is: Fragments =
(objectUnderTest must multiplyByTwo((_: Int))).foreach(testCases)
def multiplyByTwo(i: Int): Matcher[(Int) => Int] =
(=== (i * 2)) ^^
((f: Int => Int) => f(i) aka "result of applying %s to %d".format(f, i))
}
And the output is:
result of applying <function1> to 1 '3' is not equal to '2'; result of applying <function1> to 3 '5' is not equal to '6'; result of applying <function1> to 4 '6' is not equal to '8'
Disclaimer
I do not state that this is very readable. Also I'm not an expert Specs2 user.
Just as ziggystar I can't really help you with your direct question. But I strongly recommend to switch to a scala testing framework. My personal favorite is scalatest.
In the last example here: http://blog.schauderhaft.de/2011/01/16/more-on-testing-with-scalatest/ I demonstrate how simple and straight forward it is to write test that runs with multiple inputs. It is just a simple loop!