I am trying to improve test coverage on my Scala code.
One of the class looks like:
class MyConfig {
val param1: String = sys.env("some string")
val param2: String = scala.util.Properties.envOrElse(SOME_CONSTANT_NAME, null)
}
There are a total of about 30 params like that, all but one are string type (the remaining one is boolean)
Sonarqube is labeling a red tag on the line of "class MyConfig" and reporting 4 conditions not being tested. I did test the boolean param to true and false.
What other conditions should I be testing in a class like this when all are strings parameters?
Thanks for any input.
Related
I have a following User defined function in scala
val returnKey: UserDefinedFunction = udf((key: String) => {
val abc: String = key
abc
})
Now, I want to unit test whether it is returning correct or not. How do I write the Unit test for it. This is what I tried.
class CommonTest extends FunSuite with Matchers {
test("Invalid String Test") {
val key = "Test Key"
val returnedKey = returnKey(col(key));
returnedKey should equal (key);
}
But since its a UDF the returnKey is a UDF function. I am not sure how to call it or how to test this particular scenario.
A UserDefinedFunction is effectively a wrapper around your Scala function that can be used to transform Column expressions. In other words, the UDF given in the question wraps a function of String => String to create a function of Column => Column.
I usually pick 1 of 2 different approaches to testing UDFs.
Test the UDF in a spark plan. In other words, create a test DataFrame and apply the UDF to it. Then collect the DataFrame and check its contents.
// In your test
val testDF = Seq("Test Key", "", null).toDS().toDF("s")
val result = testDF.select(returnKey(col("s"))).as[String].collect.toSet
result should be(Set("Test Key", "", null))
Notice that this lets us test all our edge cases in a single spark plan. In this case, I have included tests for the empty string and null.
Extract the Scala function being wrapped by the UDF and test it as you would any other Scala function.
def returnKeyImpl(key: String) = {
val abc: String = key
abc
}
val returnKey = udf(returnKeyImpl _)
Now we can test returnKeyImpl by passing in strings and checking the string output.
Which is better?
There is a trade-off between these two approaches, and my recommendation is different depending on the situation.
If you are doing a larger test on bigger datasets, I would recommend using testing the UDF in a Spark job.
Testing the UDF in a Spark job can raise issues that you wouldn't catch by only testing the underlying Scala function. For example, if your underlying Scala function relies on a non-serializable object, then Spark will be unable to broadcast the UDF to the workers and you will get an exception.
On the other hand, starting spark jobs in every unit test for every UDF can be quite slow. If you are only doing a small unit test, it will likely be faster to just test the underlying Scala function.
I have some code that requires an Environment Variable to run correctly. But when I run my unit tests, it bombs out once it reaches that point unless I specifically export the variable in the terminal. I am using Scala and sbt. My code does something like this:
class something() {
val envVar = sys.env("ENVIRONMENT_VARIABLE")
println(envVar)
}
How can I mock this in my unit tests so that whenever sys.env("ENVIRONMENT_VARIABLE") is called, it returns a string or something like that?
If you can't wrap existing code, you can change UnmodifiableMap System.getenv() for tests.
def setEnv(key: String, value: String) = {
val field = System.getenv().getClass.getDeclaredField("m")
field.setAccessible(true)
val map = field.get(System.getenv()).asInstanceOf[java.util.Map[java.lang.String, java.lang.String]]
map.put(key, value)
}
setEnv("ENVIRONMENT_VARIABLE", "TEST_VALUE1")
If you need to test console output, you may use separate PrintStream.
You can also implement your own PrintStream.
val baos = new java.io.ByteArrayOutputStream
val ps = new java.io.PrintStream(baos)
Console.withOut(ps)(
// your test code
println(sys.env("ENVIRONMENT_VARIABLE"))
)
// Get output and verify
val output: String = baos.toString(StandardCharsets.UTF_8.toString)
println("Test Output: [%s]".format(output))
assert(output.contains("TEST_VALUE1"))
Ideally, environment access should be rewritten to retrieve the data in a safe manner. Either with a default value ...
scala> scala.util.Properties.envOrElse("SESSION", "unknown")
res70: String = Lubuntu
scala> scala.util.Properties.envOrElse("SECTION", "unknown")
res71: String = unknown
... or as an option ...
scala> scala.util.Properties.envOrNone("SESSION")
res72: Option[String] = Some(Lubuntu)
scala> scala.util.Properties.envOrNone("SECTION")
res73: Option[String] = None
... or both [see envOrSome()].
I don't know of any way to make it look like any/all random env vars are set without actually setting them before running your tests.
You shouldn't test it in unit-test.
Just extract it out
class F(val param: String) {
...
}
In your prod code you do
new Foo(sys.env("ENVIRONMENT_VARIABLE"))
I would encapsulate the configuration in a contraption which does not expose the implementation, maybe a class ConfigValue
I would put the implementation in a class ConfigValueInEnvVar extends ConfigValue
This allows me to test the code that relies on the ConfigValue without having to set or clear environment variables.
It also allows me to test the base implementation of storing a value in an environment variable as a separate feature.
It also allows me to store the configuration in a database, a file or anything else, without changing my business logic.
I select implementation in the application layer.
I put the environment variable logic in a supporting domain.
I put the business logic and the traits/interfaces in the core domain.
I am analyzing some Scala code written in a codebase written by others and I am trying to refactor the code.
What I want to accomplish is: Display the value of the statusUpdate variable, and also capture its internal variables like taskRun.filesFound.
I want to see what is being passed into the statusUpdate varible upstream to some other code, before it is written into that ChannelBuffer with the copiedBuffer method.
So, the code description is as below and in that code there is a method that is is defined in a trait which is in turn is mixed into some other class:
def sendApiUpdate(tasks:Map[String,FileTaskState]) = future{
val statusUpdate = MyTaskStatus(tasks.map(tuple=>
(tuple._1, TaskStatus(
tuple._2.task.name,
tuple._2.taskId,
tuple._2.task.taskState.dispName,
tuple._2.taskRun.trawledTime,
tuple._2.taskRun.filesDiscovered,
tuple._2.taskRun.filesWorkedOn
))).toMap)
dispatch(someUrl, ChannelBuffer.copiedBuffer(write(statusUpdate).getBytes("UTF-8") ))
}
where MyTaskStatus is a case class as below:
case class MyTaskStatus(taskMap:Map[String, TaskStatus])
tasks is:
tasks: Map[String, FileTaskState]
and FileTaskState is a case class as follows:
case class FileTaskState(trawler: ActorRef, taskId: String, runId: String, taskRun)
//there are more field values but I left them out
filesDiscovered is:
def filesDiscovered = noFilesDiscovered.getOrElse(0L)
Also, in the following line of code:
tuple._2.taskRun.filesDiscovered
taskRun is one of the fields of case class FileTaskState
taskRun is:
taskRun: TaskRun
and TaskRun is a case class described as below:
case class TaskRun( taskId: String, noFilesDiscovered)
Update:
I would like to extract the values associated with the tuples in the tasks.map(...) in their separate variable.
for example, I would like to extract tuple._2.taskRun.filesDiscovered into: val filesDisc = <>
The purpose is to help me with refactoring.
Thanks in advance
My intention is to create a function in Scala which accepts some kind of a dynamic query (similar to case expressions in pattern matching) and returns matching instances inside a List. Suppose that the list consists of instances of case class Person which has few properties with different types. The function should be able to accept a dynamic combination of values for some of the fields, and return matching Persons. I am specifically looking for a clean solution. One possible ways to use such a function would be to pass an object with an anonymous type as the query (the "pattern"):
def find(?): List<Person> = { ? }
val matches = find(new { val name = "Name"; val gender = Gender.MALE })
Ideally, I would like to develop a clean way of passing conditions, instead of concrete values, but it's not essential. Since I am learning Scala, I am not aware of all the techniques to implement such a thing. In C#, I used an anonymous type (similar to the second line of code above) and dynamic parameters to achieve something similar. What is a clean and elegant solution in Scala?
I'm not sure if this is what you are looking for but let's try it this way:
First, we define Person as case class Person(name: String, gender: Gender.Value) where Gender is an already defined enum.
Then we create a Query case class which has the same fields, but as options which default to None, and a method for comparing the query to a person:
case class Query(name: Option[String] = None,
gender: Option[Gender.Value] = None){
def ===(person: Person) = check(person.name, name) &&
check(person.gender, gender)
private def check[T](field: T, q: Option[T]) = field == q.getOrElse(field)
}
Unfortunately, in this solution === has to call check separately for each field. Let's leave it like that for now. Maybe it is sufficient (because, for example, the list of fields will not change).
Note that check returns true if the query's option is None, sot you don't have to pass all fields of the query:
val q = Query(name = Some("Ann")) // the gender is not important
q === Person("Ann", Gender.FEMALE) // returns true
And finally the find method:
def find(people: List[Person], query: Query) = people.filter(query === _)
With the following test I have an invalid stub setup. The mock requestBuilder is stubbed with the param "INCORRECT", whereas the class under test invokes it with "/socket".
My experience with Mockito in JUnit tells me that the invocation at runtime will result in null. However, I'm seeing a false positive. The test passes & the requestBuilder is returning the mock request, even though it is invoked with "/socket".
Why is this? Has it something to do with having more than one expectation? If so, the final expectation is the one that counts and it should fail.
class ChannelSpec extends Specification with Mockito with ScalaCheck with ArbitraryValues { def is = s2"""
A Channel must
send an open request to /socket with provided channel name on instantiation $sendToSocket
"""
def sendToSocket = prop{(name: String, key: Key, secret: Secret) =>
val requestBuilder = mock[(String, Seq[Map[String, String]]) => Req]
val request = mock[Req]
val httpRequestor = mock[(Req) => Future[String]]
val result = mock[Future[String]]
val params = Seq(Map("key" -> key.value, "timestamp" -> anyString, "token" -> anyString), Map("channel" -> name))
requestBuilder("INCORRECT", params) returns request
httpRequestor(request) returns result
new Channel(name, key, secret, requestBuilder = requestBuilder, httpRequestor = httpRequestor)
there was one(requestBuilder).apply("INCORRECT", params)
println("expecting " + request)
there was one(httpRequestor).apply(request)
}
While I don't know Scala, I do know that anyString doesn't do what you think it does. Specifically, all Mockito matchers work through side effects, adding the related String objects to ArgumentMatcherStorage as I described toward the end of this SO answer.
Consequently, you can't really extract matchers into variables:
// This is fine, because the parameters are evaluated in order.
takesTwoParameters(eq("A"), eq("B")) returns bar
// Now let's change it a little bit.
val secondParam = eq("SECOND")
val firstParam = eq("FIRST")
// At this point, firstParam == secondParam == null, and the hidden argument
// matcher stack looks like [eq("SECOND"), eq("FIRST")]. That means that your
// stubbing will apply to takesTwoParameters("SECOND", "FIRST"), not
// takesTwoParameters("FIRST", "SECOND")!
takesTwoParameters(firstParam, secondParam)) returns bar
Furthermore, you can't nest anyString into arbitrary types like Seq or Map; Mockito's argument matchers are designed to match entire arguments.
Your best bet is to use argThat for your params, and create a small Hamcrest matcher that checks that the argument you're checking is properly formed. That will give you the flexibility of checking that your params are roughly what you expect, while ignoring the values you don't care about.
I think this is simply the manifestation that, in an acceptance specifications, there are no exceptions to signal failed expectations. So you you have:
def test = {
1 === 2
1 === 1
}
Then test is going to pass because only the last value will be kept as the test result. You can change this behaviour by either chaining expectations:
def test = {
(1 === 2) and
(1 === 1)
}
Or by mixing in the Specification the ThrownExpectations trait:
import org.specs2.Specification
import org.specs2.matcher.ThrownExpectations
import org.specs2.mock.Mockito
class MySpec extends Specification with ThrownExpecations with Mockito {
...
}