I have a number of test files, each with their own tests. Until now, each one has a trait in which I create a configuration object.
Building that object now takes the better part of two minutes, as it has to do a lot of work calling a number of databases (don't ask - really, it has to be done).
Is there a way to share this object (tldConfigMap, below) between multiple test suites in multiple files without having to build it over and over?
Here's how I was doing it - as you can see, when brought in as a trait, load() will be called every time:
trait TLDMapAcceptanceIsLoadedSpec extends org.scalatest.fixture.FlatSpecLike with TLDConfigMap {
val tldConfigMap: Map[String, TLDConfig] = load(withAttributes = true).right.get
type FixtureParam = Map[String, TLDConfig]
def withFixture(test: OneArgTest) = {
withFixture(test.toNoArgTest(tldConfigMap)) // "loan" the fixture to the test
}
}
You could just put it into an object (assuming it's really config and isn't changed by tests):
object TldConfigMap {
val tldConfigMap: Map[String, TLDConfig] = load(withAttributes = true).right.get
}
trait TLDMapAcceptanceIsLoadedSpec extends org.scalatest.fixture.FlatSpecLike with TLDConfigMap {
def tldConfigMap: Map[String, TLDConfig] = TldConfigMap.tldConfigMap // or just use it directly
type FixtureParam = Map[String, TLDConfig]
def withFixture(test: OneArgTest) = {
withFixture(test.toNoArgTest(tldConfigMap)) // "loan" the fixture to the test
}
}
Related
I would like to use a simple example for my problem.
I have a below trait:
trait ReaderTrait {
def readTable(tableName:String, closingDate: String)(implicit sparkSession: SparkSession): DataFrame = {
sparkSession.read.table(
tableName
).filter(col("closingDate") === closingDate)
}
def read(fullTable:String, closingDate: String)(implicit sparkSession: SparkSession): DataFrame
}
I have below Object:
object Reader extends ReaderTrait
{
override final def read(fullTable: String, closingDate: String)(implicit sparkSession: SparkSession): DataFrame =
{
readTable(fullTable: String, closingDate: String)
//.filter..
//I have business logic here which I want to test, I want to mock only readTable
}
}
Now I want to mock only the readTable method in Object.
I have a Test class:
class ReaderTest extends FeatureSpec with MockitoSugar {
scenario("first") {
//sample dataframe1 created ......
val mockTrait = mock[Reader.type ]
when(mockTrait.readTable("abc", "sss")(spark)) thenReturn (dataframe1)
val result = mockTrait.read("abc", "sss")(spark)
result.show()
}
}
After running the above test class, it's giving me the below error:
java.lang.NullPointerException was thrown.
Please help me to resolve this situation; I have to mock the readTable method in read method in object without making any changes to existing code.
Many problems here:
You cannot mock objects.
mock[Reader.type] does not make sense. You could mock the trait with mock[ReaderTrait] though.
Mock doesn't work the way you seem to think.
If you have class A { def foo = ""; def bar = foo }, and then you do
val m = mock[A]
when(m.foo).thenReturn("baz")
val result = m.bar
the value of result will not be "baz" it will be null. Why? Because m is a mock – a dummy object, with all of the methods stubbed out. m.bar does not call foo, because it is a stub, it just returns null.
There is a way to stub only some methods of a class, and have others keep original behavior. If you'd like to learn more about it, look up the concept of a spy (you would do spy(new A) instead of mock[A]). I will not get into this here, because it is not a good practice to use spies. Which brings me to my third point ...
You should not stub methods of the class you are testing. When you need to, it is a symptom of your class design violating the single responsibility principle, which is not a good thing.
The proper way of writing your test in this case would be isolate the readTable "provider" class from the functionality you are testing.
For example:
trait TableReader {
def readTable(table:String, date: String)(implicit s: SparkSession) =
s.read.table(table).filter(col("closingDate") === date)
}
object TableReader extends TableReader
class Reader(val tr: TableReader) {
def read(table: String, date: String)(implicit s: SparkSession) =
tr.readTable(table, date)
}
object Reader extends Reader(TableReader)
Now you can test your Reader class with something like:
val tr = mock[TableReader]
wen(tr.readTable(any, any)(any)).thenReturn(dataframe1)
val reader = new Reader(tr)
reader.read("foo", "bar")(spark) shouldBe dataframe1
verify(tr).readTable("foo", "bar")(spark)
This will work, but kinda unclear what it is you are trying to test here.
You are not testing readTable because you stubbed it out. And there is no logic in read, so nothing to test there.
I would suggest to move the filtering logic out from TableReader into Reader, that would make for a cleaner separation of responsibilities between the two classes: one is responsible for fetching data from the table, and the other applies filtering and other business logic:
trait TableReader {
def readTable(t: String)(implicit s: SparkSession) = s.read.table(t)
}
class Reader(tr: TableReader) {
def read(table: String, date: String)(implicit s: SparkSession) =
tr.readTable(table).filter(col("closingDate") === date)
}
This gives you an ability to write meaningful tests, like:
reader.read("foo", "bar")(spark)
.select("closingDate")
.as[String]
.collect
.toList shouldBe List("bar", "bar", "bar")
(basically, checking that the filter for closingDate === "bar" was properly applied to the dataframe)
The idea here is that you do not need to test TableReader, as there is no any business logic there, it is only a simple wrapper around spark, so you can stub it out easily.
And the Reader class is only concerned about business logic that you do want to test, and does not know anything about where the actual data comes from, so you don't need to stub any of its methods, just supply a mock provider of the data (mock[TableReader]).
Lets say I have bunch of car objects in my project, for example:
object Porsche extends Car {
override def start() {...}
override def canStart(fuelInLitr: Int) = fuelInLitr > 5
override val fuelInLitr = 45
override val carId = 1234567
}
im extending Car which is just a trait to set a car structure:
trait Car {
def start(): Unit
val canStart(fuel: Double): Boolean
val fuelInLitr: Int
val carId: Int
}
Now, in the start() method I want to use some api service that will give me a car key based on its id so I cant start the car.
So I have this CarApiService:
class CarApiService (wsClient: WSClient, configuration: Configuration) {
implicit val formats: Formats = DefaultFormats
def getCarkey(carId: String): Future[Option[CarKey]] = {
val carInfoServiceApi = s"${configuration.get[String]("carsdb.carsInfo")}?carId=$carId"
wsClient.url(carInfoServiceApi).withHttpHeaders(("Content-Type", "application/json")).get.map { response =>
response.status match {
case Status.OK => Some(parse(response.body).extract[CarKey])
case Status.NO_CONTENT => None
case _ => throw new Exception(s"carsdb failed to perform operation with status: ${response.status}, and body: ${response.body}")
}
}
}
}
I want to have the ability to use getCarkey() in my car objects, so I created a CarsApiServicesModule which will give my access to the carApiService and I can use its methods:
trait CarsApiServicesModule {
/// this supply the carApiService its confuguration dependancy
lazy val configuration: Config = ConfigFactory.load()
lazy val conf: Configuration = wire[Configuration]
/// this supply the carApiService its WSClient dependancy
lazy val wsc: WSClient = wire[WSClient]
lazy val carApiService: CarApiService = wire[CarApiService]
}
and now I want to add mix this trait in my car object this way:
object Porsche extends Car with CarsApiServicesModule {
// here I want to use myApiService
// for example: carApiService.getCarkey(carId)...
}
but when compiling this I get this error:
does anyone know what is the issue?
also, is that design make sense?
You need to keep in mind that wire is just a helper macro which tries to generate new instance creation code: it's quite dumb, in fact. Here, it would try to create a new instance of WSClient.
However, not all objects can be instantiated using a simple new call - sometimes you need to invoke "factory" method.
In this case, if you take a look at the readme on GitHub, you'll see that to instantiate the WSClient, you need to create it through the StandaloneAhcWSClient() object.
So in this case, wire won't help you - you'll need to simply write the initialisation code by hand. Luckily it's not too large.
Scala functions has following methods for chaining:
fn1.andThen(fn2)
fn1.compose(fn2)
But how can be written this case:
I have function cleanUp() which has to be called always as last step.
And I have a bunch of other functions, like that:
class Helper {
private[this] val umsHelper = new UmsHelper()
private[this] val user = umsHelper.createUser()
def cleanUp = ... // delete user/ and other entities
def prepareModel(model: TestModel) = {
// create model on behalf of the user
}
def commitModel() = {
// commit model on behalf of the user
}
}
And some external code can use code something like this:
val help = new Helper()
help.prepareModel()
help.commitModel()
// last step should be called implicitly cleanUp
How this can be written in a functional way, that chaining will always
call cleanUp function implicitly as last step?
Note: I see it as analogue of destructor in C++. Some chaining (doesn't matter how this chain is done) fn1 andLater fn2 andLater fn3 have to call as last step cleanUp (fn1 andLater fn2 andLater fn3 andLater cleanUp). Wrong with directly writing cleanUp method is there is a big chance someone will miss this step and user will be leaked (will be stayed in database)
This is a more advanced alternative:
When you hear "context" and "steps", there's a functional pattern that directly comes to mind: Monads. Rolling up your own monad instance can simplify the user-side of putting valid steps together, while providing warranties that the context will be cleaned up after them.
Here, we are going to develop a "CleanableContext" construction that follows that pattern.
We base our construct on the most simple monad, one whose only function is to hold a value. We're going to call that Context
trait Context[A] { self =>
def flatMap[B](f:A => Context[B]): Context[B] = f(value)
def map[B](f:A => B): Context[B] = flatMap(f andThen ((b:B) => Context(b)))
def value: A
}
object Context {
def apply[T](x:T): Context[T] = new Context[T] { val value = x }
}
Then we have a CleanableContext, which is capable of "cleaning up after itself" provided some 'cleanup' function:
trait CleanableContext[A] extends Context[A] {
override def flatMap[B](f:A => Context[B]): Context[B] = {
val res = super.flatMap(f)
cleanup
res
}
def cleanup: Unit
}
And now, we have an object that's able to produce a cleanable UserContext that will take care of managing the creation and destruction of users.
object UserContext {
def apply(x:UserManager): CleanableContext[User] = new CleanableContext[User] {
val value = x.createUser
def cleanup = x.deleteUser(value)
}
}
Let's say that we have also our model and business functions already defined:
trait Model
trait TestModel extends Model
trait ValidatedModel extends Model
trait OpResult
object Ops {
def prepareModel(user: User, model: TestModel): Model = new Model {}
def validateModel(model: Model): ValidatedModel = new ValidatedModel {}
def commitModel(user: User, vmodel: ValidatedModel): OpResult = new OpResult {}
}
Usage
With that reusable machinery in place, our users can express our process in a succinct way:
import Ops._
val ctxResult = for {
user <- UserContext(new UserManager{})
validatedModel <- Context(Ops.prepareModel(user, testModel)).map(Ops.validateModel)
commitResult <- Context(commitModel(user, validatedModel))
} yield commitResult
The result of the process is still encapsulated, and can be taken "out" from the Context with the value method:
val result = ctxResult.value
Notice that we need to encapsulate the business operations into a Context to be used in this monadic composition. Note as well that we don't need to manually create nor cleanup the user used for the operations. That's taken care of for us.
Furthermore, if we needed more than one kind of managed resource, this method could be used to take care of managing additional resources by composing different contexts together.
With this, I just want to provide another angle to the problem. The plumbing is more complex, but it creates a solid ground for users to create safe processes through composition.
I think that the core of the question is "how to keep a resource within a managed context". i.e. provide users with a way to use the resource and prevent it to 'leak' outside its context.
One possible approach is to provide a functional access to the managed resource, where the API requires functions to operate over the resource in question. Let me illustrate this with an example:
First, we define the domain of our model: (I've added some subtypes of Model to make the example more clear)
trait User
trait Model
trait TestModel extends Model
trait ValidatedModel extends Model
trait OpResult
// Some external resource provider
trait Ums {
def createUser: User
def deleteUser(user: User)
}
Then we create a class to hold our specific context.
class Context {
private val ums = new Ums{
def createUser = new User{}
def deleteUser(user: User) = ???
}
def withUserDo[T](ops: User => T):T = {
val user = ums.createUser
val result = ops(user)
ums.deleteUser(user)
result
}
}
The companion object provides (some) operations on the managed resource. Users can provide their own functions as well.
object Context {
def prepareModel(model: TestModel): User => Model = ???
val validateModel: Model => ValidatedModel = ???
val commitModel: ValidatedModel => OpResult = ???
}
We can instantiate our context and declare operations on it, using a classic declaration, like:
val ctx = new Context
val testModel = new TestModel{}
val result = ctx.withUserDo{ user =>
val preparedModel = prepareModel(testModel)(user)
val validatedModel = validateModel(preparedModel)
commitModel(validatedModel)
}
Or, given the desire in the question to use functional composition, we could rewrite this as:
val result = ctx.withUserDo{
prepareModel(testModel) andThen validateModel andThen commitModel
}
Use autoClean this will automatically call cleanUp at the end.
create a HelperStuff trait which contains all the necessary functions.
Inside the Helper object create a private implementation of the HelperStuff and then have a method method called autoClean which does the work keeping the Helper instance private and safe way from the rouge users.
Helper.autoClean { helperStuff =>
//write all your code here. clean up will happen automatically
helper.foo()
helper.commitModel()
}
Here is the autoClean function for you
trait HelperStuff {
def foo(): Unit
def commitModel: Unit
def cleanUp(): Unit
}
object Helper {
private class Helper extends HelperStuff {
def foo(): Unit = println("foo")
def cleanUp(): Unit = println("cleaning done")
}
private val helper = new Helper()
def autoClean[T](code: HelperStuff => T): T = {
val result = code(helper)
helper.cleanUp()
result
}
}
I've built a microservice using Scala and Play and now I need to create a new version of the service that returns the same data as the previous version of the service but in a different JSON format. The service currently uses implicit Writes converters to do this. My controller looks something like this, where MyJsonWrites contains the implicit definitions.
class MyController extends Controller with MyJsonWrites {
def myAction(query: String) = Action.async {
getData(query).map {
results =>
Ok(Json.toJson(results))
}
}
}
trait MyJsonWrites {
implicit val writes1: Writes[SomeDataType]
implicit val writes2: Writes[SomeOtherDataType]
...
}
Now I need a new version of myAction where the JSON is formatted differently. The first attempt I made was to make MyController a base class and have subclasses extend it with their own trait that has the implicit values. Something like this.
class MyNewContoller extends MyController with MyNewJsonWrites
This doesn't work though because the implicit values defined on MyNewJsonWrites are not available in the methods of the super class.
It would be ideal if I could just create a new action on the controller that somehow used the converters defined in MyNewJsonWrites. Sure, I could change the trait to an object and import the implicit values in each method but then I'd have to duplicate the method body of myAction so that the implicits are in scope when I call Json.toJson. I don't want to pass them as implicit parameters to a base method because there are too many of them. I guess I could pass a method as a parameter to the base method that actually does the imports and Json.toJson call. Something like this. I just thought maybe there'd be a better way.
def myBaseAction(query: String, toJson: Seq[MyResultType] => JsValue) = Action.async {
getData(query).map {
results =>
Ok(Json.toJson(results))
}
}
def myActionV1(query: String) = {
def toJson(results: Seq[MyResultType]) = {
import MyJsonWritesV2._
Json.toJson(results)
}
myBaseAction(query, toJson)
}
Instead of relying on scala implicit resolution, you can call your writes directly:
def myBaseAction(query: String, writes: Writes[MyResultType]) = Action.async {
getData(query).map { results =>
val seqWrites: Writes[Seq[MyResultType]] = Writes.seq(writes)
Ok(seqWrites.writes(results))
}
}
def myActionV1(query: String) = myBaseAction(query, MyJsonWritesV1)
def myActionV2(query: String) = myBaseAction(query, MyJsonWritesV2)
I have some functions that access a database, which I need to mock for testing purposes.
For ease of use, I would like to define these functions within another function, where I can leverage scope to reduce the number of arguments I have to pass.
I need to test the parent function, while mocking the nested functions.
Are there any tricks to mock functions that are nested?
As a secondary question, are there ways to mock functions when nested at arbitrary depth?
And a side note: my project is light enough I'm not even using classical mocking, just stackable traits like this blog post suggests; but for this question, any kind of mocking is fine.
Here is some very simple example code:
class Storage {
def storeData(specId: Long, data: String): Unit = {
val rawPath = "/path/to/file"
def storeFsEntry: Unit = {
// do stuff
}
def storeDbEntry: Unit = {
// do stuff we need mocked
}
if ( specId == 1 )
{
storeDbEntry
storeFsEntry
}
}
}
It's not possible, but you can define a trait and implement it inside your function (if you really want this logic been implemented inside):
class Storage {
trait Storing {
def path: String //you have to define all free members
def storeDbEntry: Unit
def storeFsEntry: Unit
}
def storeData(specId: Long, data: String, storingDefault: Option[Storing] = None): Unit = {
val myStoring = new Storing {
...implement it here
}
val storing = storingDefault getOrElse myStoring
import storing._
if ( specId == 1 ) {
storeDbEntry
storeFsEntry
}
}
}
Your mock will be something like that:
trait StorageMock extends Storage {
override def storeData(specId: Long, data: String, storingDefault: Option[Storing] = None): Unit = {
val storingOverride = new Storing { ... } //mocking functionality
super.storeData(specId, data, storingDefault orElse Some(storingOverride))
}
}
new Storage with StorageMock
You may also turn storingDefault into a member instead of function parameter.
The reason why it's not possible to do same for inner functions is that they are private and also typecheck can't be performed on them (in comparison with inner traits).