I'm using scalatest_2.11 version 2.2.1. I'm trying to write test that run on a SparkContext. How can I initiate a SparkContext when the tests begin, take this Sc thorugh all the tests and then stop it when they are all done?
(I know that it suppose to stop by itself, but I'sd still like to do it myself)
You can use BeforeAndAfterAll from ScalaTest. Define a base trait that starts and stops the SparkContext and use it with your other tests.
trait SparkTest extends BeforeAndAfterAll {
self: Suite =>
#transient var sc: SparkContext = _
override def beforeAll {
val conf = new SparkConf().
setMaster("local[*]").
setAppName("test")
sc = new SparkContext(conf)
super.beforeAll()
}
override def afterAll: Unit = {
try {
sc.stop()
} finally {
super.afterAll
}
}
}
// Mix-in the trait with your tests like below.
class MyTest extends FunSuite with SparkTest {
test("my test") {
// you can access "sc" here.
}
}
Related
I am using Play 2.6 and Slick 3.2 with MySql database. But the problem is that I am not able to write test cases using h2 database. The problem is that, by using guice, I am not able to inject DatabaseConfigProvider in my repo. Following is my repo class implementation:
class CompanyRepoImpl #Inject() (dbConfigProvider: DatabaseConfigProvider)
(implicit ec: ExecutionContext) extends CompanyRepo {
val logger: Logger = LoggerFactory.getLogger(this.getClass())
private val dbConfig = dbConfigProvider.get[JdbcProfile]
import dbConfig._
import profile.api._
private val company = TableQuery[Companies]
------
}
My test case class:
class CompanyRepoSpec extends AsyncWordSpec with Matchers with BeforeAndAfterAll {
private lazy val injector: Injector = new GuiceApplicationBuilder()
.overrides(bind(classOf[CompanyRepo]).to(classOf[CompanyRepoImpl]))
.in(Mode.Test)
.injector()
private lazy val repo: CompanyRepo = injector.instanceOf[CompanyRepo]
private lazy val dbApi = injector.instanceOf[DBApi]
override protected def beforeAll(): Unit = {
Evolutions.applyEvolutions(database = dbApi.database("test"))
}
override protected def afterAll(): Unit = {
Evolutions.cleanupEvolutions(database = dbApi.database("test"))
}
-----------------------
}
Testing application.test.conf
play.evolutions.db.test.enabled=true
play.evolutions.autoApply=true
slick.dbs.test.profile="slick.jdbc.H2Profile$"
slick.dbs.test.db.driver="org.h2.Driver"
slick.dbs.test.db.url="jdbc:h2:mem:test;MODE=MySQL"
May be, my configuration are not valid for testing, so, how can we write test cases using scala, slick and play.
In "Dependency Injecting Actors" it's shown how to inject a parameter into the constructor of a child actor. The parent actor uses injectedChild to be allowed to pass to the child (at child creation time) only the non-injected parameter and then let Guice inject the rest. To do this, it extends InjectedActorSupport and gets the child's factory injected in the constructor:
class MyParent #Inject() (childFactory: MyChild.Factory,
#Assisted something: Something,
#Assisted somethingElse: SomethingElse) extends Actor with InjectedActorSupport
[..]
val child: ActorRef = injectedChild(childFactory(something, somethingElse), childName)
But what about the class that starts the parent and is not an actor but a custom ApplicationLoader?
How can I start the parent actor from there? No mention of this is in the documentation.
I tried doing the same for the loader as I did for parent:
class MyLoader #Inject() (parentFactory: MyParent.Factory) extends ApplicationLoader with Actor with InjectedActorSupport {
[..]
val parent = injectedChild(parentFactory(something, somethingElse), parentName)
would this be correct? How can I test it?
class MyModule extends AbstractModule with AkkaGuiceSupport {
def configure = {
bindActor[MyParent](parentName)
bindActor[MyLoader](loaderName)
bindActorFactory[MyChild, MyChild.Factory]
bindActorFactory[MyParent, MyParent.Factory]
}
}
So:
How do I start the parent from MyLoader while letting Guice dependency-inject what's required?
How can I test MyLoader?
This has been my test so far but now I need to pass the injected thingy to MyLoader and I don't know how (note the ***???**** in place of the argument which I do not know where to find):
class MyLoaderSpec(_system: ActorSystem, implicit val ec: ExecutionContext) extends TestKit(_system) with WordSpecLike with BeforeAndAfterAll with Matchers {
val loader = new SimstimLoader(???)
override def beforeAll(): Unit = {
loader.load(ApplicationLoader.createContext(new Environment(new File("."), ApplicationLoader.getClass.getClassLoader, Mode.Test)))
}
Thanks a million in advance!
Here is how I solved this issue.
--> How to start a parent actor who needs dependency-injection.
First of all, manually starting such an actor is impossible if you, like me, need to dependency-inject an instance which you do not know how to pass and where from. The solution is to let Guice start the actor automagically. Here is how.
First, create your binder module for Guice:
class MyModule extends AbstractModule with AkkaGuiceSupport{
override def configure(): Unit = {
bindActor[Root](Root.NAME)
bind(classOf[StartupActors]).asEagerSingleton()
}
}
Then, tell Play where your binder module is located by adding the following in your conf/application.conf:
play.modules={
enabled += "my.path.to.MyModule"
}
The StartupActors is simply a class I use to log whenever the automagic start of dependency-injected actors actually takes place. I log the event so that I can be sure of when and whether it occurs:
class StartupActors #Inject() (#Named(Root.NAME) root: ActorRef) {
play.api.Logger.info(s"Initialised $root")
}
The Root actor in my case takes care of parsing a custom configuration. Since the resulting vars from the parsing is required by my parent actor and during the tests I need to mock such resulting vars, I delegate the parsing to an actor other than the parent actor, i.e., the Root actor:
object Root {
final val NAME = "THERoot"
case class ParseConfiguration()
}
class Root #Inject()(configuration: Configuration, projectDAO: ProjectDAO) extends Actor {
val resultingVar: Something = myConfigParsing()
override def preStart(): Unit = {
context.actorOf(Props(new MyParent(resultingVar: Something, somethingElse: SomethingElse, projectDAO: ProjectDAO)))
}
override def receive: Receive = {
case ParseConfiguration => sender ! myConfigParsing()
case _ => logger.error("Root actor received an unsupported message")
}
}
The ParseConfiguration message is used uniquely for testing purposes. Normally the configuration parsing occurs instead because of the initialisation of the resultingVar attribute.
This way, MyParent wont need to get anything injected. Only StartupActors and Root will get injected. MyParent will simply get projectDAO from Root and pass it on to all its children.
class MyParent(something: Something, somethingElse: SomethingElse, projectDAO: ProjectDAO) extends Actor { ... }
Finally, for completion, I'm reporting here how I wrote the tests since I had troubles finding enough information online around this as well.
import akka.actor.{ActorRef, ActorSystem, Props}
import akka.testkit.{TestKit, TestProbe}
import com.typesafe.config.ConfigFactory
import org.mockito.Mockito.mock
import org.scalatest.{BeforeAndAfterAll, WordSpecLike}
import org.specs2.matcher.MustMatchers
import play.api.Configuration
import scala.concurrent.ExecutionContext
class RootSpec(_system: ActorSystem) extends TestKit(_system)
with WordSpecLike with BeforeAndAfterAll with MustMatchers {
implicit val ec: ExecutionContext = scala.concurrent.ExecutionContext.global
val conf: com.typesafe.config.Config = ConfigFactory.load()
val configuration: Configuration = Configuration(conf)
val projectDAOMock: ProjectDAO = mock(classOf[ProjectDAO])
private var mainActor: ActorRef = _
private var something: Something = Something.empty
def this() = this(ActorSystem("MySpec"))
override def afterAll: Unit = {
system.shutdown()
}
override def beforeAll(): Unit = {
mainActor = system.actorOf(Props(new Root(configuration, projectDAOMock)), Root.NAME)
}
"RootSpec: Root Actor" should {
val probe = TestProbe()
"successfully parse the configuration file" in {
probe.send(mainActor, ParseConfiguration)
something = probe.expectMsgPF() {
case msg => msg.asInstanceOf[Something]
}
}
}
}
and then I test MyParent by conveniently providing mock objects in place of vars resulting from the configuration parsing:
import akka.actor.{ActorRef, ActorSystem, Props}
import akka.testkit.{TestKit, TestProbe}
import org.mockito.Mockito
import org.mockito.Mockito._
import org.scalatest.{BeforeAndAfterAll, WordSpecLike}
import org.specs2.matcher.MustMatchers
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.{ExecutionContext, Future}
case class AnyProjectAPI(val projectAPI: ProjectAPI) extends AnyVal
class MyParentSpec(_system: ActorSystem, implicit val ec: ExecutionContext) extends TestKit(_system)
with WordSpecLike with BeforeAndAfterAll with MustMatchers {
val something = mock(classOf[Something])
val somethingElse = mock(classOf[somethingElse])
val projectDAOMock: ProjectDAO = mock(classOf[ProjectDAO])
val projectTest: ProjectAPI = new ProjectAPI(allMyRandomConstructorArguments),
val projectsList: List[ProjectAPI] = List(projectTest)
val expectedCreationId = 1
private var parent: ActorRef = _
def this() = this(ActorSystem("MySpec"), scala.concurrent.ExecutionContext.global)
override def afterAll: Unit = {
system.shutdown()
}
override def beforeAll(): Unit = {
parent = system.actorOf(Props(new MyParent(something, somethingElse, projectDAOMock)), MyParent.NAME)
}
"MyParentTesting: parent's pull request" should {
when(myProjApi.getAllProjects).thenReturn(Future {projectsList})
val anyProject: AnyProjectAPI = AnyProjectAPI(org.mockito.Matchers.any[ProjectAPI])
Mockito.when(projectDAOMock.create(org.mockito.Matchers.any[ProjectAPI]))
.thenReturn(Future {expectedCreationId}: Future[Int])
val probe = TestProbe()
val probe1 = TestProbe()
"be successfully satisfied by all children when multiple senders are waiting for an answer" in {
probe.send(parent, UpdateProjects)
probe1.send(parent, UpdateProjects)
allChildren.foreach(child =>
probe.expectMsg(expectedCreationId))
allChildren.foreach(child =>
probe1.expectMsg(expectedCreationId))
}
}
}
I have the following #Singleton in my Play for Scala application that loads on startup:
#Singleton
class Scheduler #Inject()(#Named("mainEtl") mainEtl: ActorRef, system: ActorSystem) {
// some code
}
This is the module where Scheduler is declared. The module is enabled in application.conf:
class Module extends AbstractModule {
def configure() = {
bind(classOf[Scheduler]).asEagerSingleton
}
}
And the related module definition to configure the #Named injected object, also declared in application.conf:
class AkkaBindings extends AbstractModule with AkkaGuiceSupport {
def configure = {
bindActor[MainEtl]("mainEtl")
}
}
When I run any ScalaTest test, apparently the singleton starts running because I get an error saying that it doesn't find MainEtl (the object injected in the Scheduler class). The point is that I don't need to run the singleton for my tests, so I need to disable it.
This is how I invoke the Play application in my tests:
class ManageBanksTest extends PlaySpec with OneAppPerSuite with MockitoSugar {
implicit override lazy val app = new GuiceApplicationBuilder().build
// more test code
}
This is how I tried to disable it, but it doesn't work as I get the same error:
implicit override lazy val app = new GuiceApplicationBuilder()
.disable[Scheduler]
.build
Alternatively, I could mock Scheduler, but I would have to mock also the #Named injected object and I couldn't find information on how to achieve that.
Any ideas?
This is the solution: to disable the Module class not to declare Scheduler as singleton:
implicit override lazy val app = new GuiceApplicationBuilder()
.disable[Module]
.build
import com.google.inject.AbstractModule
import com.google.inject.name.Names
import org.specs2.mock.Mockito
import play.api.inject.guice.{GuiceApplicationBuilder, GuiceableModule}
val modules = Option(new AbstractModule {
override def configure() = {
val mockMainETL = mock[MainEtl]
bind(classOf[ActorRef])
.annotatedWith(Names.named("mainEtl"))
.toInstance(mockMainETL)
val mock1 = mock[ManageBanksDAO]
mock1.readMany answers { _ => Future{seqMany}}
val mockManageBanks = mock[ManageBanks]
bind(classOf[ManageBanks]).toInstance(new ManageBanks(mock1))
}
})
lazy val app = new GuiceApplicationBuilder()
.overrides(modules.map(GuiceableModule.guiceable).toSeq: _*)
.build
Try configuring your mock inside modules and add those modules while initializing your application. This will inject custom mocks.
Also instead of this :
val controller = new ManageBanks(mock1)
Try this:
val controller = app.injector.instanceOf(classOf[ManageBanks])
I want to create tests for my Play Framework Application and I continue getting java.lang.RuntimeException: There is no started application. I have an Asynchronous Controller like this:
class ComputerController #Inject()(computerService: ComputerService)(implicit executionContext: ExecutionContext){
def add = Action.async {
ComputerForm.form.bindFromRequest.fold(
errorForm => Future.successful(Ok(errorForm.toString)),
data => {
val ip = data.ip
val name = data.name
val user = data.user
val password = data.password
val computer = Computer(ip,name,user,password)
val futureTask = computerService.add(newComputer)
futureTask.map(res => Redirect(routes.HomeController.home()))
}
)
}
}
A helper trait for injecting:
trait Inject {
val injector = new GuiceApplicationBuilder()
.in(new File("conf/application.conf").
.in(Mode.Test)
.injector
}
And the tests is like this:
class ComputerControllerSpec extends PlaySpec with Inject with MockitoSugar with ScalaFutures {
lazy val computerService = mock[ComputerService]
when(computerService.add(any[Computer])) thenReturn Future.successful("Computer added")
implicit lazy val executionContext = injector.instanceOf[ExecutionContext]
val controller = new ComputerController(computerService)
"Computer Controller" should {
"add a new computer" in {
val computer = ComputerFormData("127.0.0.1","Computer","user","password")
val computerForm = ComputerForm.form.fill(computer)
val result = controller.add.apply {
FakeRequest()
.withFormUrlEncodedBody(computerForm.data.toSeq: _*)
}
val bodyText = contentAsString(result)
bodyText mustBe ""
}
}
}
I have also tried:
Initializing the executionContext implicit value with ExecutionContext.global instead and got java.lang.RuntimeException: There is no started application.
Adding with OneAppPerSuite to ComputerControllerSpec and got: akka.actor.OneForOneStrategy - Cannot initialize ExecutionContext; AsyncExecutor already shut down
Changing "add a new computer" in { for "add a new computer" in new WithApplication { and got: java.lang.RuntimeException: There is no started application
I don't really know how to inject that implicit ExecutionContext to my tests.
I guess you are missing "new WithApplication()"
Try this
class ComputerControllerSpec extends PlaySpec with Inject with MockitoSugar with ScalaFutures {
lazy val computerService = mock[ComputerService]
when(computerService.add(any[Computer])) thenReturn Future.successful("Computer added")
implicit lazy val executionContext = injector.instanceOf[ExecutionContext]
val controller = new ComputerController(computerService)
"Computer Controller" should {
"add a new computer" in new WithApplication() {
val computer = ComputerFormData("127.0.0.1","Computer","user","password")
val computerForm = ComputerForm.form.fill(computer)
val result = controller.add.apply {
FakeRequest()
.withFormUrlEncodedBody(computerForm.data.toSeq: _*)
}
val bodyText = contentAsString(result)
bodyText mustBe ""
}
}
}
The way this test got to work was:
Extending PlaySpec with MockitoSugar and BeforeAndAfterAll
Overwriting:
// Before all the tests, start the fake Play application
override def beforeAll() {
application.startPlay()
}
// After the tests execution, shut down the fake application
override def afterAll() {
application.stopPlay()
}
And then all the tests run without the thrown exception.
Hey i want to build some small Funsuite test for akka actor application but after combining Testkit with FunSuiteLike i cant call th test anymore.
Somebody an idea why this is happening? is Testkit and funsuite not compatible?
import org.scalatest.{FunSuiteLike, BeforeAndAfterAll}
import akka.testkit.{ImplicitSender, TestKit, TestActorRef}
import akka.actor.{ActorSystem}
class ActorSynchroTest(_system: ActorSystem) extends TestKit(_system) with FunSuiteLike with BeforeAndAfterAll with ImplicitSender{
val actorRef = TestActorRef(new EbTreeDatabase[Int])
val actor = actorRef.underlyingActor
//override def afterAll = TestKit.shutdownActorSystem( system )
test("EbTreeDatabase InsertNewObject is invoked"){
val idList = List(1024L,1025L,1026L,1032L,1033L,1045L,1312L,1800L)
idList.
foreach(x => actorRef ! EbTreeDataObject[Int](x,x,1,None,null))
var cursor:Long = actor.uIdTree.firstKey()
var actorItems:List[Long] = List(cursor)
while(cursor!=actor.uIdTree.lastKey()){
cursor = actor.uIdTree.next(cursor)
cursor :: actorItems
}
assert(idList.diff(actorItems) == List())
}
}
The intelliJ idea test enviroment says:
One or more requested classes are not Suites: model.ActorSynchroTest
class ActorSynchroTest extends TestKit(ActorSystem("ActorSynchroTest",ConfigFactory.parseString(ActorSynchroTest.config)))
with DefaultTimeout with ImplicitSender
with FunSuiteLike with Matchers with BeforeAndAfterAll {
...
}
object ActorSynchroTest {
// Define your test specific configuration here
val config = """
akka {
loglevel = "WARNING"
}
"""
}
Different initialization of the testkit worked in the end before the standard config was used which didn't fit