ThreadPoolExecution error with scala, slick, specs2 - scala

I am trying to write a db unit test for my module using slick and specs2 in scala language. I have written two tests at the moment but the second one fails with ThreadPoolExecution error.
I have tried many approaches but still can't manage to solve my problem.
Any help would be greatly appreciated.
Here is the test code:
class UserClientSpecs extends Specification{
"App should " should {
"add value to database" in new WithApplication {
println("before first test")
val recordEntry = new UserClient(None, "Lohs_atkal", 2)
val newRecord = UserService.addUser(recordEntry)
newRecord.onComplete {
case Success(value) => println(s"Got the callback, meaning = $value")
case Failure(e) => println(e.getMessage)
}
newRecord should not beNull
val count = UserService.listAllUsers.map {
v =>
println("the number of database entries are " + v.length)
}
}
"delete a record" in new WithApplication {
println("before second test")
val recordEntry = new UserClient(Some(0), "Lielaks Lohs", 5)
val newRecord = UserService.addUser(recordEntry)
newRecord.map {
v => println("newRecord value", v)
}
newRecord.onComplete {
case Success(value) => println(s"Got the callback, meaning = $value")
case Failure(e) => println(e.getMessage)
}
val recordEntry2 = new UserClient(Some(1), "Lielaks Lohs2", 50)
val newRecord2 = UserService.addUser(recordEntry2)
val countOne = UserService.listAllUsers.map {
res =>
println(res.length)
}
val deleteUser = UserService.deleteUser(1)
val countTwo = UserService.listAllUsers.map {
res =>
res should_==(1)
res should !==(2)
}
}
}
}
And the error I am getting when running my test through sbt -> testOnly:
[play-scala] $ testOnly models.UserClientSpecs
[info] UserClientSpecs
[info]
before first test
Got the callback, meaning = Some(UserClient(None,Lohs_atkal,2))
the number of database entries are 1
[info] App should should
[info] + add value to database
before second test
Task slick.backend.DatabaseComponent$DatabaseDef$$anon$2#6d01394 rejected from java.util.concurrent.ThreadPoolExecutor#7fa5be9b[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 2]
[info] + delete a record
[info]
[info] Total for specification UserClientSpecs
[info] Finished in 2 seconds, 739 ms
[info] 2 examples, 0 failure, 0 error
[info]
[info] Passed: Total 2, Failed 0, Errors 0, Passed 2
[success] Total time: 8 s, completed Mar 10, 2016 1:40:10 PM

Related

How to mock TemporaryFile moveTo() method?

I'm trying to create a test case for the controller below but I'm encountering some issues that I believe to be related to mocking methods correctly.
Issue:
Every time my test runs through this line:
file.ref.moveTo(new File(s"${configuration.uploadFileLocation}/$filename").getCanonicalFile,true)
It throws the following error:
Error:
[info] - run test *** FAILED ***
[info] java.lang.NullPointerException:
[info] at java.nio.file.Files.provider(Files.java:97)
[info] at java.nio.file.Files.move(Files.java:1392)
[info] at play.api.libs.Files$TemporaryFile.moveTo(Files.scala:100)
Controller:
def run = Action.async(parse.multipartFormData) { implicit request =>
request.body.file("file").map { file =>
import java.io.File
val filename = randomFileName
val contentType = file.contentType
if (contentType == Some("text/csv")) {
file.ref.moveTo(new File(s"${configuration.uploadFileLocation}/$filename").getCanonicalFile, true)
val result = apiClient.match (filename)
Future.successful("ok")
} else {
Future.successful("ok")
}
}.getOrElse {
Future.successful("ok")
}
}
Test case:
test("run() test") {
val mockMessageApi = mock(classOf[MessagesApi])
val mockApiClient = mock(classOf[apiClient])
val mockConfiguration = mock(classOf[Configuration])
val mockFile = mock(classOf[File])
val buildFakeRequest = FakeRequest(POST, "", FakeHeaders(), {
val tempFile = new TemporaryFile(mockFile)
val filePart = FilePart("file", "test.file", Some("text/csv"), tempFile)
MultipartFormData(Map(), List(filePart), List())
}
)
val controller = new MultipleMatch(mockApiClient, mockMessageApi, mockConfiguration)
val response = controller.run()(buildFakeRequest)
}
Question:
How can I effectively mock the ref and the moveTo method? What is causing this to happen? How to avoid the same issue again in the future?

Non-Terminating ZeroMQ Load Balancer

Given the following slightly modified Load Balancer ZeroMQ code:
package net.broker
import org.zeromq.ZMQ
object LruQueue2 {
class ClientTask(name: String) extends Runnable {
override def run(): Unit = {
val context = ZMQ.context(1)
val client = context.socket(ZMQ.REQ)
client.setIdentity(name.getBytes)
client.connect("tcp://localhost:5555")
// send request, get reply
client.send("HELLO".getBytes, 0)
val reply = client.recv(0)
println(s"${new String(client.getIdentity)} received: ${new String(reply)}")
}
}
class WorkerTask(name: String) extends Runnable {
override def run(): Unit = {
val context = ZMQ.context(1)
val worker = context.socket(ZMQ.REQ)
worker.connect("tcp://localhost:5556")
worker.setIdentity(name.getBytes)
worker.send("READY".getBytes, 0)
while(true) {
val clientAddr = worker.recv(0)
val empty = worker.recv(0)
val clientMsg = worker.recv(0)
worker.send(clientAddr, ZMQ.SNDMORE)
worker.send("".getBytes, ZMQ.SNDMORE)
worker.send("WORLD".getBytes, 0)
println(s"${new String(worker.getIdentity)}: 3-frames to client: ${new String(clientAddr)}")
}
}
}
def main(args: Array[String]): Unit = {
val NOFLAGS = 0
// worker using REQ socket to do LRU routing
val NBR_CLIENTS = 1
val NBR_WORKERS = 1
val context = ZMQ.context(1)
val frontend = context.socket(ZMQ.ROUTER)
val backend = context.socket(ZMQ.ROUTER)
frontend.bind("tcp://*:5555")
backend.bind("tcp://*:5556")
val clients = (1 to NBR_CLIENTS).toList.map{ i => new Thread(new ClientTask(s"CLIENT$i"))}
val workers = (1 to NBR_CLIENTS).toList.map{ i => new Thread(new WorkerTask(s"WORKER$i"))}
clients.foreach(_.start)
workers.foreach(_.start)
val workerQueue = scala.collection.mutable.Queue[Array[Byte]]()
val poller = context.poller(2)
poller.register(backend, ZMQ.Poller.POLLIN)
poller.register(frontend, ZMQ.Poller.POLLIN)
var clientNbr = NBR_CLIENTS
while(true) {
println("begin to poll")
poller.poll()
println("done polling")
println("clientNbr:" + clientNbr)
println("workerQueue.length: " + workerQueue.length)
if(clientNbr == 0) {
sys.exit(0)
}
else if(poller.pollin(0) && clientNbr > 0) {
val workerAddr = backend.recv(NOFLAGS)
val empty = backend.recv(NOFLAGS)
val clientAddrOrReadyMsg = backend.recv(NOFLAGS)
workerQueue.enqueue(workerAddr)
if(new String(clientAddrOrReadyMsg) == "READY") {
// nothing to do - worker is letting us know that he's ready to work
}
else {
// retrieve remaining 2 frames of client message
// [Empty][Client Message of "HELLO"]
val empty = backend.recv(0)
val workerResponse = backend.recv(0)
frontend.send(clientAddrOrReadyMsg, ZMQ.SNDMORE)
frontend.send("".getBytes, ZMQ.SNDMORE)
frontend.send(workerResponse, NOFLAGS)
clientNbr -= 1
}
}
else if (poller.pollin(1) && workerQueue.nonEmpty) {
val clientAddr = frontend.recv(0)
val empty = frontend.recv(0)
val clientMsg = frontend.recv(0)
backend.send(workerQueue.dequeue(), ZMQ.SNDMORE)
backend.send("".getBytes, ZMQ.SNDMORE)
backend.send(clientAddr, ZMQ.SNDMORE)
backend.send("".getBytes, ZMQ.SNDMORE)
backend.send(clientMsg, NOFLAGS)
}
else {}
}
}
}
When I run the above code, I see the following output:
[info] Running net.broker.LruQueue2
[info] begin to poll
[info] done polling
[info] clientNbr:1
[info] workerQueue.length: 0
[info] begin to poll
[info] done polling
[info] clientNbr:1
[info] workerQueue.length: 1
[info] begin to poll
[info] WORKER1: 3-frames to client: CLIENT1
[info] done polling
[info] clientNbr:1
[info] workerQueue.length: 0
[info] begin to poll
[info] CLIENT1 received: WORLD
It appears that it's stuck on poller.poll(), per the output print statements.
How can I debug this behavior?

Subscribe to mongo query does not work

I use playframework with "org.mongodb.scala" %% "mongo-scala-driver" % "1.0.1" and have that code:
val collection = Mongo.db.getCollection("regionAuth")
def getRegions = {
val find: Observable[Document] = collection.find()
Logger.info("regions searching")
find.subscribe(new Observer[Document] {
override def onError(e: Throwable): Unit = Logger.error("regions error", e)
override def onSubscribe(subscription: Subscription): Unit = Logger.info("subscribed")
override def onComplete(): Unit = Logger.info("regions done")
override def onNext(result: Document): Unit = Logger.info("region accepted")
})
find.map { region =>
Logger.info("region accepted by map")
region
}
find.foreach(_ => Logger.info("region accepted by foreach"))
find.toFuture().onComplete {
case Success(r) => Logger.info("regions accepted as future seq " + r.size)
case Failure(e) => Logger.error("regions error as future", e)
}
find
}
And it prints:
[info] application - regions searching
[info] application - subscribed
[info] application - region accepted by foreach
[info] application - region accepted by foreach
[info] application - region accepted by foreach
[info] application - regions accepted as future seq 3
Why subscribe's events and map do not work?
I hope is not too late.
If this is an stand alone application where nothing else is running, you must wait until the future completes.
...
val future = find.toFuture()
future.onComplete {
case Success(r) => Logger.info("regions accepted as future seq " + r.size)
case Failure(e) => Logger.error("regions error as future", e)
}
scala.concurrent.Await.result(future)
find

Race condition in Slick Code

I have written this slick DAO and its unit test in specs2.
My code has race conditions. When I run the same tests, I get different outputs.
The race conditions exist even though in both the functions I do Await.result(future, Duration.Inf)
DAO
package com.example
import slick.backend.DatabasePublisher
import slick.driver.H2Driver.api._
import scala.concurrent.ExecutionContext.Implicits.global
import slick.jdbc.meta._
import scala.concurrent._
import ExecutionContext.Implicits.global
import scala.concurrent.duration._
case class Person(id: Int, firstname: String, lastname: String)
class People(tag: Tag) extends Table[Person](tag, "PEOPLE") {
def id = column[Int]("PERSON_ID", O.PrimaryKey)
def firstname = column[String]("PERSON_FIRST_NAME")
def lastname = column[String]("PERSON_LAST_NAME")
def * = (id, firstname, lastname) <> (Person.tupled, Person.unapply _)
}
object PersonDAO {
private def createList(numRows: Int) : List[Person] = {
def recFunc(counter: Int, result: List[Person]) : List[Person] = {
counter match {
case x if x <= numRows => recFunc(counter + 1, Person(counter, "test" + counter, "user" + counter) :: result)
case _ => result
}
}
recFunc(1, List[Person]())
}
val db = Database.forConfig("test1")
val people = TableQuery[People]
def createAndPopulate(numRows: Int) = {
val action1 = people.schema.create
val action2 = people ++= Seq(createList(numRows) : _* )
val combined = db.run(action1 andThen action2)
val future1 = combined.map { result =>
result map {x =>
println(s"number of rows inserted $x")
x
}
}
Await.result(future1, Duration.Inf).getOrElse(0)
}
def printAll() = {
val a = people.result
val b = db.run(a)
val y = b map { result =>
result map {x => x}
}
val z = Await.result(y, Duration.Inf)
println(z)
println(z.length)
z
}
}
Unit Test
import org.specs2.mutable._
import com.example._
class HelloSpec extends Specification {
"This usecase " should {
"should insert rows " in {
val x = PersonDAO.createAndPopulate(100)
x === 100
}
}
"This usecase " should {
"return 100 rows" in {
val x = PersonDAO.printAll()
val y = PersonDAO.printAll()
y.length === 100
}
}
}
When I run this same code using activator test I see 2 different types of outputs on different runs
sometimes the code gets exception
number of rows inserted 100
[info] HelloSpec
[info]
[info] This usecase should
[info] + should insert rows
[info]
[info] This usecase should
[info] ! return 100 rows
[error] JdbcSQLException: : Table PEOPLE not found; SQL statement:
[error] select x2."PERSON_ID", x2."PERSON_FIRST_NAME", x2."PERSON_LAST_NAME" from "PEOPLE" x2 [42S02-60] (Message.java:84)
[error] org.h2.message.Message.getSQLException(Message.java:84)
[error] org.h2.message.Message.getSQLException(Message.java:88)
[error] org.h2.message.Message.getSQLException(Message.java:66)
Sometimes the 1st function call returns 0 rows and the 2nd function call returns 100 values
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
number of rows inserted 100
Vector()
0
Vector(Person(100,test100,user100), Person(99,test99,user99), Person(98,test98,user98), Person(97,test97,user97), Person(96,test96,user96), Person(95,test95,user95), Person(94,test94,user94), Person(93,test93,user93), Person(92,test92,user92), Person(91,test91,user91), Person(90,test90,user90), Person(89,test89,user89), Person(88,test88,user88), Person(87,test87,user87), Person
I don't understand why does my code have these race conditions because I block on future in each method.
Your assumption that two test cases should run serial, one after the other is not right. The test cases are running parallel. Just use sequential to verify that thats the case.

Not all akka stream Sinks receive the emitted data

When running the following akka streaming FlowGraph not all the emitted Chars are received by all Sinks.
package sample.stream
import java.io.{ FileOutputStream, PrintWriter }
import akka.actor.ActorSystem
import akka.stream.ActorFlowMaterializer
import akka.stream.scaladsl.{ Broadcast, FlowGraph, Sink, Source }
import scala.concurrent.forkjoin.ThreadLocalRandom
import scala.util.{ Failure, Success, Try }
object Sample {
def main(args: Array[String]): Unit = {
println("start")
implicit val system = ActorSystem("Sys")
import system.dispatcher
implicit val materializer = ActorFlowMaterializer()
var counter = -1
val countSource: Source[Char, Unit] = Source(() => Iterator.continually { counter += 1; (counter + 'A').toChar }.take(11))
var counter1 = 0
val consoleSink1 = Sink.foreach[Char] { counter =>
println("sink1:" + counter1 + ":" + counter)
counter1 += 1
Thread.sleep(100)
//Thread.sleep(300)
}
var counter2 = 0
val consoleSink2 = Sink.foreach[Char] { counter =>
println("sink2:" + counter2 + ":" + counter)
counter2 += 1
Thread.sleep(200)
}
val materialized = FlowGraph.closed(consoleSink1, consoleSink2)((x1, x2) => x1) { implicit builder =>
(console1, console2) =>
import FlowGraph.Implicits._
val broadcast = builder.add(Broadcast[Char](2))
countSource ~> broadcast ~> console1
broadcast ~> console2
}.run()
// ensure the output file is closed and the system shutdown upon completion
materialized.onComplete {
case Success(_) =>
system.shutdown()
case Failure(e) =>
println(s"Failure: ${e.getMessage}")
system.shutdown()
}
println("waiting the remaining ones")
//scala.concurrent.Await.ready(materialized, scala.concurrent.duration.DurationInt(100).seconds)
//system.shutdown()
println("end")
}
}
After running the following output is generated
[info] Running sample.stream.Sample
[info] start
[info] waiting the remaining ones
[info] end
[info] sink2:0:A
[info] sink1:0:A
[info] sink1:1:B
[info] sink1:2:C
[info] sink2:1:B
[info] sink1:3:D
[info] sink2:2:C
[info] sink1:4:E
[info] sink1:5:F
[info] sink2:3:D
[info] sink1:6:G
[info] sink1:7:H
[info] sink2:4:E
[info] sink2:5:F
[info] sink1:8:I
[info] sink1:9:J
[info] sink2:6:G
[info] sink2:7:H
[info] sink1:10:K
The second sink doesn't receive the 8th, 9th and 10th values: IJK but still the entire flow is ended.
What should I do to wait for both Sinks to consume all the data?
I discovered that if I change the (x1,x2)=>x1 to (x1,x2)=>x2 this will wait. That is the same with sleeping 300ms in the first sink.
The function that you pass to a second parameter list of FlowGraph.closed determines what materialized value is returned when you run the flow. So when you pass in (x1,x2)=>x1 you return a future which is completed when the first sink gets all elements and then the callback on that future shuts down the actor system without the second sink having a chance receiving all of the elements.
Instead, you should get both futures out and shutdown the system only when both futures are completed.
You can actually see how this approach is used in some of the akka-stream tests here.