I have created a simple test scenario that shows this:
class Test extends AsyncFunSuite {
test("async test") {
val f = Future {
val thread = new Thread {
override def run(): Unit = {
println("OKAYY")
}
}
thread.start()
}
Await.result(f, Duration.Inf)
Future {
assert(true)
}
}
}
When executing this, the test runs forever and never completes.
You will see the same behavior with all asynchronous testing in ScalaTest (AsyncFeatureSpec, AsyncFlatSpec, AsyncFreeSpec, AsyncFunSpec, AsyncFunSuite, AsyncWordSpec).
The reason for this is that the default execution context in ScalaTest is serial execution context. You can read more about this here: https://www.scalatest.org/user_guide/async_testing. I have summarized the important points below.
Using ScalaTest's serial execution context on the JVM will ensure the same thread that produced the Future[Assertion] returned from a test body is also used to execute any tasks given to the execution context while executing the test body—and that thread will not be allowed to do anything else until the test completes.
This thread confinement strategy does mean, however, that when you are using the default execution context on the JVM, you must be sure to never block in the test body waiting for a task to be completed by the execution context. If you block, your test will never complete.
Solution 1: Override execution context
implicit override def executionContext = scala.concurrent.ExecutionContext.Implicits.global
Solution 2: Chaining
class Test extends AsyncFunSuite {
test("async test") {
val f = Future {
val thread = new Thread {
override def run(): Unit = {
println("OKAYY")
}
}
thread.start()
}
f.map { _ =>
assert(true)
}
}
}
Related
Consider the following unit test example:
class MySpec extends AsyncFlatSpec {
"this" should "fail after some timeout" in {
val resultFuture: Future[_] = Promise().future
for (result <- resultFuture) yield {
assert(result == ???) // some assertions
assert(result == ???) // some assertions
assert(result == ???) // some assertions
}
}
}
The problem
If resultFuture never completes the test suite never finishes either.
As I understand it's due to the way how SerialExecutionContext is implemented.
The question
Is there any "nice" way how to setup a timeout for this kind of tests, so that if the future isn't complete the test just fails, instead of blocking the entire test suite for eternity?
UPDATE and clarification
While solutions https://stackoverflow.com/a/65746143/96766 and https://stackoverflow.com/a/65749840/96766 (posted by #matthias-berndt and #tomer-shetah) work for the case of blocked thread, it's not exactly what I'm looking for.
Let me make an important clarification to the question. In my case the future isn't eventually complete, but never complete. For example, when a Future is obtained from the Promise that is never resolved (nobody calls success nor failure on it). In that case the proposed solutions still block infinitely.
Is there a way to work this around for AsyncSpec without resorting to using a real pool-based execution context and Await-ing on the future?
Use eventually from scalatest
extends Eventually
Use the following code to set up timeout and interval for checking
import scala.concurrent.duration._
eventually(timeout(1 minutes), interval(5 seconds)) {
resultFuture.futureValue shouldBe ???
}
Use the AsyncTimeLimitedTests trait.
https://www.scalatest.org/scaladoc/3.2.0/org/scalatest/concurrent/AsyncTimeLimitedTests.html
You can use the trait TimeLimits. For example, you can have a test class:
class MySpec extends AsyncFlatSpec with TimeLimits {
"this" should "fail" in {
failAfter(2.seconds) {
val resultFuture: Future[_] = Future {
Thread.sleep(3000)
}
assert(true)
}
}
"this" should "pass" in {
failAfter(2.seconds) {
val resultFuture: Future[_] = Future {
Thread.sleep(1000)
}
assert(true)
}
}
}
I have two methods, let's call them load() and init(). Each one starts a computation in its own thread and returns a Future on its own execution context. The two computations are independent.
val loadContext = ExecutionContext.fromExecutor(...)
def load(): Future[Unit] = {
Future
}
val initContext = ExecutionContext.fromExecutor(...)
def init(): Future[Unit] = {
Future { ... }(initContext)
}
I want to call both of these from some third thread -- say it's from main() -- and perform some other computation when both are finished.
def onBothComplete(): Unit = ...
Now:
I don't care which completes first
I don't care what thread the other computation is performed on, except:
I don't want to block either thread waiting for the other;
I don't want to block the third (calling) thread; and
I don't want to have to start a fourth thread just to set the flag.
If I use for-comprehensions, I get something like:
val loading = load()
val initialization = initialize()
for {
loaded <- loading
initialized <- initialization
} yield { onBothComplete() }
and I get Cannot find an implicit ExecutionContext.
I take this to mean Scala wants a fourth thread to wait for the completion of both futures and set the flag, either an explicit new ExecutionContext or ExecutionContext.Implicits.global. So it would appear that for-comprehensions are out.
I thought I might be able to nest callbacks:
initialization.onComplete {
case Success(_) =>
loading.onComplete {
case Success(_) => onBothComplete()
case Failure(t) => log.error("Unable to load", t)
}
case Failure(t) => log.error("Unable to initialize", t)
}
Unfortunately onComplete also takes an implicit ExecutionContext, and I get the same error. (Also this is ugly, and loses the error message from loading if initialization fails.)
Is there any way to compose Scala Futures without blocking and without introducing another ExecutionContext? If not, I might have to just throw them over for Java 8 CompletableFutures or Javaslang Vavr Futures, both of which have the ability to run callbacks on the thread that did the original work.
Updated to clarify that blocking either thread waiting for the other is also not acceptable.
Updated again to be less specific about the post-completion computation.
Why not just reuse one of your own execution contexts? Not sure what your requirements for those are but if you use a single thread executor you could just reuse that one as the execution context for your comprehension and you won't get any new threads created:
implicit val loadContext = ExecutionContext.fromExecutor(Executors.newSingleThreadExecutor)
If you really can't reuse them you may consider this as the implicit execution context:
implicit val currentThreadExecutionContext = ExecutionContext.fromExecutor(
(runnable: Runnable) => {
runnable.run()
})
Which will run futures on the current thread. However, the Scala docs explicitly recommends against this as it introduces nondeterminism in which thread runs the Future (but as you stated, you don't care which thread it runs on so this may not matter).
See Synchronous Execution Context for why this isn't advisable.
An example with that context:
val loadContext = ExecutionContext.fromExecutor(Executors.newSingleThreadExecutor)
def load(): Future[Unit] = {
Future(println("loading thread " + Thread.currentThread().getName))(loadContext)
}
val initContext = ExecutionContext.fromExecutor(Executors.newSingleThreadExecutor)
def init(): Future[Unit] = {
Future(println("init thread " + Thread.currentThread().getName))(initContext)
}
val doneFlag = new AtomicBoolean(false)
val loading = load()
val initialization = init()
implicit val currentThreadExecutionContext = ExecutionContext.fromExecutor(
(runnable: Runnable) => {
runnable.run()
})
for {
loaded <- loading
initialized <- initialization
} yield {
println("yield thread " + Thread.currentThread().getName)
doneFlag.set(true)
}
prints:
loading thread pool-1-thread-1
init thread pool-2-thread-1
yield thread main
Though the yield line may print either pool-1-thread-1 or pool-2-thread-1 depending on the run.
In Scala, a Future represents a piece of work to be executed async (i.e. concurrently to other units of work). An ExecutionContext represent a pool of threads for executing Futures. In other words, ExecutionContext is the team of worker who performs the actual work.
For efficiency and scalability, it's better to have big team(s) (e.g. single ExecutionContext with 10 threads to execute 10 Future's) rather than small teams (e.g. 5 ExecutionContext with 2 threads each to execute 10 Future's).
In your case if you want to limit the number of threads to 2, you can:
def load()(implicit teamOfWorkers: ExecutionContext): Future[Unit] = {
Future { ... } /* will use the teamOfWorkers implicitly */
}
def init()(implicit teamOfWorkers: ExecutionContext): Future[Unit] = {
Future { ... } /* will use the teamOfWorkers implicitly */
}
implicit val bigTeamOfWorkers = ExecutionContext.fromExecutorService(Executors.newFixedThreadPool(2))
/* All async works in the following will use
the same bigTeamOfWorkers implicitly and works will be shared by
the 2 workers (i.e. thread) in the team */
for {
loaded <- loading
initialized <- initialization
} yield doneFlag.set(true)
The Cannot find an implicit ExecutionContext error does not mean that Scala wants additional threads. It only means that Scala wants a ExecutionContext to do the work. And additional ExecutionContext does not necessarily implies additional 'thread', e.g. the following ExecutionContext, instead of creating new threads, will execute works in the current thread:
val currThreadExecutor = ExecutionContext.fromExecutor(new Executor {
override def execute(command: Runnable): Unit = command.run()
})
Just trying to get my first futures use up and running and doing a test similar to an example outlined in the Akka in Action MEAP book. I want to call a web service and return the result in a future. I am using scalaxb to access the web service. I have outlined the code below but when I run it the app terminates without waiting for the response from the service. Maybe someone can tell me what I am missing?
import scala.util._
import control.NonFatal
import scala.concurrent._
import ExecutionContext.Implicits.global
object Test {
val service = (new MyServiceBindings with scalaxb.Soap11Clients with scalaxb.DispatchHttpClients {}).service
def test = {
val f = future {
service.someCall() match {
case Right(resp) => resp
case Left(fault) => throw new Exception("Fault: " + fault)}
}
}
f.onComplete {
case Success(resp) => println("Resp: " + resp)
case Failure(NonFatal(e)) => println("Fail: " + e)
}
}
def main(args: Array[String]): Unit = {
test
}
}
It terminates because the main thread that is executed inside your Test has completed. The threads that are used internally by the Dispatch library don't keep the program from exiting.
You would need to wait on the future since this is the only thing your test app is doing. Put this after the onComplete statement.
import scala.concurrent.duration._
Await.ready(f, 10.seconds)
Now bear in mind that this is bad practice usually. You need it here because your test app is doing nothing else, but in a real app, you wouldn't want to block after each futures call, as that would negate the point of using futures.
You can patch this in the main funtion by adding the following logic to your test application:
def main(args: Array[String]) : Unit = {
// Assuming test is a future that did not complete yet.
while(!test.isCompleted) {
Thread.sleep(100)
}
// Software exits here only after future is completed.
}
Or Better if you have many futures you can implement it this way:
// Assuming you have --> listOfFutures = ArrayBuffer[Future]
def main(args: Array[String]) : Unit = {
while(!listOfFutures.isEmpty) {
listOfFutures.foreach {
future =>
if(future.isCompleted) {
listOfFutures -= future
}
}
//checks which futures are completed every half-a-second
Thread.sleep(500)
}
// Program exits after all futures have been completed.
}
Just trying to get my first futures use up and running and doing a test similar to an example outlined in the Akka in Action MEAP book. I want to call a web service and return the result in a future. I am using scalaxb to access the web service. I have outlined the code below but when I run it the app terminates without waiting for the response from the service. Maybe someone can tell me what I am missing?
import scala.util._
import control.NonFatal
import scala.concurrent._
import ExecutionContext.Implicits.global
object Test {
val service = (new MyServiceBindings with scalaxb.Soap11Clients with scalaxb.DispatchHttpClients {}).service
def test = {
val f = future {
service.someCall() match {
case Right(resp) => resp
case Left(fault) => throw new Exception("Fault: " + fault)}
}
}
f.onComplete {
case Success(resp) => println("Resp: " + resp)
case Failure(NonFatal(e)) => println("Fail: " + e)
}
}
def main(args: Array[String]): Unit = {
test
}
}
It terminates because the main thread that is executed inside your Test has completed. The threads that are used internally by the Dispatch library don't keep the program from exiting.
You would need to wait on the future since this is the only thing your test app is doing. Put this after the onComplete statement.
import scala.concurrent.duration._
Await.ready(f, 10.seconds)
Now bear in mind that this is bad practice usually. You need it here because your test app is doing nothing else, but in a real app, you wouldn't want to block after each futures call, as that would negate the point of using futures.
You can patch this in the main funtion by adding the following logic to your test application:
def main(args: Array[String]) : Unit = {
// Assuming test is a future that did not complete yet.
while(!test.isCompleted) {
Thread.sleep(100)
}
// Software exits here only after future is completed.
}
Or Better if you have many futures you can implement it this way:
// Assuming you have --> listOfFutures = ArrayBuffer[Future]
def main(args: Array[String]) : Unit = {
while(!listOfFutures.isEmpty) {
listOfFutures.foreach {
future =>
if(future.isCompleted) {
listOfFutures -= future
}
}
//checks which futures are completed every half-a-second
Thread.sleep(500)
}
// Program exits after all futures have been completed.
}
I am new to future modern, what I expect from the following code is :
launch future task finished!
end of task
but actualy, the "end of task" appear first.
I want the process in function long_task to run asynchronizely.
what did I missed?
import com.twitter.util.Future
object FutureTest extends App{
def long_task(): Future[_] = { //the function of asynchronize processing
Thread.sleep(5000)
println("end of task")
Future.value("")
}
val f = long_task()
println("launch future task finished!")
}
The problem you are encountering is that long_task is a synchronous function that returns a completed future after it has done all the work. A future in and of itself does not start asynchronous work, but is just a handle for notification of asynchronous. It's still up to you to start the work asynchronously.
For your example using a FutureTask with a java ExecutorService will do the trick:
object FutureTest extends App{
val pool: ExecutorService = Executors.newFixedThreadPool(10)
def long_task(): Future[Unit] = {
// create a FutureTask, which is a Callable wrapped by a Future
val f = new FutureTask[Unit](new Callable[Unit]() {
def call(): Unit = {
Thread.sleep(5000)
println("end of task")
}})
// execute the callable
pool.execute(f)
// return the uncompleted future
f
}
val f = long_task()
println("launch future task finished!")
}
If you can use the scala Future instead of Twitter's the syntax gets a little simpler, since they've abstracted the underlying Threadpool work a bit further:
import scala.concurrent.ExecutionContext.Implicits._
import scala.concurrent.{Await, future}
import scala.concurrent.duration._
object FutureTest extends App {
def long_task() = future {
Thread.sleep(5000)
println("end of task")
}
val f = long_task()
println("launch future task finished!")
Await.result(f,10.seconds)
}
Note the Await at the end, which is needed here because with scala's default executorservice, the future work is a daemon thread which will not keep the runtime from quiting as soon as it reaches the last line. Await forces the program to block until the future completes.
Try it this way (create the Future first and execute):
import com.twitter.util.{Future, FutureTask}
object FutureTest extends App{
def long_task(): Future[_] = new FutureTask { //the function of asynchronize processing
Thread.sleep(5000)
println("end of task")
""
}
val f = long_task()
println("launch future task finished!")
}