How to wrap a library async method with a threadsafe async wrapper in scala? - scala

I'm working with a library that has a method that looks something like this.
trait LibraryDependency {
/**
* This method if called while previous future is not resolved will return a failed future
* #return
*/
def foo(): Future[Boolean]
}
Here the method foo() will return a failed future if there is a previous future that was returned by foo() that has not yet completed. Since I cannot modify the library's implementation of the trait, I am trying to wrap it with my own wrapper that provides the behaviour I need.
The behaviour I need is that if there are concurrent calls to foo(), then the other futures will also block until the first future is resolved. I tried to do something like this.
class ThreadSafeLibraryWrapper(delegate: LibraryDependency) extends LibraryDependency {
private val lock: Object = new Object
private implicit val ec: ExecutionContext = ExecutionContext.Implicits.global
/**
* This one will block the other concurrent calls to foo()
* #return
*/
override def foo(): Future[Boolean] = {
val promise = Promise[Boolean]()
lock.synchronized {
val result = delegate.foo()
promise.completeWith(result)
result.onComplete { _ =>
lock.notify()
}
lock.wait()
}
promise.future
}
}
I'm running into the following issue, I'm not sure how to block the thread that is calling this method, and complete the original future, and I get IllegalMonitorStateException.
EDIT: I've solved this by using Await
class ThreadSafeLibraryWrapper(delegate: LibraryDependency) extends LibraryDependency {
private val lock: Object = new Object
private implicit val ec: ExecutionContext = ExecutionContext.Implicits.global
/**
* This one will block the other concurrent calls to foo()
* #return
*/
override def foo(): Future[Boolean] = Future {
lock.synchronized {
Await.result(delegate.foo(), Duration.Inf)
}
}
}
I'm still not sure how to do this by avoiding Await.

If I correctly understood your question, your dependency can run on Future at time, so you would like to have wrapper which will limit access to foo method so to avoid returning failed future. If so, this looks like you need to en-queue next invocations until previous one will complete.
Well, I did some prototyping and I hope it helps:
import java.time.LocalTime.now
import scala.collection.immutable.Queue
import scala.concurrent.{ExecutionContext, Future, Promise}
import scala.util.{Failure, Success, Try}
object Concurrency {
trait LibraryDependency {
/**
* This method if called while previous future is not resolved will return a failed future
* #return
*/
def foo(): Future[Boolean]
}
class DummyLibraryDependency(implicit ec: ExecutionContext) extends LibraryDependency {
override def foo(): Future[Boolean] = {
Future {
println(s"${now()} - started dependency execution")
Thread.sleep(1000)
println(s"${now()} - finished dependency execution")
true
}
}
}
class SafeLibraryDependency(delegate: LibraryDependency)(implicit ec: ExecutionContext) {
private type OnComplete = Try[Boolean] => Unit
private var currentlyRunning: Option[Future[Boolean]] = None
private var queue: Queue[Promise[Boolean]] = Queue[Promise[Boolean]]()
def foo: Future[Boolean] = {
this.synchronized {
currentlyRunning.fold(startDelegateTask(onRunningComplete))(_ => enqueueNextTask())
}
}
private def enqueueNextTask(): Future[Boolean] = {
val promise = Promise[Boolean]()
queue = queue enqueue promise
promise.future
}
private def onRunningComplete(result: Try[Boolean]): Unit = {
this.synchronized {
currentlyRunning = None
if(queue.nonEmpty) {
val (promise, newQueue) = queue.dequeue
queue = newQueue
startDelegateTask { result =>
promise.complete(result)
onRunningComplete(result)
}
}
}
}
private def startDelegateTask(f: OnComplete): Future[Boolean] = {
val task = delegate.foo()
task.onComplete(f)
currentlyRunning = Some(task)
task
}
}
def main(args: Array[String]): Unit = {
import scala.concurrent.ExecutionContext.Implicits.global
val dummyLibraryDependency = new DummyLibraryDependency
val safeLibraryDependency = new SafeLibraryDependency(dummyLibraryDependency)
safeLibraryDependency.foo.onComplete(result => println(s"${now()} - #1 complete with result: $result"))
safeLibraryDependency.foo.onComplete(result => println(s"${now()} - #2 complete with result: $result"))
safeLibraryDependency.foo.onComplete(result => println(s"${now()} - #3 complete with result: $result"))
Thread.sleep(5000)
println("Done")
}
}
So SafeLibraryDependency - this is the wrapper, which limit's invocations to single running Future at time.
On my machine output was next:
19:30:43.666 - started dependency execution
19:30:44.679 - finished dependency execution
19:30:44.681 - started dependency execution
19:30:44.679 - #1 complete with result: Success(true)
19:30:45.681 - finished dependency execution
19:30:45.681 - started dependency execution
19:30:45.681 - #2 complete with result: Success(true)
19:30:46.682 - finished dependency execution
19:30:46.682 - #3 complete with result: Success(true)
Done
Hope this helps you!

Related

Scala print foo/bar alternately

I'm trying to code this LeetCode exercise of printing foo/bar alternately in Scala using conventional Runnables with wait(), notifyAll(), but can't get it to produce the wanted output, which should be:
foo bar foo bar foo bar foo bar foo bar
Here's the code:
import scala.concurrent.ExecutionContext.Implicits.global
class Foo extends Runnable {
#Override def run(): Unit = { print("foo ") }
}
class Bar extends Runnable {
#Override def run(): Unit = { print("bar ") }
}
val printFoo = new Foo
val printBar = new Bar
class FooBar {
private var foosLoop: Boolean = false
#throws(classOf[InterruptedException])
def foo: Unit = for (_ <- 1 to 5) synchronized {
while (foosLoop) { wait() }
printFoo.run()
foosLoop = true
notifyAll()
}
#throws(classOf[InterruptedException])
def bar: Unit = for (_ <- 1 to 5) synchronized {
while (!foosLoop) { wait() }
printBar.run()
foosLoop = false
notifyAll()
}
}
val fb = new FooBar
fb.foo
fb.bar
// Output:
// foo <=== prints only first "foo "
Could someone help me figure out what I did wrong?
My second question is: Can it be implemented with Scala Futures replacing Runnables?
UPDATE:
The posted code actually works as long as fb.foo and fb.bar are to be called from separate threads.
val tFoo = new Thread (new Runnable { #Override def run(): Unit = fb.foo })
val tBar = new Thread (new Runnable { #Override def run(): Unit = fb.bar })
tFoo.start()
tBar.start()
Could someone help me figure out what I did wrong?
No idea, I haven't used Runnables in my life, and they are not used in Scala.
(and I would say that are also not used anymore in Java too)
Can it be implemented with Scala Futures replacing Runnables?
Yes, something like this:
import java.util.concurrent.Semaphore
import scala.concurrent.{ExecutionContext, Future}
object RunAlternately {
/**
* Runs two taks concurrently and alternating between the two.
* #param n the amout of times to run each task.
* #param aTaks the first task.
* #param bTaks the second task.
*/
def apply(n: Int)(aTask: => Unit)(bTask: => Unit)(implicit ec: ExecutionContext): Future[Unit] ={
val aLock = new Semaphore(1)
val bLock = new Semaphore(0)
def runOne(task: => Unit, thisLock: Semaphore, thatLock: Semaphore): Future[Unit] =
Future {
var i = 0
while (i < n) {
thisLock.acquire()
task
thatLock.release()
i += 1
}
}
val aFuture = runOne(aTask, thisLock = aLock, thatLock = bLock)
val bFuture = runOne(bTask, thisLock = bLock, thatLock = aLock)
aFuture.flatMap(_ => bFuture)
}
}
See it running here.
However, these kind of things are usually better modelled with even higher-level APIs like IO or Streams.

Async before and after for creating and dropping scala slick tables in scalatest

I'm trying to figure out a way to have async before and after statements where the next test cases aren't run until the completion of the action inside of the test case. In my case, it is the creating and dropping a table inside of a database
val table = TableQuery[BlockHeaderTable]
val dbConfig: DatabaseConfig[PostgresDriver] = DatabaseConfig.forConfig("databaseUrl")
val database: Database = dbConfig.db
before {
//Awaits need to be used to make sure this is fully executed before the next test case starts
//TODO: Figure out a way to make this asynchronous
Await.result(database.run(table.schema.create), 10.seconds)
}
"BlockHeaderDAO" must "store a blockheader in the database, then read it from the database" in {
//...
}
it must "delete a block header in the database" in {
//...
}
after {
//Awaits need to be used to make sure this is fully executed before the next test case starts
//TODO: Figure out a way to make this asynchronous
Await.result(database.run(table.schema.drop),10.seconds)
}
Is there a simple way I can remove these Await calls inside of my before and after functions?
Unfortunately, #Jeffrey Chung's solution hanged for me (since futureValue actually awaits internally). This is what I ended up doing:
import org.scalatest.{AsyncFreeSpec, FutureOutcome}
import scala.concurrent.Future
class TestTest extends AsyncFreeSpec /* Could be any AsyncSpec. */ {
// Do whatever setup you need here.
def setup(): Future[_] = ???
// Cleanup whatever you need here.
def tearDown(): Future[_] = ???
override def withFixture(test: NoArgAsyncTest) = new FutureOutcome(for {
_ <- setup()
result <- super.withFixture(test).toFuture
_ <- tearDown()
} yield result)
}
The following is the testing approach that Dennis Vriend takes in his slick-3.2.0-test project.
First, define a dropCreateSchema method. This method attempts to create a table; if that attempt fails (because, for example, the table already exists), it drops, then creates, the table:
def dropCreateSchema: Future[Unit] = {
val schema = BlockHeaderTable.schema
db.run(schema.create)
.recoverWith {
case t: Throwable =>
db.run(DBIO.seq(schema.drop, schema.create))
}
}
Second, define a createEntries method that populates the table with some sample data for use in each test case:
def createEntries: Future[Unit] = {
val setup = DBIO.seq(
// insert some rows
BlockHeaderTable ++= Seq(
BlockHeaderTableRow(/* ... */),
// ...
)
).transactionally
db.run(setup)
}
Third, define an initialize method that calls the above two methods sequentially:
def initialize: Future[Unit] = for {
_ <- dropCreateSchema
_ <- createEntries
} yield ()
In the test class, mix in the ScalaFutures trait. For example:
class TestSpec extends FlatSpec
with Matchers
with ScalaFutures
with BeforeAndAfterAll
with BeforeAndAfterEach {
// ...
}
Also in the test class, define an implicit conversion from a Future to a Try, and override the beforeEach method to call initialize:
implicit val timeout: Timeout = 10.seconds
implicit class PimpedFuture[T](self: Future[T]) {
def toTry: Try[T] = Try(self.futureValue)
}
override protected def beforeEach(): Unit = {
blockHeaderRepo.initialize // in this example, initialize is defined in a repo class
.toTry recover {
case t: Throwable =>
log.error("Could not initialize the database", t)
} should be a 'success
}
override protected def afterAll(): Unit = {
db.close()
}
With the above pieces in place, there is no need for Await.
You can simplify #Jeffrey Chung
A simplified dropCreateSchema method:
def dropCreateSchema: Future[Unit] = {
val schema = users.schema
db.run(DBIO.seq(schema.dropIfExists, schema.create))
}
Also in the test class, I simplified beforeEach method that calls initialize. I removed an implicit conversion from a Future to a Try, and use onComplete callback:
override protected def beforeEach(): Unit = {
initialize.onComplete(f =>
f recover {
case t: Throwable =>
log.error("Could not initialize the database", t)
} should be a 'success)
}
override protected def afterAll(): Unit = {
db.close()
}

In twitter future, how to set function of asynchronize return directly

I am new to future modern, what I expect from the following code is :
launch future task finished!
end of task
but actualy, the "end of task" appear first.
I want the process in function long_task to run asynchronizely.
what did I missed?
import com.twitter.util.Future
object FutureTest extends App{
def long_task(): Future[_] = { //the function of asynchronize processing
Thread.sleep(5000)
println("end of task")
Future.value("")
}
val f = long_task()
println("launch future task finished!")
}
The problem you are encountering is that long_task is a synchronous function that returns a completed future after it has done all the work. A future in and of itself does not start asynchronous work, but is just a handle for notification of asynchronous. It's still up to you to start the work asynchronously.
For your example using a FutureTask with a java ExecutorService will do the trick:
object FutureTest extends App{
val pool: ExecutorService = Executors.newFixedThreadPool(10)
def long_task(): Future[Unit] = {
// create a FutureTask, which is a Callable wrapped by a Future
val f = new FutureTask[Unit](new Callable[Unit]() {
def call(): Unit = {
Thread.sleep(5000)
println("end of task")
}})
// execute the callable
pool.execute(f)
// return the uncompleted future
f
}
val f = long_task()
println("launch future task finished!")
}
If you can use the scala Future instead of Twitter's the syntax gets a little simpler, since they've abstracted the underlying Threadpool work a bit further:
import scala.concurrent.ExecutionContext.Implicits._
import scala.concurrent.{Await, future}
import scala.concurrent.duration._
object FutureTest extends App {
def long_task() = future {
Thread.sleep(5000)
println("end of task")
}
val f = long_task()
println("launch future task finished!")
Await.result(f,10.seconds)
}
Note the Await at the end, which is needed here because with scala's default executorservice, the future work is a daemon thread which will not keep the runtime from quiting as soon as it reaches the last line. Await forces the program to block until the future completes.
Try it this way (create the Future first and execute):
import com.twitter.util.{Future, FutureTask}
object FutureTest extends App{
def long_task(): Future[_] = new FutureTask { //the function of asynchronize processing
Thread.sleep(5000)
println("end of task")
""
}
val f = long_task()
println("launch future task finished!")
}

How to communicate between threads in Scala?

I have a class with some methods in Scala, each method has an execution time, for example methodA takes 10 seconds and methodB takes 5 seconds. and each method calls asynchronously.when I call methodB, it should cancel the thread that is running another methods. I first call methodA, and 2 seconds later I call methodB. What is the best solution for this problem?
def methodA()={
async{
// a job that takes 10 seconds
}
}
def methodB()={
async{
// other methods should stop their job
// a job that takes 5 second
}
}
def async[T](fn: => Unit): Unit = scala.actors.Actor.actor {
fn
}
.....
methodA()
methodB()
Here is an idea, based on the assumption that your method is actively checking whether it should still run or cancel:
import concurrent.{ExecutionContext, Future, Promise, blocking, future, promise}
case class Cancelled() extends RuntimeException
object Job {
def apply[A](fun: (() => Boolean) => A)(implicit ctx: ExecutionContext): Job[A] =
new Job[A] {
private val p = promise[A]
def result = p.future
def cancel(): Unit = p.tryFailure(Cancelled())
p tryCompleteWith future {
fun(() => !p.isCompleted)
}
}
}
trait Job[A] {
def result: Future[A]
def cancel(): Unit
}
So Job embodies a future along with a cancel() method. Your example could be similar to this:
import ExecutionContext.Implicits.global
val a = Job { active =>
for (i <- 1 to 100 if active()) {
blocking {
Thread.sleep(1000) // doing actual heavy work here
println(s"A $i")
}
}
}
val b = Job { active =>
for (i <- 1 to 20 if active()) {
blocking {
Thread.sleep(1000) // doing actual heavy work here
println(s"B $i")
}
}
println("Goodbye A...")
a.cancel()
}
I have also seen a rather harsh variant that calls Thread.interrupt.

Kill or timeout a Future in Scala 2.10

Hi,
I'm using Scala 2.10 with the new futures library and I'm trying to write some code to test an infinite loop. I use a scala.concurrent.Future to run the code with the loop in a separate thread. I would then like to wait a little while to do some testing and then kill off the separate thread/future. I have looked at Await.result but that doesn't actually kill the future. Is there any way to timeout or kill the new Scala 2.10 futures?
I would prefer not having to add external dependencies such as Akka just for this simple part.
Do not try it at home.
import scala.concurrent._
import scala.concurrent.duration._
class MyCustomExecutionContext extends AnyRef with ExecutionContext {
import ExecutionContext.Implicits.global
#volatile var lastThread: Option[Thread] = None
override def execute(runnable: Runnable): Unit = {
ExecutionContext.Implicits.global.execute(new Runnable() {
override def run() {
lastThread = Some(Thread.currentThread)
runnable.run()
}
})
}
override def reportFailure(t: Throwable): Unit = ???
}
implicit val exec = new MyCustomExecutionContext()
val f = future[Int]{ do{}while(true); 1 }
try {
Await.result(f, 10 seconds) // 100% cpu here
} catch {
case e: TimeoutException =>
println("Stopping...")
exec.lastThread.getOrElse(throw new RuntimeException("Not started"))
.stop() // 0% cpu here
}
No - you will have to add a flag that your loop checks. If the flag is set, stop the loop. Make sure the flag is at least volatile.
See Java Concurrency in Practice, p 135-137.
I had a similar problem and wrote the following nonblocking future op:
class TerminationToken(var isTerminated: Boolean)
object TerminationToken { def apply() = new TerminationToken(false) }
implicit class FutureOps[T](future: Future[Option[T]]) {
def terminate(timeout: FiniteDuration, token: TerminationToken): Future[Option[T]] = {
val timeoutFuture = after[Option[T]](timeout, using = context.system.scheduler) {
Future[Option[T]] { token.isTerminated = true; None } }
Future.firstCompletedOf[Option[T]](Seq (future recover { case _ => None }, timeoutFuture))
}
}
Then just create a future that returns an option, and use .terminate(timeout, token) on it