I am writing a simple function to return a Future[Unit] that completes after a given delay.
def delayedFuture(delay: FiniteDuration): Future[Unit] = {
val promise = Promise[Unit]()
val timerTask = new java.util.TimerTask {
override def run(): Unit = promise.complete(Success(()))
}
new java.util.Timer().schedule(timerTask, delay.toMillis)
promise.future
}
This implementation can probably work but I don't like creating a new Timer for each invocation because each Timer instance creates a thread. I can pass a Timer instance to delayedFuture as an argument but I don't want client code to know about Timer. So I guess I cannot use java.util.Timer at all.
I would like to use ExecutionContext for task scheduling and maybe define delayedFuture like this:
def delayedFuture(delay: FiniteDuration)
(implicit ec: ExecutoionContext): Future[Unit] = ???
What is the simplest way to implement delayedFuture like this without java.util.Timer ?
You don't need to create a new Timer for each invocation. Just make one global one.
object Delayed {
private val timer = new Timer
def apply[T](delay: Duration)(task: => T): Future[T] = {
val promise = Promise[T]()
val tt = new TimerTask {
override def run(): Unit = promise.success(task)
}
timer.schedule(tt, delay.toMillis)
promise.future
}
def unit(delay: Duration) = apply[Unit](delay) { () }
}
Then, Delayed.unit(10 seconds) gives you a future unit that satisfies in 10 seconds.
You can use one of Java's ScheduledExecutorServices, e.g. create a single-thread pool and use it for scheduling only:
val svc = new ScheduledThreadPoolExecutor(1, (r: Runnable) => {
val t = new Thread(r)
t.setName("future-delayer")
t.setDaemon(true) // don't hog the app with this thread existing
t
})
def delayedFuture(d: FiniteDuration) = {
val p = Promise[Unit]()
svc.schedule(() => p.success(()), d.length, d.unit)
p.future
}
Related
Trying to wrap my head around how async tasks are chained together, for ex futures and flatMap
val a: Future[Int] = Future { 123 }
val b: Future[Int] = a.flatMap(a => Future { a + 321 })
How would i implement something similar where i build a new Future by waiting for a first and applying a f on the result. I tried looking a scala's source code but got stuck here: https://github.com/scala/scala/blob/2.13.x/src/library/scala/concurrent/Future.scala#L216
i would imagine some code like:
def myFlatMap(a: Future[Int])(f: a => Future[Int]): Future[Int] = {
Future {
// somehow wait for a to complete and the apply 'f'
a.onComplete {
case Success(x) => f(a)
}
}
}
but i guess above will return a Future[Unit] instead. I would just like to understand the "pattern" of how async tasks are chained together.
(Disclaimer: I'm the main maintainer of Scala Futures)
You wouldn't want to implement flatMap/recoverWith/transformWith yourself—it is a very complex feature to implement safely (stack-safe, memory-safe, and concurrency-safe).
You can see what I am talking about here:
https://github.com/scala/scala/blob/2.13.x/src/library/scala/concurrent/impl/Promise.scala#L442
https://github.com/scala/scala/blob/2.13.x/src/library/scala/concurrent/impl/Promise.scala#L307
https://github.com/scala/scala/blob/2.13.x/src/library/scala/concurrent/impl/Promise.scala#L274
There is more reading on the topic here: https://viktorklang.com/blog/Futures-in-Scala-2.12-part-9.html
I found that looking at the source for scala 2.12 was easier for me to understand how Future / Promise was implemented, i learned that Future is in fact a Promise and by looking at the implementation i now understand how "chaining" async tasks can be implemented.
Im posting the code that i wrote to help me better understand what was going on under the hood.
import java.util.concurrent.{ScheduledThreadPoolExecutor, TimeUnit}
import scala.collection.mutable.ListBuffer
trait MyAsyncTask[T] {
type Callback = T => Unit
var result: Option[T]
// register a new callback to be called when task is completed.
def onComplete(callback: Callback): Unit
// complete the given task, and invoke all registered callbacks.
def complete(value: T): Unit
// create a new task, that will wait for the current task to complete and apply
// the provided map function, and complete the new task when completed.
def flatMap[B](f: T => MyAsyncTask[B]): MyAsyncTask[B] = {
val wrapper = new DefaultAsyncTask[B]()
onComplete { x =>
f(x).onComplete { y =>
wrapper.complete(y)
}
}
wrapper
}
def map[B](f: T => B): MyAsyncTask[B] = flatMap(x => CompletedAsyncTask(f(x)))
}
/**
* task with fixed pre-calculated result.
*/
case class CompletedAsyncTask[T](value: T) extends MyAsyncTask[T] {
override var result: Option[T] = Some(value)
override def onComplete(callback: Callback): Unit = {
// we already have the result just call the callback.
callback(value)
}
override def complete(value: T): Unit = () // noop nothing to complete.
}
class DefaultAsyncTask[T] extends MyAsyncTask[T] {
override var result: Option[T] = None
var isCompleted = false
var listeners = new ListBuffer[Callback]()
/**
* register callback, to be called when task is completed.
*/
override def onComplete(callback: Callback): Unit = {
if (isCompleted) {
// already completed just invoke callback
callback(result.get)
} else {
// add the listener
listeners.addOne(callback)
}
}
/**
* trigger all registered callbacks to `onComplete`
*/
override def complete(value: T): Unit = {
result = Some(value)
listeners.foreach { listener =>
listener(value)
}
}
}
object MyAsyncTask {
def apply[T](body: (T => Unit) => Unit): MyAsyncTask[T] = {
val task = new DefaultAsyncTask[T]()
// pass in `complete` as callback.
body(task.complete)
task
}
}
object MyAsyncFlatMap {
/**
* helper to simulate async task.
*/
def delayedCall(body: => Unit, delay: Int): Unit = {
val scheduler = new ScheduledThreadPoolExecutor(1)
val run = new Runnable {
override def run(): Unit = {
body
}
}
scheduler.schedule(run, delay, TimeUnit.SECONDS)
}
def main(args: Array[String]): Unit = {
val getAge = MyAsyncTask[Int] { cb =>
delayedCall({ cb(66) }, 1)
}
val getName = MyAsyncTask[String] { cb =>
delayedCall({ cb("John") }, 2)
}
// same as: getAge.flatMap(age => getName.map(name => (age, name)))
val result: MyAsyncTask[(Int, String)] = for {
age <- getAge
name <- getName
} yield (age, name)
result.onComplete {
case (age, name) => println(s"hello $name $age")
}
}
}
I would like to have a Source.queue (or something analogous to it to push items on a materialized graph) that is instrumented to tell me the current level of saturation of the queue.
I'd like to do that without (re-)implementing the functionality provided by the QueueSource graph stage.
One possible solution I came up with is the following:
object InstrumentedSource {
final class InstrumentedSourceQueueWithComplete[T](
delegate: SourceQueueWithComplete[T],
bufferSize: Int,
)(implicit executionContext: ExecutionContext)
extends SourceQueueWithComplete[T] {
override def complete(): Unit = delegate.complete()
override def fail(ex: Throwable): Unit = delegate.fail(ex)
override def watchCompletion(): Future[Done] = delegate.watchCompletion()
private val buffered = new AtomicLong(0)
private[InstrumentedSource] def onDequeue(): Unit = {
val _ = buffered.decrementAndGet()
}
object BufferSaturationRatioGauge extends RatioGauge {
override def getRatio: RatioGauge.Ratio = RatioGauge.Ratio.of(buffered.get(), bufferSize)
}
lazy val bufferSaturationGauge: RatioGauge = BufferSaturationRatioGauge
override def offer(elem: T): Future[QueueOfferResult] = {
val result = delegate.offer(elem)
result.foreach {
case QueueOfferResult.Enqueued =>
val _ = buffered.incrementAndGet()
case _ => // do nothing
}
result
}
}
def queue[T](bufferSize: Int, overflowStrategy: OverflowStrategy)(
implicit executionContext: ExecutionContext,
materializer: Materializer,
): Source[T, InstrumentedSourceQueueWithComplete[T]] = {
val (queue, source) = Source.queue[T](bufferSize, overflowStrategy).preMaterialize()
val instrumentedQueue = new InstrumentedSourceQueueWithComplete[T](queue, bufferSize)
source.mapMaterializedValue(_ => instrumentedQueue).map { item =>
instrumentedQueue.onDequeue()
item
}
}
}
This mostly appears to work from some manual testing (apart from the fact that buffered is at most eventually consistent with the actual number of items in the queue, which should be fine in my case), but I was wondering if there is are solutions that perhaps make better use of built-in functionalities that I might have missed.
I'm trying to figure out how to write this piece of code in an elegant pure-functional style using scalaz7 IO and monad transformers but just can't get my head around it.
Just imagine I have this simple API:
def findUuid(request: Request): Option[String] = ???
def findProfile(uuid: String): Future[Option[Profile]] = redisClient.get[Profile](uuid)
Using this API I can easily write impure function with OptionT transformer like this:
val profileT = for {
uuid <- OptionT(Future.successful(findUuid(request)))
profile <- OptionT(findProfile(uuid))
} yield profile
val profile: Future[Option[Profile]] = profileT.run
As you have noticed - this function contains findProfile() with a side-effect. I want to isolate this effect inside of the IO monad and interpret outside of the pure function but don't know how to combine it all together.
def findProfileIO(uuid: String): IO[Future[Option[Profile]]] = IO(findProfile(uuid))
val profileT = for {
uuid <- OptionT(Future.successful(findUuid(request)))
profile <- OptionT(findProfileIO(uuid)) //??? how to put Option inside of the IO[Future[Option]]
} yield profile
val profile = profileT.run //how to run transformer and interpret IO with the unsafePerformIO()???
Any peaces of advice on how it might be done?
IO is meant more for synchronous effects. Task is more what you want!
See this question and answer: What's the difference between Task and IO in Scalaz?
You can convert your Future to Task and then have an API like this:
def findUuid(request: Request): Option[String] = ???
def findProfile(uuid: String): Task[Option[Profile]] = ???
This works because Task can represent both synchronous and asynchronous operations, so findUuid can also be wrapped in Task instead of IO.
Then you can wrap these in OptionT:
val profileT = for {
uuid <- OptionT(Task.now(findUuid(request)))
profile <- OptionT(findProfileIO(uuid))
} yield profile
Then at the end somewhere you can run it:
profileT.run.attemptRun
Check out this link for converting Futures to Tasks and vice versa: Scalaz Task <-> Future
End up with this piece of code, thought it might be useful for someone (Play 2.6).
Controller's method is a pure function since Task evaluation takes place outside of the controller inside of PureAction ActionBuilder. Thanks to Luka's answer!
Still struggling with new paradigm of Action composition in Play 2.6 though, but this is another story.
FrontendController.scala:
def index = PureAction.pure { request =>
val profileOpt = (for {
uuid <- OptionT(Task.now(request.cookies.get("uuid").map(t => uuidKey(t.value))))
profile <- OptionT(redis.get[Profile](uuid).asTask)
} yield profile).run
profileOpt.map { profileOpt =>
Logger.info(profileOpt.map(p => s"User logged in - $p").getOrElse("New user, suggesting login"))
Ok(views.html.index(profileOpt))
}
}
Actions.scala
Convenient action with Task resolution at the end
class PureAction #Inject()(parser: BodyParsers.Default)(implicit ec: ExecutionContext) extends ActionBuilderImpl(parser) {
self =>
def pure(block: Request[AnyContent] => Task[Result]): Action[AnyContent] = composeAction(new Action[AnyContent] {
override def parser: BodyParser[AnyContent] = self.parser
override def executionContext: ExecutionContext = self.ec
override def apply(request: Request[AnyContent]): Future[Result] = {
val taskResult = block(request)
taskResult.asFuture //End of the world lives here
}
})
}
Converters.scala
Task->Future and Future->Task implicit converters
implicit class FuturePimped[+T](root: => Future[T]) {
import scalaz.Scalaz._
def asTask(implicit ec: ExecutionContext): Task[T] = {
Task.async { register =>
root.onComplete {
case Success(v) => register(v.right)
case Failure(ex) => register(ex.left)
}
}
}
}
implicit class TaskPimped[T](root: => Task[T]) {
import scalaz._
val p: Promise[T] = Promise()
def asFuture: Future[T] = {
root.unsafePerformAsync {
case -\/(ex) => p.failure(ex); ()
case \/-(r) => p.success(r); ()
}
p.future
}
}
I have the following code that I need to run around 100 times:
val system = ActorSystem("mySystem")
val myActorObject = system.actorOf(Props[MyActorClass], name = "myactor")
implicit val timeout = Timeout(60 seconds)
val future = myActorObject ? Request1
val result = Await.result(future, timeout.duration)
Question is: assuming the first two statements can be called just once, should I cache these variables or Akka does it?
Do not make the actor system creation and actor creation part of repetitive code. Once the actor is created. Using the ActorRef can be done as many times as possible.
val system = ActorSystem("mySystem")
val myActorObject = system.actorOf(Props[MyActorClass], name = "myactor")
Something like this
val futureList = for(_ <- 1 to 1000) yield {
implicit val timeout = Timeout(60 seconds)
val future = myActorObject ? Request1
}
val finalF = Future.sequence(futureList)
val result = Await.result(future, timeout.duration)
In play! (+2.5.x) you can get the system by injection. e.g.
#Singleton
class MyController #Inject() (system: ActorSystem) extends Controller {
val myActor = system.actorOf(MyActorClass.props, "myactor")
//...
}
Then you could have an end point to call the actor as many times as you want, for example
def sayHello(name: String) = Action.async {
(myActor ? name).map { message => Ok(message) }
}
The code inside your actor could be something like this
class MyActorClass extends Actor {
def receive = {
case name: String => sender() ! s"hello $name"
}
}
object MyActorClass {
def props: Props = Props(classOf[MyActorClass ])
}
I want to create custom StatefulStage which should works like groupBy method and emit Source[A, Unit] elements but I don't understand how to create instance of Source[A, Unit] and push incoming element to it. Here is stub:
class GroupBy[A, Mat]() extends StatefulStage[A, Source[A, Unit]] {
override def initial: StageState[A, Source[A, Unit]] = new StageState[A, Source[A, Unit]] {
override def onPush(elem: A, ctx: Context[Source[A, Unit]]): SyncDirective = {
val source: Source[A, Unit] = ... // Need to create source here
// and push `elem` to `source` here
emit(List(source).iterator, ctx)
}
}
}
You can use the following snippet for test GroupBy flow (it should print events from created stream):
case class Tick()
case class Event(timestamp: Long, sessionUid: String, traffic: Int)
implicit val system = ActorSystem()
import system.dispatcher
implicit val materializer = ActorMaterializer()
var rnd = Random
rnd.setSeed(1)
val eventsSource = Source
.tick(FiniteDuration(0, SECONDS), FiniteDuration(1, SECONDS), () => Tick)
.map {
case _ => Event(System.currentTimeMillis / 1000, s"session-${rnd.nextInt(5)}", rnd.nextInt(10) * 10)
}
val flow = Flow[Event]
.transform(() => new GroupByUntil)
.map {
(source) => source.runForeach(println)
}
eventsSource
.via(flow)
.runWith(Sink.ignore)
.onComplete(_ => system.shutdown())
Can anybody explain me how to do it?
UPDATE:
I wrote the following onPush method base on this answer but it didn't print events. As I understand I can push element to source only when it running as part of flow but I want to run flow outside of GroupBy in test snippet. If I run flow in GroupBy as in this example then it will process events and send them to Sink.ignore. I think this is a reason why my test snippet didn't print events.
override def onPush(elem: A, ctx: Context[Source[A, Unit]]): SyncDirective = {
val source: Source[A, ActorRef] = Source.actorRef[A](1000, OverflowStrategy.fail)
val flow = Flow[A].to(Sink.ignore).runWith(source)
flow ! elem
emit(List(source.asInstanceOf[Source[A, Unit]]).iterator, ctx)
}
So, how to fix it?