I'm using RxJava in my Scala project and I need to execute my Observable in a separate thread. I know in order to achieve this I need to call observeOn method on it and pass an instance of rx.lang.scala.Scheduler as an argument.
But how can I create that instance? I did not find any apparent ways of instantiating of rx.lang.scala.Scheduler trait. For example, I have this code:
Observable.from(List(1,2,3)).observeOn(scheduler)
Can someone provide an example of working scheduler variable that will do the trick?
A trait is not instantiable.
You need to use one of the subclasses of the trait listed under "Known Subclasses" in the API documentation.
All schedulers are in the package
import rx.lang.scala.schedulers._
For blocking IO operations, use IO scheduler
Observable.from(List(1,2,3)).observeOn(IOScheduler())
For computational work, use computation scheduler
Observable.from(List(1,2,3)).observeOn(ComputationScheduler())
To execute on the current thread
Observable.from(List(1,2,3)).observeOn(ImmediateScheduler())
To execute on a new thread
Observable.from(List(1,2,3)).observeOn(NewThreadScheduler())
To queues work on the current thread to be executed after the current one
Observable.from(List(1,2,3)).observeOn(TrampolineScheduler())
If you want to use your own custom thread pool
val threadPoolExecutor = Executors.newFixedThreadPool(2)
val executionContext = ExecutionContext.fromExecutor(threadPoolExecutor)
val customScheduler = ExecutionContextScheduler(executionContext)
Observable.from(List(1,2,3)).observeOn(customScheduler)
Related
Given:
AtoB: GraphStageWithMaterializedValue[A, B, Future[AtoBRuntimeApi]];
BtoC: GraphStageWithMaterializedValue[B, C, Future[BtoCRuntimeApi]].
Wanted: AtoC: GraphStageWithMaterializedValue[A, C, Future[AtoCRuntimeApi]].
In my particular case it's really convenient to implement AtoCRuntimeApi in terms of both AtoBRuntimeApi and BtoARuntimeApi.
So I would like to define AtoCRuntimeApi as case class AtoCRuntimeApi(a2b: AtoBRuntimeApi, b2c: BtoCRuntimeApi);
And to define the new compound stage as stageAtoB.viaMat(stageBtoC)(combineIntoAtoC) where
combineIntoAtoC: (Future[AtoBRuntimeApi], Future[B2CRuntimeApi]) => Future[AtoCRuntimeApi].
Obviously the implementation of combineIntoAtoC requires some instance of ExecutionContext in order to map the futures.
The question: what execution context should I use in the described case?
Options, I would rather avoid:
bake in an instance that is currently available (while composition of stages) — the "blueprint" will not be safe to materialise if that execution-context becomes unavailable;
ExecutionContext.global — well, it's global. It seems terribly wrong to use it (probably, once per materialisation — is not that big deal).
The most wanted execution-context is the one that is available as the property of materializer (mat.executionContext). But there is no way I can access it within that combine-function.
I usually use the actor system context in such cases, which is indeed by default can be referred to from the materializer. As a general solution, though, you can pass the execution context to whatever function you construct the stream graph in with an implicit parameter:
def makeGraph(...)(implicit ec: ExecutionContext): RunnableGraph = {
def combineIntoAtoC(...) = ... // uses ec implicitly
}
This allows you to push the decision about which context to use up the call stack. At the appropriate level there most certainly will be some kind of access to the actor system's dispatcher.
The reason why I prefer to use the actor system's dispatcher instead of the global one is because it reduces the surface of dependencies - all execution contexts in this case come from one source, and you know how to configure them if the need arises.
In addition to the future's execution context provided by Scala:
import scala.concurrent.ExecutionContext.Implicits.global
Play provides another execution context:
import play.api.libs.concurrent.Execution.Implicits.defaultContext
When to use each in Play for Scala?
You can find an answer here:
Play's internal execution context
That question is not complete duplicate but very close, and the answer there cover your question as well.
In a short:
You must not use import scala.concurrent.ExecutionContext.Implicits.global in Play.
Response to the comment
The quote from the answer:
Instead, you would use
play.api.libs.concurrent.Execution.Implicits.defaultContext, which
uses an ActorSystem.
scala.concurrent.ExecutionContext.Implicits.global is an
ExecutionContext defined in the Scala standard library. It is a
special ForkJoinPool that using the blocking method to handle
potentially blocking code in order to spawn new threads in the pool.
You really shouldn't use this in a Play application, as Play will have
no control over it. It also has the potential to spawn a lot of
threads and use a ton of memory, if you're not careful.
As a general rule, if you need an ExecutionContext inside a method or class, require it as an implicit parameter (Scala) or a normal parameter (Java). Convention is to put this parameter last.
This rule allows the caller/creator to control where/how/when asynchronous effects are evaluated.
The main exception to this rule is when you already have an ExecutionContext and do not wish for the caller/creator to be in control of where the effects are evaluated.
I use the Spray API to listen for requests from a server. A computation in one specific scala class ends up blocking Spray from responding across the whole application. This is a slight simplification of the problem, but I can provide more info. if needed.
class SomeClass(implicit execc: ExecutionContext){
implicit val x = ...
val foo = Await.result(...someFunc(x))
}
I added this import and it resolved my issue:
import scala.concurrent.ExecutionContext.Implicits.global
Can anyone explain how or why this worked?
===================================================
Edit:
OuterClass instantiates SomeClass, but itself is never instantiated with the ExecutionContext parameter. It appears that it may be using the global execution context by default, and that is why it is blocking then?
class OuterClass(executor: ExecutionContext){
val s = new someClass
}
val x = (new OuterClass).someFunction
Spray route handler is a single actor that receives HTTP requests from Spray IO/Spray-can/library and passes them to the route handling function - essentially a partial function that has no concurrency on it's own. Thus if your route blocks, Spray will also block and requests will queue in the route handler actor queue.
There are 3 ways to properly handle blocking request processing in the route: spawn an actor per request, return Future of response or take request completion function and use it somewhere else unblocking the route (search for more detailed explanation if interested).
I can't be sure which execution context was used in your case, but it must have been very limited in terms of allocated threads and/or shared by your Spray route handler and long running task. This would lead to them both running on the same thread.
If you didn't have any execution context imported explicitly it must have been found through implicit resolution from regular scopes. You must have had one since you have it as a constructor parameter. Try to check your implicits in the scope to see which one it was. I'm curious myself whether it was one provided by Spray or something else.
I am trying to understand Scala futures coming from Java background: I understand you can write:
val f = Future { ... }
then I have two questions:
How is this future scheduled? Automatically?
What scheduler will it use? In Java you would use an executor that could be a thread pool etc.
Furthermore, how can I achieve a scheduledFuture, the one that executes after a specific time delay? Thanks
The Future { ... } block is syntactic sugar for a call to Future.apply (as I'm sure you know Maciej), passing in the block of code as the first argument.
Looking at the docs for this method, you can see that it takes an implicit ExecutionContext - and it is this context which determines how it will be executed. Thus to answer your second question, the future will be executed by whichever ExecutionContext is in the implicit scope (and of course if this is ambiguous, it's a compile-time error).
In many case this will be the one from import ExecutionContext.Implicits.global, which can be tweaked by system properties but by default uses a ThreadPoolExecutor with one thread per processor core.
The scheduling however is a different matter. For some use-cases you could provide your own ExecutionContext which always applied the same delay before execution. But if you want the delay to be controllable from the call site, then of course you can't use Future.apply as there are no parameters to communicate how this should be scheduled. I would suggest submitting tasks directly to a scheduled executor in this case.
Andrzej's answer already covers most of the ground in your question. Worth mention is that Scala's "default" implicit execution context (import scala.concurrent.ExecutionContext.Implicits._) is literally a java.util.concurrent.Executor, and the whole ExecutionContext concept is a very thin wrapper, but is closely aligned with Java's executor framework.
For achieving something similar to scheduled futures, as Mauricio points out, you will have to use promises, and any third party scheduling mechanism.
Not having a common mechanism for this built into Scala 2.10 futures is a pity, but nothing fatal.
A promise is a handle for an asynchronous computation. You create one (assuming ExecutionContext in scope) by calling val p = Promise[Int](). We just promised an integer.
Clients can grab a future that depends on the promise being fulfilled, simply by calling p.future, which is just a Scala future.
Fulfilling a promise is simply a matter of calling p.successful(3), at which point the future will complete.
Play 2.x solves scheduling by using promises and a plain old Java 1.4 Timer.
Here is a linkrot-proof link to the source.
Let's also take a look at the source here:
object Promise {
private val timer = new java.util.Timer()
def timeout[A](message: => A, duration: Long, unit: TimeUnit = TimeUnit.MILLISECONDS)
(implicit ec: ExecutionContext): Future[A] = {
val p = Promise[A]()
timer.schedule(new java.util.TimerTask {
def run() {
p.completeWith(Future(message)(ec))
}
}, unit.toMillis(duration))
p.future
}
}
This can then be used like so:
val future3 = Promise.timeout(3, 10000) // will complete after 10 seconds
Notice this is much nicer than plugging a Thread.sleep(10000) into your code, which will block your thread and force a context switch.
Also worth noticing in this example is the val p = Promise... at the function's beginning, and the p.future at the end. This is a common pattern when working with promises. Take it to mean that this function makes some promise to the client, and kicks off an asynchronous computation in order to fulfill it.
Take a look here for more information about Scala promises. Notice they use a lowercase future method from the concurrent package object instead of Future.apply. The former simply delegates to the latter. Personally, I prefer the lowercase future.
The Akka documentation says:
you may be tempted to just wrap the blocking call inside a Future and work with that instead, but this strategy is too simple: you are quite likely to find bottlenecks or run out of memory or threads when the application runs under increased load.
They suggest the following strategies:
Do the blocking call within a Future, ensuring an upper bound on the number of such calls at any point in time (submitting an unbounded number of tasks of this nature will exhaust your memory or thread limits).
Do the blocking call within a Future, providing a thread pool with an upper limit on the number of threads which is appropriate for the hardware on which the application runs.
Do you know about any implementation of those strategies?
Futures are run within execution contexts. This is obvious from the Future API: any call which involves attaching some callbacks to a future or to build a future from an arbitrary computation or from another future requires an implicitly available ExecutionContext object. So you can control the concurrency setup for your futures by tuning the ExecutionContext in which they run.
For instance, to implement the second strategy you can do something like
import scala.concurrent.ExecutionContext
import java.util.concurrent.Executors
import scala.concurrent.future
object Main extends App {
val ThreadCount = 10
implicit val executionContext = ExecutionContext.fromExecutor(Executors.newFixedThreadPool(ThreadCount))
val f = future {
println(s"Hello ! I'm running in an execution context with $ThreadCount threads")
}
}
Akka itself implements all this, you can wrap your blocking calls into Actors and then use dispatchers to control execution thread pools.