In the function method, I throw a nullPointerException and print out thread count (sout(Thread.activeCount). It appears that each time I retry on error, that the threadCount increases by 1. Why is this happening? Should I assume it will just get garbage collected?
public void start(int time) {
Observable.interval(time, TimeUnit.SECONDS)
.doOnNext(t -> function())
.doOnError(System.out::println)
.retry()
.subscribe();
}
The computation Scheduler used by that interval overload is limited to the number of CPUs (Runtime.getRuntime().availableProcessors()) by default. As you keep resubscribing due to retry, the Scheduler will spin up more and more worker threads. However, the total thread count should not grow endlessly.
If you don't want to have so many worker threads, consider using Schedulers.single() or Schedulers.io() which latter reuses workers:
Observable.interval(time, TimeUnit.SECONDS, Schedulers.single())
.doOnNext(t -> function())
.doOnError(System.out::println)
.retry()
.subscribe();
or
Observable.interval(time, TimeUnit.SECONDS, Schedulers.io())
.doOnNext(t -> function())
.doOnError(System.out::println)
.retry()
.subscribe();
Maybe Observable.interval() takes new thread from its computational scheduler thread pool every time you retry. Maybe it does so because previous thread is still in use at that point? IMHO it should not rise indefinitely, as it would indicate some bug.
Related
I read the tokio documentation and I wonder what is the best approach for encapsulating costly synchronous I/O in a future.
With the reactor framework, we get the advantage of a green threading model: a few OS threads handle a lot of concurrent tasks through an executor.
The future model of tokio is demand driven, which means the future itself will poll its internal state to provide informations about its completion; allowing backpressure and cancellation capabilities. As far as I understand, the polling phase of the future must be non-blocking to work well.
The I/O I want to encapsulate can be seen as a long atomic and costly operation. Ideally, an independent task would perform the I/O and the associated future would poll the I/O thread for the completion status.
The two only options I see are:
Include the blocking I/O in the poll function of the future.
spawn an OS thread to perform the I/O and use the future mechanism to poll its state, as shown in the documentation
As I understand it, neither solution is optimal and don't get the full advantage of the green-threading model (first is not advised in documentation and second don't pass through the executor provided by reactor framework). Is there another solution?
Ideally, an independent task would perform the I/O and the associated future would poll the I/O thread for the completion status.
Yes, this is the recommended approach for asynchronous execution. Note that this is not restricted to I/O, but is valid for any long-running synchronous task!
Futures crate
The ThreadPool type was created for this1.
In this case, you spawn work to run in the pool. The pool itself performs the work to check to see if the work is completed yet and returns a type that fulfills the Future trait.
use futures::{
executor::{self, ThreadPool},
future,
task::{SpawnError, SpawnExt},
}; // 0.3.1, features = ["thread-pool"]
use std::{thread, time::Duration};
async fn delay_for(pool: &ThreadPool, seconds: u64) -> Result<u64, SpawnError> {
pool.spawn_with_handle(async {
thread::sleep(Duration::from_secs(3));
3
})?
.await;
Ok(seconds)
}
fn main() -> Result<(), SpawnError> {
let pool = ThreadPool::new().expect("Unable to create threadpool");
let a = delay_for(&pool, 3);
let b = delay_for(&pool, 1);
let c = executor::block_on(async {
let (a, b) = future::join(a, b).await;
Ok(a? + b?)
});
println!("{}", c?);
Ok(())
}
You can see that the total time is only 3 seconds:
% time ./target/debug/example
4
real 3.010
user 0.002
sys 0.003
1 — There's some discussion that the current implementation may not be the best for blocking operations, but it suffices for now.
Tokio
Here, we use task::spawn_blocking
use futures::future; // 0.3.15
use std::{thread, time::Duration};
use tokio::task; // 1.7.1, features = ["full"]
async fn delay_for(seconds: u64) -> Result<u64, task::JoinError> {
task::spawn_blocking(move || {
thread::sleep(Duration::from_secs(seconds));
seconds
})
.await?;
Ok(seconds)
}
#[tokio::main]
async fn main() -> Result<(), task::JoinError> {
let a = delay_for(3);
let b = delay_for(1);
let (a, b) = future::join(a, b).await;
let c = a? + b?;
println!("{}", c);
Ok(())
}
See also CPU-bound tasks and blocking code in the Tokio documentation.
Additional points
Note that this is not an efficient way of sleeping, it's just a placeholder for some blocking operation. If you actually need to sleep, use something like futures-timer or tokio::time::sleep. See Why does Future::select choose the future with a longer sleep period first? for more details
neither solution is optimal and don't get the full advantage of the green-threading model
That's correct - because you don't have something that is asynchronous! You are trying to combine two different methodologies and there has to be an ugly bit somewhere to translate between them.
second don't pass through the executor provided by reactor framework
I'm not sure what you mean here. There's an executor implicitly created by block_on or tokio::main. The thread pool has some internal logic that checks to see if a thread is done, but that should only be triggered when the user's executor polls it.
From this tutorial https://github.com/slouc/concurrency-in-scala-with-ce#threading
async operations are divided into 3 groups and require significantly different thread pools to run on:
Non-blocking asynchronous operations:
Bounded pool with a very low number of threads (maybe even just one), with a very high priority. These threads will basically just sit idle most of the time and keep polling whether there is a new async IO notification. Time that these threads spend processing a request directly maps into application latency, so it's very important that no other work gets done in this pool apart from receiving notifications and forwarding them to the rest of the application.
Bounded pool with a very low number of threads (maybe even just one), with a very high priority. These threads will basically just sit idle most of the time and keep polling whether there is a new async IO notification. Time that these threads spend processing a request directly maps into application latency, so it's very important that no other work gets done in this pool apart from receiving notifications and forwarding them to the rest of the application.
Blocking asynchronous operations:
Unbounded cached pool. Unbounded because blocking operation can (and will) block a thread for some time, and we want to be able to serve other I/O requests in the meantime. Cached because we could run out of memory by creating too many threads, so it’s important to reuse existing threads.
CPU-heavy operations:
Fixed pool in which number of threads equals the number of CPU cores. This is pretty straightforward. Back in the day the "golden rule" was number of threads = number of CPU cores + 1, but "+1" was coming from the fact that one extra thread was always reserved for I/O (as explained above, we now have separate pools for that).
In my Cats Effect application, I use Scala Future-based ReactiveMongo lib to access MongoDB, which does NOT block threads when talking with MongoDB, e.g. performs non-blocking IO.
It needs execution context.
Cats effect provides default execution context IOApp.executionContext
My question is: which execution context should I use for non-blocking io?
IOApp.executionContext?
But, from IOApp.executionContext documemntation:
Provides a default ExecutionContext for the app.
The default on top of the JVM is lazily constructed as a fixed thread pool based on the number available of available CPUs (see PoolUtils).
Seems like this execution context falls into 3rd group I listed above - CPU-heavy operations (Fixed pool in which number of threads equals the number of CPU cores.),
and it makes me think that IOApp.executionContext is not a good candidate for non-blocking IO.
Am I right and should I create a separate context with a fixed thread pool (1 or 2 threads) for non-blocking IO (so it will fall into the first group I listed above - Non-blocking asynchronous operations: Bounded pool with a very low number of threads (maybe even just one), with a very high priority.)?
Or is IOApp.executionContext designed for both CPU-bound and Non-Blocking IO operations?
The function I use to convert Scala Future to F and excepts execution context:
def scalaFutureToF[F[_]: Async, A](
future: => Future[A]
)(implicit ec: ExecutionContext): F[A] =
Async[F].async { cb =>
future.onComplete {
case Success(value) => cb(Right(value))
case Failure(exception) => cb(Left(exception))
}
}
In Cats Effect 3, each IOApp has a Runtime:
final class IORuntime private[effect] (
val compute: ExecutionContext,
private[effect] val blocking: ExecutionContext,
val scheduler: Scheduler,
val shutdown: () => Unit,
val config: IORuntimeConfig,
private[effect] val fiberErrorCbs: FiberErrorHashtable = new FiberErrorHashtable(16)
)
You will almost always want to keep the default values and not fiddle around declaring your own runtime, except in perhaps tests or educational examples.
Inside your IOApp you can then access the compute pool via:
runtime.compute
If you want to execute a blocking operation, then you can use the blocking construct:
blocking(IO(println("foo!"))) >> IO.unit
This way, you're telling the CE3 runtime that this operation could be blocking and hence should be dispatched to a dedicated pool. See here.
What about CE2? Well, it had similar mechanisms but they were very clunky and also contained quite a few surprises. Blocking calls, for example, were scheduled using Blocker which then had to be somehow summoned out of thin air or threaded through the whole app, and thread pool definitions were done using the awkward ContextShift. If you have any choice in the matter, I highly recommend investing some effort into migrating to CE3.
Fine, but what about Reactive Mongo?
ReactiveMongo uses Netty (which is based on Java NIO API). And Netty has its own thread pool. This is changed in Netty 5 (see here), but ReactiveMongo seems to still be on Netty 4 (see here).
However, the ExecutionContext you're asking about is the thread pool that will perform the callback. This can be your compute pool.
Let's see some code. First, your translation method. I just changed async to async_ because I'm using CE3, and I added the thread printline:
def scalaFutureToF[F[_]: Async, A](future: => Future[A])(implicit ec: ExecutionContext): F[A] =
Async[F].async_ { cb =>
future.onComplete {
case Success(value) => {
println(s"Inside Callback: [${Thread.currentThread.getName}]")
cb(Right(value))
}
case Failure(exception) => cb(Left(exception))
}
}
Now let's pretend we have two execution contexts - one from our IOApp and another one that's going to represent whatever ReactiveMongo uses to run the Future. This is the made-up ReactiveMongo one:
val reactiveMongoContext: ExecutionContext =
ExecutionContext.fromExecutor(Executors.newFixedThreadPool(1))
and the other one is simply runtime.compute.
Now let's define the Future like this:
def myFuture: Future[Unit] = Future {
println(s"Inside Future: [${Thread.currentThread.getName}]")
}(reactiveMongoContext)
Note how we are pretending that this Future runs inside ReactiveMongo by passing the reactiveMongoContext to it.
Finally, let's run the app:
override def run: IO[Unit] = {
val myContext: ExecutionContext = runtime.compute
scalaFutureToF(myFuture)(implicitly[Async[IO]], myContext)
}
Here's the output:
Inside Future: [pool-1-thread-1]
Inside Callback: [io-compute-6]
The execution context we provided to scalaFutureToF merely ran the callback. Future itself ran on our separate thread pool that represents ReactiveMongo's pool. In reality, you will have no control over this pool, as it's coming from within ReactiveMongo.
Extra info
By the way, if you're not working with the type class hierarchy (F), but with IO values directly, then you could use this simplified method:
def scalaFutureToIo[A](future: => Future[A]): IO[A] =
IO.fromFuture(IO(future))
See how this one doesn't even require you to pass an ExecutionContext - it simply uses your compute pool. Or more specifically, it uses whatever is defined as def executionContext: F[ExecutionContext] for the Async[IO], which turns out to be the compute pool. Let's check:
override def run: IO[Unit] = {
IO.executionContext.map(ec => println(ec == runtime.compute))
}
// prints true
Last, but not least:
If we really had a way of specifying which thread pool ReactiveMongo's underlying Netty should be using, then yes, in that case we should definitely use a separate pool. We should never be providing our runtime.compute pool to other runtimes.
I created this example of groupBy -> map -> mergeSubstreamsWithParallelism using Akka streams. On the course that I am doing, it says that the groupBy will create X substreams regarding the parameter that I pass to it and then I have to merge the substreams to a single stream. So, I understand that the map operator is running in parallel. Is that right?
If so, why I can see the same thread executing the map operator in this code:
val textSource = Source(List(
"I love Akka streams", // odd
"this has even characters", // even
"this is amazing", // odd
"learning Akka at the Rock the JVM", // odd
"Let's rock the JVM", // even
"123", // odd
"1234" // even
))
val totalCharCountFuture = textSource
.groupBy(2, string => string.length % 2)
.map { c =>
println(s"I am running on thread [${Thread.currentThread().getId}]")
c.length
}// .async // this operator runs in parallel
.mergeSubstreamsWithParallelism(2)
.toMat(Sink.reduce[Int](_ + _))(Keep.right)
.run()
totalCharCountFuture.onComplete {
case Success(value) => println(s"total char count: $value")
case Failure(exception) => println(s"failed computation: $exception")
}
the output:
I am running on thread [16]
I am running on thread [16]
I am running on thread [16]
I am running on thread [16]
I am running on thread [16]
I am running on thread [16]
I am running on thread [16]
total char count: 116
then I added the .async to make the operator run in asynchronously. Then my output shows different threads executing the map operator:
I am running on thread [21]
I am running on thread [21]
I am running on thread [21]
I am running on thread [20]
I am running on thread [20]
I am running on thread [20]
I am running on thread [20]
I read the documentation at Akka docs about the async boundary:
Put an asynchronous boundary around this Flow. If this is a SubFlow
(created e.g. by groupBy), this creates an asynchronous boundary
around each materialized sub-flow, not the super-flow. That way, the
super-flow will communicate with sub-flows asynchronously.
So, do I need the .async after the groupBy to ensure that all substreams are executing in parallel or not? Is this test that I am doing validity the parallelism of an operator in Akka stream?
Thanks
So, do I need the .async after the groupBy to ensure that all substreams are executing in parallel or not? Is this test that I am doing validity the parallelism of an operator in Akka stream?
The short answer is "yes", you need async.
As a rule of thumb in Akka Streams (and other reactive streams spec implementation like RxJava or Project Reactor), you need to explicitly demarcate async boundaries. By default the streams are executed single threaded (or single actor in case of Akka Streams). That includes operators like groupBy. This may seem a bit counter intuitive at first, but when you think about it, parallel execution is not really a must in groupBy semantics, even though often you want parallel execution because it's the very reason you apply groupBy, be it to use all cores available for some computation task or perhaps to call some external service in parallel and get better throughput. In those cases you need to explicitly code for that parallelism to occur. One way is using async as you did in your example, where the stream execution implementation logic would introduce that parallelism or you could also user mapAsync where the parallelism is introduced by some means external to the stream execution logic.
I have some async (ZIO) code, which I need to test. If I create a testing part using Thread.sleep() it works fine and I always get response:
for {
saved <- database.save(smth)
result <- eventually {
Thread.sleep(20000)
database.search(...)
}
} yield result
But if I made same logic using timeout and interval from eventually then it never works correctly ( I got timeouts):
for {
saved <- database.save(smth)
result <- eventually(timeout(Span(20, Seconds)), interval(Span(20, Seconds))) {
database.search(...)
}
} yield result
I do not understand why timeout and interval works different then Thread.sleep. It should be doing exactly same thing. Can someone explain it to me and tell how I should change this code to do not need to use Thread.sleep()?
Assuming database.search(...) returns ZIO[] object.
eventually{database.search(...)} most probably succeeds immediately after the first try.
It successfully created a task to query the database.
Then database is queried without any retry logic.
Regarding how to make it work:
val search: ZIO[Any, Throwable, String] = ???
val retried: ZIO[Any with Clock, Throwable, Option[String]] = search.retry(Schedule.spaced(Duration.fromMillis(1000))).timeout(Duration.fromMillis(20000))
Something like that should work. But I believe that more elegant solutions exist.
The other answer from #simpadjo addresses the "what" quite succinctly. I'll add some additional context as to why you might see this behavior.
for {
saved <- database.save(smth)
result <- eventually {
Thread.sleep(20000)
database.search(...)
}
} yield result
There are three different technologies being mixed here which is causing some confusion.
First is ZIO which is an asynchronous programming library that uses it's own custom runtime and execution model to perform tasks. The second is eventually which comes from ScalaTest and is useful for checking asynchronous computations by effectively polling the state of a value. And thirdly, there is Thread.sleep which is a Java api that literally suspends the current thread and prevents task progression until the timer expires.
eventually uses a simple retry mechanism that differs based on whether you are using a normal value or a Future from the scala standard library. Basically it runs the code in the block and if it throws then it sleeps the current thread and then retries it based on some interval configuration, eventually timing out. Notably in this case the behavior is entirely synchronous, meaning that as long as the value in the {} doesn't throw an exception it won't keep retrying.
Thread.sleep is a heavy weight operation and in this case it is effectively blocking the function being passed to eventually from progressing for 20 seconds. Meaning that by the time the database.search is called the operation has likely completed.
The second variant is different, it executes the code in the eventually block immediately, if it throws an exception then it will attempt it again based on the interval/timeout logic that your provide. In this scenario the save may not have completed (or propagated if it is eventually consistent). Because you are returning a ZIO which is designed not to throw, and eventually doesn't understand ZIO it will simply return the search attempt with no retry logic.
The accepted answer:
val retried: ZIO[Any with Clock, Throwable, Option[String]] = search.retry(Schedule.spaced(Duration.fromMillis(1000))).timeout(Duration.fromMillis(20000))
works because the retry and timeout are using the built-in ZIO operators which do understand how to actually retry and timeout a ZIO. Meaning that if search fails the retry will handle it until it succeeds.
Is there any way to interrupt a parallel collection computation in Scala?
Example:
val r = new Runnable {
override def run(): Unit = {
(1 to 3).par.foreach { _ => Thread.sleep(5000000) }
}
}
val t = new Thread(r)
t.start()
Thread.sleep(300) // let them spin up
t.interrupt()
I'd expect t.interrupt to interrupt all threads spawned by par, but this is not happening, it keeps spinning inside ForkJoinTask.externalAwaitDone. Looks like that method clears the interrupted status and keeps waiting for the spawned threads to finish.
This is Scala 2.12
The thread that you t.start() is responsible just for starting parallel computations and to wait and gather the result.
It is not connected to threads that compute operations. Usually, it runs on default forkJoinPool that independent from the thread that submits computation tasks.
If you want to interrupt the computation, you can use custom execution back-end (like manually created forkJoinPool or a threadPool), and then shut it down. You can read about that here.
Or you can provide a callback from the computation.
But all those approaches are not so good for such a case.
If you producing a production solution or your case is complex and critical for the app, you probably should use something that has cancellation by design. Like Monix.Task or CancellableFuture.
Or at least use Future and cancel it with workarounds.