https://projectreactor.io/docs/core/release/api/reactor/core/publisher/doc-files/marbles/flatMapForFlux.svg
This question is regarding the flatmap function in Flux publisher in project reactor. According to our understanding within a flatmap we have to return a publisher and flatmap will internally subscribe to that publisher. What we want to know is how the threads handle this scenario. Is it going to be multiple threads or single thread?
Map and flat maps are internally handled by the reactor-event treads.
See the following image:
Related
As per the documentation:
Flux is a stream which can emit 0..N elements:
Flux<String> fl = Flux.just("a", "b", "c");
Mono is a stream of 0..1 elements:
Mono<String> mn = Mono.just("hello");
And as both are the implementations of the Publisher interface in the reactive stream.
Can't we use only Flux in most of the cases as it also can emit 0..1, thus satisfying the conditions of a Mono?
Or there are some specific conditions when only Mono needs to be used and Flux can not handle the operations?
Please suggest.
In many cases, you are doing some computation or calling a service and you expect exactly one result (or maybe zero or one result), and not a collection that contains possibly multiple results. In such cases, it's more convenient to have a Mono.
Compare it to "regular" Java: you would not use List as the return type of any method that can return zero or one result. You would use Optional instead, which makes it immediately clear that you do not expect more than one result.
Flux is equivalent to RxJava Observable is capable of emitting
- zero or more item (streams of many elements)
- and then OPTIONALLY , completing OR failing
Mono can only emit one item at the most (streams one element)
Relations:
If you concatente two Monos you will get a Flux
You can call single() on Flux to return a Mono
From the docs here
This distinction carries a bit of semantic information into the type, indicating the rough cardinality of the asynchronous processing. For instance, an HTTP request produces only one response, so there is not much sense in doing a count operation. Expressing the result of such an HTTP call as a Mono thus makes more sense than expressing it as a Flux, as it offers only operators that are relevant to a context of zero items or one item.
Simply as the Mono is used for handling zero or one result, while the Flux is used to handle zero to many results, possibly even infinite results.
And both two in common behave in a purely asynchronous and fully non-blocking.
I think it is good practice to use Mono in cases where we know we can only get one result. In this way, we make it known to other developers working on the same thing that the result can be 0 or 1.
We are following that approach on all our projects.
Here is one good tutorial on Reactive Streams and the uses of Mono and Flux -> Reactive programming in Java.
Trying to migrate to rx-java2and came across a problem with resubscribing the shared observable inside it's own flatMap. Need this pattern to get-update-refresh chain:
Get current data from network (shared observable to avoid multiple network requests if source is being subscribed by several observers at the same time).
Modify the data and send it back to server (completable)
Get the data again after update completes
The whole thing looks like this:
#Test fun sharedTest() {
val o = Observable.just(1).share()
assertEquals(1, o
.take(1)
.flatMap({
Completable.complete()
.andThen(o) })
.blockingFirst())
}
The test fails with: java.util.NoSuchElementException
If o is not shared everything works.
That behavior seems to be because the latter subscriber comes when a single value of original has already been dispatched and only onComplete event is to be seen.
Does anybody know is that a by-design behavior and documented somehow? There is a workaround of course but I need to know the cause, as this is a bit annoying. The approach worked in Rx 1.x
Currently using version 2.1.3
Edit:
Seems to be no legitimate way to "restart" a shared observable and its side-effects as there is no guarantee other subscribers are not listening at the moment.
Take a look at the bubble diagram for 'share' and you'll see why it behaves like that: Observable.share().
share() emits items that are emitted after the subscription, it does not re-emit previously emitted items. Take a look at Observable.replay() for the behavior that should be what you expect.
Seems to be no legitimate way to "restart" a shared observable and its side-effects as there is no guarantee other subscribers are not listening at the moment.
I'm wondering if Futures are better to be used in conjunction with Actors only, rather than in a program that does not use Actor. Said differently, is performing asynchronous computation with future something that should better be done within an Actors system?
Here why i'm saying that:
1 -
You perform a computation for which the result, would trigger some action that you may do in another thread.
For instance, i have a long operation to determine the price of something, from my main thread, i decide to launch an asynchronous process for it. In the mean time i could be doing other thing, then when the response is ready/availble or communicated back to me, i go on on that path.
I can see that with actor this is handy, because you can pipe a result to an actor. But with a typical threading model, you can either block or .... ?
2 -
Another issue, let say i need to update the age of a list of participant, by getting some information online. Let assume i just have one future for that task. Isn't closing over the participant list something wrong to do. Multiple thread maybe accessing that participant list at the same time. So making the update within the future would simply be wrong and in that case, we would need java concurrent collection isn't it ?
Maybe i see it the wrong way, future are not meant to do side effect
at all
But in that case, fair enough, no side effect, but we still have the problem of getting a value back from the calling thread, which can only be blocking. I mean let's imagine that, the result, would help the calling thread, to update some data structure. How to do that update asynchronously without closing over that data structure somehow.
I believe the call backs such as OnComplete can be use for
side-effecting (Am it right here?)
still, the call back would have to close over the data structure anyway. Hence i don't see how not using Actor.
PS: I like actors, i'm just trying to understand better the usage of future without actors. I read everywhere, that one should use actor only when necessary that is when state need to be manage. It seems to me that overall, using future, without actor, always involve blocking somewhere down the line, if the result need to be communicated back at some point to the thread that initiated the asynchronous task.
Actors are good when you are dealing with mutable state because they encapsulate the mutable state. and allow only message-based interaction.
You can use Future to execute in a different thread. You don't have to block on a Future because Scala's Future compose. So if you have multiple Futures in your code, you don't have to wait/block for all of them to compete. For example, if your pipeline is completely non-block or asyn (e.g., Play and Spray) you can return a Future back to the client.
Futures are lightweight compared to actors because you don't need a complete actorsystem.
Here is a quote from Martin Odersky that I really like.
There is no silver bullet for all concurrency issues; the right
solution depends on what one needs to achieve. Do you want to define
asynchronous computations that react to events or streams of values?
Or have autonomous, isolated entities communicating via messages? Or
define transactions over a mutable store? Or, maybe the primary
purpose of parallel execution is to increase the performance? For each
of these tasks, there is an abstraction that does the job: futures,
reactive streams, actors, transactional memory, or parallel
collections.
So choose your abstraction based on your use case and needs.
How well to scala parallel collection operations get along with the concurrency/parallelism used by Akka Actors (and Futures) with respect to efficient scheduling on the system?
Actors' and Futures' execution is handled by an ExecutionContext generally provided by the Dispatcher. What I find on parallel collections indicates they use a TaskSupport object. I found a ExecutionContextTaskSupport object that may connect the two but am not sure.
What is the proper way to mix the two concurrency solutions, or is it advised not to?
At present this is not supported / handled well.
Prior to Scala 2.11-M7, attempting to use the dispatcher as the ContextExecutor throws an exception.
That is, the following code in an actor's receive will throw a NotImplementedError:
val par = List(1,2,3).par
par.tasksupport = new ExecutionContextTaskSupport(context.dispatcher)
par foreach println
Incidentally, this has been fixed in 2.11-M7, though it was not done to correct the above issue.
In reading through the notes on the fix it sounds like the implementation provided by ExecutionContextTaskSupport in the above case could have some overhead over directly using one of the other TaskSupport implementations; however, I have done nothing to test that interpretation or evaluate the magnitude of any impact.
A Note on Parallel Collections:
By default Parallel Collections will use the global ExecutorContext (ExecutionContext.Implicits.global) just as you might use for Futures. While this is well behaved, if you want to be constrained by the dispatcher (using context.dispatcher)—as you are likely to do with Futures in Akka—you need to set a different TaskSupport as shown in the code sample above.
Having watched the presentation Composable Futures with Akka 2.0, I am curious to know what additional features the Akka implementation of Futures and Agents bring over Clojure's ones.
"Agents in Akka are inspired by agents in Clojure." This is the first line in Agent documentation on Akka and hopefully it clears the agents part of question. As far as futures are concerned, they are both same conceptually (i.e invoking an operation on a separate thread). The underlying implementation are based on java.util.concurrent, so both using same underlying infrastructure.
Scala part:
The important part how the composable word come into play (both for agents and futures). If you go to akka docs you will find that you can use higher-order functions like map, filter etc. on Akka futures, i.e. a map operation on a future returns another future (and similarly for filter). This allows you to easily compose/chain together futures and wait on the final future for the final value. Now, all this is possible because the map, filter, for comprehension etc. are all based on Scala (monadic) API which basically allows any new type to provide specific implementations of these functions.
Clojure part:
Now on clojure side of things, you know that map, filter etc are just normal functions that work on collections, i.e they work on something that can be traversed and hence are different concept from the Monadic API of scala. So in Clojure you will use different ways to compose futures after all Clojure (or Lisp in general) allow composability in many many ways