I know type of synch vs. async where Observable can be processed on main thread or on background thread specified by subscribeOn (potentially observeOn). This case is clear.
Can you explain me where is a different between synchrounous and asychronous operator? Asynchronous operator can change order of items from upstream?
Which operators are asynchronous? Only merge and flatMap?
When all operators in chain are synchronous that emitted item must be processed using entire chain before next item can be processed? Chain with map and filter.
Any operator that has to deal with multiple sources or takes a Scheduler is considered an asynchronous boundary - async for short.
Multi-source operators may encounter signals from multiple threads; even if there was a total order of these signals, some operators, such as merge() may not emit them in the same order. Generally though, signals from a source may end up interleaved with signals from other sources.
Synchronous chains go in deep into the chain of onNext calls and unless there are some buffering happening, they have to return up to the top-source for the next element.
Related
Can I have multiple asynchronous operations that call controller.add(myEvent) on a single broadcast StreamController? In other words, can I have multiple producers that emit stream events on the same broadcast stream?
The reason for this question is because the documentation states that it s
not allowed to call add, addError, or close before a previous call has returned.
However, add is a synchronous operation that returns void (rather than a Future). How am I to understand this statement?
I want to implement a complicated reactive program in vertx, which contains multiple blocking operation steps. There seems several ways to implement it AFAIK, there may be other ways as well, what is the most efficient way in terms of throughout and response time, in a multi-core computer?
Separate each operation step in different verticles, and use event bus to communicate with these verticles.
Make all operations in one verticle, chain all operations with Future composition
Make all operations in one verticle, chain all operations with RxJava 2
According to Vertx core document, "There is a single event bus instance for every Vert.x instance and it is obtained using the method eventBus", the 1st way seems less efficient than others because the data transmission between verticles is in a single event bus thread, while for others, multiple instances of the verticle can be created so that more cores are used as event loop thread. Do I understand correctly?
I'm trying out some of the operations that I could do on the Observable from Monix. I came across this debounce operator and could not understand its behavior:
Observable.interval(5.seconds).debounce(2.seconds)
This one above just emits a Long every 5 seconds.
Observable.interval(2.seconds).debounce(5.seconds)
This one however does not emit anything at all. So what is the real purpose of the debounce operator and in which cases could I use it?
The term debounce comes from mechanical relays. You can think of it as a frequency filter: o.debounce(5.seconds) filters out any events that are emitted more frequently than once every 5 seconds.
An example of where I've used it is where I expect to get a batch of similar events in rapid succession, and my response to each event is the same. By debouncing I can reduce the amount of work I need to do by making the batch look like just one event.
It isn't useful in situations like your examples where the input frequency is constant, as the only possibilities are that it does nothing or it filters out everything.
Have been trying to google this but getting a bit stuck.
Let's say we have a class that fires an event, and that event could be fired by several threads at the same time.
Using Observable.FromEventPattern, we create an Observable, and subscribe to that event. How exactly does Rx manage multiple those events being fired at once? Let's say we have 3 events fired in quick succession on different threads. Does it queue them internally, and then call the Subscribe delegate synchronously for each one? Let's say we were subscribing on a thread pool, can we still guarantee the Subscriptions would be processed separately in time?
Following on from that, let's say for each event, we want to perform an action, but it's a method that's potentially not thread safe, so we only want one thread to be in this method at a time. Now I see we can use an EventLoop Scheduler, and presumably we wouldn't need to implement any locking on the code?
Also, would observing on the Current Thread be an option? Is Current Thread the thread that the event was fired from, or the event the subscription was set up on? i.e. Is that current thread guaranteed to always be the same or could be have 2 threads running ending up in the method at the same time?
Thx
PS: I put an example together but I always seem to end up on the samethread in my subscrive method, even when I ObserveOn the threadpool, which is confusing :S
PSS: From doing a few more experiments, it seems that if no Schedulers are specified, then RX will just execute on whatever thread the event was fired on, meaning it processes several concurrently. As soon as I introduce a scheduler, it always runs things consecutively, no matter what the type of the scheduler is. Strange :S
According to the Rx Design Guidelines, an observable should never call OnNext of an observer concurrently. It will always wait for the current call to complete before making the next call. All Rx methods honor this convention. And, more importantly, they assume you also honor this convention. When you violate this condition, you may encounter subtle bugs in the behavior of your Observable.
For those times when you have source data that does not honor this convention (ie it can produce data concurrently), they provide Synchronize.
Observable.FromEventPattern assumes you will not be firing concurrent events and so does nothing to prevent concurrent downstream notifications. If you plan on firing events from multiple threads, sometimes concurrently, then use Synchronize() as the first operation you do after FromEventPattern:
// this will get you in trouble if your event source might fire events concurrently.
var events = Observable.FromEventPattern(...).Select(...).GroupBy(...);
// this version will protect you in that case.
var events = Observable.FromEventPattern(...).Synchronize().Select(...).GroupBy(...);
Now all of the downstream operators (and eventually your observer) are protected from concurrent notifications, as promised by the Rx Design Guidelines. Synchronize works by using a simple mutex (aka the lock statement). There is no fancy queueing or anything. If one thread attempts to raise an event while another thread is already raising it, the 2nd thread will block until the first thread finishes.
In addition to the recommendation to use Synchronize, it's probably worth having a read of the Intro to Rx section on scheduling and threading. It Covers the different schedulers and their relationship to threads, as well as the differences between ObserveOn and SubscribeOn, etc.
If you have several producers then there are RX methods for combining them in a threadsafe way
For combining streams of the same type of event into a single stream
Observable.Merge
For combining stream of different types of events into a single stream using a selector to transform the latest value on each stream into a new value.
Observable.CombineLatest
For example combining stock prices from different sources
IObservable<StockPrice> source0;
IObservable<StockPrice> source1;
IObservable<StockPrice> combinedSources = source0.Merge(source1);
or create balloons at the current position every time there is a click
IObservable<ClickEvent> clicks;
IObservable<Position> position;
IObservable<Balloons> balloons = clicks
.CombineLatest
( positions
, (click,position)=>new Balloon(position.X, position.Y)
);
To make this specifically relevant to your question you say there is a class which combines events from different threads. Then I would use Observable.Merge to combine the individual event sources and expose that as an Observable on your main class.
BTW if your threads are actually tasks that are firing events to say they have completed here is an interesting patterns
IObservable<Job> jobSource;
IObservable<IObservable<JobResult>> resultTasks = jobSource
.Select(job=>Observable.FromAsync(cancelationToken=>DoJob(token,job)));
IObservable<JobResult> results = resultTasks.Merge();
Where what is happening is you are getting a stream of jobs in. From the jobs you are creating a stream of asynchronous tasks ( not running yet ). Merge then runs the tasks and collects the results. It is an example of a mapreduce algorithm. The cancellation token can be used to cancel running async tasks if the observable is unsubscribed from (ie canceled )
Is there a good reason for the added complexity of Futures (vs parallel collections) when processing a list of items in parallel?
List(...).par.foreach(x=>longRunningAction(x))
vs
Future.traverse(List(...)) (x=>Future(longRunningAction(x)))
I think the main advantage would be that you can access the results of each future as soon as it is computed, while you would have to wait for the whole computation to be done with a parallel collection. A disadvantage might be that you end up creating lots of futures. If you later end up calling Future.sequence, there really is no advantage.
Parallel collections will kill of some threads as we get closer to processing all items. So last few items might be processed by single thread.
Please see my question for more details on this behavior Using ThreadPoolTaskSupport as tasksupport for parallel collections in scala
Future does no such thing, and all your threads are in use until all objects are processed. Hence unless your tasks are so small that you dont care about loss of parallelism for last few tasks and you are using huge number of threads, which have to killed of as soon as possible, Futures are better.
Futures become useful as soon as you want to compose your deferred / concurrent computations. Futures (the good kind, anyway, such as Akka's) are monadic and hence allow you to build arbitrarily complex computational structures with all the concurrency and synchronization handled properly by the Futures library.