I have a cache of observables and reuse them. They normally all use some sort of caching (mostly replay(1).refCount()) and I make sure, that the underlying calculation is done once only with this.
I now have cases, where the underlying stream emits items and noone is subscribed to my cached observable. I still want it to process this event. How can I do this?
Currently I only can do this like following:
val o = observable.reply(1)
o.connect() // make sure this hot observable always is connected and processes it's input
return o // this one is cached
Is there some better way? I want that the hot observable always acts as if someone is subscribed and never unsubscribes from the upstream...
Background
I have redux store like observables and those need to process EVERY input, no matter if someone is subscribed or not so that the cached values that a replayed are always the newest one...
IMO the correct answer is by #prom85 in the question comment section.
From the Learning RxJava Book by Thomas Nield
If you pass 0 to autoConnect() for the numberOfSubscribers argument,
it will start firing immediately and not wait for any Observers. This
can be handy to start firing emissions immediately without waiting for
any Observers.
If we had a hot Flowable (for example PublishProcessor) with a backpressure strategy. How it works when there are several subscribers and only one of them has problems processing received elements?
The subscribers that can attend the messages at the time are affected by the backpressure strategy that is needed by the slower one?
EDIT:
After reading #akarnokd suggested documentation and looking at the diagram I've seen that there isn't any backpressure management for PublishProcessor.
But what if I chain a .onBackPressureXXXX(). To my understanding, it seems that the strategy will be applied to each subscriber, but just want to be sure be getting a confirmation from an experimented developer.
As per the Reactive Stream paradigm,
Currently, we’ve focused primarily on cold streams. These are static, fixed length streams which are easy to deal with. A more realistic use case for reactive might be something that happens infinitely. For example, we could have a stream of mouse movements which constantly needs to be reacted to or a twitter feed. These types of streams are called hot streams, as they are always running and can be subscribed to at any point in time, missing the start of the data.
So how can we implement this hot stream?
This can be done using the ConnectableFlux as follows:
ConnectableFlux<Object> publish = Flux.create(fluxSink -> {
while(true) {
fluxSink.next(System.currentTimeMillis());
}
})
.publish();
You could use MongoDB's Capped Collections to create a hot stream. By using a #Tailable on that collection it will create a publisher which publishes every new entry. With .share() it will multicast that publisher, so that not every subscription creates a new database connection.
Let's say I have deployed an NSB endpoint that subscribes to events A,B, and C.
6 months later, version 1.1 of the endpoint adds a handler for event D, but the handler for event B is removed. What is a sensible process for removing the persisted subscription record for event B? I presume there is no automagic way for this to happen, and my choices would be:
Delete the entire contents of the subscription table and restart all endpoints.
Delete selectively based on what I know about the delta
Have some shutdown mode where my subscriber would call Unsubscribe on all its message types on the way down (and therefore would start with a clean slate on the way up)
Has anyone implemented any of these strategies, or am I missing some alternative?
The best solution would probably be option 1. The operational overhead involved in this would be fairly small:
Shut down publisher host
Clear down subscriptions db
Bounce all subscribers
Start up publisher host
Option 3 would also be possible but would involve making an unsubscribe call from every subscriber which is IMO much higher overhead (plus would require a redeployment if unscubscribe call not already implemented and then a shutdown to trigger the call).
Option 2 seems a bit hacky but would be lowest cost as you can just run a sql statement against the publisher db and bob's your mother's brother.
I would recommend option 1.
I watched the video and I know the general principles - hot happens even when nobody is subscribed, cold happens "on demand".
Also, Publish() converts cold to hot and Defer() converts hot to cold.
But still, I feel I am missing the details. Here are some questions I'd like to have answered:
Can you give a comprehensive definition for these terms?
Does it ever make sense to call Publish on a hot observable or Defer on a cold?
What are the aspects of Hot/Cold conversions - do you lose messages, for example?
Are there differences between hot and cold definitions for IObservable and IEnumerable?
What are the general principles you should take into account when programming for cold or hot?
Any other tips on hot/cold observables?
From: Anton Moiseev's Book “Angular Development with Typescript, Second Edition.” :
Hot and cold observables
There are two types of observables: hot and cold. The main
difference is that a cold observable creates a data
producer for each subscriber, whereas a hot observable
creates a data producer first, and each subscriber gets the
data from one producer, starting from the moment of subscription.
Let’s compare watching a movie on Netflix to going into a
movie theater. Think of yourself as an observer. Anyone who decides to watch Mission: Impossible on Netflix will get the entire
movie, regardless of when they hit the play button. Netflix creates a
new producer to stream a movie just for you. This is a cold
observable.
If you go to a movie theater and the showtime is 4 p.m., the producer
is created at 4 p.m., and the streaming begins. If some people
(subscribers) are late to the show, they miss the beginning of the
movie and can only watch it starting from the moment of arrival. This
is a hot observable.
A cold observable starts producing data when some code invokes a
subscribe() function on it. For example, your app may declare an observable providing a URL on the server to get certain products. The
request will be made only when you subscribe to it. If another script
makes the same request to the server, it’ll get the same set of data.
A hot observable produces data even if no subscribers are
interested in the data. For example, an accelerometer in your
smartphone produces data about the position of your device, even if no
app subscribes to this data. A server can produce the latest stock
prices even if no user is interested in this stock.
Hot observables are ones that are pushing event when you are not subscribed to the observable. Like mouse moves, or Timer ticks or anything like that. Cold observables are ones that start pushing only when you subscribe, and they start over if you subscribe again.
I hope this helps.
Can you give a comprehensive
definition for these terms?
See my blog post at: https://leecampbell.com/2010/08/19/rx-part-7-hot-and-cold-observables
Does it ever make sense to call
Publish on a hot observable or Defer
on a cold?
No, not that I can think of.
What are the aspects of Hot/Cold
conversions - do you lose messages,
for example?
It is possible to "lose" messages when the Observable is Hot, as "events" happen regardless of subscribers.
Are there differences between hot and
cold definitions for IObservable and
IEnumerable?
I dont really understand the question. I hope this analogy helps though. I would compare a Hot Observable to an Eagerly evaluated IEnumerable. ie a List or an Array are both Eagerly evaluated and have been populated even if no-one enuemerates over them. A yield statement that gets values from a file or a database could be lazily evaluated with the Yield keyword. While lazy can be good, it will by default, be reevaluated if a second enumerator runs over it. Comparing these to Observables, a Hot Observable might be an Event (Button click) or a feed of temperatures; these events will happen regardless of a subscription and would also be shared if multiple subscriptions were made to the same observale. Observable.Interval is a good example of a Cold observable. It will only start producing values when a subscription is made. If multiple subscriptions as made then the sequence will be re-evaluated and the "events" will occur at seperate times (depending on the time between subscriptions).
What are the general principles you should take into account when programming for cold or hot?
Refer to the link in point one. I would also recommend you look into Publsh being used in conjunction with RefCount. This allows you to have the ability to have Lazy evaluation semantics of Cold Observables but the sharing of events that Hot Observables get.
Any other tips on hot/cold
observables?
Get your hands dirty and have a play with them. Once you have read about them for more than 30minutes, then time spent coding with them is far more productive to you than reading any more :)
Not pretending to give a comprehensive answer, I'd like to summarize in a simplest form what I have learned since the time of this question.
Hot observable is an exact match for event. In events, values usually are fed into the handler even if no subscribers are listening. All subscribers are receiving the same set of values. Because of following the "event" pattern, hot observables are easier to understand than the cold ones.
Cold observable is also like an an event, but with a twist - Cold observable's event is not a property on a shared instance, it is a property on an object that is produced from a factory each time when somebody subscribes. In addition, subscription starts the production of the values. Because of the above, multiple subscribers are isolated and each receives its own set of values.
The most common mistake RX beginners make is creating a cold observable (well, thinking they are creating a cold observable) using some state variables within a function (f.e. accumulated total) and not wrapping it into a .Defer() statement. As a result, multiple subscribers share these variables and cause side effects between them.