Is there a Rx.Net Operator that tells the number of subscribed observers? - system.reactive

I am trying to debug if I am correctly disposing the observers to a long running services (services exposing IObservable). And I am wondering if there an operator or something that we can create to log the number of active observers, say something like.
public class NewsService
{
IObservable<Article> GetArticles();
}
NewsService.Instance
.GetArticles()
.DoCount(x=> Trace.Writeline("The current count is {x}"))
.Subscribe();
There is a solution proposed here, which works on the Subject. What if we don't have access to the Subject and a library is exposing IObservable.

Generally speaking, the concept of subscriber count for an arbitrary observable sequence is not defined.
For cold observables such as Observable.Interval, every time you subscribe to the observable, a new pipeline instance is created, which - from its point of view sees only a single observer at a time.
We can, nonetheless, warm up a cold observable, and watch subscriptions come and go.
public static IObservable<T> RefCount<T>(this IObservable<T> source, Action<int> onChange)
{
var subscribers = 0;
var shared = source.Publish().RefCount();
void callback(int count) => onChange(Interlocked.Add(ref subscribers, count));
return Observable.Create<T>(observer =>
{
callback(+1);
var subscription = shared.Subscribe(observer);
var dispose = Disposable.Create(() => callback(-1));
return new CompositeDisposable(subscription, dispose);
});
}
Demo
var values =
Observable
.Interval(TimeSpan.FromSeconds(0.1))
.RefCount(count => Console.WriteLine($"Subscribers: {count}"));
values.Take(5).Subscribe();
values.Take(10).Subscribe();
values.Take(15).Subscribe();
Output
Subscribers: 1
Subscribers: 2
Subscribers: 3
Subscribers: 2
Subscribers: 1
Subscribers: 0
Now, this works because we have a shared view of the parent observable.
So try to make all subscriptions point to the same instance.
_articles = GetArticles().RefCount(count => Console.WriteLine($"Subscribers: {count}")));
...
_articles.Subscribe();

Related

Spit IObservable<T> exceptions in a separate IObservable<Exception> and continue normally

I have an hot IObservable<T> which may throw an exception. However, I would like to continue with it. I think I could use Retry operator for that. However, it would be great if I can also listen to any error in IObservable<T> through a separate IObservable<Exception>. Is it possible?
Your case is significantly more simplified in that you have a hot observable.
OnError is a notification outside your value stream, so we could materialize the notifications to retrieve the error. This still causes the tear-down of the stream with an OnCompleted, so you'll need to re-subscribe with Repeat.
var exceptions =
source
.Materialize()
.Where(notif => notif.Kind == NotificationKind.OnError)
.Select(notif => notif.Exception)
.Repeat();
Note
If you're using a Subject<T> for your hot observable, you might run into the usual problem of re-subbing a subject. A subject will replay its OnError or OnCompleted notifications for every new observer.
var source = new Subject<int>();
source.OnNext(1);
source.OnError(new Exception());
source.Subscribe(
i => Console.WriteLine(i),
ex => Console.WriteLine("Still got exception after the throw")
);
In this case your exception stream will go into an infinite re-subscription loop.
The premise of your question violates the observable contract:
An Observable may make zero or more OnNext notifications, each representing a single emitted item, and it may then follow those emission notifications by either an OnCompleted or an OnError notification, but not both. Upon issuing an OnCompleted or OnError notification, it may not thereafter issue any further notifications. (emphasis mine)
In other words, after your hot IObservable<T> throws an exception, the observable is ended. The observable of exceptions that comes out of that has a max count of one.
If you want to support a scenario where you re-start an observable after an exception, you're producing a stream of observables, or IObservable<IObservable<T>>. To work with that, here's a code sample:
var source = new Subject<Subject<int>>();
var exceptionStream = source
.SelectMany(o => o.Materialize())
.Where(n => n.Kind == NotificationKind.OnError)
.Select(n => n.Exception);
var itemStream = source
.SelectMany(o => o.Materialize())
.Where(n => n.Kind == NotificationKind.OnNext)
.Select(n => n.Value);
var items = new List<int>();
var exceptions = new List<Exception>();
itemStream.Subscribe(i => items.Add(i));
exceptionStream.Subscribe(e => exceptions.Add(e));
var currentSubject = new Subject<int>();
source.OnNext(currentSubject);
currentSubject.OnNext(1);
currentSubject.OnNext(2);
currentSubject.OnNext(3);
currentSubject.OnError(new Exception("First error"));
var currentSubject2 = new Subject<int>();
source.OnNext(currentSubject2);
currentSubject2.OnNext(4);
currentSubject2.OnNext(5);
currentSubject2.OnNext(6);
currentSubject2.OnError(new Exception("Second error"));
items.Dump(); //Linqpad
exceptions.Dump(); //Linqpad

Cancel Punctuator on Kafka Streams after is triggered

I create a scheduled punctuator on a transformer and I schedule it to run on a periodical basis (using kafka v2.1.0). Every time I accept a specific key I do create a new one like this
scheduled = context.schedule (Duration.ofMillis(scheduleTime),
PunctuationType.WALL_CLOCK_TIME,new CustomPunctuator(context, customStateStoreName));
My issue is that all these punctuators I create run constantly and I cannot find a way to cancel them. I found a snippet in the internet to use
private Cancellable scheduled;
#Override
public void init(PorcessorContext processContext) {
this.context = processorContext;
scheduled = context.schedule(TimeUnit.SECONDS.toMillis(5), PunctuationType.WALL_CLOCK_TIME,
this::punctuateCancel);
}
private void punctuateCancel(long timestamp) {
scheduled.cancel();
}
but this unfortunately seems to cancel only the latest created Punctuator.
I am editing my post just to give some further insight regarding my approach and how this is related with comments made by wardzinia. So my approach is pretty similar just uses a Map because I need to have only one punctuator active per event key so in my Transformer class I initiate
private Map<String,Cancellable> scheduled = new HashMap<>();
And on my transform method I do execute the code below
{
final Cancellable cancelSched = scheduled.get(recordKey);
// Every time I get a new event I cancel my previous Punctuator
// and schedule a new one ( context.schedule a few lines later)
if(cancelSched != null)
cancelSched.cancel();
// This is supposed to work like a closure by capturing the currentCancellable which in the next statement
// is moved to the scheduled map. Scheduled map at any point will have the only active Punctuator for a
// specific String as it is constantly renewed
// Note: Previous registered punctuators have already been cancelled as it can be seen by the previous
// statement (cancelSched.cancel();)
Cancellable currentCancellable = context.schedule(Duration.ofMillis(scheduleTime), PunctuationType.WALL_CLOCK_TIME,
new CustomPunctuator(context, recordKey ,()-> scheduled ));
// Update Active Punctuators for a specific key.
scheduled.put(recordKey,currentCancellable);
}
And I use that registered callback on my Punctuator punctuate method to cancel the last active Punctuator
after it has started. It seems to work (not sure though) but it feels very "hacky" and not the kind of solution
that it is certainly desirable.
So how can I cancel a punctuator after is triggered. Is there a way to cope with this issue ?
I think one thing you could do is the following:
class CustomPunctuator implements Punctuator {
final Cancellable schedule;
public void punctuate(final long timestamp) {
// business logic
if (/* do cancel */) {
schedule.cancel()
}
}
}
// registering a punctuation
{
final CustomPunctuator punctuation = new CustomPunctuator();
final Cancellable currentCancellable = context.schedule(
Duration.ofMillis(scheduleTime),
PunctuationType.WALL_CLOCK_TIME,
punctuation);
punctuation.schedule = currentCancellable;
}
This way, you don't need to maintain the HashMap and give each CustomPunctuator instance a way to cancel itself.
I had the same situation, just for the people interested in scala I handle it as
val punctuation = new myPunctuation()
val scheduled:Cancellable=context.schedule(Duration.ofSeconds(5), PunctuationType.WALL_CLOCK_TIME, punctuation)
punctuation.schedule=scheduled
The class
class myPunctuation() extends Punctuator{
var schedule: Cancellable = _
override def punctuate(timestamp: Long): Unit = {
println("hello")
schedule.cancel()
}
}
Works like a charm

RXJS : Idiomatic way to create an observable stream from a paged interface

I have paged interface. Given a starting point a request will produce a list of results and a continuation indicator.
I've created an observable that is built by constructing and flat mapping an observable that reads the page. The result of this observable contains both the data for the page and a value to continue with. I pluck the data and flat map it to the subscriber. Producing a stream of values.
To handle the paging I've created a subject for the next page values. It's seeded with an initial value then each time I receive a response with a valid next page I push to the pages subject and trigger another read until such time as there is no more to read.
Is there a more idiomatic way of doing this?
function records(start = 'LATEST', limit = 1000) {
let pages = new rx.Subject();
this.connect(start)
.subscribe(page => pages.onNext(page));
let records = pages
.flatMap(page => {
return this.read(page, limit)
.doOnNext(result => {
let next = result.next;
if (next === undefined) {
pages.onCompleted();
} else {
pages.onNext(next);
}
});
})
.pluck('data')
.flatMap(data => data);
return records;
}
That's a reasonable way to do it. It has a couple of potential flaws in it (that may or may not impact you depending upon your use case):
You provide no way to observe any errors that occur in this.connect(start)
Your observable is effectively hot. If the caller does not immediately subscribe to the observable (perhaps they store it and subscribe later), then they'll miss the completion of this.connect(start) and the observable will appear to never produce anything.
You provide no way to unsubscribe from the initial connect call if the caller changes its mind and unsubscribes early. Not a real big deal, but usually when one constructs an observable, one should try to chain the disposables together so it call cleans up properly if the caller unsubscribes.
Here's a modified version:
It passes errors from this.connect to the observer.
It uses Observable.create to create a cold observable that only starts is business when the caller actually subscribes so there is no chance of missing the initial page value and stalling the stream.
It combines the this.connect subscription disposable with the overall subscription disposable
Code:
function records(start = 'LATEST', limit = 1000) {
return Rx.Observable.create(observer => {
let pages = new Rx.Subject();
let connectSub = new Rx.SingleAssignmentDisposable();
let resultsSub = new Rx.SingleAssignmentDisposable();
let sub = new Rx.CompositeDisposable(connectSub, resultsSub);
// Make sure we subscribe to pages before we issue this.connect()
// just in case this.connect() finishes synchronously (possible if it caches values or something?)
let results = pages
.flatMap(page => this.read(page, limit))
.doOnNext(r => this.next !== undefined ? pages.onNext(this.next) : pages.onCompleted())
.flatMap(r => r.data);
resultsSub.setDisposable(results.subscribe(observer));
// now query the first page
connectSub.setDisposable(this.connect(start)
.subscribe(p => pages.onNext(p), e => observer.onError(e)));
return sub;
});
}
Note: I've not used the ES6 syntax before, so hopefully I didn't mess anything up here.

Why does head not cancel subscription

Let's say you have the following Observable in rxjava-scala-0.18.4
#volatile var dorun = true
var subscriber: Subscriber[String] = null
val myObs = Observable { obs: Subscriber[String] =>
subscriber = obs
Subscription { println("unsubscribed"); dorun = false }
}
val sub = myObs.head.subscribe(println(_))
assertTrue(dorun)
subscriber.onNext("hello")
Thread.sleep(500)
assertFalse(dorun)
subscriber.onNext("world")
Thread.sleep(500)
assertFalse(dorun)
The second assertion fails, which means that head does not unsubscriby. Is my understanding of Observables wrong or should head unsubscribe after the first element got emitted?
Take a look at your subscribe() method: you loop until run is set to false, but the only way for that to happen would be to close the subscription. The problem is that nobody has the subscription yet: the loop keeps you from returning. The head operator can't terminate the underlying subscription after the first item is delivered because it hasn't finished subscribing yet. Thus, you just keep looping forever.
One solution would be to move your loop into an action scheduled on Schedulers.trampoline(). Then the events would be delivered some time after returning from subscribe().
Additionally, in your subscribe() method, it seems you need to add the new subscription object to the Subscriber that gets passed in, like so:
val myObs = Observable {
obs: rx.lang.scala.Subscriber[String] =>
...
obs.add(
Subscription {
dorun = false
println("unsubscribed")
}
)
}

rx reactive extension: how to have each subscriber get a different value (the next one) from an observable?

Using reactive extension, it is easy to subscribe 2 times to the same observable.
When a new value is available in the observable, both subscribers are called with this same value.
Is there a way to have each subscriber get a different value (the next one) from this observable ?
Ex of what i'm after:
source sequence: [1,2,3,4,5,...] (infinite)
The source is constantly adding new items at an unknown rate.
I'm trying to execute a lenghty async action for each item using N subscribers.
1st subscriber: 1,2,4,...
2nd subscriber: 3,5,...
...
or
1st subscriber: 1,3,...
2nd subscriber: 2,4,5,...
...
or
1st subscriber: 1,3,5,...
2nd subscriber: 2,4,6,...
I would agree with Asti.
You could use Rx to populate a Queue (Blocking Collection) and then have competing consumers read from the queue. This way if one process was for some reason faster it could pick up the next item potentially before the other consumer if it was still busy.
However, if you want to do it, against good advice :), then you could just use the Select operator that will provide you with the index of each element. You can then pass that down to your subscribers and they can fiter on a modulus. (Yuck! Leaky abstractions, magic numbers, potentially blocking, potentiall side effects to the source sequence etc)
var source = Obserservable.Interval(1.Seconds())
.Select((i,element)=>{new Index=i, Element=element});
var subscription1 = source.Where(x=>x.Index%2==0).Subscribe(x=>DoWithThing1(x.Element));
var subscription2 = source.Where(x=>x.Index%2==1).Subscribe(x=>DoWithThing2(x.Element));
Also remember that the work done on the OnNext handler if it is blocking will still block the scheduler that it is on. This could affect the speed of your source/producer. Another reason why Asti's answer is a better option.
Ask if that is not clear :-)
How about:
IObservable<TRet> SomeLengthyOperation(T input)
{
return Observable.Defer(() => Observable.Start(() => {
return someCalculatedValueThatTookALongTime;
}, Scheduler.TaskPoolScheduler));
}
someObservableSource
.SelectMany(x => SomeLengthyOperation(input))
.Subscribe(x => Console.WriteLine("The result was {0}", x);
You can even limit the number of concurrent operations:
someObservableSource
.Select(x => SomeLengthyOperation(input))
.Merge(4 /* at a time */)
.Subscribe(x => Console.WriteLine("The result was {0}", x);
It's important for the Merge(4) to work, that the Observable returned by SomeLengthyOperation be a Cold Observable, which is what the Defer does here - it makes the Observable.Start not happen until someone Subscribes.