Let's say I have a few Observable suppliers with the side effect on get call:
Subject<String> firstSubject = PublishSubject.create();
Supplier<Observable<String>> firstSupplier = () -> {
System.out.println("side effect of first one");
return firstSubject;
};
Subject<String> secondSubject = PublishSubject.create();
Supplier<Observable<String>> secondSupplier = () -> {
System.out.println("side effect of second one");
return secondSubject;
};
Subject<String> thirdSubject = PublishSubject.create();
Supplier<Observable<String>> thirdSupplier = () -> {
System.out.println("side effect of third one");
return thirdSubject;
};
Now I want to combine them in the following way - the get of the next supplier is called after the Observable from previous one emits a value (onNext is called).
I can do it with the following code:
firstSupplier.get()
.flatMap(__ -> secondSupplier.get())
.flatMap(__ -> thirdSupplier.get())
.subscribe();
// output: side effect of first one
firstSubject.onNext("");
// output: side effect of second one
secondSubject.onNext("");
// output: side effect of third one
How can I rewrite this code to accept unknown number of suppliers, passed as a - for example - Collection<Supplier<Observable>>?
I've reviewed various factory methods of Observable (like merge, concat), however they all are taking a collection of ObservableSource, which means I have to call get on all of my suppliers eagerly. However in my case it is important to call it lazily - only after previous Observable emits a value.
Edit 2:
I completely forgot that Observable.flatMap calls the mapper even if maxConcurrency is set to 1 and queues up the generated Observables to be run later.
I hope the following setup works as expected (i.e, it wasn't specified what should happen to the subsequent onNexts to the subjects).
Subject<String> firstSubject = PublishSubject.create();
Supplier<Observable<String>> firstSupplier = () -> {
System.out.println("side effect of first one");
return firstSubject;
};
Subject<String> secondSubject = PublishSubject.create();
Supplier<Observable<String>> secondSupplier = () -> {
System.out.println("side effect of second one");
return secondSubject;
};
Subject<String> thirdSubject = PublishSubject.create();
Supplier<Observable<String>> thirdSupplier = () -> {
System.out.println("side effect of third one");
return thirdSubject;
};
Collection<Supplier<Observable<String>>> collection =
Arrays.asList(firstSupplier, secondSupplier, thirdSupplier);
Observable.fromIterable(collection)
.concatMap(supplier -> supplier.get().take(1))
.subscribe();
System.out.println("// output: side effect of first one");
firstSubject.onNext("");
System.out.println("// output: side effect of second one");
secondSubject.onNext("");
System.out.println("// output: side effect of third one");
prints:
side effect of first one
// output: side effect of first one
side effect of second one
// output: side effect of second one
side effect of third one
// output: side effect of third one
Related
I want to combine result form two Mono based on some condition. Both Mono are results of WebClient calls:
The first one is a single call expecting fast response.
The second one is a combination of several calls with a slow response.
The idea to "cancel" the second Mono if result from the first one satisfies some condition to save time and avoid unnecessary network calls. If the first's Mono result is not enough zip it with the second Mono.
A Kotlin code sample to explain my idea:
fun getResult(): Mono<Result> {
val trivialResultMono: Mono<Result> = webClient.getResult()
val nonTrivialResultMono: Mono<Result> = webClient
.getResult()
.flatMap { webClient.getResult1(it) }
.flatMap { webClient.getResult2(it) }
.flatMap { webClient.getResult2(it) }
//here I need to check if trivial result satisfies some condition,
//for example trivialResult.size > 5 if it's true I just return
//trivialResultMono from getResult() function,
//it it's false something like this:
return Mono.zip(trivialResultMono, nonTrivialResultMono) { trivialResult, nonTrivialResult ->
trivialResult + nonTrivialResult
}
}
UPDATE:
To be more clear let's say that trivialResult comes in 1 second, nonTrivialResult in 2 seconds. I want to get my final result in 1 second in case of trivialResult.size > 5 and in 2 seconds otherwise.
Using just Mono.zip(trivialResultMono, nonTrivialResultMono) I will always get my final result in 2 seconds.
Using filter + switchIfEmpty it will take 1 second if trivialResult.size > 5 and 3 seconds otherwise. Please correct me if I wrong.
You could filter your trivialResultMono and apply switchIfEmpty operator
return trivialResultMono
.filter(trivialResult -> trivialResult.size > 5)
.switchIfEmpty(Mono.zip(...))
Update for merge approach:
Mono<Result> zipResultMono = Mono.zip...
return Flux.merge(
trivialResultMono.map(trivialResult -> Tuples.of(1, trivialResult)),
zipResultMono.map(zipResult -> Tuples.of(2, zipResult)))
.filter(tuple ->
(tuple.getT1().equals(1) && tuple.getT2().size > 5) ||
tuple.getT1().equals(2))
.next()
.map(Tuple2::getT2);
You could skip converting to the Tuple2 if zipResult always has size more then 5
You can achieve this with flatMap and map:
trivial.flatMap(trivialResult -> {
if (trivialResult.size > 5) {
return Mono.just(trivialResult);
} else {
return nonTrivial.map(nonTrivialResult -> trivialResult + nonTrivialResult);
}
});
I want to look for an entire list of items to be found before I complete and if that entire list isn't found, then an exception (a Timeout or custom one) is to be thrown. Like the built in Observable.timer() but instead of the test passing once the first item is emitted, I want it to require all of the items in a list to be found.
Here is an example. Let's say I have some test function that emits Observable<FoundNumber>. It looks like this:
var emittedList: List<String?> = listOf(null, "202", "302", "400")
data class FoundNumber(val numberId: String?)
fun scanNumbers(): Observable<FoundNumber> = Observable
.intervalRange(0,
emittedList.size.toLong(),
0,
1,
TimeUnit.SECONDS).map { index ->
FoundNumber(emittedList[index.toInt()]) }
That function will then be called to get numbers that will be compared to a list of expected numbers. It doesn't matter if there are additional numbers coming from scanForNumbers that aren't in the "target" list. They will just be ignored. Something like this:
val expectedNumbers = listOf("202", "302","999")
scanForNumbers(expectedNumbers)
.observeOn(AndroidSchedulers.mainThread())
.subscribeOn(Schedulers.io())
.subscribe { value -> Log.d(TAG, "Was returned a $value") }
So, the expected numbers (202, 302, and 999) don't exactly match with the numbers that will be emitted (202, 302, and 400). So, a timeout SHOULD occur, but with the built in version of Observable.timer(), it will not time out since at least one item was observed.
Here is kind of what I'd like to have. Anyone know how to code this up in RxJava/RxKotlin?
fun scanForNumbers(targets: List<String>): Observable<FoundNumber> {
val accumulator: Pair<Set<Any>, FoundNumber?> = targets.toSet() to null
return scanNumbers()
.SPECIAL_TIMEOUT_FOR_LIST(5, TimeUnit.SECONDS, List)
.scan(accumulator) { acc, next ->
val (set, previous) = acc
val stringSet:MutableSet<String> = hashSetOf()
set.forEach { stringSet.add(it.toString()) }
val item = if (next.numberId in stringSet) {
next
} else null
(set - next) to item // return set and nullable item
}
.filter { Log.d(TAG, "Filtering on ${it.second}")
it.second != null } // item not null
.take(targets.size.toLong()) // limit to the number of items
.map { it.second } // unwrap the item from the pair
.map { FoundController(it.numberId) } // wrap in your class
}
How do you code, hopefully using RxJava/Kotlin, a means to timeout on a list as mentioned?
I think I get it now, you want the timeout to begin counting from the moment you subscribe, not after you observe items.
If this is what you need, then the takeUntil operator could help you:
return scanNumbers()
.takeUntil(Observable.timer(5, TimeUnit.SECONDS))
.scan(accumulator) { acc, next -> ...
In this case, the timer will begin counting as soon as you subscribe. If the main observable completes before then great, if not, then the timer will complete the main observable anyways.
But takeUntil by itself will not throw an error, it will just complete. If you need it to end with an error, then you could use the following combination:
return scanNumbers()
.takeUntil(
Observable
.error<Void>(new TimeoutError("timeout!"))
.delay(5, TimeUnit.SECONDS, true))
.scan(accumulator) { acc, next -> ...
I have a data source fun getData(page : Int) : Single<List<Data>>.
I'd like to create a behaviour where I could subscribe to that source each time with different parameters each time until conditionCheck() returns true.
I imagine something like this:
getData(page)
.doOnNext { page++ }
.doOnNext { /* manipulate data */ }
.takeWhile { conditionCheck() }
.subscribe({
print "Completed"
})
Here is a different way to think about it (pseudocode-ish):
initialPage = Observable.just(1);
futurePages = PublishSubject.create();
allPossiblePages = Observable.concat(initialPage, futurePages);
allPossibleData = allPossiblePages
.map(page -> getData(page))
.do(page -> futurePages.onNext(/* manipulate page data */));
dataYouWant = allPossibleData
.filter(data -> conditionCheck(data))
.takeFirst()
.subscribe( /* data should be here */);
Maybe this isn't the best way to do this, someone will speak up if that's the case. I'm not a big fan of the line .do(page -> futurePages.onNext ..), but something like this might work.
I put together an example that runs using rxjs (easiest to prototype/demo) here: https://plnkr.co/edit/uXE53ygo8REzpbMJdvsD
I've got a simple program here that displays the number of letters in various words. It works as expected.
static void Main(string[] args) {
var word = new Subject<string>();
var wordPub = word.Publish().RefCount();
var length = word.Select(i => i.Length);
var report =
wordPub
.GroupJoin(length,
s => wordPub,
s => Observable.Empty<int>(),
(w, a) => new { Word = w, Lengths = a })
.SelectMany(i => i.Lengths.Select(j => new { Word = i.Word, Length = j }));
report.Subscribe(i => Console.WriteLine($"{i.Word} {i.Length}"));
word.OnNext("Apple");
word.OnNext("Banana");
word.OnNext("Cat");
word.OnNext("Donkey");
word.OnNext("Elephant");
word.OnNext("Zebra");
Console.ReadLine();
}
And the output is:
Apple 5
Banana 6
Cat 3
Donkey 6
Elephant 8
Zebra 5
I used the Publish().RefCount() because "wordpub" is included in "report" twice. Without it, when a word is emitted first one part of the report would get notified by a callback, and then the other part of report would be notified, double the notifications. That is kindof what happens; the output ends up having 11 items rather than 6. At least that is what I think is going on. I think of using Publish().RefCount() in this situation as simultaneously updating both parts of the report.
However if I change the length function to ALSO use the published source like this:
var length = wordPub.Select(i => i.Length);
Then the output is this:
Apple 5
Apple 6
Banana 6
Cat 3
Banana 3
Cat 6
Donkey 6
Elephant 8
Donkey 8
Elephant 5
Zebra 5
Why can't the length function also use the same published source?
This was a great challenge to solve!
So subtle the conditions that this happens.
Apologies in advance for the long explanation, but bear with me!
TL;DR
Subscriptions to the published source are processed in order, but before any other subscription directly to the unpublished source. i.e. you can jump the queue!
With GroupJoin subscription order is important to determine when windows open and close.
My first concern would be that you are publish refcounting a subject.
This should be a no-op.
Subject<T> has no subscription cost.
So when you remove the Publish().RefCount() :
var word = new Subject<string>();
var wordPub = word;//.Publish().RefCount();
var length = word.Select(i => i.Length);
then you get the same issue.
So then I look to the GroupJoin (because my intuition suggests that Publish().Refcount() is a red herring).
For me, eyeballing this alone was too hard to rationalise, so I lean on a simple debugging too I have used dozens of times of the years - a Trace or Log extension method.
public interface ILogger
{
void Log(string input);
}
public class DumpLogger : ILogger
{
public void Log(string input)
{
//LinqPad `Dump()` extension method.
// Could use Console.Write instead.
input.Dump();
}
}
public static class ObservableLoggingExtensions
{
private static int _index = 0;
public static IObservable<T> Log<T>(this IObservable<T> source, ILogger logger, string name)
{
return Observable.Create<T>(o =>
{
var index = Interlocked.Increment(ref _index);
var label = $"{index:0000}{name}";
logger.Log($"{label}.Subscribe()");
var disposed = Disposable.Create(() => logger.Log($"{label}.Dispose()"));
var subscription = source
.Do(
x => logger.Log($"{label}.OnNext({x.ToString()})"),
ex => logger.Log($"{label}.OnError({ex})"),
() => logger.Log($"{label}.OnCompleted()")
)
.Subscribe(o);
return new CompositeDisposable(subscription, disposed);
});
}
}
When I add the logging to your provided code it looks like this:
var logger = new DumpLogger();
var word = new Subject<string>();
var wordPub = word.Publish().RefCount();
var length = word.Select(i => i.Length);
var report =
wordPub.Log(logger, "lhs")
.GroupJoin(word.Select(i => i.Length).Log(logger, "rhs"),
s => wordPub.Log(logger, "lhsDuration"),
s => Observable.Empty<int>().Log(logger, "rhsDuration"),
(w, a) => new { Word = w, Lengths = a })
.SelectMany(i => i.Lengths.Select(j => new { Word = i.Word, Length = j }));
report.Subscribe(i => ($"{i.Word} {i.Length}").Dump("OnNext"));
word.OnNext("Apple");
word.OnNext("Banana");
word.OnNext("Cat");
word.OnNext("Donkey");
word.OnNext("Elephant");
word.OnNext("Zebra");
This will then output in my log something like the following
Log with Publish().RefCount() used
0001lhs.Subscribe()
0002rhs.Subscribe()
0001lhs.OnNext(Apple)
0003lhsDuration.Subscribe()
0002rhs.OnNext(5)
0004rhsDuration.Subscribe()
0004rhsDuration.OnCompleted()
0004rhsDuration.Dispose()
OnNext
Apple 5
0001lhs.OnNext(Banana)
0005lhsDuration.Subscribe()
0003lhsDuration.OnNext(Banana)
0003lhsDuration.Dispose()
0002rhs.OnNext(6)
0006rhsDuration.Subscribe()
0006rhsDuration.OnCompleted()
0006rhsDuration.Dispose()
OnNext
Banana 6
...
However when I remove the usage Publish().RefCount() the new log output is as follows:
Log without only Subject
0001lhs.Subscribe()
0002rhs.Subscribe()
0001lhs.OnNext(Apple)
0003lhsDuration.Subscribe()
0002rhs.OnNext(5)
0004rhsDuration.Subscribe()
0004rhsDuration.OnCompleted()
0004rhsDuration.Dispose()
OnNext
Apple 5
0001lhs.OnNext(Banana)
0005lhsDuration.Subscribe()
0002rhs.OnNext(6)
0006rhsDuration.Subscribe()
0006rhsDuration.OnCompleted()
0006rhsDuration.Dispose()
OnNext
Apple 6
OnNext
Banana 6
0003lhsDuration.OnNext(Banana)
0003lhsDuration.Dispose()
...
This gives us some insight, however when the issue really becomes clear is when we start annotating our logs with a logical list of subscriptions.
In the original (working) code with the RefCount our annotations might look like this
//word.Subsribers.Add(wordPub)
0001lhs.Subscribe() //wordPub.Subsribers.Add(0001lhs)
0002rhs.Subscribe() //word.Subsribers.Add(0002rhs)
0001lhs.OnNext(Apple)
0003lhsDuration.Subscribe() //wordPub.Subsribers.Add(0003lhsDuration)
0002rhs.OnNext(5)
0004rhsDuration.Subscribe()
0004rhsDuration.OnCompleted()
0004rhsDuration.Dispose()
OnNext
Apple 5
0001lhs.OnNext(Banana)
0005lhsDuration.Subscribe() //wordPub.Subsribers.Add(0005lhsDuration)
0003lhsDuration.OnNext(Banana)
0003lhsDuration.Dispose() //wordPub.Subsribers.Remove(0003lhsDuration)
0002rhs.OnNext(6)
0006rhsDuration.Subscribe()
0006rhsDuration.OnCompleted()
0006rhsDuration.Dispose()
OnNext
Banana 6
So in this example, when word.OnNext("Banana"); is executed the chain of observers is linked in this order
wordPub
0002rhs
However, wordPub has child subscriptions!
So the real subscription list looks like
wordPub
0001lhs
0003lhsDuration
0005lhsDuration
0002rhs
If we annotate the Subject only log we see where the subtlety lies
0001lhs.Subscribe() //word.Subsribers.Add(0001lhs)
0002rhs.Subscribe() //word.Subsribers.Add(0002rhs)
0001lhs.OnNext(Apple)
0003lhsDuration.Subscribe() //word.Subsribers.Add(0003lhsDuration)
0002rhs.OnNext(5)
0004rhsDuration.Subscribe()
0004rhsDuration.OnCompleted()
0004rhsDuration.Dispose()
OnNext
Apple 5
0001lhs.OnNext(Banana)
0005lhsDuration.Subscribe() //word.Subsribers.Add(0005lhsDuration)
0002rhs.OnNext(6)
0006rhsDuration.Subscribe()
0006rhsDuration.OnCompleted()
0006rhsDuration.Dispose()
OnNext
Apple 6
OnNext
Banana 6
0003lhsDuration.OnNext(Banana)
0003lhsDuration.Dispose()
So in this example, when word.OnNext("Banana"); is executed the chain of observers is linked in this order
1. 0001lhs
2. 0002rhs
3. 0003lhsDuration
4. 0005lhsDuration
As the 0003lhsDuration subscription is activated after the 0002rhs, it wont see the "Banana" value to terminate the window, until after the rhs has been sent the value, thus yielding it in the still open window.
Whew
As #francezu13k50 points out the obvious and simple solution to your problem is to just use word.Select(x => new { Word = x, Length = x.Length });, but as I think you have given us a simplified version of your real problem (appreciated) I understand why this isn't suitable.
However, as I dont know what your real problem space is I am not sure what to suggest to you to provide a solution, except that you have one with your current code, and now you should know why it works the way it does.
RefCount returns an Observable that stays connected to the source as long as there is at least one subscription to the returned Observable. When the last subscription is disposed, RefCount disposes it's connection to the source, and reconnects when a new subscription is being made. It might be the case with your report query that all subscriptions to the 'wordPub' are disposed before the query is fulfilled.
Instead of the complicated GroupJoin query you could simply do :
var report = word.Select(x => new { Word = x, Length = x.Length });
Edit:
Change your report query to this if you want to use the GroupJoin operator :
var report =
wordPub
.GroupJoin(length,
s => wordPub,
s => Observable.Empty<int>(),
(w, a) => new { Word = w, Lengths = a })
.SelectMany(i => i.Lengths.FirstAsync().Select(j => new { Word = i.Word, Length = j }));
Because GroupJoin seems to be very tricky to work with, here is another approach for correlating the inputs and outputs of functions.
static void Main(string[] args) {
var word = new Subject<string>();
var length = new Subject<int>();
var report =
word
.CombineLatest(length, (w, l) => new { Word = w, Length = l })
.Scan((a, b) => new { Word = b.Word, Length = a.Word == b.Word ? b.Length : -1 })
.Where(i => i.Length != -1);
report.Subscribe(i => Console.WriteLine($"{i.Word} {i.Length}"));
word.OnNext("Apple"); length.OnNext(5);
word.OnNext("Banana");
word.OnNext("Cat"); length.OnNext(3);
word.OnNext("Donkey");
word.OnNext("Elephant"); length.OnNext(8);
word.OnNext("Zebra"); length.OnNext(5);
Console.ReadLine();
}
This approach works if every input has 0 or more outputs subject to the constraints that (1) outputs only arrive in the same order as the inputs AND (2) each output corresponds to its most recent input. This is like a LeftJoin - each item in the first list (word) is paired with items in the right list (length) that subsequently arrive, up until another item in the first list is emitted.
Trying to use regular Join instead of GroupJoin. I thought the problem was that when a new word was created there was a race condition inside Join between creating a new window and ending the current one. So here I tried to elimate that by pairing every word with a null signifying the end of the window. Doesn't work, just like the first version did not. How is it possible that a new window is created for each word without the previous one being closed first? Completely confused.
static void Main(string[] args) {
var lgr = new DelegateLogger(Console.WriteLine);
var word = new Subject<string>();
var wordDelimited =
word
.Select(i => Observable.Return<string>(null).StartWith(i))
.SelectMany(i => i);
var wordStart = wordDelimited.Where(i => i != null);
var wordEnd = wordDelimited.Where(i => i == null);
var report = Observable
.Join(
wordStart.Log(lgr, "word"), // starts window
wordStart.Select(i => i.Length),
s => wordEnd.Log(lgr, "expireWord"), // ends current window
s => Observable.Empty<int>(),
(l, r) => new { Word = l, Length = r });
report.Subscribe(i => Console.WriteLine($"{i.Word} {i.Length}"));
word.OnNext("Apple");
word.OnNext("Banana");
word.OnNext("Cat");
word.OnNext("Zebra");
word.OnNext("Elephant");
word.OnNext("Bear");
Console.ReadLine();
}
I want to stop stream A for exactly one notification whenever stream B fires. Both streams will stay online and won't ever complete.
A: o--o--o--o--o--o--o--o--o
B: --o-----o--------o-------
R: o-----o-----o--o-----o--o
or
A: o--o--o--o--o--o--o--o--o
B: -oo----oo-------oo-------
R: o-----o-----o--o-----o--o
Here's a version of my SkipWhen operator I did for a similar question (the difference is that, in the original, multiple "B's" would skip multiple "A's"):
public static IObservable<TSource> SkipWhen<TSource, TOther>(this IObservable<TSource> source,
IObservable<TOther> other)
{
return Observable.Create<TSource>(observer =>
{
object lockObject = new object();
bool shouldSkip = false;
var otherSubscription = new MutableDisposable();
var sourceSubscription = new MutableDisposable();
otherSubscription.Disposable = other.Subscribe(
x => { lock(lockObject) { shouldSkip = true; } });
sourceSubscription.Disposable = source.Where(_ =>
{
lock(lockObject)
{
if (shouldSkip)
{
shouldSkip = false;
return false;
}
else
{
return true;
}
}
}).Subscribe(observer);
return new CompositeDisposable(
sourceSubscription, otherSubscription);
});
}
If the current implementation becomes a bottleneck, consider changing the lock implementation to use a ReaderWriterLockSlim.
This solution will work when the observable is hot (and without refCount):
streamA
.takeUntil(streamB)
.skip(1)
.repeat()
.merge(streamA.take(1))
.subscribe(console.log);
.takeUntil(streamB): make stream A complete upon stream B producing a value.
.skip(1): make stream A skip one value upon starting (or as a result of .repeat()).
.repeat(): make stream A repeat (reconnect) indefinitely.
.merge(streamA.take(1)): offset the effect of .skip(1) at the beginning of the stream.
Example of making A stream skip every 5 seconds:
var streamA,
streamB;
streamA = Rx.Observable
.interval(1000)
.map(function (x) {
return 'A:' + x;
}).publish();
streamB = Rx.Observable
.interval(5000);
streamA
.takeUntil(streamB)
.skip(1)
.repeat()
.merge(streamA.take(1))
.subscribe(console.log);
streamA.connect();
You can also use this sandbox http://jsbin.com/gijorid/4/edit?js,console to execute BACTION() in the console log at the time of running the code to manually push a value to streamB (which is helpful for analysing the code).