I use combineLatest for join of two streams with two types of tasks. Processing two types of tasks should be interleaved. Is possible to determine which stream emits last value of pair?
I use solution with timestamp, but it is not correct. Each subject contain default value.
List<Flowable<? extends Timed<? extends Task>>> sources = new ArrayList<>();
Flowable<Timed<TaskModification>> modificationSource = mTaskModificationSubject
.onBackpressureDrop()
.observeOn(Schedulers.io(), false, 1)
.timestamp();
Flowable<Timed<TaskSynchronization>> synchronizationSource = mTaskSynchronizationSubject
.onBackpressureDrop()
.observeOn(Schedulers.io(), false, 1)
.flatMap(TaskSynchronizationWrapper::getSources)
.timestamp();
sources.add(0, modificationSource);
sources.add(1, synchronizationSource);
return Flowable
.combineLatest(sources, array -> {
Timed<TaskModification> taskModification = (Timed<TaskModification>) array[0];
Timed<TaskSynchronization> taskSynchronization = (Timed<TaskSynchronization>) array[1];
return (taskModification.time() > taskSynchronization.time())
? taskModification.value()
: taskSynchronization.value();
}, 1)
.observeOn(Schedulers.io(), false, 1)
.flatMapSingle(
Task::getSource
)
.ignoreElements();
When modification task is emitted than should have priority before synchronization tasks.
Without implementing a custom operator, you could introduce queues, merge the signals, then pick items from the priority queue first:
Flowable<X> prioritySource = ...
Flowable<X> source = ...
Flowable<X> output = Flowable.defer(() -> {
Queue<X> priorityQueue = new ConcurrentLinkedQueue<>();
Queue<X> queue = new ConcurrentLinkedQueue<>();
return Flowable.merge(
prioritySource.map(v -> {
priorityQueue.offer(v);
return 1;
}),
source.map(v -> {
queue.offer(v);
return 1;
})
)
.map(v -> {
if (!priorityQueue.isEmpty()) {
return priorityQueue.poll();
}
return queue.poll();
});
});
Related
I am new to RxJava2.
I am trying to get a list of Transaction object both from cache and from server.
I want to compare the server value to cache value and if the server value is the same, then ignore it.
I was able to do it easily using .scan() because we can return null and when null is returned from the .scan() the value got ignored(filtered).
RxJava 1
private Observable<List<Transaction>> getTransactionsFromCacheAndServer() {
return Observable.concat(
getTransactionsFromCache(),
getTransactionsFromServer()
)
.scan((p1, p2) -> {
if (p1 == null && p2 != null) {
return p2;
} else if (p1 != null && !isListSame(p1, p2)) {
return p2;
} else {
return null;
}
});
}
With RxJava 2, since I cannot return null anymore, things are not easy.
RxJava 2
private Observable<List<Transaction>> getTransactionsFromCacheAndServer() {
return Observable.concat(
getTransactionsFromCache(),
getTransactionsFromServer()
)
.map(FilterObject::new)
.scan((filterObject1, filterObject2) -> {
List<Transaction> p1 = (List<Transaction>)filterObject1.value;
List<Transaction> p2 = (List<Transaction>)filterObject2.value;
if (p1.size() == 0 && p2.size() > 0) {
return filterObject2;
} else if (!isListSame(p1, p2)) {
return filterObject2;
} else {
filterObject2.filter = true;
return filterObject2;
}
})
.filter(filterObject -> !filterObject.filter)
.map(filterObject -> (List<Transaction>)filterObject.value);
}
Where FilterObject is:
public class FilterObject {
public Object value;
public boolean filter;
public FilterObject(Object value) {
this.value = value;
}
}
Even though I can achieve the same thing using above method, it seems very ugly. Also I had to include two maps which might not be so performance friendly.
Is there a simple/clean way to achieve what I want?
I don't think there is a generic solution to this problem, since an empty list and a list that needs to be filtered (which happens to be empty in all cases) are two different things (the output of the scan) and needs to be handled differently.
However, in your particular case you never emit an empty list, except maybe for the first output.
(I am using String instead Transaction, shouldn't matter)
private Observable<List<String>> getTransactionsFromCacheAndServer() {
return Observable.concat(
getTransactionsFromCache(),
getTransactionsFromServer()
)
.filter(list -> !list.isEmpty())
// If you prefer a consistent empty list over the first
// empty list emission getting filtered
.startWith((List<String>) Collections.EMPTY_LIST)
// Newly emitted value cannot be empty, it only depends only on the comparison
.distinctUntilChanged(this::isListSame);
}
That's the closest I could get with as few operators as possible. Hope it solves your problem.
Based on andras' answer, I modified little bit to achieve what I want.
private Observable<List<String>> getTransactionsFromCacheAndServer() {
return Observable.concat(
getTransactionsFromCache(),
getTransactionsFromServer()
)
.filter(list -> !list.isEmpty())
.distinctUntilChanged(this::isListSame)
.switchIfEmpty(Observable.just(new ArrayList<>()));
}
Andreas' answer will always receive an empty list and then a real data.
My solution above will receive:
1. Data from cache (and then data from server if different)
2. Empty list if both cache and server returns Empty list.
I want to look for an entire list of items to be found before I complete and if that entire list isn't found, then an exception (a Timeout or custom one) is to be thrown. Like the built in Observable.timer() but instead of the test passing once the first item is emitted, I want it to require all of the items in a list to be found.
Here is an example. Let's say I have some test function that emits Observable<FoundNumber>. It looks like this:
var emittedList: List<String?> = listOf(null, "202", "302", "400")
data class FoundNumber(val numberId: String?)
fun scanNumbers(): Observable<FoundNumber> = Observable
.intervalRange(0,
emittedList.size.toLong(),
0,
1,
TimeUnit.SECONDS).map { index ->
FoundNumber(emittedList[index.toInt()]) }
That function will then be called to get numbers that will be compared to a list of expected numbers. It doesn't matter if there are additional numbers coming from scanForNumbers that aren't in the "target" list. They will just be ignored. Something like this:
val expectedNumbers = listOf("202", "302","999")
scanForNumbers(expectedNumbers)
.observeOn(AndroidSchedulers.mainThread())
.subscribeOn(Schedulers.io())
.subscribe { value -> Log.d(TAG, "Was returned a $value") }
So, the expected numbers (202, 302, and 999) don't exactly match with the numbers that will be emitted (202, 302, and 400). So, a timeout SHOULD occur, but with the built in version of Observable.timer(), it will not time out since at least one item was observed.
Here is kind of what I'd like to have. Anyone know how to code this up in RxJava/RxKotlin?
fun scanForNumbers(targets: List<String>): Observable<FoundNumber> {
val accumulator: Pair<Set<Any>, FoundNumber?> = targets.toSet() to null
return scanNumbers()
.SPECIAL_TIMEOUT_FOR_LIST(5, TimeUnit.SECONDS, List)
.scan(accumulator) { acc, next ->
val (set, previous) = acc
val stringSet:MutableSet<String> = hashSetOf()
set.forEach { stringSet.add(it.toString()) }
val item = if (next.numberId in stringSet) {
next
} else null
(set - next) to item // return set and nullable item
}
.filter { Log.d(TAG, "Filtering on ${it.second}")
it.second != null } // item not null
.take(targets.size.toLong()) // limit to the number of items
.map { it.second } // unwrap the item from the pair
.map { FoundController(it.numberId) } // wrap in your class
}
How do you code, hopefully using RxJava/Kotlin, a means to timeout on a list as mentioned?
I think I get it now, you want the timeout to begin counting from the moment you subscribe, not after you observe items.
If this is what you need, then the takeUntil operator could help you:
return scanNumbers()
.takeUntil(Observable.timer(5, TimeUnit.SECONDS))
.scan(accumulator) { acc, next -> ...
In this case, the timer will begin counting as soon as you subscribe. If the main observable completes before then great, if not, then the timer will complete the main observable anyways.
But takeUntil by itself will not throw an error, it will just complete. If you need it to end with an error, then you could use the following combination:
return scanNumbers()
.takeUntil(
Observable
.error<Void>(new TimeoutError("timeout!"))
.delay(5, TimeUnit.SECONDS, true))
.scan(accumulator) { acc, next -> ...
I'm looking to combine many IObservable<bool> streams such that when the latest value for all of them is true, a true is emitted, and otherwise a false is emitted.
CombinedLast would allow me to build something like this for two streams easily, but a) I'm not sure the API easily allows thousands of streams to be combined and b) I'm not sure how efficient it would be even if it could.
All is kinda similar to what I want except I'm assuming that works over a single sequence and once false cannot dynamically changes back to true.
Also I need the values to be "distinct until changed", although the DistintUntilChanged operator may not be efficient for this?
I'm hoping for an O(1) algorithm.
A good approach for combining the latest is to start with a IObservable<IObservable<T>> and turn it in to a IObservable<T[]>. This becomes a very dynamic way to combine as many values you need.
Here's an extension method to do this:
public static IObservable<T[]> CombineLatest<T>(this IObservable<IObservable<T>> sources)
{
return
sources.Publish(ss =>
Observable.Create<T[]>(o =>
{
var composite = new CompositeDisposable();
var list = new List<T>();
composite.Add(
ss.Subscribe(source =>
{
var index = list.Count;
list.Add(default(T));
composite.Add(source.Subscribe(x => list[index] = x));
}));
composite.Add(ss.Merge().Select(x => list.ToArray()).Subscribe(o));
return composite;
}));
}
This nicely creates and tracks all subscriptions and uses a closure to define the index that each subscription needs to use to update its value in the list that is used for output.
If you use it like this:
var sources = new Subject<IObservable<bool>>();
var output = sources.CombineLatest();
output.Subscribe(x => Console.WriteLine(x));
var s1 = new Subject<bool>();
sources.OnNext(s1);
s1.OnNext(true);
var s2 = new Subject<bool>();
sources.OnNext(s2);
s2.OnNext(false);
var s3 = new Subject<bool>();
sources.OnNext(s3);
s3.OnNext(true);
s2.OnNext(true);
s1.OnNext(false);
Then you get this output:
If you change the definition of output to var output = sources.CombineLatest().Select(xs => xs.Aggregate((x, y) => x & y)); then you get the output that I think you're after:
True
False
False
True
False
I don't know how to do this in a classically functional way and still achieve O(1). This used mutable state, and is O(1) for observing each message, but O(n) for memory:
public IObservable<bool> CombineBooleans(this IObservable<bool>[] source)
{
return source.Select((o, i) => o.Select(b => (value: b, index: i)))
.Merge()
.Scan((array: new bool[source.Length], countFalse: source.Length), (state, item) =>
{
var countFalse = state.countFalse;
if (state.array[item.index] == item.value)
return (state.array, countFalse); //nothing to change, emit same state
else if (state.array[item.index]) //previous/current state is true, becoming false
{
countFalse++;
state.array[item.index] = false;
}
else //previous/current state is false, becoming true
{
countFalse--;
state.array[item.index] = true;
}
return (state.array, countFalse);
})
.Scan((countFalse: source.Length, oldCountFalse: source.Length), (state, item) => (countFalse: item.countFalse, oldCountFalse: state.countFalse))
.SelectMany(state =>
state.countFalse == 1 && state.oldCountFalse == 0
? Observable.Return(false)
: state.countFalse == 0 && state.oldCountFalse == 1
? Observable.Return(true)
: Observable.Empty<bool>()
)
.Publish()
.RefCount();
}
EDIT: Added .Publish().Refcount() to eliminate multiple-subscriber bugs.
In the following code:
http://jsfiddle.net/staltz/4gGgs/27/
var clickStream = Rx.Observable.fromEvent(button, 'click');
var multiClickStream = clickStream
.buffer(function() { return clickStream.throttle(250); })
.map(function(list) { return list.length; })
.filter(function(x) { return x > 1; });
// Same as above, but detects single clicks
var singleClickStream = clickStream
.buffer(function() { return clickStream.throttle(250); })
.map(function(list) { return list.length; })
.filter(function(x) { return x === 1; });
// Listen to both streams and render the text label accordingly
singleClickStream.subscribe(function (event) {
document.querySelector('h2').textContent = 'click';
});
multiClickStream.subscribe(function (numclicks) {
document.querySelector('h2').textContent = ''+numclicks+'x click';
});
Rx.Observable.merge(singleClickStream, multiClickStream)
.throttle(1000)
.subscribe(function (suggestion) {
document.querySelector('h2').textContent = '';
});
How many times clickStream sequence will be iterated after merge?
I mean, will it look like this:
case 1
for(numclicks : clickStream.length){
if (numclicks === 1){
document.querySelector('h2').textContent = 'click';
}
};
for(numclicks : clickStream.length){
if (numclicks > 1){
document.querySelector('h2').textContent = ''+numclicks+'x click';
}
};
Or it will be internally, really merged to something like this (pseudocode):
case 2
for(numclicks: clickStream.length){
if (numclicks === 1){
document.querySelector('h2').textContent = 'click';
}else if(numclicks > 1){
document.querySelector('h2').textContent = ''+numclicks+'x click';
}
}
I personally think, that merge just sequentially apply stream to its arguments (case 1).
P.S. I hope there is some standart for things like this. But if no - I particularly interested in RxCpp and Sodium implementation.
I took js example, as more interactive.
fromEvent returns a hot source and so all subscribers share the same iteration of the for loop.
Ignoring the throttle calls, the result is similar to:
for(numclicks: clickStream.length){
// first subscription
if (numclicks === 1){
document.querySelector('h2').textContent = 'click';
}
// second subscription
if(numclicks > 1){
document.querySelector('h2').textContent = ''+numclicks+'x click';
}
// merged subscription
if (numclicks === 0) {
document.querySelector('h2').textContent = '';
}
}
The throttle calls mean that the body of the sole click stream for loop is actually just pushing click events into two buffers and reseting the timer in each of the three throttle operators. h2 is set when one of the three throttle timers fires. since the timers are not shared it is like a separate for loop per throttle timer with each loop setting h2 to only one of the three possible values:
This behavior is similar in all the Rx family.
Regarding rxcpp in particular:
rxcpp is missing the buffer overload that allows a observable to trigger a transition to a new buffer.
rxcpp does not have throttle implemented yet.
rxcpp is not thread-safe by default (pay-for-play) so if the throttle timers used introduce threads, then coordinations must be used to explicitly add thread-safety.
I want to stop stream A for exactly one notification whenever stream B fires. Both streams will stay online and won't ever complete.
A: o--o--o--o--o--o--o--o--o
B: --o-----o--------o-------
R: o-----o-----o--o-----o--o
or
A: o--o--o--o--o--o--o--o--o
B: -oo----oo-------oo-------
R: o-----o-----o--o-----o--o
Here's a version of my SkipWhen operator I did for a similar question (the difference is that, in the original, multiple "B's" would skip multiple "A's"):
public static IObservable<TSource> SkipWhen<TSource, TOther>(this IObservable<TSource> source,
IObservable<TOther> other)
{
return Observable.Create<TSource>(observer =>
{
object lockObject = new object();
bool shouldSkip = false;
var otherSubscription = new MutableDisposable();
var sourceSubscription = new MutableDisposable();
otherSubscription.Disposable = other.Subscribe(
x => { lock(lockObject) { shouldSkip = true; } });
sourceSubscription.Disposable = source.Where(_ =>
{
lock(lockObject)
{
if (shouldSkip)
{
shouldSkip = false;
return false;
}
else
{
return true;
}
}
}).Subscribe(observer);
return new CompositeDisposable(
sourceSubscription, otherSubscription);
});
}
If the current implementation becomes a bottleneck, consider changing the lock implementation to use a ReaderWriterLockSlim.
This solution will work when the observable is hot (and without refCount):
streamA
.takeUntil(streamB)
.skip(1)
.repeat()
.merge(streamA.take(1))
.subscribe(console.log);
.takeUntil(streamB): make stream A complete upon stream B producing a value.
.skip(1): make stream A skip one value upon starting (or as a result of .repeat()).
.repeat(): make stream A repeat (reconnect) indefinitely.
.merge(streamA.take(1)): offset the effect of .skip(1) at the beginning of the stream.
Example of making A stream skip every 5 seconds:
var streamA,
streamB;
streamA = Rx.Observable
.interval(1000)
.map(function (x) {
return 'A:' + x;
}).publish();
streamB = Rx.Observable
.interval(5000);
streamA
.takeUntil(streamB)
.skip(1)
.repeat()
.merge(streamA.take(1))
.subscribe(console.log);
streamA.connect();
You can also use this sandbox http://jsbin.com/gijorid/4/edit?js,console to execute BACTION() in the console log at the time of running the code to manually push a value to streamB (which is helpful for analysing the code).