i'm a RxJava newcomer, and i'm having some trouble wrapping my head around how to do the following.
i'm using Retrofit to invoke a network request that returns me a Single<Foo>, which is the type i ultimately want to consume via my Subscriber instance (call it SingleFooSubscriber)
Foo has an internal property items typed as List<String>.
if Foo.items is not empty, i would like to invoke separate, concurrent network requests for each of its values. (the actual results of these requests are inconsequential for SingleFooSubscriber as the results will be cached externally).
SingleFooSubscriber.onComplete() should be invoked only when Foo and all Foo.items have been fetched.
fetchFooCall
.subscribeOn(Schedulers.io())
// Approach #1...
// the idea here would be to "merge" the results of both streams into a single
// reactive type, but i'm not sure how this would work given that the item emissions
// could be far greater than one. using zip here i don't think it would every
// complete.
.flatMap { foo ->
if(foo.items.isNotEmpty()) {
Observable.zip(
Observable.fromIterable(foo.items),
Observable.just(foo),
{ source1, source2 ->
// hmmmm...
}
).toSingle()
} else {
Single.just(foo)
}
}
// ...or Approach #2...
// i think this would result in the streams for Foo and items being handled sequentially,
// which is not really ideal because
// 1) i think it would entail nested streams (i get the feeling i should be using flatMap
// instead)
// 2) and i'm not sure SingleFooSubscriber.onComplete() would depend on the completion of
// the stream for items
.doOnSuccess { data ->
if(data.items.isNotEmpty()) {
// hmmmm...
}
}
.observeOn(AndroidSchedulers.mainThread())
.subscribe(
{ data -> /* onSuccess() */ },
{ error -> /* onError() */ }
)
any thoughts on how to approach this would be greatly appreciated!
bonus points: in trying to come up with a solution to this, i've begun to question the decision to use the Single reactive type vs the Observable reactive type. most (all, except this one Foo.items case?) of my streams actually revolve around consuming a single instance of something, so i leaned toward Single to represent my streams as i thought it would add some semantic clarity around the code. anybody have any general guidance around when to use one vs the other?
You need to nest flatMaps and then convert back to Single:
retrofit.getMainObject()
.flatMap(v ->
Flowable.fromIterable(v.items)
.flatMap(w ->
retrofit.getItem(w.id).doOnNext(x -> w.property = x)
)
.ignoreElements()
.toSingle(v)
)
Related
Sorry, I am new to reactive paradigm. Is is possible to use AtomicReference to get value of a Mono since reactive code can run asynchronously and different events run on different thread. Please see the sample below. I am also not sure if this piece of code is considered reactive
sample code:
public static void main(String[] a) {
AtomicReference<UserDTO> dto = new AtomicReference<>();
Mono.just(new UserDTO())
.doOnNext(d -> d.setUserId(123L))
.subscribe(d -> dto.set(d));
UserDTO result = dto.get();
dto.set(null);
System.out.println(result); // produce UserDTO(userId=123)
System.out.println(dto.get()); // produce null
}
The code snippet you have shared is not guaranteed to always work. There is no way to guarantee that the function inside doOnNext will happen before dto.get(). You have created a race condition.
You can run the follow code to simulate this.
AtomicReference<UserDTO> dto = new AtomicReference<>();
Mono.just(new UserDTO())
.delayElement(Duration.ofSeconds(1))
.doOnNext(d -> d.setUserId(123L))
.subscribe(dto::set);
UserDTO result = dto.get();
System.out.println(result); // produces null
To make this example fully reactive, you should print out in the subscribe operator
Mono.just(new UserDTO())
.doOnNext(d -> d.setUserId(123L))
.subscribe(System.out::println)
In a more "real world" example, your method would return a Mono<UserDTO> and you would then perform transformations on this using map or flatMap operators.
** EDIT **
If you are looking to make a blocking call within a reactive stream this previous stack overflow question contains a good answer
I am trying to manipulate my objects received from Flux with data received from a Mono where the methods emitting the Flux of object and Mono of items are both different API calls. The problem is, I don't have control over the threads and the items received from the Mono are never assigned to my object unless I intentionally block() that thread. Kindly suggest if any non-blocking way possible for this scenario.
I have also looked into Schedulers, subscribeOn, publishOn but unable to figure out the pipeline.
public Flux<Object> test {
method1().map(obj -> {
if (obj.getTotalItems() > 20) {
obj.setItems(method2(obj).block());
}
return obj;
});
}
Here method1 is emitting Flux of objects received from API hit.
And method2 is emitting a list of items fetched from another API hit.
How can I make this whole flow non-blocking?
Try flatMap or concatMap
using flatMap operator you can flatten substream in non-blocking public
Flux<Object> test {
method1().flatMap(obj -> {
if (obj.getTotalItems() > 20) {
return method2(obj)
.map(result -> {
obj.setItems(result);
return obj;
});
}
return Mono.just(obj);
});
}
flatMap allows you to flatten several streams at a time, so in case of long-running operations, you may in more efficient process elements.
One downside of flatMap is that it does not preserve the order of elements so if you have a sequence of upstream elements like [1, 2, 3, 4] with flatMap there is a chance that the order will be changed because of asynchronous nature of substreams.
To preserve order, you can use concatMap which flatten only once streams at a time, so there are guarantees that order of flattening elements will be preserved:
Flux<Object> test {
method1().concatMap(obj -> {
if (obj.getTotalItems() > 20) {
return method2(obj)
.map(result -> {
obj.setItems(result);
return obj;
});
}
return Mono.just(obj);
});
}
Note
Mutation of the objects such a way is not the best idea, and I would prefer to use immutable object pattern object in reactive programming
I am going through the tutorial:
https://marcosantadev.com/mvvmc-with-swift/
Which talks about MVVM-C design pattern. I have real trouble understanding of how and why .never() observable is used there (and in general why we would want to use .never() besides testing timeouts).
Could anyone give a reasonable example of .never() observable usage in swift code (not in testing) and explain why it is necessary and what are the alternatives?
I address all the actions from View to ViewModel. User taps on a button? Good, the signal is delivered to a ViewModel. That is why I have multiple input observables in ViewModel. And all the observables are optional. They are optional because sometimes I write tests and don't really want to provide all the fake observables to test some single function. So, I provide other observables as nil. But working with nil is not very convenient, so I provide some default behavior for all the optional observables like this:
private extension ViewModel {
func observableNavigation() -> Observable<Navigation.Button> {
return viewOutputFactory().observableNavigation ?? Observable.never()
}
func observableViewState() -> Observable<ViewState> {
return viewOutputFactory().observableViewState ?? Observable.just(.didAppear)
}
}
As you can see, if I pass nil for observableViewState I substitute it with just(.didAppear) because the ViewModel logic heavily depends on the state of view. On the other hand if I pass nil for observableNavigation I provide never() because I assume that non of the navigation button will ever be triggered.
But this whole story is just my point of view. I bet you will find your own place to use this never operator.
Maybe your ViewModel has different configurations (or you have different viewModel under the same protocol), one of which does not need to send any updates to its observers. Instead of saying that the observable does not exist for this particular case (which you would implement as an optional), you might want to be able to define an observable as a .never(). This is in my opinion cleaner.
Disclaimer - I am not a user of RxSwift, but I am assuming never is similar than in ReactiveSwift, i.e. a signal that never sends any value.
It's an open ended question, and there can be many answers, but I've found myself reaching for never on a number of cases. There are many ways to solve a problem, but recently, I was simplifying some device connection code that had a cascading fail over, and I wanted to determine if my last attempt to scan for devices yielded any results.
To do that, I wanted to create an observable that only emitted a "no scan results" event in the event that it was disposed without having seen any results, and conversely, emitted nothing if it did.
I have pruned out other details from my code to sake of brevity, but in essence:
func connect(scanDuration: TimeInterval) -> Observable<ConnectionEvent> {
let scan = scan(for: scanDuration).share(replay: 1)
let connection: Observable<ConnectionEvent> =
Observable.concat(Observable.from(restorables ?? []),
connectedPeripherals(),
scan)
.flatMapLatest { [retainedSelf = self] in retainedSelf.connect(to: $0) }
let scanDetector = scan
.toArray() // <-- sum all results as an array for final count
.asObservable()
.flatMap { results -> Observable<ConnectionEvent> in
results.isEmpty // if no scan results
? Observable.just(.noDevicesAvailable) // emit event
: Observable.never() } // else, got results, no action needed
// fold source and stream detector into common observable
return Observable.from([
connection
.filter { $0.isConnected }
.flatMapLatest { [retained = self] event -> Observable<ConnectionEvent> in
retained.didDisconnect(peripheral: event.connectedPeripheral!.peripheral)
.startWith(event) },
scanDetector])
.switchLatest()
}
For a counter point, I realized as I typed this up, that there is still a simpler way to achieve my needs, and that is to add a final error emitting observable into my concat, it fails-over until it hits the final error case, so I don't need the later error detection stream.
Observable.concat(Observable.from(restorables ?? []),
connectedPeripherals(),
scan,
hardFailureEmitNoScanResults())
That said, there are many cases where we may want to listen and filter down stream, where the concat technique is not available.
BrightFutures is a nice implementation of "future" in the Swift language.
https://github.com/Thomvis/BrightFutures
I like to control the parallelism of multicore CPU with it. Does someone know a way to control the # of CPU cores/physical threads to be used?
All closures passed to BrightFutures are executed according to BF's default threading model. It seems like you want to diverge from the default model. This is possible by passing a custom execution context.
An execution context that limits the number of parallel tasks it executes, could be created with the following function:
func executionContextWithControlledParallelism(p: Int) -> ExecutionContext {
let s = Semaphore(value: p)
let q = Queue.global.context
return { task in
s.wait()
q {
task()
s.signal()
}
}
}
I tested this briefly using the following code:
let context = executionContextWithControlledParallelism(5)
for _ in 0..<100 {
future(context:context) { () -> Int in
return fibonacci(Int(arc4random_uniform(15)))
}
}
You'll have to pass context to every map, flatMap, etc. that you want to limit the parallelism of. I'll admit that seems cumbersome. A better way (that is currently not supported by BrightFutures) would be to set the default threading model, like this:
let context = executionContextWithControlledParalelism(5)
// this is not supported right now:
BrightFutures.setDefaultThreadingModel(model: {
return context
})
If you like this, please consider filing an issue to request this or (even better) create a pull request.
Is there a way to have an observable sequence to resume execution with the next element in the sequence if an error occurs?
From this post it looks like you need to specify a new observable sequence in Catch() to resume execution, but what if you needed to just continue processing with the next element in the sequence instead? Is there a way to achieve this?
UPDATE:
The scenario is as follows:
I have a bunch of elements that I need to process. The processing is made up of a bunch of steps. I have
decomposed the steps into tasks that I would like to compose.
I followed the guidelines for ToObservable() posted here
to convert by tasks to an observables for composition.
so basically I'm doing somethng like so -
foreach(element in collection)
{
var result = from aResult in DoAAsync(element).ToObservable()
from bResult in DoBAsync(aResult).ToObservable()
from cResult in DoCAsync(bResult).ToObservable()
select cResult;
result.subscribe( register on next and error handlers here)
}
or I could something like this:
var result =
from element in collection.ToObservable()
from aResult in DoAAsync(element).ToObservable()
from bResult in DoBAsync(aResult).ToObservable()
from cResult in DoCAsync(bResult).ToObservable()
select cResult;
What is the best way here to continue processing other elements even if let's say the processing of
one of the elements throws an exception. I would like to be able to log the error and move on ideally.
Both James & Richard made some good points, but I don't think they have given you the best method for solving your problem.
James suggested using .Catch(Observable.Never<Unit>()). He was wrong when he said that "will ... allow the stream to continue" because once you hit an exception the stream must end - that is what Richard pointed out when he mentioned the contract between observers and observables.
Also, using Never in this way will cause your observables to never complete.
The short answer is that .Catch(Observable.Empty<Unit>()) is the correct way to change a sequence from one that ends with an error to one that ends with completion.
You've hit on the right idea of using SelectMany to process each value of the source collection so that you can catch each exception, but you're left with a couple of issues.
You're using tasks (TPL) just to turn a function call into an observable. This forces your observable to use task pool threads which means that the SelectMany statement will likely produce values in a non-deterministic order.
Also you hide the actual calls to process your data making refactoring and maintenance harder.
I think you're better off creating an extension method that allows the exceptions to be skipped. Here it is:
public static IObservable<R> SelectAndSkipOnException<T, R>(
this IObservable<T> source, Func<T, R> selector)
{
return
source
.Select(t =>
Observable.Start(() => selector(t)).Catch(Observable.Empty<R>()))
.Merge();
}
With this method you can now simply do this:
var result =
collection.ToObservable()
.SelectAndSkipOnException(t =>
{
var a = DoA(t);
var b = DoB(a);
var c = DoC(b);
return c;
});
This code is much simpler, but it hides the exception(s). If you want to hang on to the exceptions while letting your sequence continue then you need to do some extra funkiness. Adding a couple of overloads to the Materialize extension method works to keep the errors.
public static IObservable<Notification<R>> Materialize<T, R>(
this IObservable<T> source, Func<T, R> selector)
{
return source.Select(t => Notification.CreateOnNext(t)).Materialize(selector);
}
public static IObservable<Notification<R>> Materialize<T, R>(
this IObservable<Notification<T>> source, Func<T, R> selector)
{
Func<Notification<T>, Notification<R>> f = nt =>
{
if (nt.Kind == NotificationKind.OnNext)
{
try
{
return Notification.CreateOnNext<R>(selector(nt.Value));
}
catch (Exception ex)
{
ex.Data["Value"] = nt.Value;
ex.Data["Selector"] = selector;
return Notification.CreateOnError<R>(ex);
}
}
else
{
if (nt.Kind == NotificationKind.OnError)
{
return Notification.CreateOnError<R>(nt.Exception);
}
else
{
return Notification.CreateOnCompleted<R>();
}
}
};
return source.Select(nt => f(nt));
}
These methods allow you to write this:
var result =
collection
.ToObservable()
.Materialize(t =>
{
var a = DoA(t);
var b = DoB(a);
var c = DoC(b);
return c;
})
.Do(nt =>
{
if (nt.Kind == NotificationKind.OnError)
{
/* Process the error in `nt.Exception` */
}
})
.Where(nt => nt.Kind != NotificationKind.OnError)
.Dematerialize();
You can even chain these Materialize methods and use ex.Data["Value"] & ex.Data["Selector"] to get the value and selector function that threw the error out.
I hope this helps.
The contract between IObservable and IObserver is OnNext*(OnCompelted|OnError)? which is upheld by all operators, even if not by the source.
Your only choice is to re-subscribe to the source using Retry, but if the source returns the IObservable instance for every description you won't see any new values.
Could you supply more information on your scenario? Maybe there is another way of looking at it.
Edit: Based on your updated feedback, it sounds like you just need Catch:
var result =
from element in collection.ToObservable()
from aResult in DoAAsync(element).ToObservable().Log().Catch(Observable.Empty<TA>())
from bResult in DoBAsync(aResult).ToObservable().Log().Catch(Observable.Empty<TB>())
from cResult in DoCAsync(bResult).ToObservable().Log().Catch(Observable.Empty<TC>())
select cResult;
This replaces an error with an Empty which would not trigger the next sequence (since it uses SelectMany under the hood.