I have create one observable
Observable<Map<Integer, String>> observable = Observable.create(s -> {
try {
System.out.println("getMultipleCitiesName ==="+Thread.currentThread().getName());
List<String> cityIdsString = new ArrayList<>();
for (Integer cityId : cityIds) {
cityIdsString.add(cityId.toString());
}
MultiValueMap<String, String> formParams = new LinkedMultiValueMap<>();
formParams.put("cityIds[]", cityIdsString);
// Call the Location Client to call the API
Response<Map<Integer, String>> response = locationClient.getMultipleCitiesName(formParams);
s.onNext(response.getData());
} catch (Exception e) {
System.out.println("Inside Exception CITY NEW");
s.onError(e);
}
s.onComplete();
});
Now I want to add onErrorReturnItem() and retry() both.
So I tried it two ways
a) observable = observable.onErrorReturnItem(new HashMap<>()).retry(3);
b) observable = observable.retry(3).onErrorReturnItem(new HashMap<>());
b) is working (meaning retry and onErrorReturnItem) both are working
while in a) retry is not working?
Why is that
Order of operators matters, this is how you have to interpret the chains:
(a) you're saying something like Let take whatever comes from observable and if some error happens upstream then let have an observable that emit an empty HashMap(i.e calling onErrorReturnItem) and then retry on this observable(if some error is emitted). but no error is emitted by the observable onErrorReturnItem it just emit the empty HashMap followed by an onComplete signal. i.e whatever the error happens upstream is hidden by onErrorReturnItem and retry would never be signaled an onError to start retrying.
(b) you've saying completely the opposite, Let take whatever comes from observable and if some error happens upstream let retry, if retry can't get any valid item after 3 attempts, then signal onError downstream and onErrorReturnItem will return an empty HashMap.
Now it should be clear why (b) tend be what you're expecting to happens.
Related
could you please advise , how can I stop sending to my 3rd kafka topic, when the control reaches the catch block, currently the message is sent to both error topic as well as the topic to which it should send in case of normal processing. A snippet of code is as below:
#Component
public class Abc {
private final StreamBridge streamBridge;
public Abc (StreamBridge streamBridge)
this.streamBridge = streamBridge;
#Bean
public Function<KStream<String, KafkaClass>, KStream<String,KafkaClass>> hiProcess() {
return input -> input.map((key,value) -> {
try{
KafkaClass stream = processFunction();
}
catch(Exception e) {
Message<KakfaClass> mess = MessageBuilder.withPayload(value).build();
streamBridge.send("errProcess-out-0". mess);
}
return new KeyValue<>(key, stream);
})
}
}
This can be implemented using the following pattern:
KafkaClass stream;
return input -> input
.branch((k, v) -> {
try {
stream = processFunction();
return true;
}
catch (Exception e) {
Message<KakfaClass> mess = MessageBuilder.withPayload(value).build();
streamBridge.send("errProcess-out-0". mess);
return false;
}
},
(k, v) -> true)[0]
.map((k, v) -> new KeyValue<>(k, stream));
Here, we are using the branching feature (API) of KStream to split your input into two paths - normal flow and the one causing the errors. This is accomplished by providing two filters to the branch method call. The first filter is the normal flow in which you call the processFunction method and get a response back. If we don't get an exception, the filter returns true, and the result of the branch operation is captured in the first element of the output array [0] which is processed downstream in the map operation in which it sends the final result to the outbound topic.
On the other hand, if it throws an exception, it sends whatever is necessary to the error topic using StreamBridge and the filter returns false. Since the downstream map operation is only performed on the first element of the array from branching [0], nothing will be sent outbound. When the first filter returns false, it goes to the second filter which always returns true. This is a no-op filter where the results are completely ignored.
One downside of this particular implementation is that you need to store the response from processFunction in an instance field and then mutate on each incoming KStream record so that you can access its value in the final map method where you send the output. However, for this particular use case, this may not be an issue.
I have an hot IObservable<T> which may throw an exception. However, I would like to continue with it. I think I could use Retry operator for that. However, it would be great if I can also listen to any error in IObservable<T> through a separate IObservable<Exception>. Is it possible?
Your case is significantly more simplified in that you have a hot observable.
OnError is a notification outside your value stream, so we could materialize the notifications to retrieve the error. This still causes the tear-down of the stream with an OnCompleted, so you'll need to re-subscribe with Repeat.
var exceptions =
source
.Materialize()
.Where(notif => notif.Kind == NotificationKind.OnError)
.Select(notif => notif.Exception)
.Repeat();
Note
If you're using a Subject<T> for your hot observable, you might run into the usual problem of re-subbing a subject. A subject will replay its OnError or OnCompleted notifications for every new observer.
var source = new Subject<int>();
source.OnNext(1);
source.OnError(new Exception());
source.Subscribe(
i => Console.WriteLine(i),
ex => Console.WriteLine("Still got exception after the throw")
);
In this case your exception stream will go into an infinite re-subscription loop.
The premise of your question violates the observable contract:
An Observable may make zero or more OnNext notifications, each representing a single emitted item, and it may then follow those emission notifications by either an OnCompleted or an OnError notification, but not both. Upon issuing an OnCompleted or OnError notification, it may not thereafter issue any further notifications. (emphasis mine)
In other words, after your hot IObservable<T> throws an exception, the observable is ended. The observable of exceptions that comes out of that has a max count of one.
If you want to support a scenario where you re-start an observable after an exception, you're producing a stream of observables, or IObservable<IObservable<T>>. To work with that, here's a code sample:
var source = new Subject<Subject<int>>();
var exceptionStream = source
.SelectMany(o => o.Materialize())
.Where(n => n.Kind == NotificationKind.OnError)
.Select(n => n.Exception);
var itemStream = source
.SelectMany(o => o.Materialize())
.Where(n => n.Kind == NotificationKind.OnNext)
.Select(n => n.Value);
var items = new List<int>();
var exceptions = new List<Exception>();
itemStream.Subscribe(i => items.Add(i));
exceptionStream.Subscribe(e => exceptions.Add(e));
var currentSubject = new Subject<int>();
source.OnNext(currentSubject);
currentSubject.OnNext(1);
currentSubject.OnNext(2);
currentSubject.OnNext(3);
currentSubject.OnError(new Exception("First error"));
var currentSubject2 = new Subject<int>();
source.OnNext(currentSubject2);
currentSubject2.OnNext(4);
currentSubject2.OnNext(5);
currentSubject2.OnNext(6);
currentSubject2.OnError(new Exception("Second error"));
items.Dump(); //Linqpad
exceptions.Dump(); //Linqpad
I would like to merge two Single<MyData> such that if one of them fails but the other one succeeds then the error of the one that failed and the emission from the other one are reported, and then the resulting Single<MyData> (or Observable<MyData>) completes.
If both Single<MyData> fail then the result should also fail and also be marked as failed.
What I would like to have at the end is:
If both succeed then the emitted values and a producer marked as completed.
If one succeeds and the other fails, the emitted value, the thrown error and the producer marked as complete.
If all fail, the errors and the producer marked as failed.
It's like an 'OR' operation
This is not possible. There is only a single terminal event allowed. The contract for Single is success|error. If you need to receive a next event as well, you should consider to use Observable instead. The contract for Observable is next* complete|error, but you'll still not get a complete.
Observable.mergeDelayError(single1.toObservable(), single2.toObservable())
This can be accomplished with Single.create(SingleOnSubscribe). If your return type is Single<MyData> only one of the responses can be returned, but you could also modify this to instead return a Single<List<MyData>> or some other RxJava structure like Flowable<MyData> that supports multiple returns. In this example, the Single<MyData> returns whichever call returns last because that was the simplest to implement.
public Single<MyData> getCombinedSingle(List<Single<MyData>> singles) {
return Single.create(new SingleOnSubscribe<MyData> {
private boolean encounteredError = false;
private MyData myData;
#Override
public void subscribe(#NonNull Emitter<MyData> emitter) {
List<Disposable> disposables = new ArrayList<>();
Consumer<MyData> myDataConsumer = myData -> {
this.MyData = myData;
checkForFinish(emitter, disposables);
}
Consumer<Throwable> throwableConsumer = throwable -> {
throwable.printStackTrace();
encounteredError = true;
checkForFinish(emitter, disposables);
}
for (Single single: singles) {
disposables.put(single.subscribe(myDataConsumer, throwableConsumer);
}
}
private void checkForFinish(SingleEmitter<MyData> emitter, List<Disposable> disposables) {
if (disposables1.stream().allMatch(Disposable::isDisposed)) {
if (encounteredError) {
emitter.onError(new Throwable());
} else {
emitter.onSuccess(myData);
}
}
}
}
}
This could be modified to return a Throwable from the original Singles if needed.
I have a problem with my Rx subscription using Switch statement.
_performSearchSubject
.AsObservable()
.Select(_ => PerformQuery())
.Switch()
.ObserveOn(_synchronizationContextService.SynchronizationContext)
.Subscribe(DataArrivedForPositions, PositionQueryError, PositionQueryCompleted)
.DisposeWith(this);
The flow is:
Some properties change and the performSearchSubject.OnNext is called
The PerformPositionQuery() is called, which returns a observer each time it is hit
The service which responds through this observer calls OnNext twice and OnCompleted once when the data receive is done
Method DataArrivedForPositions is called twice as expected
Method PositionQueryCompleted is never called, though observer.OnCompleted() is called inside my data service.
Code for dataService is:
protected override void Request(Request request, IObserver<Response> observer)
{
query.Arrive += p => QueryReceive(request.RequestId, p, observer, query);
query.Error += (type, s, message) => QueryError(observer, message);
query.NoMoreData += id => QueryCompleted(observer);
query.Execute(request);
}
private void QueryError(IObserver<PositionSheetResponse> observer, string message)
{
observer.OnError(new Exception(message));
}
private void QueryCompleted(IObserver<PositionSheetResponse> observer)
{
observer.OnCompleted();
}
private void QueryReceive(Guid requestId, Qry0079Receive receiveData, IObserver<PositionSheetResponse> observer, IQry0079PositionSheet query)
{
observer.OnNext(ConvertToResponse(requestId, receiveData));
}
Switch result will only Complete when your outer observable (_performSearchSubject) completes. I assume in your case this one never does (it's probably bound to a user action performing the search).
What's unclear is when you expect PositionQueryCompleted to be called. If It's after each and every successful query is processed, then your stream needs to be modified, because Switch lost you the information that the query stream completed, but it also lacks information about the UI (wrong scheduler even) to say whether its data was actually processed.
There may be other ways to achieve it, but basically you want your query stream complete to survive through Switch (which currently ignore this event). For instance you can transform your query stream to have n+1 events, with one extra for the complete:
_performSearchSubject
.AsObservable()
.Select(_ =>
PerformQuery()
.Select(Data => new { Data, Complete = false})
.Concat(Observable.Return(new { Data = (string)null, Complete = true })))
You can safely apply .Switch().ObserveOn(_synchronizationContextService.SynchronizationContext) on it, but then you need to modify your subscription:
.Subscribe(data => {
if (data.Complete) DataArrivedForPositions(data.Data);
else PositionQueryCompleted()
}, PositionQueryError)
i've an observable that I create with the following code.
Observable.create(new Observable.OnSubscribe<ReturnType>() {
#Override
public void call(Subscriber<? super ReturnType> subscriber) {
try {
if (!subscriber.isUnsubscribed()) {
subscriber.onNext(performRequest());
}
subscriber.onCompleted();
} catch (Exception e) {
subscriber.onError(e);
}
}
});
performRequest() will perform a long running task as you might expect.
Now, since i might be launching the same Observable twice or more in a very short amount of time, I decided to write such transformer:
protected Observable.Transformer<ReturnType, ReturnType> attachToRunningTaskIfAvailable() {
return origObservable -> {
synchronized (mapOfRunningTasks) {
// If not in maps
if ( ! mapOfRunningTasks.containsKey(getCacheKey()) ) {
Timber.d("Cache miss for %s", getCacheKey());
mapOfRunningTasks.put(
getCacheKey(),
origObservable
.doOnTerminate(() -> {
Timber.d("Removed from tasks %s", getCacheKey());
synchronized (mapOfRunningTasks) {
mapOfRunningTasks.remove(getCacheKey());
}
})
.cache()
);
} else {
Timber.d("Cache Hit for %s", getCacheKey());
}
return mapOfRunningTasks.get(getCacheKey());
}
};
}
Which basically puts the original .cache observable in a HashMap<String, Observable>.
This basically disallows multiple requests with the same getCacheKey() (Example login) to call performRequest() in parallel. Instead, if a second login request arrives while another is in progress, the second request observable gets "discarded" and the already-running will be used instead. => All the calls to onNext are going to be cached and sent to both subscribers actually hitting my backend only once.
Now, suppouse this code:
// Observable loginTask
public void doLogin(Observable<UserInfo> loginTask) {
loginTask.subscribe(
(userInfo) -> {},
(throwable) -> {
if (userWantsToRetry()) {
doLogin(loinTask);
}
}
);
}
Where loginTask was composed with the previous transformer. Well, when an error occurs (might be connectivity) and the userWantsToRetry() then i'll basically re-call the method with the same observable. Unfortunately that has been cached and I'll receive the same error without hitting performRequest() again since the sequence gets replayed.
Is there a way I could have both the "same requests grouping" behavior that the transformer provides me AND the retry button?
Your question has a lot going on and it's hard to put it into direct terms. I can make a couple recommendations though. Firstly your Observable.create can be simplified by using an Observable.defer(Func0<Observable<T>>). This will run the func every time a new subscriber is subscribed and catch and channel any exceptions to the subscriber's onError.
Observable.defer(() -> {
return Observable.just(performRequest());
});
Next, you can use observable.repeatWhen(Func1<Observable<Void>, Observable<?>>) to decide when you want to retry. Repeat operators will re-subscribe to the observable after an onComplete event. This particular overload will send an event to a subject when an onComplete event is received. The function you provide will receive this subject. Your function should call something like takeWhile(predicate) and onComplete when you do not want to retry again.
Observable.just(1,2,3).flatMap((Integer num) -> {
final AtomicInteger tryCount = new AtomicInteger(0);
return Observable.just(num)
.repeatWhen((Observable<? extends Void> notifications) ->
notifications.takeWhile((x) -> num == 2 && tryCount.incrementAndGet() != 3));
})
.subscribe(System.out::println);
Output:
1
2
2
2
3
The above example shows that retries are aloud when the event is not 2 and up to a max of 22 retries. If you switch to a repeatWhen then the flatMap would contain your decision as to use a cached observable or the realWork observable. Hope this helps!