Compare List<String> to Flux<String> in non blocking way - reactive-programming

How to compare List to Flux in non blocking way
Below the code in blocking way
public static void main(String[] args) {
List<String> all = List.of("A", "B", "C", "D");
Flux<String> valid = Flux.just("A", "D");
Map<Boolean, List<String>> collect = all.stream()
.collect(Collectors.groupingBy(t -> valid.collectList().block().contains(t)));
System.out.println(collect.get(Boolean.TRUE));
System.out.println(collect.get(Boolean.FALSE));
}
how to get it working in non-blocking way?
Above is an example of what i am trying to do in web application. I receive list of object which is List all. Then i query database which return Flux . Flux returned by database will be subset of List all. I need to prepare two lists. List of items which are present in Flux of valid and List of items which are not present in Flux of valid
EDIT:
I converted Flux to Mono and List to Mono,
public static void main(String[] args) {
Mono<List<String>> all = Mono.just(List.of("A", "B", "C", "D"));
Mono<List<String>> valid = Mono.just(List.of("A", "D"));
var exist = all.flatMap(a -> valid.map(v -> a.stream().collect(Collectors.groupingBy(v::contains))));
System.out.println(exist.block().get(Boolean.TRUE));
System.out.println(exist.block().get(Boolean.FALSE));
}

There is no straightforward way of achieve this in reactive programming without breaking some of its semantics.
If you reflect back on what reactive programming tris to achieve and your problem statement, you should notice that those won't play well that much together.
Reactive programming, as the name suggests, is about reacting to events which in your case would be valid items emitted from your datastore. In a typical situation, you should have been programming your statement to compute some assertions around the emitted valid items then emit these (or some other transformations downstream). Unfortunately, you won't be able to compute the all and valid items intersection and diversion without stopping at some point (otherwise how would you know that an item you assumed non-valid is not emitted at some point by the valid publisher).
Though, to achieve the desired behavior, you will lean on memory to buffer items then trigger your validations.
Retrieving valid items should be achievable using the filterWhen operator paired with the hasElement one:
Flux<String> validItems = Flux.fromIterable(all)
.filterWhen(valid::hasElement);
To retrieve the invalid items, you can collect all and validItems merged together then filter out elements that do appear more than once:
Flux<String> inValidItems = Flux.fromIterable(all)
.mergeWith(validItems)
.collectList()
.flatMapIterable(list -> list.stream().filter(item -> Collections.frequency(list, item) == 1).collect(Collectors.toList()));

Related

Make iterations of a loop sequentially in Mutiny

I am new in the reactive programming world. I am currently working in a Java reactive application using the Mutiny library.
I need to develop a loop that waits for the previous iteration to finish in order to start the next one. For instance:
List<Uni<T>> uniList = new ArrayList<>();
for (T item : items) { //items is an already fulfilled collection
uniList.add(this.doSomethingAndReturnInUni(item));
}
return Uni.combine().all().unis(uniList).combinedWith(unisToCombine -> {
List<T> list = new ArrayList<>();
unisToCombine.forEach(x ->list.add(x));
return list;
});
The for loop in the example, generates a thread per iteration. I am wondering how to order the i-th call to the method doSomethingAndReturnInUni() waits for the (i-1) call to trigger the event, that is, make the for loop sequentially. It is possible to suscribe those events in such a way?
Could you try something like this?
Builder<Item> items = Uni.join().builder();
for (Item item : items) {
builder.add(this.doSomethingAndReturnInUni(item));
}
return builder.joinAll().andCollectFailures()
.flatMap(itemList -> do whatever you need ...) //itemList type is List<Item>
I don't know why you are using uni, as this should just handle one operation, for loops you should use multi, where you can handle the back pressure, and only get the next event, when one event is finished. Multi can be run sequentially and in parallel.
see https://quarkus.io/blog/mutiny-back-pressure/
I’ve done the same, using Multi’s see the ‘generateData()’ method here:
https://github.com/Serkan80/quarkus-quickstarts/blob/development/redis-streams-quickstart/weather-producer/src/main/java/org/acme/redis/streams/producer/ValuesGenerator.java

Can I use AtomicReference to get value of a Mono and code still remain reactive

Sorry, I am new to reactive paradigm. Is is possible to use AtomicReference to get value of a Mono since reactive code can run asynchronously and different events run on different thread. Please see the sample below. I am also not sure if this piece of code is considered reactive
sample code:
public static void main(String[] a) {
AtomicReference<UserDTO> dto = new AtomicReference<>();
Mono.just(new UserDTO())
.doOnNext(d -> d.setUserId(123L))
.subscribe(d -> dto.set(d));
UserDTO result = dto.get();
dto.set(null);
System.out.println(result); // produce UserDTO(userId=123)
System.out.println(dto.get()); // produce null
}
The code snippet you have shared is not guaranteed to always work. There is no way to guarantee that the function inside doOnNext will happen before dto.get(). You have created a race condition.
You can run the follow code to simulate this.
AtomicReference<UserDTO> dto = new AtomicReference<>();
Mono.just(new UserDTO())
.delayElement(Duration.ofSeconds(1))
.doOnNext(d -> d.setUserId(123L))
.subscribe(dto::set);
UserDTO result = dto.get();
System.out.println(result); // produces null
To make this example fully reactive, you should print out in the subscribe operator
Mono.just(new UserDTO())
.doOnNext(d -> d.setUserId(123L))
.subscribe(System.out::println)
In a more "real world" example, your method would return a Mono<UserDTO> and you would then perform transformations on this using map or flatMap operators.
** EDIT **
If you are looking to make a blocking call within a reactive stream this previous stack overflow question contains a good answer

Spring webflux/reactor using #Scheduled to read database and perform some tasks

I am new to spring webflux and my current spring boot application uses a scheduler(annotated as #Scheduled) to read list of data from DB, call a rest api concurrently in batches and then writes to event stream
I want to achieve the same in Spring webflux.
Should I use #Scheduled or use schedulePeriodically from Webflux?
How can I batch items from DB into smaller sets(say 10 items) and concurrently call rest api?
At present the app fetches max 100 records in one scheduler run and then it processes them. I am planning to shift to r2dbc, if i do so, can i limit the flow of data like 100?
Thanks
1. Should I use #Scheduled or use schedulePeriodically from Webflux?
#Scheduled is an annotation which is part of the spring framework scheduled package, while schedulePeriodically is a function which is part of reactor, so you can't really compare the two. I dont see any problems in using the annotation since it is part of the core framework.
2. How can I batch items from DB into smaller sets (say 10 items) and concurrently call rest api?
By using the Flux#buffer functions which will emit a list of items when the buffer is full.
Flux.just("1", "2", "3", "4")
.buffer(2)
.doOnNext(list -> {
System.out.println(list.size());
}).subscribe()
Will print 2 each time.
3. At present the app fetches max 100 records in one scheduler run and then it processes them. I am planning to shift to r2dbc, if i do so, can i limit the flow of data like 100?
Well you can as written before, you fetch, and then buffer the responses into lists of 100, you can then place each list in its own flux and emit items again, or process each list of 100 items. Up to you.
There are a lot of functions under the buffer segment, check them out.
Flux#buffer
Flux.buffer will combine the streams and will emit a list of streams of mentioned buffer size.
For batching purpose, you can use Flux.expand or Mono.expand. You only have to provide your condition in the expand to execute it again or finally end it.
Here are the examples:
public static void main(String[] args) {
List<String> list = new ArrayList<>();
list.add("1");
Flux.just(list)
.buffer(2)
.doOnNext(ls -> {
System.out.println(ls.getClass());
// Buffering a list returns the list of list of String
System.out.println(ls);
}).subscribe();
Flux.just(list)
.expand(listObj -> {
// Condition to finally end the batch
if (listObj.size()>4) {
return Flux.empty();
}
// Can return the size of data as much as you require
list.add("a");
return Flux.just(listObj);
}).map(ls -> {
// Here it returns list of String which was the original object type not list of list as in case of buffer
System.out.println(ls.getClass());
System.out.println(ls);
return ls;
}).subscribe();
}
Output:
class java.util.ArrayList
[[1]] /// Output of buffer list of list
class java.util.ArrayList
[1]
class java.util.ArrayList
[1, a]
class java.util.ArrayList
[1, a, a]
class java.util.ArrayList
[1, a, a, a]
class java.util.ArrayList
[1, a, a, a, a]

Concat multiple reactive requests to one Mono

I noticed in the reactive libraries there are Tuples, but what do I do if there are more than 8 Tuples?
https://projectreactor.io/docs/core/release/api/reactor/util/function/Tuples.html#fromArray-java.lang.Object:A-
Example code that seems to work, but is there a better way to use some sort of collector?
private Mono<List<String>> getContent(List<String> ids) {
List<String> allContent = new ArrayList<>();
Mono<List<String>> allContentMono = Mono.empty();
for(String id : ids) {
allContentMono = callApi(id)
.flatMap(result -> result.bodyToMono(String.class))
.map(str -> {
allContent.add(str);
return allContent;
});
}
return allContentMono;
}
Why did the tuple size stop at 8? (haven't looked around for the documentation on why, but not my main concern)
Thanks
zip (which uses TupleN) is for when you want to create values by compositon, out of a combination of sources. Eg. out of a Flux<FirstName> and Flux<LastName> you want a Flux<FullName>, that emits one FullName for each incoming FistName/LastName pair.
For your use case, where you want to execute multiple calls (possibly in parallel) and collect the results in a list, flatMap is enough:
private Mono<List<String>> getContent(List<String> ids) {
return Flux
.fromIterable(ids)
.flatMap(id -> callApi(id))
.flatMap(response -> response.bodyToMono(String.class))
.collectList();
}
Tuple is an immutable, fixed-size data structure, used by zip as convenience when you don't want to create a dedicated POJO. It doesn't make sense to try and support unlimited sizes so we stopped at eight. There is a zip variant that will aggregate more than 8 sources, but will make you work with an Object[] instead of a Tuple.

Aggregate PropertyChanged events from the collection into one IObservable<EventPattern<PropertyChangedEventArgs>>

I have collection of INotifyPropertyChanged objects and would like to stream all PropertyChanged events into a single observable sequence for further processing.
Here is a code
IObservable<EventPattern<PropertyChangedEventArgs>> _allEvents = null;
// Items contains collection of item, which implements INotifyPropertyChanged
foreach (var item in Items)
{
var seq = Observable.FromEventPattern<PropertyChangedEventArgs>(item, "PropertyChanged");
if (_allEvents == null)
_allEvents = seq;
else
_allEvents.Merge(seq);
}
// subscribe to the aggregated observable sequence
if (_allEvents != null)
_allEvents.Subscribe(evt => Trace.WriteLine(" Property Changed -> " + evt.EventArgs.PropertyName));
A single Subscribe doesn't work here for some reason in the aggregated sequence.
Looks like I aggregated (using Reactive Extensions's Merge function) incorrectly. But, subscribe inside the loop works perfectly.
Can anybody assist me here, how to aggregate a many event streams into one with reactive extensions?
Thanks
Try this:
var _allEvents = Observable
.Merge(Items
.Select(item => Observable
.FromEventPattern<PropertyChangedEventArgs>(item, "PropertyChanged")));
_allEvents.Subscribe(evt => Trace.WriteLine(" Property Changed -> " + evt.EventArgs.PropertyName));
The reason your approach doesn't work is that you were calling IObservable<T> Observable.Merge<T>(this IObservable<T> first, IObservable<T> second). The return type of this is a resulting observable. I think you might have been thinking that Merge modified the Observable, but you can think of Observables as immutable (sort of). The way to get your approach to work would have been:
_allEvents = _allEvents.Merge(seq);
But... yuck. Don't do that.