I want to combine result form two Mono based on some condition. Both Mono are results of WebClient calls:
The first one is a single call expecting fast response.
The second one is a combination of several calls with a slow response.
The idea to "cancel" the second Mono if result from the first one satisfies some condition to save time and avoid unnecessary network calls. If the first's Mono result is not enough zip it with the second Mono.
A Kotlin code sample to explain my idea:
fun getResult(): Mono<Result> {
val trivialResultMono: Mono<Result> = webClient.getResult()
val nonTrivialResultMono: Mono<Result> = webClient
.getResult()
.flatMap { webClient.getResult1(it) }
.flatMap { webClient.getResult2(it) }
.flatMap { webClient.getResult2(it) }
//here I need to check if trivial result satisfies some condition,
//for example trivialResult.size > 5 if it's true I just return
//trivialResultMono from getResult() function,
//it it's false something like this:
return Mono.zip(trivialResultMono, nonTrivialResultMono) { trivialResult, nonTrivialResult ->
trivialResult + nonTrivialResult
}
}
UPDATE:
To be more clear let's say that trivialResult comes in 1 second, nonTrivialResult in 2 seconds. I want to get my final result in 1 second in case of trivialResult.size > 5 and in 2 seconds otherwise.
Using just Mono.zip(trivialResultMono, nonTrivialResultMono) I will always get my final result in 2 seconds.
Using filter + switchIfEmpty it will take 1 second if trivialResult.size > 5 and 3 seconds otherwise. Please correct me if I wrong.
You could filter your trivialResultMono and apply switchIfEmpty operator
return trivialResultMono
.filter(trivialResult -> trivialResult.size > 5)
.switchIfEmpty(Mono.zip(...))
Update for merge approach:
Mono<Result> zipResultMono = Mono.zip...
return Flux.merge(
trivialResultMono.map(trivialResult -> Tuples.of(1, trivialResult)),
zipResultMono.map(zipResult -> Tuples.of(2, zipResult)))
.filter(tuple ->
(tuple.getT1().equals(1) && tuple.getT2().size > 5) ||
tuple.getT1().equals(2))
.next()
.map(Tuple2::getT2);
You could skip converting to the Tuple2 if zipResult always has size more then 5
You can achieve this with flatMap and map:
trivial.flatMap(trivialResult -> {
if (trivialResult.size > 5) {
return Mono.just(trivialResult);
} else {
return nonTrivial.map(nonTrivialResult -> trivialResult + nonTrivialResult);
}
});
Related
I'm learning reactive programming with webflux, and for that I'm migrating some code.
For example I'm trying to migrate this method:
public Set<Vaccine> getAll(Set<Long> vaccinesIds) throws EntityNotFoundException {
if (null == vaccinesIds) {
return null;
}
Set<Long> vaccinesToFind = new HashSet<>(vaccinesIds);
vaccinesToFind.remove(null);
Set<Vaccine> vaccines = new HashSet<>();
vaccineRepository.findByIdIn(vaccinesToFind).forEach(vaccines::add);
if (vaccines.size() != vaccinesToFind.size()) {
LOG.warn("Could not find vaccines with ids: " + vaccinesToFind.removeAll(vaccines.stream().map(Vaccine::getId).collect(Collectors.toSet())));
throw new EntityNotFoundException(VACCINE_ERROR_NOT_FOUND);
}
return vaccines;
}
To summarize the code, if the respository returns all the vaccines that are requested should return the result, if not should return an error.
For that, I thought in something like this, but is not working:
public Flux<Vaccine> getAll(Set<Long> vaccinesIds) {
if (null == vaccinesIds) {
return Flux.empty();
}
Set<Long> vaccinesToFind = new HashSet<>(vaccinesIds);
Flux<Vaccine> byIdIn = vaccineRepository.findByIdIn(vaccinesToFind);
Mono<Long> filter = vaccineRepository.findByIdIn(vaccinesToFind).count().filter(x -> x.equals(Long.valueOf(vaccinesToFind.size())));
return filter.flatMapMany(asd -> vaccineRepository.findByIdIn(vaccinesToFind)
).switchIfEmpty(Flux.error((new EntityNotFoundException(VACCINE_ERROR_NOT_FOUND))));
}
What am I doing wrong?
My first doubt is why the filter is a Mono of Long if it has a equals method in the end. My problem is about evaluating the filter in order to return the list or the error.
First of all, you are querying the same result vaccineRepository.findByIdIn(vaccinesToFind) multiple times. The same data is queried, transferred and deserialized multiple times. This is a sign that something is wrong here.
Let's assume the result set fits into the memory. Then the idea would be to transform flux into a usual collection and to decide whether to emit an error or not:
return vaccineRepository.findByIdIn(vaccinesIds)
.collectList()
.flatMapMany(result -> {
if(result.size() == vaccinesIds.size()) return Flux.fromIterable(result);
else return Flux.error(new EntityNotFoundException(VACCINE_ERROR_NOT_FOUND));
});
In the case the result is to huge for the main memory, you could do count in the db by the first query and in the positive case query the results. The solution is similar to your code:
return vaccineRepository.countByIdIn(vaccinesIds)
.filter(count -> count == vaccinesIds.size())
.flatMapMany($ -> vaccineRepository.findByIdIn(vaccinesIds))
.switchIfEmpty(Mono.error(new EntityNotFoundException(VACCINE_ERROR_NOT_FOUND)));
The result of filter is Mono<Long> because filter just takes the elements from the upstream and tests against the given predicate. If the predicate returns false, the item is filtered out and the Mono is empty. To keep all results of a the test you could use map and the type would be Mono<Boolean>.
So I am trying to create basic pagination in Gatling but failing miserably.
My current scenario is as follows:
1) First call is always a POST request, the body includes page index 1 and page size 50
{"pageIndex":1,"pageSize":50}
I am then receiving 50 object + the total count of objects on the environment:
"totalCount":232
Since I need to iterate through all objects on the environment, I will need to POST this call 5 time, each time with an updated pageIndex.
My current (failing) code looks like:
def getAndPaginate(jsonBody: String) = {
val pageSize = 50;
var totalCount: Int = 0
var currentPage: Int = 1
var totalPages: Int =0
exec(session => session.set("pageIndex", currentPage))
exec(http("Get page")
.post("/api")
.body(ElFileBody("json/" + jsonBody)).asJson
.check(jsonPath("$.result.objects[?(#.type == 'type')].id").findAll.saveAs("list")))
.check(jsonPath("$.result.totalCount").saveAs("totalCount"))
.exec(session => {
totalCount = session("totalCount").as[Int]
totalPages = Math.ceil(totalCount/pageSize).toInt
session})
.asLongAs(currentPage <= totalPages)
{
exec(http("Get assets action list")
.post("/api")
.body(ElFileBody("json/" + jsonBody)).asJson
.check(jsonPath("$.result.objects[?(#.type == 'type')].id").findAll.saveAs("list")))
currentPage = currentPage+1
exec(session => session.set("pageIndex", currentPage))
pause(Config.minDelayValue seconds, Config.maxDelayValue seconds)
}
}
Currently the pagination values are not assign to the variables that I have created at the beginning of the function, if I create the variables at the Object level then they are assigned but in a manner which I dont understand. For example the result of Math.ceil(totalCount/pageSize).toInt is 4 while it should be 5. (It is 5 if I execute it in the immediate window.... I dont get it ). I would than expect asLongAs(currentPage <= totalPages) to repeat 5 times but it only repeats twice.
I tried to create the function in a class rather than an Object because as far as I understand there is only one Object. (To prevent multiple users accessing the same variable I also ran only one user with the same result)
I am obviously missing something basic here (new to Gatling and Scala) so any help would be highly appreciated :)
using regular scala variables to hold the values isn't going to work - the gatling DSL defines builders that are only executed once at startup, so lines like
.asLongAs(currentPage <= totalPages)
will only ever execute with the initial values.
So you just need to handle everything using session variables
def getAndPaginate(jsonBody: String) = {
val pageSize = 50;
exec(session => session.set("notDone", true))
.asLongAs("${notDone}", "index") {
exec(http("Get assets action list")
.post("/api")
.body(ElFileBody("json/" + jsonBody)).asJson
.check(
jsonPath("$.result.totalCount")
//transform lets us take the result of a check (and an optional session) and modify it before storing - so we can use it to get store a boolean that reflects whether we're on the last page
.transform( (totalCount, session) => ((session("index").as[Int] + 1) * pageSize) < totalCount.toInt)
.saveAs("notDone")
)
)
.pause(Config.minDelayValue seconds, Config.maxDelayValue seconds)
}
}
I am new to RxJava2.
I am trying to get a list of Transaction object both from cache and from server.
I want to compare the server value to cache value and if the server value is the same, then ignore it.
I was able to do it easily using .scan() because we can return null and when null is returned from the .scan() the value got ignored(filtered).
RxJava 1
private Observable<List<Transaction>> getTransactionsFromCacheAndServer() {
return Observable.concat(
getTransactionsFromCache(),
getTransactionsFromServer()
)
.scan((p1, p2) -> {
if (p1 == null && p2 != null) {
return p2;
} else if (p1 != null && !isListSame(p1, p2)) {
return p2;
} else {
return null;
}
});
}
With RxJava 2, since I cannot return null anymore, things are not easy.
RxJava 2
private Observable<List<Transaction>> getTransactionsFromCacheAndServer() {
return Observable.concat(
getTransactionsFromCache(),
getTransactionsFromServer()
)
.map(FilterObject::new)
.scan((filterObject1, filterObject2) -> {
List<Transaction> p1 = (List<Transaction>)filterObject1.value;
List<Transaction> p2 = (List<Transaction>)filterObject2.value;
if (p1.size() == 0 && p2.size() > 0) {
return filterObject2;
} else if (!isListSame(p1, p2)) {
return filterObject2;
} else {
filterObject2.filter = true;
return filterObject2;
}
})
.filter(filterObject -> !filterObject.filter)
.map(filterObject -> (List<Transaction>)filterObject.value);
}
Where FilterObject is:
public class FilterObject {
public Object value;
public boolean filter;
public FilterObject(Object value) {
this.value = value;
}
}
Even though I can achieve the same thing using above method, it seems very ugly. Also I had to include two maps which might not be so performance friendly.
Is there a simple/clean way to achieve what I want?
I don't think there is a generic solution to this problem, since an empty list and a list that needs to be filtered (which happens to be empty in all cases) are two different things (the output of the scan) and needs to be handled differently.
However, in your particular case you never emit an empty list, except maybe for the first output.
(I am using String instead Transaction, shouldn't matter)
private Observable<List<String>> getTransactionsFromCacheAndServer() {
return Observable.concat(
getTransactionsFromCache(),
getTransactionsFromServer()
)
.filter(list -> !list.isEmpty())
// If you prefer a consistent empty list over the first
// empty list emission getting filtered
.startWith((List<String>) Collections.EMPTY_LIST)
// Newly emitted value cannot be empty, it only depends only on the comparison
.distinctUntilChanged(this::isListSame);
}
That's the closest I could get with as few operators as possible. Hope it solves your problem.
Based on andras' answer, I modified little bit to achieve what I want.
private Observable<List<String>> getTransactionsFromCacheAndServer() {
return Observable.concat(
getTransactionsFromCache(),
getTransactionsFromServer()
)
.filter(list -> !list.isEmpty())
.distinctUntilChanged(this::isListSame)
.switchIfEmpty(Observable.just(new ArrayList<>()));
}
Andreas' answer will always receive an empty list and then a real data.
My solution above will receive:
1. Data from cache (and then data from server if different)
2. Empty list if both cache and server returns Empty list.
I want to wait till first element gets updated to rate same as monthlyRate variable and second element gets updated to rate same as annualRate.
With below code I get:
Failed: Wait timed out after 9008ms
public waitForSubscriptionRates = (rateselector: string, monthlyRate: string, annualRate: string) => {
browser.wait(function(){
element.all(by.css(rateselector)).filter(function(elem, index) {
return elem.getText().then(function(text) {
console.log('text=' + text);
console.log('monthlyRate=' + monthlyRate);
console.log('annualrate=' + annualRate);
return (text === monthlyRate || text === annualRate);
});
}).first().click();
},9000);
}
I'm not sure do I understand your code well.
I assume that there are two selectors - first may contain text of monthlyRate and second annualRate. They can not be mixed, isn't it?
I don't get what you want to click.
Anyway, you can chain ExpectedConditions with or keyword.
See the example in the documentation: http://www.protractortest.org/#/api?view=ProtractorExpectedConditions.prototype.or
I want to look for an entire list of items to be found before I complete and if that entire list isn't found, then an exception (a Timeout or custom one) is to be thrown. Like the built in Observable.timer() but instead of the test passing once the first item is emitted, I want it to require all of the items in a list to be found.
Here is an example. Let's say I have some test function that emits Observable<FoundNumber>. It looks like this:
var emittedList: List<String?> = listOf(null, "202", "302", "400")
data class FoundNumber(val numberId: String?)
fun scanNumbers(): Observable<FoundNumber> = Observable
.intervalRange(0,
emittedList.size.toLong(),
0,
1,
TimeUnit.SECONDS).map { index ->
FoundNumber(emittedList[index.toInt()]) }
That function will then be called to get numbers that will be compared to a list of expected numbers. It doesn't matter if there are additional numbers coming from scanForNumbers that aren't in the "target" list. They will just be ignored. Something like this:
val expectedNumbers = listOf("202", "302","999")
scanForNumbers(expectedNumbers)
.observeOn(AndroidSchedulers.mainThread())
.subscribeOn(Schedulers.io())
.subscribe { value -> Log.d(TAG, "Was returned a $value") }
So, the expected numbers (202, 302, and 999) don't exactly match with the numbers that will be emitted (202, 302, and 400). So, a timeout SHOULD occur, but with the built in version of Observable.timer(), it will not time out since at least one item was observed.
Here is kind of what I'd like to have. Anyone know how to code this up in RxJava/RxKotlin?
fun scanForNumbers(targets: List<String>): Observable<FoundNumber> {
val accumulator: Pair<Set<Any>, FoundNumber?> = targets.toSet() to null
return scanNumbers()
.SPECIAL_TIMEOUT_FOR_LIST(5, TimeUnit.SECONDS, List)
.scan(accumulator) { acc, next ->
val (set, previous) = acc
val stringSet:MutableSet<String> = hashSetOf()
set.forEach { stringSet.add(it.toString()) }
val item = if (next.numberId in stringSet) {
next
} else null
(set - next) to item // return set and nullable item
}
.filter { Log.d(TAG, "Filtering on ${it.second}")
it.second != null } // item not null
.take(targets.size.toLong()) // limit to the number of items
.map { it.second } // unwrap the item from the pair
.map { FoundController(it.numberId) } // wrap in your class
}
How do you code, hopefully using RxJava/Kotlin, a means to timeout on a list as mentioned?
I think I get it now, you want the timeout to begin counting from the moment you subscribe, not after you observe items.
If this is what you need, then the takeUntil operator could help you:
return scanNumbers()
.takeUntil(Observable.timer(5, TimeUnit.SECONDS))
.scan(accumulator) { acc, next -> ...
In this case, the timer will begin counting as soon as you subscribe. If the main observable completes before then great, if not, then the timer will complete the main observable anyways.
But takeUntil by itself will not throw an error, it will just complete. If you need it to end with an error, then you could use the following combination:
return scanNumbers()
.takeUntil(
Observable
.error<Void>(new TimeoutError("timeout!"))
.delay(5, TimeUnit.SECONDS, true))
.scan(accumulator) { acc, next -> ...