I am attempting to make 3 web services calls (e.g.: getPhoneNumber, getFirstName, getLastName) and collect the answers into a common object Person. Any of the web services calls can return a Maybe.empty().
When attempting to zip the response together, rxjava2 skips over the zip operation and terminate normally (without having aggregated my answer).
For a simplified example, see below:
#Test
public void maybeZipEmptyTest() throws Exception {
Maybe<Integer> a = Maybe.just(1);
Maybe<Integer> b = Maybe.just(2);
Maybe<Integer> empty = Maybe.empty();
TestObserver<String> observer = Maybe.zip(a, b, empty, (x, y, e) -> {
String output = "test: a "+x+" b "+y+" empty "+e;
return output;
})
.doOnSuccess(output -> {
System.out.println(output);
})
.test();
observer.assertNoErrors();
}
How can we collect empty values within a zip operation instead of having the zip operation skipped/ignored? If this is the wrong pattern to solve this problem, how would you recommend solving this?
For most use cases, leveraging the defaultIfEmpty method is the right way to go.
For representing something which is ultimately optional (doesn't even use default), I used Java 8 Optional type to represent.
For example
#Test
public void maybeZipEmptyTest() throws Exception {
Maybe<Optional<Integer>> a = Maybe.just(Optional.of(1));
Maybe<Optional<Integer>> b = Maybe.just(Optional.of(2));
Maybe<Optional<Integer>> empty = Maybe.just(Optional.empty());
TestObserver<String> observer = Maybe.zip(a, b, empty, (x, y, e) -> {
String output = "test: a "+toStringOrEmpty(x)+" b "+toStringOrEmpty(y)+" empty "+toStringOrEmpty(e);
return output;
})
.doOnSuccess(output -> {
System.out.println(output);
})
.test();
observer.assertNoErrors();
}
private String toStringOrEmpty(Optional<Integer> value){
if(value.isPresent()){
return value.get().toString();
}
else {
return "";
}
}
Related
I'm learning reactive programming with webflux, and for that I'm migrating some code.
For example I'm trying to migrate this method:
public Set<Vaccine> getAll(Set<Long> vaccinesIds) throws EntityNotFoundException {
if (null == vaccinesIds) {
return null;
}
Set<Long> vaccinesToFind = new HashSet<>(vaccinesIds);
vaccinesToFind.remove(null);
Set<Vaccine> vaccines = new HashSet<>();
vaccineRepository.findByIdIn(vaccinesToFind).forEach(vaccines::add);
if (vaccines.size() != vaccinesToFind.size()) {
LOG.warn("Could not find vaccines with ids: " + vaccinesToFind.removeAll(vaccines.stream().map(Vaccine::getId).collect(Collectors.toSet())));
throw new EntityNotFoundException(VACCINE_ERROR_NOT_FOUND);
}
return vaccines;
}
To summarize the code, if the respository returns all the vaccines that are requested should return the result, if not should return an error.
For that, I thought in something like this, but is not working:
public Flux<Vaccine> getAll(Set<Long> vaccinesIds) {
if (null == vaccinesIds) {
return Flux.empty();
}
Set<Long> vaccinesToFind = new HashSet<>(vaccinesIds);
Flux<Vaccine> byIdIn = vaccineRepository.findByIdIn(vaccinesToFind);
Mono<Long> filter = vaccineRepository.findByIdIn(vaccinesToFind).count().filter(x -> x.equals(Long.valueOf(vaccinesToFind.size())));
return filter.flatMapMany(asd -> vaccineRepository.findByIdIn(vaccinesToFind)
).switchIfEmpty(Flux.error((new EntityNotFoundException(VACCINE_ERROR_NOT_FOUND))));
}
What am I doing wrong?
My first doubt is why the filter is a Mono of Long if it has a equals method in the end. My problem is about evaluating the filter in order to return the list or the error.
First of all, you are querying the same result vaccineRepository.findByIdIn(vaccinesToFind) multiple times. The same data is queried, transferred and deserialized multiple times. This is a sign that something is wrong here.
Let's assume the result set fits into the memory. Then the idea would be to transform flux into a usual collection and to decide whether to emit an error or not:
return vaccineRepository.findByIdIn(vaccinesIds)
.collectList()
.flatMapMany(result -> {
if(result.size() == vaccinesIds.size()) return Flux.fromIterable(result);
else return Flux.error(new EntityNotFoundException(VACCINE_ERROR_NOT_FOUND));
});
In the case the result is to huge for the main memory, you could do count in the db by the first query and in the positive case query the results. The solution is similar to your code:
return vaccineRepository.countByIdIn(vaccinesIds)
.filter(count -> count == vaccinesIds.size())
.flatMapMany($ -> vaccineRepository.findByIdIn(vaccinesIds))
.switchIfEmpty(Mono.error(new EntityNotFoundException(VACCINE_ERROR_NOT_FOUND)));
The result of filter is Mono<Long> because filter just takes the elements from the upstream and tests against the given predicate. If the predicate returns false, the item is filtered out and the Mono is empty. To keep all results of a the test you could use map and the type would be Mono<Boolean>.
I am new to RxJava2.
I am trying to get a list of Transaction object both from cache and from server.
I want to compare the server value to cache value and if the server value is the same, then ignore it.
I was able to do it easily using .scan() because we can return null and when null is returned from the .scan() the value got ignored(filtered).
RxJava 1
private Observable<List<Transaction>> getTransactionsFromCacheAndServer() {
return Observable.concat(
getTransactionsFromCache(),
getTransactionsFromServer()
)
.scan((p1, p2) -> {
if (p1 == null && p2 != null) {
return p2;
} else if (p1 != null && !isListSame(p1, p2)) {
return p2;
} else {
return null;
}
});
}
With RxJava 2, since I cannot return null anymore, things are not easy.
RxJava 2
private Observable<List<Transaction>> getTransactionsFromCacheAndServer() {
return Observable.concat(
getTransactionsFromCache(),
getTransactionsFromServer()
)
.map(FilterObject::new)
.scan((filterObject1, filterObject2) -> {
List<Transaction> p1 = (List<Transaction>)filterObject1.value;
List<Transaction> p2 = (List<Transaction>)filterObject2.value;
if (p1.size() == 0 && p2.size() > 0) {
return filterObject2;
} else if (!isListSame(p1, p2)) {
return filterObject2;
} else {
filterObject2.filter = true;
return filterObject2;
}
})
.filter(filterObject -> !filterObject.filter)
.map(filterObject -> (List<Transaction>)filterObject.value);
}
Where FilterObject is:
public class FilterObject {
public Object value;
public boolean filter;
public FilterObject(Object value) {
this.value = value;
}
}
Even though I can achieve the same thing using above method, it seems very ugly. Also I had to include two maps which might not be so performance friendly.
Is there a simple/clean way to achieve what I want?
I don't think there is a generic solution to this problem, since an empty list and a list that needs to be filtered (which happens to be empty in all cases) are two different things (the output of the scan) and needs to be handled differently.
However, in your particular case you never emit an empty list, except maybe for the first output.
(I am using String instead Transaction, shouldn't matter)
private Observable<List<String>> getTransactionsFromCacheAndServer() {
return Observable.concat(
getTransactionsFromCache(),
getTransactionsFromServer()
)
.filter(list -> !list.isEmpty())
// If you prefer a consistent empty list over the first
// empty list emission getting filtered
.startWith((List<String>) Collections.EMPTY_LIST)
// Newly emitted value cannot be empty, it only depends only on the comparison
.distinctUntilChanged(this::isListSame);
}
That's the closest I could get with as few operators as possible. Hope it solves your problem.
Based on andras' answer, I modified little bit to achieve what I want.
private Observable<List<String>> getTransactionsFromCacheAndServer() {
return Observable.concat(
getTransactionsFromCache(),
getTransactionsFromServer()
)
.filter(list -> !list.isEmpty())
.distinctUntilChanged(this::isListSame)
.switchIfEmpty(Observable.just(new ArrayList<>()));
}
Andreas' answer will always receive an empty list and then a real data.
My solution above will receive:
1. Data from cache (and then data from server if different)
2. Empty list if both cache and server returns Empty list.
I've created a custom Magic command with the intention of generating a spark query programatically. Here's the relevant part of my class that implements the MagicCommandFunctionality:
MagicCommandOutcomeItem execute(MagicCommandExecutionParam magicCommandExecutionParam) {
// get the string that was entered:
String input = magicCommandExecutionParam.command.substring(MAGIC.length())
// use the input to generate a query
String generatedQuery = Interpreter.interpret(input)
MIMEContainer result = Text(generatedQuery);
return new MagicCommandOutput(MagicCommandOutcomeItem.Status.OK, result.getData().toString());
}
This works splendidly. It returns the command that I generated. (As text)
My question is -- how do I coerce the notebook into evaluating that value in the cell? My guess is that a SimpleEvaluationObject and TryResult are involved, but I can't find any examples of their use
Rather than creating the MagicCommandOutput I probably want the Kernel to create one for me. I see that the KernelMagicCommand has an execute method that would do that. Anyone have any ideas?
Okay, I found one way to do it. Here's my solution:
You can ask the current kernelManager for the kernel you're interested in,
then call PythonEntryPoint.evaluate. It seems to do the job!
#Override
MagicCommandOutcomeItem execute(MagicCommandExecutionParam magicCommandExecutionParam) {
String input = magicCommandExecutionParam.command.substring(MAGIC.length() + 1)
// this is the Scala code I want to evaluate:
String codeToExecute = <your code here>
KernelFunctionality kernel = KernelManager.get()
PythonEntryPoint pep = kernel.getPythonEntryPoint(SCALA_KERNEL)
pep.evaluate(codeToExecute)
pep.getShellMsg()
List<Message> messages = new ArrayList<>()
//until there are messages on iopub channel available collect them into response
while (true) {
String iopubMsg = pep.getIopubMsg()
if (iopubMsg == "null") break
try {
Message msg = parseMessage(iopubMsg) //(I didn't show this part)
messages.add(msg)
String commId = (String) msg.getContent().get("comm_id")
if (commId != null) {
kernel.addCommIdManagerMapping(commId, SCALA_KERNEL)
}
} catch (IOException e) {
log.error("There was an error: ${e.getMessage()}")
return new MagicKernelResponse(MagicCommandOutcomeItem.Status.ERROR, messages)
}
}
return new MagicKernelResponse(MagicCommandOutcomeItem.Status.OK, messages)
}
I have found a few articles like this one:
http://devtoolshed.com/using-stored-procedures-entity-framework-scalar-return-values
Yet when I take the step to create a function import for a int32 scalar, this is what gets generated:
public ObjectResult<Nullable<global::System.Int32>> MyStoredProcedure(Nullable<global::System.Int32> orderId)
{
ObjectParameter orderIdParameter;
if (orderId.HasValue)
{
orderIdParameter = new ObjectParameter("OrderId", orderId);
}
else
{
orderIdParameter = new ObjectParameter("OrderId", typeof(global::System.Int32));
}
return base.ExecuteFunction<Nullable<global::System.Int32>>("MyStoredProcedure", orderIdParameter);
}
I am able to call the procedure with this, but am not able to get to the underlying scalar:
ObjectResult<int?> result = myEntities.MyProcedure(orderId);
In the code examples I have seen, I should get context.MyProcedure().SingleOrDefault().
Try this:
int? result = myEntities.MyProcedure(orderId).FirstOrDefault();
Here is code of very simple expression evaluator using IronRuby
public class BasicRubyExpressionEvaluator
{
ScriptEngine engine;
ScriptScope scope;
public Exception LastException
{
get; set;
}
private static readonly Dictionary<string, ScriptSource> parserCache = new Dictionary<string, ScriptSource>();
public BasicRubyExpressionEvaluator()
{
engine = Ruby.CreateEngine();
scope = engine.CreateScope();
}
public object Evaluate(string expression, DataRow context)
{
ScriptSource source;
parserCache.TryGetValue(expression, out source);
if (source == null)
{
source = engine.CreateScriptSourceFromString(expression, SourceCodeKind.SingleStatement);
parserCache.Add(expression, source);
}
var result = source.Execute(scope);
return result;
}
public void SetVariable(string variableName, object value)
{
scope.SetVariable(variableName, value);
}
}
and here is problem.
var evaluator = new BasicRubyExpressionEvaluator();
evaluator.SetVariable("a", 10);
evaluator.SetVariable("b", 1 );
evaluator.Evaluate("a+b+2", null);
vs
var evaluator = new BasicRubyExpressionEvaluator();
evaluator.Evaluate("10+1+2", null);
First Is 25 times slower than second. Any suggestions? String.Replace is not a solution for me.
I do not think the performance you are seeing is due to variable setting; the first execution of IronRuby in a program is always going to be slower than the second, regardless of what you're doing, since most of the compiler isn't loaded in until code is actually run (for startup performance reasons). Please try that example again, maybe running each version of your code in a loop, and you'll see the performance is roughly equivalent; the variable-version does have some overhead of method-dispatch to get the variables, but that should be negligible if you run it enough.
Also, in your hosting code, how come you are holding onto ScriptScopes in a dictionary? I would hold onto CompiledCode (result of engine.CreateScriptSourceFromString(...).Compile()) instead -- as that will help a lot more in repeat runs.
you can of course first build the string something like
evaluator.Evaluate(string.format("a={0}; b={1}; a + b + 2", 10, 1))
Or you can make it a method
if instead of your script you return a method then you should be able to use it like a regular C# Func object.
var script = #"
def self.addition(a, b)
a + b + 2
end
"
engine.ExecuteScript(script);
var = func = scope.GetVariable<Func<object,object,object>>("addition");
func(10,1)
This is probably not a working snippet but it shows the general idea.