Consider I have two streams of data, is there a way to merge them and apply conditions on data between these two streams? For example
Stream A : A, B, C, D....
Stream B : -, A, -, -....
Composed : (A,-),(B,A),(C,-),(D,-)....
How to get composed stream above using rxjs? I would like to apply conditions on composed streams to raise some notifications. Also would it be possible to use last known non-null data for example see the composed stream below.
Stream A : A, B, C, D....
Stream B : 1, null, 2, null....
Composed : (A,1),(B,1),(C,2),(D,2)....
I've just started playing with reactive streams idea, so please correct me if I've misunderstood the idea of reactive streams.
There are two operators that can serve for your propose.
Zip:
Reference for RxJs: https://github.com/Reactive-Extensions/RxJS/blob/master/doc/api/core/operators/zip.md
CombineLatest:
Reference for RxJs: https://github.com/Reactive-Extensions/RxJS/blob/master/doc/api/core/operators/combinelatest.md
The images explain the differences between both. Now you have merged the observable's you just need to filter, using where, that will filter if one of the values is null.
Unfortunately neither operators can get this behavior that you describe:
Stream A : A, B, C, D, E....
Stream B : 1, null, 2, null, 3....
Composed : (A,1),(B,1),(C,2),(D,2)....
If you use Zip and Where (filtering null values after), the result will be:
Composed: (A,1),(C,2),(E,3)
If you use Where (filtering null values previously) and Zip, the result will be:
Composed: (A,1),(B,2),(C,3)
If you use CombineLatest will depend of the order that the events happens in the Streams, and of course, where you will put the where operator, the result can be different that what you shown, e.g.:
Stream A : A, B, C, D....
Stream B : 1, null, 2, null....
Composed : (A,1),(B,1),(C,1),(C,2),(D,2).... // OR
Composed : (A,1),(B,1),(B,2),(C,2),(D,2)....
Unless you have more specific requirements, I think one of the options that I mentioned is what you are looking for, feel free to add information.
There are several ways to compose observable's, other operators not mentioned are:
distinctUntilChanged, could be added in the final of the composition, using the key selector function to limit for just part of zip or latest value.
switch, used to combine one observable inside another.
Related
I'm trying to create a new node with label C and relationships from a-->c and b-->c, but if and only if the whole pattern a-->c,b-->c does exist.
a and b already exist (merged before the rest of the query).
The below query is a portion of the query I want to write to accomplish this.
However, it creates a random empty node devoid of properties and labels and attaches the relationship to that node instead. This shouldn't be possible and is certainty not what I want. How do I stop that from happening?
merge (a: A {id: 1})
merge (b: B {id:1})
with *
call {with a, b
match (a)-[:is_required]->(dummy:C), (a)-[:is_required]->(b)
with count(*) as cnt
where cnt = 0
merge (temp: Temporary {id: 12948125})
merge (a)-[:is_required]->(temp)
return temp
}
return *
Thanks
I think there are a couple of problems here:
There are restrictions on how you can use variables introduced with WITH in a sub-query. This article helps to explain them https://neo4j.com/developer/kb/conditional-cypher-execution/
I think you may be expecting the WHERE to introduce conditional flow like IF does in other languages. WHERE is a filter (maybe FILTER would have been a better choice of keyword than WHERE). In this case you are filtering out 'cnt's where they are 0, but then never reference cnt again, so the merge (temp: Temporary {id: 12948125}) and merge (a)-[:is_required]->(temp) always get executed. The trouble is, due to the above restrictions on using variables inside sub-queries, the (a) node you are trying to reference doesn't exist, it's not the one in the outer query. Neo4j then just creates an empty node, with no properties or labels and links it to the :Temporary node - this is completely valid and why you are getting empty nodes.
This query should result in what you intend:
merge (a: A {id: 1})
merge (b: B {id:1})
with *
// Check if a is connected to b or :C (can't use a again otherwise we'd overwrite it)
optional match(x:A {id: 1}) where exists((a)-[:is_required]->(:C)) or exists((a)-[:is_required]->(b))
with *, count(x) as cnt
// use a case to 'fool' foreach into creating the extra :Temporary node required if a is not related to b or :C
foreach ( i in case when cnt = 0 then [1] else [] end |
merge (temp: Temporary {id: 12948125})
merge (a)-[:is_required]->(temp)
)
with *
// Fetch the :Temporary node if it was created
optional match (a)-[:is_required]->(t:Temporary)
return *
There are apoc procedures you could use to perform conditional query execution (they are mentioned in the linked article). You could also play around with looking for a path from (a) and check its length, rather than introduce a new MATCH and the variable x then checking for the existance of related nodes.
If anyone is having the same problem, the answer is that the Neo4j browser is display nonexistent nodes. The query executes fine…
Assume events of either type A, B, C or D are being emitted. I want to detect whenever an event of type A is followed by an event of type B. In other words, I want to detect a sequences, for which Esper's EPL provides the -> operator.
However, what I described above is ambiguous, what I want is the following: Whenever a B is detected, I want it to be matched with the most recent A.
I have been playing around with EPL's syntax, but the best I could come up with was that:
select * from pattern [(every a=A) -> b=B]
This, however, matches each B with the oldest A that occured after the last match. Weird...
Help is much appreciated! :P
I use joins a lot for the simple matching. The other option is match-recognize. The join like this.
select * from B unidirectional, A.std:lastevent()
Consider the following example:
List(0,3,2,1) map List("A","B","C","D")
it gives List(A, D, C, B)
Whether this order is maintained always or it can change the order for a different execution.
I want reorder the list by a generator function(index generator).
Whether we can assume the order of map method or is it undecidable?
I'd like to pair two sequences D and A with Reactive Extensions in .NET. The resulting sequence R should pair D and A in a way that whenever new data appears on D, it is paired with the latest value from A as visualized in the following diagram:
D-1--2---3---4---
A---a------b-----
R----2---3---4---
a a b
CombineLatest or Zip does not exactly what I want. Any ideas on how this can be achieved?
Thanks!
You want Observable.MostRecent:
var R = A.Publish(_A => D.SkipUntil(_A).Zip(_A.MostRecent(default(char)), Tuple.Create));
Replace char with whatever the element type of your A observable.
Conceptually, the query above is the same as the following query.
var R = D.SkipUntil(A).Zip(A.MostRecent(default(char)), Tuple.Create));
The problem with this query is that subscribing to R subscribes to A twice. This is undesirable behavior. In the first (better) query above, Publish is used to avoid subscribing to A twice. It takes a mock of A, called _A, that you can subscribe to many times in the lambda passed to Publish, while only subscribing to the real observable A once.
I am new user in Pentaho and maybe my question is very simple. I have two streams with identical columns, e.g. stream S1 has the columns: A, B, C, and stream S2 has columns: A, B, C (same name, same order, same data type). I want to merge or append these two streams into a single stream containing the columns A, B, C. However, when I use merge join (with the option FUL OUTER JOIN) my result is a stream with the columns: A, B, C, A_1, B_1, C_1. It is not what I want. I tried to use the append stream step, but in this case appeared nothing in the preview.
As per your requirement first create two stream.
Here we have taken two streams i.e. "stream1.xls" and "stream2.xls".
Then built the transformation using the "Sorted merge" join
For better understanding please refer the screenshots.