Is there a way to merge two Flux such that the result only contains unique elements? I can block on the output and then convert it to a set, but is there a way that does not depend on blocking?
Source (Kotlin)
val set1 = Flux.just(1, 2, 3, 4, 5)
val set2 = Flux.just(2, 4, 6, 8, 10)
val mergedSet = set1.mergeWith(set2)
println(mergedSet.collectList().block())
Output
[1, 2, 3, 4, 5, 2, 4, 6, 8, 10]
Desired Output (order is not important)
[1, 2, 3, 4, 5, 6, 8, 10]
You can use the Flux's merge method and then apply distinct() to it.
Flux.merge (Flux.just(1, 2, 3, 4, 5), Flux.just(2, 4, 6, 8, 10)).distinct();
This way you get a flux which produces only distinct values.
Related
Postgres has the function array_positions:
array_positions(ARRAY[1, 4, 3, 1, 3, 4, 2, 1], 1);
-> {1,4,8}
Which checks for equality of the second argument. I can't find anywhere if it has a function for comparison. E.g. (element in array > 1) which would return {2, 3, 5, 6, 7} in this example.
Does postgres have an efficient way of doing this?
SELECT position
FROM UNNEST(ARRAY[1, 4, 3, 1, 3, 4, 2, 1]) WITH ORDINALITY a(element, position)
WHERE element > 1
I have dataframe like this
+---------+--------------------+----------------------------+
| Name| rem1| quota |
+---------+--------------------+----------------------------+
|Customer_3|[258, 259, 260, 2...|[1, 2, 3, 4, 5, 6, 7,..500]|
|Customer_4|[18, 19, 20, 27, ...|[1, 2, 3, 4, 5, 6, 7,..500]|
|Customer_5|[16, 17, 51, 52, ...|[1, 2, 3, 4, 5, 6, 7,..500]|
|Customer_6|[6, 7, 8, 9, 10, ...|[1, 2, 3, 4, 5, 6, 7,..500]|
|Customer_7|[0, 30, 31, 32, 3...|[1, 2, 3, 4, 5, 6, 7,..500]|
I would like to remove list value in rem1 from quota and create as one new column. I have tried.
val dfleft = dfpci_remove2.withColumn("left",$"quota".filter($"rem1"))
<console>:123: error: value filter is not a member of org.apache.spark.sql.ColumnName
Please advise.
You can use a filter in a column in such way, you can write an udf as below
val filterList = udf((a: Seq[Int], b: Seq[Int]) => a diff b)
df.withColumn("left", filterList($"rem1", $"quota") )
This should give you the expected result.
Hope this helps!
Custom fashion is:
obs1 = [1, 3, 5, 7, 9], obs2 = [2, 4, 6, 8, 10] -> mergedObs = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
I was thinking about obs1.zipWith(obs2) and my bifunction was (a, b) -> Observable.just(a, b) and then it's not trivial for me to flatten Observable<Observable<Integer>>.
That looks like an ordered merge: merge so that the smallest is picked from the sources when they all have items ready:
Flowables.orderedMerge():
Given a fixed number of input sources (which can be self-comparable or given a Comparator) merges them into a single stream by repeatedly picking the smallest one from each source until all of them completes.
Flowables.orderedMerge(Flowable.just(1, 3, 5), Flowable.just(2, 4, 6))
.test()
.assertResult(1, 2, 3, 4, 5, 6);
Edit
If the sources are guaranteed to be the same length, you can also zip them into a structure and then flatten that:
Observable.zip(source1, source2, (a, b) -> Arrays.asList(a, b))
.flatMapIterable(list -> list)
;
I have written the program below for generating random unique numbers for several number of times by invoking the function, but it seems like I'm getting the same pattern with minimal changes.
func generateRandom(withinNumber: Int) {
var i:Int = 0
var elements = Set<Int>()
while i != withinNumber {
let num:Int = Int(arc4random())%withinNumber + 1
if elements.count <= withinNumber && elements.contains(num) == false {
elements.insert(num)
}
else {
i = i-1
}
i=i+1
}
print(elements)
elements.removeAll()
}
generateRandom(withinNumber: 10)
How does I make my program effectively run to generate several random unique numbers.
Please let me know it would be very helpful for me.
You are storing your numbers in a Set and sets are not ordered, so the order the elements are shown by print is unrelated to the order in which they were added to the set.
Rather the elements of a set are stored in some manner which enables fast checking for .contains(), and this is one reason you seeing similar sequences.
If you wish to preserve order of insertion use a collection which does this, i.e. an array. Changing to an array in your code produced the following results from 9 calls:
[8, 9, 7, 10, 5, 6, 2, 3, 1, 4]
[4, 9, 10, 3, 6, 2, 1, 7, 8, 5]
[8, 3, 5, 1, 6, 4, 9, 10, 7, 2]
[5, 7, 2, 9, 8, 1, 6, 10, 3, 4]
[2, 3, 7, 6, 9, 1, 8, 10, 5, 4]
[9, 10, 2, 4, 6, 8, 5, 7, 1, 3]
[9, 10, 2, 5, 4, 7, 3, 8, 1, 6]
[1, 6, 4, 5, 8, 2, 3, 9, 7, 10]
[6, 10, 5, 3, 2, 8, 1, 9, 7, 4]
You are also generating 10 random numbers in the range 1 to 10 and avoiding duplicates, so the results is always going to be the numbers 1 to 10 in some order.
To generate a random number in a given range do not use %, instead use the provided arc4random_uniform() which will give better a better distribution.
The function mention in your title arc4random_stir() is available in Swift.
BTW (somewhat opinion based): It is better to write !e (! being the boolean not operator) rather than e == false, and never ever write e == true which is the long form of e!
BTW (SO etiquette): Don't link to your code (or paste in images of it). Reduce to a small example which demonstrates the issue (not required in your case) and insert directly in the question. Keep tags minimal and appropriate. These edits were done for you this time by myself and others, you will know for next time.
HTH
Say I have IObservable and I want an observable that ignores the repeating numbers of the original one, how can I do that ? I tried the following
I have tried GroupBy() but it is a hot observable, which is not going to work. And all I need to compare with is with the previous one.
You want to use DistinctUntilChanged.
// yields 1, 4, 4, 2, 2, 2, 3, 4, 4, 3
IObservable<int> a = ...;
// yields 1, 4, 2, 3, 4, 3
IObservable<int> b = obs.DistinctUntilChanged();