I would like that first event to arrive will cause some work. Later I would like to throttle down work a little bit. Until now I came with the following code:
var events = Observable.FromEventPattern<...>(...);
var throttled = events.Throttle(TimeSpan.FromSeconds(1));
events.Take(1).Subscribe((x) =>
{
DoWork(x);
throttled.Subscribe((y) => DoWork(y);
});
Is there a more elegant way of expressing it?
Apparently it's quite simple:
var events = Observable.FromEventPattern<...>(...);
var throttled = events.Throttle(TimeSpan.FromSeconds(1));
events.Take(1).Concat(throttled).Subscribe((x) => DoWork(x));
Concat will wait for the first sequence to finish and than move subscription to the second.
Another common way is to use select-many. This allows you to pass data from the 1st sequence to the 2nd sequence, it also allows the two sequences to be of different types:
In query comprehension syntax;
var q = from x in xs
from y in ys
select new {x, y};
-or- you can use the standard linq operators as extension methods (losing access to the x value however)
xs.SelectMany(x=>ys)
.Select(y=>y)
Related
I need to iterate a scala Seq of Row type until a particular condition is met. i dont need to process further post the condition.
I have a seq[Row] r->WrappedArray([1/1/2020,abc,1],[1/2/2020,pqr,1],[1/3/2020,stu,0],[1/4/2020,opq,1],[1/6/2020,lmn,0])
I want to iterate through this collection for r.getInt(2) until i encounter 0. As soon as i encounter 0, i need to break the iteration and collect r.getString(1) till then. I dont need to look into any other data post that.
My output should be: Array(abc,pqr,stu)
I am new to scala programming. This seq was actually a Dataframe. I know how to handle this using Spark dataframes, but due to some restriction put forth by my organization, windows function, createDataFrame function are not available/working in our environment. Hence i have resort to Scala programming to achieve the same.
All I could come up was something like below, but not really working!
breakable{
for(i <- r)
var temp = i.getInt(3)===0
if(temp ==true)
{
val = i.getInt(2)
break()
}
}
Can someone please help me here!
You can use the takeWhile method to grab the elements while it's value is 1
s.takeWhile(_.getInt(2) == 1).map(_.getString(1))
Than will give you
List(abc, pqr)
So you still need to get the first element where the int values 0 which you can do as follows:
s.find(_.getInt(2)== 0).map(_.getString(1)).get
Putting all together (and handle possible nil values):
s.takeWhile(_.getInt(2) == 1).map(_.getString(1)) ++ s.find(_.getInt(2)== 0).map(r => List(r.getString(1))).getOrElse(Nil)
Result:
Seq[String] = List(abc, pqr, stu)
Is it possible to perform a dynamic "where/filter" in a dataframe ?
I am running a "like" operation to remove items that match specific strings
eventsDF.where(
~eventsDF.myColumn.like('FirstString%') &
~eventsDF.myColumn.like('anotherString%')
).count()
However I need to filter based on strings that come from another dataframe/list.
The solution that I was going for (which doesn't really work) involves a function that receives an index
#my_func[0] = "FirstString"
#my_func[1] = "anotherString"
def my_func(n):
return str(item[n])
newDf.where(
~newDf.useragent.like(str(my_func(1))+'%')
).count()
but I'm struggling to make it work by passing a range (mainly because it's a list instead of an integer)
newDf.where(
~newDf.useragent.like(str(my_func([i for i in range(2)])+'%'))
).count()
I don't want to go down the path of using "exec" or "eval" to perform it
str_likes = [~df.column.like(s) for s in strings] then reduce it into one expression reduce(lambda x, y: x & y, str_likes)
It's a little bit ugly but does what you want. You can also do this in a for loop like so
bool_expr = ~df.column.like(strings[0])
for s in strings[1:]:
bool_expr &= ~df.column.like(s)
df.where(bool_expr).count()
New to Scala. I'm iterating a for loop 100 times. 10 times I want condition 'a' to be met and 90 times condition 'b'. However I want the 10 a's to occur at random.
The best way I can think is to create a val of 10 random integers, then loop through 1 to 100 ints.
For example:
val z = List.fill(10)(100).map(scala.util.Random.nextInt)
z: List[Int] = List(71, 5, 2, 9, 26, 96, 69, 26, 92, 4)
Then something like:
for (i <- 1 to 100) {
whenever i == to a number in z: 'Condition a met: do something'
else {
'condition b met: do something else'
}
}
I tried using contains and == and =! but nothing seemed to work. How else can I do this?
Your generation of random numbers could yield duplicates... is that OK? Here's how you can easily generate 10 unique numbers 1-100 (by generating a randomly shuffled sequence of 1-100 and taking first ten):
val r = scala.util.Random.shuffle(1 to 100).toList.take(10)
Now you can simply partition a range 1-100 into those who are contained in your randomly generated list and those who are not:
val (listOfA, listOfB) = (1 to 100).partition(r.contains(_))
Now do whatever you want with those two lists, e.g.:
println(listOfA.mkString(","))
println(listOfB.mkString(","))
Of course, you can always simply go through the list one by one:
(1 to 100).map {
case i if (r.contains(i)) => println("yes: " + i) // or whatever
case i => println("no: " + i)
}
What you consider to be a simple for-loop actually isn't one. It's a for-comprehension and it's a syntax sugar that de-sugares into chained calls of maps, flatMaps and filters. Yes, it can be used in the same way as you would use the classical for-loop, but this is only because List is in fact a monad. Without going into too much details, if you want to do things the idiomatic Scala way (the "functional" way), you should avoid trying to write classical iterative for loops and prefer getting a collection of your data and then mapping over its elements to perform whatever it is that you need. Note that collections have a really rich library behind them which allows you to invoke cool methods such as partition.
EDIT (for completeness):
Also, you should avoid side-effects, or at least push them as far down the road as possible. I'm talking about the second example from my answer. Let's say you really need to log that stuff (you would be using a logger, but println is good enough for this example). Doing it like this is bad. Btw note that you could use foreach instead of map in that case, because you're not collecting results, just performing the side effects.
Good way would be to compute the needed stuff by modifying each element into an appropriate string. So, calculate the needed strings and accumulate them into results:
val results = (1 to 100).map {
case i if (r.contains(i)) => ("yes: " + i) // or whatever
case i => ("no: " + i)
}
// do whatever with results, e.g. print them
Now results contains a list of a hundred "yes x" and "no x" strings, but you didn't do the ugly thing and perform logging as a side effect in the mapping process. Instead, you mapped each element of the collection into a corresponding string (note that original collection remains intact, so if (1 to 100) was stored in some value, it's still there; mapping creates a new collection) and now you can do whatever you want with it, e.g. pass it on to the logger. Yes, at some point you need to do "the ugly side effect thing" and log the stuff, but at least you will have a special part of code for doing that and you will not be mixing it into your mapping logic which checks if number is contained in the random sequence.
(1 to 100).foreach { x =>
if(z.contains(x)) {
// do something
} else {
// do something else
}
}
or you can use a partial function, like so:
(1 to 100).foreach {
case x if(z.contains(x)) => // do something
case _ => // do something else
}
Say I have a map that looks like this
val map = Map("Shoes" -> 1, "heels" -> 2, "sneakers" -> 3, "dress" -> 4, "jeans" -> 5, "boyfriend jeans" -> 6)
And also I have a set or collection that looks like this:
val set = Array(Array("Shoes", "heels", "sneakers"), Array("dress", "maxi dress"), Array("jeans", "boyfriend jeans", "destroyed jeans"))
I would like to perform a filter operation on my map so that only one element in each of my set retains. Expected output should be something like this:
map = Map("Shoes" -> 1, "dress" -> 4 ,"jeans" -> 5)
The purpose of doing this is so that if I have multiple sets that indicate different categories of outfits, my output map doesn't "repeat" itself on technically the same objects.
Any help is appreciated, thanks!
So first get rid of the confusion that your sets are actually arrays. For the rest of the example I will use this definition instead:
val arrays = Array(Array("Shoes", "heels", "sneakers"), Array("dress", "maxi dress"), Array("jeans", "boyfriend jeans", "destroyed jeans"))
So in a sense you have an array of arrays of equivalent objects and want to remove all but one of them?
Well first you have to find which of the elements in an array are actually used as keys in the mep. So we just filter out all elements that are not used as keys:
array.filter(map.keySet)
Now, we have to chose one element. As you said, we just take the first one:
array.filter(map.keySet).head
As your "sets" are actually arrays, this is really the first element in your array that is also used as a key. If you would actually use sets this code would still work as sets actually have a "first element". It is just highly implementations specific and it might not even be deterministic over various executions of the same program. At least for immutable sets it should however be deterministic over several calls to head, i.e., you should always get the same element.
Instead of the first element we are actually interested in all other elements, as we want to remove them from the map:
array.filter(map.keySet).tail
Now, we just have to remove those from the map:
map -- array.filter(map.keySet).tail
And to do it for all arrays:
map -- arrays.flatMap(_.filter(map.keySet).tail)
This works fine as long as the arrays are disjoined. If they are not, we can not take the initial map to filter the array in every step. Instead, we have to use one array to compute a new map, then take the next starting with the result from the last and so on. Luckily, we do not have to do much:
arrays.foldLeft(map){(m,a) => m -- a.filter(m.keySet).tail}
Note: Sets are also functions from elements to Boolean, this is, why this solution works.
This code solves the problem:
var newMap = map
set.foreach { list =>
var remove = false
list.foreach { _key =>
if (remove) {
newMap -= _key
}
if (newMap.contains(_key)) {
remove = true
}
}
}
I'm completely new at Scala. I have taken this as my first Scala
example, please any hints from Scala's Gurus is welcome.
The basic idea is to use groupBy. Something like
map.groupBy{ case (k,v) => g(k) }.
map{ case (_, kvs) => kvs.head }
This is the general way to group similar things (using some function g). Now the question is just how to make the g that you need. One way is
val g = set.zipWithIndex.
flatMap{ case (a, i) => a.map(x => x -> i) }.
toMap
which labels each set with a number, and then forms a map so you can look it up. Maps have an apply function, so you can use it as above.
A slightly simpler version
set.flatMap(_.find(map.contains).map(y => y -> map(y)))
First off, for full disclosure I'm a n00b in RX, but I'm learning daily now...
I need to build an Observable that's going to enable a button (or automatically start an action) as long as a stream of another incoming observable averaged signals is coming in within a certain range. As far as I've learned so far, I could do that by adding a .Where to an averaged Observable in which I can then check that my observed average values created from an event handler are in fact within a given range...
What I need also however is to have these observable values influence the state of my action/button (allow it to be executed) just until its underlying/inner signals overstep the given range. Is such a thing in RX possible as a reversed .TakeUntil(inverse where clause) which I now think could maybe solve my problem, or should I just reuse the original observable and copy it with a negated .Where clause and then use that as another independent observable...if latter, is there some performance loss by reusing almost identical observables multiple times, just changing few their linq queries...how many observables is too much please?
It seems to me that something like this is sufficient.
var source = Observable.Range(0, 10);
var query =
source
.Buffer(4, 1)
.Select(xs => xs.Average())
.Select(x => x > 5.0 && x <= 7.0);
query.ObservableOn(button).Subscribe(enabled => button.Enabled = enabled);
(I've assumed Windows forms.)
This gives me:
False
False
False
False
True
True
False
False
False
False
I can improve it slightly like this:
var query =
source
.Buffer(4, 1)
.Select(xs => xs.Average())
.Select(x => x > 5.0 && x <= 7.0)
.StartWith(true)
.DistinctUntilChanged();
That then gives me:
True
False
True
False
Please let me know if I've missed anything.