I'm building a function that requires both the previous and the current value of a stream.
I managed to work that around, but I was wondering if that is some way to retrieve the second to last value of it.
You can use rxdart pairwise:
RangeStream(1, 4)
.pairwise()
.listen(print); // prints [1, 2], [2, 3], [3, 4]
You will always get a List containing the current emitted value and the last one as well. Just be aware this will only emit after there are 2 items to be emitted, so if you need the first value ASAP, this might not be the best solution for you.
A simple way to solve this is to just save the emitted value to an external variable, this usually isn't much recommended as Streams are supposed to be encapsulated from external code, but for many cases this would be simpler.
If you really need the first value you can duplicate your stream and consume the first value only once, then let pairwise() do it's magic, here's one solution using the async and rxdart packages:
Stream<int> stream = Stream.fromIterable([0, 1, 2, 3, 4, 5]);
List<Stream<int>> splitted = StreamSplitter.splitFrom(stream);
splitted[0].take(1).listen(print); // prints 0 immediately
splitted[1].pairwise().listen(print); // prints [0, 1], [1, 2], [2, 3], [3, 4], [4, 5]
Of course you can also merge them and get all of it in one stream.
Related
I'm trying to migrate a stream processing project to Spark Structured Streaming.
Within this project, there is a correlation logic like this:
A dict with init values
{
1: [2, 3],
4: [5, 6],
}
Then a new input comes, saying that 2 and 5 should be correlated together.
We know the key for 2 is 1, and for 5 is 4, so we merge all values in entry 4 to entry 1.
Finally, the dict becomes { 1: [2, 3, 4, 5, 6] }.
Currently, we use a distributed database to save the dict. But with Spark, I want to retire the database and only rely on Spark's memory state.
According to this tutorial, I created a mapping function:
def mappingFunction(
key: String,
values: Iterator[Input],
state: GroupState[State]
): Iterator[...] = {
}
But it seems I can only access the state of the specific key (first param in this func).
My questions are:
If I receive <2, 5>, how can I update the group state of 1 and delete the group state of 4?
Can we rely on Spark for maintaining a complicated state like this? Or is a distributed global state store always needed for this case?
Thanks!
I want to do multiple where query without effect data. I want to get data that include at least 1 data per array. Pseudo code
data =[1,3]
array1 = [1,2]
array2 = [3,4]
if(data.IsIntersect(array1) and data.IsIntersect(array2))
IsIntersect checks are there a intersection beetween arrays
I did so far
queryBuilder.andWhere(
'properties.id IN (:...sizeIds) AND properties.id IN (:...colorIds)',
{ sizeIds: [1, 2], colorIds: [3, 4] },
);
It returns empty because firstly checks properties for 'sizeIds' then it checks for 'colorIds'. For example
properties includes 1,3
check for sizeIds, returns 1
check for colorIds, return empty
How can I do that with typeORM?
How can properties.id be 1 and 3? And if it is, how could 1 or 3 be in both? You're asking for the impossible.
I assume you mean to ask for when properties.id is 1 or 3, because if it is [1,3] then you should use the postgres array syntax {1,3} & the ANY keyword (some variation on this: Check if value exists in Postgres array).
tldr, I think all you need is brackets and OR instead of AND:
queryBuilder.andWhere(
'(properties.id IN (:...sizeIds) OR properties.id IN (:...colorIds))',
{ sizeIds: [1, 2], colorIds: [3, 4] },
);
If properties.id is in fact an array, then please add the entity definition to your question. If you want to merge the rows where properties.id is in the list you will need a GROUP BY (https://orkhan.gitbook.io/typeorm/docs/select-query-builder).
This question already has an answer here:
For loop for Dictionary don't follow it's order in Swift [duplicate]
(1 answer)
Closed 5 years ago.
the question that I want to ask, from which directions swift starts to read dictionaries or arrays
when I put some codes like this
let interestingNumber = [
"Square": [1,4,9,16,25],
"Prime": [2,3,5,7,11,13],
"Fiboannci": [1,1,2,3,5,8],
"asd":[2,3,4,5],
"zxc":[3,4,5]
]
for (key,values) in interestingNumber{
print(values)
}
the output is
[1, 4, 9, 16, 25]
[2, 3, 5, 7, 11, 13]
[1, 1, 2, 3, 5, 8]
[3, 4, 5]
[2, 3, 4, 5]
this is not the exact order, so do you know why swift does this ? and it sometimes makes it different too!
I guessed may be it does it in string order, then I tried but I think it is not too, so why swift does do that ?
Just like for NSDictionary, Swift dictionaries are not ordered by key or value. The order will always be unspecified.
If you need the keys to be sorted, your only option is to have an array of ordered keys, and use the for loop over this array.
From apple documentation (https://developer.apple.com/library/content/documentation/Swift/Conceptual/Swift_Programming_Language/CollectionTypes.html#//apple_ref/doc/uid/TP40014097-CH8-ID105)
Dictionaries
A dictionary stores associations between keys of the same type and
values of the same type in a collection with no defined ordering. Each
value is associated with a unique key, which acts as an identifier for
that value within the dictionary. Unlike items in an array, items in a
dictionary do not have a specified order. You use a dictionary when
you need to look up values based on their identifier, in much the same
way that a real-world dictionary is used to look up the definition for
a particular word.
This question already has answers here:
NumPy selecting specific column index per row by using a list of indexes
(7 answers)
Closed 2 years ago.
Is there a better way to get the "output_array" from the "input_array" and "select_id" ?
Can we get rid of range( input_array.shape[0] ) ?
>>> input_array = numpy.array( [ [3,14], [12, 5], [75, 50] ] )
>>> select_id = [0, 1, 1]
>>> print input_array
[[ 3 14]
[12 5]
[75 50]]
>>> output_array = input_array[ range( input_array.shape[0] ), select_id ]
>>> print output_array
[ 3 5 50]
You can choose from given array using numpy.choose which constructs an array from an index array (in your case select_id) and a set of arrays (in your case input_array) to choose from. However you may first need to transpose input_array to match dimensions. The following shows a small example:
In [101]: input_array
Out[101]:
array([[ 3, 14],
[12, 5],
[75, 50]])
In [102]: input_array.shape
Out[102]: (3, 2)
In [103]: select_id
Out[103]: [0, 1, 1]
In [104]: output_array = np.choose(select_id, input_array.T)
In [105]: output_array
Out[105]: array([ 3, 5, 50])
(because I can't post this as a comment on the accepted answer)
Note that numpy.choose only works if you have 32 or fewer choices (in this case, the dimension of your array along which you're indexing must be of size 32 or smaller). Additionally, the documentation for numpy.choose says
To reduce the chance of misinterpretation, even though the following "abuse" is nominally supported, choices should neither be, nor be thought of as, a single array, i.e., the outermost sequence-like container should be either a list or a tuple.
The OP asks:
Is there a better way to get the output_array from the input_array and select_id?
I would say, the way you originally suggested seems the best out of those presented here. It is easy to understand, scales to large arrays, and is efficient.
Can we get rid of range(input_array.shape[0])?
Yes, as shown by other answers, but the accepted one doesn't work in general so well as what the OP already suggests doing.
I think enumerate is handy.
[input_array[enum, item] for enum, item in enumerate(select_id)]
How about:
[input_array[x,y] for x,y in zip(range(len(input_array[:,0])),select_id)]
How can I access the first value from
object t = (object) wsf.LinEst(y.ToArray(), x.ToArray(), false, true);
The output is
object[1..5, 1..2]}
[1, 1]: 0.17134831460674155
[1, 2]: 0.0
[2, 1]: 0.019612696690686725
[2, 2]: -2146826246
[3, 1]: 0.95020429009193053
[3, 2]: 0.82746057986828336
[4, 1]: 76.328205128205056
[4, 2]: 4.0
[5, 1]: 52.261235955056179
[5, 2]: 2.7387640449438226
I need to get only [1,1] 's value i.e. 0.17134831460674155
How to get that.
The linEst return an object only.
I am using C#3.0
Thanks
Thanks
Well, it looks like it's something you can enumerate which returns doubles - although it may not actually implement IEnumerable<double>. Try this (with a using System.Linq; directive at the top of your code):
IEnumerable t = (IEnumerable) wsf.LinEst(y.ToArray(), x.ToArray(), false, true);
double first = t.Cast<double>().First();
However, that won't work if actually it's returning something like an int[][]. Could you tell us more about what it's really returning? Yes, it's declared to just return object, but it's clearly returning something more than that. If you know it will always return an object[,] with 1-based indexes, you could cast to that:
object[,] t = (object[,]) wsf.LinEst(y.ToArray(), x.ToArray(), false, true);
double first = (double) t[1, 1];
If you don't know the lower bounds of the array, you can ask for them programmatically of course... but the LINQ way will probably be simpler.
You can force a cast to the appropriate type.
Instead of doing:
object t = (object) wsf.linEst(etc.)
try
int[,] t = (int[,]) wsf.linEst(etc.)
This is because the linEst function is not simply returning an instance of the Object class.
The output you've included in your question seems to imply that the debugger is able to determine the runtime type of your object t so try and use that information to do the proper cast.
Either that, or wsf.linEst isn't just returning an object and you are adding that (object) text which is forcing the return value to be cast to fit into the object t variable