Is it possible to get value of iteration from a tLoop component if it is in WHILE LOOP configuration? Understand that for FOR LOOP I can get the value using "tLoop_1_CURRENT_VALUE". Can this be achieved if the tLoop component is in WHILE LOOP?
Yes, you can get current iteration value in either case of FOR LOOP or WHILE LOOP by using:
(Integer)globalMap.get("tLoop_1_CURRENT_ITERATION")
Related
I am trying to iterate a linked list in BPF, can I use bpf_loop for this?, if so how?
Tried using bpf_loops, but it required a fixed number to bound the loops, what way I can do that?
These are the comments for bpf_loop.
bpf_loop
For nr_loops, call callback_fn function
with callback_ctx as the context parameter.
The callback_fn should be a static function and
the callback_ctx should be a pointer to the stack.
The flags is used to control certain aspects of the helper.
Currently, the flags must be 0. Currently, nr_loops is
limited to 1 << 23 (~8 million) loops.
long (*callback_fn)(u32 index, void *ctx);
where index is the current index in the loop. The index
is zero-indexed.
If callback_fn returns 0, the helper will continue to the next
loop. If return value is 1, the helper will skip the rest of
the loops and return. Other return values are not used now,
and will be rejected by the verifier.
Returns
The number of loops performed, -EINVAL for invalid flags,
-E2BIG if nr_loops exceeds the maximum number of loops.
You can pass the linked list via the callback_ctx which is shared between iterations.
You can hardcode the number of loops to the max of 1 << 23 (~8 million) (perhaps less, it depends on how the verifier imposes penalties).
If you can advance the linked list you do so and perform whatever logic you want.
If you are done or at the end of the linked list you return 1 to
break out of the loop.
If you expect to need more than 8M iterations, you can at least detect when you are not done by looking at the return value (perhaps running a second loop, I don't know if that is allowed or not)
I have an issue rearding the passing of more than one array parameter. I was able to do a "for each" cycle to execute my array parameter "SPPATH", but unfortunately I can pass only one, here is my code:
{"SPPATH":"#{item()}","SPFOLPATH":"#{pipeline().parameters.SPFOLPATH}","updsppath":"#{pipeline().parameters.updsppath}","Storageacct":"#{pipeline().parameters.Storageacct}","sapath":"#{pipeline().parameters.sapath}","saoppath":"#{pipeline().parameters.saoppath}"}
I want to pass "updsppath" also in the array because my output is on different locations, is it possible to do that, if so, how?
thanks in advance
I have reproduced the above and able to iterate multiple arrays inside ForEach.
For this the length of the all arrays should be same.
Use another array for indexes of these.
For Sample I have two array parameters like below.
I have created another array for index like below.
#range(0,length(pipeline().parameters.arr1))
Give this index_array to ForEach.
Create a res array variable in pipeline and inside ForEach, use append variable with the below dynamic content.
#json(concat('{"arr1":"',pipeline().parameters.arr1[item()],'","SPFOLPATH":"',pipeline().parameters.arr2[item()],'"}'))
After ForEach if you look at variable result (for showing here I have assigned to another variable), it will give you the desired JSON.
Result:
You can use this procedure to generate the desired array of objects and pass it to the logic apps as per your requirement.
I am new new to scala and spark and trying to understand few basic stuff out here.
Spark version used 1.5.
why does value of sum does not get updated in below foreach loop.
var sum=1;
df.select("column1").distinct().foreach(row=>{
sum = sum +1
})
println("SUM = "sum)
--> SUM = 1
I am trying to understand whats scope of variable referred in for-each. What if i need to do some math inside and get the result of it outside the for loop.
My use case to understand above is to get unique values in loop and append it to list of String.
The way you reason about the program is wrong. foreach is executed independently on each executor and modifies its own copy of sum. There is no global shared state here. Just count values directly:
df.select("column1").distinct.count
If you really want to handle this manually you'll need some type of reduce:
df.select("column1").distinct.rdd.map(_ => 1L).reduce(_ + _)
Read the Programming Guide, it has a section devoted to this: Understanding Closures. If you actually need to collect some state, you can use Accumulators (but note that you can't access the value from the executor nodes, only amend it). But try doing without them first: think in terms of available transformations instead of mutating state.
My CoffeeScript is as follows:
for i in [1..3]
i++ if i is 1
console.log i
Expected output is
2
3
Generated output is
2
2
3
The issue is that CoffeeScript keeps a private variable to keep track of the iteration, _i, but if I try to increment that _i++, then the private variable changes to _j and constantly evades me.
So how can I increment the loop manually using CoffeeScript?
You can't manually increment the loop's counting variable. Instead, you need to use continue to skip one or more iterations.
for i in [1..3]
continue if i is 1
console.log i
You should never attempt to access or modify CoffeeScript's generate variables, those are an implementation detail and you cannot rely on them being present.
Trying to understand what this function does:
yr:{ yr:.z.T+1000*x; -1 string .z.Z; while[(.z.T < yr); ZZ,::.z.Z]}
I understand .z.Z gets the datetime, and that execution is from right to left. what is ZZ? What is .z.T?
Essentially, what does the line accomplish?
Here :: means assign the value on the right to the global variable on
the left
ZZ is a global variable
ZZ,::.z.Z is shorthand for ZZ::ZZ,.z.Z
So it appends the latest time to the global variable ZZ.
e.g.
q)f:{ZZ,::2}
q)f[]
q)ZZ
,2
q)f[]
q)ZZ
2 2
.z.T is the time.
.z.T is the current time in the time datatype; the underlying number in times is milliseconds since midnight, so adding 1000*x gives a time x seconds in the future.
-1 string .z.Z prints the current datetime to stdout.
while[(.z.T < yr); ..] loops as long as the current time is less than yr (x seconds in the future).
ZZ,::.z.Z appends the current datetime to a global variable named ZZ.
some additional notes:
the datetime datatype is in general deprecated in favor of timestamp.
the parentheses around the test condition are redundant.
the second : is also redundant, but for more interesting reasons: ,: (like all of the two-argument "writeback" functions (x+:1, y-:2, etc.)) always modifies either a global or a local variable, depending on whether a local of that name exists in the function.
proof:
q)delete from`.;
q){ZZ:(); yr:.z.T+1000*x; -1 string .z.Z; while[(.z.T < yr); ZZ,::.z.Z]}1
2014.04.30T18:26:24.592
q)ZZ
'ZZ
q)
the ZZ being modified inside the while loop in that version of the function was the local variable declared in the first statement, not a global one as one might assume given the presence of a :: in the modification statement.
anyway, if you want to do it like this, it should at the very least be rewritten to yr:{ yr:.z.T+1000*x; -1 string .z.Z; while[.z.T < yr; ZZ,:.z.Z]}.
There is a nicer rewrite of this to make it much more idiomatic. It takes advantage of the overload of \ to iterate. Check out http://code.kx.com/q/ref/adverbs/#converge-iterate
The current function you have, takes the current time (.z.T) and adds x number of seconds (it multiplies by 1000 to make it milliseconds). This becomes your bound, and then as long as the current time is less than that, you append .z.Z (the datetime marker) to a global list ZZ.
The version below will do the same with a couple advantages:you avoid using a global and you write more idiomatic code
f2:{{.z.Z}\[{y; .z.T<x}.z.T+1e3*x; .z.Z]}
While your condition {y; .z.T<x}.z.T+1e3*x is true, your loop will continue to iterate. Note that we make sure to pass in the time limit from outside, so that it is not revaluated it each time, additionally we need a dummy parameter y, as it will try to apply this condition to the result of our iteration, but we don't really want that, since that is .z.Z. Each time it iterates it will evaluate .z.Z returning the datetime. Since you are using \ and not / you will get the entire list generated.
q)f2:{{.z.Z}\[{y; .z.T<x}.z.T+1e3*x; .z.Z]}
q)f2 1
2014.04.30T17:40:23.357 2014.04.30T17:40:23.357 .......
That being said, if you're looking to generate a sequence of datetime values, from now to now+x, this is not the best way to do that, since your .z.Z stamp will reflect the time it takes your machine to actually evaluate that.
CORRECTION:
I did not correctly explain the reason our condition testing lambda is {y; .z.T<x}.z.T+1e3*x. You want to make sure your .z.T+1e3*x is only evaluated at the start, so you do not want your bound to be inside the function. Additionally, if you were to leave out the y param, then {z.T<x}.z.T+1e3*x would immediately evaluate to false, and your iteration would return a type error as it tries to apply your iteration result to 0b at each cycle of the loop. By keeping y and just not using it, we make sure to delay this evaluation and create a lambda that correctly tests our bound.
What This Function is doing:
This function accepts one parameter(should be int) and add those many seconds to current time(lets say it future time). Then it starts a loop from current time to future time and in every iteration, append current date and time to global list(referred by variable ZZ).
How to Call: yr[2]
Explanation:
yr:.z.T+1000*x
.z.T give you current local time.
1000 is equivalent to 1 seconds. So it is adding x seconds to current time and storing it in variable yr. ( : means assignment (=))
ZZ,::.z.Z
:: is used to assign something to global variable. So ZZ is a
global variable which is list (like array).
, is used to append. Above statement is equivalent to ZZ : ZZ , .z.Z
.z.Z gives current local date and time.
while[(.z.T < yr);...]
This is a condition for the loop. So it takes the current time(.z.T)
and check whether it is lesser than future time that was calculated
in first statement.
Loop will end once current time =yr(future time).
Finally you'll have global list which you can access using variable
ZZ.