ADF Pipeline Use variable inside another Set Variable - azure-data-factory

I have two activities inside a for each
Set variable 1 - Value is a string. I want to add this string to the content of Set Variable 2
#item().MessageType (eg :- EVENT, TYPE , STATUS etc. These are the output of a dataflow activity)
Set variable 2 :
#activity('df_dataflow_activity').output.runStatus.output.CacheWaterMarkValue.value[0].EVENT
Here, I want to dynamically change the last part to EVENT,STATUS etc.
#activity('df_dataflow_activity').output.runStatus.output.CacheWaterMarkValue.value[0].{get value of Variable 1)
I tried but getting unable to parse error.
`#activity('df_dataflow_activity').output.runStatus.output.CacheWaterMarkValue.value[0].EVENT`
gives me result of 99999 . The last part EVENT/TYPE/STATUS I want it to be dynamic

Set Variable is used to store the value into a variable. You can use that variable in later activities.
You can include the variable which is used in set variable1 in set variable2.
Set variable1:
Set Variable2:
#concat('sample_',variables('var1'))

Related

kdb - passing in a symbol as parameter for variable name

I've been trying to pass in a symbol from a process outside KDB and am wishing to use this symbol to create a variable where we then load a specific file in case it doesn't exist.
//create some sample data and write to disk
.data.area52.varName1: ([] name:`billy`allen`terry; val:90 100 200);
.data.area52.varName2: 10 #200;
.data.area53.varName1: 9 #100;
.data.area53.varName2: ([] name:`joe`jim`bob; val:10 20 30);
.data.area53.varName3: `random;
`:area52 set .data.area52;
`:area53 set .data.area52;
I am trying to pass in the parameter for reference area52for example which will test a conditional if it exists (do nothing, else) then create the variable loading viaget `:area52
First, I wrote a conditional below which seems to check (however trhe `.areaXX isn't parametrized yet.
// fresh instance of KDB
.data.area52: $[`area52 in key `.data; .data.area52; get `:area52
And then stumbled onto the link here kdb+: use string as variable name
Which gets me most of the way there.
Is there a way to parameterize the beginning of the lambda below passing in some combination of `.area52 ? Much of this variable can be assembled / edited outside KDB and only passing in the
for an example, we can have several hundred `.areaXX that we could pass into KDB as data is changed and refreshed.
{.data.area52:()!(); #[`.data; `area52;:; (get `:area52)]} [];
To safeguard against the situation where the namespace .data doesn't exist you could use the following lambda which takes parameter area52 as an argument.
q){(` sv ``data,x) set ()!(); #[`.data; x; :; get hsym x]}`area52
`.data
Tying it back to the original problem, this lambda will create the variable if it does not exist by reading from the file of the same name:
/ .data does not exist yet
q).data
'.data
[0] .data
^
/ create variable if necessary
q){if[not x in key `.data; (` sv ``data,x)set get hsym x]}`area52
/ check .data
q).data
| ::
area52| ``varName1`varName2!(::;+`name`val!(`billy`allen`terry;90 100 200);20..

Azure-data-Factory Copy data If a certain file exists

I have many files in a blob container. However I wanted to run a Stored procedure only IF a certain file (e.g. SRManifest.csv) exists on the blob Container. I used Get metadata and IF Condition on Data Factory. Can you please help me with the dynamic script for this. I tried this #bool(startswith(
activity('Get Metadata1').output.childitems.ItemName,
'SRManifest.csv')). It doesnt work.
Then I thought, what if i used #greaterOREquals(activity('Get Metadata1').output.LastModified,adddays(utcnow(),-2))But this checks the last modified within 2 days of the Bloob not the file exist. Thank you.
Please see below my diagram
I have understood your requirement differently I think.
I wanted to run a Stored procedure only IF a certain file (e.g. SRManifest.csv) exists on the blob Container
1 Change your metadata activity to look for existence of sentinel file (SRManifest.csv)
2 Follow with an IF activity, use this condition:
3 Put your sp in the True part of the IF activity
If you also needed the file list passed to the sp then you'll need the GetMetadata with childitems option inside the IF-True activity
Based on your diagram, since you are looping over all the blob names already, you can add a Boolean variable to the pipeline and set its default value to false:
Inside the ForEach activity, you only want to attempt to set the variable if the value is still false, and if the blob name is found, set it to true. Since Set Variable cannot be self-referential, do this inside the False branch of an If activity:
This will only attempt to process if the value is false (so the file name has not been found yet), and will do nothing if the value is true. Now set the variable based on your file name:
[NOTE: This value can be hard coded, parameterized, or based on a variable]
When you execute the pipeline, you'll see the Set Variable stops attempting once the value is set to true:
In the main pipeline, after the ForEach activity has completed, you can use the variable to set the condition of your final If activity. If the blob is never found, it will still be false, so put the Stored Procedure activity inside the True branch.

Retrive Variable in a variable group using another variable

I have variable group variable as given below
opp453.Name = "Raj"
pqr325.Name = "Shyam"
I know can I retrieve the variable group as below
$(opp453.Name) or $(pqr325.Name)
If I have another variable
$ptrref=opp453 or
$ptrref=pqr325
Then how can I refer to this variable inside the first variable
$($ptrref.Name)
to retrieve the original value
I think this is impossible to achieve at present because Nested variables are not supported yet in build pipeline/variable group.You can check this similar thread for some more details.
We could also use the same workaround, using InlinePowershell task to determine the value of ptrref, and set the value of opp453.Name/ pqr325.Name according to the value of ptrref.

Enter a query in a default value expression

In Jasperreports I would like to enter a Default Value Expression to a Parameter as a Query string to be able to dynamically provide the user with a default value that is correct, but not force him to choose it.
Is there anyway?
I guess the result should look something like this (even though it doesn't work):
I am using this for a form with a single-value selection method (the user can write which ever number he/she wants but I want the default value to be selected from the database).
Here's how I handle this:
The user selects a value from an input control (in my example, I will call it $P{time_toggle}). Then, I have another parameter ($P{time_constraint}) that takes the user input from $P{time_toggle} and decides what SQL string to inject into the main query based on it. The default value expression for $P{time_constraint} looks like:
$P{time_toggle} == "rolling_quarter" ? "..." : (
$P{time_toggle} == "rolling_30_days" ? "..." : (...
)
)
Then, in my main report query I reference $P{time_constraint}:
SELECT * FROM tblTable WHERE $P!{time_constraint}
To set a default time period, I set the default value expression for $P{time_toggle} to my desired default.

Variable expression contains another variable

I am trying to put some variable inside Variable Expression of another variable.
For example:
$V{sum} = $F{quantity} * ${price}, where sum is simple variable without any calculation
$V{total} = $F{disb} * $V{price}, where total has 'Sum' calculation type.
As a result I receive the wrong amount.
But If I use:
$V{total} = $F{disb} * $F{quantity} * ${price}
the amount is valid.
Is there any reason that variable inside variable expression gives wrong value? Thank you
If you are outputting the $V{total} of your first example in a textfield, then you will need to make sure the Evaluation Time is set correctly per your report. Most likely you will want to set the field evaluation time to "Report".
The evaluation time determines when dynamically calculated variables are actually processed during the report's generation life-cycle.