I have my ADF pipeline, Where my final output from set variable activity is something like this {name:test, value:1234},
The input coming to this variable is
{
"variableName": "test",
"value": "test:1234"
}
The expression provided in Set variable Item column is #item().ColumnName. And the ColumnName in my JSon file is something like this "ColumnName":"test:1234"
How can I change it so that I get only 1234. I am only interested in the value coming here.
It looks like you need to split the value by colon which you can do using Azure Data Factory (ADF) expressions and functions: the split function, which splits a string into an array and the last function to get the last item from the array. This works quite neatly in this case:
#last(split(variables('varWorking'), ':'))
Sample results:
Change the variable name to suit your case. You can also use string methods like lastIndexOf to locate the colon, and grab the rest of the string from there. A sample expression would be something like this:
#substring(variables('varWorking'),add(indexof(variables('varWorking'), ':'),1),4)
It's a bit more complicated but may work for you, depending on the requirement.
It seems like you are using it inside of an iterator since you got item but however, I tried with a simple json lookup value
#last(split(activity('Lookup').output.value[0].ColumnName,':'))
Related
My requirement is to extract a string from filenames using a ADF variable, I need to extract the string until the final underscore '_' and the number of underscores vary in every filename as seen in the below example.
abc_xyz_20221221.txt --> abc_xyz
abc_xyz_a1_20221221.txt --> abc_xyz_a1
abc_c_ab_a1_20221221.txt --> abc_c_ab_a1
abc_c_ab_a1_a11_20221221.txt --> abc_c_ab_a1_a11
I tried to get it done using indexof() to get the position of the final underscore but it does not accept negative values, so I got the below logic which works in KQL (Azure Data Explorer) but fails in ADF because there is no CountOf() in this tool. Is there any equivalent function in ADF or can you please suggest me how to achieve the same in ADF?
substring("abc_xyz_20221221.txt", 0,
indexof("abc_xyz_20221221.txt", "_", 0,
strlen("abc_xyz_20221221.txt"),
countof("abc_xyz_20221221.txt", '_')))
You can try like this also using split and join inside ForEach activity.
Array for ForEach activity:
["abc_xyz_20221221.txt","abc_xyz_a1_20221221.txt","abc_c_ab_a1_20221221.txt","abc_c_ab_a1_a11_20221221.txt"]
Append variable inside ForEach:
#join(take(split(item(), '_'),add(length(split(item(), '_')),-1)),'_')
Result in an array variable:
As mentioned by #Joel Cochran, use the below expression in the append variable inside ForEach with lastIndexOf().
#substring(item(),0,lastindexof(item(),'_'))
This is a just a simpler form of what #Rakesh called out above . The only difference being , his implementation is iterating . In my case the file name is stored in a variable named foo
#substring(variables('foo'),0,lastindexof(variables('foo'),'_'))
output
I am running my ADF pipeline with Dataflow and I am getting the output as json as something like this
{"key1":"value1","key2":"[vaq:233,popo:basic5542]"}
However, my actual requirement is to have something like this.
{"key1":"value1","key2":["vaq:233","popo:basic5542"]}
Check the placement of double inverted commas for key "key2".In my Data factory pipeline I am using Derived column action in Dataflow and for key2 I am doing concat ("[",Data1,",popo:basic5542]" ) and Data1 has value vaq:233.
How can I adjust the double inverted comma here?
You could you use the below expression and check whether this meets your requirement.
array(Data1,"popo:basic5542")
Instead of the concat function.
Output :
["pea:P1013","popo:basic5542"]
Considering popo:basic5542 is a static value, you can try as below expression.
concat("[","\"",Data1,"\"",",","\"","popo:basic5542","\"","]")
Or if you are getting popo and basic5542 dynamically, you can try as below.
concat("[","\"",Data1,"\"",",","\"",popo,":",basic5542,"\"","]")
Example:
I am having lookup wherein salesforce query is there. I am using elements (item()) in subsequent activities. Till now i had item().name or item().email but now i have item().NVMStatsSF__Related_Lead__r.FirstName which has (dot) in the field name.
How should i parse it through body tag so that it reads it correctly?
So I have the following data in item()
{
"NVMStatsSF__Related_Lead__c": "00QE000egrtgrAK",
"NVMStatsSF__Agent__r.Name": "ABC",
"NVMStatsSF__Related_Lead__r.Email": "geggegg#gmail.com",
"NVMStatsSF__Related_Lead__r.FirstName": "ABC",
"NVMStatsSF__Related_Lead__r.OwnerId": "0025434535IIAW"
}
now when i use item().NVMStatsSF__Agent__r.Name it will not parse because of (dot) after NVMStatsSF__Agent__r. And it is giving me the following error.
'item().NVMStatsSF__Related_Lead__r.Email' cannot be evaluated because property 'NVMStatsSF__Related_Lead__r' doesn't exist, available properties are 'NVMStatsSF__Related_Lead__c, NVMStatsSF__Agent__r.Name, NVMStatsSF__Related_Lead__r.Email, NVMStatsSF__Related_Lead__r.FirstName, NVMStatsSF__Related_Lead__r.OwnerId'.",
"failureType": "UserError",
"target": "WebActivityToAddPerson"
this is because ADF uses '.' for object reading.
Could you find a way to rename the field name which contains '.'?
Seems like you need a built-in function to get the value of an object according to the key. Like getValue(item(), 'key.nestkey'). But unfortunately, seems there isn't such a function. You may need handle your key first.
Finally, it worked. I was being silly.
Instead of taking the value from the child table with the help of (dot) operator I just used subquery. Silly see.
And it worked.
I know you can set a Date field explicitly like so:
"date_brewed":{
"__type":"Date",
"iso":"2009-10-15T00:00:00.000Z"
}
But is there anyway to explicitly set the column type of 'Number' using REST? For instance, I'd like to set the column 'batch_size' to a Number instead of a string but when POST'ing via rest it keeps getting created as a string type column.
Meh, this was more of a Perl issue than a Parse issue.
What I had to do to tell Perl to treat the number like an actual number was to add a zero to the value. :/
I've research this topic extensibly and I'm asking as a last resort before assuming that there is no wildcard for what I want to do.
I need to pull up all the text input elements from the document and add it to an array. However, I only want to add the input elements that have an id.
I know you can use the \S* wildcard when using an id selector such as $(#\S*), however I can't use this because I need to filter the results by text type only as well, so I searching by attribute.
I currently have this:
values_inputs = $("input[type='text'][id^='a']");
This works how I want it to but it brings back only the text input elements that start with an 'a'. I want to get all the text input elements with an 'id' of anything.
I can't use:
values_inputs = $("input[type='text'][id^='']"); //or
values_inputs = $("input[type='text'][id^='*']"); //or
values_inputs = $("input[type='text'][id^='\\S*']"); //or
values_inputs = $("input[type='text'][id^=\\S*]");
//I either get no values returned or a syntax error for these
I guess I'm just looking for the equivalent of * in SQL for JQuery attribute selectors.
Is there no such thing, or am I just approaching this problem the wrong way?
Actually, it's quite simple:
var values_inputs = $("input[type=text][id]");
Your logic is a bit ambiguous. I believe you don't want elements with any id, but rather elements where id does not equal an empty string. Use this.
values_inputs = $("input[type='text']")
.filter(function() {
return this.id != '';
});
Try changing your selector to:
$("input[type='text'][id]")
I figured out another way to use wild cards very simply. This helped me a lot so I thought I'd share it.
You can use attribute wildcards in the selectors in the following way to emulate the use of '*'. Let's say you have dynamically generated form in which elements are created with the same naming convention except for dynamically changing digits representing the index:
id='part_x_name' //where x represents a digit
If you want to retrieve only the text input ones that have certain parts of the id name and element type you can do the following:
var inputs = $("input[type='text'][id^='part_'][id$='_name']");
and voila, it will retrieve all the text input elements that have "part_" in the beginning of the id string and "_name" at the end of the string. If you have something like
id='part_x_name_y' // again x and y representing digits
you could do:
var inputs = $("input[type='text'][id^='part_'][id*='_name_']"); //the *= operator means that it will retrieve this part of the string from anywhere where it appears in the string.
Depending on what the names of other id's are it may start to get a little trickier if other element id's have similar naming conventions in your document. You may have to get a little more creative in specifying your wildcards. In most common cases this will be enough to get what you need.