I'm using Azure function to retrieve an array, it works very well but the problem I can't pass that array into a foreach activity, it does not iterate the array.
the result of the function activity :
the set variable :
#uriComponentToString(replace(uriComponent(activity('Azure Function1').output.Response), '%0D%0A', ''))
the result of set variable :
foreach activity :
when I execute the pipeline the append activity executes just one time :
Now the problem is the foreach activity, it treats the array as one value and it pass it to the append vaiable inside the foreach, how can resolve this proble please .
result variable is already an array. I think you need not convert to array like this while passing on to the foreach loop.'
#array(variables('result'))
Instead, just pass on the variable value directly like
#variables('result')
Thank you GregGalloway. Posting your suggestion as an answer to help other community members.
Use #json(variables('result')) instead of #array(variables('result')) in item field of ForEach activity
Related
My requirement is to extract a string from filenames using a ADF variable, I need to extract the string until the final underscore '_' and the number of underscores vary in every filename as seen in the below example.
abc_xyz_20221221.txt --> abc_xyz
abc_xyz_a1_20221221.txt --> abc_xyz_a1
abc_c_ab_a1_20221221.txt --> abc_c_ab_a1
abc_c_ab_a1_a11_20221221.txt --> abc_c_ab_a1_a11
I tried to get it done using indexof() to get the position of the final underscore but it does not accept negative values, so I got the below logic which works in KQL (Azure Data Explorer) but fails in ADF because there is no CountOf() in this tool. Is there any equivalent function in ADF or can you please suggest me how to achieve the same in ADF?
substring("abc_xyz_20221221.txt", 0,
indexof("abc_xyz_20221221.txt", "_", 0,
strlen("abc_xyz_20221221.txt"),
countof("abc_xyz_20221221.txt", '_')))
You can try like this also using split and join inside ForEach activity.
Array for ForEach activity:
["abc_xyz_20221221.txt","abc_xyz_a1_20221221.txt","abc_c_ab_a1_20221221.txt","abc_c_ab_a1_a11_20221221.txt"]
Append variable inside ForEach:
#join(take(split(item(), '_'),add(length(split(item(), '_')),-1)),'_')
Result in an array variable:
As mentioned by #Joel Cochran, use the below expression in the append variable inside ForEach with lastIndexOf().
#substring(item(),0,lastindexof(item(),'_'))
This is a just a simpler form of what #Rakesh called out above . The only difference being , his implementation is iterating . In my case the file name is stored in a variable named foo
#substring(variables('foo'),0,lastindexof(variables('foo'),'_'))
output
I have my ADF pipeline, Where my final output from set variable activity is something like this {name:test, value:1234},
The input coming to this variable is
{
"variableName": "test",
"value": "test:1234"
}
The expression provided in Set variable Item column is #item().ColumnName. And the ColumnName in my JSon file is something like this "ColumnName":"test:1234"
How can I change it so that I get only 1234. I am only interested in the value coming here.
It looks like you need to split the value by colon which you can do using Azure Data Factory (ADF) expressions and functions: the split function, which splits a string into an array and the last function to get the last item from the array. This works quite neatly in this case:
#last(split(variables('varWorking'), ':'))
Sample results:
Change the variable name to suit your case. You can also use string methods like lastIndexOf to locate the colon, and grab the rest of the string from there. A sample expression would be something like this:
#substring(variables('varWorking'),add(indexof(variables('varWorking'), ':'),1),4)
It's a bit more complicated but may work for you, depending on the requirement.
It seems like you are using it inside of an iterator since you got item but however, I tried with a simple json lookup value
#last(split(activity('Lookup').output.value[0].ColumnName,':'))
I have a pipeline that is retrieving data to an array variable in a ForEach loop. I'd like to next use this array variable and save this array data into a new file (JSON or CSV is preferred).
How would I accomplish this?
In a ForEach loop and We can use the item().<KeyName> to access every item's value.
For exmaple :
If the file is formatted as the following: {"id":"ba3cbfdd-c668-4dab-88c0-ac2f3af183cf"} {"id":"e90f527b-efe0-43ee-b398-e9e6b7f2abb5"}, then we can use item().id to access the value.
I have a set of hardcoded values for which I need to execute ForEach activity. I can create the array in the following manner and specify this as an expression in the Items setting for the ForEach activity
createArray('obj1', 'obj2','obj3')
But I get the following warning:-
Expression of type: 'String' does not match the field: 'items'
Is there any way workaround for this so that ForEach activity will work for this?
I think createArray built-in function could be implemented.
Inside ForEach Avtivity,i just configure one SetVariableActivity.
Result:
I have a strange one I have searched the existing Q&A and haven't found a match.
I have written my functions using parameter validation using the basic format
function FunctioName
{
[CmdletBinding()]
Param(
[parameter(Mandatory)]
[String]$VariableName
)
When I set the parameter to Mandatory as above I get a parameter binding exception indicating a null value was passed. Running the script in debug I can see the function parameter being passed is not null and is a valid string.
When I run the script in the exact same way without the mandatory flag the string is passed into the function and it executes correctly.
Has anyone got any ideas, what could be the issue. This problem is affecting a number of functions in my application interestingly it appears that the affected functions all have only a single parameter functions with multiple parameters do not appear to be affected.
Ok thanks guys for your feedback its much appreciated. BTW i am using powershell 5 .
Further to the issue, looking into it further I found that the variable was being passed to the function as an array of strings, however an empty string value was being appended into the array which I believe was the cause for the issue. This is where it starts to get interesting, I will need to give a bit more background.
The script I am running queries active directory for user attributes meeting specific conditions, those that match I create an array of strings with each value a delimited value of the user,hostname and other attribute properties.
To ensure that I am getting the latest values I use the ASDI GetInfo method,which seems to trigger the odd behavior.
At a high level the functions are
Function GetuserAttr
{
$inscopeusers = New-Object System.Collections.ArrayList
$accountlist = (Get-ADUser -Filter { attribute1 -eq "value"} -Properties attribute1).SamAccountName
foreach ($user in $accountlist)
{
$DN = getDN($user) # basically a funtion I wrote to create ASDI object for user account.
$DN.GetInfo() # this method call appears to cause issues
$attr1 = $DN.Get("Attribute1")
$attr2 = $DN.Get("Attribute2")
$hoststring = "$($user)|$($attr1)|$($attr2)"
$inscopeusers.Add($hoststring) > null
}
return $inscopeusers
}
The string array returned in this function is fed into a number of other functions, one of which is the one that was giving the error that I originally brought up.
The thing is when I use the GetInfo method the array returned by this function contains several null values in the array, when I remove the command the array has no null strings.
Even more strange when I am operating on the array in other functions it appears that the array looses some of its properties when the GetInfo method is used. So for instance I am able to use the foreach loop to iterate through array values but I cannot access an array value by index such as $array[1].
By simply commenting out the GetInfo method call in the function the array returned seems to function normally and you can access array values by index.
I have another function that also uses GetInfo and returns a hash table, when I try to operate on the returned hashtable I cannot access values using a key value such as $hashtable['key'], but I can access them using $hashtable.key. I know this is really weird and can't really think what it could be
Has any one else experienced a similar problem.
You're missing an argument.
Function Test
{
[CmdletBinding()]
Param(
[Parameter(Mandatory=$True)]
[String]
$Variable
)
Write "$Variable"
}