I have initialized a pipeline parameter, lets say a flag, with a value ('true'). Now inside the pipeline I want to update the flag value based on some condition.
So lets say I have a web activity inside the pipeline, I want to update the value of that flag to 'false' if i dont get the response from the web activity.
Any idea how to do that?
You could try #empty(body('web1'))
So if I understand correctly you want to call a Web activity from your pipeline and if there is no response then set a variable?
In this case, could you not just set a low timeout and use a "Set Variable" activity on the Fail condition? So if there was no response it would fail and set the variable
Related
I have a pipeline in Synapse that reloads data from a number of REST-endpoints. It looks as follows:
What I want to achieve is that the 'CheckAccessTokenValidity' activity checks whether my API-token is still valid. If yes, all is fine, proceed to ForEachDivision activity. If no, refresh tokens, Proceed to ForEachDivision activity.
I thought I had implemented this logic in the screenshot provided. However, this is not the case. My pipeline is now deadlocked because the ForEachDivision apparently expects all three preceding activities to be succesful before running (Which is impossible by design).
How do I implement the logic described above in Azure Synapse?
You can modify your pipeline and accommodate following changes:
Create a variable 'var1' , by default set it to 'False'
After 'CheckAccessTokenValidity' , add set variable activity 'set variable1' pointing to 'var1' and set its value as 'True'
After SetRefreshToken , add another set variable activity 'set variable2' pointing to same 'var1' and set its value as 'True' , so that either of the step 2 or 3 can turn the value of variable to 'True'.
Remove all activities starting ForEach block from this pipeline and cut paste those activities and keep in another new pipeline 'pipeline2'
After 'set variable2' activity, attach an If activity with 'success' and 'Skipped' conditional path . If step 2 gets successful, 'set variable2' would be skipped and it will call the next activity that is 'Step6' . If step 2 fails, and step 3 is called, it will call 'set variable2' and after its success, Step6 will be completed.
In this if activity, write the condition to check the value of var1 . If var1=='True' , then execute the 'pipeline2'
I'm trying to execute Azure Durable Function in ADF.
I have "Get Current Function Status" Activity inside Until activity.
Possible value are pending, completed, running and exception.
I wish to execute Until runtimeStatus Completed or Exception is reached.
It this possible to do following way or should I create flag variable?
I try to set expression, but getting warning "String does not match" :
or(equals(activity('Get Current Function Status').output.runtimeStatus,'Pending'),
equals(activity('Get Current Function Status').output.runtimeStatus,'Running'))
Can you try the below expression, there are few tweaks.
#or(equals(activity('Azure Function').output.runtimeStatus,'Pending'),equals(activity('Azure Function').output.runtimeStatus,'Running'))
I have many files in a blob container. However I wanted to run a Stored procedure only IF a certain file (e.g. SRManifest.csv) exists on the blob Container. I used Get metadata and IF Condition on Data Factory. Can you please help me with the dynamic script for this. I tried this #bool(startswith(
activity('Get Metadata1').output.childitems.ItemName,
'SRManifest.csv')). It doesnt work.
Then I thought, what if i used #greaterOREquals(activity('Get Metadata1').output.LastModified,adddays(utcnow(),-2))But this checks the last modified within 2 days of the Bloob not the file exist. Thank you.
Please see below my diagram
I have understood your requirement differently I think.
I wanted to run a Stored procedure only IF a certain file (e.g. SRManifest.csv) exists on the blob Container
1 Change your metadata activity to look for existence of sentinel file (SRManifest.csv)
2 Follow with an IF activity, use this condition:
3 Put your sp in the True part of the IF activity
If you also needed the file list passed to the sp then you'll need the GetMetadata with childitems option inside the IF-True activity
Based on your diagram, since you are looping over all the blob names already, you can add a Boolean variable to the pipeline and set its default value to false:
Inside the ForEach activity, you only want to attempt to set the variable if the value is still false, and if the blob name is found, set it to true. Since Set Variable cannot be self-referential, do this inside the False branch of an If activity:
This will only attempt to process if the value is false (so the file name has not been found yet), and will do nothing if the value is true. Now set the variable based on your file name:
[NOTE: This value can be hard coded, parameterized, or based on a variable]
When you execute the pipeline, you'll see the Set Variable stops attempting once the value is set to true:
In the main pipeline, after the ForEach activity has completed, you can use the variable to set the condition of your final If activity. If the blob is never found, it will still be false, so put the Stored Procedure activity inside the True branch.
How can I spy into my values when I'm on ADF debug mode ?
I want to build a simple pipeline that digs into a storage account table. For each row, enter the value of the second column, use it to create a URL and call a web service.
I saw the output of the Lookup command but how can I saw, for example, the content of each input() into the foreach activity. Can I used the user properties for debugging reason ?
When debugging, I frequently make use of the 'Set Variable' activity. Viewing the output of a 'Set Variable' activity is spying on the value.
You want to see the input to each iteration of your ForEach. Prepend the inner activity with a Set Variable activity. Dynamic content #string(item()) should be enough.
I have a concat expression defined in the Function Name setting of an Azure Function in my pipeline, where it concatenates the API Query with the current filename that I want to run on this function. When I debug the pipeline, it fails without giving me any feedback. It just says "AzureFunction failed:"
If I manually insert the string, it works fine.
the concat expression is:
#concat('HttpTrigger?filename=', variables('filename'))
I'm new to Azure, any way I can debug this?
try this way:
#concat(variables('FirstName') ,variables('LastName'))
You could use Set Variable Activity with your Azure Function Activity.
In the Variable Activity, set the value of the variable.
Then refer the variable in the Azure Function Activity:
#concat('HttpTriggerJS1?name=',variables('name'))