if any of my stage fails, I need to pick it up from the same failed stage in my next run instead of starting from the first stage, how to achieve this is ADF? and how to send email whenever an stage fails so that all the users are notified?
if any of my stage fails, I need to pick it up from the same failed stage in my next run instead of starting from the first stage, how to achieve this is ADF?
Failde activity
To Rerun this pipeline from copy activity, click on Rerun from this activity symbol
Click on Ok
Output
It skipped first wait activity which is already runned successfully and started from copy activity which is failed.
how to send email whenever a stage fails so that all the users are notified?
For failure alert for each activity an alert can be created in the Azure Data Factory (ADF) Monitor section under the Alerts and Actions option.
For more understanding you can refer this SO Thread by #UtkarshPal-MT
Or else,
You can create a logic app and then configure it with Azure Data factory through that you cand send mail when activity fails you
For more information you can refer this Article by #JEROEN SMANS
Related
I have an ADF Pipeline with a trigger that runs the pipeline once every day. I would like to find out the status of the pipeline run (latest pipeline run) and log it somewhere (maybe log analytics). How do I do that?
In case if you want to have a log pertaining to only the recent run, then you would have to write a custom logic within your pipeline (like a script activity that would write in a database the status of the pipeline at the end)
sample reference:
https://datasharkx.wordpress.com/2021/08/19/error-logging-and-the-art-of-avoiding-redundant-activities-in-azure-data-factory/
In case if you are comfortable querying a list of logs and filtering out the latest one based on some filter logics, then you can use log analytics or storage blob by enabling diagnostic settings
I am using Azure DevOps pipeline to run my automation test cases.
I am facing an issue that whenever my test run task fails, it doesn't display me the exact error that why the task failed, it just displays "cmd.exe exited with code '1' " for all test lists in the Summary section and the email.
Is there a way I can recieve the information that which test case failed and why in the summary section and email?
I have 5 cmd tasks in my pipeline, each of them has same configuraion, just the test list name is different.
Azure Data Factory pipeline have been working fine for 2 years with Self-hosted Integration runtime (Azure VM 16GB)
In last few weeks pipeline have got very unreliable and data is no longer processed correctly.
Data Factory Activity using "MyAzureIntegrationRuntime" instance is failing with "timeout" error.
Some of activities get successfully completed, but most of them get failed.
ADF Monitor tell that node is "Unavailable". Typical ADF activity is Azure SQL lookup.
There is no event log errors in Virtual Machine. There seems to be enough CPU/RAM to execute IR activities.
Reboot of VM has once helped to recover connectivity and pipelines.
However latest VM reboot restored status to "Running" from “Unavailable”, but many pipeline activities get failed.
Integration Runtime is currently not in High Available Cluster.
There is single VM serving Sandbox, Dev, Test and Prod ADF. It has worked fine for last years except last 2 weeks.
How could I find what is the problem and fixed it?
How could I find what is the problem and fixed it?
For failed activities that are running on a self-hosted IR or a shared IR, the service supports viewing and uploading error logs. To get the error report ID, follow the instructions here, and then enter the report ID to search for related known issues.
On the Monitor page for the service UI, select Pipeline runs.
Under Activity runs, in the Error column, select the highlighted button to display the activity logs, as shown in the following screenshot:
The activity logs are displayed for the failed activity run.
For further assistance, select Send logs.
The Share the self-hosted integration runtime (IR) logs with Microsoft window opens.
Select which logs you want to send.
For a self-hosted IR, you can upload logs that are related to the failed activity or all logs on the self-hosted IR node.
For a shared IR, you can upload only logs that are related to the failed activity.
When the logs are uploaded, keep a record of the Report ID for later use if you need further assistance to solve the issue.
I am trying to run the Test Compelete Scripts on a virtual machine through Azure DevOps pipeline.
When I run the Azure Devops pipeline release. Execution starts in VM. But when I check the logs I see that the user action in the Test scripts are not performed.
Eg: My application is launched, and next steps when I have to select a radio button execution fails and also in
TE log I can see "User session is disconnected and also I can see that it is not able to identify the object)
You typically get this error when the test is run from a scheduled task with the Run whether user is logged on or not option. The error occurs because in this case the test is run in a non-interactive session and cannot interact with the GUI.
To resolve the problem, do the following:
Modify properties of the scheduled task to use the Run only when the
user is logged on option and specify the user account under which
the task will be run.
Make sure that the specified user is logged into the system when the
task is triggered.
Here is the document you can refer to.
Using VSTS online and trying to do gated check. Want to learn if the previous task failed.
How do we write that custom condition?
There already has the condition Only when a previous task has failed (failed()). So you can use this item for your current task, so that only if previous task is failed, current task will be executed.
For more detail for custom condtions, you can refer Specify conditions for running a task.