I have created a pipeline (LogPipeline) that logs other pipelines' status to a database. The idea is that every pipeline will call the LogPipeline at the start and at the end by passing pipeline name and pipeline ID along with other parameters like started/ended/failed.
The last parameter is "Reason" where I want to capture the error message of why a pipeline may have failed.
However, in a given pipeline there are multiple activities that could fail. So I want to direct any and all failed activities to my Execute Pipeline activity and pass the error message.
But on the Execute Pipeline when filling out the parameters, I can only reference an activity by its name, e.g. Reason = #activity['Caller Activity'].Error.Message.
However, since multiple activities are calling this Execute Pipeline, is there a way to say
Reason = #activity[activityThatCalledExecutePipeline].Error.Message?
If my understanding is correct,there are multiple activities call the LogPipeline and you want to get those failed activities' names so that you could know the names inside LogPipeline. Per my knowledge,your requirement is not supported in ADF.
I'm not sure why you have to construct such complex scenario,even though you just want to log the specific fail activities and error messages anyway which is common requirement.There are many monitor ways supported by ADF,please follow below links:
1.https://learn.microsoft.com/en-us/azure/data-factory/monitor-using-azure-monitor#alerts
2.https://learn.microsoft.com/en-us/azure/data-factory/monitor-programmatically
I would suggest you getting an idea of Alerts and Monitor in ADF portal:
And you could set the Target Criteria
It includes:
Related
I know one can use REST API to query Pipeline activities but is there anyway to query Pipeline components ( i.e, listing of linked services, sources, sinks, parameters, etc.)
Right now I'm manually recording all the components I see in the pipeline and querying these components would make this listing faster and more accurate
enter image description here
Thanks, Jeannie
I haven't found a way to get all the components of a pipeline directly. If the information you want to obtain is defined in the YAML file, you cannot directly obtain it, you may need to parse the YAML file to obtain it.
If the information you want to get is defined on the UI, such as variables, you can get it through the REST API Definitions - Get. And you can get the resources of the pipeline through Resources - List.
I'm trying to execute simple step from Azure Apps to get the pipeline run statistics, said pipeline calls Logic Apps in the Web activity:
However I'm receiving the error and I don't understand what exactly the step expects as input here:
Could you please assist in resolving above?
You should not use http requests to pass in your Run Id, because Run Id changes every time you run the pipeline.
You should use Create a pipeline run action first, then you can pass the run ID of the output of this operation to the Get a pipeline run action.
You can refer to this question.
There should be a file identifier logic to be added it seems in your case:
You need to take the Output body of JSON file in next block.
I have a pipeline built on Azure data Factory. It has:
a "LookUp" task that has an SQL query that returns a column [CountRecs]. This columns holds a value 0 or more.
an "if" task to check this returned value. I want to fail the pipeline when the value of [CountRecs]>0
Is this possible?
You could probably achieve this by having a Web Activity when your IF Condition is true ([CountRecs]>0) in which the web activity should call the below REST API to cancel the pipeline run by using the pipelinerunID (you can get this value by using dynamic expression - #pipeline().RunId)
Sample Dynamic Expression for Condition: #greater(activity('LookupTableRecordCount').output.firstRow.COUNTRECS, 0)
REST API to Cancel the Pipeline Run: POST https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.DataFactory/factories/{factoryName}/pipelineruns/{runId}/cancel?api-version=2018-06-01
MS Doc related to Rest API: ADF Pipeline Runs - Cancel
One other possible way is to have an invalid URL in your web activity which will fail the Web activity in-turn it will fail the IfCondition activity, which inturn will result in your pipeline to fail.
There is an existing feature request related to the same requirement in ADF user voice forum suggested by other ADF users. I would recommend you please up-vote and/or comment on this feedback which will help to increase the priority of the feature request implementation.
ADF User voice feedback related to this requirement: https://feedback.azure.com/forums/270578-data-factory/suggestions/38143873-a-new-activity-for-cancelling-the-pipeline-executi
Hope this helps.
As a sort-of hack-solution you can create a "Set variable" activity which incurs division by zero if a certain condition is met. I don't like it but it works.
#string(
div(
1
, if(
greater( int(variables('date_diff')), 100 )
, 0
, 1
)
)
)
Is there a way by which we can get the list of skipped tasks from the build?
For example, I have 2 tasks that run conditionally only based on external factors. So how can I see, whether the tasks were skipped or actually ran from Azure DevOps REST API?
I need to trigger another build conditionally based on the above factor.
Any help will be appreciated!
You should look into the Build Timeline REST API. If you issue the following GET request:
GET https://dev.azure.com/{organization}/{project}/_apis/build/builds/{buildId}/timeline
where buildId is the ID of the build you're examining, it returns the Timeline object. It is essentially a collection of TimelineRecord objects, each representing an entry in a build's timeline. You should filter out this collection to leave only those, where "type": "Task" and "result": "skipped".
TL;DR: How do I relate a release to the tests visible in the Tests tab via the API?
I am running a Release Pipeline which executes tests against the website being deployed. The tests generate files (within my test code) and I want to use the DevOps REST API to attach those files to the Test report associated with the Release.
Note: I am running these tests as smoke tests after release as part of the release pipeline so that I can test the deployed website. This would not be possible as part of the build pipeline.
I can get the release information, but I'm having trouble identifying how to find the related test(s) and attach a file.
Additional: The link in this SO post appears to have been changed and no longer points to the intended page, but now points to the overall documentation for the REST API.
Currently, I believe the process should be:
Get release details
Get test run id
Get test case result id
Create test result attachment
The primary problem is getting the test run id and test case result id from the release information.
I guess I'm a bit late with this, but still.
You are right about the process. Getting Test run ID is possible via accessing logs of corresponding task inside the release. Here's the needed endpoint:
https://{organizationName}.vsrm.visualstudio.com/{projectName}/_apis/release/releases/{releaseID}/environments/{stageID}/deployPhases/{deployPhaseID}/tasks/{taskId}/logs
Please note that test run ID and test IDs will only be available after the test run task is finished.
I'd recommend getting release ID, stage ID and deployment phase ID directly from the release by accessing default variables provided by azure devops (e.g. $(Release.ReleaseId), other ones are easily googled) rather than fetching via API and matching by name, cause it takes time and does not return all releases, only 100 first or smth like that.
After this, you need to get TaskID. I found it possible and applicable looking for that by Task Name. Just fetch the release by it's ID via API and look for task which matches needed name using this endpoint:
https://{organizationName}.vsrm.visualstudio.com/{projectName}/_apis/release/releases/{releaseId}?api-version=5.0
Going through all the deploySteps, deployPhases, deployJobs to get the task name is a bit cumbersome, but possible (I used LINQ).
After getting these 4 base IDs, you are now able to get logs of the Run tests task. In received log I have used a regex to extract test run ID. Then, having test run ID, you can grab the list of test run results:
https://{organizationName}.visualstudio.com/{projectName}/_apis/test/runs/{testRunId}/results
To attach some files to a test, you will need this test's AzDo ID. How can you get it? Well, I can think of mapping the tests by their names (you will need to store a list of those beforehand) and then looping through fetched test results and the stored list. After a match is found, get the respective id, and attach anything you want:
https://{organizationName}.visualstudio.com/{projectName}/_apistest/Runs/{testRunId}/Results/{testId}/attachments?api-version=6.1-preview.1