I am building a pipeline in ADF and I must save in the database the name of the pipeline and the activities that are being used, how can I save this information in the database?
You would get a better answer if you could be more specific on when/where you want to do that, i.e the usage scenario. Without that, my best-guess answer is that you can use PowerShell to obtain that information.
Specifically, you can use the cmdlet Get-AzDataFactoryV2Pipeline, as specified here: https://learn.microsoft.com/en-us/powershell/module/az.datafactory/get-azdatafactoryv2pipeline?view=azps-5.8.0
You can use a python script to parse these details and then load it into the database this can all be done using Azure DevOps pipelines.
Related
I know one can use REST API to query Pipeline activities but is there anyway to query Pipeline components ( i.e, listing of linked services, sources, sinks, parameters, etc.)
Right now I'm manually recording all the components I see in the pipeline and querying these components would make this listing faster and more accurate
enter image description here
Thanks, Jeannie
I haven't found a way to get all the components of a pipeline directly. If the information you want to obtain is defined in the YAML file, you cannot directly obtain it, you may need to parse the YAML file to obtain it.
If the information you want to get is defined on the UI, such as variables, you can get it through the REST API Definitions - Get. And you can get the resources of the pipeline through Resources - List.
I'm trying to execute simple step from Azure Apps to get the pipeline run statistics, said pipeline calls Logic Apps in the Web activity:
However I'm receiving the error and I don't understand what exactly the step expects as input here:
Could you please assist in resolving above?
You should not use http requests to pass in your Run Id, because Run Id changes every time you run the pipeline.
You should use Create a pipeline run action first, then you can pass the run ID of the output of this operation to the Get a pipeline run action.
You can refer to this question.
There should be a file identifier logic to be added it seems in your case:
You need to take the Output body of JSON file in next block.
I have created a pipeline (LogPipeline) that logs other pipelines' status to a database. The idea is that every pipeline will call the LogPipeline at the start and at the end by passing pipeline name and pipeline ID along with other parameters like started/ended/failed.
The last parameter is "Reason" where I want to capture the error message of why a pipeline may have failed.
However, in a given pipeline there are multiple activities that could fail. So I want to direct any and all failed activities to my Execute Pipeline activity and pass the error message.
But on the Execute Pipeline when filling out the parameters, I can only reference an activity by its name, e.g. Reason = #activity['Caller Activity'].Error.Message.
However, since multiple activities are calling this Execute Pipeline, is there a way to say
Reason = #activity[activityThatCalledExecutePipeline].Error.Message?
If my understanding is correct,there are multiple activities call the LogPipeline and you want to get those failed activities' names so that you could know the names inside LogPipeline. Per my knowledge,your requirement is not supported in ADF.
I'm not sure why you have to construct such complex scenario,even though you just want to log the specific fail activities and error messages anyway which is common requirement.There are many monitor ways supported by ADF,please follow below links:
1.https://learn.microsoft.com/en-us/azure/data-factory/monitor-using-azure-monitor#alerts
2.https://learn.microsoft.com/en-us/azure/data-factory/monitor-programmatically
I would suggest you getting an idea of Alerts and Monitor in ADF portal:
And you could set the Target Criteria
It includes:
I've a tool(web application) which creates work-items in azure devops. (skipping the unnecessary details), just like how we assign any user a particular work-item from the Assigned To dropdown in azure devops, I too have a dropdown which when user enters any name/alias, a list of users starts showing based on the input.
The api which I was using in the backend was
https://abcorganization.vsaex.visualstudio.com/_apis/UserEntitlements?top=10&filter=name+eq+%27Tejas
Here filter=name+eq+%27Tejas in the query parameter helps to query the api and used to give set of users whose name starts with Tejas. It can be email alias too.
But for some reason, that doesn't work anymore. My guess, they've deprecated that API version
So in my search to find the alternative/answer, I came across the following documentation:
(https://learn.microsoft.com/en-us/rest/api/azure/devops/graph/users/get?view=azure-devops-rest-5.1) in which the API given is: https://vssps.dev.azure.com/abcorganization/_apis/graph/users/{userDescriptor}?api-version=5.1-preview.1
Here the userDescriptor is some sort of unique key of AAD related to a particular user. (which I certainly can't use to fulfill my purpose).
The other thing which I've tried is the below query parameters but it still didn't worked out
https://vssps.dev.azure.com/abcorganization/_apis/graph/users?subjectTypes={subjectTypes}&continuationToken={continuationToken}&api-version=5.1-preview.1
So is there anyway/api which can fullfil my purpose or is it that I'm using this new API in a wrong way or something?
Any help would be much appreciated
I believe it should be $filter in the query. You are also missing the closing quote.
(See docs for more details)
https://abcorganization.vsaex.visualstudio.com/_apis/UserEntitlements?top=10&$filter=name+eq+'Tejas'
Use this. I tested it and worked for me:
https://vsaex.dev.azure.com/{organization}/_apis/userentitlements?$filter=name+eq+%27{name}%27&api-version=6.0-preview.3
Replace {organization} and {name} as needed
I am trying to collect some metrics on releases in Azure Devops via a Powershell script.
I have very limited dev experience and am new to PowerShell. And this is the first time I have worked with an API. So far I have been able to authenticate, return a list of releases, loop through them and export the data to a file. Now I need to filter the releases based on a substring of the release name. For the record I have been doing my initial testing in Postman to make sure my syntax and results are correct. Then I migrated working syntax to Powershell.
https://{{organization}}.vsrm.visualstudio.com/{{project}}/_apis/release/releases?api-version=5.0
If I add the id filter as shown here:
https://{{organization}}.vsrm.visualstudio.com/{{project}}/_apis/release/releases?api-version=5.0&releaseId=34567
I get this result:
"id": 34567,
"name": "Test-Release-MyService",
But if use the same filter format for Release Name,
https://{{organization}}.vsrm.visualstudio.com/{{project}}/_apis/release/releases?api-version=5.0&releaseName="Test-Release-MyService"
I get back 50 results of which none match that criteria, whether I wrap the string in quotes or not. Furthermore, what i really want to do is to have the response only include records where the releaseName contains "XYZ".
So the question: Is there a filter operator for "contains" so I only get back records where the release name contains the "XYZ" substring?
Thanks in advance for your advice.
Every parameter you used in Azure DevOps REST API needs to be consistent with the description in the document, Azure DevOps REST API does not support custom parameters. For your question, the parameter searchText is used to filter the the searching result with the release name containing the keyword. I have tested with POSTMAN to call the api, it works fine. In addition, the value of parameter searchText is not case-sensitive. Filter release name
If you want to do more filter, in fact you can use powershell or other client library to deserialize the json response to an object, and do some convert or filter. Following documents may be helpful for you:
https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/convertfrom-json?view=powershell-6
https://devblogs.microsoft.com/scripting/playing-with-json-and-powershell/