Is there a way to run (queue) a specific azure pipeline from the command line or via http web-hook or an API ? I would like to automatically trigger a pipeline without the need to change git or whatever.
You can use the AzureDevOps Rest API
POST:
https://account.visualstudio.com/project/_apis/build/builds?api-version=4.1
Body:
{
"definition": {
"id": number
}
}
Refer my answer here
Related
I'm looking for a way to automate/script Azure DevOps project creation. I know that I can use VSTeam PowerShell module to create a project, but it does not provide a way to programmatically set project settings (the ones on the screenshot below). Is there a way to do that with PowerShell or AZ CLI? In particular I'm looking for a way to control/change these two settings:
Limit job authorization scope to current project for non-release pipelines
Limit job authorization scope to referenced Azure DevOps repositories
You can try to use Rest API:
How to use PowerShell with rest API: Modify Azure Devops Test Case Parameters Through REST API
Rest API Methods: General Settings - Get; General Settings - Update
To create a project you will need to send a POST to https://dev.azure.com/<devopsOrgName>/_apis/projects?api-version=5.0-preview.3
with a body like the below. To get the "template type" value used in the body, you will need to set a GET to https://dev.azure.com/<devopsOrgName>/_apis/process/processes?api-version=5.0-preview.1
` {
"description": "Tailspin Toys",
"name": "Tailspin_TOYS",
"capabilities": {
"versioncontrol": {
"sourceControlType": "Git"
},
"processTemplate": {
"templateTypeId": "24268e03-7eed-4ac0-a178-700881565b99"
}
}
}`
To change specific settings, that you can't find in the documentation, I would suggest using the developer tools in your browser (f12). Watch the network tab and you should see the requests going when you change your setting.
In this instance, I can see the below request when I try these settings
URL: https://dev.azure.com/<devopsOrgName>/_apis/Contribution/HierarchyQuery
Type: POST
Body:
{ "contributionIds":["ms.vss-build-web.pipelines-general-settings-data-provider"],"dataProviderContext":{"properties":{"enforceJobAuthScopeForReleases":"false","sourcePage":{"url":"https://dev.azure.com/<devopsOrgName>/<projectName>/_settings/settings","routeId":"ms.vss-admin-web.project-admin-hub-route","routeValues":{"project":"<projectName>","adminPivot":"settings","controller":"ContributedPage","action":"Execute","serviceHost":"495d404e-cdeb-496a-8496-fccf9df3d9fa (<devopsOrgName>)"}}}} }
Azure DevOps REST isn't the most documented, but when used with your browser develop tools, I haven't found something I couldn't set.
We use Synapse git Integration to deploy artifacts such as linked services generated by a Data Warehouse automation tool (JSON files)
It is different then deploying ARM template in ADF.
We created one Azure Key Vault (AKV) per environment so we do have an Azure Key Vault LinkedService in each environment and the linked services has the same name. But each AKV as his own URL so we need to change the URL in the deployed linked services during the CI/CD process.
I read this https://learn.microsoft.com/en-us/azure/synapse-analytics/cicd/continuous-integration-deployment#use-custom-parameters-of-the-workspace-template
I think I need to create a template to change "Microsoft.Synapse/workspaces/linkedServices"
But I didn't find any example on how to modify the KV url parameters.
Here is the linked services I want to modify,https://myKeyVaultDev.vault.azure.net as to be changed when deploying
{
"name": "myKeyVault",
"properties": {
"type": "AzureKeyVault",
"typeProperties": {
"baseUrl": "https://myKeyVaultDev.vault.azure.net"
}
}
}
Not much familiar with the ci/cd and azure devOps yet, but still I need to do it...
I have done this using Azure Devops. When you create the Release pipeline within Azure Devops, one of the options is to "override parameters". at this point you can specify the name of the keyvault and the corresponding value. The corresponding value is configured in a pipeline variable set - which itself can come from the same keyvault.
You don't need to create the template. Synapse already does that and stores it in the publish branch (“workspace_publish”). If you look in that branch you will see the template along with the available parameters that you can override.
More info is available here:
https://www.drware.com/how-to-use-ci-cd-integration-to-automate-the-deploy-of-a-synapse-workspace-to-multiple-environments/
https://techcommunity.microsoft.com/t5/data-architecture-blog/ci-cd-in-azure-synapse-analytics-part-1/ba-p/1964172
From the Azure Key Vault side of things, I believe you're right - you have change the Linked Services section within the template to point to the correct Key Vault base URL.
Azure Key Vault linked service
I don't know if you still are looking for the solution.
In order to parametrize linked service property and specially AKV reference, I think you should modify the template-parameters-definition.json, and add the following section:
"Microsoft.Synapse/workspaces/linkedServices":
{ "*":
{ "properties":
{ "typeProperties":
{ "baseUrl": "|:-connectionString:secureString" }
}
}
}
This will create a parameter for each linked service. The next step is to overrideParameters on SynapseWorkspaceDeployment task on Azure Devops.
I am trying to create an azure ci/cd pipeline for my azure data factory in which I have used databricks notebook. Pipeline got created successfully with the ARM template for ADF but I am not able to see any override parameter for databricks workspace URL, that's why i got the same databricks URL in my dev and prod environment.
Can anyone help me to set databricks workspace URL for Dev and prod dynamically?
For anyone facing the same challenge, I just added this code to my arm-template-parameters-definition.json:
"AzureDatabricks": {
"properties": {
"typeProperties": {
"domain": "=",
"existingClusterId":"=",
"accessToken": {
"secretName": "="
}
}
}
}
It will make the parameters explicit to be overriden.
If you are using the default template provided by Microsoft, I inserted the code just under the "ODBC" section factories/linkedServices.
you need to create a token in qa and prod and use that in ur ci cd pipeline..tokens are unique all
over and automatically linked to respective service
I have a parent pipeline which publish pipeline artifacts.
And the child pipeline which has a parent pipeline added as a resource pipeline and consumes the parent's artifacts.
I would like to programatically (by REST API) create a run of multistage child pipeline and provide the parent pipeline resource.
How to craft a request body for pipelines' run endpoint?
According to ms docs PipelineResourceParameters contains only version? Are there any examples how to use it?
Is it a resource name and buildnumber?
https://learn.microsoft.com/en-us/rest/api/azure/devops/pipelines/runs/run%20pipeline?view=azure-devops-rest-6.1#pipelineresourceparameters
Where can I find any examples of requests bodies?
The pipeline resource is set in advance in the pipelines. By default, the pipeline will select last successful run as the resource. We can choose the pipeline resource version when we run the pipeline.
Here is my request body sample:
{
"resources":{
"pipelines":{
"Parent":{
"version":"20201225.1"
}
}
}
}
If you want to find the sample of Rest API, in addition to referring to the examples in the official documentation, you can also check the developer tool (F12) in the browser. For example, run a pipeline manually and check the rest api:
Is it possible to trigger an azure DevOps pipeline via ADF through web activity?
If not then how to trigger via ADF?
It is not possible to trigger an Azure Devops release pipeline from ADF.
But you can use a logic app to trigger the same and in turn call the logic app through ADF via web activity.
As of this time, however, there isn't an existing function that support Azure Data Factory to trigger Azure DevOps pipeline.
But there is a REST API Runs - Run Pipeline that can queue a pipeline outside the Azure DevOps which may help you.
POST https://dev.azure.com/{organization}/{project}/_apis/pipelines/{pipelineId}/runs?api-version=6.0-preview.1
Here is an example of the request body:
{
"stagesToSkip": [],
"resources": {
"repositories": {
"self": {
"refName": "refs/heads/main"
}
}
},
"variables": {}
}
For data factory, you only can add the pipeline trigger:
When the pipeline runs, the web active will run and call the azure DevOps pipeline. You can use REST API like Jane said. We can not trigger the azure DevOps pipeline directly. There isn't a active can achieve that.
HTH.