Azure Devops - How to call one pipeline from another - azure-devops

Could someone help me with how to call one pipeline from another pipeline in Azure DevOps?
I have to run a pipeline and this should trigger another pipeline in different project.

You can try to use Trigger Azure DevOps Pipeline task to trigger another pipeline in different projects.
Depending on your choice in the task it will trigger a build or a release pipeline.
To be able to use the extension an Azure DevOps API endpoint needs to be created.
For the service connection to work as it should you need to configure the following parameters:
Organization Url: The URL of the organization.
(https://dev.azure.com/[organization])
Release API Url: The URL of the Azure DevOps Release API
(https://vsrm.dev.azure.com/[organization])
Personal Access Token: The personal access token.
How you can create a personal access token can be found here: Use personal access tokens to authenticate.
Make sure the personal access token has the following rights:
Triggering a Release: Release – Read, write & execute – Build Read &
Execute
Triggering a Build: Read & Execute

I think resources for Azure Pipelines is what you looking for.
Add a resource in the pipeline that shall be called from another one and name the source pipeline:
# Explicitly set none for repository trigger
trigger:
- none
resources:
pipelines:
- pipeline: myappbuild # Name of the pipeline resource
source: myapp-build-pipeline # Name of the triggering pipeline
trigger:
branches:
- master

You can use API for triggering build. Here is the example that I use to trigger another build pipeline.
- powershell: |
# Write your PowerShell commands here.
Write-Host " ***** Start Script ***** "
$body = '
{
"parameters": "{\"parameter1\": \"value1\"}",
"definition": {"id": "1234"},
"sourceBranch": "git/branch",
"templateParameters": {"templateparameter": "paramvalue"}
}
'
$bodyJson=$body | ConvertFrom-Json
Write-Output $bodyJson
$bodyString=$bodyJson | ConvertTo-Json -Depth 100
Write-Output $bodyString
$user="$(user)"
$token="$(token)"
$base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f $user,$token)))
$Uri = "https://tfs.com:8443/Organization/_apis/build/builds?api-version=6.1-preview.6"
$buildresponse = Invoke-RestMethod -Method Post -UseDefaultCredentials -ContentType application/json -Uri $Uri -Body $bodyString -Headers #{Authorization=("Basic {0}" -f $base64AuthInfo)}
write-host $buildresponse
$buildID = $buildresponse.id
write-host $buildID
Write-Output "Build ID is $buildID... Sleep for 5 seconds.."
Start-Sleep -Seconds 5
$buildInfo = ( Invoke-RestMethod -Method Get -UseDefaultCredentials -Uri "https://tfs.com:8443/Organization/_apis/build/builds/${buildID}?api-version=6.1-preview.6" -Headers #{Authorization=("Basic {0}" -f $base64AuthInfo)} )
while($buildInfo.status -eq "inProgress" -or $buildInfo.status -eq "notStarted") # keep checking till build completed
{
Write-Output "Build is $($buildInfo.status)... Sleep for 5 seconds.."
Start-Sleep -Seconds 5 # Start sleep for 5 seconds
$buildInfo = ( Invoke-RestMethod -Method Get -UseDefaultCredentials -Uri "https://tfs.com:8443/Organization/_apis/build/builds/${buildID}?api-version=6.1-preview.6" -Headers #{Authorization=("Basic {0}" -f $base64AuthInfo)} )
}
Write-Output "Build Status : $($buildInfo.status)" # print build status
Write-Output "Build Result : $($buildInfo.result)" # print build result
displayName: 'Trigger Another Build Pipeline'

You can install az devops extension in your pipeline agent and then you can call az pipeline CLI commands" to manage other build or release pipelines. Next, you can call az pipeline CLI commands from your main pipeline and for this you can use AzureCLI task or Bash task.

Here is an implementation I use with the following:
PowerShell CmdLet "Invoke-RestMethod" https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/invoke-restmethod?view=powershell-5.1
Azure DevOps REST API https://learn.microsoft.com/en-us/rest/api/azure/devops/build/builds/queue?view=azure-devops-rest-5.0
The $(System.AccessToken) variable from DevOps. This has the
advantage that a PAT is not necessary. More Infomation at https://learn.microsoft.com/en-us/azure/devops/pipelines/build/variables?view=azure-devops&tabs=yaml#systemaccesstoken
Unfortunately the target pipeline is not in another project
I created a stage
######### stage_call_other_pipelines ###################
#########################################################
- stage: stage_call_other_pipelines
displayName: "call other pipelines"
jobs:
#XYZ deployment
- job: job_call_XYZ_deployment
displayName: "execute XYZ deployment"
steps:
- checkout: none
- task: PowerShell#2
displayName: "via REST API"
env:
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
inputs:
targetType: 'inline'
script: |
#url
$url = 'https://dev.azure.com/XYZOrganization/XYZProject/_apis/build/builds?api-version=5.0'
#header
$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("Content-Type", "application/json")
$headers.Add("Authorization","Bearer $env:SYSTEM_ACCESSTOKEN");
#body
$body = " {
`n `"definition`": {
`n `"id`": 134
`n },
`n `"templateParameters`": {
`n `"ParameterA`": `"ParameterValueA`",
`n `"ParameterB`": `"ParameterValueB`"
`n }
`n }"
#call rest api
$response = Invoke-RestMethod $url -Method 'POST' -Headers $headers -Body $body
#output
$response | ConvertTo-Json
failOnStderr: true
pwsh: true
The token is passed to the agent as an environment variable. The pipeline and its parameters are defined in the body.
Additional
On the pipeline that is to be executed, permissions must be adjusted.
Go to the desired pipeline, click in the right upper corner on the
menu button and select "Manage security"
A form will apear. Choose the Build Service service principal and set "Queue builds" on "Allow"

Since the OP didn't specify how they wanted to accomplish this, I'll share how I'm doing this without YAML pipelines. I use the Edit pipeline option then the meatball menu to select triggers and I can specify which triggering build with branch filters.

Related

How to use my token to az pipelines run command?

I have created a new pipeline to run another pipeline with pwsh as below,
steps:
- pwsh: |
az pipelines run --branch "$(Build.SourceBranch)" `
--name "another-pipeline" `
--organization "$(System.CollectionUri)" `
--project "$(System.TeamProject)"
displayName: 'Enqueue another pipeline'
env:
AZURE_DEVOPS_EXT_PAT: $(System.AccessToken)
After ran this step, it will start another-pipeline but it is not me who starts the pipeline because it is using $(System.AccessToken).
Let's say anyone in my team can run the new pipeline.
How to get their access token and use it or any others solution?
The variable: $(System.AccessToken) is a predefined variable. It represents the role of the build service account.
So when you run the Az Pipelines run command with this PAT, it will show that the pipeline is running by build service account.
How to get my access token and use it?
You need to manually create Personal Access Token. Refer to this doc: Create PAT
Then you can add an variable in Pipeline and set it as secret.
For example:
You can use the secret variable in the pipeline.
steps:
- pwsh: |
az pipelines run --branch "$(Build.SourceBranch)" `
--name "another-pipeline" `
--organization "$(System.CollectionUri)" `
--project "$(System.TeamProject)"
displayName: 'Enqueue another pipeline'
env:
AZURE_DEVOPS_EXT_PAT: $(PAT)
Then the pipeline will running by your account.
Update:
When you use the Az Pipeline run to run the pipeline, it will run the pipeline with the user represented by PAT. It has no option to set the request user in the Azure CLI so you need to create PAT for all users.
Based on your requirement, you can change to use the Rest API to run the pipeline: Builds - Queue
When you use Rest API, you can set the requestedBy field in the body to set the user alias to run the pipeline.
For example:
$body = '
{
"definition": {
"id": number
} ,
"requestedFor": {
"id": "userid"
}
}
'
$bodyJson=$body | ConvertFrom-Json
Write-Output $bodyJson
$bodyString=$bodyJson | ConvertTo-Json -Depth 100
Write-Output $bodyString
$user="name"
$token="PAT"
$base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f $user,$token)))
$Uri = "https://account.visualstudio.com/project/_apis/build/builds?api-version=4.1"
$buildresponse = Invoke-RestMethod -Method Post -UseDefaultCredentials -ContentType application/json -Uri $Uri -Body $bodyString -Headers #{Authorization=("Basic {0}" -f $base64AuthInfo)}
write-host $buildresponse
Or you can use the Trigger Build task: Queue build in extension: Trigger Build Task
For example:
- task: TriggerBuild#4
displayName: 'Trigger a new build of 797'
inputs:
buildDefinition: name or id
queueBuildForUserThatTriggeredBuild: true
password: PAT
You can set the queueBuildForUserThatTriggeredBuild to true. Then the triggered pipeline will run by the same user as current pipeline run

Configuring Pipeline dependencies in Azure Pipelines

I have 2 Azure Pipelines, deploy and test. As their names imply one is used for deploying a product and the other is used for testing. When a developer wants to run their own tests on the existing deployment they trigger test. When a deployment is required they trigger deploy. If the test pipeline is in execution when the deploy pipeline is triggered I want the deploy to wait till the test has finished executing.
Is there a way to configure this dependency within the pipeline.yaml themselves, or a workaround to achieve the mentioned requirement
Is there a way to configure this dependency within the pipeline.yaml themselves, or a workaround to achieve the mentioned requirement
Here are two methods could meet your requirement:
1.You could add the Environment in your Yaml Pipeline. Add you could add Invoke Rest API check in the environment. Rest API: Latest - Get
In Yaml Pipeline, you could call this environment.
Example:
stages:
- stage: deploy
jobs:
- deployment: DeployWeb
displayName: deploy Web App
pool:
vmImage: 'Ubuntu-latest'
environment: 'EnvironmentName'
strategy:
runOnce:
deploy:
steps:
...
When you run the pipeline, the environment will check the latest build status of the test Pipeline. If the build has completed , it will run the deploy pipeline.
Result:
2.You could directly add a Powershell task in the Deploy task to check the status of the Test Pipeline.
$token = "PAT"
$url="https://dev.azure.com/{OrganizationName}/{ProjectName}/_apis/build/definitions/{DefinitionID}?includeLatestBuilds=true&api-version=5.1"
$token = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$($token)"))
$response = Invoke-RestMethod -Uri $url -Headers #{Authorization = "Basic $token"} -Method Get -ContentType application/json
$buildid = $response.latestBuild.id
echo $buildid
$success = $false
do{
try{
$Buildurl2 = "https://dev.azure.com/{OrganizationName}/{ProjectName}/_apis/build/builds/$($buildid)?api-version=5.0"
$Buildinfo2 = Invoke-RestMethod -Method Get -ContentType application/json -Uri $Buildurl2 -Headers #{Authorization=("Basic {0}" -f $base64AuthInfo)}
$BuildStatus= $Buildinfo2.status
$result = $Buildinfo2.result
echo $result
echo $BuildStatus
if($BuildStatus -eq "completed" ) {
write-output "No Running Pipeline, starting Next Pipeline"
$success = $true
} else {
Write-output "Pipeline Build In Progress, Waiting for it to finish!"
Write-output "Next attempt in 30 seconds"
Start-sleep -Seconds 30
}
}
catch{
Write-output "catch - Next attempt in 30 seconds"
write-output "1"
Start-sleep -Seconds 30
# Put the start-sleep in the catch statemtnt so we
# don't sleep if the condition is true and waste time
}
$count++
}until($count -eq 2000 -or $success -eq $true )
if ($result -ne "succeeded" )
{
echo "##vso[task.logissue type=error]Something went very wrong."
}
if(-not($success)){exit}
You can also refer to my another ticket.
You will probably have to merge the pipelines into one and, depending on the steps in them, you can convert them to jobs or even to stages. In both cases, you can specify dependencies via dependsOn (e.g. docs for jobs).
So you will have something like:
jobs:
- job: test
steps:
- step1...
- step2...
- job: deploy
dependsOn: test
steps:
- step1...
- step2...
Also, if you go this way, consider using deployment jobs for deployment, they have some related build-in functionality.

Execute Mutliple release pipeline

I have 50+ release pipelines for the production environment and each time doing production has to manually approve each of these pipelines for deployment.
Is there any way to automate and in a single click, all the pipeline gets approved and gets deployed?
The only way I am aware of accomplishing this would be to combine the 50+ pipelines into one master release and have those dependent on a separate stage who is deploying to a gated environment. Assign an approval on that environment. Thus once that stage in the "approval environment" is approved all subsequent stages will run.
In Azure DevOps, we can configure approvals, It checks all the resources used in that stage, such as source code and follow the target server deployed, this should let managers to check and agree this deploy, batch approval is meaningless.
If you insist on approving multiple release pipeline stages at the same time. we recommend that you can remove these approvals.
Or we can use API to list all Approvals and get the approval id, then update status of an approval.
In addition, if there are multiple approvals in the pipeline, it will only approve the current stage.
Steps:
Create a PAT token->create a build pipeline->click the tab variable and add the variable pat = {Token}, then set it to secret variable->add task powershell and enter the following script to approve.
$connectionToken="$(pat)"
$base64AuthInfo= [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$($connectionToken)"))
$PipelineUrl = "https://vsrm.dev.azure.com/{Org name}/{Project name}/_apis/release/approvals?api-version=5.1"
$Pipelines = (Invoke-RestMethod -Uri $PipelineUrl -Method Get -UseDefaultCredential -Headers #{Authorization=("Basic {0}" -f $base64AuthInfo)})
$ ApprovalId = $Pipelines.value.id
Write-Host $ ApprovalId
# List all Approvals and get approval id
ForEach ($Id in $ ApprovalId)
{
$baseUrl = "https://vsrm.dev.azure.com/{Org name}/{Project name}/_apis/release/approvals/$($Id)?api-version=5.1"
$body ="{
`"status`": `"approved`"
}"
$response = Invoke-RestMethod -Uri $baseUrl -ContentType "application/json" -Body $body -Headers #{Authorization=("Basic {0}" -f $base64AuthInfo)} -Method PATCH
}

How can I read the PR tag inside a pipeline triggered by another pipeline?

I had my first question answered here Is it possible to read the PR tag on a pipeline task? but my scenario is a little bit different. I need to read the PR tag from a pipeline that was triggered by another pipeline.
PR triggers the CI which checks if everything's ok for merge. If it is, the CI triggers the CD which will in turn read the PR tag.
PR -> CI -> CD (access the tag here)
I have a PowerShell task named Get PR tag with the following script (courtesy of Lance):
$url = "$($env:SYSTEM_TEAMFOUNDATIONSERVERURI)$env:SYSTEM_TEAMPROJECTID/_apis/git/repositories/$($env:BUILD_REPOSITORY_NAME)/pullRequests/$($env:SYSTEM_PULLREQUEST_PULLREQUESTID)/labels?api-version=5.1-preview.1"
$response = Invoke-RestMethod -Uri $url -Method Get -Headers #{
Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"
}
Write-Host "##vso[task.setvariable variable=PullRequestTag;isOutput=true]$($response.value.name)"
But I keep getting "The request is invalid.":
========================== Starting Command Output ===========================
/usr/bin/pwsh -NoLogo -NoProfile -NonInteractive -Command . '/home/vsts/work/_temp/74b14931-e33a-4389-b19f-3db7faa53e8d.ps1'
Invoke-RestMethod: /home/vsts/work/_temp/74b14931-e33a-4389-b19f-3db7faa53e8d.ps1:3
Line |
3 | $response = Invoke-RestMethod -Uri $url -Method Get -Headers #{
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| {"count":1,"value":{"Message":"The request is invalid."}}
##[error]PowerShell exited with code '1'.
Finishing: Get PR tag
My agent job is set to use the OAuth token:
Update
In release pipeline, the variable name is not the same as the variable name in build, we need update the url info in the script, we can also check the release pipeline variable in the Initialize job log.
Steps:
a. Configure branch policy and add the policy Build Validation-> add build pipeline A
b. Create release->select the build A as the Source type->Enable the feature Pull request trigger->open Pre-deployment conditions and enable the option Pull request deployment
c. Open the release->enable the feature Allow scripts to access the OAuth token (Click Agent Job Name=>Additional options) add task powershell and enter the script below
$url = "$($env:SYSTEM_TASKDEFINITIONSURI)$env:BUILD_PROJECTID/_apis/git/repositories/$($env:BUILD_REPOSITORY_NAME)/pullRequests/$($env:BUILD_PULLREQUEST_ID)/labels?api-version=5.1-preview.1"
$response = Invoke-RestMethod -Uri $url -Method Get -Headers #{
Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"
}
Write-Host "##vso[task.setvariable variable=PullRequestTag;isOutput=true]$($response.value.name)"
d. Configure the Reference name as PS and add task cmd to output the tags.
CMD script:
echo $(PS.PullRequestTag)
e. Create pull request and add tags
Result:
Update2
The pull request triggers the CI build pipeline(power shell), after the build pipeline is completed, another build pipeline(power shell test) will be triggered.
b. Open the build pipeline power shell test and add a new variable PullRequestID and grant test Build Service (xxx) account the Edit build pipeline permission. (open the build pipeline(power shell test)--> ... --> Security --> Edit build pipeline set to Allow)
c. enable the feature Allow scripts to access the OAuth token (Click Agent Job Name=>Additional options) add task powershell(Get the tag value) and enter the script below. click the powershell task->Output Variables->enter PS->add a task cmd and use the code echo $(PS.PullRequestTag) to output the tag value
$url = "$($env:SYSTEM_TEAMFOUNDATIONSERVERURI)$env:SYSTEM_TEAMPROJECTID/_apis/git/repositories/$($env:BUILD_REPOSITORY_NAME)/pullRequests/$(PullRequestID)/labels?api-version=5.1-preview.1"
$response = Invoke-RestMethod -Uri $url -Method Get -Headers #{
Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"
}
Write-Host "##vso[task.setvariable variable=PullRequestTag;isOutput=true]$($response.value.name)"
d. Open build pipeline power shell, enable the feature Allow scripts to access the OAuth token (Click Agent Job Name=>Additional options) add task powershell and enter the script below to update the pipeline(power shell test) variable PullRequestID value.
$url = "$($env:SYSTEM_TEAMFOUNDATIONSERVERURI)$env:SYSTEM_TEAMPROJECTID/_apis/build/definitions/55?api-version=5.1"
Write-Host "URL: $url"
$pipeline = Invoke-RestMethod -Uri $url -Headers #{
Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"
}
Write-Host "Pipeline = $($pipeline | ConvertTo-Json -Depth 100)"
# Update an existing variable named PullRequestID to its new value pull request ID
$pipeline.variables.PullRequestID.value= $($env:SYSTEM_PULLREQUEST_PULLREQUESTID)
####****************** update the modified object **************************
$json = #($pipeline) | ConvertTo-Json -Depth 99
$updatedef = Invoke-RestMethod -Uri $url -Method Put -Body $json -ContentType "application/json" -Headers #{Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"}
write-host "=========================================================="
Write-host "The value of Varialbe 'PullRequestID ' is updated to" $updatedef.variables.PullRequestID.value
write-host "=========================================================="

How to set the Description of an Azure DevOps Release via script?

In Azure DevOps I'm trying to set the release description via PowerShell / CMD in order to have a dynamic description of my releases based on the input of the artifacts from the build step.
I've tried setting the release variables via powershell like:
Write-Host "##vso[task.setvariable variable=release.releasedescription;]bar"
Write-Host "##vso[task.setvariable variable=RELEASE_RELEASEDESCRIPTION;]bar"
But that didn't work and the description field remains always empty.
Is there any tweak / setting that would help achieve this behavior?
What you tried to do is just to set the environment variable that contains the release description data and not set the "real" release description, so after the release finished the description not changed.
If you want to set the release description during the release you can try to do it with Azure DevOps Rest API - Update Release.
So add a PowerShell task that executes the Rest API with Invoke-RestMethod, get the current release with GET method and then update the release with PUT, in the body change the description to the new one.
You could do this too:
- bash: |
echo "Updating pipeline job Run description"
echo "##vso[build.updatebuildnumber]$(Build.BuildNumber) $(App_Name)"
displayName: "Set pipeline job Run description for Azure DevOps console"
For anyone who simply wants to set the name of an Azure Pipelines run, you can find the description here.
TLDR: Set the top-level name: attribute.
Here is the PowerShell script to set release description in DevOps. Before the stage that need approval, add a PowerShell task to invoke REST API to change the release description. Please use your own orgName, projectName, PAT and modify the release description.
$url = https://vsrm.dev.azure.com/<orgName>/<ProjectName>/_apis/release/releases/$(Release.ReleaseId)?api-version=6.0
$token="PAT"
$base64AuthInfo= [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$($token)"))
$head = #{ Authorization =" Basic $base64AuthInfo" }
$pipeline = Invoke-RestMethod -Uri $url -Headers $head -Method Get
$Pipeline.description = "<Modify the release description>"
$body = $Pipeline | ConvertTo-Json -Depth 100
$url2=https://vsrm.dev.azure.com/<orgName>/<ProjectName>/_apis/release/releases/$(Release.ReleaseId)?api-version=6.0
$resp = Invoke-RestMethod -Uri $url2 -Method Put -Headers $head -Body $body -ContentType application/json