How to add a manual intervention step in Azure Pipelines yaml - azure-devops

How do you add a manual intervention step into a multi-stage Azure Devops YAML pipeline?
In jenkins you can do some thing like:
stage ('approve-prod') {
steps {
input "Approve deployment to production?"
}
}
I am looking for the equivalent in Azure Devops YAML.
Note: this is for the newly released multi-stage Azure Devops pipelines, not the old style release pipelines. Related announcement here https://devblogs.microsoft.com/devops/whats-new-with-azure-pipelines/

Microsoft have now made available a brand new official Manual Validation task that allows for manual intervention to be added into a YAML pipeline.
Quick example of how to use this task is as follows:
jobs:
- job: waitForValidation
displayName: Wait for external validation
pool: server
timeoutInMinutes: 4320 # job times out in 3 days
steps:
- task: ManualValidation#0
timeoutInMinutes: 1440 # task times out in 1 day
inputs:
notifyUsers: |
test#test.com
example#example.com
instructions: 'Please validate the build configuration and resume'
onTimeout: 'resume'
Some key constraints to be aware of:
This task is only supported in YAML pipelines
Can be used only in an agentless job of a YAML pipeline.

Azure DevOps/Pipelines now has a feature called Environments which supports approvals.
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/environments?view=azure-devops#approvals
We are using them as a workaround. Basically we have specified two environments ApprovalNotRequired and ApprovalRequired in Azure DevOps. On the latter we have specified who can approve deployments. Then in the pipeline we reference the environment like this.
- stage: 'Approval not required'
jobs:
- deployment: 'MyDeployment'
displayName: MyDeployment
environment: 'ApprovalNotRequired'
strategy:
runOnce:
deploy:
# whatever
- stage: 'Approval required'
jobs:
- deployment: 'MyDeployment2'
displayName: MyDeployment2
environment: 'ApprovalRequired'
strategy:
runOnce:
deploy:
# whatever
The first stage will run without interference and the second will pause until it's approved.

This doesn't appear to be available yet, but there is a GitHub Issue tracking this:
https://github.com/MicrosoftDocs/vsts-docs/issues/4241
From the issue:
So what I heard from the product team is that this "approval per stage" policy isn't available yet but is on their backlog.
There is also a Roadmap work item tracking it:
https://dev.azure.com/mseng/AzureDevOpsRoadmap/_workitems/edit/1510336/

Because there's a long time since Microsoft is ignoring this, and because this is a critical missing functionality , I will add an workaround here (for the moment, it's working only to ignore the entire step for all machines in case of a multi stage YAML but I think this can be solved also, but I am not looking into it for the moment).
Unfortunately, there is a task that needs to be added before each task. This can be solved also by iterative insertion (https://learn.microsoft.com/en-us/azure/devops/pipelines/process/templates?view=azure-devops).
Shortly, in order to be able to ignore a specific task:
T1 is checking the build run for "IgnoreStep"tag. If found, it will set IgnoreStep variable to true and remove the tag
T2 is running only if previous IgnoreStep is on false
When something is failing and I want to ignore the step, I will add "IgnoreStep" tag for the run and retry.
For adding tags, I am using the API because there is no task to do it yet. For request details, F21 in Chrome and check what it sent to server after you will add a tag, and export the request to power shell.
Below you have the YAML:
trigger: none
jobs:
- deployment: Dev
environment:
name: Dev
resourceType: virtualMachine
tags: online
strategy:
runOnce:
deploy:
steps:
- task: PowerShell#2
displayName: CheckIfWeShouldIgnoreStep
name: CheckIfWeShouldIgnoreStep
inputs:
targetType: 'inline'
script: |
$user = "user"
$pass= "pass"
$secpasswd = ConvertTo-SecureString $pass -AsPlainText -Force
$credential = New-Object System.Management.Automation.PSCredential($user, $secpasswd)
$response = Invoke-RestMethod -Uri "https://server/tfs/collection/projectId/_apis/build/builds/$(Build.BuildId)/tags" `
-Method "GET" `
-Headers #{
"accept"="application/json;api-version=6.0;excludeUrls=true;enumsAsNumbers=true;msDateFormat=true;noArrayWrap=true"
} `
-ContentType "application/json" `
-Credential $credential -UseBasicParsing
Write-Host "##vso[task.setvariable variable=IgnoreStep]false"
Write-Host "Tags: $response"
foreach($tag in $response)
{
if($tag -eq "IgnoreStep")
{
Write-Host "##vso[task.setvariable variable=IgnoreStep]true"
Invoke-RestMethod -Uri "https://server/tfs/collection/projectId/_apis/build/builds/$(Build.BuildId)/tags/IgnoreStep" `
-Method "DELETE" `
-Headers #{
"accept"="application/json;api-version=6.0;excludeUrls=true;enumsAsNumbers=true;msDateFormat=true;noArrayWrap=true"
}`
-Credential $credential -UseBasicParsing
}
}
- task: PowerShell#2
displayName: Throw Error
condition: eq (variables.IgnoreStep, false)
inputs:
targetType: 'inline'
script: |
throw "Error"

Related

How to use key vault secret on Powershell that run on DevOps pipeline

everyone,
There are 2 tasks in my DevOps pipeline, the first is for getting the secret value from Azure Key Vault,
trigger: none
jobs:
- job: PBICDSolution
pool:
vmImage: windows-latest
steps:
- checkout: self
- task: AzureKeyVault#2
inputs:
azureSubscription: '<my subscription>'
KeyVaultName: 'PA01'
SecretsFilter: '<my secret name>'
RunAsPreJob: false
Next I want to use this secret value inside my powershell script for login a service principal account. Here is my powershell code,
$azureAplicationId = "<my service principal client id>"
$azureTenantId= "<my tanant id>"
Write-Output "Generate Credential"
$azurePassword = ConvertTo-SecureString <here should be the variable of AKV secret value> -AsPlainText -Force
$psCred = New-Object System.Management.Automation.PSCredential($azureAplicationId , $azurePassword)
Write-Output "Login SP"
Connect-PowerBIServiceAccount -Tenant $azureTenantId -ServicePrincipal -Credential $psCred
Here is the pipeline yml code for above task,
- task: AzurePowerShell#5
inputs:
azureSubscription: '<my subscripton name>'
ScriptType: 'FilePath'
ScriptPath: '$(Build.SourcesDirectory)\<my ps file name>.ps1'
ScriptArguments: '<how can i set the variable here?>'
azurePowerShellVersion: 'LatestVersion'
So the question is here, how can I get the output value from task1, then pass this value into task2 (PowerShell script)?
I have refer to this docs but it's not helpful since I don't need to download the secret to a txt file.
Any solution would be grateful!
how can I get the output value from task1, then pass this value into task2 (PowerShell script)?
The key vault you downloaded in the azure key vault task can be used as a Pipeline variable.
Refer to the following steps to use the Key vault.
In Azure Powershell Task, you can define the arguement: -azurePassword $(azurePassword)
For example:
- task: AzurePowerShell#5
displayName: 'Azure PowerShell script: FilePath'
inputs:
azureSubscription: xx
ScriptPath: test.ps1
ScriptArguments: '-azurePassword $(azurePassword)'
preferredAzurePowerShellVersion: 3.1.0
In Powershell file, you can define the param.
For example:
param (
[string]$azurePassword
)

Bicep Deployment error : An error occurred reading file. Could not find a part of the path '/home/vsts/work/1/s/bicep/storageaccount.bicep'

I'm trying to deploy a bicep template using a powershell task in a devOps yml pipeline.
I have the following task:
- task: AzurePowerShell#4
displayName: "4.3) Deploy xxxxxx Synapse Infra"
enabled: true
inputs:
azureSubscription: ${{parameters.azureServiceConnection}}
ScriptType: "InlineScript"
azurePowerShellVersion: "LatestVersion"
continueOnError: true
errorActionPreference : "continue"
Inline: |
echo "Deploy Bicep template"
$deployment = New-AzResourceGroupDeployment `
-ResourceGroupName "rg-emdi-data-${{parameters.environment}}" `
-TemplateFile "$env:BUILD_SOURCESDIRECTORY\bicep\storageaccount.bicep" `
-envName "${{parameters.environment}}" `
-location "${{parameters.location}}" `
-storageId "$(storageID)" `
However, when I run it, I get the following error message :
ERROR: An error occurred reading file. Could not find a part of the path '/home/vsts/work/1/s/bicep/storageaccount.bicep'.
I can't understand why the file path is not resolved. My file structure is:
It works if I deploy main.bicep but fails when deploying the storage account module.
Any help would be great.
A deployment job does not automatically clone the source repo, so you can either use the artifact approach as mentioned by Thomas or you can add a checkout: self step.
From the documentation: Deployment Jobs
A deployment job doesn't automatically clone the source repo. You can
checkout the source repo within your job with checkout: self.
Deployment jobs only support one checkout step.
This would look like something like this inside the YAML pipeline:
- stage: Deployment
jobs:
- deployment: DeployBicep
environment: $(Environment)
strategy:
runOnce:
deploy:
steps:
- checkout: self
- task: AzurePowerShell#4
displayName: "4.3) Deploy xxxxxx Synapse Infra"
enabled: true
inputs:
azureSubscription: ${{parameters.azureServiceConnection}}
ScriptType: "InlineScript"
azurePowerShellVersion: "LatestVersion"
continueOnError: true
errorActionPreference : "continue"
Inline: |
echo "Deploy Bicep template"
$deployment = New-AzResourceGroupDeployment `
-ResourceGroupName "rg-emdi-data-${{parameters.environment}}" `
-TemplateFile "$env:BUILD_SOURCESDIRECTORY\bicep\storageaccount.bicep" `
-envName "${{parameters.environment}}" `
-location "${{parameters.location}}" `
-storageId "$(storageID)" `

Identify build policy responsible for run of pull request build in Azure DevOps pipeline

I would like to identify the build policy for a build that was run by clicking the Queue (or Re-queue) button against a required/optional check from within a pull request. I wish to identify the policy programmatically from within a pipeline; e.g. a script task. Open to any approach, been exploring the az CLI but no luck thus far.
I've setup two build policies against a branch that both target the same build definition - Policy A and Policy B. Both are setup to be run manually - A is required, B is optional. Both will surface in the UI for a pull request as checks - A being required, B being optional. When a build is run by clicking the Queue (or Re-queue) button against either check, I would like to be able to identify which of the two policies the run was initiated from, (which policy provided the Queue or Re-queue button that was clicked).
EDIT: A bit more background on what I'm doing ...
I've got a single pipeline for building an application.
I've recently got a request to update the pipeline to support publishing to Chromatic.
I've added a Publish to Chromatic parameter to the pipeline and a task to push to Chromatic when the parameter is set to true.
I received a subsequent request to make it easier to publish changes from a feature branch to Chromatic. One engineer threw out the idea of having an optional check available in pull requests to give a single button click experience.
While researching my options, I was wondering if it would be possible to enhance the existing pipeline to set the Publish to Chromatic parameter to true during a run. I found this comment on Reddit which ultimately led to me posting here ...
set a default for your parameter (I like to use 'auto') add a script >task near the beginning that reads the pull request comment and sets a variable for you to use in later logic if the parameter is auto . you can even condition this to only run on a PR.
I am aware that I could create a separate pipeline for publishing to Chromatic instead of updating the existing one; that's one of a few options I have. At this point, I'm more-so curious whether or not this particular approach is technically feasible even if I opt not to go forward with it.
Hope that adds some clarity!
The policy that queued the pipeline isn't something that is visible to the pipeline as a pipeline variable. In fact, there doesn't seem to be any indication if the PullRequest was queued manually or automatically.
There might be a few other ways to approach this...
I would start by putting a publishChromatic parameter in the pipeline and then building up conditions in the pipeline execution around this variable. By default, let's assume that the value is false so that if you're manually queueing a pipeline run you can opt-in.
triggers:
- include:
branches:
- develop
parameters:
- name: publishChromatic
displayName: 'Publish build to Chromatic'
type: boolean
default: false
jobs:
- job: Build
variables:
publishChromatic: ${{ parameters.publishChromatic }}
steps:
... pre-execution steps
- task: CmdLine#2
displayName: Publish to Chromatic
condition: and(succeeded(), eq(variables['publishChromatic'], 'true'))
inputs:
script: npx chromatic --project-token=$(CHROMATIC_PROJECT_TOKEN) --branch=$(Build.SourceBranch)
... post execution steps
Option 1: Pull Request Labels
One option might be to inspect the pull request for the presence of a label as outlined in this answer. As a pre-execution step, a simple script could flip the flag when the label is present:
- pwsh: |
$urlFormat = "{0}/{1}/_apis/git/repositories/{1}/pullRequests/{2}/labels?api-version=6.0-preview.1"
$url = $urlFormat -f `
$env:SYSTEM_TEAMFOUNDATIONSERVERURI, `
$env:SYSTEM_TEAMPROJECTID, `
$env:BUILD_REPOSITORY_NAME, `
$env:SYSTEM_PULLREQUEST_PULLREQUESTID
$headers = #{
Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"
}
$response = Invoke-RestMethod -Uri $url -Method Get -Headers $headers
$labels = $response.value.name
Write-Host "##vso[task.setvariable variable=PullRequestTag]$labels"
displayName: 'Fetch Pull Request Labels'
condition: and( succeeded(), eq(variables['Build.Reason'], 'PullRequest'))
env:
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
- pwsh: |
if ("$(PullRequestLabels)" -like "*chromatic*") {
Write-Host "##vso[task.setvariable variable=publishChromatic]true"
}
condition: and( succeeded(), eq(variables['Build.Reason'], 'PullRequest'))
displayName: 'Check for Chromatic label'
I like this option in that it provides a bit of traceability for which Pull Requests were deployed. Unfortunately, there's no way to queue a build automatically when the PR labels are modified so you'd need to have the tag on the PR before triggering the pipeline.
You could also establish a different pattern such as triggering based on a convention like a value that appears in the name of the Pull Request, etc.
Option 2: Pipeline to Trigger Chromatic
If you'd rather have a Build Validation option labeled 'Deploy to Chromatic' to automate triggering your deployment to Chromatic, a simple option would be to create a pipeline that triggers your pipeline with the publishChromatic parameter.
trigger: none
steps:
- checkout: none
- pwsh: |
$pipelineId = 1234
$urlFormat = "{0}/{1}/_apis/pipelines/{2}/runs?api-version=6.0-preview.1
$url = $urlFormat -f `
$env:SYSTEM_TEAMFOUNDATIONSERVERURI, `
$env:SYSTEM_TEAMPROJECTID `
$pipelineId
$headers = #{
Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"
}
$body = #{
resources = #{ repositories = #{ self = #{ refName = "$(Build.SourceBranch)" } } }
variables = #{
originalPrId = #{
value = "$(System.PullRequest.PullRequestId)
}
}
templateParameters = #{
publishChromatic = $true
}
}
Invoke-RestMethod -Uri $url -Method Post -Body $body -Headers $headers
displayName: 'Trigger Chromatic Pipeline'
condition: eq(variables['Build.Reason'],'PullRequest')
env:
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
This simple script performs a fire-and-forget approach to triggering your original pipeline.
If you need to have a successful deployment to Chromatic as part of your PR, you could adjust the original pipeline to report a pull-request status.
In your original pipeline, add the following as a post-execution step:
- pwsh: |
$urlFormat = "{0}/{1}/_apis/git/repositories/{2}/pullRequests/{3}/statuses?api-version=6.0-preview.1
$url = $urlFormat -f `
$env:SYSTEM_TEAMFOUNDATIONSERVERURI, `
$env:SYSTEM_TEAMPROJECTID, `
$env:BUILD_REPOSITORY_NAME, `
"$(originalPrId)"
$headers = #{
Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"
}
$body = #{
status = "succeeded"
description = "completed chromatic regression"
context = #{
name = "qualitygate/chromatic"
}
targetUrl = "http://chromatic.com/your/buildid"
}
Invoke-RestMethod -Uri $url -Method POST -Body $body -Headers $headers
displayName: Report status to PR
condition: and( succeeded(), ne(variables['originalPrId'],''))
env:
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
To require a successful chromatic quality gate, add a Status Check to your Branch Policy with the same name mentioned above qualitygate/chromatic.
Option 3: Further down the Rabbit hole
You can establish even deeper integration with Chromatic by building a custom extension that allows you to add specialized menus to the Pull Request checks menu. The custom menu could include javascript-enabled buttons to trigger your pipeline without the need for a custom pipeline mentioned in Option 2.
While not necessarily dependent on writing a custom extension, you could also create an Azure Function App that listens for webhooks from Chromatic and posts status updates back to your PR with the a custom UI that links back to the Chromatic build. You'd simply need to query the Azure DevOps API to map the branch name in the Chromatic payload to the corresponding PR.

##[error]Required: 'ConnectedServiceNameARM' input --- Azure DevOps

##[error]Required: 'ConnectedServiceNameARM' input
This is in Azure DevOps using YAML inline script.
Need help with what to enter to fix this error? I am really new at YAML. This is a inline YAML and what tried seems to break the YAML script. The ConnectedServiceNameARM is just the Azure Subscription name? My service connection in azure devops has a working azure subscription name so I am wondering what is wrong?
Also need this YAML code to run so that the output file is placed in agent/_work/_tasks folder and not the artifacts folder. How would I move the file from the _tasks/Powershell folder to something that can be copied to share?
trigger:
- main
pool:
name: 'CloudUiPath001'
demands:
- agent.name -equals UiPathAgent01
steps:
- task: AzurePowerShell#3
displayName: 'Azure PowerShell script: InlineScript'
inputs:
ScriptType: InlineScript
Inline: |
$filePath='C:\Program Files (x86)\UiPath\Studio'
$dir=(New-Object -com scripting.filesystemobject).getFolder($filePath).ShortPath
$ProjectFilePath= "$(System.DefaultWorkingDirectory)/_TESTREPO7/project.json"
$ExecutableFilePath=$dir+"\UiPath.Studio.CommandLine.exe"
$OutputFilePath=".\$(Get-Date -Format 'yyyy-MM-dd-HH-mm-ss')-Workflow-Analysis.json"
#This was an attempt to write the filename to a pipeline variable: Feel free to continue on this path if possible
Write-Host "##vso[task.setvariable variable=jsonFilePath]$OutputFilePath"
Write-Output "$(Get-Date -Format 'HH:mm:ss') - STARTED - Workflow Analyzer CLI Script"
$Command = "$ExecutableFilePath analyze -p $ProjectFilePath"
Invoke-Expression $Command | Out-File -FilePath $OutputFilePath
Write-Output "$(Get-Date -Format 'HH:mm:ss') - COMPLETED - Workflow Analyzer CLI Script"
azurePowerShellVersion: LatestVersion
How do I fix his error within a INLINE YAML script. I am new to YAML and when I tried to enter a input I got errors.
##[error]Required: 'ConnectedServiceNameARM' input
According to your AzurePowerShell task definition, you don’t seem to specify the azureSubscription field.
steps:
- task: AzurePowerShell#5
displayName: 'Azure PowerShell script: InlineScript'
inputs:
azureSubscription: 'xxx'
ScriptType: InlineScript
Inline: xxx
azurePowerShellVersion: 'LatestVersion'
You can click the Settings shown in the figure below to specify the subscription.
About Azure PowerShell task, please refer to this document for details.
to go around this error change from task: AzurePowerShell#5 to pwsh:
- pwsh: |
InlineScript
displayName: 'Azure PowerShell script: InlineScript'

Azure pipeline build bring TFVC project references into git build

I have a git solution in azure repositories that has three project references to TFVC projects. I'm trying to create a build pipeline right now using YAML, but I can't find any step that downloads projects from TFVC. This is giving me this sort of error:
Error MSB3202: The project file "(path to project)\Standard.Logging.csproj" was not found
I know this is because the project folders aren't part of the repository, but i'm not sure how to bring them from the tfsvc repo into my build agent.
Here is my azure-pipelines.yml:
pool: 'MyPool'
variables:
solution: '**/*.sln'
buildPlatform: 'Any CPU'
buildConfiguration: 'Release'
steps:
- task: NuGetToolInstaller#1
- task: NuGetCommand#2
inputs:
command: 'restore'
restoreSolution: '$(solution)'
feedsToUse: 'select'
vstsFeed: '{company feed}'
- task: VSBuild#1
inputs:
solution: '$(solution)'
msbuildArgs: '/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:DesktopBuildPackageLocation="$(build.artifactStagingDirectory)\WebApp.zip"
platform: '$(buildPlatform)'
configuration: '$(buildConfiguration)'
How do I download these projects?
I am afraid YAML pipeline doesnot support TFVC currently.
As workaround you can migrate TFVC to Git.
You can also create a classic UI build pipeline instead of YAML pipeline. TFVC is supported on classic UI pipeline.
User voice to support TFVC in Yaml has been submitted to Microsoft development. You can vote it up or create a new one. see here.
Update:
To bring the code from tfsvc repo into build agent.
you can use TFVC Get items rest api to get the items. Add a script task in your pipeline to call the rest api and save the items in the build agent. Please check about below example powershell script: To get a Personal access token. Please refer to document here.
$url = "$(System.TeamFoundationCollectionUri)$(System.TeamProject)/_apis/tfvc/items?scopePath=path&recursionLevel=full&api-version=5.1"
$PAT= "Personal access token"
$base64AuthInfo= [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$($PAT)"))
$result = Invoke-RestMethod -Uri $url -Method Get -Header #{Authorization = "Basic $base64AuthInfo"}
$files= $result.value | where { !$_.isFolder} | select path, url
foreach($file in $files){
$path = "$(System.DefaultWorkingDirectory)\" + $file.path.Substring(2)
New-Item -Path $path -ItemType File -Force
Invoke-RestMethod -Uri $file.url -Method get -Header #{Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"} -OutFile $path
}
Above script call the Get Items Api and get the items url and path($files= $result.value | where { !$_.isFolder} | select path, url)
Then get each item and save to $(System.DefaultWorkingDirectory). For example if my scopePath is $/MyProject/, then the items will be save to $(System.DefaultWorkingDirectory)/MyProject/