Injecting an input to a step provided in a stepList? - azure-devops

I have a pipeline template that takes a stepList:
parameters:
- name: applicationDeploySteps
type: stepList
default: []
And injects the stepList into the template:
- deployment: Deploy_App
displayName: Deploy Application
pool: ${{ variables.AgentPool }}
environment: ${{ parameters.Stage }}
variables:
- name: ServiceConnection
value: SomeServiceConnection
strategy:
runOnce:
deploy:
steps:
- ${{ each step in parameters.applicationDeploySteps }}:
- ${{ each pair in step }}:
${{ pair.key }}: ${{ pair.value }}
However, I'd like to provide an AzureCLI#2 step, with the azureSubscription parameter being sourced from a variable inaccessible to the AzureCLI#2 step at the time of template compilation:
extends:
template: main.yml
parameters:
applicationDeploySteps:
- task: AzureCLI#2
inputs:
azureSubscription: $(ServiceConnection)
addSpnToEnvironment: true
scriptType: 'bash'
scriptLocation: 'inlineScript'
inlineScript: |
echo "do azurey things here"
The problem is in azureSubscription: $(ServiceConnection). Obviously, that variable can't resolve. So the solution I'm shooting for is to inject the azureSubscription value in the template. However, I can't find a way to effectively iterate over the values provided in the input block.
- ${{ each pair in step }}:
${{ pair.key }}: ${{ pair.value }}
will let me interrogate the type of the step. Trying to take it further just gives me a null reference exception when trying to queue the pipeline:
- ${{ each pair in step }}:
${{ if eq(pair.key, 'inputs') }}:
- ${{ each input in pair.value }}:
${{ if eq(input.key, 'azureSubscription') }}:
${{ input.key }}: ${{ variables.ServiceConnection }}
${{ else }}:
${{ input.key }}: ${{ input.value }}
${{ else }}:
${{ pair.key }}: ${{ pair.value }}
That attempt gives me: Object reference not set to an instance of an object. with no corresponding line number. I'm guessing it's failing to iterate over pair.value, but I have no idea how to troubleshoot it further or get an idea of what I can and cannot iterate over. The documentation does not include more comprehensive examples, just checking if, say, it's a script task and blocking execution.
Note that this is similar, but not the scenario that I'm implementing.

I figured it out.
- ${{ each step in parameters.applicationDeploySteps }}:
- ${{ each pair in step }}:
${{ if eq(pair.key, 'inputs') }}:
inputs:
${{ each input in pair.value }}:
${{ if eq(input.key, 'azureSubscription') }}:
${{ input.key }}: ${{ variables.ServiceConnection }}
${{ else }}:
${{ input.key }}: ${{ input.value }}
${{ else }}:
${{ pair.key }}: ${{ pair.value }}
This document was useful for pointing me in the right direction, but honestly it was mostly trial-and-error.

Related

Not able to set variable based on parameters in YAML

I'm trying to build a new yaml file that reads keyvault secrets based on the parameters at the runtime and declared variables with the condition as per the parameters, but this isn't working.
- name: azure_subscription
displayName: " Select subscription "
type: string
default: "service-connection-dev"
values:
- 'service-connection-dev'
- 'service-connection-sit'
- 'service-connection-tes'
- 'service-connection-prd'
variables:
- ${{ if eq('${{ parameters.azure_subscription }}', 'service-connection-sit') }}:
name: key_vault
value: 'core-kv-sit'
- ${{ if eq('${{ parameters.azure_subscription }}', 'service-connection-dev') }}:
name: key_vault
value: 'core-kv-dev'
stages:
- stage: Validate
${{ if eq(parameters.azure_subscription, 'service-connection-dev') }}:
pool:
name: agent-pool-win-dev
${{ if eq(parameters.azure_subscription, 'service-connection-sit') }}:
pool:
name: agent-pool-win-sit
jobs:
- job: Validate
steps:
- task: AzureKeyVault#2
inputs:
KeyVaultName: "${{variables.key_vault}}"
SecretsFilter: "*"
RunAsPreJob: false
azureSubscription: ${{ parameters.azure_subscription }}
I've tried using variables inside jobs, but that is also not working. Can someone please help?
Also, I'll have to declare 2 more variables as per the parameters input, Is it possible ?
Thanks in advance
- ${{ if eq('${{ parameters.azure_subscription }}', 'service-connection-sit') }}:
You're using a literal value of "${{ parameters.azure_subscription }}" for the left side of the comparison. The comparison should just be parameters.azure_subscription.
So: - ${{ if eq(parameters.azure_subscription, 'service-connection-sit') }}:

How to get the value of parameter in condition azure pipelines yaml

Here's a section of my YAML pipeline:
parameters:
- name: appEnv
default: None
type: string
values:
- None
- Dev
- QA
- Stage
- Stage2
- Prod
variables:
- name: envConfiguration
${{ if and(eq('${{parameters.appEnv}}', 'None'), startsWith(variables['Build.SourceBranchName'], 'release/')) }}:
value: Prod
${{ elseif and(eq('${{parameters.appEnv}}', 'None'), startsWith(variables['Build.SourceBranchName'], 'hotfix/')) }}:
value: Prod
${{ elseif and(eq('${{parameters.appEnv}}', 'None'), startsWith(variables['System.PullRequest.TargetBranch'], 'release/')) }}:
value: Stage
${{ elseif and(eq('${{parameters.appEnv}}', 'None'), startsWith(variables['System.PullRequest.TargetBranch'], 'hotfix/')) }}:
value: Stage2
${{ elseif eq('${{parameters.appEnv}}', 'None') }}:
value: Dev
${{ else }}:
value: ${{parameters.appEnv}}
- name: envFileName
${{ if eq(variables.envConfiguration, 'Dev') }}:
value: .env.dev
${{ if eq(variables.envConfiguration, 'QA') }}:
value: .env.qa
${{ if eq(variables.envConfiguration, 'Stage') }}:
value: .env.stage
${{ if eq(variables.envConfiguration, 'Stage2') }}:
value: .env.stage2
${{ if eq(variables.envConfiguration, 'Prod') }}:
value: .env.prod
The problem is that the envConfiguration conditions are never honoured. Whenever I create a PR targeting the release/ branch the value is always None. I think it is because of runtime and compile time variables (not sure).
All I want to do is select the correct .env file based on the appEnv parameter and the source and target branches.
You are using incorrect syntax.
${{ if and(eq('${{parameters.appEnv}}', 'None'), startsWith(variables['Build.SourceBranchName'], 'release/')) }}:
should be
${{ if and(eq(parameters.appEnv, 'None'), startsWith(variables['Build.SourceBranchName'], 'release/')) }}:
As written, it's doing a literal comparison of the values ${{parameters.appEnv}} and None, which will, of course, always be false.
In general, refer to the documentation for examples of correct syntax.

YAML Extends Template - Unrecognized value: 'else'. Located at position 1 within expression: else

Expanding on my first question. I am now experiencing a new error Unrecognized value: 'else'. Located at position 1 within expression: else that I cannot figure out a solution to. The issue happens when I add the 'else' condition within the foreach of the job.steps.
Build YAML
resources:
repositories:
- repository: self
type: git
ref: refs/heads/Development
- repository: AdoRestrictions
type: git
name: utl-yaml-templates
ref: refs/heads/main
trigger: none
pool:
name: PROD
extends:
template: ADO_Stage_Restrictions_Dev.yml#AdoRestrictions
parameters:
stageObjs:
- stage: 'BuildStage'
displayName: 'Build Test'
jobs:
- job: 'BuildJob'
displayName: 'Build'
steps:
- task: PowerShell#2
displayName: 'PS Hello World'
inputs:
targetType: inline
script: |
Write-Host "Hello World"
Working Extends YAML
parameters:
- name: stageObjs
type: stageList
default: []
stages:
- ${{ each stage in parameters.stageObjs }}:
- stage: ${{ stage.stage }}
displayName: ${{ stage.displayName }}
jobs:
- ${{ each job in stage.jobs }}:
- job: ${{ job.job }}
displayName: ${{ job.displayName }}
steps:
- ${{ each step in job.steps }}:
- ${{ if or(startsWith(step.task, 'PowerShell'),startsWith(step.task, 'CmdLine'),startsWith(step.task, 'Bash'),startsWith(step.task, 'ShellScript'),containsValue(step.task, 'Script'),containsValue(step.task, 'CLI'),containsValue(step.task, 'PowerShell')) }}:
- task: PowerShell#2
displayName: 'Unapproved - ${{ step.displayName }}'
inputs:
targetType: inline
script: Write-Output "Unapproved Task - Scripting and CLI tasks are not approved for use in yaml pipelines"
Broken Extends YAML
parameters:
- name: stageObjs
type: stageList
default: []
stages:
- ${{ each stage in parameters.stageObjs }}:
- stage: ${{ stage.stage }}
displayName: ${{ stage.displayName }}
jobs:
- ${{ each job in stage.jobs }}:
- job: ${{ job.job }}
displayName: ${{ job.displayName }}
steps:
- ${{ each step in job.steps }}:
- ${{ if or(startsWith(step.task, 'PowerShell'),startsWith(step.task, 'CmdLine'),startsWith(step.task, 'Bash'),startsWith(step.task, 'ShellScript'),containsValue(step.task, 'Script'),containsValue(step.task, 'CLI'),containsValue(step.task, 'PowerShell')) }}:
- task: PowerShell#2
displayName: 'Unapproved - ${{ step.displayName }}'
inputs:
targetType: inline
script: Write-Output "Unapproved Task - Scripting and CLI tasks are not approved for use in yaml pipelines"
- ${{ else }}:
- ${{ step }}
Full Error Message
(Line: 22, Col: 13): Unrecognized value: 'else'. Located at position 1 within expression: else. (Line: 22, Col: 13): Expected at least one key-value pair in the mapping
Update - Working Solution
parameters:
- name: stageObjs
type: stageList
default: []
stages:
- ${{ each stage in parameters.stageObjs }}:
- stage: ${{ stage.stage }}
displayName: ${{ stage.displayName }}
jobs:
- ${{ each job in stage.jobs }}:
- job: ${{ job.job }}
displayName: ${{ job.displayName }}
steps:
- ${{ each step in job.steps }}:
#Disallow any type of scripting tasks
- ${{ if or(startsWith(step.task, 'PowerShell'),startsWith(step.task, 'CmdLine'),startsWith(step.task, 'Bash'),startsWith(step.task, 'ShellScript'),contains(step.task, 'Script'),contains(step.task, 'CLI'),contains(step.task, 'PowerShell')) }}:
- task: PowerShell#2
displayName: 'Unapproved Task - ${{ step.displayName }}'
inputs:
targetType: inline
script: throw "Unapproved Task - Scripting and CLI tasks are not approved for use in yaml pipelines"
#Not a scripting task perform additional checks
- ${{ if and(not(startsWith(step.task, 'PowerShell')),not(startsWith(step.task, 'CmdLine')),not(startsWith(step.task, 'Bash')),not(startsWith(step.task, 'ShellScript')),not(contains(step.task, 'Script')),not(contains(step.task, 'CLI')),not(contains(step.task, 'PowerShell')) ) }}:
#Validate azure subscription provided for the environment
- ${{ if contains(step.task, 'Azure') }}:
- ${{ each pair in step }}:
${{ if eq(pair.key, 'inputs') }}:
inputs:
${{ each attribute in pair.value }}:
${{ if contains(attribute.key, 'Subscription') }}:
${{ if and(ne(attribute.value, 'sub-name1'), ne(attribute.value, 'sub-name2'), ne(attribute.value, 'sub-name3')) }}:
${{ attribute.value }}: ''
${{ if or(eq(attribute.value, 'sub-name1'), eq(attribute.value, 'sub-name2'), eq(attribute.value, 'sub-anme3')) }}:
${{ pair.key }}: ${{ pair.value }}
${{ if ne(pair.key, 'inputs') }}:
${{ if eq(pair.key, 'displayName') }}:
${{ pair.key }}: 'Invalid Azure Subscription - ${{ pair.value }}'
${{ if ne(pair.key, 'displayName') }}:
${{ pair.key }}: ${{ pair.value }}
#Allow all other tasks
- ${{ if not(contains(step.task, 'Azure')) }}:
- ${{ step }}

In Azure DevOps YAML build definition how can we parameterize the name of some dependent template?

I have two templates (build-tsmain-project.yml and build-single-project.yml) which I call from another template like this:
steps:
- ${{ each project in parameters.projects }}:
- ${{ if ne(project.disable, 'true') }}:
- ${{ if eq(project.name, 'TSMain') }}:
- template: build-tsmain-project.yml
parameters:
name: ${{ project.name }}
shortName: ${{ project.shortName }}
path: ${{ project.name }}.sln
moreBuildArgs: ${{ parameters.restoreArgs }} ${{ parameters.moreBuildArgs }} ${{ project.moreBuildArgs }}
testFilter: ${{ project.testFilter }}
${{ if parameters.withCoverage }}:
moreTestArgs: ${{ parameters.coverageArgs }}
publishTestResults: false
noTest: ${{ project.noTest }}
- ${{ if ne(project.name, 'TSMain') }}:
- template: build-single-project.yml
parameters:
name: ${{ project.name }}
shortName: ${{ project.shortName }}
path: ${{ project.name }}.sln
moreBuildArgs: ${{ parameters.restoreArgs }} ${{ parameters.moreBuildArgs }} ${{ project.moreBuildArgs }}
testFilter: ${{ project.testFilter }}
${{ if parameters.withCoverage }}:
moreTestArgs: ${{ parameters.coverageArgs }}
publishTestResults: false
noTest: ${{ project.noTest }}
That is a problem, because I duplicate the parameters - not DRY.
The only working solution that is DRY which I could find involves introducing a new template (build-single-project-or-tsmain.yml):
parameters:
- name: template_name
type: string
- name: template_parameters
type: object
steps:
- template: ${{ parameters.template_name }}
parameters: ${{ parameters.template_parameters }}
Which allows me to rewrite the original code like this:
steps:
- ${{ each project in parameters.projects }}:
- ${{ if ne(project.disable, 'true') }}:
- template: build-single-project-or-tsmain.yml
parameters:
${{ if eq(project.name, 'TSMain') }}:
template_name: build-tsmain-project.yml
${{ if ne(project.name, 'TSMain') }}:
template_name: build-single-project.yml
template_parameters:
name: ${{ project.name }}
shortName: ${{ project.shortName }}
path: ${{ project.name }}.sln
moreBuildArgs: ${{ parameters.restoreArgs }} ${{ parameters.moreBuildArgs }} ${{ project.moreBuildArgs }}
testFilter: ${{ project.testFilter }}
${{ if parameters.withCoverage }}:
moreTestArgs: ${{ parameters.coverageArgs }}
publishTestResults: false
noTest: ${{ project.noTest }}
Now I am wondering if there is a way to avoid introducing a new template and still have a DRY solution?
We use Azure DevOps Server 2020 (on-prem).

Can Conditional Variable Assignment be Done in Azure Pipelines?

Azure Pipelines has Expressions and Conditions, but I can find no way to assign one of two values to a variable, based on a condition.
Is there any way to accomplish what this pseudo-code would?
${{ if endsWith( variables['Build.SourceBranchName'], '/master' ) }}:
buildVersion: variables['mavenVersion']
${{ else }}:
buildVersion: variables['Build.SourceBranchName']
As an extension to #Mike Murray's answer, if you are using variable groups you must define additional variables as name value pairs. To use conditional variable assignment in this case would be as follows:
variables:
- group: 'my-variable-group'
- name: myfirstadditionalvariable
value: 100
- name: myconditionalvariable
${{ if eq( variables['Build.SourceBranchName'], 'master' ) }}:
value: masterBranchValue
${{ if ne( variables['Build.SourceBranchName'], 'master' ) }}:
value: featureBranchValue
I was closer than I thought. This is not pretty, but it worked. (with more yaml context)
variables:
${{ if eq( variables['Build.SourceBranchName'], 'master' ) }}:
buildVersion: ${{ variables['mavenVersion'] }}
${{ if ne( variables['Build.SourceBranchName'], 'master' ) }}:
buildVersion: ${{ variables['Build.SourceBranchName'] }}
buildKey: ${{ format('{0}_{1}', variables['supportReleaseNumber'], variables['buildVersion']) }}
buildNum: $[counter(variables['buildKey'], 1)] # same as $(Rev:r), but more widely usable
name: $(buildKey)_$(buildNum) # build run name
This should do the trick....
BuildVersion is initialised as $(Build.SourceBranch)
if it's the master branch you change that to the $(mavenVersion)
else no change.
variables:
mavenVersion: '1.0'
buildVersion: $(Build.SourceBranch)
pool:
vmImage: 'ubuntu-latest'
steps:
- script: echo '##vso[task.setvariable variable=buildVersion]$(mavenVersion)'
displayName: "Set the buildVersion as mavenVersion if the Build.SourceBranch = 'refs/heads/master' "
condition: eq(variables['Build.SourceBranch'], 'refs/heads/master')
- script: echo $(buildVersion)
displayName: 'Printing the variable'
non-master branches prints 'refs/heads/branch_name' which is mavenVersion
master branch prints 1.0 which is mavenVersion
With this update your YAML is valid:
${{ if endsWith( variables['Build.SourceBranchName'], '/master' ) }}:
buildVersion: variables['mavenVersion']
${{ else }}:
buildVersion: variables['Build.SourceBranchName']
All works fine with this
trigger: none
name: if-else
pool:
vmImage: ubuntu-latest
variables:
${{ if endsWith( variables['Build.SourceBranchName'], 'master' ) }}:
buildVersion: 'master'
${{ else }}:
buildVersion: 'none-master'
steps:
- script: |
echo "$(Build.SourceBranchName)"
echo "$(buildVersion)"
displayName: 'Display buildVersion'
Simply Build.SourceBranchName contains just master.
Generating script.
========================== Starting Command Output ===========================
/usr/bin/bash --noprofile --norc /home/vsts/work/_temp/1121efc8-6445-4e19-be5a-8d525a76467e.sh
master
master
Finishing: Display buildVersion
Note: This doesn't work on Azure DevOps Server - please be aware of that! Thanks #psulek for highlighting this!
#Mike Murray, thank you for this! I have been trying to solve this for ages.
When builds are triggered from pull requests the SourceBranchName is always 'merge'. Your answer helped me come up with this solution for getting the target branch name for both scenarios, manual builds and builds triggered by pull-requests:
${{ if ne( variables['Build.SourceBranchName'], 'merge' ) }}:
environment: ${{ variables['Build.SourceBranchName'] }}
${{ if endsWith( variables['System.PullRequest.TargetBranch'], 'dev' ) }}:
environment: dev
${{ if endsWith( variables['System.PullRequest.TargetBranch'], 'staging' ) }}:
environment: staging
${{ if endsWith( variables['System.PullRequest.TargetBranch'], 'master' ) }}:
environment: prod
Not very pretty, but finally works.
You can have conditional variables based on parameters:
parameters:
- name: example
default: master
values:
- master
- branch1
- branch2
variables:
${{ if eq( parameters.example, 'master' ) }}:
buildVersion: 'this'
${{ else }}:
buildVersion: 'that' # buildVersion = this
Won't work if the variables are in a separate template.