I have scaleset agents configured and working fine. But i would also like to take advantage of Microsoft Hosted agents whenever possible. So i added a parameter to the pipeline to choose which agent pool to use for that particular pipeline run.
I manged to set the pool/image variables based on the chosen parameter but unable to make a container job use the selected pool due to a difference in syntax as shown below.
Is there a way to modify the schema of yaml template based on the user selected parameter.
I think we need to not display vmImage property conditionally. I tried below snippet but it doesn't work. vmImage property is always missing.
pool:
${{ if ne(parameters.pool, 'Azure Pipelines') }}:
name: ${{ parameters.pool }}
${{ if eq(parameters.pool, 'Azure Pipelines') }}:
vmImage: 'ubuntu-latest'
UPDATED:
parameters:
- name: pool
displayName: Agent Pool
type: string
default: Scalesets
values:
- Scalesets
- Hosted
jobs:
- deployment: '${{ parameters.jobName }}'
displayName: ${{ coalesce(parameters.jobDisplayName, 'Deploy Infrastructure') }}
${{ if parameters.dependsOn }}:
dependsOn: '${{ parameters.dependsOn }}'
${{ if parameters.condition }}:
condition: '${{ parameters.condition }}'
pool:
name: ${{ parameters.pool }}
vmImage: 'ubuntu-latest'
container:
image: '${{ parameters.image }}'
endpoint: 'Container Registry - Prd'
environment: ${{ parameters.environment }}
variables:
- name: 'Release-Tag'
value: '${{ parameters.release }}'
- name: 'Layer'
value: '${{ parameters.environmentLayer }}'
strategy:
runOnce:
deploy:
steps:
- script: |
echo $(Release-Tag) $(Layer)
displayName: 'Show release condidate'
You can set the data type of your parameters as object. The value of object type can be any YAML structure. Please find more details about data types of parameters in this document.
Here is my sample:
parameters:
- name: pool
type: object
default:
pool:
vmImage: ubuntu-latest
jobs:
- job: build
pool: ${{ parameters.pool }}
steps:
- script: echo Hello, world!
You can also choose the job to run based on the value of your parameters:
trigger: none
parameters:
- name: type
type: string
values:
- Scalesets
- Hosted
jobs:
- ${{ if eq(parameters.type, 'Scalesets') }}:
- job: Scalesets
pool: Default
steps:
- script: echo Scalesets
- ${{ if eq(parameters.type, 'Hosted') }}:
- job: Hosted
pool:
vmImage: ubuntu-latest
steps:
- script: echo Hosted
Related
Looking to try and get some help figuring out a solution to what seems like a simple task. I am trying to develop some extend YAML templates for security within our ADO pipelines but first I have to get passed this error I am experiencing.
The error being returned by ADO when I try to validate or run the pipeline is
/ADO_Stage_Restrictions_Dev.yml#AdoRestrictions (Line: 7, Col: 3): Unexpected value job
Working Extend Template YAML
This template validates and executes without issue, which to me means I am successfully passing the stages object into the extends template
parameters:
- name: stageObjs
type: stageList
default: []
stages:
- ${{ each stage in parameters.stageObjs }}:
${{ stage }}
Broken Extend Template YAML
This template does not validate and throws the 'Unexpected value job' exception. based on the stage schema I would assume that I would be able to loop the jobs property within the stage.
parameters:
- name: stageObjs
type: stageList
default: []
stages:
- ${{ each stage in parameters.stageObjs }}:
- ${{ each job in stage.jobs }}:
${{ job }}
Build YAML
The main yaml file that extends stages
resources:
repositories:
- repository: self
type: git
ref: refs/heads/Development
- repository: AdoRestrictions
type: git
name: utl-yaml-templates
ref: refs/heads/main
trigger: none
pool:
name: PROD
extends:
template: ADO_Stage_Restrictions_Dev.yml#AdoRestrictions
parameters:
stageObjs:
- stage: 'BuildStage'
displayName: 'Build Test'
jobs:
- job: 'BuildJob'
displayName: 'Build'
steps:
- task: PowerShell#2
displayName: 'Hello World'
inputs:
targetType: inline
script: |
Write-Host "Hello World"
There's nothing in your yaml which defines the start of the stage, or declares the start of the list of jobs; that's why it wasn't expecting to find 'job' there. You can add those parts in, like this:
parameters:
- name: stageObjs
type: stageList
default: []
stages:
- ${{ each stage in parameters.stageObjs }}:
- stage: ${{ stage.stage }}
displayName: ${{ stage.displayName }}
jobs:
- ${{ each job in stage.jobs }}:
${{ job }}
I use Azure Pipelines to Build my solution. When building manually the user can decide which Build Configuration to use. Now I want the default value (and so the value when triggered automatically) to be different according to the branch. This is my (non-working) approach:
name: PR-$(Build.SourceBranchName)-$(Date:yyyyMMdd)$(Rev:.r)
trigger:
- develop
- test
- master
pool:
vmImage: 'windows-latest'
parameters:
- name: BuildConfiguration
displayName: Build Configuration
${{ if eq(variables['Build.SourceBranch'], 'refs/heads/develop') }}:
default: Debug
${{ if eq(variables['Build.SourceBranch'], 'refs/heads/test') }}:
default: Release
${{ if eq(variables['Build.SourceBranch'], 'refs/heads/master') }}:
default: Release
values:
- Debug
- Release
The conditional part (${{...}) simply does not work. (*"A template expression is not allowed in this context")
Is there any other approach to make this happen?
This is not posssible to use expressins to select default value for paramataers.
Paramaters alco can't be optionla:
Parameters must contain a name and data type. Parameters cannot be optional. A default value needs to be assigned in your YAML file or when you run your pipeline. If you do not assign a default value or set default to false, the first available value will be used.
What you can do is use parameters and variables:
name: PR-$(Build.SourceBranchName)-$(Date:yyyyMMdd)$(Rev:.r)
trigger:
- develop
- test
- master
pool:
vmImage: 'windows-latest'
parameters:
- name: BuildConfiguration
displayName: Build Configuration
default: Default
values:
- Debug
- Release
- Default
variables:
- name: buildConfiguration
${{ if and(eq(variables['Build.SourceBranch'], 'refs/heads/develop'), eq(parameters.BuildConfiguration, 'Default')) }}:
value: Debug
${{ elseif and(eq(variables['Build.SourceBranch'], 'refs/heads/test'), eq(parameters.BuildConfiguration, 'Default')) }}:
value: Release
${{ elseif and(eq(variables['Build.SourceBranch'], 'refs/heads/master'), eq(parameters.BuildConfiguration, 'Default')) }}:
value: Release
${{ else }}:
value: ${{parameters.BuildConfiguration}}
steps:
- script: |
echo $(buildConfiguration)
echo $(Build.SourceBranch)
I am trying to select a service connection based on if else logic.
I have following two pipelines azure-caller.yml and template.yaml.
azure-caller.yml
parameters:
- name: directory
displayName: 'select appropriate environment directory'
values:
- Dev
- Stage
stages:
- template: job-templates/template.yml
parameters:
selectdirectory: $(select-directory)
template.yml
parameters:
selectdirectory: default
stages:
- stage: A
variables:
- name: service-connection
${{ if eq(parameters.selectdirectory, 'Dev') }}:
value: Dev-Service-connection
${{ elseif eq(parameters.selectdirectory, 'Stage') }}:
value: Stage-Service-connection
jobs:
- job: Install_terraform
displayName: Intialize Terraform
continueOnError: true
steps:
- checkout: self
- task: TerraformInstaller#0
inputs:
terraformVersion: '1.0.5'
- job: Validate_terraform_Plan
displayName: Validate Terraform
continueOnError: true
dependsOn: Install_terraform
steps:
- checkout: self
- task: TerraformTaskV2#2
inputs:
provider: 'azurerm'
command: 'init'
backendServiceArm: ${{ variables.service-connection }}
backendAzureRmResourceGroupName: 'any-rg'
backendAzureRmStorageAccountName: 'any-storage'
backendAzureRmContainerName: 'statefile'
backendAzureRmKey: abcd.tfstate
- task: TerraformTaskV2#2
displayName: 'plan'
inputs:
provider: 'azurerm'
command: 'plan'
commandOptions: '-out="plan.out"'
environmentServiceNameAzureRM: ${{ variables.service-connection }}
My tasks are failing because it fetches the value of the service connection at compile time, which is empty.
Azure Caller is a wrapper/main pipeline, and the template is the only one that is allowed to use Dev or Stage service connections for deployment purposes to achieve security. Is there a way to accomplish this?
Your input will be helpful. thx
It is not supported to dynamically specify value of pipeline variable conditionally.
In the YMAL pipeline, you can use the if conditional to judge using or not using a pipeline variable. You can not use it to judge specifying a value or another one to a pipeline variable.
However, you can dynamically set the input conditionally on a task. See "Conditionally set a task input".
You also can dynamically specify value of parameter conditionally.
Below is an example to dynamically specify value of parameter conditionally. You can reference it to change your YAML files.
template.yml
parameters:
- name: ServiceConnection
type: string
stages:
- stage: A
displayName: 'Stage A'
jobs:
- job: A1
displayName: 'Job A1'
pool:
vmImage: ubuntu-latest
steps:
- task: Bash#3
displayName: 'Show parameter value'
inputs:
targetType: inline
script: echo "The value of ServiceConnection is ${{ parameters.ServiceConnection }}"
azure-pipelines.yml
parameters:
- name: directory
displayName: 'Select appropriate environment directory:'
type: string
values:
- Dev
- Stage
stages:
- template: template.yml
parameters:
${{ if eq(parameters.directory, 'Dev') }}:
ServiceConnection: Dev-Service-connection
${{ if eq(parameters.directory, 'Stage') }}:
ServiceConnection: Stage-Service-connection
Result
[UPDATE]
Dynamically set the input conditionally on the task.
template.yml
parameters:
- name: selectdirectory
type: string
stages:
- stage: A
displayName: 'Stage A'
jobs:
- job: Validate_terraform_Plan
displayName: 'Validate Terraform'
. . .
steps:
. . .
- task: TerraformTaskV2#2
inputs:
provider: 'azurerm'
command: 'init'
${{ if eq(parameters.selectdirectory, 'Dev') }}:
backendServiceArm: Dev-Service-connection
${{ if eq(parameters.selectdirectory, 'Stage') }}:
backendServiceArm: Stage-Service-connection
backendAzureRmResourceGroupName: 'any-rg'
backendAzureRmStorageAccountName: 'any-storage'
backendAzureRmContainerName: 'statefile'
backendAzureRmKey: abcd.tfstate
. . .
azure-pipelines.yml
parameters:
- name: directory
displayName: 'Select appropriate environment directory:'
type: string
values:
- Dev
- Stage
stages:
- template: template.yml
parameters:
selectdirectory: ${{ parameters.directory }}
Have you checked this answer out: AzureDevops: Can we pass dynamic value as service connection in yaml
Essentially rather then pass in your service connection name via an if statement have the service connection name stored in a variable.yml file that is determined off your selected directory and will load the appropriate service connection name.
We have two environments defined, "stage" and "prod". Prod requires approval for deploying, where as stage does not.
What I'm trying to achieve is that if the pipeline runs against the master branch, it should use the prod environment, and all other branches should use the stage environment.
Problem is that I can't seem to be able to change the environment in runtime. I've also tried using templates, which sort of works, except that I can't pass runtime information as parameters to the template.
I also tried using two deployment jobs, one for each environment and then chose job based on conditions, however, as soon as the pipeline involves the prod environment, whether it's going to be used or not, it will ask for approval.
I will start with first last one:
I also tried using two deployment jobs, one for each environment and then chose job based on conditions, however, as soon as the pipeline involves the prod environment, whether it's going to be used or not, it will ask for approval.
Resource restriction are evaluated on stage level so if you put those deployment jobs on the same stage (or you didn't define a stage - then you have implicit stage) - this is the reason why it asks for approval. If you move deployment jobs to seprate stages you won't be asked for approval. Please keep in mind to move condition to a stage level.
Another apporach it would be conditional for envrionment as follows:
jobs:
- deployment: DeployWeb
displayName: deploy Web App
${{ if eq(variables['Build.SourceBranchName'], 'master') }}:
environment: 'Dev'
${{ if ne(variables['Build.SourceBranchName'], 'master') }}:
environment: 'campo-dev'
strategy:
# Default deployment strategy, more coming...
runOnce:
deploy:
steps:
- checkout: self
- script: echo my first deployment
I do this with an extends yaml. You can define your stages via the environment's parameter (defaulted) and then your calling yaml extends it and passes tasks for build and deploy. The build section is ran once. And the deploy section will loop for each environment.
The approvals comes from the Environment approvals setup in Devops
#Extends Yaml
parameters:
- name: buildSteps
type: stepList
default: []
- name: deploySteps
type: stepList
default: []
- name: environments
type: object
default:
- name: Stage
branch: any
dependsOn:
- Builds
- name: Prod
branch: any
dependsOn:
- Stage
stages:
- stage: Builds
jobs:
- job: Building
steps:
- ${{ each step in parameters.buildSteps }}:
- ${{ each pair in step }}:
${{ pair.key }}: ${{ pair.value }}
- ${{ each env in parameters.environments }}:
- stage: ${{ env.name }}
dependsOn: ${{ env.dependsOn }}
condition: and(succeeded(), or(startsWith(variables['Build.SourceBranch'], 'refs/heads/${{ env.branch }}'), eq('${{ env.branch }}', 'any')))
jobs:
- deployment: Deploy${{ env.name }}
displayName: Deploy to ${{ env.name }}
variables:
- group: ENV-${{ env.name }}
environment: ${{ env.name }}
strategy:
runOnce:
deploy:
steps:
- ${{ each step in parameters.deploySteps }}:
- ${{ each pair in step }}:
${{ pair.key }}: ${{ pair.value }}
#Calling Yaml
#*Trimmed* header stuff
resources:
repositories:
- repository: cicd
type: git
name: SharedRepo
ref: main
extends:
template: extends-deploy.yaml#cicd
parameters:
buildSteps:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: Write-Host 'Build Stuff 1'
- task: PowerShell#2
inputs:
targetType: 'inline'
script: Write-Host 'Build Stuff 2'
deploySteps:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: Write-Host 'Deploy Stuff 1'
- task: PowerShell#2
inputs:
targetType: 'inline'
script: Write-Host 'Deploy Stuff 2'
I'm trying to pass stage list from azure pipeline shown below
# File: azure-pipelines.yml
trigger:
- master
extends:
template: start_stage.yml
parameters:
cdstages:
- stage: secure_buildstage
pool: Hosted VS2017
jobs:
- job: secure_buildjob
steps:
- bash: echo This happens before code
displayName: 'Base: Pre-build'
- bash: echo Building
displayName: 'Base: Build'
- bash: echo This happens after code
displayName: 'Base: Signing'
- stage: secure_deploystage
pool: Hosted VS2017
jobs:
- job: secure_deployjob
steps:
- bash: echo This happens before code
displayName: 'Base: Pre-build'
- bash: echo Building
displayName: 'Base: Build'
- script: echo This happens after code
displayName: 'Base: Signing'
to extend template shown below
parameters:
- name: cdstages # the name of the parameter is buildSteps
type: stageList # data type is StepList
default: [] # default value of buildSteps
stages:
- ${{ each stage in parameters.cdstages }}:
- ${{ each job in stage.jobs }}:
- ${{ each step in job.steps }}:
- ${{ each pair in step }}:
${{ if ne(pair.value, 'CmdLine#2') }}:
${{ pair.key }}: ${{ pair.value }}
${{ if eq(pair.value, 'CmdLine#2') }}:
'${{ pair.value }}': error
The goal is to take stage list and validate if users are only running steps approved by firm's compliance team.
I'm getting error
[][1
not sure why getting "task" error, there are no task keywords used anywhere.
Any help?
Based on my test , it seems that the stagelist in start_stage.yml doesn't support to add eachdirective to get deeper content (e.g. job and steps).
When you use the stagelist, it could get the stage and use it for comparison.
For example:
parameters:
- name: cdstages # the name of the parameter is buildSteps
type: stageList # data type is StepList
default: [] # default value of buildSteps
stages:
- ${{ each stage in parameters.cdstages }}:
- ${{ each pair in stage }}:
${{ if ne(pair.value, 'abc') }}:
${{ pair.key }}: ${{ pair.value }}
${{ if eq(pair.value, 'abc') }}:
'${{ pair.value }}': error
This Yaml template could work.
But when I add the each directive behind the stage to get the jobs. The jobs are is not available.
parameters:
- name: cdstages
type: stageList
default: []
stages:
- ${{ each stage in parameters.cdstages }}:
- ${{ each job in stage.jobs }}:
- ${{ each pair in job }}:
${{ if ne(pair.value, 'abc') }}:
${{ pair.key }}: ${{ pair.value }}
${{ if eq(pair.value, 'abc') }}:
'${{ pair.value }}': error
According to your requirements, you need to get the build step and use it for comparison.
You could try to directly use the steplist type.
Here is an example about steplist, you could refer to it.
Hope this helps.