I have the following YAML code:
name: $(major).$(minor).$(Date:yyMM).$(Rev:r)
resources:
repositories:
- repository: repoxyz
type: git
name: OneXXXXX/XYZ
ref: develop
trigger:
- develop
- main
- users/*
pool:
vmImage: "windows-latest"
variables:
"major": '2'
"minor": '0'
"BuildConfiguration": 'debug'
stages:
- template: build-services.yml#repoxyz
parameters:
repoToCheckout: repoxyz
checkoutPath: '$(Build.BuildNumber)'
- template: build-functionapps.yml#repoxyz
parameters:
repoToCheckout: repoxyz
checkoutPath: '$(Build.BuildNumber)'
- template: deploy-pipeline.yml
parameters:
solutionAbbreviation: '$(SolutionAbbreviation)'
environmentAbbreviation: 'int'
- template: deploy-pipeline.yml
parameters:
solutionAbbreviation: '$(SolutionAbbreviation)'
environmentAbbreviation: 'ua'
- template: deploy-pipeline.yml
parameters:
solutionAbbreviation: '$(SolutionAbbreviation)'
environmentAbbreviation: 'prod'
Here I checkout another repository repoxyz and run build-services.yml and build-functionapps.yml in repoxyz. Then I deploy to int, ua, prod using deploy-pipeline.yml in the current repository. I use ref: develop. Is it possible to parameterize ref such that for int, use develop and for ua/prod, use main branch?
For example:
${{ if in(parameters.environmentAbbreviation, 'prod', 'ua') }}:
ref: main
${{ else }}:
ref: develop
Related
I have this problem. When trying to use variable groups with conditional templates pipeline gives me this error:
templates/vars-qa.yml (Line: 9, Col: 1): While parsing a block mapping, did not find expected key.
Here is the azure-pipelines.yml
trigger:
- main
- qa
- development
- staging
#//
resources:
- repo: self
variables:
- ${{ if eq(variables['Build.SourceBranch'], 'refs/heads/main') }}:
- template: templates/vars-qa.yml
- ${{ if eq(variables['Build.SourceBranch'], 'refs/heads/qa') }}:
- template: templates/vars-qa.yml
- ${{ if eq(variables['Build.SourceBranch'], 'refs/heads/development') }}:
- template: templates/vars-dev.yml
- ${{ if eq(variables['Build.SourceBranch'], 'refs/heads/staging') }}:
- template: templates/vars-qa.yml
stages:
- template: templates/transform-settings.yml
- template: templates/build-image.yml
And vars-qa.yml where error is happen:
variables:
imageRepository: 'service-qa'
dockerRegistryServiceConnection: 'dec124f0-814f-4511-94d3-b3396698767508'
containerRegistry: 'test.azurecr.io'
imagePullSecret: 'test1334fe31-auth'
dockerfilePath: '**/Dockerfile'
vmImageName: 'ubuntu-latest'
tag: '$(Build.BuildId)'
- group: dev-secrets
- name: ConnectionStrings.DefaultConnection
value: $(psql-conn-str-dev)
This is happen in -group: dev-secrets.The group is existing and the psql-conn-str-dev already to.I have tried with $(variables.psql-conn-str-dev) but result was the same. Everything works correctly without templates
There is a problem in the indent of your template YAML.
You cannot define a sequence item when in a mapping.
Please try and use an online YAML validator tool to check this:
https://codebeautify.org/yaml-validator
I converted your variables to sequence items.
I think this will work better:
variables:
- name: imageRepository
value: 'service-qa'
- name: dockerRegistryServiceConnection
value: 'dec124f0-814f-4511-94d3-b3396698767508'
- name: containerRegistry
value: 'test.azurecr.io'
- name: imagePullSecret
value: 'test1334fe31-auth'
- name: dockerfilePath
value: '**/Dockerfile'
- name: vmImageName
value: 'ubuntu-latest'
- name: tag
value: '$(Build.BuildId)'
- group: dev-secrets
- name: ConnectionStrings.DefaultConnection
value: $(psql-conn-str-dev)
I have reusable build template job. In my pipeline, I'm reusing this template every time I want to build new application. Inside this build template, it has inline checkout step. Problem occurs if I checkout same repository and branch.
Template structure
# **pipeline.yaml**
stages:
- template: templates/group-build.yaml
parameters:
branchRef: ${{ variables.branchRef}}
# -----------------------------------------
# **templates/group-build.yaml**
parameters:
- name: branchRef
type: string
stages:
- stage: 'Build'
displayName: 'Build'
variables:
jobs:
- template: template/functionapp-build-job.yaml
parameters:
branchRef: ${{ parameters.branchRef}}
- template: template/webapp-build-job.yaml
parameters:
branchRef: ${{ parameters.branchRef}}
# -----------------------------------------
# **template/functionapp-build-job.yaml**
parameters:
- name: branchRef
type: string
jobs:
- template: templates/function/build-job.yaml
parameters:
repository: repo
branchRef: ${{ parameters.branchRef}}
project: functionapp/function.csproj'
# -----------------------------------------
# **templates/function/build-job.yaml**
parameters:
- name: repository
type: string
- name: project
type: string
- name: componentName
type: string
- name: branchRef
type: string
jobs:
- job: JOB_${{ parameters.componentName}}_BUILD # <- Component is unique
displayName: Build ${{ parameters.componentName}} Job
steps:
// Checking out same repo is not allowed here
- checkout: git://project/${{ parameters.repository }}#${{ parameters.branchRef}}
- Other Task here...
# -----------------------------------------
# **template/webapp-build-job.yaml**
# Same structure as function app...
This is how it looks like structurally
Pipeline
Stages:
- Stage:
Jobs:
- Job: Web App Build
Steps:
- checkout: 'Here I call same repo and branch but different build steps'
- Job: Function App Build
Steps:
- checkout: 'Here I call same repo and branch but different build steps'
Validation issue image
Adding resources in pipeline is not my option because refs is being stored in variable group, which is not support to resolve due to limitation of azure yaml as stated here.
Any suggestion or idea to circumvent this issue is highly appreciated.
From your sample, the template webapp-build-job.yaml and functionapp-build-job.yaml are reference the same yaml：build-job.yaml.
In this case, the job names (- job: JOB_BUILD) are the same key and this can be the root cause of this issue.
To solve this issue, you can define job name as a parameter and assign value in webapp-build-job.yaml and functionapp-build-job.yaml.
For example:
build-job.yaml
parameters:
- name: repository
type: string
- name: project
type: string
- name: jobname
type: string
- name: branchRef
type: string
jobs:
- job: ${{ parameters.jobname }}
displayName: Build Job
steps:
- checkout: git://Artifacts/${{ parameters.repository }}#${{ parameters.branchRef}}
functionapp-build-job.yaml
parameters:
- name: branchRef
type: string
jobs:
- template: build-job.yaml
parameters:
repository: Repo11
branchRef: ${{ parameters.branchRef}}
jobname: test1
project: functionapp/function.csproj'
webapp-build-job.yaml
parameters:
- name: branchRef
type: string
jobs:
- template: build-job.yaml
parameters:
repository: Repo11
branchRef: ${{ parameters.branchRef}}
jobname: test2
project: functionapp/function.csproj'
group-build.yaml
parameters:
- name: branchRef
type: string
stages:
- stage: 'Build'
displayName: 'Build'
jobs:
- template: functionapp-build-job.yaml
parameters:
branchRef: ${{ parameters.branchRef}}
- template: webapp-build-job.yaml
parameters:
branchRef: ${{ parameters.branchRef}}
Pipeline.yaml
variables:
- group: test
stages:
- template: group-build.yaml
parameters:
branchRef: ${{ variables.branchRef}}
Can anybody point me in the right direction here?
When I attempt to kick off the pipeline from the main yaml template the reference template gives me the unexpected 'steps' value.
I attempted to add a stage before the defined job and then receive the unexpected 'stage' message. Looking at some similar previously asked questions I understand a step can't be place directly under a 'stage' but this is not the case here.
parameters:
- name: baseEnvironmentName
type: string
- name: environmentSuffix
type: string
- name: postDeploySteps
type: stepList
default: []
- name: publishedPackageName
type: string
- name: serviceName
type: string
- name: dnsHostName
type: string
jobs:
- deployment: ${{ parameters.environmentSuffix }}Deployment
displayName: '${{ parameters.environmentSuffix }} Deployment'
pool:
vmImage: 'windows-2019'
environment: ${{ parameters.baseEnvironmentName }}-${{ parameters.environmentSuffix }}
variables:
- template: variables/variables.common.yaml
- template: variables/variables.${{ parameters.environmentSuffix }}.yaml
- name: MonitorExceptionAlert_Name
value: '${{ variables.IdPrefix }}-${{ parameters.environmentSuffix }}-${{ parameters.dnsHostName }}-ExceptionAlert'
steps:
- checkout: self
- checkout: common_iac
- task: Random Task Name
displayName: 'Create Environment Resource Group'
inputs:
ConnectedServiceName: '$(ADOS.ServiceConnectionName)'
resourceGroupName: '$(ResourceGroup.Name)'
location: '$(ResourceGroup.Location)'
condition: and(succeeded(), ne(variables['CreateResourceGroupInTaskGroup'], 'false'))
- task: Random Task Name 2
displayName: 'Create Application Insights'
inputs:
ConnectedServiceName: '$(ADOS.ServiceConnectionName)'
ResourceGroupName: '$(ResourceGroup.Name)'
Location: '$(ResourceGroup.Location)'
instanceName: '$(ApplicationInsights.Name)'
You are using deployment jobs. Try defining a strategy. Try replacing steps: inline with deployment with:
- deployment: ....
strategy:
runOnce:
deploy:
steps:
From the example at the Azure docs templates page, jobs: should contain a - job: which contains steps:. I don't know that jobs: can directly contain steps:.
The dynamic variables with the checkout step do not work with the template.
The same works without a template.
#- checkout: git://MY_PROJ/MY_REPO#refs/tags/${{parameters.tag_name}} ## DOES NOT WORK
Getting below error while running pipeline:
"ERROR: An item with the same key has already been added"
Working:
my_pipeline.yml without template works
parameters:
- name: RELEASE_TAG
displayName: Enter The Master Release Tag Name Example 1.0.0-RELEASE
default: 1.0.0-RELEASE
type: string
trigger:
- none
variables:
db_resource_path: '$(System.DefaultWorkingDirectory)/resources/db'
pipeline_environment_name: 'PROD_ENV'
db_env_name: 'prod'
stages:
- stage: "PROD_DB_DEPLOYMENT"
displayName: "PROD DB Deployment"
pool:
name: $(param.agent.pool.name)
variables:
- group: PROD_VG
jobs:
- job:
steps:
- script: |
echo param release tag: ${{parameters.RELEASE_TAG}}
- deployment: Deploy
environment: PROD_ENV
strategy:
runOnce:
deploy:
steps:
- checkout: git://MY_PROJ/MY_REPO#refs/tags/${{parameters.RELEASE_TAG}} ## WORKS
#- checkout: git://MY_PROJ/MY_REPO#refs/tags/${{variables.tag_name}} ## WORKS WITH VAR ALSO
Not Working:
Gives ERROR: An item with the same key has already been added.
my azure-pipelines.yml is:
parameters:
- name: RELEASE_TAG
displayName: Enter The Master Release Tag Name Example 1.0.0-RELEASE
default: 1.0.0-RELEASE
type: string
resources:
repositories:
- repository: templates
type: git
name: MY_PROJECT/MY_TEMPLATE_REPO
trigger:
- none
variables:
tagName: '${{ parameters.RELEASE_TAG }}'
stages:
- stage: "PROD_DB_DEPLOYMENT"
displayName: "PROD DB Deployment"
variables:
- group: PROD_VG
jobs:
- template: my_template.yml#templates
parameters:
tag_name: $(tagName)
db_env_name: 'prod'
agent_pool_name: $(param.agent.pool.name)
db_resource_path: $(System.DefaultWorkingDirectory)/resources/db
pipeline_environment_name: PROD_ENV
is_release: 'true'
My Template is:
parameters:
- name: 'agent_pool_name'
type: string
- name: 'db_resource_path'
type: string
- name: 'pipeline_environment_name'
type: string
- name: 'db_env_name'
type: string
- name: 'is_release'
type: string
default: 'false'
- name: 'tag_name'
type: string
default: '1.0.0-RELEASE'
jobs:
- job:
pool:
name: ${{parameters.agent_pool_name}}
steps:
- script: |
echo param tag_name: ${{parameters.tag_name}}
echo var tag_name: $(tagName)
- deployment: Deploy
pool:
name: ${{parameters.agent_pool_name}}
environment: ${{parameters.pipeline_environment_name}}
strategy:
runOnce:
deploy:
steps:
- checkout: self
- ${{ if eq(parameters.is_release, true) }}:
#- checkout: git://MY_PROJ/MY_REPO#refs/tags/1.0.0-RELEASE ## WORKS
#- checkout: git://MY_PROJ/MY_REPO#refs/tags/$(tag_name) ## DOES NOT WORK
#- checkout: git://MY_PROJ/MY_REPO#refs/tags/${{parameters.tag_name}} ## DOES NOT WORK
Tried below variable option also but get error:Unexpected value 'variables'
Any suggestion, please.
This is due to that you passed $(var) as parameters of the template.
You can pass ${{variables.tagName}} instead.
checked on my side screenshot
In a pipeline, template expression variables (${{ variables.var }}) get processed at compile time, before runtime starts. Macro syntax variables ($(var)) get processed during runtime before a task runs.
Because templates are expanded before the pipeline execution gets planned, so you cannot pass $(var) as parameter to the template.
Please check official doc for the variable syntax.
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch#understand-variable-syntax
Edit: for error "Unexpected value 'variables'":
Move variables to stage scope: my yaml here
My template here
I was trying to use a variable as tag while executing the deploy job using the environment template. But from https://developercommunity.visualstudio.com/content/problem/1015274/yaml-variable-names-not-expanded-for-tags-on-envir.html, it's evident that ADO doesn't support variables as tags in environment deployments. For example, this works
variables:
envName: 'EnvironmentName'
vmTag: 'vm-test-01'
environment:
name: "EnvironmentName"
resourceName: 'vm-test-01'
resourceType: "virtualMachine"
tags: "vm01"
strategy:
runOnce:
deploy:
steps:
- powershell: write-host "This is $(VMName)"
but this doesn't
name: "$(envName)"
resourceType: "virtualMachine"
tags: "$(vmTag)"
strategy:
runOnce:
deploy:
steps:
- powershell: write-host "This is $(VMName)"
Does anyone know of an alternate or a workaround for this?
Thanks
UPDATE: This is what I've tried
multistage-pipeline.yml
- stage: "deploytouservm"
displayName: "Stage - Deploy To User VM"
dependsOn: "build"
variables:
- name: envName
value: "environmentName"
- name: userVM
value: "vmName01" #this value is dynamically generated
- template: templates\jobs\deploy-template.yml
parameters:
envName: $(envName)
vmName: $(userVM)
deploy-template.yml
parameters:
- name: envName
- name: vmName
jobs:
- deployment: "deployJob"
environment: ${{ parameters['envName']}}.${{ parameters['vmName']}}
displayName: "Deploy - SCOM To User VM"
strategy:
runOnce:
deploy:
steps:
Try using parameters instead of variables:
parameters:
- name: envName
default: EnvironmentName
jobs:
- deployment: VMDeploy
displayName: Test_script
environment:
name: ${{ parameters.envName}}
resourceType: VirtualMachine
azure-pipelines.yml
stages:
- stage:
variables:
EnvironmentName: Prod
jobs:
- template: steps.yml
parameters:
myParameter: Test
....
steps:yml
parameters:
- name: myParameter
jobs:
- deployment: deployment
environment: ${{ parameters['myParameter']}}
strategy:
...