Parse Set variables output to template - Azure yml pipeline - azure-devops

I have a pipeline that runs a template pipeline. It looks like this:
resources:
repositories:
- repository: repoName
type: git
name: projectName/repoName
ref: branchName
stages:
- stage: GetLastCommitId
jobs:
- job: lastCommitId
steps:
- checkout: repoName
- bash: |
cd repoName
echo "##vso[task.setvariable variable=commitId;isOutput=true]$(git rev-parse HEAD)"
name: a
- bash: |
echo $(a.commitId)
- checkout: self
- template: templates/bicep.yml
parameters:
environment: dev
lastCommitId: $[stageDependencies.GetLastCommitId.lastCommitId.outputs['a.commitId']]
bash returns me the required ID. Everything as it is meant to be.
Now I want to pass this output value to the template as parameter - lastCommitId.
Is there a way to do this?
Here's what I tried:
- template: templates/bicep.yml
parameters:
environment: dev
lastCommitId: $(a.commitId)
Error: Empty string
- template: templates/bicep.yml
dependsOn: GetLastCommitId
parameters:
environment: dev
lastCommitId: $[stageDependencies.GetLastCommitId.lastCommitId.outputs['a.commitId']]
Error: Can not start pipeline. dependsOn not expected here
- template: templates/bicep.yml
parameters:
environment: dev
dependsOn: GetLastCommitId
lastCommitId: $[stageDependencies.GetLastCommitId.lastCommitId.outputs['a.commitId']]
Error: syntax error: invalid arithmetic operator (error token is ".GetLastCommitId.lastCommitId.outputs['a.commitId']
syntax according to microsoft doc: $[stageDependencies.A.A1.outputs['MyOutputVar.myStageVal']]
Here is documentation from Microsoft: https://learn.microsoft.com/en-us/azure/devops/pipelines/process/set-variables-scripts?view=azure-devops&tabs=bash

The value of the parameter your are trying to set is done before the actually run of the tasks. Hence, the lastCommitId is not set as the value and is resulting in your different approaches into a dissapointed result.
But, depending on what is in your bicep.yml, there is a solution!
With the azure-pipeline.yml and bicep.yml below, you are able to use your lastCommitId in the template:
trigger:
- main
pool:
vmImage: ubuntu-latest
resources:
repositories:
- repository: repoName
type: git
name: projectName/repoName
ref: branchName
stages:
- stage: GetLastCommitId
jobs:
- job: lastCommitId
steps:
- checkout: repoName
- bash: |
cd repoName
echo "##vso[task.setvariable variable=commitId;isOutput=true]$(git rev-parse HEAD)"
name: a
- bash: |
echo $(a.commitId)
- checkout: self
- template: templates/bicep.yml
And the bicep.yml:
stages:
- stage: Template
jobs:
- job: JobInTemplate
variables:
lastCommitId: $[stageDependencies.GetLastCommitId.lastCommitId.outputs['a.commitId']]
steps:
- script: echo the value is $(lastCommitId)

Related

Azure DevOps YAML - if statement not working through template with number/int

Checking a version number in a yaml pipeline with an if statement and wanting to run a particular task dependent on version. Works fine directly but I'm using templates and passing the value through it doesn't seem to work.
This works:
trigger:
- main
pool:
vmImage: ubuntu-latest
parameters:
- name: TF_VERSION
default: 1.3.3
steps:
- ${{ if lt(parameters.TF_VERSION, 1) }}:
- powershell: Write-Host "I'm running lt 1.0"
- ${{ if ge(parameters.TF_VERSION, 1) }}:
- powershell: Write-Host "I'm running ge 1.0"
This doesn't with the first condition always returning true:
trigger:
- main
pool:
vmImage: ubuntu-latest
resources:
repositories:
- repository: templates
type: git
name: YAML-Templates
ref: refs/heads/main
variables:
- name: TF_VERSION
value: 1.3.3
steps:
- template: if/if.yml#templates
parameters:
TF_VERSION: $(TF_VERSION)
template:
parameters:
- name: TF_VERSION
steps:
- ${{ if lt(parameters.TF_VERSION, 1) }}:
- powershell: Write-Host "I'm running lt 1.0"
- ${{ if ge(parameters.TF_VERSION, 1) }}:
- powershell: Write-Host "I'm running ge 1.0"
Anyone know why???
I can reproduce your problem.
I can partly solve it by adding the mandatory type to the parameter in the template:
parameters:
- name: TF_VERSION
type: number
steps:
- ${{ if lt(parameters.TF_VERSION, 1) }}:
- powershell: Write-Host "I'm running lt 1.0"
- ${{ if ge(parameters.TF_VERSION, 1) }}:
- powershell: Write-Host "I'm running ge 1.0"
The only down side seems to be that I'm not able to pass a variable from the calling YAML due to this error.
This issue can be overcome by setting the number directly in the parameter like this:
trigger:
- main
pool:
vmImage: ubuntu-latest
resources:
repositories:
- repository: templates
type: git
name: YAML-Templates
ref: refs/heads/main
steps:
- template: if/if.yml#templates
parameters:
TF_VERSION: 1.5
Like you mentioned, 1.3.3, a version number, is still not a number.
Alternative
Alternatively you could go for startsWith like so:
parameters:
- name: TF_VERSION
steps:
- ${{ if startsWith(parameters.TF_VERSION, '0.') }}:
- powershell: Write-Host "I'm running 0.x"
- ${{ if startsWith(parameters.TF_VERSION, '1.') }}:
- powershell: Write-Host "I'm running 1.x"
I hope this answers the question, why it is not working and what a possible alternative is.
Edit-2
The solution of Antonia Wu-MSFT's also work, the why is still a bit unclear. But I think it has to do with: https://learn.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch#template-expression-syntax
But to be honest, still don't understand why a type is mandatory according to the documentation, but not used in this case.
I have tried only to change the TF_VERSION: $(TF_VERSION) to TF_VERSION: ${{ variables.TF_VERSION }} in the main yaml.
trigger:
- main
pool:
vmImage: ubuntu-latest
resources:
repositories:
- repository: templates
type: git
name: YAML-Templates
ref: refs/heads/main
variables:
- name: TF_VERSION
value: 1.3.3
steps:
- template: /if/if.yml#templates
parameters:
TF_VERSION: ${{ variables.TF_VERSION }}
Then run the pipeline with the template it seems work and get this:
Please kindly try whether it works on your side. And hope it could do some help:>

Azure Devops pipeline can't find a checkouted template

My issue
I have an Azure devops project with a template on another repository than main one's.
I try to load the template but I get this:
/azure-pipelines.yml: File /TemplateRepository/build.yml not found in
repository
https://dev.azure.com/XXXXX/TestAzureDevops/_git/TemplateRepository
branch refs/heads/main
What I did
build.yml is my template into repository: TemplateRepository.
azure-pipelines.yml is my main file.
this is my template:
parameters:
- name: 'name'
default: 'name'
type: 'string'
steps:
- script: echo ${{parameters.name}}
This is my main:
pool:
vmimage: 'windows-latest'
parameters:
- name: contexts
type: object
default: [{
name: macOS,
pool: 'macOS-latest',
sign: false
},
{
name: Windows,
pool: 'windows-latest',
sign: true
}]
resources:
repositories:
- repository: Tuto-Ressources
ref: main
type: git
name: TemplateRepository
stages:
- stage: BUILD
jobs:
- job: test
steps:
- checkout: self
- checkout: Tuto-Ressources
- script: dir
displayName: Dir
- ${{ each context in parameters.contexts }}:
- template: .\TemplateRepository\build.yml#Tuto-Ressources
parameters:
name: ${{context.name}}
pool: ${{context.pool}}
sign: ${{context.sign}}
buildSteps:
- checkout: self
- checkout: Tuto-Ressources
- bash: echo "Test module"
What I tested
If I remove the template lines from main script so I just have script: dir I can see my checkout on the VM and build.yml into \TemplateRepository directory.
So there is no reason it can't find my template.
I checked the branch name, it's main and I have no other one. I can see the file from the Devops Portal
What I need
I would like to understand what happen and what I can do.
I know there are artifact, but in this situation I don't understand whay it is not working like this because I don't change job neither stage.
The template line should reference the template file by its path relative to TemplateRepository. If your build template resides at \build.yml in TemplateRepository, then you should just do:
- template: build.yml#Tuto-Ressources
Also, you don't need to do - checkout: Tuto-Ressources - the pipeline engine will know to fetch the template file from referenced repository.

Can't run a templated yaml Azure Devops pipeline with dynamically created stage

My problem
I try to create templated yaml Azure Devops pipeline:
parameters:
- name: envs
type: object
default:
- QUAL
- PROD
- PREPROD
stages:
- template: Orchestrator.yml
parameters:
name: envs
type: object
default:
- QUAL
- PROD
- PREPROD
This is my template:
parameters:
envs: {}
stages:
- ${{ each env in parameters.envs }}:
- stage: ${{ env }}
jobs:
- job: Deploy
steps:
- script: echo Deploy project
displayName: 'Deploy'
- job: Tests
steps:
- script: echo Unit tests
displayName: Test 1
But I get this error message:
An error occurred while loading the YAML build pipeline. The array
must contain at least one element. Parameter name: stages
OK, I modify my main script like this:
parameters:
- name: envs
type: object
default:
- QUAL
- PROD
- PREPROD
stages:
- stage: Build
jobs:
- job: Build
steps:
- script: echo Compilation completed...
displayName: 'Compile'
- template: Orchestrator.yml
parameters:
name: envs
type: object
default:
- QUAL
- PROD
- PREPROD
This time pipeline runs, but only first Job. The Template is not loaded.
What I need
I was able to make this scenario working with a single file script, but I would like to make it working with a templated script.
Is this scenario supported? How I can do?
thanks
Here I have a sample as reference:
In the template YAML file (here I name it template.yaml), write as this.
parameters:
- name: envs
type: object
default:
- QUAL
- PROD
- PREPROD
stages:
- ${{ each env in parameters.envs }}:
- stage: ${{ env }}
displayName: 'Stage ${{ env }}'
jobs:
- job: job1
displayName: 'Job 1'
steps:
- bash: echo "Current job is job1 in ${{ env }}"
- job: job2
displayName: 'Job 2'
steps:
- bash: echo "Current job is job2 in ${{ env }}"
In the pipeline YAML (here I name it pipeline.yaml), write as this.
trigger:
- main
extends:
template: template.yaml
Result
If you do not want to hardcode the value of the parameter 'envs' in template.yaml, you can write like as below.
In template.yaml write as this.
parameters:
- name: envs
type: object
default: []
stages:
- ${{ each env in parameters.envs }}:
- stage: ${{ env }}
displayName: 'Stage ${{ env }}'
jobs:
- job: job1
displayName: 'Job 1'
steps:
- bash: echo "Current job is job1 in ${{ env }}"
- job: job2
displayName: 'Job 2'
steps:
- bash: echo "Current job is job2 in ${{ env }}"
In pipeline.yaml write as this.
trigger:
- main
stages:
- template: template.yaml
parameters:
envs:
- QUAL
- PROD
- PREPROD
Result. Same as above.
To view more details, you can see "YAML schema reference".

Deselect Stages By Default

In Azure Devops multistage YAML pipeline we got multiple environments.
In stages to run normally we do a build and deploy only in QA, so we need to deselect each stage manually. By default all stages are selected is it possible to have exact opposite, where all stages are deselected by default???
trigger: none
pr: none
stages:
- stage: 'Build'
jobs:
- deployment: 'Build'
pool:
name: Default
# testing
environment: INT
strategy:
runOnce:
deploy:
steps:
- checkout: none
- powershell: |
echo "Hello Testing"
Start-Sleep -Seconds 10
- stage: 'Sandbox'
jobs:
- job: 'Sandbox'
pool:
name: Default
steps:
- checkout: none
# testing
- powershell: |
echo "Hello Testing"
- stage: 'Test'
jobs:
- job: 'DEV'
pool:
name: Default
steps:
- checkout: none
- powershell: |
echo "Hello Testing"
- stage: 'QA'
dependsOn: ['Test','Test1','Test2']
jobs:
- job: 'QA'
pool:
name: Default
steps:
- checkout: none
# Testing
- powershell: |
echo "Hello Testing"
I am afraid that there is no UI (like stage to run) method that can meet your needs.
You could try to add parameters to your Yaml Sample.
Here is an example:
trigger: none
pr: none
parameters:
- name: stageTest
displayName: Run Stage test
type: boolean
default: false
- name: stageBuild
displayName: Run Stage build
type: boolean
default: false
stages:
- ${{ if eq(parameters.stageBuild, true) }}:
- stage: 'Build'
jobs:
- deployment: 'Build'
pool:
name: Default
environment: INT
strategy:
runOnce:
deploy:
steps:
- checkout: none
- powershell: |
echo "Hello Testing"
Start-Sleep -Seconds 10
- ${{ if eq(parameters.stageTest, true) }}:
- stage: Test
dependsOn: []
jobs:
- job: B1
steps:
- script: echo "B1"
The parameters are used to determine whether to run these stages. You could add expressions before the stage to check if the parameter value could meet expression.
The default value is false. This means that the stage will not run by default.
Here is the result:
You can select the stage you need to run by clicking the selection box.
Update
Workaround has some limitations. When the select stage has depenencies, you need to select all dependent stages to run.
For example:
- stage: 'QA'
dependsOn: ['Test','Test1','Test2']
On the other hand, I have created a suggestion ticket to report this feature request. Here is the suggestion ticket link: Pipeline Deselect Stages By Default You could vote and add comment in it .
I've used this solution to build a nuget-package, and:
always push packages from master
conditionally push packages from other branches
Using GitVersion ensures that the packages from other branches get prerelease version numbers, e.g. 2.2.12-my-branch-name.3 or 2.2.12-PullRequest7803.4. The main branch simply gets 2.2.12, so the master branch is recognized as a "regular" version.
The reason I'm repeating the answer above, is that I chose to make the stage conditional instead of using an if:
trigger:
- master
parameters:
- name: pushPackage
displayName: Push the NuGet package
type: boolean
default: false
stages:
- stage: Build
jobs:
- job: DoBuild
steps:
- script: echo "I'm building a NuGet package (versioned with GitVersion)"
- stage: Push
condition: and(succeeded('build'), or(eq('${{ parameters.pushPackage }}', true), eq(variables['build.sourceBranch'], 'refs/heads/master')))
jobs:
- job: DoPush
steps:
- script: echo "I'm pushing the NuGet package"
Like the other answer, this results in a dialog:
But what's different from the (equally valid) solution with '${{ if }}', is that the stage is always shown (even if it's skipped):

Azure DevOps template as a dependency for a job

I use Azure DevOps Templates in Stage and I want some job to start only when Job from template is completed (dependsOn):
- stage: stage1
jobs:
- job: job1
steps:
- bash: |
...
- template: template1.yml
parameters:
param1: 'val1'
- job: job2
**dependsOn: how to put `template: template1.yml` here?**
steps:
- bash: |
...
How could it be done?
Building on Eric Smith's answer you can pass in the name of the job that the template will depend on as a parameter.
#template1.yml
jobs:
- job: mytemplateJob1
steps:
- script: npm install
#template2.yml
parameters:
DependsOn: []
jobs:
- job: mytemplateJob2
dependsOn: ${{ parameters.DependsOn }}
steps:
- bash: pwd
By setting the default value for DependsOn to [] you ensure that the template will run if no value is passed in for DependsOn but you can optionally create a dependency like this:
stages:
- stage: stage1
jobs:
- template: template1.yml # Template reference
- template: template2.yml
parameters:
DependsOn: 'mytemplateJob1'
You can accomplish this by using the name of the job, as it is defined in your template in the dependsOn.
#template1.yml
jobs:
- job: mytemplateJob
steps:
- script: npm install
and
stages:
- stage: stage1
jobs:
- job: job1
steps:
- bash: pwd
- template: template1.yml # Template reference
parameters:
param: 'val1'
- job: job2
dependsOn: mytemplateJob
steps:
- bash: pwd