I'd like to set up an Azure pipeline using yaml template. What I struggle with is the path triggers definition. I'd like the individual pipelines to start up, only if they actually differ from their earlier versions.
core.yaml is a generic template, which is used across the different projects.
parameters:
- name: projectPath
type: string
trigger:
branches:
include:
- master
- feature/*
paths:
include:
- ${{ parameters.projectPath }} # this is the problematic part
steps:
... # restore, build, test, etc tasks are defined here
build.yaml files are defined for every project. They only extend on core.yaml and supply the
required parameters:
extends:
template: core.yaml
parameters:
projectPath: src/project1
Using the value ${{ parameters.projectPath }} fails as "Template expression is not allowed in the given context". Same occurs if I surround it with quotes.
Using the value $(parameters.projectPath) works, but it triggers even if I change a file outside the defined path.
I also tried using the actual values instead of a parameter (e.g. "src/project1"), but it also triggers for every change, even if I only touch the file "src/project2/foo.md".
The examples I saw did not use triggers within a template. But normally I got a syntax error when the pipeline is triggered (e.g. unexpected value) when I misplaced a node. So it's just my expectation that it should work.
Any suggestions?
This is not possible, because this part
trigger:
branches:
include:
- master
- feature/*
paths:
include:
- ${{ parameters.projectPath }} # this is the problematic part
needs to be known on compilation time, but you provide parameter which is evaluated at runtime.
And this syntax $(parameters.projectPath) doesn't make any sense, and probably is evaluated to empy value and then everything is included into trigger.
Related
I am trying to use variables defined at the root level on a YAML pipeline inside Azure DevOps inside templates via the template syntax, but it seems that the variables are not available inside the templates, but when adding the steps directly to the pipeline, the exact same thing works perfectly.
So with a pipeline snippet like that
variables:
- name: test
value: asdf
stages:
- stage:
jobs:
- job: test_job
steps:
- script: echo "${{ variables.test }}"
- template: ./test.yaml
And a test.yaml like that
jobs:
- job: test
steps:
- script: echo "${{ variables.test }}"
The script inside the test_job job writes out asdf while the job inside the template just resolves to echo "".
Since my understanding of pipeline templates is, that those basically get inserted into the main pipeline, this seems like a bug. Any ideas on how to use the root variables in a template syntax inside templates or why this is not working? (Macro synatx is not an option as I need the variable inside a templated condition like ${{ if eq(variables['test'], 'asdf') }})
For security reasons, we only allow you to pass information into
templated code via explicit parameters.
https://docs.microsoft.com/en-us/azure/devops/pipelines/process/templates?view=azure-devops
The means the author of the pipeline using your template needs to
commit changes where they explicitly pass the needed info into your
template code.
There are some exceptions to this, where the variable is statically
defined in the same file or at pipeline compile time, but generally
speaking, it’s probably better to use parameters for everything that
does not involve system-defined read-only dynamic variable and
custom-defined dynamic output variables.
This behavior is by design, check this thread in the developer community
So you can either pass the variable as a parameter to the template or define a centralized variables file to include in the template like here
Is it possible to have different variable template files that would be used based on the branch being built? I was trying to do something like this:
jobs:
- job: BuildandPublish
variables:
- template: /env/$(Build.SourceBranchName).vars.yml
But that doesn't work, I'm guessing do to the order that those variables are replaced.
Is it possible to have different variable template files that would be used based on the branch being built?
From your YAML sample, you are using the format: $(Build.SourceBranchName). The variable value will be expanded at runtime.
But the template will read the variable at Compile time.
To solve this issue, you need to change the format : ${{ variables['Build.SourceBranchName'] }}
Here is an example:
jobs:
- job: BuildandPublish
variables:
- template: /env/${{ variables['Build.SourceBranchName'] }}.vars.yml
Template:
parameters:
- name: PathPrefix
displayName: 'Path prefix'
type: string
default: ''
steps:
- task: DotNetCoreCLI#2
displayName: 'dotnet restore'
inputs:
command: restore
projects: ${{parameters.PathPrefix}}**/$(Build.DefinitionName).sln
Pipeline:
resources:
repositories:
- repository: devops
name: foo/devops
type: git
ref: master
trigger:
branches:
include:
- refs/heads/*
jobs:
- job: Job_1
displayName: Agent job 1
pool:
vmImage: windows-latest
steps:
- checkout: self
- template: azure/pipelines/pipeline.yaml#devops
parameters:
PathPrefix: $(Build.DefinitionName)
Run error during the restore step:
##[error]No files matched the search pattern.
Even setting verbosityRestore: detailed on the restore step doesn't give me any more information.
If I don't set PathPrefix, it seems to use the default empty string and find the solution file (in some cases). However, if I do set the prefix, which is needed in some repos, it can't find the file. I've tried various ways of referencing the parameter within the template (${{}}, $(), $[] and others) and different ways of specifying it within the pipeline, including hard-coding the path (though I want to use the variable instead), but nothing works.
I thought maybe variables would work instead, so I also tried specifying variables in the pipeline and using them in the template, but that results in the same error. Defining the variable in the template gave me a compilation error for the template (unexpected token 'variable' or something similar).
Look at what you're passing and mentally expand the results.
${{parameters.PathPrefix}}**/$(Build.DefinitionName).sln
If Build.DefinitionName is foo, and you pass that in as PathPrefix, then what you get is:
foo**/foo.sln
It looks like you want an extra forward slash in there, so you get foo/**/foo.sln.
I have repo A that holds pipeline templates. Repo A has the following azure-template.yml:
# Repo A / azure-template.yml
stages:
- stage: ${{ variables.stageName }}
jobs:
- job:
steps:
- task:
A lot of the code in repo A that has the templates refers to variables in the following format:
${{ variables.variableName }}. The variable file is in a different folder in repo A. (e.g. variables/variables.yaml)
Now let's move to repo B. Repo B has my azure-pipeline.yml that needs to build from repo A:
# Repo B / azure-pipeline.yml
resources:
repositories:
- repository: templates
type: git
name: repoA
ref: refs/heads/develop
variables:
- template: variables/variables.yml#templates
stages:
- template: azure-template.yml#templates # Template reference
When I run azure-pipeline.yml, I get the following error:
An error occurred while loading the YAML build pipeline. The string must have at least one character. Parameter name: environmentName
That parameter is not one of mine. I don't have it declared or set anywhere. This tells me it is Azure specific but I have no idea where/why it's there or where it is even set.
How can I run a pipeline with a yaml template that refers to a
variables file in a different repo with Azure DevOps?
You're in correct direction, at least a working direction. It's supported to do that like what you've done above. About the error you got, I assume there could be something wrong with your yaml syntax. You can try following steps to locate the issue:
Copy the content of azure-template.yml and variables.yaml directly into azure-pipeline.yml file, and run the pipeline again to check if the issue persists.
In your Azure-pipeline.yml, specify the trigger and pool.
In azure-template.yml, try replacing the ${{ variables.stageName }} with hard-code value.
This is my first time to see this error message, but according to Parameter name: environmentName.You can also check if Release.EnvironmentName has valid value in one PS task. Hope it helps :)
I'm attempting to create multiple pipelines in Azure DevOps but I would like to reuse the same pipeline YAML file with the differences per environment being loaded from a separate template variables file.
For that purpose I've created two variable files, which are located in the same folder as the pipeline definition:
# vars.dev.yml
variables:
- name: EnvironmentName
value: Development
# vars.prd.yml
variables:
- name: EnvironmentName
value: Production
And the definition of the pipeline is the following:
trigger: none
pr: none
variables:
- name: EnvironmentCode
value: dev
- name: EnvironmentFileName
value: vars.$('EnvironmentCode').yml
stages:
- stage: LoadVariablesPerEnvironment
displayName: Load Variables Per Environment
variables:
- template: $(EnvironmentFileName)
jobs:
- job: ShowcaseLoadedVariables
steps:
- pwsh: Write-Host "Variables have been loaded for the '$ENV:ENVIRONMENTNAME' environment"
displayName: Output Environment Variables
After importing the pipelines using the Azure DevOps UI, I can go to settings of each and set the Environment Code variable to whatever desired environment code:
However I'm always getting the same error when I try to run the pipeline, regardless of the code I fill in the variable value:
So the question here is: Is this kind of variable expansion not supported or is there a different way that I should use to accomplish this?
Thanks!
EDIT
I was able to expand the variables using another method. The new version of the pipeline is as such:
variables:
- name: EnvironmentCode
value: dev
- name: EnvironmentFileName
value: vars.${{ variables.EnvironmentCode }}.yml
stages:
- stage: LoadVariablesPerEnvironment
displayName: Load Variables Per Environment
variables:
- template: ${{ variables.EnvironmentFileName }}
jobs:
- job: ShowcaseLoadedVariables
steps:
- pwsh: Write-Host "Variables have been loaded for the '$ENV:ENVIRONMENTNAME' environment"
displayName: Output Environment Variables
However there is yet the issue of loading different files. I made different attempts and verified the following:
If you give a different environment code using the UI, when running
the pipeline, the value it assumes is still the one that's on the
pipeline definition;
If you remove from the pipeline definition the
default value or the variable entirely the expression
${{variables.EnvironmentCode}} will return an empty string
assuming the filename to be vars..yml which doesn't exist.
Is this kind of variable expansion not supported or is there a
different way that I should use to accomplish this?
If I am not misunderstand, at first, you want to use $() to get the variable you defined using the UI but failed. But later, ${{ }} can give you the value of the variable EnvironmentCode.
In fact, while you change to use ${{ }}, it just accessing the variable you predefined in the YAML files instead of the one you defined with UI. Just see this doc: Variable templates.
For the variable you defined with UI, it can be get and used with the format $()(Note: ${{ }} is the format of get the variables which defined in YAML file). But also, there some things you need to pay attention is for the variables you defined in UI, it can only be get/accessed after the build begin to run, because the variable which defined with UI exists in environment only after the build compiled begin. In one word, they are the agent-scope variable. That's why the value it used is still the one that's on the pipeline definition instead of on the UI.
If you remove from the pipeline definition the default value or the
variable entirely the expression ${{variables.EnvironmentCode}} will
return an empty string assuming the filename to be vars..yml which
doesn't exist.
As the document defined and I mentioned before, ${{}} is format which used to get the value of variable which defined in YAML files rather than the one which defined using UI.
In the steps of job, the variable that defined in the UI or defined in the YAML file can all be get/accessed with the format $(). Still, for the variable which defined in the YAML file can also get with ${{variables.xxxx}}. But at this time, if the variable name which defined in YAML file is same with the one defined with UI, the server can only get the one defined in YAML file.