I have 1 parameter file for each environment. Let's call it :
dev.json
test.json
Each one of these files have parameters needed by other templates file:
ex: Dev environment has multiple templates such as:
Lambda.yaml
SNS.yaml
All parameters for these files come from dev.json file.
Question: How do I make parameters optional in parameter file so that if it is not needed in lambda.yaml then it will be ignored.
I am getting errors such as [parameters] do not exists in template.
Just set a default value for your parameters.
# File: template.yml
parameters:
- name: environmentFile
type: string
default: ''
steps:
- script: echo ${{ parameters.environmentFile }}
This way you can consume your template without passing any value to its parameters.
# File: template-caller.yml
extends:
template: simple-param.yml
Related
I have one variable group in ADO library which store different paths and some other variables.
In my main "master" pipeline I use it as below:
variables:
- group: myGroupName
- name: nameOfMyVariable(from variables group) or JustAnyName
- value: $[variables.nameOfMyVariable] or $[variables.JustAnyName]
then in job in the first Stage (for testing, there is only one stage and job for now) I'm trying to using template yaml:
jobs:
- template: my-template.yaml
parameters:
path: $(nameOfMyVariable) or $(JustAnyName)
then in my-template.yaml I have this code:
parameters:
- name: path
type: string
default: ''
jobs:
- job: BuildSomething
steps:
- task: CopyFiles#2
inputs:
Contents: |
${{ parameters.path }}
TargetFolder: '$Build.ArtifactStagingDirectory)'
....
Rest is not that important as it just can't find files to copy and when I try to print parameters.path with echo I get error :
syntax error: invalid arithmetic operator(error token is ".nameOfMyVariable").
I do not know how to fix it so I can access variables from variable group in some of my templates. Do I need to use ##vso[task.setvariables] or something else?
If you want use variable from variable group it is enough to just include this group
variables:
- group: myGroupName
And then use variable by name $(nameOfMyVariable)
In your example it seems you try unnecessary try to declare this variable again in first yaml example.
This example is additionally incorrect because you are addings dash directly before 'value' keyword and it may cause undefined behaviour.
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch#specify-variables
Just like Kontekst says, when it comes to variable groups, you don't need to declare the variable names and values in your yaml. Once you declare the variable groups in your yaml, you could use the variables from the groups directly.
And if you are using parameters against templates, for your scenario, I suppose that you could declare the parameters in your main yaml, and input the parameter value into your template. (and you don't have to use the variable groups)
My main yaml as below.
trigger:
- none
pool:
vmImage: ubuntu-latest
parameters:
- name: pathmain
displayName: Source Path
type: string
default: README.md
values:
- azure-pipelines.yml
- README.md
jobs:
- job:
steps:
- script: echo ${{ parameters.pathmain }}
- template: test-template.yml
parameters:
path: ${{parameters.pathmain}}
My test-template as below.
parameters:
- name: path
displayName: Source Path
type: string
jobs:
- job: BuildSomething
steps:
- task: CopyFiles#2
inputs:
Contents: |
${{ parameters.path }}
TargetFolder: '$Build.ArtifactStagingDirectory'
I have Azure DevOps build pipeline yml that has parameters, variables defined and at the same time I have an external tool that kicks of the jobs in the yml including passing in some parameters. At first I thought the parameters in the yml and incoming parameters should be the same, so I lined up all the names to match. Only to find out that the incoming parameters through REST API is actually a variable!
parameters:
- name: machineList
type: string
default: any
then I have variable declared like so
variables:
testsettingsjson: ${{ parameters.machineList }}
From the REST API that get's passed in and when I query the request coming in, calling the get build data api, I see
"parameters" : "{ \"machineList\" : \"machine1\" }"
Now trying to use the variable it works in some places and not in others.
- script: |
echo "Output variable values"
echo "ML json : '$(testsettingsjson)'"
echo "ML json : '$(machineList)'"
echo "ML json variables : '${{ variables.machineList }}'"
the output I see for the above is :
Output variable values
ML json : any
ML json : machine1
ML json variables:
So $(machineList) works, I thought.
- task: PowerShell#2
displayName: 'Read ML input value'
inputs:
targetType: filePath
filePath: ./ReadMachineList.ps1
continueOnError: true
env:
TESTSETTINGSJSON: $(machineList)
When I read TESTSETTINGSJSON using $env:TESTSETTINGSJSON from the powershell script above, the value is "any"!
I am super confused. What am I doing wrong?
Only to find out that the incoming parameters through REST API is
actually a variable!
This is as expected. The parameters you specify in the request body of the rest api actually represents the variable. If you want to replace the runtime parameter, you should use templateParameters instead.
$json= #'
{
"templateParameters": {
"machineList": "machine1"
}
}
Solved it. To make it work, I first structured my solution working with 3 separate set of "variables", here's an example.
parameters:
- name: MachineList
type: string
default: any
variables:
${{ if ne(variables['machineList'], '') }}:
machinelist: $(machineList)
${{ if ne(parameters.MachineList, 'Any') }}:
machinelist: ${{ parameters.MachineList }}
MachineList is the input to the pipeline.
machineList is the input value from Power Automate (or REST API).
machinelist is the local variable in yml.
Now in the rest of the yml, I can use $(machinelist) switching between the values for the pipeline versus the REST API.
I have defined a variable in the pipeline and set it to false/true. However, when the pipeline is set, I can see that the value for parameters.RunUnitTest is $(RunUnitTest), and this is not the value I set up in the pipeline. So what I am doing wrong here?
trigger: none
extends:
template: ThunderPipeline.yaml
parameters:
MergeBetweenBranches: true
FromBranch: 'master'
ToBranch: 'R_Current_Sprint'
RunUnitTest: '$(RunUnitTest)'
Variables can be defined in one YAML and included in another template. This could be useful if you want to store all of your variables in one file. If you are using a template to include variables in a pipeline, the included template can only be used to define variables. You can use steps and more complex logic when you are extending from a template. Use parameters instead of variables when you want to restrict type.
In this example, the variable favoriteVeggie is included in azure-pipelines.yml.
# File: vars.yml
variables:
favoriteVeggie: 'brussels sprouts'
# File: azure-pipelines.yml
variables:
- template: vars.yml # Template reference
steps:
- script: echo My favorite vegetable is ${{ variables.favoriteVeggie }}.
Or something like:
# File: templates/steps-with-params.yml
parameters:
- name: 'runExtendedTests' # defaults for any parameters that aren't specified
type: boolean
default: false
steps:
- script: npm test
- ${{ if eq(parameters.runExtendedTests, true) }}:
- script: npm test --extended
# File: azure-pipelines.yml
steps:
- script: npm install
- template: templates/steps-with-params.yml # Template reference
parameters:
runExtendedTests: 'true'
I would like to trigger a job template with an object as parameter.
Unfortunately, even based on the examples I couldn't find a way to do that.
I would appreciate if someone could guide me how to achieve this.
What I want to achieve, is to replace the ["DEPLOY", "CONFIG"] part with a dynamic variable:
- template: job-template.yaml
parameters:
jobs: ["DEPLOY", "CONFIG"]
This is not possible. YAML is very limited here and you may read more about this here
Yaml variables have always been string: string mappings.
So for instance you can define paramaters as complex type
Template file
parameters:
- name: 'instances'
type: object
default: {}
- name: 'server'
type: string
default: ''
steps:
- ${{ each instance in parameters.instances }}:
- script: echo ${{ parameters.server }}:${{ instance }}
Main file
steps:
- template: template.yaml
parameters:
instances:
- test1
- test2
server: someServer
But you are not able to do it dynamically/programmatically as every output you will create will end up as simple string.
What you can do is to pass as string and then using powershell split that string. But it all depends what you want to run further because you won't be able to simply iterate over yaml structure in that way. All what you can do is to run in in powershell loop and do something, but it can be not enough for you.
It's possible with some logic. see below
- template: job-template.yaml
parameters:
param: ["DEPLOY", "CONFIG"]
and in job-template.yaml file you can define. So every job name will be different
parameters:
param: []
jobs:
- ${{each jobName in parameters.param}}:
- job: ${{jobName}}
steps:
- task: Downl......
I'm attempting to create multiple pipelines in Azure DevOps but I would like to reuse the same pipeline YAML file with the differences per environment being loaded from a separate template variables file.
For that purpose I've created two variable files, which are located in the same folder as the pipeline definition:
# vars.dev.yml
variables:
- name: EnvironmentName
value: Development
# vars.prd.yml
variables:
- name: EnvironmentName
value: Production
And the definition of the pipeline is the following:
trigger: none
pr: none
variables:
- name: EnvironmentCode
value: dev
- name: EnvironmentFileName
value: vars.$('EnvironmentCode').yml
stages:
- stage: LoadVariablesPerEnvironment
displayName: Load Variables Per Environment
variables:
- template: $(EnvironmentFileName)
jobs:
- job: ShowcaseLoadedVariables
steps:
- pwsh: Write-Host "Variables have been loaded for the '$ENV:ENVIRONMENTNAME' environment"
displayName: Output Environment Variables
After importing the pipelines using the Azure DevOps UI, I can go to settings of each and set the Environment Code variable to whatever desired environment code:
However I'm always getting the same error when I try to run the pipeline, regardless of the code I fill in the variable value:
So the question here is: Is this kind of variable expansion not supported or is there a different way that I should use to accomplish this?
Thanks!
EDIT
I was able to expand the variables using another method. The new version of the pipeline is as such:
variables:
- name: EnvironmentCode
value: dev
- name: EnvironmentFileName
value: vars.${{ variables.EnvironmentCode }}.yml
stages:
- stage: LoadVariablesPerEnvironment
displayName: Load Variables Per Environment
variables:
- template: ${{ variables.EnvironmentFileName }}
jobs:
- job: ShowcaseLoadedVariables
steps:
- pwsh: Write-Host "Variables have been loaded for the '$ENV:ENVIRONMENTNAME' environment"
displayName: Output Environment Variables
However there is yet the issue of loading different files. I made different attempts and verified the following:
If you give a different environment code using the UI, when running
the pipeline, the value it assumes is still the one that's on the
pipeline definition;
If you remove from the pipeline definition the
default value or the variable entirely the expression
${{variables.EnvironmentCode}} will return an empty string
assuming the filename to be vars..yml which doesn't exist.
Is this kind of variable expansion not supported or is there a
different way that I should use to accomplish this?
If I am not misunderstand, at first, you want to use $() to get the variable you defined using the UI but failed. But later, ${{ }} can give you the value of the variable EnvironmentCode.
In fact, while you change to use ${{ }}, it just accessing the variable you predefined in the YAML files instead of the one you defined with UI. Just see this doc: Variable templates.
For the variable you defined with UI, it can be get and used with the format $()(Note: ${{ }} is the format of get the variables which defined in YAML file). But also, there some things you need to pay attention is for the variables you defined in UI, it can only be get/accessed after the build begin to run, because the variable which defined with UI exists in environment only after the build compiled begin. In one word, they are the agent-scope variable. That's why the value it used is still the one that's on the pipeline definition instead of on the UI.
If you remove from the pipeline definition the default value or the
variable entirely the expression ${{variables.EnvironmentCode}} will
return an empty string assuming the filename to be vars..yml which
doesn't exist.
As the document defined and I mentioned before, ${{}} is format which used to get the value of variable which defined in YAML files rather than the one which defined using UI.
In the steps of job, the variable that defined in the UI or defined in the YAML file can all be get/accessed with the format $(). Still, for the variable which defined in the YAML file can also get with ${{variables.xxxx}}. But at this time, if the variable name which defined in YAML file is same with the one defined with UI, the server can only get the one defined in YAML file.