I would like to use a yml variables file, rather than an ARM parameters file, so that I can use a single variable file for multiple bicep deployment tasks and to use the variables in other pipeline tasks without duplication, but am having trouble with the syntax. Below is what I have. It seems to not see it as valid syntax. I get the following error:
There was an error while overriding 'tags' parameter because of 'SyntaxError: Unexpected end of JSON input', make sure it follows JavaScript Object Notation (JSON)
What is the correct syntax or is there a better way that meets the criteria?
# vars.yml contents
rsgName: "rsg1"
location: "westus"
tags: |
{
"tag1": "tagA",
"tag2": "tagB"
}
# deploy.yml contents
- task: AzureResourceManagerTemplateDeployment#3
inputs:
deploymentScope: 'Subscription'
azureResourceManagerConnection: ${{ parameters.azureServiceConnection }}
subscriptionId: ${{ variables.subId }}
templateLocation: 'Linked artifact'
csmFile: ./template.bicep
overrideParameters: >
-rsgName ${{ variables.rsgName }}
-location ${{ variables.location }}
-tags ${{ variables.tags }}
deploymentMode: 'Validation'
# template.bicep contents
param rsgName string
param location string
targetScope = 'subscription'
resource resourceGroup 'Microsoft.Resources/resourceGroups#2021-04-01' = {
name: rsgName
location: location
tags: tags
As per the task doc, it supports multiple parameters overwrite on the task.
If you use classic UI editor of the task, click ... , it could be more clear.
The error should be caused by the incorrect format on the tag definition on your vars.yml .
Remove extra , behind "tag2": "tagB" to fix the json format for a check.
Or you can try to use simple tag content for a check.
Related
I tried to find some information in internet. But unfortunately I could not found any information.
Im trying to send pipeline parameters from pipeline into powershell script
pipeline below:
parameters:
- name: env
displayName: Select Environment
type: string
default: development
stages:
- stage: test
displayName: test var
jobs:
- job: PostgresSQL
steps:
- task: PowerShell#2
inputs:
filePath: '$(5ystem. DefaultWorkingDirectory)/test.psl'
errorActionPreference: 'continue'
enabled: true
I need to send ${{ parameters.env }} to powershell.
I tried different type of define param like a variable into powershell. but it does not work.
I would be very happy if anybody can help me and share relevant documentation for that.
Thanks all
First approach is to provide arguments using 'arguments' keyword (available in PowerShell by 'param')
filePath: xyz
arguments: -input1 ${{ parameters.env }}
documentation and example - https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/reference/powershell-v2?view=azure-pipelines#call-powershell-script-with-multiple-arguments
Second approach you can map parameters to environment variables provided to script using 'env' keyword
env:
input1: ${{ parameters.env }}
documentation - https://learn.microsoft.com/en-us/azure/devops/pipelines/process/tasks?view=azure-devops&tabs=yaml#environment-variables
I am quite new in Yaml and AzureDevOps and I am trying to initialize variable based on condition
variables:
- name: DisplayName
${{ if eq('$(env)', 'wh') }}:
value: 'DisplayNAME-WH'
${{ if eq('$(env)', 'lw') }}:
value: 'DisplayNAME-LW'
Where my 'env' is variable passed via UI in Azure DevOps
Issue what I am getting that "DisplayName" stays still empty (is not initialized).
Can you help me out?
Thanks.
Azure DevOps conditional initialization of variables is not working
According to the Understand variable syntax:
In a pipeline, template expression variables (${{ variables.var }})
get processed at compile time, before runtime starts. Macro syntax
variables ($(var)) get processed during runtime before a task runs.
So, we could not use the $(env) in the ${{ if eq('$(env)', 'wh') }}. That because the syntax of ${{ if eq('$(env)', 'wh') }} is parsed before the syntax of $(env).
To resolve this issue, we could to defined the variable as Runtime parameters:
parameters:
- name: 'env'
default: 'wh'
type: string
values:
- wh
- lw
variables:
${{ if eq(parameters.env, 'wh') }}:
DisplayName: 'DisplayNAME-WH'
${{ if eq(parameters.env, 'lw') }}:
DisplayName: 'DisplayNAME-LW'
steps:
- script:
echo $(DisplayName)
The test result:
I'm trying to use the lower() function in a YAML template, but do not understand the behavior.
I've a variable group named Infra.Environment.ShortName with the value "Dev".
In my YAML template, I've defined a variable from a variable group:
variables:
- name: environmentShortName
value: $(Infra.Environment.ShortName)
In a task, I refer to this variable:
- task: AzureResourceGroupDeployment#2
displayName: 'Deploy Storage Account'
inputs:
azureSubscription: ${{ parameters.subscription }}
resourceGroupName: mst-${{ lower(variables.environmentShortName) }}-infra
location: '$(Infra.Environment.Region.Primary)'
csmFile: '$(Pipeline.Workspace)/$(Build.DefinitionName)/Resources/infra-storageAccount.json'
csmParametersFile: '$(Pipeline.Workspace)/$(Build.DefinitionName)/Resources/infra-storageAccount.parameters.json'
deploymentOutputs: ArmOutputs
I've experimented with different expressions, but do not understand why I cannot convert the variable group value to lowercase:
resourceGroupName: mst-${{ lower(variables.environmentShortName) }}-infra
=> mst-Dev-infra (lower does not seem to work)
resourceGroupName: ${{ format('mst-{0}-infra', lower(variables.environmentShortName)) }}
=> mst-Dev-infra (format works, but lower does not)
resourceGroupName: $[format('mst-{0}-infra', lower(variables.environmentShortName))]
=> $[format('mst-{0}-infra', lower(variables.environmentShortName))] (expression not evaluated)
resourceGroupName: mst-${{ lower(variables['Infra.Environment.ShortName']) }}-infra
=> mst--infra (empty value)
resourceGroupName: mst-${{ lower('Dev') }}-infra
=> mst-dev-infra (lower works with a constant value)
The solution was to use runtime variables as the variable group is not resolved at compile time. Runtime expressions have some limitations, but the following works:
- name: environmentShortName
value: $[lower(variables['Infra.Environment.ShortName'])]
resourceGroupName: mst-$(environmentShortName)-infra
Following snippet will work correctly and write "mst-test-infra" as the output:
variables:
environmentShortName: TEST
steps:
- powershell: Write-Host "${{ format('mst-{0}-infra', lower(variables.environmentShortName)) }}"
This is because the variable set to literal value and the value is known on a compile time.
If the value of the environmentShortName is dependant on another variable, like:
environmentShortName: $(System.TeamProject)
The lower() will not work correctly and behaves just like you described.
There is (my) logical explanation: the expression ${{ variables.environmentShortName}} is resolves at compile time and perhaps your variable value is not ready yet to be resolved.
To deal with a such behaviour, consider to change a location where lower() is applied and move it to a variable declaration part:
variables:
environmentShortName: ${{ lower(variables['System.TeamProject'] ) }}
steps:
- powershell: Write-Host "mst-$(environmentShortName)-infra"
I am using the DownloadPipelineArtifact#2 task and looping over an array of pipelines to download the artifacts for each of them. I would like to use the $(BuildNumber) output of each of these tasks in subsequent tasks.
${{ each p in pipelines }}:
- task: DownloadPipelineArtifact#2
displayName: 'Download ${{ p.alias }} Artifact - ${{ p.artifact }}'
name: 'Download_${{ p.alias }}'
inputs:
buildType: specific
project: $(System.TeamProjectId)
pipeline: ${{ p.id }}
preferTriggeringPipeline: false
runVersion: latestFromBranch
runBranch: 'refs/heads/${{ p.branch }}'
artifact: ${{ p.artifact }}
patterns: ${{ p.patterns }}
targetPath: '$(Pipeline.Workspace)/artifacts/${{ p.alias }}/${{ p.artifact }}'
Unfortunately, I am unable to do so as I would need to refer to the task by it's name and I am not able to assign dynamic names to the tasks. I get an error saying
Valid names may only contain alphanumeric characters and '_' and may not start with a number.
I tried using runtime variables, expressions and a counter for the name but all fail with the same error. I am assuming here the name field is not dynamically assignable.
All works well without the name field. But from what I understand I would need the name of the task to refer to it and extract it's output. Something like:
{{ taskName.BuildNumber }}
If anyone knows of any way in which I can assign the name dynamically or through which I can extract the $(BuildNumber) for each task in the loop, please let me know. Thanks!
Is there a way to dynamically assign names to tasks in yaml pipelines for Azure Devops?
I am afraid there is no such way to dynamically assign names to tasks in yaml pipelines for Azure Devops
Because the task itself does not contain the Name field, the Nameof the downloaded artifact is determined by the name of the artifact generated during building. This task only needs to download the artifact, so this task does not include the field to rewrite the name. When we use any variable in the value of this field, the compiler will parse this field, then it will throw us the error mentioned in your question:
Looks like you can do it, this guy has it working 1 ... as long as the array of iteration is known at the YAML compile time.
Here what was working for me (Had to santize the name to get /A-z0-9_/):
- ${{ each environmentFull in parameters.environmentFullList }}:
- bash: |
echo "##vso[task.setvariable variable=environment]$(echo ${{ environmentFull }} | cut -d. -f1)"
echo "##vso[task.setvariable variable=namespace]$(echo ${{ environmentFull }} | cut -d. -f2)"
failOnStderr: true
displayName: Important pipeline variables
- bash: |
echo "Namespace: $(namespace)"
echo "Environment: $(environment)"
echo "Task name: kubectl_${{ replace(replace(environmentFull,'.','_'),'-','_') }}"
failOnStderr: true
displayName: Print info
- task: Kubernetes#1
name: kubectl_${{ replace(replace(environmentFull,'.','_'),'-','_') }}
inputs:
connectionType: 'Kubernetes Service Connection'
kubernetesServiceEndpoint: '${{ environmentFull }}'
namespace: '$(namespace)'
command: 'get'
arguments: 'pods'
secretType: 'dockerRegistry'
containerRegistryType: 'Azure Container Registry'
outputFormat: 'jsonpath=''{range .items[*].status.containerStatuses}{#}{end}'''
displayName: "Get all pods"
- bash: |
export TASKNAME_LC="kubectl_${{ replace(replace(environmentFull,'.','_'),'-','_') }}_KubectlOutput"
export TASKNAME_UC=${TASKNAME_LC^^}
echo "====Using name: $TASKNAME_UC"
export TASK_VALUE=${!TASKNAME_UC}
echo "$TASK_VALUE"
Image of worked run:
1) Azure Pipeline dynamic parameters to template file from YAML pipeline
In Devops in release pipeline, I am configuring variables (name of the resources to be deployed for Dev, preprod stages).Hence under override template parameters, i am specifying -accountName $(accountName) but the name as mentioned in variables (for Dev, preprod) is not accepted, instead it takes the values from template parameters json. Please suggest solution.
You may try to use ${{ variables.accountName}} instead $(accountName) in the pipeline to pass the parameters to the template. You can set the variable system.debug as true to check the full log of the pipeline. In the last section in InitializePipeline, you can see how the resulting yaml looks. It seems that ${{ }} is evaluated during processing the templates etc. but $() is evaluated later.
I had a similar issue as well but a slightly different solution.
My sample azure-pipelines.yml where flag parameter wasn't being overridden in the template.
parameters:
- name: var1
type: string
- name: var2
type: string
- name: bool1
type: bool
...
omitted
...
- task: AzureResourceManagerTemplateDeployment#3
inputs:
deploymentScope: 'Resource Group'
azureResourceManagerConnection: ''
subscriptionId: ''
action: 'Create Or Update Resource Group'
resourceGroupName: rg-${{parameters.var1}}-${{parameters.var2}}
location: ''
templateLocation: 'Linked artifact'
csmFile: $(System.DefaultWorkingDirectory)/arm.json
csmParametersFile: $(System.DefaultWorkingDirectory)/arm.parameters.json
deploymentMode: 'Incremental'
overrideParameters: >-
-string1 ${{parameters.var1}}
-string2 ${{parameters.var2}}
-flag ${{parameters.bool1}}
The problem was with my overrideParameters block style.
I was using >- which removes all newlines at the end but > works which puts a newline at the end. Learn more here.