In Devops in release pipeline, I am configuring variables (name of the resources to be deployed for Dev, preprod stages).Hence under override template parameters, i am specifying -accountName $(accountName) but the name as mentioned in variables (for Dev, preprod) is not accepted, instead it takes the values from template parameters json. Please suggest solution.
You may try to use ${{ variables.accountName}} instead $(accountName) in the pipeline to pass the parameters to the template. You can set the variable system.debug as true to check the full log of the pipeline. In the last section in InitializePipeline, you can see how the resulting yaml looks. It seems that ${{ }} is evaluated during processing the templates etc. but $() is evaluated later.
I had a similar issue as well but a slightly different solution.
My sample azure-pipelines.yml where flag parameter wasn't being overridden in the template.
parameters:
- name: var1
type: string
- name: var2
type: string
- name: bool1
type: bool
...
omitted
...
- task: AzureResourceManagerTemplateDeployment#3
inputs:
deploymentScope: 'Resource Group'
azureResourceManagerConnection: ''
subscriptionId: ''
action: 'Create Or Update Resource Group'
resourceGroupName: rg-${{parameters.var1}}-${{parameters.var2}}
location: ''
templateLocation: 'Linked artifact'
csmFile: $(System.DefaultWorkingDirectory)/arm.json
csmParametersFile: $(System.DefaultWorkingDirectory)/arm.parameters.json
deploymentMode: 'Incremental'
overrideParameters: >-
-string1 ${{parameters.var1}}
-string2 ${{parameters.var2}}
-flag ${{parameters.bool1}}
The problem was with my overrideParameters block style.
I was using >- which removes all newlines at the end but > works which puts a newline at the end. Learn more here.
Related
I would like to use a yml variables file, rather than an ARM parameters file, so that I can use a single variable file for multiple bicep deployment tasks and to use the variables in other pipeline tasks without duplication, but am having trouble with the syntax. Below is what I have. It seems to not see it as valid syntax. I get the following error:
There was an error while overriding 'tags' parameter because of 'SyntaxError: Unexpected end of JSON input', make sure it follows JavaScript Object Notation (JSON)
What is the correct syntax or is there a better way that meets the criteria?
# vars.yml contents
rsgName: "rsg1"
location: "westus"
tags: |
{
"tag1": "tagA",
"tag2": "tagB"
}
# deploy.yml contents
- task: AzureResourceManagerTemplateDeployment#3
inputs:
deploymentScope: 'Subscription'
azureResourceManagerConnection: ${{ parameters.azureServiceConnection }}
subscriptionId: ${{ variables.subId }}
templateLocation: 'Linked artifact'
csmFile: ./template.bicep
overrideParameters: >
-rsgName ${{ variables.rsgName }}
-location ${{ variables.location }}
-tags ${{ variables.tags }}
deploymentMode: 'Validation'
# template.bicep contents
param rsgName string
param location string
targetScope = 'subscription'
resource resourceGroup 'Microsoft.Resources/resourceGroups#2021-04-01' = {
name: rsgName
location: location
tags: tags
As per the task doc, it supports multiple parameters overwrite on the task.
If you use classic UI editor of the task, click ... , it could be more clear.
The error should be caused by the incorrect format on the tag definition on your vars.yml .
Remove extra , behind "tag2": "tagB" to fix the json format for a check.
Or you can try to use simple tag content for a check.
I am trying to find a way to define a variable group at stage level and then access it in below jobs through a template? How would I go about doing this?
# Template file getchangedfilesandvariables.yaml
parameters:
- name: "previouscommitid"
type: string
steps:
- task: PowerShell#2
displayName: 'Get the changed files'
name: CommitIds
inputs:
targetType: 'filePath'
filePath: '$(Build.SourcesDirectory)\AzureDevOpsPipelines\Get-COChangedfiles.ps1'
arguments: >
-old_commit_id ${{ previouscommitid }}
- task: PowerShell#2
name: PassOutput
displayName: 'Getting Variables for Packaging'
inputs:
targetType: 'filepath'
filepath: '$(System.DefaultWorkingDirectory)\AzureDevOpsPipelines\Get-COADOvariables.ps1'
And below is my yaml file.
trigger: none
name: $(BuildID)
variables:
system.debug: true
CodeSigningCertThumbprint: "somethumbprint"
# Triggering builds on a branch itself.
${{ if startsWith(variables['Build.SourceBranch'], 'refs/heads/') }}:
branchName: $[ replace(variables['Build.SourceBranch'], 'refs/heads/', '') ]
# Triggering builds from a Pull Request.
${{ if startsWith(variables['Build.SourceBranch'], 'refs/pull/') }}:
branchName: $[ replace(variables['System.PullRequest.SourceBranch'], 'refs/heads/', '') ]
## it will create pipeline package and it will push it private or public feed artifacts
stages:
- stage: Stage1
variables:
- group: Cloudops
- name: oldcommitid
value: $[variables.lastcommitid]
jobs:
- job: IdentifyChangedFilesAndGetADOVariables
pool:
name: OnPrem
workspace:
clean: all # Ensure the agent's directories are wiped clean before building.
steps:
- powershell: |
[System.Version]$PlatformVersion = ((Get-Content "$(Build.SourcesDirectory)\AzureDevOpsPipelines\PlatformVersion.json") | ConvertFrom-Json).PlatformVersion
Write-Output "The repository's PlatformVersion is: $($PlatformVersion.ToString())"
$NewPackageVersion = New-Object -TypeName "System.Version" -ArgumentList #($PlatformVersion.Major, $PlatformVersion.Minor, $(Build.BuildId))
Write-Output "This run's package version is $($NewPackageVersion.ToString())"
echo "##vso[task.setvariable variable=NewPackageVersion]$($NewPackageVersion.ToString())"
echo "##vso[task.setvariable variable=commitidold;isOutput=true]$(oldcommitid)"
displayName: 'Define package version.'
name: commitidorpackageversion
errorActionPreference: stop
- template: getchangedfilesandvariables.yaml
parameters:
previouscommitid:
- $(commitidorpackageversion.commitidold)
# - $(oldcommitid)
I get the error at the second last line of the code that
/AzureDevOpsPipelines/azure-pipelines.yml (Line: 49, Col: 13): The 'previouscommitid' parameter is not a valid String.
I tried different combinations but I am still getting the errors.
Any ideas?
Thanks for your response. I already had the variable group setup in my library. I was just not able to use it.
The way I was able to achieve this I created another template file and supplied it to variables section under my stage. After doing this I was able to actually able to use the variables from my variable group in my successive jobs.
For more information you can review this doc : https://learn.microsoft.com/en-us/azure/devops/pipelines/library/variable-groups?view=azure-devops&tabs=yaml
stagevariables.yaml
variables:
- group: Cloudops
azure-pipelines.yml
stages:
- stage: Stage1
variables:
- template: stagevariables.yaml
jobs:
- job: CheckwhichfeedsAreAvailable
In YAML pipeline, you can't define a new variable group under the variables key.
Actually, we do not have the syntax can be available to create new variable group when running the YAML pipeline.
Under the variables key, you can:
Define new variables with the specified values.
Override the existing variables with new values.
Reference the variables from the existing variable groups and variable templates.
So, if you want to use a variable group with some variables in the pipeline, you should manually define the variable group on the Pipelines > Library page, then reference it in the pipeline.
Am working on Azure DevOps templates lately. And came across this double curly braces syntax. Just want to know how to use double curly braces in Azure DevOps.
Have seen some posts regarding the same.
Assign conditional value to variables in Azure DevOps
Curly braces in Yaml files
If else in Azure DevOps
Conditional Variable Assignment in Azure DevOps
So lets say, I have a variable defined in a group as follows.
Also we can define variables as follows in a yaml file.
variables:
- name: BuildConfiguration
value: 'Release'
- name: finalBuildArtifactName
value: 'TempFolderName'
When should we use the double curly brace syntax?
I have idea about and using the following ways to reference variables.
variables['MyVar']
variables.MyVar
What can we accomplish with double curly braces in contrast to the above two?
Its very difficult to get things working in a yaml pipeline. Do the change, checkin, run the pipeline, see the result and circle back again. This is highly time consuming, and not smooth to say the least.
${{}} syntax is used for expressions. What's more there is another $[ <expression> ]. And here is the difference:
# Two examples of expressions used to define variables
# The first one, a, is evaluated when the YAML file is compiled into a plan.
# The second one, b, is evaluated at runtime.
# Note the syntax ${{}} for compile time and $[] for runtime expressions.
variables:
a: ${{ <expression> }}
b: $[ <expression> ]
Please check documentation here.
And as you may see here
steps:
- task: PublishPipelineArtifact#1
inputs:
targetPath: '$(Pipeline.Workspace)'
${{ if eq(variables['Build.SourceBranchName'], 'main') }}:
artifact: 'prod'
${{ else }}:
artifact: 'dev'
publishLocation: 'pipeline'
you can use variables['Build.SourceBranchName'] syntax for accessing variables. But,
variables:
- name: foo
value: fabrikam # triggers else condition
pool:
vmImage: 'ubuntu-latest'
steps:
- script: echo "start"
- ${{ if eq(variables.foo, 'adaptum') }}:
- script: echo "this is adaptum"
- ${{ elseif eq(variables.foo, 'contoso') }}:
- script: echo "this is contoso"
- ${{ else }}:
- script: echo "the value is not adaptum or contoso"
You may also use variables.foo.
Expressons are often used for conditional evaluation, dynamic steps/jobs/stages configuration etc.
I have a yaml. where I need to pass serviceconnections to a script/template based on an another task which retrieve all required subscriptions.
Question is : Can I pass a dynamic value to a service connection? It is giving me compile time error.
My code below:
trigger: none
pr: none
parameters:
- name: AzureSubscription
type: object
default:
xxx:
Sub: xxx
yyy:
Sub: yyy
jobs:
- job: Updating
condition: succeeded()
pool:
vmImage: "windows-latest"
strategy:
maxParallel: 10
matrix: ${{ parameters.AzureSubscription }}
steps:
- task: AzurePowerShell#5
displayName: Tes
inputs:
azureSubscription: 'zzz'
ScriptType: 'InlineScript'
Inline: |
Write-Output "subcriptionList ---- $(Sub)"
FailOnStandardError: true
azurePowerShellVersion: 'LatestVersion'
pwsh: true
- task: AzurePowerShell#4
displayName: Updating
inputs:
**azureSubscription: ${{ sub }}** # here it is giving me error?
ScriptType: 'FilePath'
ScriptPath: '$(System.DefaultWorkingDirectory)/Foundation/xxxxx.ps1'
azurePowerShellVersion: 'LatestVersion'
So in the 2nd task, I am passing subscription from my parameter.
Error is : Unrecognized value: 'sub'.
Can someone help me?
This is possible with some creativity but not native to the task. Rather using variables and dynamically loading the variable file.
I usually declare the Service Connection name as a variable template file in a separate repository. This allows for reuse across all projects in the org, not required but find it easier that way. Part of the template name would be the environment being deployed to. So a template file might be called azure.dev.yml or azure.uat.yml and look like:
variables:
AzureSubscriptionServiceConnectionName: Azure - Dev
Then a variable defined within the scope of the stage/job would load the template file like below, assuming that a parameter or a local variable would be passed in with the given environmentName.
variables:
- template: /azure.${{ parameters.environmentName }}.yml
Then the stage/job can reference this variable via:
${{ variables.AzureSubscriptionServiceConnectionName }}
Here is some more Microsoft Documentation on YAML Pipeline Variable Scope
You cannot set azureSubcription dynamically. It is known limitation.
#JoeGaggler this feature isn't supported today. Usage of service endpoints (Azure Subscription is one of kind) in release/build definition is controlled by some permissions. At the time of saving a release/build definition service validates that the author (whoever is saving the definition) has appropriate permissions on the endpoint. If we support variable replacements for service endpoint input then service can't validate that the author has required permissions or not and it might become a security issue.
I'm trying to use the lower() function in a YAML template, but do not understand the behavior.
I've a variable group named Infra.Environment.ShortName with the value "Dev".
In my YAML template, I've defined a variable from a variable group:
variables:
- name: environmentShortName
value: $(Infra.Environment.ShortName)
In a task, I refer to this variable:
- task: AzureResourceGroupDeployment#2
displayName: 'Deploy Storage Account'
inputs:
azureSubscription: ${{ parameters.subscription }}
resourceGroupName: mst-${{ lower(variables.environmentShortName) }}-infra
location: '$(Infra.Environment.Region.Primary)'
csmFile: '$(Pipeline.Workspace)/$(Build.DefinitionName)/Resources/infra-storageAccount.json'
csmParametersFile: '$(Pipeline.Workspace)/$(Build.DefinitionName)/Resources/infra-storageAccount.parameters.json'
deploymentOutputs: ArmOutputs
I've experimented with different expressions, but do not understand why I cannot convert the variable group value to lowercase:
resourceGroupName: mst-${{ lower(variables.environmentShortName) }}-infra
=> mst-Dev-infra (lower does not seem to work)
resourceGroupName: ${{ format('mst-{0}-infra', lower(variables.environmentShortName)) }}
=> mst-Dev-infra (format works, but lower does not)
resourceGroupName: $[format('mst-{0}-infra', lower(variables.environmentShortName))]
=> $[format('mst-{0}-infra', lower(variables.environmentShortName))] (expression not evaluated)
resourceGroupName: mst-${{ lower(variables['Infra.Environment.ShortName']) }}-infra
=> mst--infra (empty value)
resourceGroupName: mst-${{ lower('Dev') }}-infra
=> mst-dev-infra (lower works with a constant value)
The solution was to use runtime variables as the variable group is not resolved at compile time. Runtime expressions have some limitations, but the following works:
- name: environmentShortName
value: $[lower(variables['Infra.Environment.ShortName'])]
resourceGroupName: mst-$(environmentShortName)-infra
Following snippet will work correctly and write "mst-test-infra" as the output:
variables:
environmentShortName: TEST
steps:
- powershell: Write-Host "${{ format('mst-{0}-infra', lower(variables.environmentShortName)) }}"
This is because the variable set to literal value and the value is known on a compile time.
If the value of the environmentShortName is dependant on another variable, like:
environmentShortName: $(System.TeamProject)
The lower() will not work correctly and behaves just like you described.
There is (my) logical explanation: the expression ${{ variables.environmentShortName}} is resolves at compile time and perhaps your variable value is not ready yet to be resolved.
To deal with a such behaviour, consider to change a location where lower() is applied and move it to a variable declaration part:
variables:
environmentShortName: ${{ lower(variables['System.TeamProject'] ) }}
steps:
- powershell: Write-Host "mst-$(environmentShortName)-infra"