YAML Extends Template - Unrecognized value: 'else'. Located at position 1 within expression: else - azure-devops

Expanding on my first question. I am now experiencing a new error Unrecognized value: 'else'. Located at position 1 within expression: else that I cannot figure out a solution to. The issue happens when I add the 'else' condition within the foreach of the job.steps.
Build YAML
resources:
repositories:
- repository: self
type: git
ref: refs/heads/Development
- repository: AdoRestrictions
type: git
name: utl-yaml-templates
ref: refs/heads/main
trigger: none
pool:
name: PROD
extends:
template: ADO_Stage_Restrictions_Dev.yml#AdoRestrictions
parameters:
stageObjs:
- stage: 'BuildStage'
displayName: 'Build Test'
jobs:
- job: 'BuildJob'
displayName: 'Build'
steps:
- task: PowerShell#2
displayName: 'PS Hello World'
inputs:
targetType: inline
script: |
Write-Host "Hello World"
Working Extends YAML
parameters:
- name: stageObjs
type: stageList
default: []
stages:
- ${{ each stage in parameters.stageObjs }}:
- stage: ${{ stage.stage }}
displayName: ${{ stage.displayName }}
jobs:
- ${{ each job in stage.jobs }}:
- job: ${{ job.job }}
displayName: ${{ job.displayName }}
steps:
- ${{ each step in job.steps }}:
- ${{ if or(startsWith(step.task, 'PowerShell'),startsWith(step.task, 'CmdLine'),startsWith(step.task, 'Bash'),startsWith(step.task, 'ShellScript'),containsValue(step.task, 'Script'),containsValue(step.task, 'CLI'),containsValue(step.task, 'PowerShell')) }}:
- task: PowerShell#2
displayName: 'Unapproved - ${{ step.displayName }}'
inputs:
targetType: inline
script: Write-Output "Unapproved Task - Scripting and CLI tasks are not approved for use in yaml pipelines"
Broken Extends YAML
parameters:
- name: stageObjs
type: stageList
default: []
stages:
- ${{ each stage in parameters.stageObjs }}:
- stage: ${{ stage.stage }}
displayName: ${{ stage.displayName }}
jobs:
- ${{ each job in stage.jobs }}:
- job: ${{ job.job }}
displayName: ${{ job.displayName }}
steps:
- ${{ each step in job.steps }}:
- ${{ if or(startsWith(step.task, 'PowerShell'),startsWith(step.task, 'CmdLine'),startsWith(step.task, 'Bash'),startsWith(step.task, 'ShellScript'),containsValue(step.task, 'Script'),containsValue(step.task, 'CLI'),containsValue(step.task, 'PowerShell')) }}:
- task: PowerShell#2
displayName: 'Unapproved - ${{ step.displayName }}'
inputs:
targetType: inline
script: Write-Output "Unapproved Task - Scripting and CLI tasks are not approved for use in yaml pipelines"
- ${{ else }}:
- ${{ step }}
Full Error Message
(Line: 22, Col: 13): Unrecognized value: 'else'. Located at position 1 within expression: else. (Line: 22, Col: 13): Expected at least one key-value pair in the mapping
Update - Working Solution
parameters:
- name: stageObjs
type: stageList
default: []
stages:
- ${{ each stage in parameters.stageObjs }}:
- stage: ${{ stage.stage }}
displayName: ${{ stage.displayName }}
jobs:
- ${{ each job in stage.jobs }}:
- job: ${{ job.job }}
displayName: ${{ job.displayName }}
steps:
- ${{ each step in job.steps }}:
#Disallow any type of scripting tasks
- ${{ if or(startsWith(step.task, 'PowerShell'),startsWith(step.task, 'CmdLine'),startsWith(step.task, 'Bash'),startsWith(step.task, 'ShellScript'),contains(step.task, 'Script'),contains(step.task, 'CLI'),contains(step.task, 'PowerShell')) }}:
- task: PowerShell#2
displayName: 'Unapproved Task - ${{ step.displayName }}'
inputs:
targetType: inline
script: throw "Unapproved Task - Scripting and CLI tasks are not approved for use in yaml pipelines"
#Not a scripting task perform additional checks
- ${{ if and(not(startsWith(step.task, 'PowerShell')),not(startsWith(step.task, 'CmdLine')),not(startsWith(step.task, 'Bash')),not(startsWith(step.task, 'ShellScript')),not(contains(step.task, 'Script')),not(contains(step.task, 'CLI')),not(contains(step.task, 'PowerShell')) ) }}:
#Validate azure subscription provided for the environment
- ${{ if contains(step.task, 'Azure') }}:
- ${{ each pair in step }}:
${{ if eq(pair.key, 'inputs') }}:
inputs:
${{ each attribute in pair.value }}:
${{ if contains(attribute.key, 'Subscription') }}:
${{ if and(ne(attribute.value, 'sub-name1'), ne(attribute.value, 'sub-name2'), ne(attribute.value, 'sub-name3')) }}:
${{ attribute.value }}: ''
${{ if or(eq(attribute.value, 'sub-name1'), eq(attribute.value, 'sub-name2'), eq(attribute.value, 'sub-anme3')) }}:
${{ pair.key }}: ${{ pair.value }}
${{ if ne(pair.key, 'inputs') }}:
${{ if eq(pair.key, 'displayName') }}:
${{ pair.key }}: 'Invalid Azure Subscription - ${{ pair.value }}'
${{ if ne(pair.key, 'displayName') }}:
${{ pair.key }}: ${{ pair.value }}
#Allow all other tasks
- ${{ if not(contains(step.task, 'Azure')) }}:
- ${{ step }}

Related

Injecting an input to a step provided in a stepList?

I have a pipeline template that takes a stepList:
parameters:
- name: applicationDeploySteps
type: stepList
default: []
And injects the stepList into the template:
- deployment: Deploy_App
displayName: Deploy Application
pool: ${{ variables.AgentPool }}
environment: ${{ parameters.Stage }}
variables:
- name: ServiceConnection
value: SomeServiceConnection
strategy:
runOnce:
deploy:
steps:
- ${{ each step in parameters.applicationDeploySteps }}:
- ${{ each pair in step }}:
${{ pair.key }}: ${{ pair.value }}
However, I'd like to provide an AzureCLI#2 step, with the azureSubscription parameter being sourced from a variable inaccessible to the AzureCLI#2 step at the time of template compilation:
extends:
template: main.yml
parameters:
applicationDeploySteps:
- task: AzureCLI#2
inputs:
azureSubscription: $(ServiceConnection)
addSpnToEnvironment: true
scriptType: 'bash'
scriptLocation: 'inlineScript'
inlineScript: |
echo "do azurey things here"
The problem is in azureSubscription: $(ServiceConnection). Obviously, that variable can't resolve. So the solution I'm shooting for is to inject the azureSubscription value in the template. However, I can't find a way to effectively iterate over the values provided in the input block.
- ${{ each pair in step }}:
${{ pair.key }}: ${{ pair.value }}
will let me interrogate the type of the step. Trying to take it further just gives me a null reference exception when trying to queue the pipeline:
- ${{ each pair in step }}:
${{ if eq(pair.key, 'inputs') }}:
- ${{ each input in pair.value }}:
${{ if eq(input.key, 'azureSubscription') }}:
${{ input.key }}: ${{ variables.ServiceConnection }}
${{ else }}:
${{ input.key }}: ${{ input.value }}
${{ else }}:
${{ pair.key }}: ${{ pair.value }}
That attempt gives me: Object reference not set to an instance of an object. with no corresponding line number. I'm guessing it's failing to iterate over pair.value, but I have no idea how to troubleshoot it further or get an idea of what I can and cannot iterate over. The documentation does not include more comprehensive examples, just checking if, say, it's a script task and blocking execution.
Note that this is similar, but not the scenario that I'm implementing.
I figured it out.
- ${{ each step in parameters.applicationDeploySteps }}:
- ${{ each pair in step }}:
${{ if eq(pair.key, 'inputs') }}:
inputs:
${{ each input in pair.value }}:
${{ if eq(input.key, 'azureSubscription') }}:
${{ input.key }}: ${{ variables.ServiceConnection }}
${{ else }}:
${{ input.key }}: ${{ input.value }}
${{ else }}:
${{ pair.key }}: ${{ pair.value }}
This document was useful for pointing me in the right direction, but honestly it was mostly trial-and-error.

Not able to set variable based on parameters in YAML

I'm trying to build a new yaml file that reads keyvault secrets based on the parameters at the runtime and declared variables with the condition as per the parameters, but this isn't working.
- name: azure_subscription
displayName: " Select subscription "
type: string
default: "service-connection-dev"
values:
- 'service-connection-dev'
- 'service-connection-sit'
- 'service-connection-tes'
- 'service-connection-prd'
variables:
- ${{ if eq('${{ parameters.azure_subscription }}', 'service-connection-sit') }}:
name: key_vault
value: 'core-kv-sit'
- ${{ if eq('${{ parameters.azure_subscription }}', 'service-connection-dev') }}:
name: key_vault
value: 'core-kv-dev'
stages:
- stage: Validate
${{ if eq(parameters.azure_subscription, 'service-connection-dev') }}:
pool:
name: agent-pool-win-dev
${{ if eq(parameters.azure_subscription, 'service-connection-sit') }}:
pool:
name: agent-pool-win-sit
jobs:
- job: Validate
steps:
- task: AzureKeyVault#2
inputs:
KeyVaultName: "${{variables.key_vault}}"
SecretsFilter: "*"
RunAsPreJob: false
azureSubscription: ${{ parameters.azure_subscription }}
I've tried using variables inside jobs, but that is also not working. Can someone please help?
Also, I'll have to declare 2 more variables as per the parameters input, Is it possible ?
Thanks in advance
- ${{ if eq('${{ parameters.azure_subscription }}', 'service-connection-sit') }}:
You're using a literal value of "${{ parameters.azure_subscription }}" for the left side of the comparison. The comparison should just be parameters.azure_subscription.
So: - ${{ if eq(parameters.azure_subscription, 'service-connection-sit') }}:

How to use a dependsOn for the previous loop in Azure Devops YAML pipeline while using a each argument

I have a pipeline setup that is very basic and using the below template
parameters:
- name: countries
type: object
default:
- country: 'uk'
envs: ['development', 'preproduction', 'production']
- country: 'us'
envs: ['development', 'preproduction', 'production']
stages:
- ${{ each c in parameters.countries }}:
- stage: ${{ c.country }}
displayName: ${{ c.country }}
# some variable here to control if to use [] to deploy to all stages or just one
dependsOn: []
jobs:
- ${{ each env in c.envs }}:
- job: ${{ c.country }}_${{ env }}
steps:
- script: echo "Hello world"
condition: contains(variables['System.JobDisplayName'], 'production')
- script: echo "Hello world 2"
I want to have it so it only deploys to the new job loop in the pipeline if the previous job loop has succeeded. I can't seem to figure out a way of doing this. Any help will be appreicated. Thanks!
I managed to achieve this with the below job spec
- job: ${{ c.country }}_${{ env }}
${{if eq(env, 'preproduction')}}:
condition: eq(dependencies.${{ c.country }}_development.result, 'Succeeded')
dependsOn: ${{ c.country }}_development
${{if eq(env, 'production')}}:
condition: eq(dependencies.${{ c.country }}_preproduction.result, 'Succeeded')
dependsOn: ${{ c.country }}_preproduction
steps:

Passing a dictionary to a template in azure devops yaml

I want to run a loop in a template to download two artifacts with specific versions.
I have been trying to formulate solutions for this but no luck yet, this is what I've came up until now but i think its not supported.
Can someone point me in the right direction if this is possible?
pipeline.yml
variables:
- template: project.variables.yml
jobs:
- job: 'Deploy'
steps:
- template: instantclient.template.yml
parameters:
artifacts:
oracle-instantclient:
package: 'oracle-instantclient'
packageVersion: ${{ variables.oracle-instantclient }}
oracle-data-access-components:
package: 'oracle-data-access-components'
packageVersion: ${{ variables.oracle-data-access-components }}
instantclient.template.yml
parameters:
- name: artifacts
type: object
- name: feed
default: ahitapplicationteam
- name: downloadDirectory
default: deployment/s
steps:
- ${{ each artifact in parameters.artifacts}}:
- template: artifacts.template.yml
parameters:
packageVersion: ${{ packageVersion }}
feed: ${{ parameters.feed }}
package: ${{ package }}
downloadDirectory: ${{ parameters.downloadDirectory }}
artifacts.template.yml
parameters:
- name: packageVersion
- name: feed
- name: package
- name: downloadDirectory
steps:
- task: UniversalPackages#0
displayName: 'Downloading | Feed: ${{ parameters.feed }} | Package: ${{ parameters.package }}' #| PackageVersion: ${{ parameters.packageVersion }}
inputs:
command: 'download'
downloadDirectory: ${{ parameters.downloadDirectory }}
vstsFeed: ${{ parameters.feed }}
vstsFeedPackage: ${{ parameters.package }}
vstsPackageVersion: ${{ parameters.packageVersion }}
verbosity: 'Debug'
You're on the right track. You need to add the - character to each item in your object to convert it into an array. The object can be an array of simple strings or complex objects. As an object, you can access the properties of your objects in the each loop.
The use of ${{ variables.oracle-data-access-components }} assumes that the oracle-data-access-components variable is available at compile-time when the pipeline is initially processed.
Whether you want to break it into 3 templates is a stylistic decision. I went with 2 templates to simplify readability, but a third template will provide you will some validation for required parameters.
pipeline.yml
variables:
- template: project.variables.yml
jobs:
- job: 'Deploy'
steps:
- template: instantclient.template.yml
parameters:
artifacts:
- name: 'oracle-instantclient'
version: ${{ variables.oracle-instantclient }}
- name: 'oracle-data-access-components'
version: ${{ variables.oracle-data-access-components }}
instantclient.template.yml
parameters:
# list of package to download (name, version)
- name: artifacts
type: object
# azure artifact feed name
- name: feed
type: string
default: 'ahitapplicationteam'
# download path for artifacts
- name: downloadDirectory
type: string
default: 'deployment/s'
steps:
# loop through the artifacts (name, version)
- ${{ each artifact in parameters.artifacts}}:
# download the artifact
- task: UniversalPackages#0
displayName: 'Downloading | Feed: ${{ parameters.feed }} | Package: ${{ artifact.name }}' #| PackageVersion: ${{ artifact.version }}
inputs:
command: 'download'
downloadDirectory: ${{ parameters.downloadDirectory }}
vstsFeed: ${{ parameters.feed }}
vstsFeedPackage: ${{ artifact.name }}
vstsPackageVersion: ${{ artifact.version }}
verbosity: 'Debug'

set and refer to variables in yaml

I have the following yml code that sets and refers to some variables as follows:
<one.yml>
- task: AzurePowerShell#4
displayName: 'Copy functions templates'
inputs:
azureSubscription: ${{parameters.serviceConnection}}
ScriptPath: ${{ parameters.root }}/Scripts/ReleaseManagement/CopyChildTemplatesToContainer.ps1
ScriptArguments: '-resourceGroupName ''${{ parameters.solutionAbbreviation}}-data-${{ parameters.environmentAbbreviation}}''
name: copyFunctionsTemplates
- powershell: |
Write-Host "##vso[task.setvariable variable=data_containerSASToken;isOutput=true]$(copyFunctionsTemplates.containerSASToken)"
Write-Host "##vso[task.setvariable variable=data_containerEndPoint;isOutput=true]$(copyFunctionsTemplates.containerEndPoint)"
displayName: 'set data output variables'
name: dataVariables
<two.yml>
stages:
- deployment: ${{ parameters.stageName }}_DeployResources
displayName: ${{ parameters.stageName }}_DeployResources
- stage: ${{ parameters.stageName }}
dependsOn: ${{ parameters.dependsOn }}
condition: ${{ parameters.condition }}
jobs:
- deployment: ${{ parameters.stageName }}_DeployResources
displayName: ${{ parameters.stageName }}_DeployResources
steps:
- template: one.yml
jobs:
- job: ${{ parameters.stageName }}_DeployFunctions
dependsOn: ${{ parameters.stageName }}_DeployResources
variables:
data_containerEndPoint: $[ dependencies.DeployResources.outputs['DeployResources.dataVariables.data_containerEndPoint'] ]
data_containerSASToken: $[ dependencies.DeployResources.outputs['DeployResources.dataVariables.data_containerSASToken'] ]
steps:
- ${{ each func in parameters.functionApps }}:
- template: three.yml
<three.yml>
steps:
- task: AzureResourceGroupDeployment#2
displayName: 'deploy ${{ parameters.name }} data resources'
inputs:
azureSubscription: ${{parameters.serviceConnection}}
resourceGroupName: ${{parameters.solutionAbbreviation}}-data-${{parameters.environmentAbbreviation}}
location: ${{parameters.location}}
csmFile: ${{ parameters.root }}/functions_arm_templates/${{ parameters.name }}/Infrastructure/data/template.json
csmParametersFile: ${{ parameters.root }}/functions_arm_templates/${{ parameters.name }}/Infrastructure/data/parameters/parameters.${{parameters.environmentAbbreviation}}.json
overrideParameters: -environmentAbbreviation "${{parameters.environmentAbbreviation}}"
-tenantId "${{parameters.tenantId}}"
-solutionAbbreviation "${{parameters.solutionAbbreviation}}"
-containerBaseUrl "$(data_containerEndPoint)functions/${{ parameters.name }}/Infrastructure/data/"
-containerSasToken "$(data_containerSASToken)"
deploymentMode: 'Incremental'
On enabling the debug mode while running pipeline, I see values printed for data_containerSASToken and data_containerEndPoint from the task 'Copy functions templates' however I see empty values from the task 'deploy data resources'. What am I missing?
Your problem may be in when you retrieve the output from the previous job:
data_containerEndPoint: $[ dependencies.DeployResources.outputs['DeployResources.dataVariables.data_containerEndPoint'] ]
That's looking for a prior job called DeployResources, but the prior job is actually called {{ parameters.stageName }}_DeployResources.