I'm trying to build a new yaml file that reads keyvault secrets based on the parameters at the runtime and declared variables with the condition as per the parameters, but this isn't working.
- name: azure_subscription
displayName: " Select subscription "
type: string
default: "service-connection-dev"
values:
- 'service-connection-dev'
- 'service-connection-sit'
- 'service-connection-tes'
- 'service-connection-prd'
variables:
- ${{ if eq('${{ parameters.azure_subscription }}', 'service-connection-sit') }}:
name: key_vault
value: 'core-kv-sit'
- ${{ if eq('${{ parameters.azure_subscription }}', 'service-connection-dev') }}:
name: key_vault
value: 'core-kv-dev'
stages:
- stage: Validate
${{ if eq(parameters.azure_subscription, 'service-connection-dev') }}:
pool:
name: agent-pool-win-dev
${{ if eq(parameters.azure_subscription, 'service-connection-sit') }}:
pool:
name: agent-pool-win-sit
jobs:
- job: Validate
steps:
- task: AzureKeyVault#2
inputs:
KeyVaultName: "${{variables.key_vault}}"
SecretsFilter: "*"
RunAsPreJob: false
azureSubscription: ${{ parameters.azure_subscription }}
I've tried using variables inside jobs, but that is also not working. Can someone please help?
Also, I'll have to declare 2 more variables as per the parameters input, Is it possible ?
Thanks in advance
- ${{ if eq('${{ parameters.azure_subscription }}', 'service-connection-sit') }}:
You're using a literal value of "${{ parameters.azure_subscription }}" for the left side of the comparison. The comparison should just be parameters.azure_subscription.
So: - ${{ if eq(parameters.azure_subscription, 'service-connection-sit') }}:
Expanding on my first question. I am now experiencing a new error Unrecognized value: 'else'. Located at position 1 within expression: else that I cannot figure out a solution to. The issue happens when I add the 'else' condition within the foreach of the job.steps.
Build YAML
resources:
repositories:
- repository: self
type: git
ref: refs/heads/Development
- repository: AdoRestrictions
type: git
name: utl-yaml-templates
ref: refs/heads/main
trigger: none
pool:
name: PROD
extends:
template: ADO_Stage_Restrictions_Dev.yml#AdoRestrictions
parameters:
stageObjs:
- stage: 'BuildStage'
displayName: 'Build Test'
jobs:
- job: 'BuildJob'
displayName: 'Build'
steps:
- task: PowerShell#2
displayName: 'PS Hello World'
inputs:
targetType: inline
script: |
Write-Host "Hello World"
Working Extends YAML
parameters:
- name: stageObjs
type: stageList
default: []
stages:
- ${{ each stage in parameters.stageObjs }}:
- stage: ${{ stage.stage }}
displayName: ${{ stage.displayName }}
jobs:
- ${{ each job in stage.jobs }}:
- job: ${{ job.job }}
displayName: ${{ job.displayName }}
steps:
- ${{ each step in job.steps }}:
- ${{ if or(startsWith(step.task, 'PowerShell'),startsWith(step.task, 'CmdLine'),startsWith(step.task, 'Bash'),startsWith(step.task, 'ShellScript'),containsValue(step.task, 'Script'),containsValue(step.task, 'CLI'),containsValue(step.task, 'PowerShell')) }}:
- task: PowerShell#2
displayName: 'Unapproved - ${{ step.displayName }}'
inputs:
targetType: inline
script: Write-Output "Unapproved Task - Scripting and CLI tasks are not approved for use in yaml pipelines"
Broken Extends YAML
parameters:
- name: stageObjs
type: stageList
default: []
stages:
- ${{ each stage in parameters.stageObjs }}:
- stage: ${{ stage.stage }}
displayName: ${{ stage.displayName }}
jobs:
- ${{ each job in stage.jobs }}:
- job: ${{ job.job }}
displayName: ${{ job.displayName }}
steps:
- ${{ each step in job.steps }}:
- ${{ if or(startsWith(step.task, 'PowerShell'),startsWith(step.task, 'CmdLine'),startsWith(step.task, 'Bash'),startsWith(step.task, 'ShellScript'),containsValue(step.task, 'Script'),containsValue(step.task, 'CLI'),containsValue(step.task, 'PowerShell')) }}:
- task: PowerShell#2
displayName: 'Unapproved - ${{ step.displayName }}'
inputs:
targetType: inline
script: Write-Output "Unapproved Task - Scripting and CLI tasks are not approved for use in yaml pipelines"
- ${{ else }}:
- ${{ step }}
Full Error Message
(Line: 22, Col: 13): Unrecognized value: 'else'. Located at position 1 within expression: else. (Line: 22, Col: 13): Expected at least one key-value pair in the mapping
Update - Working Solution
parameters:
- name: stageObjs
type: stageList
default: []
stages:
- ${{ each stage in parameters.stageObjs }}:
- stage: ${{ stage.stage }}
displayName: ${{ stage.displayName }}
jobs:
- ${{ each job in stage.jobs }}:
- job: ${{ job.job }}
displayName: ${{ job.displayName }}
steps:
- ${{ each step in job.steps }}:
#Disallow any type of scripting tasks
- ${{ if or(startsWith(step.task, 'PowerShell'),startsWith(step.task, 'CmdLine'),startsWith(step.task, 'Bash'),startsWith(step.task, 'ShellScript'),contains(step.task, 'Script'),contains(step.task, 'CLI'),contains(step.task, 'PowerShell')) }}:
- task: PowerShell#2
displayName: 'Unapproved Task - ${{ step.displayName }}'
inputs:
targetType: inline
script: throw "Unapproved Task - Scripting and CLI tasks are not approved for use in yaml pipelines"
#Not a scripting task perform additional checks
- ${{ if and(not(startsWith(step.task, 'PowerShell')),not(startsWith(step.task, 'CmdLine')),not(startsWith(step.task, 'Bash')),not(startsWith(step.task, 'ShellScript')),not(contains(step.task, 'Script')),not(contains(step.task, 'CLI')),not(contains(step.task, 'PowerShell')) ) }}:
#Validate azure subscription provided for the environment
- ${{ if contains(step.task, 'Azure') }}:
- ${{ each pair in step }}:
${{ if eq(pair.key, 'inputs') }}:
inputs:
${{ each attribute in pair.value }}:
${{ if contains(attribute.key, 'Subscription') }}:
${{ if and(ne(attribute.value, 'sub-name1'), ne(attribute.value, 'sub-name2'), ne(attribute.value, 'sub-name3')) }}:
${{ attribute.value }}: ''
${{ if or(eq(attribute.value, 'sub-name1'), eq(attribute.value, 'sub-name2'), eq(attribute.value, 'sub-anme3')) }}:
${{ pair.key }}: ${{ pair.value }}
${{ if ne(pair.key, 'inputs') }}:
${{ if eq(pair.key, 'displayName') }}:
${{ pair.key }}: 'Invalid Azure Subscription - ${{ pair.value }}'
${{ if ne(pair.key, 'displayName') }}:
${{ pair.key }}: ${{ pair.value }}
#Allow all other tasks
- ${{ if not(contains(step.task, 'Azure')) }}:
- ${{ step }}
I am using outputs on each job as a hack to enable Github environments to control if my reusable workflow runs.
My only concern is the "ENV_AWS_ACCESS_KEY_ID" & "ENV_AWS_SECRET_ACCESS_KEY". These secrets are environment specific. How does the reusable workflow know what secret I am passing in?
Is there a risk with the current setup it could get overwritten if two environments got ran at the same time?
name: Used to rollback docker containers
on:
workflow_call:
inputs:
tag_to_identify_containers:
description: The last known containers prior to deployment
type: choice
required: true
options:
- last-known-testing
- last-known-integrate
- last-known-production
new_tag_to_apply_to_containers:
type: choice
required: true
options:
- testing-latest
- integrate-latest
- production-latest
jobs:
rollback_on_testing:
runs-on: ubuntu-latest
name: Rollback on testing
outputs:
signal_deployment: ${{ steps.step_id.outputs.environment }}
environment:
name: test
url: https://test.###/
steps:
- id: step_id
run: echo "::set-output name=environment::test"
retag_and_rollback_test:
needs: rollback_on_testing
if: needs.rollback_on_testing.outputs.signal_deployment == 'test'
uses: ###/###/.github/workflows/container-tagger.yml#main
with:
tag_to_identify_containers: ${{ github.event.inputs.tag_to_identify_containers }}
new_tag_to_apply_to_containers: ${{ github.event.inputs.new_tag_to_apply_to_containers }}
aws-region: eu-west-2
run_cron_and_cycle_containers: true
secrets:
AWS_ACCESS_KEY_ID: ${{ secrets.SHARED_AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.SHARED_AWS_SECRET_ACCESS_KEY }}
ENV_AWS_ACCESS_KEY_ID: ${{ secrets.THIS_AWS_ACCESS_KEY_ID }}
ENV_AWS_SECRET_ACCESS_KEY: ${{ secrets.THIS_AWS_SECRET_ACCESS_KEY }}
rollback_on_integrate:
runs-on: ubuntu-latest
name: Rollback on Integrate
outputs:
signal_deployment: ${{ steps.step_id.outputs.environment }}
environment:
name: integrate
url: https://integrate.###/
steps:
- id: step_id
run: echo "::set-output name=environment::integrate"
retag_and_rollback_integrate:
needs: rollback_on_integrate
if: needs.rollback_on_integrate.outputs.signal_deployment == 'integrate'
uses: ###/###/.github/workflows/container-tagger.yml#main
with:
tag_to_identify_containers: ${{ github.event.inputs.tag_to_identify_containers }}
new_tag_to_apply_to_containers: ${{ github.event.inputs.new_tag_to_apply_to_containers }}
aws-region: eu-west-2
run_cron_and_cycle_containers: true
secrets:
AWS_ACCESS_KEY_ID: ${{ secrets.SHARED_AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.SHARED_AWS_SECRET_ACCESS_KEY }}
ENV_AWS_ACCESS_KEY_ID: ${{ secrets.THIS_AWS_ACCESS_KEY_ID }}
ENV_AWS_SECRET_ACCESS_KEY: ${{ secrets.THIS_AWS_SECRET_ACCESS_KEY }}
rollback_on_production:
runs-on: ubuntu-latest
name: Rollback on Production
outputs:
signal_deployment: ${{ steps.step_id.outputs.environment }}
environment:
name: production
url: https://###/
steps:
- id: step_id
run: echo "::set-output name=environment::production"
retag_and_rollback_production:
needs: rollback_on_integrate
if: needs.rollback_on_integrate.outputs.signal_deployment == 'production'
uses: ###/###/.github/workflows/container-tagger.yml#main
with:
tag_to_identify_containers: ${{ github.event.inputs.tag_to_identify_containers }}
new_tag_to_apply_to_containers: ${{ github.event.inputs.new_tag_to_apply_to_containers }}
aws-region: eu-west-2
run_cron_and_cycle_containers: true
secrets:
AWS_ACCESS_KEY_ID: ${{ secrets.SHARED_AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.SHARED_AWS_SECRET_ACCESS_KEY }}
ENV_AWS_ACCESS_KEY_ID: ${{ secrets.THIS_AWS_ACCESS_KEY_ID }}
ENV_AWS_SECRET_ACCESS_KEY: ${{ secrets.THIS_AWS_SECRET_ACCESS_KEY }}
An idea would be to use a matrix for your GitHub reusable workflow.
name: Reusable workflow with matrix strategy
on:
push:
jobs:
ReuseableMatrixJobForDeployment:
strategy:
matrix:
stage: [test, integration, production]
uses: octocat/octo-repo/.github/workflows/deployment.yml#main
with:
environment: ${{ matrix.stage }}
tag_to_identify_containers: ${{ github.event.inputs.tag_to_identify_containers }}
new_tag_to_apply_to_containers: ${{ github.event.inputs.new_tag_to_apply_to_containers }}
aws-region: eu-west-2
run_cron_and_cycle_containers: true
secrets: inherit
When GitHub runs the workflow, your reusable workflow should have the environment "name" set to:
jobs:
rollback_on_testing:
runs-on: ubuntu-latest
name: Rollback on testing
outputs:
signal_deployment: ${{ steps.step_id.outputs.environment }}
environment:
name: ${{inputs.environment}}
url: https://test.###/
which should give you access to the environment's secrets inherited... "secrets: inherit".
Essentially I am trying to use a if else logic inside a yaml template instead of the caller pipeline.
I have following two pipelines azure-caller.yml and template.yaml
azure-caller.yml
parameters:
- name: test
displayName: 'select true or false'
values:
- true
- false
variables:
- name: test-true
${{ if eq(parameters.test, 'true') }}:
value: false
${{ elseif eq(parameters.test, 'false') }}:
value: true
stages:
- template: job-templates/template.yml
parameters:
testrue: $(test-true)
template.yml
parameters:
testrue: test_true
stages:
- stage: A
jobs:
- job: JA
steps:
- script: |
echo "Reverted value is" ${{ parameters.testrue }}
name: DetermineResult
How can I move the if else logic in the template.yml instead of azure-caller.yml? Your input will be helpful. thx
Below pipeline should do the trick with a limitation that the scope of variable test-true in the template will be limited only for stage: A
azure-caller.yml
parameters:
- name: test
displayName: 'select true or false'
values:
- true
- false
stages:
- template: job-templates/template.yml
parameters:
testrue: ${{ parameters.test }}
template.yml
parameters:
testrue: default
stages:
- stage: A
variables:
- name: test-true
${{ if eq(parameters.testrue, 'true') }}:
value: NewValueForTrueParam
${{ elseif eq(parameters.testrue, 'false') }}:
value: NewValueForFalseParam
jobs:
- job: JA
steps:
- script: |
echo "Echo value is" $(test-true)
Result
I want to pass variable group to my yaml template with deployment job.
Is it possible to use one of the values from variable group as environment name and connection name in deployment job?
My case is a little complicated, sorry for that :D
variables:
- group: ServiceVariables
pool: 'My Agent Pool'
extends:
template: /releases/deployment.dev.yml
parameters:
variableGroups:
- Gen2DEVVariables
- PaymentService
deployment.dev.yml
parameters:
- name: variableGroups
type: object
default: []
stages:
- stage: Deployment
variables:
- ${{ each value in parameters.variableGroups }}:
- group: ${{ value }}
jobs:
- template: /templates/jobs/deployToKubernetes.yml
parameters:
environmentName: $(azure_environment_name)
dockerRegistryConnection: $(serviceConnection_dockerRegistry)
kubernetesServiceConnection: $(serviceConnection_kubernetes)
variableGroups: ${{ parameters.variableGroups }}
and deployToKubernetes.yml
parameters:
- name: variableGroups
type: object
default: []
- name: environmentName
type: string
values:
- Develop
- Test
- UAT
- Production
- name: dockerRegistryConnection
- name: kubernetesServiceConnection
jobs:
- deployment: ${{ parameters.environmentName }}
environment: ${{ parameters.environmentName }}
variables:
- ${{ each value in parameters.variableGroups }}:
- group: ${{ value }}
strategy:
runOnce:
deploy:
steps:
- download: none
- task: KubernetesManifest#0
displayName: Create Secret
inputs:
action: 'createSecret'
kubernetesServiceConnection: ${{ parameters.kubernetesServiceConnection }}
namespace: 'value'
secretType: 'dockerRegistry'
secretName: 'value'
dockerRegistryEndpoint: ${{ parameters.dockerRegistryConnection }}
- task: KubernetesManifest#0
displayName: Deploy to Kubernetes cluster
inputs:
action: 'deploy'
kubernetesServiceConnection: ${{ parameters.kubernetesServiceConnection }}
namespace: 'value'
manifests: 'value'