I'm creating a template in which I pass a job list. In this template, I add a "pre" job to the job list, and then add all jobs. I'd like to be able to add a dependency on the "pre" job to all jobs in the job list, without breaking any dependencies between the jobs in the joblist parameter. My template looks something like this:
parameters:
- name: Jobs
type: jobList
jobs:
- job: prejob
steps:
- powershell: |
Write-Host "This is my prejob"
- ${{ each job in parameters.Jobs }}:
- ${{ each pair in job }}:
${{ if and(ne(pair.key, 'steps'),ne(pair.key, 'dependsOn')) }}:
${{ pair.key }}: ${{ pair.value }}
dependsOn: 'prejob'
steps:
- ${{ job.steps }}
now this of course overwrites the dependsOn of the jobs passed.
What can I change in the template so I can join the passed job dependencies with the prejob job? The join expression joins 2 arrays, but my prejob text is not an array, and I'm not sure the dependsOn property of a job is always an array.
Please try this:
parameters:
- name: Jobs
type: jobList
jobs:
- job: prejob
steps:
- powershell: |
Write-Host "This is my prejob"
- ${{ each job in parameters.Jobs }}:
- ${{ each pair in job }}:
${{ if and(ne(pair.key, 'steps'),ne(pair.key, 'dependsOn')) }}:
${{ pair.key }}: ${{ pair.value }}
dependsOn:
- 'prejob'
- ${{ if job.dependsOn }}:
- ${{ job.dependsOn }}
steps:
- ${{ job.steps }}
In that way you should be able to make dependency to prejob and keep dependencies from jobs.
Related
I have an extend template that takes a stageList parameter. I need to check if the stageList contains a checkout task so I can insert one if it's missing. This is because I am checking out another repo later on and the default action if no checkout is defined is to checkout self, but if different repo is checked out the self checkout needs to be made explicitly. I need to add this so if the consumer did not have a checkout in their original pipeline it will still checkout their code.
My template takes a stageList parameter:
parameters:
- name: buildStages
type: stageList
default: []
How do I check for the existence the checkout task? I inspected the yaml and see the following structure:
stages:
- stage: ''
jobs:
- job: ''
steps:
- task: 6d15af64-176c-496d-b583-fd2ae21d4df4#1
I attempted to check for the task with the containsValue function, but this approach is obviously incorrect as the resulting yaml file still has two checkout tasks.
stages:
- ${{ each stage in parameters.buildStages }}:
- stage: ${{ stage.stage }}
jobs:
- ${{ each job in stage.jobs }}:
- ${{ each pair in job }}:
${{ if ne(pair.key, 'steps') }}:
${{ pair.key }}: ${{ pair.value }}
steps:
- ${{ if not(containsValue(parameters.buildStages.stage.jobs.job.steps.task.value, '6d15af64-176c-496d-b583-fd2ae21d4df4#1'))}}:
- checkout: self
- ${{ job.steps }}
I found how to reference the arrays in the buildstages via this link
Solution:
steps:
- ${{ if not(containsValue(job.steps.*.task, '6d15af64-176c-496d-b583-fd2ae21d4df4#1')) }}:
- checkout: self
displayName: base-Checkout
Humorously, before I found that I did get this working by iterating through all the tasks and setting a variable to true if the checkout task was encountered. Happy to have found the article above and removed that code from my template.
We have two environments defined, "stage" and "prod". Prod requires approval for deploying, where as stage does not.
What I'm trying to achieve is that if the pipeline runs against the master branch, it should use the prod environment, and all other branches should use the stage environment.
Problem is that I can't seem to be able to change the environment in runtime. I've also tried using templates, which sort of works, except that I can't pass runtime information as parameters to the template.
I also tried using two deployment jobs, one for each environment and then chose job based on conditions, however, as soon as the pipeline involves the prod environment, whether it's going to be used or not, it will ask for approval.
I will start with first last one:
I also tried using two deployment jobs, one for each environment and then chose job based on conditions, however, as soon as the pipeline involves the prod environment, whether it's going to be used or not, it will ask for approval.
Resource restriction are evaluated on stage level so if you put those deployment jobs on the same stage (or you didn't define a stage - then you have implicit stage) - this is the reason why it asks for approval. If you move deployment jobs to seprate stages you won't be asked for approval. Please keep in mind to move condition to a stage level.
Another apporach it would be conditional for envrionment as follows:
jobs:
- deployment: DeployWeb
displayName: deploy Web App
${{ if eq(variables['Build.SourceBranchName'], 'master') }}:
environment: 'Dev'
${{ if ne(variables['Build.SourceBranchName'], 'master') }}:
environment: 'campo-dev'
strategy:
# Default deployment strategy, more coming...
runOnce:
deploy:
steps:
- checkout: self
- script: echo my first deployment
I do this with an extends yaml. You can define your stages via the environment's parameter (defaulted) and then your calling yaml extends it and passes tasks for build and deploy. The build section is ran once. And the deploy section will loop for each environment.
The approvals comes from the Environment approvals setup in Devops
#Extends Yaml
parameters:
- name: buildSteps
type: stepList
default: []
- name: deploySteps
type: stepList
default: []
- name: environments
type: object
default:
- name: Stage
branch: any
dependsOn:
- Builds
- name: Prod
branch: any
dependsOn:
- Stage
stages:
- stage: Builds
jobs:
- job: Building
steps:
- ${{ each step in parameters.buildSteps }}:
- ${{ each pair in step }}:
${{ pair.key }}: ${{ pair.value }}
- ${{ each env in parameters.environments }}:
- stage: ${{ env.name }}
dependsOn: ${{ env.dependsOn }}
condition: and(succeeded(), or(startsWith(variables['Build.SourceBranch'], 'refs/heads/${{ env.branch }}'), eq('${{ env.branch }}', 'any')))
jobs:
- deployment: Deploy${{ env.name }}
displayName: Deploy to ${{ env.name }}
variables:
- group: ENV-${{ env.name }}
environment: ${{ env.name }}
strategy:
runOnce:
deploy:
steps:
- ${{ each step in parameters.deploySteps }}:
- ${{ each pair in step }}:
${{ pair.key }}: ${{ pair.value }}
#Calling Yaml
#*Trimmed* header stuff
resources:
repositories:
- repository: cicd
type: git
name: SharedRepo
ref: main
extends:
template: extends-deploy.yaml#cicd
parameters:
buildSteps:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: Write-Host 'Build Stuff 1'
- task: PowerShell#2
inputs:
targetType: 'inline'
script: Write-Host 'Build Stuff 2'
deploySteps:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: Write-Host 'Deploy Stuff 1'
- task: PowerShell#2
inputs:
targetType: 'inline'
script: Write-Host 'Deploy Stuff 2'
I have below main.yml build pipeline,
parameters:
- name: ListOfEnvironments
type: object
default:
- dev
- tst
- acc
- prd
stages:
- ${{ each value in parameters.ListOfEnvironments }}:
- template: templates/stage.yml
parameters:
environment_name: ${{ value }}
Now above template works and able to call different env specific stage.yml templates.
now inside stage.yml I have the below structure:
stages:
- stage: deploy_to_env
displayName: Deploy
jobs:
- job: deploy
displayName: Deploy
variables:
- name: environment
value: ${{ parameters.environment_name }}
steps:
${{ if eq(parameters.environment_name, "dev") }}:
template: ../steps_templates/dev_steps.yml
${{ if eq(parameters.environment_name, "acc") }}:
template: ../steps_templates/acc_steps.yml
I'm not able to call steps template based on the conditions. Also, I tried to do like below
${{ if eq(parameters.environment_name, "dev") }}:
steps:
- template: ../steps_templates/dev_steps.yml
${{ if eq(parameters.environment_name, "acc") }}:
steps:
- template: ../steps_templates/acc_steps.yml
It's not working, does anyone have any idea how to call steps based on the conditional parameters.
UPDATE:
I finally managed to go further like this, where it recognized the env variable but now it complains Unrecognized value: '"dev"'. Located at position 33 within expression: eq(parameters.environment_name, dev)
steps:
- ${{ if eq(parameters.environment_name, "dev") }}:
- template: ../steps_templates/dev_steps.yml
- ${{ if eq(parameters.environment_name, "acc") }}:
- template: ../steps_templates/acc_steps.yml
it's working now, after changing " to ' . It's all good now.
May be someone will find this helpful:
The below layout will work fine:
steps:
- ${{ if eq(parameters.environment_name, 'dev') }}:
- template: ../steps_templates/dev_steps.yml
- ${{ if eq(parameters.environment_name, 'acc') }}:
- template: ../steps_templates/acc_steps.yml
Is there a way to reference a job parameter's name in an Azure DevOps YAML template? I know that I could pass in the job name as its own string parameter, but I was hoping for something that's a little less clumsy.
template.yml
parameters:
- name: MyJob
type: job
jobs:
- ${{ parameters.MyJob }}
- job: Job2
dependsOn: # How to make this depend on MyJob?
azure-pipelines.yml
stages:
- stage: Stage1
jobs:
- template: template.yml
parameters:
MyJob:
job: SomeJobName
steps:
- script: echo Hello
I tried accessing ${{ parameters.MyJob.name }} but it doesn't appear to exist.
I've figured out, though it took a lot of trial and error to get the exact syntax and spacing right. The docs aren't super clear about spacing and expressions, and when to lead with a dash.
parameters:
- name: MyJob
type: job
jobs:
- ${{ parameters.MyJob }}
- job: Job2
${{ each pair in parameters.MyJob }}:
${{ if eq(pair.key, 'job') }}:
dependsOn: ${{ pair.value }}
steps:
...
Can't comment, but as you see in previous answer, the property is not "name", but "job". You don't need the foreach then:
parameters:
- name: MyJob
type: job
jobs:
- ${{ parameters.MyJob }}
- job: Job2
dependsOn: ${{ parameters.MyJob.job }}
steps:
...
I'm trying to pass stage list from azure pipeline shown below
# File: azure-pipelines.yml
trigger:
- master
extends:
template: start_stage.yml
parameters:
cdstages:
- stage: secure_buildstage
pool: Hosted VS2017
jobs:
- job: secure_buildjob
steps:
- bash: echo This happens before code
displayName: 'Base: Pre-build'
- bash: echo Building
displayName: 'Base: Build'
- bash: echo This happens after code
displayName: 'Base: Signing'
- stage: secure_deploystage
pool: Hosted VS2017
jobs:
- job: secure_deployjob
steps:
- bash: echo This happens before code
displayName: 'Base: Pre-build'
- bash: echo Building
displayName: 'Base: Build'
- script: echo This happens after code
displayName: 'Base: Signing'
to extend template shown below
parameters:
- name: cdstages # the name of the parameter is buildSteps
type: stageList # data type is StepList
default: [] # default value of buildSteps
stages:
- ${{ each stage in parameters.cdstages }}:
- ${{ each job in stage.jobs }}:
- ${{ each step in job.steps }}:
- ${{ each pair in step }}:
${{ if ne(pair.value, 'CmdLine#2') }}:
${{ pair.key }}: ${{ pair.value }}
${{ if eq(pair.value, 'CmdLine#2') }}:
'${{ pair.value }}': error
The goal is to take stage list and validate if users are only running steps approved by firm's compliance team.
I'm getting error
[][1
not sure why getting "task" error, there are no task keywords used anywhere.
Any help?
Based on my test , it seems that the stagelist in start_stage.yml doesn't support to add eachdirective to get deeper content (e.g. job and steps).
When you use the stagelist, it could get the stage and use it for comparison.
For example:
parameters:
- name: cdstages # the name of the parameter is buildSteps
type: stageList # data type is StepList
default: [] # default value of buildSteps
stages:
- ${{ each stage in parameters.cdstages }}:
- ${{ each pair in stage }}:
${{ if ne(pair.value, 'abc') }}:
${{ pair.key }}: ${{ pair.value }}
${{ if eq(pair.value, 'abc') }}:
'${{ pair.value }}': error
This Yaml template could work.
But when I add the each directive behind the stage to get the jobs. The jobs are is not available.
parameters:
- name: cdstages
type: stageList
default: []
stages:
- ${{ each stage in parameters.cdstages }}:
- ${{ each job in stage.jobs }}:
- ${{ each pair in job }}:
${{ if ne(pair.value, 'abc') }}:
${{ pair.key }}: ${{ pair.value }}
${{ if eq(pair.value, 'abc') }}:
'${{ pair.value }}': error
According to your requirements, you need to get the build step and use it for comparison.
You could try to directly use the steplist type.
Here is an example about steplist, you could refer to it.
Hope this helps.