Looking to try and get some help figuring out a solution to what seems like a simple task. I am trying to develop some extend YAML templates for security within our ADO pipelines but first I have to get passed this error I am experiencing.
The error being returned by ADO when I try to validate or run the pipeline is
/ADO_Stage_Restrictions_Dev.yml#AdoRestrictions (Line: 7, Col: 3): Unexpected value job
Working Extend Template YAML
This template validates and executes without issue, which to me means I am successfully passing the stages object into the extends template
parameters:
- name: stageObjs
type: stageList
default: []
stages:
- ${{ each stage in parameters.stageObjs }}:
${{ stage }}
Broken Extend Template YAML
This template does not validate and throws the 'Unexpected value job' exception. based on the stage schema I would assume that I would be able to loop the jobs property within the stage.
parameters:
- name: stageObjs
type: stageList
default: []
stages:
- ${{ each stage in parameters.stageObjs }}:
- ${{ each job in stage.jobs }}:
${{ job }}
Build YAML
The main yaml file that extends stages
resources:
repositories:
- repository: self
type: git
ref: refs/heads/Development
- repository: AdoRestrictions
type: git
name: utl-yaml-templates
ref: refs/heads/main
trigger: none
pool:
name: PROD
extends:
template: ADO_Stage_Restrictions_Dev.yml#AdoRestrictions
parameters:
stageObjs:
- stage: 'BuildStage'
displayName: 'Build Test'
jobs:
- job: 'BuildJob'
displayName: 'Build'
steps:
- task: PowerShell#2
displayName: 'Hello World'
inputs:
targetType: inline
script: |
Write-Host "Hello World"
There's nothing in your yaml which defines the start of the stage, or declares the start of the list of jobs; that's why it wasn't expecting to find 'job' there. You can add those parts in, like this:
parameters:
- name: stageObjs
type: stageList
default: []
stages:
- ${{ each stage in parameters.stageObjs }}:
- stage: ${{ stage.stage }}
displayName: ${{ stage.displayName }}
jobs:
- ${{ each job in stage.jobs }}:
${{ job }}
Related
I am trying to make a pipeline template that takes a JobList a parameter and runs all the jobs, while ensuring that they run on the same agent every time. Basically the approach I've been taking is to try to adapt this answer into a genericized template format.
This is what I have so far, and I've tried a lot of slight tweaks of this with nothing passing the Validate test on the pipeline that calls it.
parameters:
- name: jobsToRun
type: jobList
- name: pool
type: string
default: Default
- name: demands
type: object
default: []
jobs:
- job:
steps:
- script: echo "##vso[task.setvariable variable=agentName;isOutput=true;]$(Agent.Name)"
pool:
name: ${{ parameters.pool }}
demands:
- ${{ each demand in parameters.demands }}:
${{ demand }}
- ${{ each j in parameters.jobsToRun }}:
${{ each pair in j }}:
${{ pair.key }} : ${{ pair.value }}
pool:
name: Default
demands:
- Agent.Name -equals $(agentName)
What am I doing wrong here? It seems like it should be possible if that answer I reference is correct, but it seems like I'm just a bit off.
Name missing on the job.., example below.
- job: 'test-Name'
Steps need a associated job and pool to run is declared inside
jobsToRun:
- job: sample_job1
displayName: "sample_job1"
pool:
name: "your_PoolName"
steps:
- script: |
echo "Hi"
On this bottom pool declaration...
pool:
name: Default
demands:
- Agent.Name -equals $(agentName)
i am not sure but i have tried this many times but i think this can't be included separate to the job since each individual job is passed as parameter. pool definition needs to be inside the job or inside the job template if you are using templates..
example:
jobsToRun:
- job: output_message_job1
displayName: "in pipe Output Message Job"
pool:
name: "your_PoolName"
steps:
- script: |
echo "Hi"
We have two environments defined, "stage" and "prod". Prod requires approval for deploying, where as stage does not.
What I'm trying to achieve is that if the pipeline runs against the master branch, it should use the prod environment, and all other branches should use the stage environment.
Problem is that I can't seem to be able to change the environment in runtime. I've also tried using templates, which sort of works, except that I can't pass runtime information as parameters to the template.
I also tried using two deployment jobs, one for each environment and then chose job based on conditions, however, as soon as the pipeline involves the prod environment, whether it's going to be used or not, it will ask for approval.
I will start with first last one:
I also tried using two deployment jobs, one for each environment and then chose job based on conditions, however, as soon as the pipeline involves the prod environment, whether it's going to be used or not, it will ask for approval.
Resource restriction are evaluated on stage level so if you put those deployment jobs on the same stage (or you didn't define a stage - then you have implicit stage) - this is the reason why it asks for approval. If you move deployment jobs to seprate stages you won't be asked for approval. Please keep in mind to move condition to a stage level.
Another apporach it would be conditional for envrionment as follows:
jobs:
- deployment: DeployWeb
displayName: deploy Web App
${{ if eq(variables['Build.SourceBranchName'], 'master') }}:
environment: 'Dev'
${{ if ne(variables['Build.SourceBranchName'], 'master') }}:
environment: 'campo-dev'
strategy:
# Default deployment strategy, more coming...
runOnce:
deploy:
steps:
- checkout: self
- script: echo my first deployment
I do this with an extends yaml. You can define your stages via the environment's parameter (defaulted) and then your calling yaml extends it and passes tasks for build and deploy. The build section is ran once. And the deploy section will loop for each environment.
The approvals comes from the Environment approvals setup in Devops
#Extends Yaml
parameters:
- name: buildSteps
type: stepList
default: []
- name: deploySteps
type: stepList
default: []
- name: environments
type: object
default:
- name: Stage
branch: any
dependsOn:
- Builds
- name: Prod
branch: any
dependsOn:
- Stage
stages:
- stage: Builds
jobs:
- job: Building
steps:
- ${{ each step in parameters.buildSteps }}:
- ${{ each pair in step }}:
${{ pair.key }}: ${{ pair.value }}
- ${{ each env in parameters.environments }}:
- stage: ${{ env.name }}
dependsOn: ${{ env.dependsOn }}
condition: and(succeeded(), or(startsWith(variables['Build.SourceBranch'], 'refs/heads/${{ env.branch }}'), eq('${{ env.branch }}', 'any')))
jobs:
- deployment: Deploy${{ env.name }}
displayName: Deploy to ${{ env.name }}
variables:
- group: ENV-${{ env.name }}
environment: ${{ env.name }}
strategy:
runOnce:
deploy:
steps:
- ${{ each step in parameters.deploySteps }}:
- ${{ each pair in step }}:
${{ pair.key }}: ${{ pair.value }}
#Calling Yaml
#*Trimmed* header stuff
resources:
repositories:
- repository: cicd
type: git
name: SharedRepo
ref: main
extends:
template: extends-deploy.yaml#cicd
parameters:
buildSteps:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: Write-Host 'Build Stuff 1'
- task: PowerShell#2
inputs:
targetType: 'inline'
script: Write-Host 'Build Stuff 2'
deploySteps:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: Write-Host 'Deploy Stuff 1'
- task: PowerShell#2
inputs:
targetType: 'inline'
script: Write-Host 'Deploy Stuff 2'
My problem
I try to create templated yaml Azure Devops pipeline:
parameters:
- name: envs
type: object
default:
- QUAL
- PROD
- PREPROD
stages:
- template: Orchestrator.yml
parameters:
name: envs
type: object
default:
- QUAL
- PROD
- PREPROD
This is my template:
parameters:
envs: {}
stages:
- ${{ each env in parameters.envs }}:
- stage: ${{ env }}
jobs:
- job: Deploy
steps:
- script: echo Deploy project
displayName: 'Deploy'
- job: Tests
steps:
- script: echo Unit tests
displayName: Test 1
But I get this error message:
An error occurred while loading the YAML build pipeline. The array
must contain at least one element. Parameter name: stages
OK, I modify my main script like this:
parameters:
- name: envs
type: object
default:
- QUAL
- PROD
- PREPROD
stages:
- stage: Build
jobs:
- job: Build
steps:
- script: echo Compilation completed...
displayName: 'Compile'
- template: Orchestrator.yml
parameters:
name: envs
type: object
default:
- QUAL
- PROD
- PREPROD
This time pipeline runs, but only first Job. The Template is not loaded.
What I need
I was able to make this scenario working with a single file script, but I would like to make it working with a templated script.
Is this scenario supported? How I can do?
thanks
Here I have a sample as reference:
In the template YAML file (here I name it template.yaml), write as this.
parameters:
- name: envs
type: object
default:
- QUAL
- PROD
- PREPROD
stages:
- ${{ each env in parameters.envs }}:
- stage: ${{ env }}
displayName: 'Stage ${{ env }}'
jobs:
- job: job1
displayName: 'Job 1'
steps:
- bash: echo "Current job is job1 in ${{ env }}"
- job: job2
displayName: 'Job 2'
steps:
- bash: echo "Current job is job2 in ${{ env }}"
In the pipeline YAML (here I name it pipeline.yaml), write as this.
trigger:
- main
extends:
template: template.yaml
Result
If you do not want to hardcode the value of the parameter 'envs' in template.yaml, you can write like as below.
In template.yaml write as this.
parameters:
- name: envs
type: object
default: []
stages:
- ${{ each env in parameters.envs }}:
- stage: ${{ env }}
displayName: 'Stage ${{ env }}'
jobs:
- job: job1
displayName: 'Job 1'
steps:
- bash: echo "Current job is job1 in ${{ env }}"
- job: job2
displayName: 'Job 2'
steps:
- bash: echo "Current job is job2 in ${{ env }}"
In pipeline.yaml write as this.
trigger:
- main
stages:
- template: template.yaml
parameters:
envs:
- QUAL
- PROD
- PREPROD
Result. Same as above.
To view more details, you can see "YAML schema reference".
Is there a way to reference a job parameter's name in an Azure DevOps YAML template? I know that I could pass in the job name as its own string parameter, but I was hoping for something that's a little less clumsy.
template.yml
parameters:
- name: MyJob
type: job
jobs:
- ${{ parameters.MyJob }}
- job: Job2
dependsOn: # How to make this depend on MyJob?
azure-pipelines.yml
stages:
- stage: Stage1
jobs:
- template: template.yml
parameters:
MyJob:
job: SomeJobName
steps:
- script: echo Hello
I tried accessing ${{ parameters.MyJob.name }} but it doesn't appear to exist.
I've figured out, though it took a lot of trial and error to get the exact syntax and spacing right. The docs aren't super clear about spacing and expressions, and when to lead with a dash.
parameters:
- name: MyJob
type: job
jobs:
- ${{ parameters.MyJob }}
- job: Job2
${{ each pair in parameters.MyJob }}:
${{ if eq(pair.key, 'job') }}:
dependsOn: ${{ pair.value }}
steps:
...
Can't comment, but as you see in previous answer, the property is not "name", but "job". You don't need the foreach then:
parameters:
- name: MyJob
type: job
jobs:
- ${{ parameters.MyJob }}
- job: Job2
dependsOn: ${{ parameters.MyJob.job }}
steps:
...
I have scaleset agents configured and working fine. But i would also like to take advantage of Microsoft Hosted agents whenever possible. So i added a parameter to the pipeline to choose which agent pool to use for that particular pipeline run.
I manged to set the pool/image variables based on the chosen parameter but unable to make a container job use the selected pool due to a difference in syntax as shown below.
Is there a way to modify the schema of yaml template based on the user selected parameter.
I think we need to not display vmImage property conditionally. I tried below snippet but it doesn't work. vmImage property is always missing.
pool:
${{ if ne(parameters.pool, 'Azure Pipelines') }}:
name: ${{ parameters.pool }}
${{ if eq(parameters.pool, 'Azure Pipelines') }}:
vmImage: 'ubuntu-latest'
UPDATED:
parameters:
- name: pool
displayName: Agent Pool
type: string
default: Scalesets
values:
- Scalesets
- Hosted
jobs:
- deployment: '${{ parameters.jobName }}'
displayName: ${{ coalesce(parameters.jobDisplayName, 'Deploy Infrastructure') }}
${{ if parameters.dependsOn }}:
dependsOn: '${{ parameters.dependsOn }}'
${{ if parameters.condition }}:
condition: '${{ parameters.condition }}'
pool:
name: ${{ parameters.pool }}
vmImage: 'ubuntu-latest'
container:
image: '${{ parameters.image }}'
endpoint: 'Container Registry - Prd'
environment: ${{ parameters.environment }}
variables:
- name: 'Release-Tag'
value: '${{ parameters.release }}'
- name: 'Layer'
value: '${{ parameters.environmentLayer }}'
strategy:
runOnce:
deploy:
steps:
- script: |
echo $(Release-Tag) $(Layer)
displayName: 'Show release condidate'
You can set the data type of your parameters as object. The value of object type can be any YAML structure. Please find more details about data types of parameters in this document.
Here is my sample:
parameters:
- name: pool
type: object
default:
pool:
vmImage: ubuntu-latest
jobs:
- job: build
pool: ${{ parameters.pool }}
steps:
- script: echo Hello, world!
You can also choose the job to run based on the value of your parameters:
trigger: none
parameters:
- name: type
type: string
values:
- Scalesets
- Hosted
jobs:
- ${{ if eq(parameters.type, 'Scalesets') }}:
- job: Scalesets
pool: Default
steps:
- script: echo Scalesets
- ${{ if eq(parameters.type, 'Hosted') }}:
- job: Hosted
pool:
vmImage: ubuntu-latest
steps:
- script: echo Hosted