I am trying to execute an Azure Devops Job on a specific pool based on a condition.
The goal is to switch between self-hosted agent and microsoft agent.
Here is the configuration:
parameters:
custom_agent: true
jobs:
- job: Test
displayName: Test job
- ${{ if eq(parameters.custom_agent, true) }}:
- pool:
name: mypool
demands:
- agent.os -equals Linux
- ${{ if eq(parameters.custom_agent, false) }}:
- pool:
vmImage: 'ubuntu-latest'
steps:
- task: npmAuthenticate#0
Any ideas ?
The below example solved my requirement
parameters:
- name: 'vmImage'
type: string
default: 'ubuntu-latest'
- name: 'agentPool'
type: string
default: ''
jobs:
- job: 'Example'
pool:
${{ if ne(parameters.agentPool, '') }}:
name: ${{ parameters.agentPool }}
${{ if eq(parameters.agentPool, '') }}:
vmImage: ${{ parameters.vmImage }}
steps:
- script: example
Another apporach to conditionally select pools if you use non-vm pools:
variables:
- ${{ if eq(parameters.custom_agent, true) }}:
- name: testJobPool
value: mypool
- ${{ if eq(parameters.custom_agent, false) }}:
- name: testJobPool
value: mypool_second
jobs:
- job: Test
displayName: Test job
pool:
name: $(testJobPool)
steps:
- task: npmAuthenticate#0
This has proved working.
We can specify conditions under which a step, job, or stage will run. We can configure the jobs in the pipeline with different condition entries, and set demands based on those conditions.
A skeleton version looks like this:
parameters:
- name: custom_agent
displayName: Pool Image
type: boolean
default: True
jobs:
- job: selfhostedagent
condition: eq(${{ parameters.custom_agent }}, True)
displayName: 'self_hosted agent'
pool:
name: Default
demands:
- Agent.Name -equals WS-VITOL-01
steps:
- script: echo self_hosted agent
- job: hostedagent
condition: eq(${{ parameters.custom_agent }}, False)
displayName: 'hosted agent'
pool:
vmImage: 'ubuntu-latest'
steps:
- script: echo hosted agent
Update1
In addition, we can configure task template, then use the template in the steps.
Result:
It looks like pool is not a valid property of a job type
Try switching your job to a deployment type:
jobs:
- deployment: Test
- ${{ if eq(parameters.custom_agent, true) }}:
pool:
name: mypool
demands:
agent.os -equals Linux
strategy:
runOnce:
deploy:
steps:
Related
I'm trying to build a new yaml file that reads keyvault secrets based on the parameters at the runtime and declared variables with the condition as per the parameters, but this isn't working.
- name: azure_subscription
displayName: " Select subscription "
type: string
default: "service-connection-dev"
values:
- 'service-connection-dev'
- 'service-connection-sit'
- 'service-connection-tes'
- 'service-connection-prd'
variables:
- ${{ if eq('${{ parameters.azure_subscription }}', 'service-connection-sit') }}:
name: key_vault
value: 'core-kv-sit'
- ${{ if eq('${{ parameters.azure_subscription }}', 'service-connection-dev') }}:
name: key_vault
value: 'core-kv-dev'
stages:
- stage: Validate
${{ if eq(parameters.azure_subscription, 'service-connection-dev') }}:
pool:
name: agent-pool-win-dev
${{ if eq(parameters.azure_subscription, 'service-connection-sit') }}:
pool:
name: agent-pool-win-sit
jobs:
- job: Validate
steps:
- task: AzureKeyVault#2
inputs:
KeyVaultName: "${{variables.key_vault}}"
SecretsFilter: "*"
RunAsPreJob: false
azureSubscription: ${{ parameters.azure_subscription }}
I've tried using variables inside jobs, but that is also not working. Can someone please help?
Also, I'll have to declare 2 more variables as per the parameters input, Is it possible ?
Thanks in advance
- ${{ if eq('${{ parameters.azure_subscription }}', 'service-connection-sit') }}:
You're using a literal value of "${{ parameters.azure_subscription }}" for the left side of the comparison. The comparison should just be parameters.azure_subscription.
So: - ${{ if eq(parameters.azure_subscription, 'service-connection-sit') }}:
Short question: how do I do the "enable" part in the pipeline below?
Long question: I have 3 jobs that I need to generate dynamically based on the parameters "params" due to business requirements. The job generated only works if the Job1, Job2 or Job3 is selected as true. Can I access parameters using string like dynamic in c#? ie: parameters["Job1"] or parameters["Job2"] or parameters["Job3"] ?
trigger: none
#Package Parameter
parameters:
- name: Job1
displayName: 'Job1'
type: boolean
default: false
- name: Job2
displayName: 'Job2'
type: boolean
default: false
- name: Job3
displayName: 'All Teams'
type: boolean
default: false
- name: 'params'
type: object
default: ["Job1",
"Job2",
"Job3"]
pool:
vmImage: ubuntu-latest
stages:
- stage: Build_dev
dependsOn:
jobs:
- ${{ each testJob in parameters.params }}:
- job:
enable: **If(parameters[testJob] == true) then enable this job**
displayName: Build ${{ testJob }}
steps:
- script: Build ${{ testJob }}
This is one way you can disable the jobs that are not set as true in the parameters. You can also use conditions for your jobs to skip them
azure-pipeline.yml
trigger: none
#Package Parameter
parameters:
- name: Job1
displayName: 'Job1'
type: boolean
default: false
- name: Job2
displayName: 'Job2'
type: boolean
default: false
- name: Job3
displayName: 'All Teams'
type: boolean
default: false
pool:
vmImage: ubuntu-latest
stages:
- stage: Build_dev
dependsOn: []
jobs:
- ${{ each parameter in parameters }}:
- ${{ if eq(parameter.Value, 'true') }}:
- template: job-template.yml
parameters:
jobName: ${{ parameter.Key }}
job-template.yml
parameters:
jobName: ''
jobs:
- job: ${{ parameters.jobName }}
displayName: Build ${{ parameters.jobName }}
steps:
- script: echo "Build ${{ parameters.jobName }}"
I have two Azure Devops pipelines: 'Starter' and 'Processing'. 'Starter' triggers 'Processing' and passes some parameters to it.
Starter:
trigger: none
pool:
vmImage: 'windows-2019'
stages:
- stage: A
jobs:
- template: Processing.yml
parameters:
products: $(Products)
creds: $(Creds)
Processing:
parameters:
- name: products
type: object
default: []
- name: creds
default: ''
jobs:
- ${{ each product in parameters.products }}:
- task: PowerShell#2
displayName: Importing ${{ product }} solution
inputs:
targetType: 'inline'
script: |
#Code
Key detail here is opportunity to loop through 'products' variable (each product in parameters.products), which must be setted in 'Starter' variables:
In other words, starting my pipeline I must pass list of products as 'string' and then loop through this list in second pipeline. 'Is it generally possible? Maybe products should be another type? I tried some work around but didn't get appropriate solution:
- job: Prepare_Products_Array
steps:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
# Write your PowerShell commands here.
$productsArray = []
$productsArray = $(Products)
$productsArray = $productsArray.Split(',')
Write-Host ("##vso[task.setvariable variable=productsArray;]$productsArray")
- template: Processing.yml
parameters:
products: $env:productsArray
Exception:
From your yaml sample, you are defining the variable in YAML Pipeline UI and using parameters in YAML Template.
In this case, the variables defined on the UI will be assigned at runtime, but the parameters and expressions in the YAML template will be expanded at compile time.
Therefore, YAML UI variables cannot be passed to the Pipeline YAML Template.
And it will show the error:
Expected a...... Actual value $(Product)
This means that the pipeline variable not expanded at compile time.
I am afraid that there is no such method can pass the UI Pipeline variable to YAML Template.
Here are the workarounds:
Method 1 : You can use parameters in Starter yaml to pass the Object type parameters to YAML template.
Starter:
trigger: none
parameters:
- name: products
type: object
default: []
- name: creds
default: ''
pool:
vmImage: 'windows-2019'
stages:
- stage: A
jobs:
- template: Processing.yml
parameters:
products: ${{ parameters.products }}
creds: ${{ parameters.creds }}
Processing:
parameters:
- name: products
type: object
default: []
- name: creds
default: ''
jobs:
- job: test
steps:
- ${{ each product in parameters.products }}:
- task: PowerShell#2
displayName: Importing ${{ product }} solution
inputs:
targetType: 'inline'
script: |
echo ${{ product }}
Result: You can input the value when you run the pipeline.
Method2: You need to define the variable in Starter pipeline and change the products parameters as String type. Then you can use the expression - ${{ each product in split(parameters.products, ',')}}: to split the string.
Starter:
pool:
vmImage: 'windows-2019'
variables:
products: '1,2,3'
creds: test
stages:
- stage: A
jobs:
- template: Processing.yml
parameters:
products: ${{ variables.products }}
creds: ${{ variables.creds }}
Processing:
parameters:
- name: products
type: string
default: ''
- name: creds
default: ''
jobs:
- job: test
steps:
- ${{ each product in split(parameters.products, ',')}}:
- task: PowerShell#2
displayName: Importing ${{ product }} solution
inputs:
targetType: 'inline'
script: |
echo ${{ product }}
Can anybody point me in the right direction here?
When I attempt to kick off the pipeline from the main yaml template the reference template gives me the unexpected 'steps' value.
I attempted to add a stage before the defined job and then receive the unexpected 'stage' message. Looking at some similar previously asked questions I understand a step can't be place directly under a 'stage' but this is not the case here.
parameters:
- name: baseEnvironmentName
type: string
- name: environmentSuffix
type: string
- name: postDeploySteps
type: stepList
default: []
- name: publishedPackageName
type: string
- name: serviceName
type: string
- name: dnsHostName
type: string
jobs:
- deployment: ${{ parameters.environmentSuffix }}Deployment
displayName: '${{ parameters.environmentSuffix }} Deployment'
pool:
vmImage: 'windows-2019'
environment: ${{ parameters.baseEnvironmentName }}-${{ parameters.environmentSuffix }}
variables:
- template: variables/variables.common.yaml
- template: variables/variables.${{ parameters.environmentSuffix }}.yaml
- name: MonitorExceptionAlert_Name
value: '${{ variables.IdPrefix }}-${{ parameters.environmentSuffix }}-${{ parameters.dnsHostName }}-ExceptionAlert'
steps:
- checkout: self
- checkout: common_iac
- task: Random Task Name
displayName: 'Create Environment Resource Group'
inputs:
ConnectedServiceName: '$(ADOS.ServiceConnectionName)'
resourceGroupName: '$(ResourceGroup.Name)'
location: '$(ResourceGroup.Location)'
condition: and(succeeded(), ne(variables['CreateResourceGroupInTaskGroup'], 'false'))
- task: Random Task Name 2
displayName: 'Create Application Insights'
inputs:
ConnectedServiceName: '$(ADOS.ServiceConnectionName)'
ResourceGroupName: '$(ResourceGroup.Name)'
Location: '$(ResourceGroup.Location)'
instanceName: '$(ApplicationInsights.Name)'
You are using deployment jobs. Try defining a strategy. Try replacing steps: inline with deployment with:
- deployment: ....
strategy:
runOnce:
deploy:
steps:
From the example at the Azure docs templates page, jobs: should contain a - job: which contains steps:. I don't know that jobs: can directly contain steps:.
I have a number of jobs for different platforms I'd like to run in parallel. I'd like to build a different set of platforms for different situations (i.e. full build, smoke, pull request, etc.). How can I make a list of jobs dynamic based on variables?
For example, if this is one of the "hard-coded" implementations:
jobs:
- job: Platform1
pool: Pool1
steps:
- template: minimal_template.yml
parameters:
BuildTarget: Platform1
- job: Platform2
pool: Pool1
steps:
- template: minimal_template.yml
parameters:
BuildTarget: Platform2
- job: Platform3
pool: Pool2
steps:
- template: minimal_template.yml
parameters:
BuildTarget: Platform3
How could I instead extract out a collection of variable sets, i.e.
[[Platform1, Pool1], [Platform2, Pool1], [Platform3, Pool2]]
And execute that on a pipeline like:
jobs:
??(Foreach platform in platforms)??
- job: $(platform[0])
pool: $(platform[1])
steps:
- template: minimal_template.yml
parameters:
BuildTarget: $(platform[0])
You can define it in the parameters and loop it:
parameters:
- name: Platforms
type: object
default:
- name: 'Platform1'
pool: 'Platform1Pool'
- name: 'Platform2'
pool: 'Platform2Pool'
jobs:
- ${{ each platform in parameters.Platforms}}:
- job: ${{ platform.name }}
pool: ${{ platform.pool }}
steps:
- template: minimal_template.yml
You may alos use 'jobList' type for template parameters:
parameters:
- name: 'testsJobsList'
type: jobList
default: []
jobs:
- ${{ each job in parameters.testsJobsList }}: # Each job
- ${{ each pair in job }}: # Insert all properties other than "steps"
${{ if ne(pair.key, 'steps') }}:
${{ pair.key }}: ${{ pair.value }}
steps: # Wrap the steps
- ${{ job.steps }} # Users steps
And then:
trigger:
- none
pool:
vmImage: 'windows-latest'
jobs:
- template: deployment-template.yml
parameters:
testsJobsList:
- job: Platform1
pool: Platform1Pool
steps:
- template: minimal_template.yml
- job: Platform2
pool: Platform2Pool
steps:
- template: minimal_template.yml
You’re looking for conditions: https://learn.microsoft.com/en-us/azure/devops/pipelines/process/conditions?view=azure-devops&tabs=yaml
Stages, jobs, and steps can all have a condition defined.
For example, running a job if a variable is set to true:
condition: eq(variables['System.debug'], 'true')