Use query-time variables in pipeline YAML - azure-devops

I have set up a pipeline with variables users can enter using the UI like this:
UI for userinput of variable called 'forceRelease'
I now want to use this variable in the pipeline yaml inside an if-statement like this:
jobs:
- job: Demo
steps:
- ${{ if eq(variables['forceRelease'], 'true') }}:
...some more stuff...
This does'nt work. I've tried different approaches but could not find the right syntax.
If I use the variable inside a condition, it works fine. Like this:
jobs:
- job: MAVEN_Build
- task: Bash#3
condition: eq(variables['forceRelease'], 'true')
I also tried to map the variable inside the variables block to a new pipeline variable like this:
variables:
isReleaseBranch: ${{ startsWith(variables['build.sourcebranch'],'refs/heads/pipelines-playground') }}
isForceRelease: $(forceRelease)
The first variable using 'build.sourcebranch' works fine. My approach using forceRelease doesnt work :(
Any ideas would be appreciated!
Cheers,
Dirk

AFAIK this is working as intended. User set variables are not expanded during parsing of the template.
You can read more on pipeline processing here:
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/runs?view=azure-devops
You should instead use parameters.
parameters:
- name: "forceRelease"
type: boolean
default: "false"
- name: "someOtherParameter"
type: string
default: "someValue"
stages:
- ${{ if eq(parameters['forceRelease'], true)}}:
- stage: build
jobs:
- job: bash_job
steps:
- task: Bash#3
inputs:
targetType: 'inline'
script: |
# Write your commands here
And then when you run the pipeline you have the option to enable the parameter forceRelease or add someOtherParameter string.

Related

Azure DevOps send parameters from pipeline to powershell

I tried to find some information in internet. But unfortunately I could not found any information.
Im trying to send pipeline parameters from pipeline into powershell script
pipeline below:
parameters:
- name: env
displayName: Select Environment
type: string
default: development
stages:
- stage: test
displayName: test var
jobs:
- job: PostgresSQL
steps:
- task: PowerShell#2
inputs:
filePath: '$(5ystem. DefaultWorkingDirectory)/test.psl'
errorActionPreference: 'continue'
enabled: true
I need to send ${{ parameters.env }} to powershell.
I tried different type of define param like a variable into powershell. but it does not work.
I would be very happy if anybody can help me and share relevant documentation for that.
Thanks all
First approach is to provide arguments using 'arguments' keyword (available in PowerShell by 'param')
filePath: xyz
arguments: -input1 ${{ parameters.env }}
documentation and example - https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/reference/powershell-v2?view=azure-pipelines#call-powershell-script-with-multiple-arguments
Second approach you can map parameters to environment variables provided to script using 'env' keyword
env:
input1: ${{ parameters.env }}
documentation - https://learn.microsoft.com/en-us/azure/devops/pipelines/process/tasks?view=azure-devops&tabs=yaml#environment-variables

use stageDependencies in if-statements in Azure DevOps yaml

I have a pretty complex setup of my Pipelines in Azure DevOps for various reasons but I'm kind of stuck in a special scenario now. Let me explain a bit.
There is a Stage_A with Job_A setting a Variable_A. Now there is a Stage_B with Job_B, need to use the Variable_A from Stage_A.Job_A.
The variable in Job_A is set by this:
echo ##vso[task.setvariable variable=Variable_A;isOutput=true]$value
Now, Job_B in Stage_B can access the variable in a condition with
variables:
Variable_A_FromStageA: $[stageDependencies.Stage_A.Job_A.outputs['task_A.Variable_A']]
I can also do an echo on the variable by using
echo $(Variable_A_FromStageA)
the Question is now, how can I use this in an if-statement? I tried different approaches:
- ${{ if eq($(Variable_A_FromStageA), 'True') }}:
- ${{ if eq(variables.Variable_A_FromStageA, 'True') }}:
- ${{ if eq(variables['Variable_A_FromStageA'], 'True') }}:
- ${{ if eq(stageDependencies.Stage_A.Job_A.outputs['task_A.Variable_A'], "True") }}:
Nothing actually works. Either the system complains about syntax issues or it doesn't evaluate it correctly. I don't really know how to use the information in my if statement in the yaml file. The documentation is not really clear about it. It only mentions the usage of a stage dependency in a condition and that's it.
Hope anyone can help me here!
Cheers,
Frank
use stageDependencies in if-statements in Azure DevOps yaml
If you mean you want to use conditional insertion to use the variables output from the logging command, then answer is NO.
The reason is the conditional insertion needs compile time value(you must provide them before pipeline run.), but the variable that the logging command output is runtime. Conditional Insertion will be unable to get it.
The right way is to use "condition" instead of "Conditional Insertion". Using condition can achieve your situation.
I write a demo for you as below:
trigger:
- none
pool:
vmImage: ubuntu-latest
stages:
- stage: A
jobs:
- job: A1
steps:
- bash: echo "##vso[task.setvariable variable=shouldrun;isOutput=true]true"
# or on Windows:
# - script: echo ##vso[task.setvariable variable=shouldrun;isOutput=true]true
name: printvar
- stage: B
condition: and(succeeded(), eq(dependencies.A.outputs['A1.printvar.shouldrun'], 'true'))
variables:
myStageAVar: $[stageDependencies.A.A1.outputs['printvar.shouldrun']]
dependsOn: A
jobs:
- job: B1
steps:
- script: echo $(myStageAVar)

Set Azure DevOps pipeline variable to array of values

I am trying to run a set of tests across a set of ADO builds.
I can retrieve the list of ADO builds using PowerShell. However, once I have the list, I need to export that to an ADO variable and then iterate across the values.
I've seen how to export values from Powershell to ADO using logging, but that appears to export the value as a string, not a list.
Is there a way to export variables so that I could iterate across them; e.g., using ${{ each foo in exportedVars }}?
First, for the usage you mentioned:
${{ each foo in exportedVars }}
This is a compile-time usage, it is expanded at the beginning, and you can't get the variables generated by the pipeline runtime through it.
Second, the pipeline can output variables through the logging command, but the variables set in this way can only be strings. This is by design, and the documentation has said it very clearly:
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch#set-variables-in-scripts
All variables set by this method are treated as strings.
It is not difficult to parse a string and put it into an array, just use the function(split()) that comes with the string type to split and restore.
Here is an example:
trigger:
- none
# 1
stages:
- stage: s1
displayName: setvars
jobs:
- job: testJob
steps:
- task: PowerShell#2
name: setvar
inputs:
targetType: 'inline'
script: |
# logic here. For example you get the vars and put it into this format:
$testvars = "testvar1,testvar2,testvar3"
Write-Host "##vso[task.setvariable variable=outputvars;isOutput=true]$testvars"
# 2
- stage: s2
displayName: getvars
dependsOn: s1
variables:
vars: $[ stageDependencies.s1.testJob.outputs['setvar.outputvars'] ]
jobs:
- job:
steps:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
$varsArr = "$(vars)".Split(',')
foreach ($var in $varsArr)
{
Write-Host "$var`r`n"
}
Result:

Is there a way to have a variable group defined at stage level? If so how to access it at Job Level?

I am trying to find a way to define a variable group at stage level and then access it in below jobs through a template? How would I go about doing this?
# Template file getchangedfilesandvariables.yaml
parameters:
- name: "previouscommitid"
type: string
steps:
- task: PowerShell#2
displayName: 'Get the changed files'
name: CommitIds
inputs:
targetType: 'filePath'
filePath: '$(Build.SourcesDirectory)\AzureDevOpsPipelines\Get-COChangedfiles.ps1'
arguments: >
-old_commit_id ${{ previouscommitid }}
- task: PowerShell#2
name: PassOutput
displayName: 'Getting Variables for Packaging'
inputs:
targetType: 'filepath'
filepath: '$(System.DefaultWorkingDirectory)\AzureDevOpsPipelines\Get-COADOvariables.ps1'
And below is my yaml file.
trigger: none
name: $(BuildID)
variables:
system.debug: true
CodeSigningCertThumbprint: "somethumbprint"
# Triggering builds on a branch itself.
${{ if startsWith(variables['Build.SourceBranch'], 'refs/heads/') }}:
branchName: $[ replace(variables['Build.SourceBranch'], 'refs/heads/', '') ]
# Triggering builds from a Pull Request.
${{ if startsWith(variables['Build.SourceBranch'], 'refs/pull/') }}:
branchName: $[ replace(variables['System.PullRequest.SourceBranch'], 'refs/heads/', '') ]
## it will create pipeline package and it will push it private or public feed artifacts
stages:
- stage: Stage1
variables:
- group: Cloudops
- name: oldcommitid
value: $[variables.lastcommitid]
jobs:
- job: IdentifyChangedFilesAndGetADOVariables
pool:
name: OnPrem
workspace:
clean: all # Ensure the agent's directories are wiped clean before building.
steps:
- powershell: |
[System.Version]$PlatformVersion = ((Get-Content "$(Build.SourcesDirectory)\AzureDevOpsPipelines\PlatformVersion.json") | ConvertFrom-Json).PlatformVersion
Write-Output "The repository's PlatformVersion is: $($PlatformVersion.ToString())"
$NewPackageVersion = New-Object -TypeName "System.Version" -ArgumentList #($PlatformVersion.Major, $PlatformVersion.Minor, $(Build.BuildId))
Write-Output "This run's package version is $($NewPackageVersion.ToString())"
echo "##vso[task.setvariable variable=NewPackageVersion]$($NewPackageVersion.ToString())"
echo "##vso[task.setvariable variable=commitidold;isOutput=true]$(oldcommitid)"
displayName: 'Define package version.'
name: commitidorpackageversion
errorActionPreference: stop
- template: getchangedfilesandvariables.yaml
parameters:
previouscommitid:
- $(commitidorpackageversion.commitidold)
# - $(oldcommitid)
I get the error at the second last line of the code that
/AzureDevOpsPipelines/azure-pipelines.yml (Line: 49, Col: 13): The 'previouscommitid' parameter is not a valid String.
I tried different combinations but I am still getting the errors.
Any ideas?
Thanks for your response. I already had the variable group setup in my library. I was just not able to use it.
The way I was able to achieve this I created another template file and supplied it to variables section under my stage. After doing this I was able to actually able to use the variables from my variable group in my successive jobs.
For more information you can review this doc : https://learn.microsoft.com/en-us/azure/devops/pipelines/library/variable-groups?view=azure-devops&tabs=yaml
stagevariables.yaml
variables:
- group: Cloudops
azure-pipelines.yml
stages:
- stage: Stage1
variables:
- template: stagevariables.yaml
jobs:
- job: CheckwhichfeedsAreAvailable
In YAML pipeline, you can't define a new variable group under the variables key.
Actually, we do not have the syntax can be available to create new variable group when running the YAML pipeline.
Under the variables key, you can:
Define new variables with the specified values.
Override the existing variables with new values.
Reference the variables from the existing variable groups and variable templates.
So, if you want to use a variable group with some variables in the pipeline, you should manually define the variable group on the Pipelines > Library page, then reference it in the pipeline.

Azure Devops - passing variables between job templates

Normal (non-template) jobs in Azure DevOps yaml support inter-job variable passing as follows:
jobs:
- job: A
steps:
- script: "echo ##vso[task.setvariable variable=skipsubsequent;isOutput=true]false"
name: printvar
- job: B
condition: and(succeeded(), ne(dependencies.A.outputs['printvar.skipsubsequent'], 'true'))
dependsOn: A
steps:
- script: echo hello from B
How do I do something similar in the following, given that templates don't support the dependsOn syntax? I need to get an output from the first template and pass it as 'environmentSlice' to the second template.
- stage: Deploy
displayName: Deploy stage
jobs:
- template: build-templates/get-environment-slice.yml#templates
parameters:
configFileLocation: 'config/config.json'
- template: build-templates/node-app-deploy.yml#templates
parameters:
# Build agent VM image name
vmImageName: $(Common.BuildVmImage)
environmentPrefix: 'Dev'
environmentSlice: '-$(dependencies.GetEnvironmentSlice.outputs['getEnvironmentSlice.environmentSlice'])'
The reason I want the separation between the two templates is the second one is a deployment template and I would like input from the first template in naming the environment in the second template. I.e. initial part of node-app-deploy.yml (2nd template) is:
jobs:
- deployment: Deploy
displayName: Deploy
# Because we use the environmentSlice to name the environment, we have to have it passed in rather than
# extracting it from the config file in steps below
environment: ${{ parameters.environmentPrefix }}${{ parameters.environmentSlice }}
Update:
The accepted solution does allow you to pass variables between separate templates, but won't work for my particular use case. I wanted to be able to name the 'environment' section of the 2nd template dynamically, i.e. environment: ${{ parameters.environmentPrefix }}${{ parameters.environmentSlice }}, but this can only be named statically since templates are compiled on pipeline startup.
The downside of the solution is that it introduces a hidden coupling between the templates. I would have preferred the calling pipeline to orchestrate the parameter passing between templates.
You can apply the depend on and dependency variable into templates.
See below sample:
To make sample more clear, here has 2 template files, one is azure-pipelines-1.yml, and another is azure-pipeline-1-copy.yml.
In azure-pipelines-1.yml, specify the environment value as output variable:
parameters:
  environment: ''
jobs:
- job: preDeploy
  variables:
    EnvironmentName: preDeploy-${{ parameters.environment }}
  steps:
  - checkout: none
  - pwsh: |
      echo "##vso[task.setvariable variable=EnvironmentName;isOutput=true]$($env:ENVIRONMENTNAME)"
    name: outputVars
And then, in azure-pipeline-1-copy.yml use dependency to get this output variable:
jobs:
- job: deployment
  dependsOn: preDeploy
  variables:
    EnvironmentNameCopy: $[dependencies.preDeploy.outputs['outputVars.EnvironmentName']]
  steps:
  - checkout: none
  - pwsh: |
      Write-Host "$(EnvironmentNameCopy)"
    name: outputVars
At last, in YAML pipeline, just need to pass the environment value
stages:
  - stage: deployQA
    jobs:
    - template: azure-pipelines-1.yml
      parameters:
        environment: FromTemplate1
    - template: azure-pipeline-1-copy.yml
Now, you can see the value get successfully in the second template job:
It is possible to avoid the dependency in the called template. However, as the OP says, the environment name cannot be created dynamically.
Here is an example of the "calling" template, which firstly calls another template (devops-variables.yml) that sets some environment variables that we wish to consume in a later template (devops-callee.yml):
stages:
- stage: 'Caller_Stage'
displayName: 'Caller Stage'
jobs:
- template: 'devops-variables.yml'
parameters:
InitialEnvironment: "Development"
- template: 'devops-callee.yml'
parameters:
SomeParameter: $[dependencies.Variables_Job.outputs['Variables_Job.Variables.SomeParameter']]
In the devops-variables.yml file, I have this:
"##vso[task.setvariable variable=SomeParameter;isOutput=true;]Wibble"
Then, in the "devops-callee.yml", I just consume it something like this:
parameters:
- name: SomeParameter
default: ''
jobs:
- deployment: 'Called_Job'
condition: succeeded()
displayName: 'Called Job'
environment: "Development"
pool:
vmImage: 'windows-2019'
dependsOn:
- Variables_Job
variables:
SomeParameter: ${{parameters.SomeParameter}}
strategy:
runOnce:
deploy:
steps:
- download: none
- task: AzureCLI#2
condition: succeeded()
displayName: 'An Echo Task'
inputs:
azureSubscription: "$(TheServiceConnection)"
scriptType: pscore
scriptLocation: inlineScript
inlineScript: |
echo "Before"
echo "$(SomeParameter)"
echo "After"
Output:
2021-04-10T09:22:29.6188535Z Before
2021-04-10T09:22:29.6196620Z Wibble
2021-04-10T09:22:29.6197124Z After
This way, the callee doesn't reference the caller. Unfortunately, setting the environment in the callee thus:
environment: "$(SomeParameter)"
doesn't work - you'll just get an environment with the literal characters '$(SomeParameter)'.