Azure Devops YAML - Dynamic Condition Evaluation - azure-devops

I am attempting to use a condition to break out of a loop based on the result of a sql query. So I am assigning the query output to a variable "sqloutput" and if that value is empty string go around again. I have confirmed that the query is outputting a value however on subsequent loops the variable is still evaluating to NULL.
Also tried to explicitly declare the variable to no avail.
parameters:
- name: loop
type: object
default : ["1","2","3","4","5","6","7","8","9","10"]
stages:
- stage: WaitforProcessing
displayName: Wait For Processing
jobs:
- deployment: Deployment
pool:
vmImage: ubuntu-latest
environment:
name: 'Client_Data_Tests'
resourceType: VirtualMachine
strategy:
runOnce:
deploy:
steps:
- ${{each Looptimes in parameters.loop}}:
- task: PowerShell#2
name: checkImportProgress${{ Looptimes }}
condition: and(succeeded(), eq(variables['sqloutput'], ''))
displayName: Check if job finished
inputs:
targetType: inline
script: |
$query = "IF EXISTS (select 1 from Job where JobFinishTime is null)
BEGIN
select '' as result
END
ELSE
select '1' as result
"
$sqloutput = Invoke-Sqlcmd -query $query -ServerInstance "$(DB_SERVER)" -Database "$(DB_DATABASE)" -Username "$(DB_USERNAME)" -Password "$(DB_PASSWORD)" -QueryTimeout 36000 -Verbose
$value = $sqloutput["result"]
Write-Host $value
Write-Host "##vso[task.setvariable variable=sqloutput]$value"
- task: PowerShell#2
condition: and(succeeded(), eq(variables['sqloutput'], ''))
displayName: 'Sleep for 30 seconds'
inputs:
targetType: inline
script: |
Start-Sleep 30

The below YAML should work.
stages:
- stage: A
jobs:
- job: A
displayName: SQL generate variable
steps:
- powershell: |
$value = 'somevalue'
echo "##vso[task.setvariable variable=sqloutput;isOutput=true]$value"
name: setvarStep
- script: echo $(setvarStep.sqloutput)
name: echovar
- job: B
dependsOn: A
displayName: Use variable that SQL script output as condition
variables:
myVarFromJobA: $[ dependencies.A.outputs['setvarStep.sqloutput'] ] # map in the variable
myvar: true
condition: eq(variables['myVarFromJobA'],'somevalue')
steps:
- script: echo $(myVarFromJobA)
name: echovar
Result:

Related

How to pass a value between stages in a pipeline returned by invoking a PowerShell script?

I have an Azure DevOps pipeline:
stages:
- stage: Stage1
jobs:
- job: RunScript
steps:
- task: PowerShell#2
inputs:
pwsh: true
targetType: 'filePath'
filePath: $(System.DefaultWorkingDirectory)/scripts/myscript1.ps1
- stage: Stage2
jobs:
- job: RunScript
steps:
- task: PowerShell#2
inputs:
pwsh: true
targetType: 'filePath'
filePath: $(System.DefaultWorkingDirectory)/scripts/myscript1.ps1
arguments: >
-Arg ??? # <======= HERE
Stage1 runs myscript1.ps1 which is (for simplicity):
$OutputValue = 'Hello'
$OutputValue
It is just a script which will eventually return something.
The problem
The issue is that the value produced by myscript1.ps1 in Stage1 must be consumed by myscript2.ps1 in Stage2.
How can I achieve this?
You need to use output variables as here
With the script myscript1.ps1 as this:
echo "##vso[task.setvariable variable=myOutputVar;isOutput=true]this is the value"
and pipeline:
stages:
- stage: A
jobs:
- job: A1
steps:
- task: PowerShell#2
inputs:
pwsh: true
targetType: 'filePath'
filePath: $(System.DefaultWorkingDirectory)/scripts/myscript1.ps1
name: setvarStep
- stage: B
dependsOn: A
variables:
myVarfromStageA: $[ stageDependencies.A.A1.outputs['setvarStep.myOutputVar'] ]
jobs:
- job: B1
steps:
- script: echo $(myVarfromStageA)
You can retrive value in the next stage.
It doesn't matter if you use it inside the script file. But important here is to have step named as later you will use it to reference output variable created by it.

Azure DevOps variable empty between steps

I am trying to read a text file using bash in a AZDO pipeline template, but for some reason the variable containing the name of the file is empty:
##deploy-to-env.yml
parameters:
- name: env
type: string
default: ""
- name: envLong
type: string
default: ""
stages:
- stage: "deployTo${{ parameters.envLong }}"
displayName: "Deploy to ${{ parameters.env }}"
variables:
- name: releasesFile
value: ""
jobs:
- deployment:
environment: "${{ parameters.env }}"
displayName: "Deploy to ${{ parameters.env }}"
strategy:
runOnce:
deploy:
steps:
- task: Bash#3
name: FindReleaseFile
displayName: "Find the releases.txt file"
inputs:
targetType: 'inline'
script: |
echo "##vso[task.setvariable variable=releasesFile;isOutput=true]$(find $(Build.SourcesDirectory) -name releases.txt -maxdepth 1 -type f 2>/dev/null)"
- task: Bash#3
name: DownloadUsingArtifactTool
displayName: "Download Using Artifact Tool"
inputs:
targetType: 'inline'
script: |
echo "$(FindReleaseFile.releasesFile)"
The pipeline looks like this:
trigger:
branches:
include:
- azure-pipelines
paths:
include:
- "releases.txt"
stages:
- template: deploy-to-env.yml
parameters:
env: "tst"
envLong: "Test"
How can I pass the releasesFile variable to the 2nd task in deploy-to-env.yml template?
The problem was the the repository was not cloned, adding checkout: self in the steps fixed it

trying to use set variable from predefined variables and use it in condition for stage in azure devops pipeline

I am trying to setup variable from predefined variable of pipeline resources and use it in condition for stage, but i could not succeeded. below is the my pipeline
resources:
pipelines:
- pipeline: pipeline1
project: appcom
source: pipeline-api
trigger:
branches:
- develop
- feat/*
- pipeline: pipeline2
project: appcom
source: pipeline2-api
trigger:
branches:
- develop
- feat/*
variables:
- name: alias
value: $(resources.triggeringAlias)
stages:
- stage: DEV
displayName: Deploying to DEV
jobs:
- deployment: Deployment
displayName: Deploying to Dev
environment: dev
pool:
name: 'CDaaSLinux'
strategy:
runOnce:
deploy:
steps:
- template: /variables/variable.yaml
- script: echo $(alias)
- task: Bash#3
inputs:
targetType: 'inline'
script: |
if [ "$(alias)" == "pipeline1" ]; then
echo "##vso[task.setvariable variable=apiname]$(resources.pipeline.pipeline1.pipelineName)"
echo "##vso[task.setvariable variable=branchName]$(resources.pipeline.pipeline1.sourceBranch)"
echo "##vso[task.setvariable variable=dockertag]$(resources.pipeline.pipeline1.sourceCommit) | cut -c -7"
echo "##vso[task.setvariable variable=helmpath]P02565Mallorca/pipeline1-api"
elif [ "$(alias)" = "pipeline2" ]; then
echo "##vso[task.setvariable variable=apiname]$(resources.pipeline.pipeline2.pipelineName)"
echo "##vso[task.setvariable variable=branchName]$(resources.pipeline.pipeline2.sourceBranch)"
echo "##vso[task.setvariable variable=dockertag]$(resources.pipeline.pipeline2.sourceCommit) | cut -c -7"
echo "##vso[task.setvariable variable=helmpath]P02565Mallorca/pipeline2-api"
fi
- script: echo $(dockertag)
- script: echo $(helmpath)
- script: echo $(apiname)
- script: echo $(branchName)
but I would like to execute above task on top of all stage and set them as global variables and use those variable in condition on following stages.
needed condition would be as below
stages:
- stage: DockerBuildtask
condition: and(succeeded(), or(eq(variables['$(branchName'], 'refs/heads/develop'), eq(variables['$(branchName'], 'refs/heads/release'), eq(variables['$(branchName'], 'refs/heads/feat*.'), eq(variables['$(branchName'], 'refs/heads/bugfix*.')))
below is the variable.yaml content
steps:
- task: Bash#3
inputs:
targetType: 'inline'
script: |
if [ "$(alias)" == "pipeline1" ]; then
echo "##vso[task.setvariable variable=apibuild]$(resources.pipeline.pipeline1.pipelineName)"
echo "##vso[task.setvariable variable=branchName;isOutput=true]$(resources.pipeline.pipeline1.sourceBranch)"
echo "##vso[task.setvariable variable=dockertag]$(echo '$(resources.pipeline.pipeline1.sourceCommit)' | cut -c -7)"
echo "##vso[task.setvariable variable=apiname]pipeline1-api"
could someone help me on this issue. thanks in advance
trying to use set variable from predefined variables and use it in condition for stage in azure devops pipeline
You could use Output variables from previous stages:
Dependencies.stageName.outputs['jobName.stepName.variableName']
Please check the document Jobs can access output variables from previous stages for some more details.
And we need also add a property ;isOutput=true and name to the previous stages:
echo "##vso[task.setvariable variable=branchName;isOutput=true]$(resources.pipeline.pipeline1.sourceBranch)"
- task: PowerShell#2
displayName: 'Inline Powershell'
inputs:
TargetType: inline
Script: |
if ("$(alias)" -eq "PipelineA")
{
...
}
pwsh: true
name: powershelltest
Then we could use the Dependencies.stageName.outputs['jobName.stepName.variableName'] as condition in next stage:
- stage: DockerBuildtask
and(succeeded(), or(eq(dependencies.ScanImage.outputs['ScanImage.powershelltest.branchName'], 'refs/heads/develop'), eq(dependencies.ScanImage.outputs['ScanImage.powershelltest.branchName'], 'refs/heads/release'), eq(dependencies.ScanImage.outputs['ScanImage.powershelltest.branchName'], 'refs/heads/feat*.'), eq(dependencies.ScanImage.outputs['ScanImage.powershelltest.branchName'], 'refs/heads/bugfix*.')))
Note: Since you need add the condition on top of all stage, we need update the stages refer from stageDependencies.stageName.jobName.outputs['stepName.variableName'] To Dependencies.stageName.outputs['jobName.stepName.variableName']
Update2:
here i am setting variable(variable.yaml) under deployment not job. so
how would i frame below one
Dependencies.stageNam.outputs['jobName.stepName.variableName'] should
i meniton Deployment as jobName ??
The answer is yes.
You could use the Dependencies.stageName.outputs['jobName.jobName.stepName.variableName'] as condition.
Note: There are two jobName in the conditions.

Azure pipeline expression does not work in template

I have some templates and i'd like to check if variables are set or not.
So i tried this:
Template that gets included:
- ${{if not(variables.assemblyVersion)}}:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
Write-Host $(assemblyVersion)
throw "assemblyVersion was not set"
But even if set assemblyVersion to something (e.g. 1.2.3.4) that task is being ran.
What am i doing wrong?
Edit: tried the answer from Krzysztof Madej, and i got false-positives:
Output:
2020-11-19T09:36:30.7383677Z ##[section]Starting: PowerShell
2020-11-19T09:36:30.7500976Z ==============================================================================
2020-11-19T09:36:30.7501272Z Task : PowerShell
2020-11-19T09:36:30.7501523Z Description : Run a PowerShell script on Linux, macOS, or Windows
2020-11-19T09:36:30.7501778Z Version : 2.170.1
2020-11-19T09:36:30.7501984Z Author : Microsoft Corporation
2020-11-19T09:36:30.7502296Z Help : https://learn.microsoft.com/azure/devops/pipelines/tasks/utility/powershell
2020-11-19T09:36:30.7502761Z ==============================================================================
2020-11-19T09:36:31.9944831Z Generating script.
2020-11-19T09:36:32.1228113Z ========================== Starting Command Output ===========================
2020-11-19T09:36:32.1504771Z ##[command]"C:\windows\System32\WindowsPowerShell\v1.0\powershell.exe" -NoLogo -NoProfile -NonInteractive -ExecutionPolicy Unrestricted -Command ". 'D:\a\_temp\2af656c2-be5b-4a73-8812-17185fff83cc.ps1'"
2020-11-19T09:36:32.4080866Z 1.2.3.4
2020-11-19T09:36:32.7066175Z assemblyVersion was not set
2020-11-19T09:36:32.7066649Z At D:\a\_temp\2af656c2-be5b-4a73-8812-17185fff83cc.ps1:4 char:1
2020-11-19T09:36:32.7067329Z + throw "assemblyVersion was not set"
2020-11-19T09:36:32.7068304Z + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2020-11-19T09:36:32.7069103Z + CategoryInfo : OperationStopped: (assemblyVersion was not set:String) [], RuntimeException
2020-11-19T09:36:32.7069840Z + FullyQualifiedErrorId : assemblyVersion was not set
2020-11-19T09:36:32.7070536Z
2020-11-19T09:36:32.8288504Z ##[error]PowerShell exited with code '1'.
2020-11-19T09:36:32.8929792Z ##[section]Finishing: PowerShell
Main pipeline where yaml is set:
trigger:
- master
- feature/*
pool:
vmImage: 'windows-latest'
variables:
- group: HmiSulis
- name: solution
value: AllProjects.sln
- name: buildPlatform
value: x86
- name: buildConfiguration
value: Debug
- name: assemblyVersion
value: 1.2.3.4
- name: fileVersion
value: 5.6.7.8
- name: informationalVersion
value: 9.10.11.12
resources:
repositories:
- repository: BuildTemplates
type: git
name: HMI/BuildTemplates
extends:
template: netFx/Jobs/netFx.Build.yml#BuildTemplates
The job used:
jobs:
- job: Build
steps:
- template: ../../NuGet/Steps/NuGet.Restore.yml
- template: ../Steps/netFx.Build.Version.yml
- template: ../Steps/netFx.Build.yml
The steps with the checks (netFx.Build.Version.yml):
steps:
- ${{if not(variables['assemblyVersion'])}}:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
Write-Host $(assemblyVersion)
throw "assemblyVersion was not set"
- ${{if not(variables['fileVersion'])}}:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
throw "fileVersion was not set"
- ${{if not(variables['informationalVersion'])}}:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
throw "informationalVersion was not set"
- task: Assembly-Info-NetFramework#2
inputs:
Path: '$(Build.SourcesDirectory)'
FileNames: |
**\AssemblyInfo.cs
**\AssemblyInfo.vb
InsertAttributes: true
FileEncoding: 'auto'
WriteBOM: false
VersionNumber: '$(assemblyVersion)'
# File version in windows explorer
FileVersionNumber: '$(fileVersion)'
# Product version in windows explorer
InformationalVersion: '$(informationalVersion)'
LogLevel: 'verbose'
FailOnWarning: false
DisableTelemetry: false
Please use correct syntax as it is shown here:
variables:
${{ if eq(variables['Build.SourceBranchName'], 'master') }}: # only works if you have a master branch
stageName: prod
pool:
vmImage: 'ubuntu-latest'
steps:
- script: echo ${{variables.stageName}}
so in your case it would be
- ${{if not(variables['assemblyVersion'])}}:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
Write-Host $(assemblyVersion)
throw "assemblyVersion was not set"
You can use an if clause to conditionally assign the value or a variable or set inputs for tasks. Conditionals only work when using template syntax.
I simplified your case and it works:
variables:
- name: solution
value: AllProjects.sln
- name: buildPlatform
value: x86
- name: buildConfiguration
value: Debug
- name: assemblyVersion
value: ''
# value: 1.2.3.4
- name: fileVersion
value: 5.6.7.8
- name: informationalVersion
value: 9.10.11.12
pool:
vmImage: 'ubuntu-latest'
jobs:
- job: Build
variables:
- name: test
value: $[not(variables['assemblyVersion'])]
steps:
- ${{if not(variables['assemblyVersion'])}}:
- task: PowerShell#2
continueOnError: true
inputs:
targetType: 'inline'
script: |
Write-Host $(assemblyVersion)
throw "assemblyVersion was not set"
- task: PowerShell#2
continueOnError: true
inputs:
targetType: 'inline'
script: |
Write-Host $(assemblyVersion)
Write-Host $(test)
So when assemblyVersion is given task is skipped. And when is not given task is ran and fail. Be aware $(assemblyVersion) it means that variable have to be available even if not set.
I made further tests moving logic to template. So this is my template:
jobs:
- job: BuildFromTemplate
dependsOn: []
variables:
- name: test
value: $[not(variables['assemblyVersion'])]
steps:
- ${{if not(variables['assemblyVersion'])}}:
- task: PowerShell#2
continueOnError: true
inputs:
targetType: 'inline'
script: |
Write-Host $(assemblyVersion)
throw "assemblyVersion was not set"
- task: PowerShell#2
continueOnError: true
inputs:
targetType: 'inline'
script: |
Write-Host $(assemblyVersion)
Write-Host $(test)
and here is the pipeline:
variables:
- name: assemblyVersion
# value: ''
value: 1.2.3.4
pool:
vmImage: 'ubuntu-latest'
stages:
- stage: A
jobs:
- job: Build
variables:
- name: test
value: $[not(variables['assemblyVersion'])]
steps:
- ${{if not(variables['assemblyVersion'])}}:
- task: PowerShell#2
continueOnError: true
inputs:
targetType: 'inline'
script: |
Write-Host $(assemblyVersion)
throw "assemblyVersion was not set"
- task: PowerShell#2
continueOnError: true
inputs:
targetType: 'inline'
script: |
Write-Host $(assemblyVersion)
Write-Host $(test)
- template: template.yaml
and yes job in template failed:
What seems strange, because:
The difference between runtime and compile time expression syntaxes is primarily what context is available. In a compile-time expression (${{ }}), you have access to parameters and statically defined variables. In a runtime expression ($[ ]), you have access to more variables but no parameters.
and assemblyVersion is statically defined variable.
It looks that we can use only paramaters in template expression in templates and not variables.
I created a bug on developer community for this.
IMHO if you want to use template you need to move this condition into task and just skip logic.
- task: PowerShell#2
continueOnError: true
inputs:
targetType: 'inline'
script: |
$ver = '$(assemblyVersion)'
Write-Host "Ver: $($ver)"
if (!$ver)
{
throw "assemblyVersion was not set"
}
Or use templates variable and reuse it in main file and template file:
parameters:
- name: 'variabaleTemplate'
default: 'variables.yaml'
type: string
jobs:
- job: BuildFromTemplate
dependsOn: []
variables:
- template: ${{parameters.variabaleTemplate}}
- name: test
value: $[not(variables['assemblyVersion'])]
steps:
- ${{if not(variables['assemblyVersion'])}}:
- task: PowerShell#2
continueOnError: true
inputs:
targetType: 'inline'
script: |
Write-Host $(assemblyVersion)
throw "assemblyVersion was not set"
with variables.yaml like
variables:
- name: assemblyVersion
# value: ''
value: 1.2.3.4
and main file like:
stages:
- stage: A
variables:
- template: variables.yaml
jobs:
- template: template2.yaml
So I got reply from MS
For security reasons, we only allow you to pass information into templated code via explicit parameters.
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/templates?view=azure-devops
The means the author of the pipeline using your template needs to commit changes where they explicitly pass the needed info into your template code.
There are some exceptions to this, where the variable is statically defined in the same file or at pipeline compile time, but generally speaking, it’s probably better to use parameters for everything that does not involve system-defined read-only dynamic variable and custom-defined dynamic output variables.
So solution with separate variable template yaml is the best what is possible I think. Otherwise you need to pass all values via template parameters.

Is it possible to use aliases for pipeline parameters in Azure DevOps?

Let's say I have the following parameters in my pipeline:
parameters:
- name: environment
displayName: 'Environment'
default: 'SERVER_0562'
type: string
values:
- 'SERVER_0562'
- 'SERVER_5149'
- 'SERVER_1892'
But I don't want whoever is running the pipeline to see these names when selecting the Environment from the dropdown list. I want then to see the following names:
- QA Server
- DEV Server
- PROD Server
Is there any "easy" way to do so or the only way would be to deal with conditionals?
${{ if eq(parameters.environment, 'DEV Server') }}:
Please check this:
parameters:
- name: image
displayName: Pool Image
type: string
default: Ubuntu
values:
- Windows
- Ubuntu
- MacOS
trigger: none
jobs:
- job: build
displayName: build
pool:
${{ if eq(parameters.image, 'Ubuntu') }}:
vmImage: 'ubuntu-latest'
${{ if eq(parameters.image, 'Windows') }}:
vmImage: 'windows-latest'
${{ if eq(parameters.image, 'MacOS') }}:
vmImage: 'macOS-latest'
steps:
- script: echo building $(Build.BuildNumber) with ${{ parameters.image }}
After you changed a question I think this may help you
parameters:
- name: environment
displayName: 'Environment'
default: 'QA Server'
type: string
values:
- 'QA Server'
- 'DEV Server'
- 'PROD Server'
trigger: none
variables:
- name: QA
value: 'SERVER_0562'
- name: DEV
value: 'SERVER_5149'
- name: PROD
value: 'SERVER_1892'
jobs:
- job: build
displayName: build
pool:
vmImage: 'windows-latest'
steps:
- script: echo building $(Build.BuildNumber) with ${{ parameters.environment }}
- powershell: |
If('${{ parameters.environment }}' -eq 'QA Server') {
Write-Host '##vso[task.setvariable variable=folderName;isOutput=true]$(QA)'
} ElseIf('${{ parameters.environment }}' -eq 'DEV Server') {
Write-Host '##vso[task.setvariable variable=folderName;isOutput=true]$(DEV)'
} ElseIf('${{ parameters.environment }}' -eq 'PROD Server') {
Write-Host '##vso[task.setvariable variable=folderName;isOutput=true]$(PROD)'
} Else {
Write-Host 'I am here'
}
name: setFolderName
- powershell: |
Write-Host '$(setFolderName.folderName)'
Inspired by the contribution of #Krzysztof Madej, I ended up solving my own issue by conditionally assigning variables:
parameters:
- name: environment
displayName: 'Environment'
default: 'QA Server'
type: string
values:
- 'QA Server'
- 'DEV Server'
- 'PROD Server'
trigger: none
variables:
${{ if eq(parameters.environment, 'QA Server') }}:
bacon: SERVER_0562
${{ if eq(parameters.environment, 'DEV Server') }}:
bacon: SERVER_5149
${{ if eq(parameters.environment, 'PROD Server') }}:
bacon: SERVER_1892
jobs:
- job: build
displayName: build
steps:
- script: echo building $(Build.BuildNumber) with ${{ variables.bacon }}
I believe this is an easier implementation since I'm not dependent on a PowerShell script.