My azure-pipeline.yml sets a special magical variable called name that causes the pipeline name to be set.
I want to use this value inside a PowerShell script to put this value into a text file as a setting.
variables:
System.Debug: true
versionNumber: '0.0.0'
namePrefix: 'yml_$(versionNumber).'
buildConfiguration: 'Release'
tag: '-yml-$(versionNumber)'
publishBuildArtifactsSubfolder: 'yml\$(Build.Repository.Name)_yml_$(versionNumber).'
name: ${{ variables['namePrefix']}}$(Build.BuildId)
steps:
# Find and replace in files. PowerShell because bash isn't available.
- task: PowerShell#2
displayName: Set version number in environment.*.ts
inputs:
targetType: 'inline'
script: |
Write-Host "Build.DefinitionName is $(Build.DefinitionName)"
Write-Host "env-name is $($env:name)"
Write-Host "name is $($name)"
#Get-ChildItem 'environment.*.ts' -Recurse | ForEach {
# (Get-Content $_ | ForEach { $_ -replace '[[name]]', '${Build.DefinitionName}' }) | Set-Content $_
#}
Is it possible to get the value of name inside the PowerShell script? How? (Is this documented anywhere?)
Workaround
It appears to be impossible to get the special name thing, therefore set a variable with the same value.
variables:
System.Debug: true
versionNumber: '0.0.0'
namePrefix: 'yml_$(versionNumber).'
buildConfiguration: 'Release'
tag: '-yml-$(versionNumber)'
publishBuildArtifactsSubfolder: 'yml\$(Build.Repository.Name)_yml_$(versionNumber).'
name: ${{ variables['namePrefix']}}$(Build.BuildId)
steps:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
Write-Host "name is $($name)"
Yes, the name it's a pre-defined variable Build.BuildNumber, so just use $(Build.BuildNumber).
Related
I have a parameter that consists of two executable paths and I want to loop through it to sign each executable. For this, I use a PowerShell#2 task that is passed via a template in Azure DevOps.
I don't seem to loop correctly through the executable array via any of the methods I've tried in my template code.
What is the best way to loop through the array parameter? Thanks for your help.
Here is my latest code:
parameters:
- name: executableList
type: object
default: []
steps:
- task: PowerShell#2
displayName: 'Sign executables'
inputs:
targetType: 'inline'
script: |
#...more code here
${{ foreach (executable in parameters.executableList) }} {
& $signingExecutable sign /sm /s My /n "Sign Text" /tr http://rfc3161timestamp.globalsign.com/advanced /td sha256 /fd sha256 "$(Pipeline.Workspace)$executable"
}
Error: While scanning a simple key, could not find expected ':'.
You can try convertToJson function. Small example:
parameters:
- name: myArray
type: object
default:
- "1"
- "2"
- "4"
- "8"
- "16"
steps:
- task: PowerShell#2
displayName: 'Sign executables'
inputs:
targetType: 'inline'
script: |
$mayarrstr= #"
${{ convertToJson(parameters.myArray) }}
"#
$myarr = ConvertFrom-Json $mayarrstr
foreach ($myvalue in $myarr)
{
Write-Host "Value: " $myvalue
}
I'm a bit of a beginner with YAMl and PS, so bear with me a bit.
The scenario:
I have SQL in Source Control using Redgate SoC and a build pipeline that contains the Redgate Build Extension, I have a task that checks whether there were any changes to Functions, SP's, Tables etc and a condition in the Redgate Build task that will skip the task if there were no changes to the above folders, when there are only scripts(data changes) that were pushed, and all of this works great. My next challenge (deployment pipeline) was to check if there was a Redgate Build performed to skip the deployment of the Redgate Deployment task.
My thinking here was, create a variable that I can save in the build pipeline and then use in the deployment pipeline (using the Variable Tools extension), under run this task (in the Redgate Deploy Pipeline section, set it to Custom Conditions and then check if the variable is true or not.
Following is my YAML Build pipeline:
trigger:
- dev
variables:
FunctionsFlag: $false
ProceduresFlag: $false
TablesFlag: $false
DataFlag: $false
ViewsFlag: $false
RGDeployFlag: $false
jobs:
- job: AdventureWorks
timeoutInMinutes: 0
pool: Enterprise
cancelTimeoutInMinutes: 1
steps:
- checkout: self
clean: true
- powershell: |
## get the changed files
$files=$(git diff HEAD HEAD~ --name-only)
$temp=$files -split ' '
$count=$temp.Length
echo "Total changed $count files"
For ($i=0; $i -lt $temp.Length; $i++)
{
$name=$temp[$i]
echo "this is $name file"
if ($name -like 'Functions/*')
{
##set the flag variable FunctionsFlagFlag to true
Write-Host "##vso[task.setvariable variable=FunctionsFlag]$True"
Write-Host "##vso[task.setvariable variable=RGDeployFlag]$True"
##Set-Variable -Name $(RGDeployFlag) -Value $True
echo "The answer is $(RGDeployFlag)"
}
if ($name -like 'Procedures/*')
{
##set the flag variable StoredProceduresFlagFlag to true
Write-Host "##vso[task.setvariable variable=ProceduresFlag]$True"
}
if ($name -like 'Tables/*')
{
##set the flag variable TablesFlagFlag to true
Write-Host "##vso[task.setvariable variable=TablesFlag]$True"
}
if ($name -like 'Data/*')
{
##set the flag variable StoredProceduresFlagFlag to true
Write-Host "##vso[task.setvariable variable=DataFlag]$True"
}
if ($name -like 'Views/*')
{
##set the flag variable TablesFlagFlag to true
Write-Host "##vso[task.setvariable variable=ViewsFlag]$True"
}
if (($name -like 'Functions/*') -or ($name -like 'Procedures/*') -or ($name -like 'Tables/*') -or ($name -like 'Data/*') -or($name -like 'Views/*'))
{
##set the flag variable RGDeployFlag to true
Write-Host "##vso[task.setvariable variable=RGDeployFlag;isoutput=true]$True"
$RGDeployFlag=$True
echo "The answer is $(RGDeployFlag)"
}
echo "The answer is $(RGDeployFlag)"
}
- task: RedgateSqlChangeAutomationBuild#4
inputs:
operation: Build
dbFolder: RootFolder
packageName: AdventureWorks
tempServerTypeBuild: localDB
buildAdvanced: true
compareOptionsBuild: 'NoTransactions, IgnoreFileGroups'
dataCompareOptionsBuild: 'DisableAndReenableDMLTriggers, SkipFkChecks'
transactionIsolationLevelBuild: readCommitted
queryBatchTimeoutBuild: '0'
excludeEnviromentVariables: 'AGENT_*;BUILD_*;SYSTEM_*'
nugetPackageVersionSelector: Specific
nugetPackageVersion: '1.0'
nugetPackageVersionUseBuildId: true
requiredVersionOfDlma: latestInstalled
condition: and(succeeded(), or(eq(variables.FunctionsFlag, true),eq(variables.ProceduresFlag, true),eq(variables.TablesFlag, true),eq(variables.DataFlag, true),eq(variables.ViewsFlag, true)))
- task: CopyFiles#2
inputs:
SourceFolder: '$(System.DefaultWorkingDirectory)/Scripts'
Contents: '**'
TargetFolder: '$(System.ArtifactsDirectory)/Scripts'
CleanTargetFolder: true
OverWrite: true
preserveTimestamp: true
- task: variabledehydration#0
inputs:
prefixes: 'GITVERSION,BUILD,RGDeployFlag'
outpath: '$(Build.ArtifactStagingDirectory)'
- task: PublishBuildArtifacts#1
displayName: 'Publish Artifact: drop'
inputs:
PathtoPublish: $(System.ArtifactsDirectory)
Release Pipeline
Custom Conditions
So my main problem is setting the variable to be used in the deployment pipeline. Here is where I require some guidance:
Am I supposed to create multiple stages and or jobs in order to do this, or is my YAML structure correct, and that I'm just setting the values incorrectly?
Inside the if statements of my PS section, I have been trying a few tests to manipulate the outcome, but so far with little success.
Regardless of how I set the variable, it keeps evaluating to false.
I look forward to your comments.
Regards
Wic
So I figured out why this did not work:
Firstly, I had to remove the variable 'declaration' from the YAML as I was setting it to be cast in stone:
Added an initialization for my variable.
And lastly added the set variable.
Voila, I am able to use that variable in my release pipeline.
The complete YAML pipeline now looks like:
trigger:
- dev
## Removed the variables section here
jobs:
- job: AdventureWorks
timeoutInMinutes: 0
pool: Enterprise
cancelTimeoutInMinutes: 1
steps:
- checkout: self
clean: true
- powershell: |
## Initialize my variable here
$RGDeployFlag = $false
## get the changed files
$files=$(git diff HEAD HEAD~ --name-only)
$temp=$files -split ' '
$count=$temp.Length
echo "Total changed $count files"
For ($i=0; $i -lt $temp.Length; $i++)
{
$name=$temp[$i]
echo "this is $name file"
if ($name -like 'Functions/*')
{
##set the flag variable FunctionsFlagFlag to true
Write-Host "##vso[task.setvariable variable=FunctionsFlag]$True"
}
if ($name -like 'Procedures/*')
{
##set the flag variable StoredProceduresFlagFlag to true
Write-Host "##vso[task.setvariable variable=ProceduresFlag]$True"
}
if ($name -like 'Tables/*')
{
##set the flag variable TablesFlagFlag to true
Write-Host "##vso[task.setvariable variable=TablesFlag]$True"
}
if ($name -like 'Data/*')
{
##set the flag variable StoredProceduresFlagFlag to true
Write-Host "##vso[task.setvariable variable=DataFlag]$True"
}
if ($name -like 'Views/*')
{
##set the flag variable TablesFlagFlag to true
Write-Host "##vso[task.setvariable variable=ViewsFlag]$True"
}
if (($name -like 'Functions/*') -or ($name -like 'Procedures/*') -or ($name -like 'Tables/*') -or ($name -like 'Data/*') -or($name -like 'Views/*'))
{
## set the flag variable RGDeployFlag to true
$RGDeployFlag = $true
}
## Test to see what the variable value is
"RGDeployFlag : $RGDeployFlag"
}
- task: RedgateSqlChangeAutomationBuild#4
inputs:
operation: Build
dbFolder: RootFolder
packageName: AdventureWorks
tempServerTypeBuild: localDB
buildAdvanced: true
compareOptionsBuild: 'NoTransactions, IgnoreFileGroups'
dataCompareOptionsBuild: 'DisableAndReenableDMLTriggers, SkipFkChecks'
transactionIsolationLevelBuild: readCommitted
queryBatchTimeoutBuild: '0'
excludeEnviromentVariables: 'AGENT_*;BUILD_*;SYSTEM_*'
nugetPackageVersionSelector: Specific
nugetPackageVersion: '1.0'
nugetPackageVersionUseBuildId: true
requiredVersionOfDlma: latestInstalled
condition: and(succeeded(), or(eq(variables.FunctionsFlag, true),eq(variables.ProceduresFlag, true),eq(variables.TablesFlag, true),eq(variables.DataFlag, true),eq(variables.ViewsFlag, true)))
- task: CopyFiles#2
inputs:
SourceFolder: '$(System.DefaultWorkingDirectory)/Scripts'
Contents: '**'
TargetFolder: '$(System.ArtifactsDirectory)/Scripts'
CleanTargetFolder: true
OverWrite: true
preserveTimestamp: true
- task: variabledehydration#0
inputs:
prefixes: 'GITVERSION,BUILD,RGDeployFlag'
outpath: '$(Build.ArtifactStagingDirectory)'
- task: PublishBuildArtifacts#1
displayName: 'Publish Artifact: drop'
inputs:
PathtoPublish: $(System.ArtifactsDirectory)
I am attempting to use a condition to break out of a loop based on the result of a sql query. So I am assigning the query output to a variable "sqloutput" and if that value is empty string go around again. I have confirmed that the query is outputting a value however on subsequent loops the variable is still evaluating to NULL.
Also tried to explicitly declare the variable to no avail.
parameters:
- name: loop
type: object
default : ["1","2","3","4","5","6","7","8","9","10"]
stages:
- stage: WaitforProcessing
displayName: Wait For Processing
jobs:
- deployment: Deployment
pool:
vmImage: ubuntu-latest
environment:
name: 'Client_Data_Tests'
resourceType: VirtualMachine
strategy:
runOnce:
deploy:
steps:
- ${{each Looptimes in parameters.loop}}:
- task: PowerShell#2
name: checkImportProgress${{ Looptimes }}
condition: and(succeeded(), eq(variables['sqloutput'], ''))
displayName: Check if job finished
inputs:
targetType: inline
script: |
$query = "IF EXISTS (select 1 from Job where JobFinishTime is null)
BEGIN
select '' as result
END
ELSE
select '1' as result
"
$sqloutput = Invoke-Sqlcmd -query $query -ServerInstance "$(DB_SERVER)" -Database "$(DB_DATABASE)" -Username "$(DB_USERNAME)" -Password "$(DB_PASSWORD)" -QueryTimeout 36000 -Verbose
$value = $sqloutput["result"]
Write-Host $value
Write-Host "##vso[task.setvariable variable=sqloutput]$value"
- task: PowerShell#2
condition: and(succeeded(), eq(variables['sqloutput'], ''))
displayName: 'Sleep for 30 seconds'
inputs:
targetType: inline
script: |
Start-Sleep 30
The below YAML should work.
stages:
- stage: A
jobs:
- job: A
displayName: SQL generate variable
steps:
- powershell: |
$value = 'somevalue'
echo "##vso[task.setvariable variable=sqloutput;isOutput=true]$value"
name: setvarStep
- script: echo $(setvarStep.sqloutput)
name: echovar
- job: B
dependsOn: A
displayName: Use variable that SQL script output as condition
variables:
myVarFromJobA: $[ dependencies.A.outputs['setvarStep.sqloutput'] ] # map in the variable
myvar: true
condition: eq(variables['myVarFromJobA'],'somevalue')
steps:
- script: echo $(myVarFromJobA)
name: echovar
Result:
I have some templates and i'd like to check if variables are set or not.
So i tried this:
Template that gets included:
- ${{if not(variables.assemblyVersion)}}:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
Write-Host $(assemblyVersion)
throw "assemblyVersion was not set"
But even if set assemblyVersion to something (e.g. 1.2.3.4) that task is being ran.
What am i doing wrong?
Edit: tried the answer from Krzysztof Madej, and i got false-positives:
Output:
2020-11-19T09:36:30.7383677Z ##[section]Starting: PowerShell
2020-11-19T09:36:30.7500976Z ==============================================================================
2020-11-19T09:36:30.7501272Z Task : PowerShell
2020-11-19T09:36:30.7501523Z Description : Run a PowerShell script on Linux, macOS, or Windows
2020-11-19T09:36:30.7501778Z Version : 2.170.1
2020-11-19T09:36:30.7501984Z Author : Microsoft Corporation
2020-11-19T09:36:30.7502296Z Help : https://learn.microsoft.com/azure/devops/pipelines/tasks/utility/powershell
2020-11-19T09:36:30.7502761Z ==============================================================================
2020-11-19T09:36:31.9944831Z Generating script.
2020-11-19T09:36:32.1228113Z ========================== Starting Command Output ===========================
2020-11-19T09:36:32.1504771Z ##[command]"C:\windows\System32\WindowsPowerShell\v1.0\powershell.exe" -NoLogo -NoProfile -NonInteractive -ExecutionPolicy Unrestricted -Command ". 'D:\a\_temp\2af656c2-be5b-4a73-8812-17185fff83cc.ps1'"
2020-11-19T09:36:32.4080866Z 1.2.3.4
2020-11-19T09:36:32.7066175Z assemblyVersion was not set
2020-11-19T09:36:32.7066649Z At D:\a\_temp\2af656c2-be5b-4a73-8812-17185fff83cc.ps1:4 char:1
2020-11-19T09:36:32.7067329Z + throw "assemblyVersion was not set"
2020-11-19T09:36:32.7068304Z + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2020-11-19T09:36:32.7069103Z + CategoryInfo : OperationStopped: (assemblyVersion was not set:String) [], RuntimeException
2020-11-19T09:36:32.7069840Z + FullyQualifiedErrorId : assemblyVersion was not set
2020-11-19T09:36:32.7070536Z
2020-11-19T09:36:32.8288504Z ##[error]PowerShell exited with code '1'.
2020-11-19T09:36:32.8929792Z ##[section]Finishing: PowerShell
Main pipeline where yaml is set:
trigger:
- master
- feature/*
pool:
vmImage: 'windows-latest'
variables:
- group: HmiSulis
- name: solution
value: AllProjects.sln
- name: buildPlatform
value: x86
- name: buildConfiguration
value: Debug
- name: assemblyVersion
value: 1.2.3.4
- name: fileVersion
value: 5.6.7.8
- name: informationalVersion
value: 9.10.11.12
resources:
repositories:
- repository: BuildTemplates
type: git
name: HMI/BuildTemplates
extends:
template: netFx/Jobs/netFx.Build.yml#BuildTemplates
The job used:
jobs:
- job: Build
steps:
- template: ../../NuGet/Steps/NuGet.Restore.yml
- template: ../Steps/netFx.Build.Version.yml
- template: ../Steps/netFx.Build.yml
The steps with the checks (netFx.Build.Version.yml):
steps:
- ${{if not(variables['assemblyVersion'])}}:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
Write-Host $(assemblyVersion)
throw "assemblyVersion was not set"
- ${{if not(variables['fileVersion'])}}:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
throw "fileVersion was not set"
- ${{if not(variables['informationalVersion'])}}:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
throw "informationalVersion was not set"
- task: Assembly-Info-NetFramework#2
inputs:
Path: '$(Build.SourcesDirectory)'
FileNames: |
**\AssemblyInfo.cs
**\AssemblyInfo.vb
InsertAttributes: true
FileEncoding: 'auto'
WriteBOM: false
VersionNumber: '$(assemblyVersion)'
# File version in windows explorer
FileVersionNumber: '$(fileVersion)'
# Product version in windows explorer
InformationalVersion: '$(informationalVersion)'
LogLevel: 'verbose'
FailOnWarning: false
DisableTelemetry: false
Please use correct syntax as it is shown here:
variables:
${{ if eq(variables['Build.SourceBranchName'], 'master') }}: # only works if you have a master branch
stageName: prod
pool:
vmImage: 'ubuntu-latest'
steps:
- script: echo ${{variables.stageName}}
so in your case it would be
- ${{if not(variables['assemblyVersion'])}}:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
Write-Host $(assemblyVersion)
throw "assemblyVersion was not set"
You can use an if clause to conditionally assign the value or a variable or set inputs for tasks. Conditionals only work when using template syntax.
I simplified your case and it works:
variables:
- name: solution
value: AllProjects.sln
- name: buildPlatform
value: x86
- name: buildConfiguration
value: Debug
- name: assemblyVersion
value: ''
# value: 1.2.3.4
- name: fileVersion
value: 5.6.7.8
- name: informationalVersion
value: 9.10.11.12
pool:
vmImage: 'ubuntu-latest'
jobs:
- job: Build
variables:
- name: test
value: $[not(variables['assemblyVersion'])]
steps:
- ${{if not(variables['assemblyVersion'])}}:
- task: PowerShell#2
continueOnError: true
inputs:
targetType: 'inline'
script: |
Write-Host $(assemblyVersion)
throw "assemblyVersion was not set"
- task: PowerShell#2
continueOnError: true
inputs:
targetType: 'inline'
script: |
Write-Host $(assemblyVersion)
Write-Host $(test)
So when assemblyVersion is given task is skipped. And when is not given task is ran and fail. Be aware $(assemblyVersion) it means that variable have to be available even if not set.
I made further tests moving logic to template. So this is my template:
jobs:
- job: BuildFromTemplate
dependsOn: []
variables:
- name: test
value: $[not(variables['assemblyVersion'])]
steps:
- ${{if not(variables['assemblyVersion'])}}:
- task: PowerShell#2
continueOnError: true
inputs:
targetType: 'inline'
script: |
Write-Host $(assemblyVersion)
throw "assemblyVersion was not set"
- task: PowerShell#2
continueOnError: true
inputs:
targetType: 'inline'
script: |
Write-Host $(assemblyVersion)
Write-Host $(test)
and here is the pipeline:
variables:
- name: assemblyVersion
# value: ''
value: 1.2.3.4
pool:
vmImage: 'ubuntu-latest'
stages:
- stage: A
jobs:
- job: Build
variables:
- name: test
value: $[not(variables['assemblyVersion'])]
steps:
- ${{if not(variables['assemblyVersion'])}}:
- task: PowerShell#2
continueOnError: true
inputs:
targetType: 'inline'
script: |
Write-Host $(assemblyVersion)
throw "assemblyVersion was not set"
- task: PowerShell#2
continueOnError: true
inputs:
targetType: 'inline'
script: |
Write-Host $(assemblyVersion)
Write-Host $(test)
- template: template.yaml
and yes job in template failed:
What seems strange, because:
The difference between runtime and compile time expression syntaxes is primarily what context is available. In a compile-time expression (${{ }}), you have access to parameters and statically defined variables. In a runtime expression ($[ ]), you have access to more variables but no parameters.
and assemblyVersion is statically defined variable.
It looks that we can use only paramaters in template expression in templates and not variables.
I created a bug on developer community for this.
IMHO if you want to use template you need to move this condition into task and just skip logic.
- task: PowerShell#2
continueOnError: true
inputs:
targetType: 'inline'
script: |
$ver = '$(assemblyVersion)'
Write-Host "Ver: $($ver)"
if (!$ver)
{
throw "assemblyVersion was not set"
}
Or use templates variable and reuse it in main file and template file:
parameters:
- name: 'variabaleTemplate'
default: 'variables.yaml'
type: string
jobs:
- job: BuildFromTemplate
dependsOn: []
variables:
- template: ${{parameters.variabaleTemplate}}
- name: test
value: $[not(variables['assemblyVersion'])]
steps:
- ${{if not(variables['assemblyVersion'])}}:
- task: PowerShell#2
continueOnError: true
inputs:
targetType: 'inline'
script: |
Write-Host $(assemblyVersion)
throw "assemblyVersion was not set"
with variables.yaml like
variables:
- name: assemblyVersion
# value: ''
value: 1.2.3.4
and main file like:
stages:
- stage: A
variables:
- template: variables.yaml
jobs:
- template: template2.yaml
So I got reply from MS
For security reasons, we only allow you to pass information into templated code via explicit parameters.
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/templates?view=azure-devops
The means the author of the pipeline using your template needs to commit changes where they explicitly pass the needed info into your template code.
There are some exceptions to this, where the variable is statically defined in the same file or at pipeline compile time, but generally speaking, it’s probably better to use parameters for everything that does not involve system-defined read-only dynamic variable and custom-defined dynamic output variables.
So solution with separate variable template yaml is the best what is possible I think. Otherwise you need to pass all values via template parameters.
I want to generate a list in a Powershell task as an output variable and use it in a yml template as a list to perform a loop for a task.
#Main
- task: PowerShell#2
condition: succeeded()
displayName: "Create a list"
inputs:
targetType: 'inline'
script: |
$myList = New-Object System.Collections.ArrayList
$myList.Add("wow")
Write-Output ("##vso[task.setvariable variable=MyList;]$myList")
- template: myRandomListTaskTemplate.yml
parameters:
MyList: $(MyList)
#Template
parameters:
MyList: []
steps:
- ${{ each myList in parameters.MyList }}:
- task: PowerShell#2
condition: succeeded()
displayName: "WOW a list"
inputs:
targetType: 'inline'
script: |
Write-Host("A list string: ${{ myList }}")
I am getting the error
Expected a sequence or mapping. Actual value '$(MyList)'
Note: The Powershell Task in the Template is just an example, it can be other Tasks that are not Powershell related (example: DotNetCoreCLI#2)
It will not work as you expect.
$myList = New-Object System.Collections.ArrayList
$myList.Add("wow")
Write-Output ("##vso[task.setvariable variable=MyList;]$myList")
You are not going here to have an array. Once you assign this to variable it will be a plain string. This is why you are getting this error.
If you run this:
- template: myRandomListTaskTemplate2.yml
parameters:
MyList:
- item1
- item2
all went fine. However, it is not possible to do what you at the moment.
Please check this developer community topic.