I have a parameter that consists of two executable paths and I want to loop through it to sign each executable. For this, I use a PowerShell#2 task that is passed via a template in Azure DevOps.
I don't seem to loop correctly through the executable array via any of the methods I've tried in my template code.
What is the best way to loop through the array parameter? Thanks for your help.
Here is my latest code:
parameters:
- name: executableList
type: object
default: []
steps:
- task: PowerShell#2
displayName: 'Sign executables'
inputs:
targetType: 'inline'
script: |
#...more code here
${{ foreach (executable in parameters.executableList) }} {
& $signingExecutable sign /sm /s My /n "Sign Text" /tr http://rfc3161timestamp.globalsign.com/advanced /td sha256 /fd sha256 "$(Pipeline.Workspace)$executable"
}
Error: While scanning a simple key, could not find expected ':'.
You can try convertToJson function. Small example:
parameters:
- name: myArray
type: object
default:
- "1"
- "2"
- "4"
- "8"
- "16"
steps:
- task: PowerShell#2
displayName: 'Sign executables'
inputs:
targetType: 'inline'
script: |
$mayarrstr= #"
${{ convertToJson(parameters.myArray) }}
"#
$myarr = ConvertFrom-Json $mayarrstr
foreach ($myvalue in $myarr)
{
Write-Host "Value: " $myvalue
}
Related
I am attempting to use a condition to break out of a loop based on the result of a sql query. So I am assigning the query output to a variable "sqloutput" and if that value is empty string go around again. I have confirmed that the query is outputting a value however on subsequent loops the variable is still evaluating to NULL.
Also tried to explicitly declare the variable to no avail.
parameters:
- name: loop
type: object
default : ["1","2","3","4","5","6","7","8","9","10"]
stages:
- stage: WaitforProcessing
displayName: Wait For Processing
jobs:
- deployment: Deployment
pool:
vmImage: ubuntu-latest
environment:
name: 'Client_Data_Tests'
resourceType: VirtualMachine
strategy:
runOnce:
deploy:
steps:
- ${{each Looptimes in parameters.loop}}:
- task: PowerShell#2
name: checkImportProgress${{ Looptimes }}
condition: and(succeeded(), eq(variables['sqloutput'], ''))
displayName: Check if job finished
inputs:
targetType: inline
script: |
$query = "IF EXISTS (select 1 from Job where JobFinishTime is null)
BEGIN
select '' as result
END
ELSE
select '1' as result
"
$sqloutput = Invoke-Sqlcmd -query $query -ServerInstance "$(DB_SERVER)" -Database "$(DB_DATABASE)" -Username "$(DB_USERNAME)" -Password "$(DB_PASSWORD)" -QueryTimeout 36000 -Verbose
$value = $sqloutput["result"]
Write-Host $value
Write-Host "##vso[task.setvariable variable=sqloutput]$value"
- task: PowerShell#2
condition: and(succeeded(), eq(variables['sqloutput'], ''))
displayName: 'Sleep for 30 seconds'
inputs:
targetType: inline
script: |
Start-Sleep 30
The below YAML should work.
stages:
- stage: A
jobs:
- job: A
displayName: SQL generate variable
steps:
- powershell: |
$value = 'somevalue'
echo "##vso[task.setvariable variable=sqloutput;isOutput=true]$value"
name: setvarStep
- script: echo $(setvarStep.sqloutput)
name: echovar
- job: B
dependsOn: A
displayName: Use variable that SQL script output as condition
variables:
myVarFromJobA: $[ dependencies.A.outputs['setvarStep.sqloutput'] ] # map in the variable
myvar: true
condition: eq(variables['myVarFromJobA'],'somevalue')
steps:
- script: echo $(myVarFromJobA)
name: echovar
Result:
My azure-pipeline.yml sets a special magical variable called name that causes the pipeline name to be set.
I want to use this value inside a PowerShell script to put this value into a text file as a setting.
variables:
System.Debug: true
versionNumber: '0.0.0'
namePrefix: 'yml_$(versionNumber).'
buildConfiguration: 'Release'
tag: '-yml-$(versionNumber)'
publishBuildArtifactsSubfolder: 'yml\$(Build.Repository.Name)_yml_$(versionNumber).'
name: ${{ variables['namePrefix']}}$(Build.BuildId)
steps:
# Find and replace in files. PowerShell because bash isn't available.
- task: PowerShell#2
displayName: Set version number in environment.*.ts
inputs:
targetType: 'inline'
script: |
Write-Host "Build.DefinitionName is $(Build.DefinitionName)"
Write-Host "env-name is $($env:name)"
Write-Host "name is $($name)"
#Get-ChildItem 'environment.*.ts' -Recurse | ForEach {
# (Get-Content $_ | ForEach { $_ -replace '[[name]]', '${Build.DefinitionName}' }) | Set-Content $_
#}
Is it possible to get the value of name inside the PowerShell script? How? (Is this documented anywhere?)
Workaround
It appears to be impossible to get the special name thing, therefore set a variable with the same value.
variables:
System.Debug: true
versionNumber: '0.0.0'
namePrefix: 'yml_$(versionNumber).'
buildConfiguration: 'Release'
tag: '-yml-$(versionNumber)'
publishBuildArtifactsSubfolder: 'yml\$(Build.Repository.Name)_yml_$(versionNumber).'
name: ${{ variables['namePrefix']}}$(Build.BuildId)
steps:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
Write-Host "name is $($name)"
Yes, the name it's a pre-defined variable Build.BuildNumber, so just use $(Build.BuildNumber).
I have a template that is called and passed a set of parameters. I would like to use these in a bash script. When I define an inline bash script I can easily print the parameters out with ${{ parameter.foo }}. However if I define a bash script as a file and call it I'm unable to access the parameters.
Here is a sample template, 'Inline Test' will spit out the parameters where as 'File Test' wont. So far two possible solutions I can see are, defining the parameter value as a 'task variable' and accessing it as $foo (example Inline Test), or potentially pass the parameter in as an argument to the 'File Test' task (example File Test 2). Is there an option to access yaml template parameters in an existing bash script from file and not one that is defined inline?
parameters:
- name: webappName #webappName
type: string
default: webappName #set default for testing
- name: resourceGroup #resourceGroup
type: string
default: resourceGroupName #set default for testing
- name: subscription #subscription
type: string
default: s #set default for testing
- name: preprod
type: boolean
default: false #default to false
- name: env
type: string
default: test
- name: role
type: string
default: s
steps:
- checkout: 'x' #the repo that test.sh lives in
- task: Bash#3
inputs:
displayName: 'Inline test'
targetType: 'inline'
script: |
echo "-----"
echo ${{ parameters.webappName }}
echo "-----"
echo ${{ parameters.resourceGroup }}
echo "-----"
echo ${{ parameters.subscription }}
echo "-----"
echo ${{ parameters.preprod }}
echo "-----"
echo ${{ parameters.env }}
echo "-----"
echo ${{ parameters.role }}
echo "-----"
echo "##vso[task.setvariable variable=TESTROLE]${{ parameters.role }}"
echo "-----"
- task: Bash#3
displayName: 'File test'
inputs:
filePath: './test.sh'
- task: Bash#3
displayName: 'File test 2'
inputs:
arguments: ${{ parameters.role }}
filePath: './test.sh'
AzDev just substitute values in the inline scripts, so the ${{ name }} is some kind of placeholder. And AzDev doesn't read script files, so doesn't substitute values there.
You can use environment variables for bash task:
- task: Bash#3
displayName: 'File test 2'
inputs:
filePath: './test.sh'
env:
APP_ROLE: ${{ parameters.role }}
And then you could use this env variable in your script file:
echo $APP_ROLE
I have some templates and i'd like to check if variables are set or not.
So i tried this:
Template that gets included:
- ${{if not(variables.assemblyVersion)}}:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
Write-Host $(assemblyVersion)
throw "assemblyVersion was not set"
But even if set assemblyVersion to something (e.g. 1.2.3.4) that task is being ran.
What am i doing wrong?
Edit: tried the answer from Krzysztof Madej, and i got false-positives:
Output:
2020-11-19T09:36:30.7383677Z ##[section]Starting: PowerShell
2020-11-19T09:36:30.7500976Z ==============================================================================
2020-11-19T09:36:30.7501272Z Task : PowerShell
2020-11-19T09:36:30.7501523Z Description : Run a PowerShell script on Linux, macOS, or Windows
2020-11-19T09:36:30.7501778Z Version : 2.170.1
2020-11-19T09:36:30.7501984Z Author : Microsoft Corporation
2020-11-19T09:36:30.7502296Z Help : https://learn.microsoft.com/azure/devops/pipelines/tasks/utility/powershell
2020-11-19T09:36:30.7502761Z ==============================================================================
2020-11-19T09:36:31.9944831Z Generating script.
2020-11-19T09:36:32.1228113Z ========================== Starting Command Output ===========================
2020-11-19T09:36:32.1504771Z ##[command]"C:\windows\System32\WindowsPowerShell\v1.0\powershell.exe" -NoLogo -NoProfile -NonInteractive -ExecutionPolicy Unrestricted -Command ". 'D:\a\_temp\2af656c2-be5b-4a73-8812-17185fff83cc.ps1'"
2020-11-19T09:36:32.4080866Z 1.2.3.4
2020-11-19T09:36:32.7066175Z assemblyVersion was not set
2020-11-19T09:36:32.7066649Z At D:\a\_temp\2af656c2-be5b-4a73-8812-17185fff83cc.ps1:4 char:1
2020-11-19T09:36:32.7067329Z + throw "assemblyVersion was not set"
2020-11-19T09:36:32.7068304Z + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2020-11-19T09:36:32.7069103Z + CategoryInfo : OperationStopped: (assemblyVersion was not set:String) [], RuntimeException
2020-11-19T09:36:32.7069840Z + FullyQualifiedErrorId : assemblyVersion was not set
2020-11-19T09:36:32.7070536Z
2020-11-19T09:36:32.8288504Z ##[error]PowerShell exited with code '1'.
2020-11-19T09:36:32.8929792Z ##[section]Finishing: PowerShell
Main pipeline where yaml is set:
trigger:
- master
- feature/*
pool:
vmImage: 'windows-latest'
variables:
- group: HmiSulis
- name: solution
value: AllProjects.sln
- name: buildPlatform
value: x86
- name: buildConfiguration
value: Debug
- name: assemblyVersion
value: 1.2.3.4
- name: fileVersion
value: 5.6.7.8
- name: informationalVersion
value: 9.10.11.12
resources:
repositories:
- repository: BuildTemplates
type: git
name: HMI/BuildTemplates
extends:
template: netFx/Jobs/netFx.Build.yml#BuildTemplates
The job used:
jobs:
- job: Build
steps:
- template: ../../NuGet/Steps/NuGet.Restore.yml
- template: ../Steps/netFx.Build.Version.yml
- template: ../Steps/netFx.Build.yml
The steps with the checks (netFx.Build.Version.yml):
steps:
- ${{if not(variables['assemblyVersion'])}}:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
Write-Host $(assemblyVersion)
throw "assemblyVersion was not set"
- ${{if not(variables['fileVersion'])}}:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
throw "fileVersion was not set"
- ${{if not(variables['informationalVersion'])}}:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
throw "informationalVersion was not set"
- task: Assembly-Info-NetFramework#2
inputs:
Path: '$(Build.SourcesDirectory)'
FileNames: |
**\AssemblyInfo.cs
**\AssemblyInfo.vb
InsertAttributes: true
FileEncoding: 'auto'
WriteBOM: false
VersionNumber: '$(assemblyVersion)'
# File version in windows explorer
FileVersionNumber: '$(fileVersion)'
# Product version in windows explorer
InformationalVersion: '$(informationalVersion)'
LogLevel: 'verbose'
FailOnWarning: false
DisableTelemetry: false
Please use correct syntax as it is shown here:
variables:
${{ if eq(variables['Build.SourceBranchName'], 'master') }}: # only works if you have a master branch
stageName: prod
pool:
vmImage: 'ubuntu-latest'
steps:
- script: echo ${{variables.stageName}}
so in your case it would be
- ${{if not(variables['assemblyVersion'])}}:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
Write-Host $(assemblyVersion)
throw "assemblyVersion was not set"
You can use an if clause to conditionally assign the value or a variable or set inputs for tasks. Conditionals only work when using template syntax.
I simplified your case and it works:
variables:
- name: solution
value: AllProjects.sln
- name: buildPlatform
value: x86
- name: buildConfiguration
value: Debug
- name: assemblyVersion
value: ''
# value: 1.2.3.4
- name: fileVersion
value: 5.6.7.8
- name: informationalVersion
value: 9.10.11.12
pool:
vmImage: 'ubuntu-latest'
jobs:
- job: Build
variables:
- name: test
value: $[not(variables['assemblyVersion'])]
steps:
- ${{if not(variables['assemblyVersion'])}}:
- task: PowerShell#2
continueOnError: true
inputs:
targetType: 'inline'
script: |
Write-Host $(assemblyVersion)
throw "assemblyVersion was not set"
- task: PowerShell#2
continueOnError: true
inputs:
targetType: 'inline'
script: |
Write-Host $(assemblyVersion)
Write-Host $(test)
So when assemblyVersion is given task is skipped. And when is not given task is ran and fail. Be aware $(assemblyVersion) it means that variable have to be available even if not set.
I made further tests moving logic to template. So this is my template:
jobs:
- job: BuildFromTemplate
dependsOn: []
variables:
- name: test
value: $[not(variables['assemblyVersion'])]
steps:
- ${{if not(variables['assemblyVersion'])}}:
- task: PowerShell#2
continueOnError: true
inputs:
targetType: 'inline'
script: |
Write-Host $(assemblyVersion)
throw "assemblyVersion was not set"
- task: PowerShell#2
continueOnError: true
inputs:
targetType: 'inline'
script: |
Write-Host $(assemblyVersion)
Write-Host $(test)
and here is the pipeline:
variables:
- name: assemblyVersion
# value: ''
value: 1.2.3.4
pool:
vmImage: 'ubuntu-latest'
stages:
- stage: A
jobs:
- job: Build
variables:
- name: test
value: $[not(variables['assemblyVersion'])]
steps:
- ${{if not(variables['assemblyVersion'])}}:
- task: PowerShell#2
continueOnError: true
inputs:
targetType: 'inline'
script: |
Write-Host $(assemblyVersion)
throw "assemblyVersion was not set"
- task: PowerShell#2
continueOnError: true
inputs:
targetType: 'inline'
script: |
Write-Host $(assemblyVersion)
Write-Host $(test)
- template: template.yaml
and yes job in template failed:
What seems strange, because:
The difference between runtime and compile time expression syntaxes is primarily what context is available. In a compile-time expression (${{ }}), you have access to parameters and statically defined variables. In a runtime expression ($[ ]), you have access to more variables but no parameters.
and assemblyVersion is statically defined variable.
It looks that we can use only paramaters in template expression in templates and not variables.
I created a bug on developer community for this.
IMHO if you want to use template you need to move this condition into task and just skip logic.
- task: PowerShell#2
continueOnError: true
inputs:
targetType: 'inline'
script: |
$ver = '$(assemblyVersion)'
Write-Host "Ver: $($ver)"
if (!$ver)
{
throw "assemblyVersion was not set"
}
Or use templates variable and reuse it in main file and template file:
parameters:
- name: 'variabaleTemplate'
default: 'variables.yaml'
type: string
jobs:
- job: BuildFromTemplate
dependsOn: []
variables:
- template: ${{parameters.variabaleTemplate}}
- name: test
value: $[not(variables['assemblyVersion'])]
steps:
- ${{if not(variables['assemblyVersion'])}}:
- task: PowerShell#2
continueOnError: true
inputs:
targetType: 'inline'
script: |
Write-Host $(assemblyVersion)
throw "assemblyVersion was not set"
with variables.yaml like
variables:
- name: assemblyVersion
# value: ''
value: 1.2.3.4
and main file like:
stages:
- stage: A
variables:
- template: variables.yaml
jobs:
- template: template2.yaml
So I got reply from MS
For security reasons, we only allow you to pass information into templated code via explicit parameters.
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/templates?view=azure-devops
The means the author of the pipeline using your template needs to commit changes where they explicitly pass the needed info into your template code.
There are some exceptions to this, where the variable is statically defined in the same file or at pipeline compile time, but generally speaking, it’s probably better to use parameters for everything that does not involve system-defined read-only dynamic variable and custom-defined dynamic output variables.
So solution with separate variable template yaml is the best what is possible I think. Otherwise you need to pass all values via template parameters.
I want to generate a list in a Powershell task as an output variable and use it in a yml template as a list to perform a loop for a task.
#Main
- task: PowerShell#2
condition: succeeded()
displayName: "Create a list"
inputs:
targetType: 'inline'
script: |
$myList = New-Object System.Collections.ArrayList
$myList.Add("wow")
Write-Output ("##vso[task.setvariable variable=MyList;]$myList")
- template: myRandomListTaskTemplate.yml
parameters:
MyList: $(MyList)
#Template
parameters:
MyList: []
steps:
- ${{ each myList in parameters.MyList }}:
- task: PowerShell#2
condition: succeeded()
displayName: "WOW a list"
inputs:
targetType: 'inline'
script: |
Write-Host("A list string: ${{ myList }}")
I am getting the error
Expected a sequence or mapping. Actual value '$(MyList)'
Note: The Powershell Task in the Template is just an example, it can be other Tasks that are not Powershell related (example: DotNetCoreCLI#2)
It will not work as you expect.
$myList = New-Object System.Collections.ArrayList
$myList.Add("wow")
Write-Output ("##vso[task.setvariable variable=MyList;]$myList")
You are not going here to have an array. Once you assign this to variable it will be a plain string. This is why you are getting this error.
If you run this:
- template: myRandomListTaskTemplate2.yml
parameters:
MyList:
- item1
- item2
all went fine. However, it is not possible to do what you at the moment.
Please check this developer community topic.