I have an azure pipeline YAML file and I can't figure out how to declare a list variable in a script and use it on the next step.
What I would like is something like that:
- script: echo "##vso[task.setvariable variable=packages]#('a','b')"
- template: publish-template.yml
parameters:
packages: packages
or
- script: echo "##vso[task.setvariable variable=packages]#('a','b')"
- ${{ each package in parameters.packages }}:
- script: echo ${{ package }}
But I can't find out how it's possible.
Thanks :)
You can't do such thing. Variables are kept as strings. So this script: echo "##vso[task.setvariable variable=packages]#('a','b')" will produce a string ('a','b') and not an array. You can parse is script task and treat as array but there is no way to have array served at to level as you show in your example.
Please check it here:
Yaml variables have always been string: string mappings. The doc appears to be currently correct, though we may have had a bug when last you visited.
We are preparing to release a feature in the near future to allow you to pass more complex structures. Stay tuned!
Related
I am trying to use variables defined at the root level on a YAML pipeline inside Azure DevOps inside templates via the template syntax, but it seems that the variables are not available inside the templates, but when adding the steps directly to the pipeline, the exact same thing works perfectly.
So with a pipeline snippet like that
variables:
- name: test
value: asdf
stages:
- stage:
jobs:
- job: test_job
steps:
- script: echo "${{ variables.test }}"
- template: ./test.yaml
And a test.yaml like that
jobs:
- job: test
steps:
- script: echo "${{ variables.test }}"
The script inside the test_job job writes out asdf while the job inside the template just resolves to echo "".
Since my understanding of pipeline templates is, that those basically get inserted into the main pipeline, this seems like a bug. Any ideas on how to use the root variables in a template syntax inside templates or why this is not working? (Macro synatx is not an option as I need the variable inside a templated condition like ${{ if eq(variables['test'], 'asdf') }})
For security reasons, we only allow you to pass information into
templated code via explicit parameters.
https://docs.microsoft.com/en-us/azure/devops/pipelines/process/templates?view=azure-devops
The means the author of the pipeline using your template needs to
commit changes where they explicitly pass the needed info into your
template code.
There are some exceptions to this, where the variable is statically
defined in the same file or at pipeline compile time, but generally
speaking, it’s probably better to use parameters for everything that
does not involve system-defined read-only dynamic variable and
custom-defined dynamic output variables.
This behavior is by design, check this thread in the developer community
So you can either pass the variable as a parameter to the template or define a centralized variables file to include in the template like here
Here is a part of the workflow file:
env:
resourceGroupName: 'rg-${GITHUB_REF#refs/heads/}'
I am trying to create an environment variable that concats a string and another environment variable but in the logs all I see from an echo from either way I can see is the following:
echo "$resourceGroupName" -> rg-$***GITHUB_REF#refs/heads/***
echo "{{ env.resourceGroupName}}" -> *** env.resourceGroupName***
Instead of what I'd have expected such as:
rg-the-name-of-the-branch
The docs do not seem particularly good in this regard. It also seems to be trying and failing to mask the variables as it's placing erroneous extra asterisks.
You need to have ${{ github.ref }} for it to work with the GitHub Actions vars. The linux envs are also exposed, so you can also use $GITHUB_REF directly without the brackets.
Alternatively, you can also use the format option to combine strings if you like that more:
format("rb-{0}", ${{ github.ref }})
I am trying this for quite sometime now & not able to figure out how to proceed further. My requirement is to dynamically calculate a variable in a bash task & then in next step use the same as a parameter to the gulp task. Below are my two tasks as part of the build pipeline (removed lines for simplicity)
- task: Bash#3
name: SetVariableValue
inputs:
targetType: inline
script: >
// removed
myvariableValue=$(do something & calculate here, assume value will be 'abc')
// Set to an output variable
echo "##vso[task.setvariable variable=myVar;isOutput=true]$myvariableValue"
- task: gulp#0
displayName: Publish front-end
inputs:
gulpFile: $(Build.SourcesDirectory)/../gulpfile.js
targets: publish
arguments: >-
--buildId $(Build.BuildId) --buildNumber $(Build.BuildNumber) --sourceBranch $(Build.SourceBranch) --var $(myVar)
gulpjs: >-
$(Build.SourcesDirectory)/../node_modules/gulp/bin/gulp.js
enableCodeCoverage: false
I am using linux machines to build angular packages and the gulp publish command is used to package our code depending on the said variable.
With above steps all the rest parameters like BuildId, BuildNumber, SourceBranch are getting passed correctly but the 'var' parameter is being passed as $(myVar) only, rather abc
Can you please help me on what am I missing here? I tried multiple things like --var $(Build.myVar), but not able to make it work.
Thanks
Sanjay
Oh man all I needed was to correct the name of the step & refer it without mistyping :(
--var $(SetVariableValue.myVar) did the trick. Thanks.
We looking to create a pipeline to update our multi-tenant azure environment. We need to perform some actions during the update per tenant. To accomplish this, we would like to create a job per tenant, so we can process tenants in parallel. To accomplish this, I want to use a runtime parameter to pass the tenants to update to my pipeline as follows:
parameters:
- name: tenants
type: object
the value of the tenants parameter might look like something like this:
- Name: "customer1"
Someotherproperty: "some value"
- Name: "customer2"
Someotherproperty: "some other value"
to generate the jobs, we do something like this:
stages:
- stage:
jobs:
- job: Update_Tenant
strategy:
matrix:
${{ each tenant in parameters.Tenants }}:
${{ tenant.tenantName }}:
name: ${{ tenant.tenantName }}
someproperty: ${{ tenant.otherProperty }}
maxParallel: 2
steps:
- checkout: none
- script: echo $(name).$(someproperty)
Now what we need, is some way to fill this tenants parameter. Now I tried a few solutions:
Ideally I would like to put a build stage before the Update_Tenants stage to call a REST api to get the tenants, and expand the tenants parameter when the Update_Tenants stage starts, but this is not supported AFAIK, since parameter expansion is done when the pipeline starts.
A less ideal but still workable option would have been to create a variable group yaml file containing the tenants, and include this variable group in my pipeline, and use the ${{ variables.Tenants }} syntax to reference them. However, for some reason, variables can only be strings.
The only solution I can currently think of, is to create a pipeline that calls a REST api to get the tenants to update, and then uses the azure devops api to queue the actual update process with the correct parameter value. But this feels like a bit of a clunky workaround to accomplish this.
Now my question is, are there any (better?) alternatives to accomplish what I want to do?
Maybe this can help. I was able to use external source (.txt file) to fill array variable in azure pipelines.
Working example
# Create a variable
- bash: |
arrVar=()
for images in `cat my_images.txt`;do
arrVar+=$images
arrVar+=","
done;
echo "##vso[task.setvariable variable=list_images]$arrVar"
# Use the variable
# "$(list_images)" is replaced by the contents of the `list_images` variable by Azure Pipelines
# before handing the body of the script to the shell.
- bash: |
echo my pipeline variable is $(list_images)
Sources (there is also example for matrix)
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch#set-a-job-scoped-variable-from-a-script
Other sources
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/runtime-parameters?view=azure-devops&tabs=script
To accomplish this, we would like to create a job per tenant, so we
can process tenants in parallel.
Apart from rolling deployment strategy, you can also check Strategies and Matrix.
You can try something like this unless you have to use Runtime parameters:
jobs:
- job: Update
strategy:
matrix:
tenant1:
Someotherproperty1: '1.1'
Someotherproperty2: '1.2'
tenant2:
Someotherproperty1: '2.1'
Someotherproperty2: '2.2'
tenant3:
Someotherproperty1: '3.1'
Someotherproperty2: '3.2'
maxParallel: 3
steps:
- checkout: none
- script: echo $(Someotherproperty1).$(Someotherproperty2)
displayName: 'Echo something'
I'm looking for a way to define a variable in my azure-pipelines.yml file where I can substring the 'Build.SourceVersion' -> Use the first 7 characters only.
Seems like there is no build-in function that can do such string operations in the documentation. Is there something I'm missing?
My other approach would be to use a bash task and overwrite the variable there but finding something build-in that can do this would be way better solution.
My other approach would be to use a bash task and overwrite the variable there but finding something build-in that can do this would be way better solution.
I agree with Lucas. There is no such built-in task to get the first 7 characters of $(Build.SourceVersion) in Azure DevOps.
We could use the command line/powershell task to split long sha into short sha:
echo $(Build.SourceVersion)
set TestVar=$(Build.SourceVersion)
set MyCustomVar=%TestVar:~0,7%
echo %MyCustomVar%
echo ##vso[task.setvariable variable=ShortSourceVersion]%MyCustomVar%
In this case, we could get the short versions of Build.SourceVersion and set it as environment variable.
Then we could set this command line task as a task group:
So, we could use this task to set the ShortSourceVersion directly.
Hope this helps.
Here is the shortest version that I use for this job;
- bash: |
longcommithash=$(Build.SourceVersion)
echo "##vso[task.setvariable variable=shorthash;]$(echo ${longcommithash::9})"
It gives you the output as shown below;
On Azure devops it's possible to be done using batch technique indeed - like other answers propose in here - however I wanted to remove string from the end of string - and with batch this gets more complex ( See following link ).
Then I've concluded to use Azure's built-in expressions, for example like this:
- name: sourceBranchName
value: ${{ replace(variables['Build.SourceBranch'], 'refs/heads/') }}
replace will have similar effect to substring - so it will remove unnecessary part from original string.
If however you want to concatenate some string and then remove it from original name - it's possible to be done using for instance format function:
- name: originBranchName
value: ${{ replace(variables.sourceBranchName, format('-to-{0}', variables.dynamicvar ) ) }}
If dynamicvar is a - then function will remove -to-a from branch name.
You are right, there is no native way to do it. You will have to write a script to transform the variable.
Here is an example:
trigger:
- master
resources:
- repo: self
stages:
- stage: Build
displayName: Build image
jobs:
- job: Build
displayName: Build
pool:
vmImage: 'ubuntu-latest'
steps:
- task: CmdLine#2
inputs:
script: ' x=`echo "$(Build.SourceVersion)" | head -c 7`; echo "##vso[task.setvariable variable=MyVar]$x"'
- task: CmdLine#2
inputs:
script: 'echo "$(MyVar)"'
Sure you can! If you absolutely must. Here is a runtime computation taking the first 7 characters of the Build.SourceVersion variable.
variables:
example: ${{ format('{0}{1}{2}{3}{4}{5}{6}', variables['Build.SourceVersion'][0], variables['Build.SourceVersion'][1], variables['Build.SourceVersion'][2], variables['Build.SourceVersion'][3], variables['Build.SourceVersion'][4], variables['Build.SourceVersion'][5], variables['Build.SourceVersion'][6]) }}
NB: I can't get it to work with $[...] syntax as the variable is apparently empty at the initial.