Using runtime expressions in azure pipelines yaml script - azure-devops

I'm trying to execute script by passing variable from the azure pipeline. Here is my simple test pipeline:
trigger:
- master
pool:
vmImage: 'windows-latest'
variables:
major: 1.2
minor: $[counter(variables['major'], 1)]
version: $[format('{0}.{1}', variables.major, variables.minor)]
name: $[format('{0} v{1}', 'Yaml Testing', variables['version'])]
steps:
- script: |
echo variables['version']
echo $(variables.version)
echo '$(variables.version)'
echo "$(variables.version')"
echo $[ variables['version'] ]
echo ${{ variables['version'] }}
echo $(Build.BuildNumber)
displayName: 'Run a multi-line script'
- script: $[format('{0} {1}', 'echo', variables['version'])]
displayName: 'Echo Formatted String'
Outputs of scripts are:
Generating script.
========================== Starting Command Output ===========================
##[command]"C:\windows\system32\cmd.exe" /D /E:ON /V:OFF /S /C "CALL "d:\a\_temp\3cb45b74-f6cd-4d2f-bf65-f635779b9d86.cmd""
variables['version']
$(variables.version)
'$(variables.version)'
"$(variables.version')"
$[ variables['version'] ]
$[format('{0}.{1}', variables.major, variables.minor)]
Yaml Testing v1.2.11
##[section]Finishing: Run a multi-line script
and
Generating script.
Script contents:
$[format('{0} {1}', 'echo', variables['version'])]
========================== Starting Command Output ===========================
##[command]"C:\windows\system32\cmd.exe" /D /E:ON /V:OFF /S /C "CALL "d:\a\_temp\5e42dc54-e027-4b9a-9af4-0db02e879b0f.cmd""
'$[format' is not recognized as an internal or external command,
operable program or batch file.
##[error]Cmd.exe exited with code '1'.
##[section]Finishing: Echo Formatted String
Strangely, code work fine in the name, but not when trying to use in the script.
What am I doing wrong?

$[] is evaluated at runtime, that is why it is not working. You can pass ${{expression}} to script like below:
- script: ${{format('{0} {1}', 'echo', '$(version)')}}
displayName: 'Echo Formatted String'
Expression in ${{}} will be evaluated at parse time. Before the -script is actually executed, expression in
${{}} is parsed into a valid command.
You can directly refer to the self-defined variables like this '$(variableName)' instead of $(variables.Name)

Related

Runtime Variable in Yaml Azure DevOps

If have created the Pipeline for One of our CRM Solution in Yaml
I have Created variables SolutionExit,SolutionName,SolutionVersion
#back-up.yaml
parameters:
- name: Solution
displayName: Solution Need to be backed up
type: object
default: [solution1,Solution2]
trigger:
- none
variables:
SolutionExit: ''
SolutionVersion: ''
SolutionName: ''
pool:
vmImage: windows-2019
steps:
- task: MSCRMToolInstaller#12
inputs:
nugetFeed: 'official'
psFeed: 'official'
- ${{each value in parameters.Solution}}:
- task: MSCRMGetSolution#12
inputs:
crmConnectionString: '$(ConnectStr)'
solutionName: '${{value}}'
existsVariableName: 'SolutionExit' #replace existing variable with output value True or False
versionVariableName: 'SolutionVersion' #replace existing variable with output value
displayVariableName: 'SolutionName' #replace existing variable with output value
- template: template/validate-solution.yml
parameters:
SolutionExit: $(SolutionExit)
#validate-solution.yml
parameters:
SolutionExit: ""
steps:
- ${{if eq(parameters.SolutionExit, 'True')}}:
- script: echo "Solution exist $(SolutionExit),$(SolutionVersion),$(SolutionName),${{parameters.SolutionExit}}"
- ${{else}}:
- bash: echo "solution doesnt exit $(SolutionExit),$(SolutionVersion),$(SolutionName),${{parameters.SolutionExit}}"
But runtime parameters are not accepted If condition never satisfies. How to pass the runtime parameter
This Is the result for both cases
If the solution available
Starting: Bash
==============================================================================
Task : Bash
Description : Run a Bash script on macOS, Linux, or Windows
Version : 3.214.0
Author : Microsoft Corporation
Help : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/bash
==============================================================================
Generating script.
Script contents:
echo "solution doesnt exit True,1.0.0.11,Solution1,True"
"C:\Program Files\Git\bin\bash.exe" -c pwd
/d/a/_temp
========================== Starting Command Output ===========================
"C:\Program Files\Git\bin\bash.exe" /d/a/_temp/237f7dc2-debc-4c47-ba88-41a6a97c49d9.sh
solution doesnt exit True,1.0.0.11,soultion1,True
Finishing: Bash
if solution not available
Starting: Bash
==============================================================================
Task : Bash
Description : Run a Bash script on macOS, Linux, or Windows
Version : 3.214.0
Author : Microsoft Corporation
Help : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/bash
==============================================================================
Generating script.
Script contents:
echo "solution doesnt exit False,,,False"
"C:\Program Files\Git\bin\bash.exe" -c pwd
/d/a/_temp
========================== Starting Command Output ===========================
"C:\Program Files\Git\bin\bash.exe" /d/a/_temp/49e5bea1-b380-4a74-a0ce-b89d0dfb3a07.sh
solution doesnt exit False,,,False
Finishing: Bash
How to fix this issue. There is compile time and Runtime variables,as per microsoft I tried ${{if eq(variables['SolutionExit'], 'True')}}: this too didnt work. We cannot use the macro variable in conditions like $(SolutionExit)
From your YAML definition, the variable will generate during the process of Pipeline Run.
The Variable should be Runtime variable. It will be expand at runtime. But the If expression and parameters will be expand at Compile time.
In this case, If expression will not work.
To solve this issue, you need to change to use Condition to replace the If expression.
At the same time, you don't need to use parameters to pass the variable value. You can directly use the variable in the condition.
For example:
Main YAML :
parameters:
- name: Solution
displayName: Solution Need to be backed up
type: object
default: [solution1,Solution2]
trigger:
- none
variables:
SolutionExit: ''
SolutionVersion: ''
SolutionName: ''
pool:
vmImage: windows-2019
steps:
- task: MSCRMToolInstaller#12
inputs:
nugetFeed: 'official'
psFeed: 'official'
- ${{each value in parameters.Solution}}:
- task: MSCRMGetSolution#12
inputs:
crmConnectionString: '$(ConnectStr)'
solutionName: '${{value}}'
existsVariableName: 'SolutionExit' #replace existing variable with output value True or False
versionVariableName: 'SolutionVersion' #replace existing variable with output value
displayVariableName: 'SolutionName' #replace existing variable with output value
- template: template/validate-solution.yml
Template YAML:
steps:
- script: echo "Solution exist $(SolutionExit),$(SolutionVersion),$(SolutionName),${{parameters.SolutionExit}}"
condition: eq(variables['SolutionExit'], 'true')
- bash: echo "solution doesnt exit $(SolutionExit),$(SolutionVersion),$(SolutionName),${{parameters.SolutionExit}}"
condition: ne(variables['SolutionExit'], 'true')

retrieve value from nested object parameter

my question is closely releted to this one, but I this one doesnt seem to help me :( How to retrieve value from nested object?
I am trying to loop over an object that i get as a parameter, I use a default value as an example, the values in this object are also objects.
I have a template:
parameters:
- name: fruits
type: object
default:
- Banana:
name: " acuminata"
url: "www.banana.com/ acuminata"
- Apple:
name: "jonaGold"
url: "www.apple.com"
steps:
- bash: echo ${{parameters.fruits[0].url}}
- ${{ each fruit in parameters.fruits }}:
- script: echo ${{fruit.name}}
Both the first bash and the second bash are not working. Is it possible to use objects in azure pipelines like this?
This is my output for both bash steps:
2022-04-13T13:01:21.8035586Z ##[section]Starting: CmdLine
2022-04-13T13:01:21.8178179Z ==============================================================================
2022-04-13T13:01:21.8178533Z Task : Command line
2022-04-13T13:01:21.8178874Z Description : Run a command line script using Bash on Linux and macOS and cmd.exe on Windows
2022-04-13T13:01:21.8179171Z Version : 2.201.1
2022-04-13T13:01:21.8179614Z Author : Microsoft Corporation
2022-04-13T13:01:21.8179954Z Help : https://learn.microsoft.com/azure/devops/pipelines/tasks/utility/command-line
2022-04-13T13:01:21.8180355Z ==============================================================================
2022-04-13T13:01:23.2732262Z Generating script.
2022-04-13T13:01:23.2840010Z Script contents: shell
2022-04-13T13:01:23.2849739Z echo
2022-04-13T13:01:23.3217106Z ========================== Starting Command Output ===========================
2022-04-13T13:01:23.3517504Z ##[command]"C:\Windows\system32\cmd.exe" /D /E:ON /V:OFF /S /C "CALL "D:\a\_temp\7a91b1cc-432d-4a20-b39c-7c6b0c78724b.cmd""
2022-04-13T13:01:23.3655983Z ECHO is off.
2022-04-13T13:01:23.3990627Z ##[section]Finishing: CmdLine
I just found a solution. The other thread does provide an answer, I just need to read better :P.
You can access the values with the 'value' keyword. I also removed the '-' before the keywords so its not an array anymore. The code now looks like this:
parameters:
- name: fruits
type: object
default:
Banana:
name: " acuminata"
url: "www.banana.com/ acuminata"
Apple:
name: "jonaGold"
url: "www.apple.com"
steps:
- ${{ each fruit in parameters.fruits }}:
- script: echo ${{fruit.value.name}}
And I get this output:
I did not yet get the first one working, but i don't really need this for my project
- bash: echo ${{parameters.fruits[0].value.url}}

Azure DevOps Variable Groups Password not working

I'm having userName and passWord field in my Azure DevOps Variable group, and I would like to pass the userName and passWord to my python script as a command line argument, I'm able to get the userName without any issue but passWord is not coming when I pass it from my pipeline script. I tried printing using echo - its printing empty - not evening printing masked value like this '*****'
Please find my pipeline code below:
trigger:
- none
variables:
- group: myVariableGrp
pool:
name: myAgents
stages:
- stage: Testing
jobs:
- job: Test
displayName: Test script
steps:
- task: Bash#3
displayName: Test variable
inputs:
targetType: inline
script: |
echo "hello variables"
echo $USER_NAME
echo $MY_PASSWORD
- task: CmdLine#2
displayName: Test Python script
inputs:
script: python test.py $USER_NAME $MY_PASSWORD
test.py
import sys
# total arguments
#n = len(sys.argv)
#print("Total arguments passed:", n)
print("UserName:", sys.argv[1])
print("Password:", sys.argv[2])
I'm getting the below error:
Traceback (most recent call last):
File "test.py", line 12, in <module>
print("Password:", sys.argv[2])
IndexError: list index out of range
##[error]Bash exited with code '1'.
Finishing: Test Python script
Secret variables are not automatically exposed as env variables. You have to explicitly map them: Reference secret variables in variable groups.
In this case, it would be:
- task: Bash#3
displayName: Test variable
inputs:
targetType: inline
env:
MAPPED_PASSWORD: $(MY_PASSWORD)
script: |
echo "hello variables"
echo $USER_NAME
echo $MAPPED_PASSWORD

Set bash command output into an azure yml variable

I'm working with Azure DevOps and I need to set the return of bash commands into some variables, for example I have the following:
variables:
VERSION: 7.2 # works fine
FILE_VERSION: ${{cat public/VERSION}} # syntax error
I tried some some variations from ${{}} without success and I could not found the right syntax for it, but I think it must be possible.
You should use a bash step for that.
Like this:
steps:
- bash: |
echo "##vso[task.setvariable variable=FILE_VERSION]$(cat public/VERSION)"
You need to do this in two steps
STEP 1: Set static value first
variables:
VERSION: 7.2
STEP 2: Create step to calculate the value
Note this will only be available for subsequent steps
- bash: |
FILE_VERSION="$(cat public/VERSION)"
echo "##vso[task.setvariable variable=FILE_VERSION]$FILE_VERSION"
Resources:
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch
We shouldn't use cat command when defining variables. The command should be located in tasks/steps.
According to your description, I assume you're trying to pass the content of public/7.2 to FILE_VERSION variable. Here's my test:
1.Azure Devops Git repo:
2.Define the VERSION variable:
variables:
VERSION: 7.2
Run the cat command and set job-scoped variable:
- bash: |
FILE_VERSION=$(cat public/$(VERSION))
echo "##vso[task.setvariable variable=FILE_VERSION]$FILE_VERSION"
The whole yaml:
trigger:
- master
pool:
vmImage: 'ubuntu-latest'
variables:
VERSION: 7.2
steps:
- bash: |
FILE_VERSION=$(cat public/$(VERSION))
echo "##vso[task.setvariable variable=FILE_VERSION]$FILE_VERSION"
#Use second bash task to test the variable.
- task: Bash#3
inputs:
targetType: 'inline'
script: |
echo $(FILE_VERSION)

Adding multiple commands under scripts stage in Azure YAML pipeline

I need to execute following commands in the project repo to a do a build
echo Building Software/linux_framework
source /opt/pkg/linux/settings.sh
cd Software/linux_framework
make images HARDWARE=../my_xsa/ BOARD=local
Snippet from my YAML file:
pool:
name: Default
steps:
- script: echo Building Software/linux_framework
source /opt/pkg/linux/settings.sh
cd Software/linux_framework
make images HARDWARE=../my_xsa/ BOARD=local
displayName: 'Make Project'
When I run the build all 4 commands are just echoed on the terminal. How to execute them as separate commands in the same terminal session
Try something like this:
- script: |
echo Building Software/linux_framework
source /opt/pkg/linux/settings.sh
cd Software/linux_framework
make images HARDWARE=../my_xsa/ BOARD=local
displayName: 'Make Project'
Pay attention to the difference between one-line script and multi-line script:
- script: echo Hello, world!
displayName: 'Run a one-line script'
- script: |
echo Hello, world!
echo Hello, world!
displayName: 'Run a multi-line script'