Set bash command output into an azure yml variable - azure-devops

I'm working with Azure DevOps and I need to set the return of bash commands into some variables, for example I have the following:
variables:
VERSION: 7.2 # works fine
FILE_VERSION: ${{cat public/VERSION}} # syntax error
I tried some some variations from ${{}} without success and I could not found the right syntax for it, but I think it must be possible.

You should use a bash step for that.
Like this:
steps:
- bash: |
echo "##vso[task.setvariable variable=FILE_VERSION]$(cat public/VERSION)"

You need to do this in two steps
STEP 1: Set static value first
variables:
VERSION: 7.2
STEP 2: Create step to calculate the value
Note this will only be available for subsequent steps
- bash: |
FILE_VERSION="$(cat public/VERSION)"
echo "##vso[task.setvariable variable=FILE_VERSION]$FILE_VERSION"
Resources:
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch

We shouldn't use cat command when defining variables. The command should be located in tasks/steps.
According to your description, I assume you're trying to pass the content of public/7.2 to FILE_VERSION variable. Here's my test:
1.Azure Devops Git repo:
2.Define the VERSION variable:
variables:
VERSION: 7.2
Run the cat command and set job-scoped variable:
- bash: |
FILE_VERSION=$(cat public/$(VERSION))
echo "##vso[task.setvariable variable=FILE_VERSION]$FILE_VERSION"
The whole yaml:
trigger:
- master
pool:
vmImage: 'ubuntu-latest'
variables:
VERSION: 7.2
steps:
- bash: |
FILE_VERSION=$(cat public/$(VERSION))
echo "##vso[task.setvariable variable=FILE_VERSION]$FILE_VERSION"
#Use second bash task to test the variable.
- task: Bash#3
inputs:
targetType: 'inline'
script: |
echo $(FILE_VERSION)

Related

Runtime Variable in Yaml Azure DevOps

If have created the Pipeline for One of our CRM Solution in Yaml
I have Created variables SolutionExit,SolutionName,SolutionVersion
#back-up.yaml
parameters:
- name: Solution
displayName: Solution Need to be backed up
type: object
default: [solution1,Solution2]
trigger:
- none
variables:
SolutionExit: ''
SolutionVersion: ''
SolutionName: ''
pool:
vmImage: windows-2019
steps:
- task: MSCRMToolInstaller#12
inputs:
nugetFeed: 'official'
psFeed: 'official'
- ${{each value in parameters.Solution}}:
- task: MSCRMGetSolution#12
inputs:
crmConnectionString: '$(ConnectStr)'
solutionName: '${{value}}'
existsVariableName: 'SolutionExit' #replace existing variable with output value True or False
versionVariableName: 'SolutionVersion' #replace existing variable with output value
displayVariableName: 'SolutionName' #replace existing variable with output value
- template: template/validate-solution.yml
parameters:
SolutionExit: $(SolutionExit)
#validate-solution.yml
parameters:
SolutionExit: ""
steps:
- ${{if eq(parameters.SolutionExit, 'True')}}:
- script: echo "Solution exist $(SolutionExit),$(SolutionVersion),$(SolutionName),${{parameters.SolutionExit}}"
- ${{else}}:
- bash: echo "solution doesnt exit $(SolutionExit),$(SolutionVersion),$(SolutionName),${{parameters.SolutionExit}}"
But runtime parameters are not accepted If condition never satisfies. How to pass the runtime parameter
This Is the result for both cases
If the solution available
Starting: Bash
==============================================================================
Task : Bash
Description : Run a Bash script on macOS, Linux, or Windows
Version : 3.214.0
Author : Microsoft Corporation
Help : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/bash
==============================================================================
Generating script.
Script contents:
echo "solution doesnt exit True,1.0.0.11,Solution1,True"
"C:\Program Files\Git\bin\bash.exe" -c pwd
/d/a/_temp
========================== Starting Command Output ===========================
"C:\Program Files\Git\bin\bash.exe" /d/a/_temp/237f7dc2-debc-4c47-ba88-41a6a97c49d9.sh
solution doesnt exit True,1.0.0.11,soultion1,True
Finishing: Bash
if solution not available
Starting: Bash
==============================================================================
Task : Bash
Description : Run a Bash script on macOS, Linux, or Windows
Version : 3.214.0
Author : Microsoft Corporation
Help : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/bash
==============================================================================
Generating script.
Script contents:
echo "solution doesnt exit False,,,False"
"C:\Program Files\Git\bin\bash.exe" -c pwd
/d/a/_temp
========================== Starting Command Output ===========================
"C:\Program Files\Git\bin\bash.exe" /d/a/_temp/49e5bea1-b380-4a74-a0ce-b89d0dfb3a07.sh
solution doesnt exit False,,,False
Finishing: Bash
How to fix this issue. There is compile time and Runtime variables,as per microsoft I tried ${{if eq(variables['SolutionExit'], 'True')}}: this too didnt work. We cannot use the macro variable in conditions like $(SolutionExit)
From your YAML definition, the variable will generate during the process of Pipeline Run.
The Variable should be Runtime variable. It will be expand at runtime. But the If expression and parameters will be expand at Compile time.
In this case, If expression will not work.
To solve this issue, you need to change to use Condition to replace the If expression.
At the same time, you don't need to use parameters to pass the variable value. You can directly use the variable in the condition.
For example:
Main YAML :
parameters:
- name: Solution
displayName: Solution Need to be backed up
type: object
default: [solution1,Solution2]
trigger:
- none
variables:
SolutionExit: ''
SolutionVersion: ''
SolutionName: ''
pool:
vmImage: windows-2019
steps:
- task: MSCRMToolInstaller#12
inputs:
nugetFeed: 'official'
psFeed: 'official'
- ${{each value in parameters.Solution}}:
- task: MSCRMGetSolution#12
inputs:
crmConnectionString: '$(ConnectStr)'
solutionName: '${{value}}'
existsVariableName: 'SolutionExit' #replace existing variable with output value True or False
versionVariableName: 'SolutionVersion' #replace existing variable with output value
displayVariableName: 'SolutionName' #replace existing variable with output value
- template: template/validate-solution.yml
Template YAML:
steps:
- script: echo "Solution exist $(SolutionExit),$(SolutionVersion),$(SolutionName),${{parameters.SolutionExit}}"
condition: eq(variables['SolutionExit'], 'true')
- bash: echo "solution doesnt exit $(SolutionExit),$(SolutionVersion),$(SolutionName),${{parameters.SolutionExit}}"
condition: ne(variables['SolutionExit'], 'true')

Azure DevOps set env variable

I am trying to set an environment variable to use it in another task but it doesn't work.
The first task should set the variable "versionTag" so I can use it in the next task as $(versionTag).
Can anyone help me with this?
- task: Bash#3
displayName: Create version tag
inputs:
targetType: 'inline'
script: |
versionTag=$(echo "$(Build.BuildNumber)" | tr '+' '-')
echo "versionTag: ${versionTag}"
echo "##vso[task.setvariable variable=versionTag]${versionTag}"
- task: Docker#2
displayName: Create runtime docker image
inputs:
containerRegistry: '$(dockerRegistryServiceConnection)'
repository: '$(imageRepository)'
command: 'build'
Dockerfile: '$(dockerfilePath)'
buildContext: '$(Build.SourcesDirectory)'
tags: |
$(tags)
$(versionTag)
There's a magic command string you can write to the log:
echo "##vso[task.prependpath]c:\my\directory\path"
The path will be updated for the scope of the Job. If your pipeline has multiple jobs, you need to issue the same command for future jobs as well.
The updated path will be available in the next step. Not in the step in which you issue the command.

How to reference a variable in Azure Devops on an Linux Agent which contains a dash/minus/-

There is this question which asks about using variables with a minus on linux agents. I tried an indirect reference as suggested here by doing ${!RELEASE_ARTIFACTS__BUILDPROJECT_SOURCEBRANCH} but this does not work, and causes an error in my pipeline. I tried some other approaches using the env command but they too throw errors... Seems odd that this issue persists I guess not as much attention is put on Linux Agents as to Windows. But I am wondering if anyone has found a solution to this?
Here is some code on a Linux agent with variables with dashes.
Try running a bash task and running printenv which will show you all variables available in the linux agent.
Please clarify further what you are trying to do because I can see vars with dashes are working fine in the Linux agents.
pool: default
variables:
- name: test
value: test-value-with-dashes
- name: test-name
value: test-name-with-dashes
- name: System.Debug
value: true
trigger:
- never
stages:
- stage: approval_check
jobs:
- job: bash
steps:
- task: Bash#3
inputs:
targetType: 'inline'
script: |
# Write your commands here
set -x
echo $(test)
echo $(test-name)
echo "print all env vars"
printenv
And here is the output of printenv you can see the vars available in the build agent.
echo test-value-with-dashes
echo test-name-with-dashes
echo 'print all env vars'
printenv
.... lots of env vars printed...
BUILD_SOURCEBRANCH=refs/heads/main
BUILD_ARTIFACTSTAGINGDIRECTORY=/home/james/_work/3/a
...

Is there a short 7-digit version of $(SourceVersion) in Azure Devops?

I am trying to set our build names to a format of...
$(BuildDefinitionName)_$(versionMajor).$(versionMinor).$(versionPatch)+$(SourceBranchName).$(SourceVersion) e.g.
OurBigLibraryCI_1.2.3+master.10bbc577
However I coudn't find any predefined variable holding the "short" (7-digit) version of the commit hash. $(SourceVersion) holds the full SHA-1 hash.
How would one shorten that in yaml based pipeline?
You can use traditional command substitution via backticks to obtain the short git hash (SHA-1), assuming that the code is being checked out in $(Build.SourcesDirectory):
- bash: |
short_hash=`git rev-parse --short=7 HEAD` ## At least 7 digits, more if needed for uniqueness
echo ""
echo "Full git hash: $(Build.SourceVersion)"
echo "Short git hash: $short_hash"
echo "##vso[task.setvariable variable=short_hash]$short_hash" ## Store variable for subsequent steps
workingDirectory: $(Build.SourcesDirectory)
displayName: Get short git hash
Output:
Full git hash: f8d63b1aaa20cf348a9b5fc6477ac80ed23d5ca0
Short git hash: f8d63b1
The following steps in the pipeline can then use the short hash via the variable $(short_hash).
(This is better than manually trimming down the full git hash to seven characters, since this will add extra digits if needed to uniquely identify the commit, see https://stackoverflow.com/a/21015031/1447415.)
Update: Improved version
The following improved version checks that the git hashes match (that the full hash starts with the short hash) and fails the step otherwise:
- bash: |
short_hash=`git rev-parse --short=7 HEAD`
echo ""
echo "Full git hash: $(Build.SourceVersion)"
echo "Short git hash: $short_hash"
echo ""
## Fail step if full hash does not start with short hash
if [[ $(Build.SourceVersion) != $short_hash* ]]; then
echo "--> Hashes do not match! Aborting."
exit 1
fi
echo "--> Hashes match. Storing short hash for subsequent steps."
## Store variable for subsequent steps
echo "##vso[task.setvariable variable=short_hash]$short_hash"
workingDirectory: $(Build.SourcesDirectory)
displayName: Get short git hash
- script: |
echo $sourceVersion
commitHash=${sourceVersion:0:7}
echo $commitHash
echo "##vso[task.setvariable variable=commitHash]$commitHash" ## Set variable for using in other tasks.
env: { sourceVersion: $(Build.SourceVersion) }
displayName: Git Hash 7-digit
workingDirectory: #workingDirectory
- task: Docker#2
displayName: Build & Push image
inputs:
command: 'buildAndPush'
containerRegistry: '$(myRegistry)'
repository: $(myContainerRepository)
Dockerfile: $(myDockerfile)
buildContext: '$(myBuildContext)'
tags: $(commitHash) ## The variable was defined above.
Here's example for vmImage: "ubuntu-latest". Step:
Split 7-characters from Pre-defined GitHash
Assign it to Pipeline variable. Don't confuse $(azure_pipeline_variable) with ${bash_shell_variable} or $bash_shell_variable.
Use it by $(commitHash)
Read more:
Assign variable in Azure pipeline
Using script in Azure pipeline
How would one shorten that in yaml based pipeline?
There is no out of box variable to get the 7-digit version of $(SourceVersion) in Azure Devops. Because the ShortSha is 8-digit version.
So, to resolve this issue, just like #4c74356b41 said, we have to use bash\powershell script to split long sha into short sha.
You can check my following sample for some more details:
steps:
- script: |
echo $(Build.SourceVersion)
set TestVar=$(Build.SourceVersion)
set MyCustomVar= %TestVar:~0,7%
echo %MyCustomVar%
displayName: 'Command Line Script'
The result:
========================== Starting Command Output ===========================
##[command]"C:\WINDOWS\system32\cmd.exe" /D /E:ON /V:OFF /S /C "CALL "C:\VS2017Agent\_work\_temp\be5f6293-77d8-41b7-a537-49e3b2e7bc6c.cmd""
cb124539c4cb7f19dc8e50e1b021f93c5ffaf226
cb12453
##[section]Finishing: Command Line Script
So, we could get the 7-digit version of $(SourceVersion) is cb12453.
Hope this helps.
you could use gitversion for that, it would expose shortsha under $(GitVersion.ShortSha) variable after you run the gitversion task.
on the other hand shortsha is just first 7 characters of a real sha, so you can just use some sort of bash\powershell script to split long sha into short sha
In Git, what is the difference between long and short hashes?
- task: gittools.gittools.setup-gitversion-task.gitversion/setup#0
displayName: gitversion/setup
inputs:
versionSpec: 5.x
- task: gittools.gittools.execute-gitversion-task.gitversion/execute#0
displayName: gitversion/execute
replacement for the deprecated extension
As the docs say you can't use $(Build.SourceVersion) variable directly in the build name, nor is there an in-built substring Expression
But you can update the Build Number variable dynamically:
pool:
vmImage: 'ubuntu-latest' #'windows-latest' - works on either
steps:
- checkout: none
- powershell: |
$shortHash = "$(Build.SourceVersion)".Substring(0, 7)
Write-Host "##vso[task.setvariable variable=shortHash]$shortHash"
displayName: Set Short Hash
- powershell: Write-Host $(shortHash)
displayName: Display Short Hash
- powershell: Write-Host "##vso[build.updatebuildnumber]$(Build.DefinitionName)_AnyOtherValues_$(shortHash)"
displayName: Update Build Number
- powershell: Write-Host $(Build.BuildNumber)
displayName: Display Build Number

Azure Devops - Setting variable in YAML script to datetime

I'm trying to generate a name for a NuGet package in an Azure DevOps YAML pipeline.
The context is creating packages on each automated build with the name of the package, the name of the branch, the date and the incrementing revision number.
packageName-branchName-year-month-day-revision
This way when a new feature branch is created it will generate a unique package which can be used for testing in projects which need it.
I'm struggling to set a variable in the YAML file from environmental variables such as the date or revision number.
Using the ##vso command I'm able to set 'myVariable' to hello and print it out.
- script: |
echo '##vso[task.setvariable variable=myVariable]hello'
- script: |
echo my variable is $(myVariable)
When I try setting the variable from PowerShell as below I get the following error '#$dateStr' is not recognized as an internal or external command'.
# Create a variable
- script: |
#$dateStr = (Get-Date).ToString('yyyy-MM-dd')
echo '##vso[task.setvariable variable=myVariable]#$dateStr'
# Print the variable
- script: |
echo my variable is $(myVariable)
When I try to set a variable in the variables section of the YAML file as so.
variables:
solution: '**/*.sln'
foo: $(Date:yyyyMMdd)
- script: |
echo my variable is $(foo)
The variable is not interpolated and it outputs as.
'my variable is $(Date:yyyyMMdd)'
How do I create variables based on environmental variables such as $(rev) and $(Date)?
i dont think there is a built-in date variable, but for the powershell case you just need to drop # before variable and it has to be enclosed with " else powershell wont expand your variable
echo "##vso[task.setvariable variable=myVariable]$dateStr"
Thanks for the help.
This is how I solved the issue in the end. With non release branches I use build# + beta + branch name to generate a unique name for the nuget packages I'm creating. With release branches I just use the build #.
# Create version number for non release package(s) - 1.0.xxx-beta-develop for example
- powershell: |
[String]$buildNumber = $Env:BUILD_BUILDNUMBER.Substring($Env:BUILD_BUILDNUMBER.LastIndexOf('.') + 1)
[String]$branchName = $Env:BUILD_BUILDNUMBER.Substring(0, $Env:BUILD_BUILDNUMBER.LastIndexOf('.'))
Write-Host "##vso[task.setvariable variable=nugetVersion]1.0.$($buildNumber)-beta-$($branchName)"
displayName: 'Create beta version number for production nuget packages - run when in non release branch'
condition: and(succeeded(), ne(variables['Build.SourceBranch'], 'release/'))
# Create version number for release package(s) - 1.0.xxx for example
- powershell: |
[String]$buildNumber = $Env:BUILD_BUILDNUMBER.Substring($Env:BUILD_BUILDNUMBER.LastIndexOf('.') + 1)
Write-Host "##vso[task.setvariable variable=nugetVersion]1.0.' + $($buildNumber)
displayName: 'Create version number for production nuget packages - run when in release branch'
condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'release/'))
- task: NuGetCommand#2
inputs:
command: 'pack' # Options: restore, pack, push, custom
versioningScheme: byEnvVar
versionEnvVar: nugetVersion
packDestination: '$(Build.ArtifactStagingDirectory)\nuget'
packagesToPack: '**/Floww.*.nuspec'
using the -script is great for running commands that can be executed cross-platform like npm i
As the date is not a global variable you will need to create it. The issue is that depending on what build agent you are running on (window|mac|linx) you will have different ways to set that variable, see the documentation...
The following will allow you to use a $d variable like other variables
steps:
- bash: |
export d=$(date +%F)
echo "The Date is $d"
echo "##vso[task.setvariable variable=fileName]$d"
condition: eq( variables['Agent.OS'], 'Linux' )
- powershell: |
Set-Variable -Name d -Value (Get-Date).ToString('yyyy-MM-dd')
Write-Host "##vso[task.setvariable variable=fileName]$d"
condition: eq( variables['Agent.OS'], 'Windows_NT' )