I am trying to set up an environment variable that can be used from the Azure Devops Library that by default is set to null value. This part is working, but I want the development teams to be able to overwrite that value ad hoc. I don't want to add this manually to all environment variables and I would like to build a condition in my playbooks to say when var != "" rather then when: var != "$(rollback)"
Here is my config:
name: $(Date:yyyyMMdd)$(Rev:.r)
trigger: none
pr: none
resources:
pipelines:
- pipeline: my-ui
source: my-ui-ci-dev
trigger: true
variables:
- group: Dev
jobs:
- job: my_cd
pool:
vmImage: "ubuntu-20.04"
container:
image: "myacr.azurecr.io/devops/services-build:$(services_build_tag_version)"
endpoint: "Docker Registry"
steps:
- task: Bash#3
displayName: "My Playbook"
env:
git_username: "$(git_username)"
git_password: "$(git_password)"
config_repo: "$(config_repo)"
service_principal_id: "$(service_principal_id)"
service_principal_secret: "$(service_principal_secret)"
subscription_id: "$(subscription_id)"
tenant_id: "$(tenant_id)"
rollback: "$(rollback)"
source_dir: "$(Build.SourcesDirectory)"
env_dir: "$(Agent.BuildDirectory)/env"
HELM_EXPERIMENTAL_OCI: 1
inputs:
targetType: "inline"
script: |
ansible-playbook cicd/ansible/cd.yaml -i "localhost, " -v
When choosing to run the pipeline, I would like the developers to just go to Run > Add Variable > Manually add the variable and value > run pipeline
And then in the playbook the value is "" if not defined or if it is then show as the value they type. Any suggestions on how I can do this with AZDO?
You can do this more easily with a run-time parameter
parameters:
- name: rollback
type: string
default: ' '
variables:
- group: dev
- ${{ if ne(parameters.rollback, ' ') }}:
- name: rollback
value: ${{ parameters.rollback }}
The way this works in practice is:
the pipeline queue dialog automatically includes a 'rollback' text field:
if the developer types a value into the rollback parameter field,
that value is used to override the rollback variable
otherwise, the value from the variable group is used.
Note that you need to give the parameter a default value of a single space; otherwise the pipeline won't let you leave it empty.
Related
I would like to pass the variable group as a runtime parameter so that whenever I run the pipeline, it should allow me to provide the input variable group name, and based on the input value for the variable group during runtime my pipeline should proceed.
I want to achieve this when we click on the run button, then there's a variable section also. So, I want you to accept the variable group names from there.
Pipeline.yml:
stages:
- stage: VMBackupandValidate
displayName: 'VM Backup and Validate using RSV'
jobs:
- job: VMBackupValidate
displayName: 'Azure VM Backup'
steps:
- task: AzurePowerShell#5
inputs:
azureSubscription: $(azure_sc)
ScriptType: 'FilePath'
ScriptPath: 'pipelines/automation/scripts/vmbackup.ps1'
ScriptArguments: '-ResourceGroupName $(ResourceGroupName) -Storagetype $(Storagetype) -SourceVMname $(SourceVMname) -RSVaultname $(RSVaultname) -Location $(Location) -WorkLoadType $(WorkLoadType) -Policyname $(Policyname) -Verbose'
azurePowerShellVersion: 'LatestVersion'
pwsh: true
Based on comments communication with OP.
I suggest using a parameter with a default value. It will ask you for input if want other values, before you hit run then make a condition to select the right variable based on input.
Here is a minified sample of the pipeline:
parameters:
- name: environment
displayName: Deploy Environment
type: string
default: TEST
values:
- TEST
- PROD
trigger:
- 'none'
variables:
- name: environment
${{ if contains(parameters.environment, 'TEST') }}:
value: TEST
${{ if contains(parameters.environment, 'PROD') }}:
value: PROD
stages:
- stage: TEST
displayName: Build
condition: ${{ eq(variables.environment, 'TEST') }}
jobs:
- job:
pool:
vmImage: 'ubuntu-20.04'
steps:
- script: |
echo ${{ variables.environment}}
displayName: 'Print environment info'
You can extend the logic, or replace it with other values and consume it in code later. You can create multiple stages with conditions as well as shown.
Lets say you have two variable groups with names prod and test. You could use the below pipeline:
trigger:
- main
parameters:
- name: environment
displayName: Where to deploy?
type: string
default: test
values:
- prod
- test
pool:
vmImage: ubuntu-latest
variables:
- group: ${{parameters.environment}}
steps:
- script: |
echo $(ENV)
echo $(VERSION)
displayName: Step 1 - print version and environment
- script: pwd ENV ${{parameters.environment}}
displayName: Step 2 - print parameter
You should define ENV, VERSION values on both variable groups.
Your stage should stay as is. In your case you will delete the steps I provided and use only the first part of the pipeline
Adding a reference article.
https://blog.geralexgr.com/azure/deploy-between-different-environments-with-variable-groups-azure-devops?msclkid=002b01eab02f11ec8dffa95dc3a34094
I am trying to find a way to define a variable group at stage level and then access it in below jobs through a template? How would I go about doing this?
# Template file getchangedfilesandvariables.yaml
parameters:
- name: "previouscommitid"
type: string
steps:
- task: PowerShell#2
displayName: 'Get the changed files'
name: CommitIds
inputs:
targetType: 'filePath'
filePath: '$(Build.SourcesDirectory)\AzureDevOpsPipelines\Get-COChangedfiles.ps1'
arguments: >
-old_commit_id ${{ previouscommitid }}
- task: PowerShell#2
name: PassOutput
displayName: 'Getting Variables for Packaging'
inputs:
targetType: 'filepath'
filepath: '$(System.DefaultWorkingDirectory)\AzureDevOpsPipelines\Get-COADOvariables.ps1'
And below is my yaml file.
trigger: none
name: $(BuildID)
variables:
system.debug: true
CodeSigningCertThumbprint: "somethumbprint"
# Triggering builds on a branch itself.
${{ if startsWith(variables['Build.SourceBranch'], 'refs/heads/') }}:
branchName: $[ replace(variables['Build.SourceBranch'], 'refs/heads/', '') ]
# Triggering builds from a Pull Request.
${{ if startsWith(variables['Build.SourceBranch'], 'refs/pull/') }}:
branchName: $[ replace(variables['System.PullRequest.SourceBranch'], 'refs/heads/', '') ]
## it will create pipeline package and it will push it private or public feed artifacts
stages:
- stage: Stage1
variables:
- group: Cloudops
- name: oldcommitid
value: $[variables.lastcommitid]
jobs:
- job: IdentifyChangedFilesAndGetADOVariables
pool:
name: OnPrem
workspace:
clean: all # Ensure the agent's directories are wiped clean before building.
steps:
- powershell: |
[System.Version]$PlatformVersion = ((Get-Content "$(Build.SourcesDirectory)\AzureDevOpsPipelines\PlatformVersion.json") | ConvertFrom-Json).PlatformVersion
Write-Output "The repository's PlatformVersion is: $($PlatformVersion.ToString())"
$NewPackageVersion = New-Object -TypeName "System.Version" -ArgumentList #($PlatformVersion.Major, $PlatformVersion.Minor, $(Build.BuildId))
Write-Output "This run's package version is $($NewPackageVersion.ToString())"
echo "##vso[task.setvariable variable=NewPackageVersion]$($NewPackageVersion.ToString())"
echo "##vso[task.setvariable variable=commitidold;isOutput=true]$(oldcommitid)"
displayName: 'Define package version.'
name: commitidorpackageversion
errorActionPreference: stop
- template: getchangedfilesandvariables.yaml
parameters:
previouscommitid:
- $(commitidorpackageversion.commitidold)
# - $(oldcommitid)
I get the error at the second last line of the code that
/AzureDevOpsPipelines/azure-pipelines.yml (Line: 49, Col: 13): The 'previouscommitid' parameter is not a valid String.
I tried different combinations but I am still getting the errors.
Any ideas?
Thanks for your response. I already had the variable group setup in my library. I was just not able to use it.
The way I was able to achieve this I created another template file and supplied it to variables section under my stage. After doing this I was able to actually able to use the variables from my variable group in my successive jobs.
For more information you can review this doc : https://learn.microsoft.com/en-us/azure/devops/pipelines/library/variable-groups?view=azure-devops&tabs=yaml
stagevariables.yaml
variables:
- group: Cloudops
azure-pipelines.yml
stages:
- stage: Stage1
variables:
- template: stagevariables.yaml
jobs:
- job: CheckwhichfeedsAreAvailable
In YAML pipeline, you can't define a new variable group under the variables key.
Actually, we do not have the syntax can be available to create new variable group when running the YAML pipeline.
Under the variables key, you can:
Define new variables with the specified values.
Override the existing variables with new values.
Reference the variables from the existing variable groups and variable templates.
So, if you want to use a variable group with some variables in the pipeline, you should manually define the variable group on the Pipelines > Library page, then reference it in the pipeline.
I know that it is possible to conditionally set a variable in azure pipelines yml.
See https://learn.microsoft.com/en-us/azure/devops/pipelines/process/expressions?view=azure-devops#conditional-insertion
Is it also somehow possible to conditionally use a variable group?
Let's say if a pipeline run variable is set or has a certain value then the pipeline shall use a variable group. If not then the group shall not be used.
Thank you
Is it possible like here for instance:
trigger: none
pr: none
variables:
isProd: true
stages:
- stage: Test
displayName: Build and restore
variables:
- ${{ if eq(variables['isProd'], 'false') }}:
- group: QA
- ${{ if eq(variables['isProd'], 'true') }}:
- group: PROD
jobs:
- job: A
steps:
- bash: echo $(name)
I'm trying to learn how to use the new yaml configured pipeline system for Azure Devops, and I'm having a bit of trouble getting my head around the way the variables are supposed to work.
When I setup the pipeline, it created a file azure-pipelines.yml and committed this to the master branch.
By default, this file looks like so...
trigger:
- master
pool:
vmImage: 'windows-latest'
variables:
solution: '**/*.sln'
buildPlatform: 'Any CPU'
buildConfiguration: 'Release'
My project is setup with the following build configurations... "prod", "staging", "develop".
What I'm confused about, is where am I supposed to override these default variables for the actual pipelines?
I can modify the values directly in this file, but that's not really going to work. When I merge the changes back from "master" to "staging" etc, then presumably the pipelines for these lower environments will then be trying to build with "prod" configuration.
Surely there must be some way to configure variables independent of the source code.
There are 2 places where I can see an option to add Variables...
When I choose "Edit" for the pipeline, up in the top right, there is a "Variables" button next to run.
I can add variables there, but they don't appear to do anything. They are not applied when I run the pipeline.
Also, to make things more confusing, when I choose to "Run pipeline", there is also an option to define variables, but likewise, these don't seem to do anything. The build still just runs with the pre-defined values from the yaml file.
Agree with Shayki Abramczyk. This method could manually override the variable value on the UI interface.
I would like to share the method of automatically appending values to variables.
You could use Expressions to judge different situations(e.g. build branch). Then you could set the value for different situations.
Here is an example:
trigger:
- '*'
pool:
vmImage: 'windows-latest'
variables:
${{ if eq(variables['Build.SourceBranchName'], 'master') }}:
buildConfiguration: Prod
${{ if eq(variables['Build.SourceBranchName'], 'staging') }}:
buildConfiguration: Staging
steps:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
# Write your PowerShell commands here.
Write-Host $(buildConfiguration)
This sample code can select the corresponding value according to different trigger branch names. (master: Prod , staging: staging)
Hope this helps.
You can define the variables with Let users override this value when running this pipeline:
Use the variable in the build step with $(BuildConfiguration ).
When you run the build you can override the value:
May be you need not a variables but a parameters?
parameters:
- name: STAND_NAME
displayName: Select Stand to deploy
type: string
default: none
values:
- dev
- stage
- prod
variables:
- group: global-variables # use global variable from library
- name: STAND_NAME
value: ${{ parameters.STAND_NAME }}
- ${{ if eq(parameters['STAND_NAME'], 'prod') }}:
- name: variable_depends_on_stand
value: "prod_value" #
- ${{ if eq(parameters['STAND_NAME'], 'stage') }}:
- name: variable_depends_on_stand
value: "stage_value"
- ${{ if eq(parameters['STAND_NAME'], 'dev') }}:
- name: variable_depends_on_stand
value: "dev_value"
- name: SOME_OTHER_GLOBAL_VARIABLE
value: some_other_value
It would be display like this in pipeline:
screenshot of the pipeline WUI
Normal (non-template) jobs in Azure DevOps yaml support inter-job variable passing as follows:
jobs:
- job: A
steps:
- script: "echo ##vso[task.setvariable variable=skipsubsequent;isOutput=true]false"
name: printvar
- job: B
condition: and(succeeded(), ne(dependencies.A.outputs['printvar.skipsubsequent'], 'true'))
dependsOn: A
steps:
- script: echo hello from B
How do I do something similar in the following, given that templates don't support the dependsOn syntax? I need to get an output from the first template and pass it as 'environmentSlice' to the second template.
- stage: Deploy
displayName: Deploy stage
jobs:
- template: build-templates/get-environment-slice.yml#templates
parameters:
configFileLocation: 'config/config.json'
- template: build-templates/node-app-deploy.yml#templates
parameters:
# Build agent VM image name
vmImageName: $(Common.BuildVmImage)
environmentPrefix: 'Dev'
environmentSlice: '-$(dependencies.GetEnvironmentSlice.outputs['getEnvironmentSlice.environmentSlice'])'
The reason I want the separation between the two templates is the second one is a deployment template and I would like input from the first template in naming the environment in the second template. I.e. initial part of node-app-deploy.yml (2nd template) is:
jobs:
- deployment: Deploy
displayName: Deploy
# Because we use the environmentSlice to name the environment, we have to have it passed in rather than
# extracting it from the config file in steps below
environment: ${{ parameters.environmentPrefix }}${{ parameters.environmentSlice }}
Update:
The accepted solution does allow you to pass variables between separate templates, but won't work for my particular use case. I wanted to be able to name the 'environment' section of the 2nd template dynamically, i.e. environment: ${{ parameters.environmentPrefix }}${{ parameters.environmentSlice }}, but this can only be named statically since templates are compiled on pipeline startup.
The downside of the solution is that it introduces a hidden coupling between the templates. I would have preferred the calling pipeline to orchestrate the parameter passing between templates.
You can apply the depend on and dependency variable into templates.
See below sample:
To make sample more clear, here has 2 template files, one is azure-pipelines-1.yml, and another is azure-pipeline-1-copy.yml.
In azure-pipelines-1.yml, specify the environment value as output variable:
parameters:
environment: ''
jobs:
- job: preDeploy
variables:
EnvironmentName: preDeploy-${{ parameters.environment }}
steps:
- checkout: none
- pwsh: |
echo "##vso[task.setvariable variable=EnvironmentName;isOutput=true]$($env:ENVIRONMENTNAME)"
name: outputVars
And then, in azure-pipeline-1-copy.yml use dependency to get this output variable:
jobs:
- job: deployment
dependsOn: preDeploy
variables:
EnvironmentNameCopy: $[dependencies.preDeploy.outputs['outputVars.EnvironmentName']]
steps:
- checkout: none
- pwsh: |
Write-Host "$(EnvironmentNameCopy)"
name: outputVars
At last, in YAML pipeline, just need to pass the environment value
stages:
- stage: deployQA
jobs:
- template: azure-pipelines-1.yml
parameters:
environment: FromTemplate1
- template: azure-pipeline-1-copy.yml
Now, you can see the value get successfully in the second template job:
It is possible to avoid the dependency in the called template. However, as the OP says, the environment name cannot be created dynamically.
Here is an example of the "calling" template, which firstly calls another template (devops-variables.yml) that sets some environment variables that we wish to consume in a later template (devops-callee.yml):
stages:
- stage: 'Caller_Stage'
displayName: 'Caller Stage'
jobs:
- template: 'devops-variables.yml'
parameters:
InitialEnvironment: "Development"
- template: 'devops-callee.yml'
parameters:
SomeParameter: $[dependencies.Variables_Job.outputs['Variables_Job.Variables.SomeParameter']]
In the devops-variables.yml file, I have this:
"##vso[task.setvariable variable=SomeParameter;isOutput=true;]Wibble"
Then, in the "devops-callee.yml", I just consume it something like this:
parameters:
- name: SomeParameter
default: ''
jobs:
- deployment: 'Called_Job'
condition: succeeded()
displayName: 'Called Job'
environment: "Development"
pool:
vmImage: 'windows-2019'
dependsOn:
- Variables_Job
variables:
SomeParameter: ${{parameters.SomeParameter}}
strategy:
runOnce:
deploy:
steps:
- download: none
- task: AzureCLI#2
condition: succeeded()
displayName: 'An Echo Task'
inputs:
azureSubscription: "$(TheServiceConnection)"
scriptType: pscore
scriptLocation: inlineScript
inlineScript: |
echo "Before"
echo "$(SomeParameter)"
echo "After"
Output:
2021-04-10T09:22:29.6188535Z Before
2021-04-10T09:22:29.6196620Z Wibble
2021-04-10T09:22:29.6197124Z After
This way, the callee doesn't reference the caller. Unfortunately, setting the environment in the callee thus:
environment: "$(SomeParameter)"
doesn't work - you'll just get an environment with the literal characters '$(SomeParameter)'.