Pipeline variable is not getting evaluated correctly - azure-devops

I have defined a Pipeline variable using the UI interface for my pipeline:
And I am consuming that variable in my pipeline azure-pipelines.yml:
name: MyRepo.Deployment.$(date:yy)$(DayOfYear)$(rev:.r)
...
jobs:
- job:
steps:
...
- template: azure-pipelines.yml#Common-YAML
parameters:
...
enable_datasync_job: ${{ eq(variables.enable_datasync_job, 'true') }}
That variable then gets passed to a template that expects a boolean.
The issue is that when I run the pipeline and the azure-pipelines.yml#Common-YAML template gets expanded, the value of enable_datasync_job is always False
What am I missing here? Is it because both my pipeline and the template share the same variable/parameter name?

Assume that enable_datasync_job is defined in the UI as false
Option 1
To override the above variable that is defined in the UI, you can add this bash task to your pipeline
- bash: |
echo $(ENABLE_DATASYNC_JOB) # outputs as false
echo "##vso[task.setvariable variable=enable_datasync_job;]true" # override
- bash: |
echo $(ENABLE_DATASYNC_JOB) # outputs as true
Since your template accepts the parameter of type boolean, you would have something like the below
# template.yml
parameters:
- name: enable_datasync_job
type: boolean
steps:
- bash: |
echo $(ENABLE_DATASYNC_JOB)
And your main yml file
# azure-pipelines.yml
steps:
- bash: |
echo "##vso[task.setvariable variable=enable_datasync_job;]true" # override
- template: template.yml
parameters:
enable_datasync_job: $(enable_datasync_job) # fail
But the above azure-pipelines.yml will fail because of typecasting of string into boolean. Check this SO post for more details
/azure-pipelines.yml (Line: 11, Col: 28): The 'enable_datasync_job'
parameter value '$(enable_datasync_job)' is not a valid Boolean.
According to the above post, you can change the data type of parameter to string in your template.yml
parameters:
- name: enable_datasync_job
type: string
Final result of option 1
# template.yml
parameters:
- name: enable_datasync_job
type: string
steps:
- bash: |
echo $(ENABLE_DATASYNC_JOB) #outputs true
# azure-pipelines.yml
steps:
- bash: |
echo "##vso[task.setvariable variable=enable_datasync_job;]true" # override
- template: template.yml
parameters:
enable_datasync_job: $(enable_datasync_job)
Option 2
If you are looking to just pass the value true to your template without overriding the pipeline variable, you can reference the value that you passed like this
# template.yml
parameters:
- name: enable_datasync_job
type: boolean
steps:
- bash: |
echo ${{ parameters.enable_datasync_job }} # outputs true
echo $(ENABLE_DATASYNC_JOB) # outputs false
# azure-pipelines.yml
steps:
- template: template.yml
parameters:
enable_datasync_job: true
The reason why it was returning False for you is because you have used this expression while passing value to the parameter
${{ eq(variables.enable_datasync_job, 'true') }}
According to the documentation, the eq function
Evaluates True if parameters are equal
You tried to evaluate if the UI variable was equal to true instead of setting it
The expression that you used above will always return False because false(UI variable value) is not equal to true(set in the expression)

Related

How to pass a Pipeline variable as boolean to a template?

I am trying to pass a Pipeline variable (enable_datasync_job) that I have defined using the UI:
to a template that is used within my main pipeline azure-pipelines.yml:
name: MyRepo.Deployment.$(date:yy)$(DayOfYear)$(rev:.r)
...
jobs:
- job:
steps:
...
- template: azure-pipelines.yml#Common-YAML
parameters:
...
enable_datasync_job: $(enable_datasync_job)
The Common-YAML template defines that variable as boolean:
parameters:
...
- name: enable_datasync_job
type: boolean
default: false
When I try to run my main pipeline, it currently breaks as it completes that I am not passing a boolean value to my template
I know that all pipeline variables as of type string. How do I convert that string to a boolean so that my template accepts it?
If you define the parameter that is in the azure-pipelines.yml also as boolean you will get a checkbox (boolean) instead of inputbox (string):
You should be able to pass this along the Common-YAML.
Example code
The azure-pipelines.yml:
trigger:
- main
pool:
vmImage: ubuntu-latest
parameters:
- name: enable_datasync_job
type: boolean
default: false
extends:
template: boolean.yml
parameters:
enable_datasync_job: ${{ parameters.enable_datasync_job }}
The azure-pipelines.yml, without extends:
trigger:
- main
pool:
vmImage: ubuntu-latest
parameters:
- name: enable_datasync_job
type: boolean
default: false
steps:
- template: boolean.yml
parameters:
enable_datasync_job: ${{ parameters.enable_datasync_job }}
The boolean.yml (Common-YAML):
parameters:
- name: enable_datasync_job
type: boolean
default: false
steps:
- script: |
echo the value is ${{ parameters.enable_datasync_job }}
The result

Evaluate an expression at template expansion time

I have an azure pipeline template that takes parameters, and I'd like to set one of the parameters based on an expression:
- template: templates/mytemplate.yml
parameters:
TestBackCompat: eq('${{ parameters.CIBuildId }}', 'MasterLatestBuildId')
CreateNupkg: ${{ parameters.CreateNupkg }}
As written, it doesn't work, because the expression isn't evaluated until runtime.
Is there a way to evaluate the expression at compile-time? Simple variable replacement (e.g., the usage of CreateNuPkg in the script above) works OK.
From this official document:
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/templates?view=azure-devops#template-expressions
Template expressions can expand template parameters, and also
variables. You can use parameters to influence how a template is
expanded. The object works like the variables object in an expression.
How the variable object works:
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch#runtime-expression-syntax
So the usage in your situation should be like this:
give_parameters.yml
trigger:
- none
parameters:
- name: CIBuildId
type: string
default: 'MasterLatestBuildId'
extends:
template: using_parameters.yml
parameters:
TestBackCompat: ${{ eq(parameters.CIBuildId, 'MasterLatestBuildId')}}
using_parameters.yml
parameters:
- name: CIBuildId
type: string
default: 'x'
- name: TestBackCompat
type: string
default: 'x'
variables:
- name: test1
value: ${{ parameters.TestBackCompat}}
- name: test2
value: ${{ parameters.CIBuildId}}
steps:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
Write-Host $(test1)
Write-Host $(test2)
Write-Host ${{ parameters.TestBackCompat}}
Result:

Azure Devops pipelines: Condition inside loop not working?

In trying to run a script on a loop only if a variable is true but the script never runs :(
It works if i set it as "condition:" to the script but i would like for the task to not even be printed if not to run, therefore would like to use the ${{if...}}
(tried with boolean and string)
parameters:
charts: []
outputChartsDebugInfo: false
steps:
- ${{ each chart in parameters.charts }}:
- script: |
echo "TEST ${{parameters.outputChartsDebugInfo}}" #this outputs true
echo "Deployment values:"
echo "chart name - ${{chart.Name}}"
#...
displayName: Deploy ${{chart.Name}}
- ${{ if eq(parameters.outputChartsDebugInfo, true) }}:
- script: |
echo "DEBUG:"
#...
displayName: Show ${{chart.Name}} debug info
Should this be working?
The first thing that you need to change is how you set your parameters. Then, you should loop over an object that contains some values. Your charts object is empty in your code.
parameters:
- name: charts
type: object
default: [one,two]
- name: outputChartsDebugInfo
default: false
steps:
- ${{ each chart in parameters.charts }}:
- script: |
echo "TEST ${{parameters.outputChartsDebugInfo}}" #this outputs true
echo "Deployment values:"
echo "chart name - ${{chart}}"
displayName: Deploy ${{chart}}
- ${{ if eq(parameters.outputChartsDebugInfo, true ) }}:
- script: |
echo "DEBUG:"
displayName: Show ${{chart }} debug info
Just it case its handy for anyone, the if inside the loop works fine, the parameters outputChartsDebugInfo was being read from a variable, but the syntax used $(var), expands only at runtime before a task executes, and the necessary is that it expands at compile time, so the fix in this case was change the syntax on the variable in the calling script:
steps:
- template: charts-pipeline-template.yml
parameters:
outputChartsDebugInfo: ${{variables.outputChartsDebugInfo}}
charts:
- Name: xxx
- Name: yyy
instead of
steps:
- template: charts-pipeline-template.yml
parameters:
outputChartsDebugInfo: $(outputChartsDebugInfo)
charts:
- Name: xxx
- Name: yyy

Azure Yaml DevOps variable handling

So I have a YAML file where I am trying to do something like the following:
# top of the file, declare global variables
variables:
- name: MyName
value: 'apple'
...
parameters:
- name: SwitchName
type: boolean
default: false
...
stages:
- stage: Build
displayName: 'Build'
...
jobs:
variables:
- ${{ if eq(parameters.SwitchName, true) }}:
- name: MyName
value: '$(MyName)_pie'
...
steps:
- task: PowerShell#1
inputs:
scriptType: inlineScript
inlineScript: |
Write-Output $(MyName)
...
#end of script
The goal is to control the pipeline to print apple in default run and apple_pie when the user selects that parameter in the UI to be true.
I understand that parameters are compile-time and variables are run-time. I know that to overwrite a global variable you can create a variable at a job level and change it as yo see fit. Sadly, the way how template works do not let me redeclare that variable properly and I end up with $(MyName)_pie. For some reason, Yaml fails to see that inside that template there is a runtime variable that is needed to be defined.
What are my options here to achieve desired behavior?
Anything missing in my understanding here?
You can change to use the ${{ variables.MyName }} to call the variable in root level.
For example:
variables:
- name: MyName
value: 'apple'
parameters:
- name: SwitchName
type: boolean
default: false
stages:
- stage: Build
displayName: 'Build'
jobs:
- job: A
variables:
- ${{ if eq(parameters.SwitchName, true) }}:
- name: MyName
value: '${{ variables.MyName }}_pie'
steps:
- script: "echo $(MyName)"
Result:
Since you used an if expression to reassign value, the variable will be assigned at compile time.
Refer to this doc: Runtime expression syntax
You can use the template expression format: ${{ variables.var }}

Assigning AWS Cloudformation output variables in Azure DevOps Pipelines

My team is using Azure Devops pipelines to deploy Cloudformation stacks to AWS. We have common Cloudformation templates that we are reusing (e.g. a single Lambda.yml Cloudformation template that is reused to deploy multiple Lambda functions).
Along with this I am setting up a step function to orchestrate logical flows. The step function steps need to reference the ARNs of the various parts of the infrastructure (for this example, focus on the lambda functions).
Since the Lambda.yml is reused, the output variables are overwritten each time a task reuses the template to deploy a new lambda. I'd like to capture the output after each task in a variable or parameter in Azure Devops. I've looked at numerous examples and the documentation on variables, and I have been unable to get it to work. The relevant parts of the Cloudformation template and the deployment script are listed here (YAML format).
Lambda.yml (Cloudformation template)
AWSTemplateFormatVersion: '2010-09-09'
Description: Template for Lambda function.
Parameters:
FunctionName:
Description: The name of the Lambda function
Type: String
Resources:
LambdaFunction:
Type: AWS::Lambda::Function
Properties:
FunctionName: !Ref FunctionName
.....
Outputs:
LambdaFunctionName:
Description: Lambda function name
Value: !Ref LambdaFunction
LambdaFunctionARN:
Description: Lambda ARN
Value:
Fn::GetAtt:
- LambdaFunction
- Arn
Deploy.yml (Azure Devops pipeline deployment)
# Deploy.yml is a series of deployment tasks called from a job/stage defined in another.yml file
# LambdaFunctionNumber1
- task: AmazonWebServices.aws-vsts-tools.CloudFormationCreateOrUpdateStack.CloudFormationCreateOrUpdateStack#1
displayName: 'Stack: LambdaFunctionNumber1'
inputs:
templateSource: s3
s3BucketName: ${{ parameters.azdoS3ArtifactBucket }}
s3ObjectKey: '${{ parameters.azdoS3ArtifactPrefix }}/cfn/Lambda.yml'
templateParametersSource: inline
templateParameters: |
-
ParameterKey: FunctionName
ParameterValue: LambdaFunctionNumber1
....
captureStackOutputs: asVariables
# I would like to capture Cloudformation output variables here for use in other tasks
# variables lambdaNumberOneFunctionName and lambdaNumberOneFunctionArn are defined as variables by the job triggering these deployment tasks, and initialized to empty strings
- bash:
echo "##vso[task.setvariable variable=lambdaNumbertOneFunctionName;isOutput=true]$LAMBDAFUNCTIONNAME"
echo "##vso[task.setvariable variable=lambdaNumberOneFunctionArn;isOutput=true]$LAMBDAFUNCTIONARN"
# LambdaFunctionNumber2
- task: AmazonWebServices.aws-vsts-tools.CloudFormationCreateOrUpdateStack.CloudFormationCreateOrUpdateStack#1
displayName: 'Stack: LambdaFunctionNumber2'
inputs:
templateSource: s3
s3BucketName: ${{ parameters.azdoS3ArtifactBucket }}
s3ObjectKey: '${{ parameters.azdoS3ArtifactPrefix }}/cfn/Lambda.yml'
templateParametersSource: inline
templateParameters: |
-
ParameterKey: FunctionName
ParameterValue: LambdaFunctionNumber2
....
captureStackOutputs: asVariables
# At this point the $(LambdaFunctionARN) output variable is set from Cloudformation of LambdaFunctionNumber2.
# The ARN from LambdaFunctionNumber1 is lost unless I capture it somehow.
# I need multiple ARNs to provide as input to the Step Function that gets deployed here
- task: AmazonWebServices.aws-vsts-tools.CloudFormationCreateOrUpdateStack.CloudFormationCreateOrUpdateStack#1
displayName: 'Stack: Step Function'
inputs:
stackName:step-function
templateSource: s3
s3BucketName: ${{ parameters.azdoS3ArtifactBucket }}
s3ObjectKey: '${{ parameters.azdoS3ArtifactPrefix }}/cfn/StepFunction.yml'
....
templateParametersSource: inline
templateParameters: |
-
ParameterKey: StateMachineName
ParameterValue: orchestration-step-function
-
# This doesn't work
ParameterKey: State1LambdaArn
ParameterValue: $[lambdaNumberOneFunctionArn]
-
# This works but only allows capturing the most recent output variable
ParameterKey: State2LambdaArn
ParameterValue: $(LambdaFunctionARN)
You may try to name step where you set a variable and then use name of step as part of variable name. You can even define variable name as template parameter so each time you will get an unique name. In code it may look like this:
template.yaml
parameters:
- name: name
type: string
default: 'lambdaNumbertOneFunctionName'
- name: arn
type: string
default: 'lambdaNumberOneFunctionArn'
- name: stepName
type: string
default: 'StepOne'
- name: value
type: string
default: 'testValue'
steps:
- bash: |
echo "##vso[task.setvariable variable=${{ parameters.name }};isOutput=true]${{ parameters.value }}"
echo "##vso[task.setvariable variable=${{ parameters.arn }};isOutput=true]${{ parameters.value }}"
echo "##vso[task.setvariable variable=Test;isOutput=true]${{ parameters.value }}"
echo "##vso[task.setvariable variable=Test2;isOutput=true]Yesy2"
name: ${{ parameters.stepName }}
- script: |
echo $(${{ parameters.stepName }}.${{ parameters.name }})
echo $(${{ parameters.stepName }}.${{ parameters.arn }})
echo $(${{ parameters.stepName }}.Test)
echo $(${{ parameters.stepName }}.Test2)
and main file
jobs:
- job: myJob
timeoutInMinutes: 10
pool:
vmImage: 'ubuntu-16.04'
steps:
- template: template.yaml
parameters:
name: 'lambdaNumbertOneFunctionName'
arn: 'lambdaNumberOneFunctionArn'
value: 'value-1'
stepName: 'StepOne'
- template: template.yaml
parameters:
name: 'lambdaNumberTwoFunctionName'
arn: 'lambdaNumberTwoFunctionArn'
value: 'value-2'
stepName: 'StepTwo'
- script: |
echo 'lambdaNumbertOneFunctionName - $(StepOne.lambdaNumbertOneFunctionName)'
echo 'lambdaNumberOneFunctionArn - $(StepOne.lambdaNumberOneFunctionArn)'
echo 'lambdaNumberTwoFunctionName - $(StepTwo.lambdaNumberTwoFunctionName)'
echo 'lambdaNumberTwoFunctionArn - $(StepTwo.lambdaNumberTwoFunctionArn)'
For that I got:
lambdaNumbertOneFunctionName - value-1
lambdaNumberOneFunctionArn - value-1
lambdaNumberTwoFunctionName - value-2
lambdaNumberTwoFunctionArn - value-2
I hope I understood you and your issue and it solves it :)