I am using the AzureRMWebAppDeployment#4 task to deploy a logic app using an Azure Devops yaml pipeline. I want to update values in both the host.json and appsettings.json files using json variable substitution.
According to the documentation, the JSONFiles parameter can accept "a newline-separated list of json files to substitute the variable values". How can I specify that?
I have tried the following:
- task: AzureRMWebAppDeployment#4
inputs:
...
JSONFiles: '**/parameters.json
**/host.json'
and also the following:
JSONFiles: '**/parameters.json|**/host.json'
There must be something simple I'm missing.
How do I specify a pattern which matches two or more files by name?
Using | should work.
From this answer:
The pipe symbol at the end of a line in YAML signifies that any indented text that follows should be interpreted as a multi-line scalar value. See the YAML spec.
So in you case it should be something like that:
- task: AzureRmWebAppDeployment#4
inputs:
...
JSONFiles: |
**/parameters.json
**/host.json
Related
I've tried several articles and threads from Stackoverflow but can't seem to get anywhere. I am trying to take a variable from a .py file which is called in a YAML step and set that variable globally to be used.
In my .py file i have
print(f'##vso[task.setvariable variable=AMLPipelineId;isOutput=true]{pipelineId}')
Then in my YAML pipeline step i have
- task: AzurePowerShell#5
displayName: 'Run AML Pipeline'
inputs:
azureSubscription: '$(azureSubscription)'
ScriptType: 'InlineScript'
name: AmlPipeline
azurePowerShellVersion: 'LatestVersion'
pwsh: true
Inline: |
$username = "$(ARM_CLIENT_ID)"
$password = "$(ARM_CLIENT_SECRET)"
$tenantId = "$(ARM_TENANT_ID)"
python $(Pipeline.Workspace)/AML_Pipeline/build_aml_pipeline.py --wsName $(wsName) --resourceGroup $(ResourceGroupName) --subscriptionId $(subId)
$MLPipelineId = $AmlPipeline.AMLPipelineId
But it seems like this variable is empty. I know there are other ways of using the "set variable" but this is my latest attempt i.e. something like print('##vso[task.setvariable variable=version;]%s' % (version))
My current approach i followed: https://learn.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch
You don't need isOutput=true - that's only needed for referencing variables between different jobs or stages.
"You cannot use the variable in the step that it is defined." - split that script into two steps: one that runs your .py file, second one that uses the newly defined variable.
I used print('##vso[task.setvariable variable=<Variable-in-Pipeline]+<output-variable>')
Variable-in-Pipeline // the given name should be used in Azure Devops pipeline and should be added to pipeline variables as an empty string
A very minimal example for everyone struggling with this. The documentation is kind of lacking on this for my taste. As #qbik said, dont set and use the variable in the same step, make it seperate steps.
set_variable.py
if __name__ == '__main__':
# set name of the variable
name = 'COLOR'
# set value of the variable
value = 'red'
# set variable
print(f'##vso[task.setvariable variable={name};]{value}')
azure-pipelines.yml
trigger:
- main
pool:
vmImage: ubuntu-latest
steps:
- task: UsePythonVersion#0
inputs:
versionSpec: '3.9'
displayName: 'Use Python 3.9'
# run the script to set the variable
- task: PythonScript#0
inputs:
scriptSource: filePath
scriptPath: set_variable.py
# now you can use the variable in the next step
- bash: echo my favorite color is: $(COLOR)
Now you can technically do all kinds of cool stuff in python, then set, and reference the variables in the following steps. In my case I have to extract specific package version numbers from a JSON/YAML file based on an id that is set earlier in the pipeline and parse the information as an args for a docker build. Hope that helps other people stumbling across this answer looking for a minimal working example :)
I have an Azure DevOps Yaml build pipeline. The yaml file calls a PowerShell script, like so:
- task: PowerShell#1
name: SetBuildNumber
displayName: 'Set Build Number'
inputs:
scriptName: '$(Build.SourcesDirectory)\BuildScripts\UpdateBuildNumber.ps1'
This powershell script is supposed to output the value of a variable. Here's what I have there:
Write-Output '##vso[task.setvariable variable=updatedCounter;isOutput=true]$(updatedCounter)'
&updatedCounter is a variable in that script that gets set to a number. I want the yaml file that called it to then be able to use that number in a parameter sent to another file. This is what I have for that:
- template: ${{ variables.buildtemplate }}
parameters:
BuildVersion: ${{ variables.BuildVersion }}$[SetBuildNumber.updatedCounter]
But what is getting passed to the template is the value of the BuildVersion variable concatenated with the string "$[SetBuildNumber.updatedCount]". So it ends up like this, for example, where the value of variables.BuildVersion is "1.2.3.":
"1.2.3.$[SetBuildNumber.updatedCounter]"
What am I doing wrong?
Output variables can be referenced by macro syntax: $(), so it should be:
BuildVersion: ${{ variables.BuildVersion }}$(SetBuildNumber.updatedCounter)
This will only work if you reference output variable from the same stage and same job. In that case, you could also remove isOutput=true and simply use $(updatedCounter).
If the reference takes place in a different stage or job, then the syntax is a bit more complex.
I have an Azure DevOps project with a couple of YAML build pipelines that share a common template for some of their tasks.
I now want to add a DownloadSecureFile task to that template, but I can't find a way to get it to work.
The following snippet results in an error when added to the template, but works fine in the parent pipeline definition (Assuming I also replace the ${{ xx }} syntax for the variable names with the $(xx) version):
- task: DownloadSecureFile#1
name: securezip
displayName: Download latest files
inputs:
secureFile: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
retryCount: 5
- task: ExtractFiles#1
displayName: Extract files
inputs:
archiveFilePatterns: ${{ variables.securezip.secureFilePath }}
destinationFolder: '${{ parameters.sourcesDir }}\secure\'
cleanDestinationFolder: true
The error occurs on the 'Extract File' step and is Input required: archiveFilePatterns, so it looks like it's just not finding the variable.
As a workaround, I could move the download task to the parent pipeline scripts and pass the file path as a parameter. However, that means duplicating the task, which seems like a bit of a hack.
Variables in dollar-double-curly-brackets are resolved at template expansion time. They are not the output of tasks.
Output variables from tasks are referenced by dollar-single-parentheses and they don't need to start with the word "variables."
So I believe the line you're looking for is like this, and it isn't affected by the template mechanism.
archiveFilePatterns: $(securezip.secureFilePath)
Is it possible in Azure Devops YAML pipelines to dynamically create additional steps based on some variable data (without creating our own plugin)
The thing is I see that I want to iterate through several directories, but I don't want to just lump it all in a single step since it makes it harder to scan through to find an error.
Is it possible in Azure Devops YAML pipelines to dynamically create
additional steps based on some variable data (without creating our own
plugin)
No, Yaml pipelines(azure-pipeline.yml) are under Version Control. So what you want (for your original title) is to dynamically commit changes to the azure-pipeline.yml file when executing the pipeline. That's not a recommended workflow.
1.Instead you can consider using Azure Devops Conditions to dynamically enable/disable the additional steps.
Use a template parameter as part of a condition
Use the output variable from a job in a condition in a subsequent job
Or Use some predefined variables:
condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/master'))
2.If you're not using Conditions, you can check conditional template as Simon suggests above.
Also, both #1 and #2 can work with new feature runtime parameters.
3.However, if the dynamic variable you mean comes from the result of components = result of ls -1 $(Pipeline.Workspace)/components command, above tips won't work for this situation. For this you can try something like this:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
# Write your PowerShell commands here.
# some logic to run `components = result of ls -1 $(Pipeline.Workspace)/components` and determine whether to set the WhetherToRun=true.
'Write-Host "##vso[task.setvariable variable=WhetherToRun]True"'
- task: CmdLine#2
inputs:
script: |
echo Hello world
condition: eq(variables['WhetherToRun'], 'True')
It is possible to include steps conditionally with an if statement.
I think the example of extending a template on the same page will give you a good indication of how to iterate through a list parameter and create / run a step based on each value.
I am writing azure pipelines and using variables from variable groups. As part of adding flexibility, we are trying to make most of the parts of pipeline configurable ( more variables). So, i was looking for a way to add default value to variable if it is not present in variable group.
The way we use is like below:
variables:
dockerId: $(docker_id)
imageName: $(DOCKER_IMAGE_NAME)
Is there option to specify default for the variable, if it doesn't find it from variable group. Something like below:
variables:
dockerId: $(docker_id:"defaultDockerId")
imageName: $(DOCKER_IMAGE_NAME:"defaultDockerImageName")
You cannot achieve this directly in azure pipeline. Azure pipeline doesnot have this feature currently and this syntax $(docker_id:"defaultDockerId") is not supported.
As workaround, you may have to write scripts in script tasks to assign the default value if the variables are not exist.
Please check out below python script:
The first python script task checks if the variable exists and set a default value for it, and define the variable using print("##vso[task.setvariable variable=variableName]value").
(Please note the variable variableTest value can only be referenced in the following tasks.)
- task: PythonScript#0
displayName: 'setDefaultValue'
inputs:
scriptSource: inline
script: |
import os
b = os.getenv("variableTest","default value for variableTest")
print("##vso[task.setvariable variable=variableTest]{b}".format(b=b))
- task: PythonScript#0
displayName: 'Run a Python script'
inputs:
scriptSource: inline
script: |
print("$(variableTest)")
Hope above helps!
The comment is correct, you cannot have defaults (unless you are using templates and parameters, which you might want to look at, but they are not exactly the thing you are after). You can either use an if condition if you know when the variable exists or not (in the yaml file) or use a script task like the comment suggests and calculate the value in the script task and emit it back to the pipeline.