I'm trying to create a YAML based pipeline that takes a parameter, then trigger the pipeline to run from a Azure DevOps REST API. I'm able to see the build gets queued, but the parameter was not overridden from my POST body.
My template my-template.yaml.
parameters:
- name: testParam
type: string
default: 'N/A'
steps:
- script: echo ${{ parameters.testParam }}
My pipeline yaml that extends the template.
trigger:
- master
extends:
template: my-template.yaml
Then I trigger this pipeline using the queue build REST API: https://dev.azure.com/{organization}/{project}/_apis/build/builds?api-version=5.1 with a POST body as below.
{
"parameters": "{\"testParam\": \"hello world\"}",
"definition": {
"id": 50642
},
"sourceBranch": "refs/heads/master"
}
So I'm expecting the pipeline execution will echo hello world instead of N/A. Unfortunately, I'm still seeing N/A in the pipeline results.
Anyone has idea of what happened? Am I miss anything?
I ran into the exact same problem - a pipeline taking runtime parameters that worked when run via the UI, but not via the Queue Build REST API.
I was able to solve this by using an undocumented API, the exact same one that the Az DevOps Pipelines UI calls when running a pipeline:
https://dev.azure.com/{organization}/{project}/_apis/pipelines/{pipelineId}/runs?api-version=5.1-preview
With the following POST body:
{
"stagesToSkip": [],
"resources": {
"repositories": {
"self": {
"refName": "refs/heads/master"
}
}
},
"templateParameters": {
"testParam": "hello world"
},
"variables": {}
}
Note that with this API, your runtime parameters are being submitted as actual JSON, not stringified JSON, and under the key templateParameters.
As well, don't forget to include the standard headers one might expect for this call:
Content-Type: application/json
Accept: application/json
AUTHORIZATION: bearer $SYSTEM_ACCESSTOKEN.
Using this approach, in the called pipeline, you will always be able to access the value of ${{ parameters.testParam }} whether the pipeline is called via REST API or manually in the UI.
While you're correct that the value is accessible as $(testParam) when executed via REST API, that variable is not populated when running the pipeline in the UI.
As such, I'd recommend using this undocumented API, since the called pipeline can use ${{ parameters.testParam }} without regard to how it's being called. Of course, it's (as of writing) undocumented, so.... ¯_(ツ)_/¯
As well, it should be noted that your pipeline must be formatted as #Josh Gust suggested:
my-template.yaml:
parameters:
- name: testParam
type: string
default: 'N/A'
steps:
- script: echo ${{ parameters.testParam }}
azure-pipelines.yaml:
parameters:
- name: testParam
type: string
default: 'N/A'
trigger:
- master
extends:
template: my-template.yaml
parameters:
testParam: ${{ parameters.testParam }}
Got the solution after spending 2 to 3 Hour:
https://dev.azure.com/{organization}/{project}/_apis/pipelines/2/runs?api-version=6.0-preview.1
Where 2= {pipelineId}
Header :
Authorization: Personal access token. Use any value for the user name and the token as the password.
Type: basic
Content-Type : application/json
Accept : application/json
Right Now I'm using: Postman for testing this API So sharing posting main screenshot:
In the Body part :
{"previewRun":false,"stagesToSkip": [],"resources": {"repositories": {"self": {"refName": "refs/heads/master"}}},"templateParameters": {"testParam": "rawat Rob" },"variables": {}}
previewRun :{If true, don't actually create a new run. Instead, return the final YAML document after parsing templates.}
It is working for me And having test around 5 to 7 time
Seems that it's an issue in Azure DevOps Rest API: https://developercommunity.visualstudio.com/content/problem/1000544/parameters-to-api-rest-build-queue-method.html
I encountered the same problem and noticed that the runtime parameter is introduced to the pipeline run as variable. Thus using $(MyParam) instead of ${{parameters.MyParam}} in the yaml fixes the problem.
Looks like the parameters are not necessary in this case, I merged the yaml into one like below.
# File: azure-pipelines.yml
trigger:
- master
steps:
- script: echo $(testParam)
Note the difference between $(testParam) and ${{ parameters.testParam }}.
Then I trigger it from REST API and it works totally fine.
You're not sending your parameter from your pipeline to the template.
Take a look at how the documentation says this should happen. I haven't tested but I think if you wire the parameter to the template properly, you'll get what you expect using the template.
Basically your template should look like this:
# File: simple-param.yml
parameters:
- name: yesNo # name of the parameter; required
type: boolean # data type of the parameter; required
default: false
steps:
- script: echo ${{ parameters.yesNo }}
And your pipeline should be thus:
# File: azure-pipelines.yml
trigger:
- master
extends:
template: simple-param.yml
parameters:
yesNo: false # set to a non-boolean value to have the build fail
Notice the parameters: yesNo: false
Also, the Runtime Parameters Documentation might suggest that you should define your pipeline parameters as explicit parameters.
The only thing you need to change is the body of the rest api request
It should be as given below
$body='{
"definition": { "id": "50642" },
"sourceBranch": "refs/heads/master",
"templateParameters": {
"testParam": "hello world"
},
"variables": {}
}'
Related
I've been trying to set up the most basic GitHub action that can be triggered via API, and I managed to trigger it, but now I'm having trouble passing down the "inputs" and using them in the jobs...
I tried reading the documentation and all, and it should work but I'm probably missing some syntax or something...
Here's the action code:
name: Test
on:
repository_dispatch:
inputs:
body:
default: 'testdefaultvalue'
description: 'Test desc'
required: true
jobs:
print_inputs:
runs-on: ubuntu-latest
steps:
- name: Print inputs
run: echo "The inputs are ${{ inputs.body }}"
And here's the body that I'm trying to send using POST which hits and triggers this action
{"event_type": "my_event", "client_payload": {"body": "Hello, world!"}}
I keep getting only the first part of the echo, like on this screenshot
I even tried just printing out the inputs body with the default value using a different kinds of syntaxes but nothing worked. Hopefully, this is not a duplicate and someone will help me and it'll be useful for someone in the future as well!
According to repository_dispatch, you need to refer to the complete event context to get the values.
So, this should work in your case:
echo ${{ github.event.client_payload.body }}
I have a yaml pipeline task that is creating a JSON string and assigning it to an output variable using Write-Host "##vso[task.setvariable syntax.
What I want is to use that JSON variable to dynamically create some other jobs/tasks.
Say the JSON contains a collection like this
[
{projectPath: "source\client\webui\webui.csproj", name: "testUI", type: "webUI"},
{projectPath: "source\microservice\test\test.csproj", name: 'testService', type: "microservice"}
]
I would like to then take that JSON and generate a pipeline that runs job(s) or even task(s) for each record.
This is VERY general pseudocode - what I need is the actual syntax, if this is even doable.
$each project
{
job: DoSomeWorkFor$(project.name)
steps:
template: 'do.build.yml'
parameters:
projectPath: $(project.path)
type: $(project.type)
}
I'll also point out that if the above is not a recommend approach, I am open to other options using Azure Devops with a monorepo.
I have Azure DevOps build pipeline yml that has parameters, variables defined and at the same time I have an external tool that kicks of the jobs in the yml including passing in some parameters. At first I thought the parameters in the yml and incoming parameters should be the same, so I lined up all the names to match. Only to find out that the incoming parameters through REST API is actually a variable!
parameters:
- name: machineList
type: string
default: any
then I have variable declared like so
variables:
testsettingsjson: ${{ parameters.machineList }}
From the REST API that get's passed in and when I query the request coming in, calling the get build data api, I see
"parameters" : "{ \"machineList\" : \"machine1\" }"
Now trying to use the variable it works in some places and not in others.
- script: |
echo "Output variable values"
echo "ML json : '$(testsettingsjson)'"
echo "ML json : '$(machineList)'"
echo "ML json variables : '${{ variables.machineList }}'"
the output I see for the above is :
Output variable values
ML json : any
ML json : machine1
ML json variables:
So $(machineList) works, I thought.
- task: PowerShell#2
displayName: 'Read ML input value'
inputs:
targetType: filePath
filePath: ./ReadMachineList.ps1
continueOnError: true
env:
TESTSETTINGSJSON: $(machineList)
When I read TESTSETTINGSJSON using $env:TESTSETTINGSJSON from the powershell script above, the value is "any"!
I am super confused. What am I doing wrong?
Only to find out that the incoming parameters through REST API is
actually a variable!
This is as expected. The parameters you specify in the request body of the rest api actually represents the variable. If you want to replace the runtime parameter, you should use templateParameters instead.
$json= #'
{
"templateParameters": {
"machineList": "machine1"
}
}
Solved it. To make it work, I first structured my solution working with 3 separate set of "variables", here's an example.
parameters:
- name: MachineList
type: string
default: any
variables:
${{ if ne(variables['machineList'], '') }}:
machinelist: $(machineList)
${{ if ne(parameters.MachineList, 'Any') }}:
machinelist: ${{ parameters.MachineList }}
MachineList is the input to the pipeline.
machineList is the input value from Power Automate (or REST API).
machinelist is the local variable in yml.
Now in the rest of the yml, I can use $(machinelist) switching between the values for the pipeline versus the REST API.
We looking to create a pipeline to update our multi-tenant azure environment. We need to perform some actions during the update per tenant. To accomplish this, we would like to create a job per tenant, so we can process tenants in parallel. To accomplish this, I want to use a runtime parameter to pass the tenants to update to my pipeline as follows:
parameters:
- name: tenants
type: object
the value of the tenants parameter might look like something like this:
- Name: "customer1"
Someotherproperty: "some value"
- Name: "customer2"
Someotherproperty: "some other value"
to generate the jobs, we do something like this:
stages:
- stage:
jobs:
- job: Update_Tenant
strategy:
matrix:
${{ each tenant in parameters.Tenants }}:
${{ tenant.tenantName }}:
name: ${{ tenant.tenantName }}
someproperty: ${{ tenant.otherProperty }}
maxParallel: 2
steps:
- checkout: none
- script: echo $(name).$(someproperty)
Now what we need, is some way to fill this tenants parameter. Now I tried a few solutions:
Ideally I would like to put a build stage before the Update_Tenants stage to call a REST api to get the tenants, and expand the tenants parameter when the Update_Tenants stage starts, but this is not supported AFAIK, since parameter expansion is done when the pipeline starts.
A less ideal but still workable option would have been to create a variable group yaml file containing the tenants, and include this variable group in my pipeline, and use the ${{ variables.Tenants }} syntax to reference them. However, for some reason, variables can only be strings.
The only solution I can currently think of, is to create a pipeline that calls a REST api to get the tenants to update, and then uses the azure devops api to queue the actual update process with the correct parameter value. But this feels like a bit of a clunky workaround to accomplish this.
Now my question is, are there any (better?) alternatives to accomplish what I want to do?
Maybe this can help. I was able to use external source (.txt file) to fill array variable in azure pipelines.
Working example
# Create a variable
- bash: |
arrVar=()
for images in `cat my_images.txt`;do
arrVar+=$images
arrVar+=","
done;
echo "##vso[task.setvariable variable=list_images]$arrVar"
# Use the variable
# "$(list_images)" is replaced by the contents of the `list_images` variable by Azure Pipelines
# before handing the body of the script to the shell.
- bash: |
echo my pipeline variable is $(list_images)
Sources (there is also example for matrix)
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch#set-a-job-scoped-variable-from-a-script
Other sources
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/runtime-parameters?view=azure-devops&tabs=script
To accomplish this, we would like to create a job per tenant, so we
can process tenants in parallel.
Apart from rolling deployment strategy, you can also check Strategies and Matrix.
You can try something like this unless you have to use Runtime parameters:
jobs:
- job: Update
strategy:
matrix:
tenant1:
Someotherproperty1: '1.1'
Someotherproperty2: '1.2'
tenant2:
Someotherproperty1: '2.1'
Someotherproperty2: '2.2'
tenant3:
Someotherproperty1: '3.1'
Someotherproperty2: '3.2'
maxParallel: 3
steps:
- checkout: none
- script: echo $(Someotherproperty1).$(Someotherproperty2)
displayName: 'Echo something'
I have a scenario where I need to have both:
runtime parameters, so that the pipeline can be triggered manually from the UI, where users triggering it can choose from a predefined set of options (defined in YAML)
variables, so that the pipeline can be invoked via REST APIs
Regarding runtime parameters, I was able to create the following sample pipeline:
parameters:
- name: image
displayName: Pool Image
type: string
default: ubuntu-latest
values:
- windows-latest
- ubuntu-latest
trigger: none
stages:
- stage: A
jobs:
- job: A
steps:
- pwsh: |
echo "This should be triggering against image: $MY_IMAGE_NAME"
env:
MY_IMAGE_NAME: ${{ parameters.image }}
When I run it, I can see the dropdown list where I can choose the image name and it is reflected in the output message of the PowerShell script.
Regarding variables, I have defined one called "image" here (notice the value is empty):
The idea now is to invoke the pipeline from REST APIs and have the image name replaced by the value coming from the variable:
{
"definition": {
"id": 1
},
"sourceBranch": "master",
"parameters": "{\"image\": \"windows-latest\" }"
}
In order to make the step print the value I'm passing here, I need to correct the environment variable in some way. I thought it would be sufficient to write something like:
env:
MY_IMAGE_NAME: ${{ coalesce(variables.image, parameters.image) }}
That's because I want to give the priority to the variables, then to parameters, so that in case none is specified, I always have a default value the pipeline can use.
However, this approach doesn't work, probably because we're dealing with different expansion times for variables, but I don't really know what I should be writing instead (if there is a viable option, of course).
What I also tried is:
env:
MY_IMAGE_NAME: ${{ coalesce($(image), parameters.image) }}
MY_IMAGE_NAME: ${{ coalesce('$(image)', parameters.image) }}
MY_IMAGE_NAME: $[ coalesce(variables.image, parameters.image) ]
MY_IMAGE_NAME: $[ coalesce($(image), parameters.image) ]
None of those are working, so I suspect this may not be feasible at all.
There is a workaround that I'm currently thinking of, which is to create two different pipelines so that those can be invoked independently, but while this is quite easy for me to accomplish, given I'm using a lot of templates, I don't find it the right way to proceed, so I'm open to any suggestion.
I tested and found you might need to define a variable and assign the parameter's value to it (eg. Mimage: ${{parameters.image}}). And define another variable(eg. Vimage) and assign $[coalesce(variables.image, variables.Vimage)] to it. Then refer to $(Vimage) in the env field of powershell task. Please check out below yaml.
parameters:
- name: image
displayName: Pool Image
type: string
default: ubuntu-latest
values:
- windows-latest
- ubuntu-latest
trigger: none
stages:
- stage: A
jobs:
- job: A
variables:
Mimage: ${{parameters.image}}
Vimage: $[coalesce(variables.image, variables.Mimage)]
steps:
- pwsh: |
echo "This should be triggering against image: $env:MY_IMAGE_NAME"
env:
MY_IMAGE_NAME: $(Vimage)
Env field of powershell task is usually for mapping secret variables. You can directly refer to $(Vimage) in the powershell script: echo "This should be triggering against image: $(Vimage).
Note: To queue a build via REST API with provided parameters, you need to check Let users override this value when running this pipeline to make the varilabe to be settable at queue time.
Update:
You can try passing the variables to the parameters of the template to make the parameters for template dynamic. Please check below simple yaml.
jobs:
- template: template.yaml
parameters:
MTimage: ${{parameters.image}}
VTimage: $(Vimage)
template.yaml:
parameters:
MTimage:
VTimage:
jobs:
- job: buildjob
steps:
- powershell: |
echo "${{parameters.VTimage}}"
echo "${{parameters.MTimage}}"