Azure DevOps REST api - Run pipeline with variables - azure-devops

I have a pipeline on Azure Devops that I'm trying to run programatically/headless using the REST api: https://learn.microsoft.com/en-us/rest/api/azure/devops/pipelines/runs/run%20pipeline?view=azure-devops-rest-6.0
So far so good, I can auth and start a run. I would like to pass data to this pipeline which the docs suggests is possible using variables in the request body. My request body:
{
"variables": {
"HELLO_WORLD": {
"isSecret": false,
"value": "HelloWorldValue"
}
}
}
My pipeline YAML looks like this:
trigger: none
pr: none
pool:
vmImage: 'ubuntu-latest'
steps:
- task: Bash#3
inputs:
targetType: 'inline'
script: |
KEY=$(HELLO_WORLD)
echo "Hello world key: " $KEY
This however gives me an error that "HELLO_WORLD: command not found".
I have tried adding a "HELLO_WORLD" variable to the pipeline and enabled the "Let users override this value when running this pipeline"-setting. This results in the HELLO_WORLD variable no longer being unknown, but instead its stuck on its initial value and not set when i trigger a run with the REST api
How do you pass variables to a pipeline using the REST api? It is important that the variable value is set only for a specific run/build
I found another API to run a build, but it seems like you cannot use Personal Access Token auth with it, like you can with the pipeline api - only OAuth2 - https://learn.microsoft.com/en-us/rest/api/azure/devops/build/builds/queue?view=azure-devops-rest-6.0

You can do it with both the Runs API and Build Queue API, both work with Personal Access Tokens. For which one is the better/preferred, see this question: Difference between Azure Devops Builds - Queue vs run pipeline REST APIs, but in short the Runs API will be the more future proof option
Option 1: Runs API
POST https://dev.azure.com/{{organization}}/{{project}}/_apis/pipelines/{{PipelineId}}/runs?api-version=6.0-preview.1
Your body will be of type application/json (HTTP header Content-Type is set to application/json) and similar to the below, just replace resources.repositories.self.refName with the appropriate value
{
"resources": {
"repositories": {
"self": {
"refName": "refs/heads/main"
}
}
},
"variables": {
"HELLO_WORLD": {
"isSecret": false,
"value": "HelloWorldValue"
}
}
}
Option 2: Build API
POST https://dev.azure.com/{{organization}}/{{project}}/_apis/build/builds?api-version=6.0
Your body will be of type application/json (HTTP header Content-Type is set to application/json), something similar to below, just replace definition.id and sourcebranch with appropriate values. Please also note the "stringified" content of the parameter section (it should be a string representation of a json map)
{
"parameters": "{\"HELLO_WORLD\":\"HelloWorldValue\"}",
"definition": {
"id": 1
},
"sourceBranch": "refs/heads/main"
}

Here's the way I solved it....
The REST call:
POST https://dev.azure.com/<myOrg>/<myProject>/_apis/pipelines/17/runs?api-version=6.0-preview.1
 
The body of the request:
{
    "resources": {
        "repositories": {
            "self": {
                "refName": "refs/heads/main"
            }
        }
    },
    "templateParameters": {
        "A_Parameter": "And now for something completely different."
    }
}
Note: I added an authorization header with basic auth containing a username (any name will do) and password (your PAT token value). Also added a Content-Type application/json header.
 
Here's the entire yaml pipeline I used:
 
parameters:
- name: A_Parameter
  displayName: A parameter
  default: noValue
  type: string
 
trigger:
- none
 
pool:
  vmImage: ubuntu-latest
 
steps:
 
- script: |
    echo '1 - using dollar sign parens, p dot A_Parameter is now: ' $(parameters.A_Parameter)
    echo '2 - using dollar sign double curly braces, p dot A_Parameter is now::' ${{ parameters.A_Parameter }} '::'
    echo '3 - using dollar sign and only the var name: ' $(A_Parameter)
  displayName: 'Run a multi-line script'
 
 
And here's the output from the pipeline log. Note that only the second way properly displayed the value.  
 
1 - using dollar sign parens, p dot A_Parameter is now: 
2 - using dollar sign double curly braces, p dot A_Parameter is now:: And now for something completely different. :: 
3 - using dollar sign and only the var name:

Related

Deploy only certain resources into Azure using DevOps Pipeline

I have a main.bicep template with multiple resources that I deploy to Azure via my DevOps pipeline. My template is currently configured to either deploy all modules to Azure or none. However, I would like to customize this in my pipeline so that only certain resources show up in my resource group so that it acts like a selection catalog.
However, I need advice and help on how to solve the problem as I can’t find the right approach. Attached you can see a snippet from my bicep template below.
– main.bicep–
param deployAppInsights bool = true param deployNetworkWatcher bool = true param deploySentinel bool = true
Example for a module
// Deploy App Insights resource
module appInsights 'AzureMonitor/AppInsights/appInsights-temp.bicep' = if(deployAppInsights) { name: 'appInsightsDeployment' params: { appName: appInsightsName regionId: regionId // tagsArray: tagsArray requestSource: requestSource workspaceResourceId: workspace.outputs.resourceWorkspaceIdOutput } }
This snippet is in my pipeline - yml
(I would like to shorten the parameters block by using override parameters.)
parameters:
- name: appInsights
type: boolean
default: True
values:
- True
- False
inputs: overrideParameters: -deployAppInsights false -deployNetworkWatcher true -deploySentinel true
Currently, my override parameters are hardcoded, but I’d like to keep it in a general way, like for example:
overrideParameters: -deployAppInsights ${{appInsights}} -deployNetworkWatcher ${{networkWatcher}} -deploySentinel ${{sentinel}}
However, this does not unfortunately work. Could anyone please help me in this regard, how I can solve this or how the variables need to look like?
Thank you very much!
Best regards
Jennifer
As far as I understand, the syntax you are using is wrong in the below code
overrideParameters: -deployAppInsights ${{appInsights}} -deployNetworkWatcher ${{networkWatcher}} -deploySentinel ${{sentinel}}
Since you have already defined parameters section in the Yml file. The correct syntax is
overrideParameters: -deployAppInsights ${{parameters.appInsights}} -deployNetworkWatcher ${{parameters.networkWatcher}} -deploySentinel ${{parameters.sentinel}}
A sample code snippet for your reference
parameters:
- name: var1
type: string
- name: var2
type: string
- name: bool1
type: bool
...
omitted
...
- task: AzureResourceManagerTemplateDeployment#3
inputs:
deploymentScope: 'Resource Group'
azureResourceManagerConnection: ''
subscriptionId: ''
action: 'Create Or Update Resource Group'
resourceGroupName: rg-${{parameters.var1}}-${{parameters.var2}}
location: ''
templateLocation: 'Linked artifact'
csmFile: $(System.DefaultWorkingDirectory)/arm.json
csmParametersFile: $(System.DefaultWorkingDirectory)/arm.parameters.json
deploymentMode: 'Incremental'
overrideParameters:
-string1 ${{parameters.var1}}
-string2 ${{parameters.var2}}
-flag ${{parameters.bool1}}
For more, please refer to the link from Microsoft. Azure Devops Template
Edit: After the first comment
Please update your 'appInsights' parameter as follows
parameters:
- name: appInsights
type: boolean
default: true

Unable to use | character in AzureFunctionApp appSettings:

I am setting a load of appSettings in my AzureFunctionApp#1 deployment task - but whenever I try to put each on a new line using the | character I get the error:
##[error]Error: Failed to update App service '{{functionName}}' application settings. Error: BadRequest - Parameter name cannot be empty. (CODE: 400)
The output above that seems to show that it has indeed built the JSON with an empty parameter name. But I don't know why? I've tested with the values on separate lines, and still in a single line, so neither of these work:
appSettings: |
'-Values:Setting1 "$(SettingVal1)"
-Values:Setting2 "$(SettingVal2)"'
appSettings: |
'-Value:Setting1 "$(SettingVal1)" -Values:Setting2 "$(SettingVal2)"'
But this does:
appSettings: '-Value:Setting1 "$(SettingVal1)" -Values:Setting2 "$(SettingVal2)"'
I've also tried without the ' - but that made no difference either.
As per your feedback - Converting my comment as an answer, also tried locally in my system.
Multi-line json input works for setting the multiple values in the app settings as this is the closest way.
appSettings: |
[
{
"name": "APPINSIGHTS_INSTRUMENTATIONKEY",
"value": "$(Key)",
"slotSetting": false
},
{
"name": "MYSQL_DATABASE_NAME",
"value": "$(DB_Name)",
"slotSetting": false
}
]
Multiline JSON doesn't work with the AzureFunctionApp#1 task's appSettings parameter (for some reason).
If you try to use the multiline JSON appSettings with the AzureFunctionApp#1 task, you will get an error: BadRequest - Parameter name cannot be empty. (CODE: 400)
To use the multiline JSON appSettings, you need to use a separate AzureAppServiceSettings#1 task - as mentioned in this document.
I can confirm this works after the AzureFunctionApp#1 task. So my pipeline now has:
steps:
...
- task: AzureFunctionApp#1
displayName: Deploy the Function App
condition: succeeded()
inputs:
azureSubscription: "${{parameters.AppAzureSubscription}}"
appName: "${{parameters.functionAppName}}"
package: "$(Pipeline.Workspace)/drop/$(Build.BuildId).zip"
- task: AzureAppServiceSettings#1
displayName: Update app settings
inputs:
azureSubscription: "${{parameters.AppAzureSubscription}}"
appName: "${{parameters.functionAppName}}"
appSettings: |
[
{
"name": "Values:DbConnectionString",
"value": "$(DbConnectionString)",
"slotSetting": false
},
...
]

Azure devops pass build parameters through Rest API

I'm trying to pass a parameters through the build rest API using jira, but it doesn't override the parameter.
Pipeline:
parameters:
- name: "Testplan"
type: string
default: "NoPlanDefined"
stage: Test
jobs:
- job: Testing_And_Transfer
- task: PowerShell#2
displayName: "Testing API Call"
inputs:
targetType: 'filepath'
filePath: './script/Jira.ps1'
arguments:
-Jira_id ${{ parameters.Testplan }}
Jira.ps1 content:
Param(
[string]$Jira_id = "no ID"
)
#-----------------------Jira API--------------------------
echo 'This is a test \n ID: '
echo $Jira_id
My rest command is setup like so:
URL: https://dev.azure.com/{My corp}/MyHello/_apis/build/builds?api-version=6.0
Body:
{
"definition": {
"id": 1
},
"parameters": "{ \"Testplan\":\"Postman\" }"
}
When using the trigger, the ps1 return NoPlanDefined as expected.
When using a manual trigger and changing the parameter, the parameter
get changed as expected.
When trying to change the parameter through
the Rest api, Testplan is empty instead of Postman.
I'm I doing something wrong with the REST API?
That's because those are not parameters, despite the name used in the REST call. They are run-time variables, which behave differently and are available at a different scope than parameters.
There is a different API that allows you to specify templateParameters: https://learn.microsoft.com/en-us/rest/api/azure/devops/pipelines/runs/run-pipeline?view=azure-devops-rest-6.1
If you are familiar with PowerShell you can use the AzurePipelinesPS module and the command below to pass parameters to pipelines when invoking them.
$id = '2' # your pipeline id
$templateParameters = #{
Testplan = 'myTestPlan' # your parameter and value
}
Invoke-APPipeline -Instance https://dev.azure.com -Collection 'yourCollectionName' -Project 'yourProjectName' -ApiVersion '6.0-preview.1' -PipelineId $id -TemplateParameters $templateParameters
The module supports "sessions" to limit the number of required parameters. See the module's github page on how to create a session.

AZP: Is there a best practice to be able to "namespace" script tasks in yaml templates for usage of variables?

In Azure Pipelines: my main problem is, if I create a yml template and have some logic inside that template in a script task where I want to set a variable, i need the
name: "pseudonamespace" to reference that variable further down in that template via
$(pseudonamespace.variablename)
An example, where the script part does nothing overtly useful, but should demonstrate my problem:
mytemplate.yml:
parameters:
- name: isWindowsOnTarget
type: boolean
default: true
steps:
- script: |
if [ "${{lower(parameters.isWindowsOnTarget)}}" == "true" ]; then
delimiter="\\"
else
delimiter="/"
fi
echo "##vso[task.setvariable variable=myCoolVariable;isOutput=true]$delimiter"
name: MyFakeNameSpace
...
- task: SomeTask#0
inputs:
myInput: $(MyFakeNameSpace.myCoolVariable)
This codeblock works; but only, if, in a job, I only instanciate it once:
- template: mytemplate.yml#templates
parameters:
isWindowsOnTarget: true
If I would need that template twice, differently parameterized, I get the error that the name of the script block needs to be unique.
Is there any useful possibility I'm not currently thinking about other than to have an extra parameter for the template that I could basically just call "UniqueNamespace"?
There is no much space to move. Your task needs a unique name as later as you mention for output parameters it works like a namespace. So the best and the only way you have is to provide another parameter which would be task name.
parameters:
- name: isWindowsOnTarget
type: boolean
default: true
- name: taskName
type: string
steps:
- script: |
if [ "${{lower(parameters.isWindowsOnTarget)}}" == "true" ]; then
delimiter="\\"
else
delimiter="/"
fi
echo "##vso[task.setvariable variable=myCoolVariable;isOutput=true]$delimiter"
name: ${{ parameters.taskName }}
...
- task: SomeTask#0
inputs:
myInput: $(MyFakeNameSpace.myCoolVariable)
and then:
- template: mytemplate.yml#templates
parameters:
isWindowsOnTarget: true
taskName: MyFakeNameSpace
- template: mytemplate.yml#templates
parameters:
isWindowsOnTarget: true
taskName: MyFakeNameSpace2
In fact when you do not provide a name Azure DevOps assign a unique name. However, in this way you don't know the name till runtime.

Publish Nunit Test Results in Post Always Section

I'm trying to run a pipeline that does some Pester Testing and publish the NUnit results.
New tests were introduced and for whatever the reason, Jenkins no longer publishes the test results and errors out immediately after the powershell script. Hence, it doesn't get to the nunit publish piece. I receive this:
ERROR: script returned exit code 128
Finished: FAILURE
I've been trying to include the publish in the always section of the post section of the Jenkinsfile, however, I'm running into problems on how to make that NUnit test file available.
I've tried establishing an agent and unstash the file (even though it probably won't stash if the powershell script cancels the whole pipeline). When I use agent I get the following exception:
java.lang.NoSuchMethodError: No such DSL method 'agent' found among steps
Here is the Jenkinsfile:
pipeline {
agent none
environment {
svcpath = 'D:\\svc\\'
unitTestFile = 'UnitTests.xml'
}
stages {
stage ('Checkout and Stash') {
agent {label 'Agent1'}
steps {
stash name: 'Modules', includes: 'Modules/*/**'
stash name: 'Tests', includes: 'Tests/*/**'
}
}
stage ('Unit Tests') {
agent {label 'Agent1'}
steps {
dir(svcpath + 'Modules\\'){deleteDir()}
dir(svcpath + 'Tests\\'){deleteDir()}
dir(svcpath){
unstash name: 'Modules'
unstash name: 'Tests'
}
dir(svcpath + 'Tests\\'){
powershell """
\$requiredCoverageThreshold = 0.90
\$modules = Get-ChildItem ../Modules/ -File -Recurse -Include *.psm1
\$result = Invoke-Pester -CodeCoverage \$modules -PassThru -OutputFile ${unitTestFile} -OutputFormat NUnitXml
\$codeCoverage = \$result.CodeCoverage.NumberOfCommandsExecuted / \$result.CodeCoverage.NumberOfCommandsAnalyzed
Write-Output \$codeCoverage
if (\$codeCoverage -lt \$requiredCoverageThreshold) {
Write-Output "Build failed: required code coverage threshold of \$(\$requiredCoverageThreshold * 100)% not met. Current coverage: \$(\$codeCoverage * 100)%."
exit 1
} else {
write-output "Required code coverage threshold of \$(\$requiredCoverageThreshold * 100)% met. Current coverage: \$(\$codeCoverage * 100)%."
}
"""
stash name: 'TestResults', includes: unitTestFile
nunit testResultsPattern: unitTestFile
}
}
post {
always {
echo 'This will always run'
agent {label 'Agent1'}
unstash name: 'TestResults'
nunit testResultsPattern: unitTestFile
}
success {
echo 'This will run only if successful'
}
failure {
echo 'This will run only if failed'
}
unstable {
echo 'This will run only if the run was marked as unstable'
}
changed {
echo 'This will run only if the state of the Pipeline has changed'
echo 'For example, if the Pipeline was previously failing but is now successful'
}
}
}
Any and all input is welcome! Thanks!
The exception you are getting is due to Jenkins' strict pipeline DSL. Documentation of allowable uses of agent are here.
Currently agent {...} is not allowed to be used in the post section. Maybe this will change in the future. If you require the whole job to run on the node that services label 'Agent1' the only way to currently do that is to
Put agent {label 'Agent1'} immediately under pipeline { to make it global
Remove all instances of agent {label 'Agent1'} in each stage
Remove the agent {label 'Agent1'} from the post section.
The post section acts more like traditional scripted DSL than the pipeline declarative DSL. So you have to use node() instead of agent.
I believe I've had this same question myself, and this SO post has the answer and some good context.
This Jenkins issue isn't exactly the same thing but shows the node syntax in the post stage.