How to modify Azure DevOps release definition variable from a release task? - azure-devops

What is the easiest way to get key rotation to work for azure storage accounts from a AzureDevOps relase task? The current plan is to re-generate the old key after release to invalidate it, and have a fresh key that can be used on next deployment. But to get that to work it seems like I at least need to store the name of the key to use in a release variable.
I had a look at he logging tasks (https://github.com/Microsoft/azure-pipelines-tasks/blob/master/docs/authoring/commands.md), but that only changes the value in the current release and does not modify the release definition.

You can use the REST API (Definitions - Update) to update the value of the release definition variable from a release task.
Go to the Agent Phase and select Allow Scripts to Access OAuth Token. See Use the OAuth token to access the REST API
Grant Project Collection Build Service (xxx) account the edit
release pipeline permission. (Select the release pipeline --> ... --> Security --> Edit release definition set to Allow)
Add a PowerShell task in your release pipeline
Run inline script: (Update the value of variable v1030 in below sample)
$url = "$($env:SYSTEM_TEAMFOUNDATIONSERVERURI)$env:SYSTEM_TEAMPROJECTID/_apis/Release/definitions/$($env:RELEASE_DEFINITIONID)?api-version=5.0-preview.3"
Write-Host "URL: $url"
$pipeline = Invoke-RestMethod -Uri $url -Headers #{
Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"
}
Write-Host "Pipeline = $($pipeline | ConvertTo-Json -Depth 100)"
# Update an existing variable named v1030 to its new value 1035
$pipeline.variables.v1030.value = "1035"
####****************** update the modified object **************************
$json = #($pipeline) | ConvertTo-Json -Depth 99
$updatedef = Invoke-RestMethod -Uri $url -Method Put -Body $json -ContentType "application/json" -Headers #{Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"}
write-host "=========================================================="
Write-host "The value of Varialbe 'v1030' is updated to" $updatedef.variables.v1030.value
write-host "=========================================================="

Here is a much cleaner and better solution that also allows for multiple builds being triggered simultaneously.
https://tsuyoshiushio.medium.com/how-to-pass-variables-with-pipeline-trigger-in-azure-pipeline-5771c5f18f91
Essentially your triggering build produces artifacts that your triggered build reads and turns into variables.
Still not at all great, but better than nothing and better than REST calls setting static global variable groups.

The other answer above talks about how to update Release pipelines.
If you would like to update a Build Pipeline's variables, here is how you do that:
Edit build pipeline
Go to the Agent Phase and select Allow Scripts to Access OAuth Token. See Use the OAuth token to access the REST API
Go to Manage Security -> Users -> Select Project Collection Build Service (YOUR TEAM NAME HERE)
Change "Edit Build Definitions" to Allow
Now add a powershell stage - 2.x - inline script called Update variables.
Script inline contents:
$api_version='5.0-preview.6'
$url = "$($env:SYSTEM_TEAMFOUNDATIONSERVERURI)$env:SYSTEM_TEAMPROJECTID/_apis/build/definitions/$(System.DefinitionId)?api-version=${api_version}"
Write-Host "URL: $url"
$pipeline = Invoke-RestMethod -Uri $url -Headers #{
Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"
}
Write-Host "Pipeline = $($pipeline | ConvertTo-Json -Depth 100)"
# Update variables as desired here:
$pipeline.variables.mavenBuildVersionPatch.value = "2401"
####****************** update the modified object **************************
$json = #($pipeline) | ConvertTo-Json -Depth 99
$updatedef = Invoke-RestMethod -Uri $url -Method Put -Body $json -ContentType "application/json" -Headers #{Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"}
write-host "=========================================================="
Write-host "The value of Variable mavenBuildVersionPatch is updated to" $updatedef.variables.mavenBuildVersionPatch.value
write-host "=========================================================="
Take note of the API version in this script is 5.0-preview.6.
If your version of Azure Devops is newer, you may need to update this in the future.
Save build pipeline.
Now when you run job, after the job completes this powershell stage, this variable will be set.
Important: If you want to update a variable then make the updated variable for other pipeline stages, then you do it with a powershell stage with the following inline script:
$mavenBuildVersionPatch = [int]"$(mavenBuildVersionPatch)"
$mavenBuildVersionPatch = $mavenBuildVersionPatch + 1
Write-Host "##vso[task.setvariable variable=mavenBuildVersionPatch;]$mavenBuildVersionPatch"
This example would take our existing patch number and increment it by 1.
This does not save the variable at the end of the job, you still need to do that with another powershell script if desired.

Related

How to update Azure build definition agent from vs2017-win2016 to windows-2019 programmatically

I am trying to programatically update all my build pipeline agents from vs2017-win2016 to windows-2019. I have access to AZ CLI, PowerShell and REST API but I haven't found the magic incantation to make it work. Specifically I would like to update the Azure Pipeline agent for a specific project and build definition name.
How to update Azure build definition agent from vs2017-win2016 to
windows-2019 programmatically
We could use the REST API Definitions - Get to get all the info about this definition in the body, then we could update the body and use the Definitions - Update to update the value of the release definition variable from a build pipeline:
PUT https://dev.azure.com/{organization}/{project}/_apis/build/definitions/{definitionId}?api-version=5.0
Following is my test inline powershell scripts:
$url = "https://dev.azure.com/{organization}/{project}/_apis/build/definitions/57?api-version=5.0"
Write-Host "URL: $url"
$pipeline = Invoke-RestMethod -Uri $url -Headers #{
Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"
}
Write-Host "Pipeline = $($pipeline | ConvertTo-Json -Depth 100)"
# Update an existing build definition named and build definition agent to its new value 2
$pipeline.process.target.agentSpecification.identifier= "windows-2019"
$pipeline.name= "UpdateVariabletest"
####****************** update the modified object **************************
$json = #($pipeline) | ConvertTo-Json -Depth 99
$updatedef = Invoke-RestMethod -Uri $url -Method Put -Body $json -ContentType "application/json" -Headers #{Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"}
write-host "=========================================================="
Write-host "The value of build definition named and build definition agent is updated to"
$updatedef.process.target.agentSpecification.identifier
$updatedef.name
You could check the similar thread for some more details.

How can I read the PR tag inside a pipeline triggered by another pipeline?

I had my first question answered here Is it possible to read the PR tag on a pipeline task? but my scenario is a little bit different. I need to read the PR tag from a pipeline that was triggered by another pipeline.
PR triggers the CI which checks if everything's ok for merge. If it is, the CI triggers the CD which will in turn read the PR tag.
PR -> CI -> CD (access the tag here)
I have a PowerShell task named Get PR tag with the following script (courtesy of Lance):
$url = "$($env:SYSTEM_TEAMFOUNDATIONSERVERURI)$env:SYSTEM_TEAMPROJECTID/_apis/git/repositories/$($env:BUILD_REPOSITORY_NAME)/pullRequests/$($env:SYSTEM_PULLREQUEST_PULLREQUESTID)/labels?api-version=5.1-preview.1"
$response = Invoke-RestMethod -Uri $url -Method Get -Headers #{
Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"
}
Write-Host "##vso[task.setvariable variable=PullRequestTag;isOutput=true]$($response.value.name)"
But I keep getting "The request is invalid.":
========================== Starting Command Output ===========================
/usr/bin/pwsh -NoLogo -NoProfile -NonInteractive -Command . '/home/vsts/work/_temp/74b14931-e33a-4389-b19f-3db7faa53e8d.ps1'
Invoke-RestMethod: /home/vsts/work/_temp/74b14931-e33a-4389-b19f-3db7faa53e8d.ps1:3
Line |
3 | $response = Invoke-RestMethod -Uri $url -Method Get -Headers #{
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| {"count":1,"value":{"Message":"The request is invalid."}}
##[error]PowerShell exited with code '1'.
Finishing: Get PR tag
My agent job is set to use the OAuth token:
Update
In release pipeline, the variable name is not the same as the variable name in build, we need update the url info in the script, we can also check the release pipeline variable in the Initialize job log.
Steps:
a. Configure branch policy and add the policy Build Validation-> add build pipeline A
b. Create release->select the build A as the Source type->Enable the feature Pull request trigger->open Pre-deployment conditions and enable the option Pull request deployment
c. Open the release->enable the feature Allow scripts to access the OAuth token (Click Agent Job Name=>Additional options) add task powershell and enter the script below
$url = "$($env:SYSTEM_TASKDEFINITIONSURI)$env:BUILD_PROJECTID/_apis/git/repositories/$($env:BUILD_REPOSITORY_NAME)/pullRequests/$($env:BUILD_PULLREQUEST_ID)/labels?api-version=5.1-preview.1"
$response = Invoke-RestMethod -Uri $url -Method Get -Headers #{
Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"
}
Write-Host "##vso[task.setvariable variable=PullRequestTag;isOutput=true]$($response.value.name)"
d. Configure the Reference name as PS and add task cmd to output the tags.
CMD script:
echo $(PS.PullRequestTag)
e. Create pull request and add tags
Result:
Update2
The pull request triggers the CI build pipeline(power shell), after the build pipeline is completed, another build pipeline(power shell test) will be triggered.
b. Open the build pipeline power shell test and add a new variable PullRequestID and grant test Build Service (xxx) account the Edit build pipeline permission. (open the build pipeline(power shell test)--> ... --> Security --> Edit build pipeline set to Allow)
c. enable the feature Allow scripts to access the OAuth token (Click Agent Job Name=>Additional options) add task powershell(Get the tag value) and enter the script below. click the powershell task->Output Variables->enter PS->add a task cmd and use the code echo $(PS.PullRequestTag) to output the tag value
$url = "$($env:SYSTEM_TEAMFOUNDATIONSERVERURI)$env:SYSTEM_TEAMPROJECTID/_apis/git/repositories/$($env:BUILD_REPOSITORY_NAME)/pullRequests/$(PullRequestID)/labels?api-version=5.1-preview.1"
$response = Invoke-RestMethod -Uri $url -Method Get -Headers #{
Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"
}
Write-Host "##vso[task.setvariable variable=PullRequestTag;isOutput=true]$($response.value.name)"
d. Open build pipeline power shell, enable the feature Allow scripts to access the OAuth token (Click Agent Job Name=>Additional options) add task powershell and enter the script below to update the pipeline(power shell test) variable PullRequestID value.
$url = "$($env:SYSTEM_TEAMFOUNDATIONSERVERURI)$env:SYSTEM_TEAMPROJECTID/_apis/build/definitions/55?api-version=5.1"
Write-Host "URL: $url"
$pipeline = Invoke-RestMethod -Uri $url -Headers #{
Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"
}
Write-Host "Pipeline = $($pipeline | ConvertTo-Json -Depth 100)"
# Update an existing variable named PullRequestID to its new value pull request ID
$pipeline.variables.PullRequestID.value= $($env:SYSTEM_PULLREQUEST_PULLREQUESTID)
####****************** update the modified object **************************
$json = #($pipeline) | ConvertTo-Json -Depth 99
$updatedef = Invoke-RestMethod -Uri $url -Method Put -Body $json -ContentType "application/json" -Headers #{Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"}
write-host "=========================================================="
Write-host "The value of Varialbe 'PullRequestID ' is updated to" $updatedef.variables.PullRequestID.value
write-host "=========================================================="

How to read environment variables in azure devops pipeline using powershell?

I created a PowerShell job and used the below code to set the environment variable in the azure pipeline using Powershell.
[Environment]::SetEnvironmentVariable("key", "value")
I can print the value using the $env:key in the same job itself.
But when I tried to display the value using $env:key in the next job nothing is printed. How to use the above environment variable through out the azure pipeline. Is there any other way to set and read custom environment variables.
you pretty much have to either use library variable groups (or sets, dont remember the name) or you have to use a specific way to share variables across jobs:
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch#use-output-variables-from-tasks
According to this, using outputs in a different job is not supported in Classic UI Format.
As workarounds in this scenario, you can share variables via Pipeline Variables(share variables across jobs in same pipeline) or Variable Groups(share variables across pipelines that use same Variable Group, it also works across jobs).
Since you only want to share variables across jobs in same pipeline, pipeline variable is enough for you.
1.You can set a key variable in pipeline variables:
2.Add one Powershell Inline task with content below in your first job:
$url = "$($env:SYSTEM_TEAMFOUNDATIONCOLLECTIONURI)$env:SYSTEM_TEAMPROJECTID/_apis/build/definitions/$($env:SYSTEM_DEFINITIONID)?api-version=5.0"
Write-Host "URL: $url"
$pipeline = Invoke-RestMethod -Uri $url -Headers #{
Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"
}
Write-Host "Pipeline = $($pipeline | ConvertTo-Json -Depth 100)"
# Update an existing variable to its new value
$pipeline.variables.key.value = "value"
####****************** update the modified object **************************
$json = #($pipeline) | ConvertTo-Json -Depth 99
$updatedef = Invoke-RestMethod -Uri $url -Method Put -Body $json -ContentType "application/json" -Headers #{Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"}
write-host "=========================================================="
Write-host "The value of Varialbe key is updated to" $updatedef.variables.key.value
write-host "=========================================================="
3.Run the pipeline we can find the value of key variable is successfully updated:
So you can run the ps script in first job to update the value of key variable, then all next jobs can access the updated variable easily.
Note:
For the script itself, you only need to change lines $pipeline.variables.key.value = "value"(necessary) and Write-host "The value of Varialbe key is updated to" $updatedef.variables.key.value(optional).
If I want to set the variable named MyTest to value MyValue, the lines should be $pipeline.variables.MyTest.value = "MyValue" and Write-host "The value of Varialbe MyTest is updated to" $updatedef.variables.MyTest.value.
To make sure the ps task in one job can access OAuth Token, we should Allow Scripts to Access OAuth Token. Click the agent job name and check the box:
To enable the pipeline has the permission to update pipeline variable (edit build pipeline), go pipeline security to set the Edit build pipeline allow for user xxx(ProjectName) build service.
Hope all above helps :)

Store result of Azure CLI task into release variable

I have a release pipeline with multiple jobs.
On the first agent job there is an Azure CLI task that retrieves the keys of a storage account on azure.
The command it executes is :
az storage account keys list --account-name "$(diagnosticsStorageAccountName)" --resource-group "$(resourceGroup)"
What I want to do is store the result of this command and utilize it in a task that is running under a deployment group job.
I've already looked into these resource :
Set Output Variable in Azure CLI task on VSTS
How to modify Azure DevOps release definition variable from a release task?
I've tried the first one but I didn't get it working.
I didn't bothered with the second because it seems way to hacky.
Is there any way do achieve this that isn't hacky ?
The output values didn't get stored properly.
The output of az storage account keys list --account-name "$(diagnosticsStorageAccountName)" --resource-group "$(resourceGroup)"is spread over multiple lines when you use the following syntax:
echo "##vso[task.setvariable variable=testvar;]%myvar%"
So one of the problems was that only the first line of the JSON array was being stored into the variable.
I solved this problem in the following way:
keys=`az storage account keys list --account-name "$(diagnosticsStorageAccountName)" --resource-group "$(resourceGroup)"`
taskvariable="##vso[task.setvariable variable=_[tempVariable];]"
echo $taskvariable$keys
According to documentation echo "##vso[task.setvariable variable=testvar output=true;]%myvar% should make the variable available for the whole release. Unfortunately I had no luck with this.
I overcame this using a Powershell task under the same Agent Job (Ubuntu 16.0) :
$url = "$($env:SYSTEM_TEAMFOUNDATIONSERVERURI)$env:SYSTEM_TEAMPROJECTID/_apis/Release/definitions/$($env:RELEASE_DEFINITIONID)?api-version=5.1"
Write-Host "URL: $url"
$pipeline = Invoke-RestMethod -Uri $url -Headers #{
Authorization = "Bearer $(System.AccessToken)"
}
#Parsing JSON string from previous task
$keys = $(#"
$(_tempVariable)
"# | ConvertFrom-Json)
# Assignment of variable value
$pipeline.variables.[variableName].value = $keys[0].value
####****************** update the modified object **************************
$json = #($pipeline) | ConvertTo-Json -Depth 99
$updatedef = Invoke-RestMethod -Uri $url -Method Put -Body $json -ContentType "application/json" -Headers #{Authorization = "Bearer $(System.AccessToken)"}
write-host "=========================================================="
Write-host "The value of Variable '[variableName]' is updated to" $updatedef.variables.[variableName].value
write-host "=========================================================="
Please note that in order to get this working there are a couple of things you need to do.
First you need to allow access to the OAuth token on the Agent Job.
On top of that you need to give the "Project Collection Build Service".
Click on security on the release and click on the "Project Collection Build Service" user.
Change the values for the "Edit release" and "Manage release" to allow and save the changes.

How to Increase/Update Variable Group value using Azure Devops Build Definition?

Am working on Azure Devops CI&CD. Here, my release name must be in Version number using tags. Am getting this with the help of Variable Groups, by adding tags and value to it. Here am getting the tags value as constant like a static for every release like 1.1,1.2,1.3 etc.
Now am trying to increase/update my tag value dynamically for every new release triggered after completion of my Build Definition successfully which looks like 1.1,1.2,2.1,2.2,3.1,3.2 etc.It is possible with the help of statically by variable group, but manually we need to update it.
Is it possible to Increase/Update the tags value in Variable Group with the Build Definition tasks or other process.If possible, please suggest me to "How to done this?"
You can overwrite/update the value of the variables by using the logging command to set the variables again in Azure Devops Build pipleline:
Write-Host "##vso[task.setvariable variable=testvar;]testvalue"
To increase the value dynamically, you need to use the token $(Rev:.r). You can custom the variables based on the $(Build.BuildNumber) or $(Release.ReleaseName)as they will increase the value dynamically...
Just reference this thread to custom the variables:https://github.com/MicrosoftDocs/vsts-docs/issues/666#issuecomment-386769445
UPDATE:
If you just want to update the value of the variables which defined in a specific Variable Group, then you can call REST API in build pipeline to achieve that:
PUT https://{account}.visualstudio.com/{ProjectName or ID}/_apis/distributedtask/variablegroups/{Variable Group ID}?api-version=5.0-preview.1
Content-Type: application/json
Request Body:
{"id":2,"type":"Vsts","name":"VG0926","variables":{"TEST0926":{"isSecret":false,"value":"0930"}}}
UPDATE2:
You can write a PowerShell script to call the REST API, then add a PowerShell task to run the script in your build pipeline: (Use the OAuth token to access the REST API)
Below sample for your reference:
$url = "$($env:SYSTEM_TEAMFOUNDATIONCOLLECTIONURI)$env:SYSTEM_TEAMPROJECTID/_apis/distributedtask/variablegroups/{Variable Group ID}?api-version=5.0-preview.1"
Write-Host $url
function CreateJsonBody
{
$value = #"
{"id":2,"type":"Vsts","name":"VG0926","variables":{"TEST0926":{"isSecret":false,"value":"0930"}}}
"#
return $value
}
$json = CreateJsonBody
$pipeline = Invoke-RestMethod -Uri $url -Method Put -Body $json -ContentType "application/json" -Headers #{
Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"
}
Write-Host "New Variable Value:" $pipeline.variables.TEST0926.value
UPDATE3:
Well, tested again, below scripts works for me as well. You can try it, just replace the parameters accordingly:
# Base64-encodes the Personal Access Token (PAT) appropriately
$base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f "","PAT here")))
$url = "https://dev.azure.com/xxx/Test0924/_apis/distributedtask/variablegroups/1?api-version=5.0-preview.1"
$json = '{"id":1,"type":"Vsts","name":"VG0928","variables":{"TEST0928":{"isSecret":false,"value":"0931"}}}'
$pipeline = Invoke-RestMethod -Uri $url -Method Put -Body $json -ContentType "application/json" -Headers #{Authorization=("Basic {0}" -f $base64AuthInfo)}
Write-Host "New Variable Value:" $pipeline.variables.TEST0928.value
You can simply update any number of variables in an Azure Devops variable group using its built-in az pipelines variable-group variable update command. You can use this command in a script task in the pipeline definition as shown below.
ps: Replace all the upper case values with corresponding values except the SYSTEM_ACCESSTOKEN.
variables:
- group: VARIABLE_GROUP_NAME
jobs:
- job: UpdateVarGroup
steps:
- script: |
newValue="This is the updated value"
echo $SYSTEM_ACCESSTOKEN | az devops login
az pipelines variable-group variable update --group-id $(group_id) \
--name NAME_OF_THE_VARIABLE \
--value "${newValue}" \
--org https://dev.azure.com/YOUR_ORGANIZATION_NAME \
--project AZURE_DEVOPS_PROJECT
displayName: 'Update variable inside a variable group'
env:
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
In order for this example to work, you need to have a variable inside your variable group called group_id. The value of that needs to be set to the group id of the variable group, which can be obtained by simply looking at the url of your variable group. (The group id is the value for variableGroupId in the url of your browser when you are inside the variable group)
System.AccessToken is required for az devops login
I used this task to update the value of my variables inside my group.
Shared variable updater (preview)
Dont forget to set those settings :
Requires 'Allow scripts to access the OAuth token' in agent job additional options
Set administrator role to 'Project Collection Build Service' in the variable group.
In case of using a YAML pipeline
When using a YAML pipeline, the OAuth token is automatically added (no need for step 1 above), but requires a bit of work to make accessible for the powershell script. Use the guidance here to be able to use the token.
If you want to update a variable group's value, use the REST.API methods.
Rather than constructing the PUT request body manually, use a GET first to get the original JSON, update only what you need, then replay it as a PUT. I used to keep the variable group id also as a variable in that group to avoid hard-coding.
variables:
- group: MyVarGroup
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
$url = "$($env:SYSTEM_TEAMFOUNDATIONCOLLECTIONURI)$env:SYSTEM_TEAMPROJECTID/_apis/distributedtask/variablegroups/$(VariableGroupId)?api-version=6.0-preview.1"
$header = #{
"Authorization" = "Bearer $(System.AccessToken)"
}
$def = Invoke-RestMethod -Uri $url -Headers $header
$def.variables.MyTestVar.value = "NewValue"
$body = #($def) | ConvertTo-Json -Depth 100 -Compress
$def = Invoke-RestMethod -Method 'Put' -Uri $url -ContentType 'application/json' -Headers $header -Body $body
You can overwrite variables using the REST API with a PowerShell task without needing to create a PAT.
You'll need to first going into the agent job and set "Allow scripts to access OAuth token".
You'll need too go to your variable group and add the 'Project Collection Build Service' as an administrator.
Now you can call the Rest API using the OAuth Bearer token. Code for reference:
$id = <variable group id>
# This is using some environment variables provided by the pipeline to build the URL
$url = ("$($env:SYSTEM_TEAMFOUNDATIONCOLLECTIONURI)$env:SYSTEM_TEAMPROJECTID/_apis/distributedtask/variablegroups/{0}?api-version=5.0-preview" -f $id)
# You might find it useful to us a GET method to grab the variable group, update it and then convert it to this json string rather than doing this here
$json = '{"id":$id,"type":"Vsts","name":"<Variable Group Name>","<Variable Name":{"ThisIsMyVariable":{"isSecret":false,"value":"20"}}}'
$pipeline = Invoke-RestMethod -Uri $url -Method Put -Body $json -ContentType "application/json" -Headers #{Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"}
I made this task to manage variable groups from Pipelines:
ManageVariableGroupTask
Using this you can Create/Update/Delete variable groups and do the same operations on variables contained in them.
Hope that's helpfull!