How can I read the PR tag inside a pipeline triggered by another pipeline? - azure-devops

I had my first question answered here Is it possible to read the PR tag on a pipeline task? but my scenario is a little bit different. I need to read the PR tag from a pipeline that was triggered by another pipeline.
PR triggers the CI which checks if everything's ok for merge. If it is, the CI triggers the CD which will in turn read the PR tag.
PR -> CI -> CD (access the tag here)
I have a PowerShell task named Get PR tag with the following script (courtesy of Lance):
$url = "$($env:SYSTEM_TEAMFOUNDATIONSERVERURI)$env:SYSTEM_TEAMPROJECTID/_apis/git/repositories/$($env:BUILD_REPOSITORY_NAME)/pullRequests/$($env:SYSTEM_PULLREQUEST_PULLREQUESTID)/labels?api-version=5.1-preview.1"
$response = Invoke-RestMethod -Uri $url -Method Get -Headers #{
Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"
}
Write-Host "##vso[task.setvariable variable=PullRequestTag;isOutput=true]$($response.value.name)"
But I keep getting "The request is invalid.":
========================== Starting Command Output ===========================
/usr/bin/pwsh -NoLogo -NoProfile -NonInteractive -Command . '/home/vsts/work/_temp/74b14931-e33a-4389-b19f-3db7faa53e8d.ps1'
Invoke-RestMethod: /home/vsts/work/_temp/74b14931-e33a-4389-b19f-3db7faa53e8d.ps1:3
Line |
3 | $response = Invoke-RestMethod -Uri $url -Method Get -Headers #{
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| {"count":1,"value":{"Message":"The request is invalid."}}
##[error]PowerShell exited with code '1'.
Finishing: Get PR tag
My agent job is set to use the OAuth token:

Update
In release pipeline, the variable name is not the same as the variable name in build, we need update the url info in the script, we can also check the release pipeline variable in the Initialize job log.
Steps:
a. Configure branch policy and add the policy Build Validation-> add build pipeline A
b. Create release->select the build A as the Source type->Enable the feature Pull request trigger->open Pre-deployment conditions and enable the option Pull request deployment
c. Open the release->enable the feature Allow scripts to access the OAuth token (Click Agent Job Name=>Additional options) add task powershell and enter the script below
$url = "$($env:SYSTEM_TASKDEFINITIONSURI)$env:BUILD_PROJECTID/_apis/git/repositories/$($env:BUILD_REPOSITORY_NAME)/pullRequests/$($env:BUILD_PULLREQUEST_ID)/labels?api-version=5.1-preview.1"
$response = Invoke-RestMethod -Uri $url -Method Get -Headers #{
Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"
}
Write-Host "##vso[task.setvariable variable=PullRequestTag;isOutput=true]$($response.value.name)"
d. Configure the Reference name as PS and add task cmd to output the tags.
CMD script:
echo $(PS.PullRequestTag)
e. Create pull request and add tags
Result:
Update2
The pull request triggers the CI build pipeline(power shell), after the build pipeline is completed, another build pipeline(power shell test) will be triggered.
b. Open the build pipeline power shell test and add a new variable PullRequestID and grant test Build Service (xxx) account the Edit build pipeline permission. (open the build pipeline(power shell test)--> ... --> Security --> Edit build pipeline set to Allow)
c. enable the feature Allow scripts to access the OAuth token (Click Agent Job Name=>Additional options) add task powershell(Get the tag value) and enter the script below. click the powershell task->Output Variables->enter PS->add a task cmd and use the code echo $(PS.PullRequestTag) to output the tag value
$url = "$($env:SYSTEM_TEAMFOUNDATIONSERVERURI)$env:SYSTEM_TEAMPROJECTID/_apis/git/repositories/$($env:BUILD_REPOSITORY_NAME)/pullRequests/$(PullRequestID)/labels?api-version=5.1-preview.1"
$response = Invoke-RestMethod -Uri $url -Method Get -Headers #{
Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"
}
Write-Host "##vso[task.setvariable variable=PullRequestTag;isOutput=true]$($response.value.name)"
d. Open build pipeline power shell, enable the feature Allow scripts to access the OAuth token (Click Agent Job Name=>Additional options) add task powershell and enter the script below to update the pipeline(power shell test) variable PullRequestID value.
$url = "$($env:SYSTEM_TEAMFOUNDATIONSERVERURI)$env:SYSTEM_TEAMPROJECTID/_apis/build/definitions/55?api-version=5.1"
Write-Host "URL: $url"
$pipeline = Invoke-RestMethod -Uri $url -Headers #{
Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"
}
Write-Host "Pipeline = $($pipeline | ConvertTo-Json -Depth 100)"
# Update an existing variable named PullRequestID to its new value pull request ID
$pipeline.variables.PullRequestID.value= $($env:SYSTEM_PULLREQUEST_PULLREQUESTID)
####****************** update the modified object **************************
$json = #($pipeline) | ConvertTo-Json -Depth 99
$updatedef = Invoke-RestMethod -Uri $url -Method Put -Body $json -ContentType "application/json" -Headers #{Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"}
write-host "=========================================================="
Write-host "The value of Varialbe 'PullRequestID ' is updated to" $updatedef.variables.PullRequestID.value
write-host "=========================================================="

Related

How to update Azure build definition agent from vs2017-win2016 to windows-2019 programmatically

I am trying to programatically update all my build pipeline agents from vs2017-win2016 to windows-2019. I have access to AZ CLI, PowerShell and REST API but I haven't found the magic incantation to make it work. Specifically I would like to update the Azure Pipeline agent for a specific project and build definition name.
How to update Azure build definition agent from vs2017-win2016 to
windows-2019 programmatically
We could use the REST API Definitions - Get to get all the info about this definition in the body, then we could update the body and use the Definitions - Update to update the value of the release definition variable from a build pipeline:
PUT https://dev.azure.com/{organization}/{project}/_apis/build/definitions/{definitionId}?api-version=5.0
Following is my test inline powershell scripts:
$url = "https://dev.azure.com/{organization}/{project}/_apis/build/definitions/57?api-version=5.0"
Write-Host "URL: $url"
$pipeline = Invoke-RestMethod -Uri $url -Headers #{
Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"
}
Write-Host "Pipeline = $($pipeline | ConvertTo-Json -Depth 100)"
# Update an existing build definition named and build definition agent to its new value 2
$pipeline.process.target.agentSpecification.identifier= "windows-2019"
$pipeline.name= "UpdateVariabletest"
####****************** update the modified object **************************
$json = #($pipeline) | ConvertTo-Json -Depth 99
$updatedef = Invoke-RestMethod -Uri $url -Method Put -Body $json -ContentType "application/json" -Headers #{Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"}
write-host "=========================================================="
Write-host "The value of build definition named and build definition agent is updated to"
$updatedef.process.target.agentSpecification.identifier
$updatedef.name
You could check the similar thread for some more details.

Managing branches with Azure DevOps Classic Build Pipelines and TFVC

How do you manage building from branches when using Azure DevOps Classic Build Pipelines and TFVC?
I believe that the only viable option is to copy the build pipeline with a new name and update the source code mapping to point to the new TFVC branch.
I see the ADO web UI provides the option to clone an individual build definition, yet as I have over 200+ build pipelines to clone whenever I branch is there a more efficient way to do this? Or is the only option to write a custom tool to leverage the ADO REST Api?
Since you need to clone pipelines in batches, using scripts to run the Rest API will be a reasonable method. As far as I know, there is no easy way out of the box other than this.
You could try the following PowerShell Script Sample:
$DefinitionIds = "PipelineIDs" #InPut All Pipelineids(e.g. "324,323,xxx" )
$DefinitionId = $DefinitionIds.split(",");
$token = "PAT Token"
$token = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$($token)"))
foreach ($i in $DefinitionId)
{
echo $i
$url="https://dev.azure.com/{OrganizationName}/{ProjectName}/_apis/build/definitions/$($i)?api-version=6.0"
$response = Invoke-RestMethod -Uri $url -Headers #{Authorization = "Basic $token"} -Method Get -ContentType application/json
Write-Host "$($response | ConvertTo-Json -Depth 100)"
$response.repository.properties.tfvcMapping= '{"mappings":[{"serverPath":"$/TFVCBranchName","mappingType":"map","localPath":"\\"}]}' # ServerPath is the Branch name
$response.repository.name = "TFVCRepoName" #Repo Source Name
$response.name = "Pipeline $i Clone" # Cloned PipelineName
echo $response.name
$url1= "https://dev.azure.com/{OrganizationName}/{ProjectName}/_apis/build/definitions?api-version=6.0"
$json = #($response) | ConvertTo-Json -Depth 100
$response1 = Invoke-RestMethod -Uri $url1 -Headers #{Authorization = "Basic $token"} -Method Post -Body $json -ContentType application/json
}
Here are the Two Rest APIs used in the Script:
Definitions - Get
Definitions - Create
Result:
The cloned Pipeline will be set to the new TFVC branch and Build definition name.

"Download pipeline artifact" task output

"Download build artifacts" task has the output variable BuildNumber that makes it possible to get the id of the build the artifact was downloaded from. The new "Download pipeline artifact" task (which is preferred over the "Download build artifacts") does not have any the output variables. Is there a way to get the id of the build the artifact was downloaded from?
Also posted an issue on GitHub.
Microsoft is about to release a fix.
There is workaround to get the build id of the downloaded artifact using restful api.
To get the BuildId of the Specific version. You can refer to below below example.
First add a powershell task in your pipeline to run below scripts.
To get the definition id provided definition name(definition id will be used in the following scripts), check here for rest API.
$urldefinition ="https://dev.azure.com/<org>/<proj>/_apis/build/definitions?name=AboutSite-ASP.NET-CI&api-version=5.1"
$result = Invoke-RestMethod -Uri $urldefinition -Headers #{Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"} -Method Get
$definition= $result.value
$definitionId = $definition[0].id
Then you can get the build id with below scripts:
$url="https://dev.azure.com/<org>/<proj>/_apis/build/builds?definitions=$definitionId&buildNumber=20191109AboutSite-ASP.NET-CI&statusFilter=completed&resultFilter=succeeded&api-version=5.1"
$result = Invoke-RestMethod -Uri $urldefinition -Headers #{Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"} -Method Get
$build = $result.value
$id = $build[0].id
And then you can output the buildid for the following task to use with below script:
echo "##vso[task.setvariable variable=ArtifactBuildId;isOutput=true]$id"
To get the latest build id you can simply use below api
GET https://dev.azure.com/{organization}/{project}/_apis/build/latest/{definition}?branchName={branchName}&api-version=5.1-preview.1

How to modify Azure DevOps release definition variable from a release task?

What is the easiest way to get key rotation to work for azure storage accounts from a AzureDevOps relase task? The current plan is to re-generate the old key after release to invalidate it, and have a fresh key that can be used on next deployment. But to get that to work it seems like I at least need to store the name of the key to use in a release variable.
I had a look at he logging tasks (https://github.com/Microsoft/azure-pipelines-tasks/blob/master/docs/authoring/commands.md), but that only changes the value in the current release and does not modify the release definition.
You can use the REST API (Definitions - Update) to update the value of the release definition variable from a release task.
Go to the Agent Phase and select Allow Scripts to Access OAuth Token. See Use the OAuth token to access the REST API
Grant Project Collection Build Service (xxx) account the edit
release pipeline permission. (Select the release pipeline --> ... --> Security --> Edit release definition set to Allow)
Add a PowerShell task in your release pipeline
Run inline script: (Update the value of variable v1030 in below sample)
$url = "$($env:SYSTEM_TEAMFOUNDATIONSERVERURI)$env:SYSTEM_TEAMPROJECTID/_apis/Release/definitions/$($env:RELEASE_DEFINITIONID)?api-version=5.0-preview.3"
Write-Host "URL: $url"
$pipeline = Invoke-RestMethod -Uri $url -Headers #{
Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"
}
Write-Host "Pipeline = $($pipeline | ConvertTo-Json -Depth 100)"
# Update an existing variable named v1030 to its new value 1035
$pipeline.variables.v1030.value = "1035"
####****************** update the modified object **************************
$json = #($pipeline) | ConvertTo-Json -Depth 99
$updatedef = Invoke-RestMethod -Uri $url -Method Put -Body $json -ContentType "application/json" -Headers #{Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"}
write-host "=========================================================="
Write-host "The value of Varialbe 'v1030' is updated to" $updatedef.variables.v1030.value
write-host "=========================================================="
Here is a much cleaner and better solution that also allows for multiple builds being triggered simultaneously.
https://tsuyoshiushio.medium.com/how-to-pass-variables-with-pipeline-trigger-in-azure-pipeline-5771c5f18f91
Essentially your triggering build produces artifacts that your triggered build reads and turns into variables.
Still not at all great, but better than nothing and better than REST calls setting static global variable groups.
The other answer above talks about how to update Release pipelines.
If you would like to update a Build Pipeline's variables, here is how you do that:
Edit build pipeline
Go to the Agent Phase and select Allow Scripts to Access OAuth Token. See Use the OAuth token to access the REST API
Go to Manage Security -> Users -> Select Project Collection Build Service (YOUR TEAM NAME HERE)
Change "Edit Build Definitions" to Allow
Now add a powershell stage - 2.x - inline script called Update variables.
Script inline contents:
$api_version='5.0-preview.6'
$url = "$($env:SYSTEM_TEAMFOUNDATIONSERVERURI)$env:SYSTEM_TEAMPROJECTID/_apis/build/definitions/$(System.DefinitionId)?api-version=${api_version}"
Write-Host "URL: $url"
$pipeline = Invoke-RestMethod -Uri $url -Headers #{
Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"
}
Write-Host "Pipeline = $($pipeline | ConvertTo-Json -Depth 100)"
# Update variables as desired here:
$pipeline.variables.mavenBuildVersionPatch.value = "2401"
####****************** update the modified object **************************
$json = #($pipeline) | ConvertTo-Json -Depth 99
$updatedef = Invoke-RestMethod -Uri $url -Method Put -Body $json -ContentType "application/json" -Headers #{Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"}
write-host "=========================================================="
Write-host "The value of Variable mavenBuildVersionPatch is updated to" $updatedef.variables.mavenBuildVersionPatch.value
write-host "=========================================================="
Take note of the API version in this script is 5.0-preview.6.
If your version of Azure Devops is newer, you may need to update this in the future.
Save build pipeline.
Now when you run job, after the job completes this powershell stage, this variable will be set.
Important: If you want to update a variable then make the updated variable for other pipeline stages, then you do it with a powershell stage with the following inline script:
$mavenBuildVersionPatch = [int]"$(mavenBuildVersionPatch)"
$mavenBuildVersionPatch = $mavenBuildVersionPatch + 1
Write-Host "##vso[task.setvariable variable=mavenBuildVersionPatch;]$mavenBuildVersionPatch"
This example would take our existing patch number and increment it by 1.
This does not save the variable at the end of the job, you still need to do that with another powershell script if desired.

How to Increase/Update Variable Group value using Azure Devops Build Definition?

Am working on Azure Devops CI&CD. Here, my release name must be in Version number using tags. Am getting this with the help of Variable Groups, by adding tags and value to it. Here am getting the tags value as constant like a static for every release like 1.1,1.2,1.3 etc.
Now am trying to increase/update my tag value dynamically for every new release triggered after completion of my Build Definition successfully which looks like 1.1,1.2,2.1,2.2,3.1,3.2 etc.It is possible with the help of statically by variable group, but manually we need to update it.
Is it possible to Increase/Update the tags value in Variable Group with the Build Definition tasks or other process.If possible, please suggest me to "How to done this?"
You can overwrite/update the value of the variables by using the logging command to set the variables again in Azure Devops Build pipleline:
Write-Host "##vso[task.setvariable variable=testvar;]testvalue"
To increase the value dynamically, you need to use the token $(Rev:.r). You can custom the variables based on the $(Build.BuildNumber) or $(Release.ReleaseName)as they will increase the value dynamically...
Just reference this thread to custom the variables:https://github.com/MicrosoftDocs/vsts-docs/issues/666#issuecomment-386769445
UPDATE:
If you just want to update the value of the variables which defined in a specific Variable Group, then you can call REST API in build pipeline to achieve that:
PUT https://{account}.visualstudio.com/{ProjectName or ID}/_apis/distributedtask/variablegroups/{Variable Group ID}?api-version=5.0-preview.1
Content-Type: application/json
Request Body:
{"id":2,"type":"Vsts","name":"VG0926","variables":{"TEST0926":{"isSecret":false,"value":"0930"}}}
UPDATE2:
You can write a PowerShell script to call the REST API, then add a PowerShell task to run the script in your build pipeline: (Use the OAuth token to access the REST API)
Below sample for your reference:
$url = "$($env:SYSTEM_TEAMFOUNDATIONCOLLECTIONURI)$env:SYSTEM_TEAMPROJECTID/_apis/distributedtask/variablegroups/{Variable Group ID}?api-version=5.0-preview.1"
Write-Host $url
function CreateJsonBody
{
$value = #"
{"id":2,"type":"Vsts","name":"VG0926","variables":{"TEST0926":{"isSecret":false,"value":"0930"}}}
"#
return $value
}
$json = CreateJsonBody
$pipeline = Invoke-RestMethod -Uri $url -Method Put -Body $json -ContentType "application/json" -Headers #{
Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"
}
Write-Host "New Variable Value:" $pipeline.variables.TEST0926.value
UPDATE3:
Well, tested again, below scripts works for me as well. You can try it, just replace the parameters accordingly:
# Base64-encodes the Personal Access Token (PAT) appropriately
$base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f "","PAT here")))
$url = "https://dev.azure.com/xxx/Test0924/_apis/distributedtask/variablegroups/1?api-version=5.0-preview.1"
$json = '{"id":1,"type":"Vsts","name":"VG0928","variables":{"TEST0928":{"isSecret":false,"value":"0931"}}}'
$pipeline = Invoke-RestMethod -Uri $url -Method Put -Body $json -ContentType "application/json" -Headers #{Authorization=("Basic {0}" -f $base64AuthInfo)}
Write-Host "New Variable Value:" $pipeline.variables.TEST0928.value
You can simply update any number of variables in an Azure Devops variable group using its built-in az pipelines variable-group variable update command. You can use this command in a script task in the pipeline definition as shown below.
ps: Replace all the upper case values with corresponding values except the SYSTEM_ACCESSTOKEN.
variables:
- group: VARIABLE_GROUP_NAME
jobs:
- job: UpdateVarGroup
steps:
- script: |
newValue="This is the updated value"
echo $SYSTEM_ACCESSTOKEN | az devops login
az pipelines variable-group variable update --group-id $(group_id) \
--name NAME_OF_THE_VARIABLE \
--value "${newValue}" \
--org https://dev.azure.com/YOUR_ORGANIZATION_NAME \
--project AZURE_DEVOPS_PROJECT
displayName: 'Update variable inside a variable group'
env:
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
In order for this example to work, you need to have a variable inside your variable group called group_id. The value of that needs to be set to the group id of the variable group, which can be obtained by simply looking at the url of your variable group. (The group id is the value for variableGroupId in the url of your browser when you are inside the variable group)
System.AccessToken is required for az devops login
I used this task to update the value of my variables inside my group.
Shared variable updater (preview)
Dont forget to set those settings :
Requires 'Allow scripts to access the OAuth token' in agent job additional options
Set administrator role to 'Project Collection Build Service' in the variable group.
In case of using a YAML pipeline
When using a YAML pipeline, the OAuth token is automatically added (no need for step 1 above), but requires a bit of work to make accessible for the powershell script. Use the guidance here to be able to use the token.
If you want to update a variable group's value, use the REST.API methods.
Rather than constructing the PUT request body manually, use a GET first to get the original JSON, update only what you need, then replay it as a PUT. I used to keep the variable group id also as a variable in that group to avoid hard-coding.
variables:
- group: MyVarGroup
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
$url = "$($env:SYSTEM_TEAMFOUNDATIONCOLLECTIONURI)$env:SYSTEM_TEAMPROJECTID/_apis/distributedtask/variablegroups/$(VariableGroupId)?api-version=6.0-preview.1"
$header = #{
"Authorization" = "Bearer $(System.AccessToken)"
}
$def = Invoke-RestMethod -Uri $url -Headers $header
$def.variables.MyTestVar.value = "NewValue"
$body = #($def) | ConvertTo-Json -Depth 100 -Compress
$def = Invoke-RestMethod -Method 'Put' -Uri $url -ContentType 'application/json' -Headers $header -Body $body
You can overwrite variables using the REST API with a PowerShell task without needing to create a PAT.
You'll need to first going into the agent job and set "Allow scripts to access OAuth token".
You'll need too go to your variable group and add the 'Project Collection Build Service' as an administrator.
Now you can call the Rest API using the OAuth Bearer token. Code for reference:
$id = <variable group id>
# This is using some environment variables provided by the pipeline to build the URL
$url = ("$($env:SYSTEM_TEAMFOUNDATIONCOLLECTIONURI)$env:SYSTEM_TEAMPROJECTID/_apis/distributedtask/variablegroups/{0}?api-version=5.0-preview" -f $id)
# You might find it useful to us a GET method to grab the variable group, update it and then convert it to this json string rather than doing this here
$json = '{"id":$id,"type":"Vsts","name":"<Variable Group Name>","<Variable Name":{"ThisIsMyVariable":{"isSecret":false,"value":"20"}}}'
$pipeline = Invoke-RestMethod -Uri $url -Method Put -Body $json -ContentType "application/json" -Headers #{Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"}
I made this task to manage variable groups from Pipelines:
ManageVariableGroupTask
Using this you can Create/Update/Delete variable groups and do the same operations on variables contained in them.
Hope that's helpfull!