Automatically Export Build and Release definitions using Powershell or DEVOPS tasks - azure-devops

Have a requirement to automatically export specific Azure DEVOPS Build/Release definitions. I know the names of the definitions required. The process would run weekly to capture the information. I know the export can be done manually but want to automate process. Hoping Powershell script can be used.
Thanks
Joe

If you want to export the build/release definition automatically, you'd better use Powershell task with Rest API. But if this, it is not enough for just know the build definition name.
Refer to these docs: get build definition and get release definition. You can see that definitionid is necessary. In fact, this definitionid is very easy to get. Just click the relevant pipeline you want to export, the definitionid will display in URL:
To export the definition, you can use the follow script in powershell:
$headers = #{ Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN" }
$projectsUrl = "https://dev.azure.com/{org}/{project}/_apis/build/definitions/{build definitionid}?api-version=5.1"
$result = Invoke-RestMethod -Uri $projectsUrl -Method Get -Headers $headers
$filename=$result.name+".json"
$filePath="D:\"
$file=$filePath+$filename
$result | ConvertTo-Json | Out-File -FilePath $file
In this script, I specified the build name as the file name($filename=$result.name+".json"), and also, convert the result content as JSON to make the local file more readable:
Similarly, to get the release definition, just change the url as get release difinition:
$headers = #{ Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN" }
$projectsUrl = "https://vsrm.dev.azure.com/{org}/{project}/_apis/release/definitions/{definitionId}?api-version=5.1"
$result = Invoke-RestMethod -Uri $projectsUrl -Method Get -Headers $headers
$filename=$result.name+".json"
$filePath="D:\"
$file=$filePath+$filename
$result | ConvertTo-Json | Out-File -FilePath $file
Note: While use #{ Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN" }, you'd enable Allow scripts to access the OAuth token to make the environment variable available during build pipeline.
In addition, as what you want is capture the information weekly, you can Schedule the pipeline which has these export task:
Now, these export pipeline will run and export the definition weekly.

You'll be looking at the az pipelines release and az pipelines build commands from Azure DevOps CLI
Commands Reference
Extension Reference

Related

Azure cli: clone pipeline

Looking at az pipelines documentation it seems it's not possible to clone a pipeline using cli.
I've looked at getting the yaml (az pipelines show -name=x > x_orig.yaml) and then trying to change json and create pipeline from modified yaml, but that feels like a lot of work that could break after next update.
Is there a way to clone a pipline without going the the Web UI?
Currently, there indeed is not available Azure CLI that can clone or export/import a pipeline to create a new pipeline.
I also searched and tried the Azure DevOps REST API for Pipelines, but did not find the available API.
Ideally, the Azure CLI "az pipelines create" can provide an input parameter that allows users specify an existing pipeline as a starting point for the new pipeline.
If your projects really need this feature, I recommend that you can directly report a feature request on the "Azure/azure-cli" repository to ask adding the parameter like as above mentioned. That will allow you directly interact with the appropriate engineering team, and make it more convenient for the engineering team to collect and categorize your suggestions.
As a workaround, we could clone the build definition via power shell script to call REST API.
Note: We need to change the original build definition name.
REST API
Get build definition:
GET https://dev.azure.com/{organization}/{project}/_apis/build/definitions/{definitionId}?api-version=6.0
Create build definition
POST https://dev.azure.com/{organization}/{project}/_apis/build/definitions?api-version=6.0
Power shell script
$connectionToken="{pat}"
$base64AuthInfo= [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$($connectionToken)"))
$BuildDefinitionInfoURL = "https://dev.azure.com/{org name}/{project name}/_apis/build/definitions/386"
$BuildDefinitionInfo = Invoke-RestMethod -Uri $BuildDefinitionInfoURL -Headers #{authorization = "Basic $base64AuthInfo"} -Method Get
Write-Host $BuildDefinitionInfo.name
$BuildDefinitionInfo.name = $BuildDefinitionInfo.name +" clone"
Write-Host $BuildDefinitionInfo.name
$body = $BuildDefinitionInfo | ConvertTo-Json -Depth 99
$createBuildDefinitionURL = "https://dev.azure.com/{org name}/{project name}/_apis/build/definitions?api-version=6.0"
$response = Invoke-RestMethod -Uri $createBuildDefinitionURL -ContentType "application/json" -Body $body -Headers #{Authorization=("Basic {0}" -f $base64AuthInfo)} -Method POST
Write-Host $response.id
Result:

Store result of Azure CLI task into release variable

I have a release pipeline with multiple jobs.
On the first agent job there is an Azure CLI task that retrieves the keys of a storage account on azure.
The command it executes is :
az storage account keys list --account-name "$(diagnosticsStorageAccountName)" --resource-group "$(resourceGroup)"
What I want to do is store the result of this command and utilize it in a task that is running under a deployment group job.
I've already looked into these resource :
Set Output Variable in Azure CLI task on VSTS
How to modify Azure DevOps release definition variable from a release task?
I've tried the first one but I didn't get it working.
I didn't bothered with the second because it seems way to hacky.
Is there any way do achieve this that isn't hacky ?
The output values didn't get stored properly.
The output of az storage account keys list --account-name "$(diagnosticsStorageAccountName)" --resource-group "$(resourceGroup)"is spread over multiple lines when you use the following syntax:
echo "##vso[task.setvariable variable=testvar;]%myvar%"
So one of the problems was that only the first line of the JSON array was being stored into the variable.
I solved this problem in the following way:
keys=`az storage account keys list --account-name "$(diagnosticsStorageAccountName)" --resource-group "$(resourceGroup)"`
taskvariable="##vso[task.setvariable variable=_[tempVariable];]"
echo $taskvariable$keys
According to documentation echo "##vso[task.setvariable variable=testvar output=true;]%myvar% should make the variable available for the whole release. Unfortunately I had no luck with this.
I overcame this using a Powershell task under the same Agent Job (Ubuntu 16.0) :
$url = "$($env:SYSTEM_TEAMFOUNDATIONSERVERURI)$env:SYSTEM_TEAMPROJECTID/_apis/Release/definitions/$($env:RELEASE_DEFINITIONID)?api-version=5.1"
Write-Host "URL: $url"
$pipeline = Invoke-RestMethod -Uri $url -Headers #{
Authorization = "Bearer $(System.AccessToken)"
}
#Parsing JSON string from previous task
$keys = $(#"
$(_tempVariable)
"# | ConvertFrom-Json)
# Assignment of variable value
$pipeline.variables.[variableName].value = $keys[0].value
####****************** update the modified object **************************
$json = #($pipeline) | ConvertTo-Json -Depth 99
$updatedef = Invoke-RestMethod -Uri $url -Method Put -Body $json -ContentType "application/json" -Headers #{Authorization = "Bearer $(System.AccessToken)"}
write-host "=========================================================="
Write-host "The value of Variable '[variableName]' is updated to" $updatedef.variables.[variableName].value
write-host "=========================================================="
Please note that in order to get this working there are a couple of things you need to do.
First you need to allow access to the OAuth token on the Agent Job.
On top of that you need to give the "Project Collection Build Service".
Click on security on the release and click on the "Project Collection Build Service" user.
Change the values for the "Edit release" and "Manage release" to allow and save the changes.

How to set the Description of an Azure DevOps Release via script?

In Azure DevOps I'm trying to set the release description via PowerShell / CMD in order to have a dynamic description of my releases based on the input of the artifacts from the build step.
I've tried setting the release variables via powershell like:
Write-Host "##vso[task.setvariable variable=release.releasedescription;]bar"
Write-Host "##vso[task.setvariable variable=RELEASE_RELEASEDESCRIPTION;]bar"
But that didn't work and the description field remains always empty.
Is there any tweak / setting that would help achieve this behavior?
What you tried to do is just to set the environment variable that contains the release description data and not set the "real" release description, so after the release finished the description not changed.
If you want to set the release description during the release you can try to do it with Azure DevOps Rest API - Update Release.
So add a PowerShell task that executes the Rest API with Invoke-RestMethod, get the current release with GET method and then update the release with PUT, in the body change the description to the new one.
You could do this too:
- bash: |
echo "Updating pipeline job Run description"
echo "##vso[build.updatebuildnumber]$(Build.BuildNumber) $(App_Name)"
displayName: "Set pipeline job Run description for Azure DevOps console"
For anyone who simply wants to set the name of an Azure Pipelines run, you can find the description here.
TLDR: Set the top-level name: attribute.
Here is the PowerShell script to set release description in DevOps. Before the stage that need approval, add a PowerShell task to invoke REST API to change the release description. Please use your own orgName, projectName, PAT and modify the release description.
$url = https://vsrm.dev.azure.com/<orgName>/<ProjectName>/_apis/release/releases/$(Release.ReleaseId)?api-version=6.0
$token="PAT"
$base64AuthInfo= [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$($token)"))
$head = #{ Authorization =" Basic $base64AuthInfo" }
$pipeline = Invoke-RestMethod -Uri $url -Headers $head -Method Get
$Pipeline.description = "<Modify the release description>"
$body = $Pipeline | ConvertTo-Json -Depth 100
$url2=https://vsrm.dev.azure.com/<orgName>/<ProjectName>/_apis/release/releases/$(Release.ReleaseId)?api-version=6.0
$resp = Invoke-RestMethod -Uri $url2 -Method Put -Headers $head -Body $body -ContentType application/json

Authenticate Azure Pipelines Powershell task against Azure DevOps Rest API

Requirements
My requirement is that for my Azure devops release pipeline I want to tag a specific commit with an annotated tag which contains the build number and the date (which is auto-set on an annotated tag).
Proposed solution
My solution to this is to use a the Azure Powershell pipeline task, the one shown here:
The task (ignore what's in the script box right now) will use the Azure Subscription that I have set in order to authenticate towards the Azure DevOps REST API. I have successfully been able to perform the task I want using a personal access token (PAT) but this is not stable long-term for a whole team and I want to use our Azure Subscription.
The problem
My problem is that I'm not sure how to use the authentication of the Azure Subscription correctly. I seem to get some data using Get-AzureRmContext (see current code below) and then I found a GitHub issue which seems to do sort of the same thing. The code gets some kind of OAuth token but using the code below, Azure still returns to me that I need to sign in, so I assume it's not the correct token. I don't understand how things hatch into each other.
Note that my subscription should have all the permissions it needs to do what I want.
Code so far:
Function Get-AccessToken($tenantId) {
$cache = [Microsoft.IdentityModel.Clients.ActiveDirectory.TokenCache]::DefaultShared
$cacheItem = $cache.ReadItems() | Where-Object { $_.TenantId -eq $tenantId } | Select-Object -First 1
return $cacheItem.AccessToken
}
$context = Get-AzureRmContext
$uri = "https://dev.azure.com/<my_org>/<my_area>/_apis/git/repositories/<project_sha>/annotatedtags?api-version=5.0-preview.1"
$token = Get-AccessToken $context.tenantID
$body = #"
{
"taggedObject": {
"objectId": "$(BUILD.SOURCEVERSION)"
},
"name": "D-$(Build.BuildNumber)",
"message": "dummy"
}
"#
$header = #{"Authorization" = "Bearer" + $token}
Invoke-RestMethod -Uri $uri -Method Post -ContentType "application/json" -Body $body -Headers $header
Any help is greatly appreciated!
There is the example for the build tasks: Use a PowerShell script to customize your build pipeline
You have to enable access to token (option Allow Scripts to Access OAuth Token)
Then use it in your script. Script from the example:
$url = "$($env:SYSTEM_TEAMFOUNDATIONCOLLECTIONURI)$env:SYSTEM_TEAMPROJECTID/_apis/build/definitions/$($env:SYSTEM_DEFINITIONID)?api-version=5.0"
Write-Host "URL: $url"
$pipeline = Invoke-RestMethod -Uri $url -Headers #{
Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"
}
Write-Host "Pipeline = $($pipeline | ConvertTo-Json -Depth 100)"

How to modify Azure DevOps release definition variable from a release task?

What is the easiest way to get key rotation to work for azure storage accounts from a AzureDevOps relase task? The current plan is to re-generate the old key after release to invalidate it, and have a fresh key that can be used on next deployment. But to get that to work it seems like I at least need to store the name of the key to use in a release variable.
I had a look at he logging tasks (https://github.com/Microsoft/azure-pipelines-tasks/blob/master/docs/authoring/commands.md), but that only changes the value in the current release and does not modify the release definition.
You can use the REST API (Definitions - Update) to update the value of the release definition variable from a release task.
Go to the Agent Phase and select Allow Scripts to Access OAuth Token. See Use the OAuth token to access the REST API
Grant Project Collection Build Service (xxx) account the edit
release pipeline permission. (Select the release pipeline --> ... --> Security --> Edit release definition set to Allow)
Add a PowerShell task in your release pipeline
Run inline script: (Update the value of variable v1030 in below sample)
$url = "$($env:SYSTEM_TEAMFOUNDATIONSERVERURI)$env:SYSTEM_TEAMPROJECTID/_apis/Release/definitions/$($env:RELEASE_DEFINITIONID)?api-version=5.0-preview.3"
Write-Host "URL: $url"
$pipeline = Invoke-RestMethod -Uri $url -Headers #{
Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"
}
Write-Host "Pipeline = $($pipeline | ConvertTo-Json -Depth 100)"
# Update an existing variable named v1030 to its new value 1035
$pipeline.variables.v1030.value = "1035"
####****************** update the modified object **************************
$json = #($pipeline) | ConvertTo-Json -Depth 99
$updatedef = Invoke-RestMethod -Uri $url -Method Put -Body $json -ContentType "application/json" -Headers #{Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"}
write-host "=========================================================="
Write-host "The value of Varialbe 'v1030' is updated to" $updatedef.variables.v1030.value
write-host "=========================================================="
Here is a much cleaner and better solution that also allows for multiple builds being triggered simultaneously.
https://tsuyoshiushio.medium.com/how-to-pass-variables-with-pipeline-trigger-in-azure-pipeline-5771c5f18f91
Essentially your triggering build produces artifacts that your triggered build reads and turns into variables.
Still not at all great, but better than nothing and better than REST calls setting static global variable groups.
The other answer above talks about how to update Release pipelines.
If you would like to update a Build Pipeline's variables, here is how you do that:
Edit build pipeline
Go to the Agent Phase and select Allow Scripts to Access OAuth Token. See Use the OAuth token to access the REST API
Go to Manage Security -> Users -> Select Project Collection Build Service (YOUR TEAM NAME HERE)
Change "Edit Build Definitions" to Allow
Now add a powershell stage - 2.x - inline script called Update variables.
Script inline contents:
$api_version='5.0-preview.6'
$url = "$($env:SYSTEM_TEAMFOUNDATIONSERVERURI)$env:SYSTEM_TEAMPROJECTID/_apis/build/definitions/$(System.DefinitionId)?api-version=${api_version}"
Write-Host "URL: $url"
$pipeline = Invoke-RestMethod -Uri $url -Headers #{
Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"
}
Write-Host "Pipeline = $($pipeline | ConvertTo-Json -Depth 100)"
# Update variables as desired here:
$pipeline.variables.mavenBuildVersionPatch.value = "2401"
####****************** update the modified object **************************
$json = #($pipeline) | ConvertTo-Json -Depth 99
$updatedef = Invoke-RestMethod -Uri $url -Method Put -Body $json -ContentType "application/json" -Headers #{Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"}
write-host "=========================================================="
Write-host "The value of Variable mavenBuildVersionPatch is updated to" $updatedef.variables.mavenBuildVersionPatch.value
write-host "=========================================================="
Take note of the API version in this script is 5.0-preview.6.
If your version of Azure Devops is newer, you may need to update this in the future.
Save build pipeline.
Now when you run job, after the job completes this powershell stage, this variable will be set.
Important: If you want to update a variable then make the updated variable for other pipeline stages, then you do it with a powershell stage with the following inline script:
$mavenBuildVersionPatch = [int]"$(mavenBuildVersionPatch)"
$mavenBuildVersionPatch = $mavenBuildVersionPatch + 1
Write-Host "##vso[task.setvariable variable=mavenBuildVersionPatch;]$mavenBuildVersionPatch"
This example would take our existing patch number and increment it by 1.
This does not save the variable at the end of the job, you still need to do that with another powershell script if desired.