How to GET variables used in a specific BUILD in Azure Devops / TFS via api - powershell

From within both Build & Release pipelines in Azure Devops, I'm triggering other pipelines via api.
I am trying to get my release pipeline to harvest custom variables that were discovered / used during the associated build.
I'm able to get a list of builds via get api, and i'm able to spawn other builds and pass parameters to them.
While I can spawn a release via api, I have been unsuccessful in passing parameters... or harvesting parameters on-demand at the time of Create Release.
I have reviewed Microsoft's documentation and successfully queued Builds via api with custom parameters and created releases without custom parameters.
[String]$buildID = "$env:BUILD_BUILDID"
[String]$project = "$env:SYSTEM_TEAMPROJECT"
[String]$projecturi = "$env:SYSTEM_TEAMFOUNDATIONCOLLECTIONURI"
[String]$alias = "drop"
$headers = #{ Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN" }
$url= $projecturi + $project + "/_apis/release/releases?api-version=5.0-preview"
$JSON = #"
{
"definitionId": 9,
"description": "Testing API Release",
"artifacts": [
{
"alias": "$alias",
"instanceReference": {
"id": "$buildID",
"name": null
}
}
],
"isDraft": false,
"reason": "none",
"manualEnvironments": null
}
"#
Write-Host $url
$responseRelease = Invoke-RestMethod -Uri $url -headers $headers -Method Post -ContentType "application/json" -Body $JSON
Write-Host $responseRelease
The results of the current code creates a release, but don't pass custom variables (or harvest custom variables from a completed build). I would rather pass or harvest them than recalculate them.

First as you have mentioned in the comment, it's not able to directly change variables when deploy a release. As a user voice on Road map:
Change release variables' value when deploy a release
https://developercommunity.visualstudio.com/idea/365596/change-release-variables-value-when-deploy-a-relea.html.
According to your description, seems you want to access the Build variables from Release definition. This is also not a build-in feature at present.
As a workaround, try to use this 3rd-party Azure DevOps extension Variable Tools for Azure DevOps Services.
In the "build pipeline" you can create a JSON file using "save variables". This file needs to be published as artifact or with existing artifact.
In the "release pipeline" you can restore the variables using "load variables" from the JSON file.
Another way is storing the variable’s value in Variable Group and link the variable group into your release definition.
During release, you can get variable groups firstly, and filter the variable under the variable group-$(Build.BuildId).

Related

Retrieving info from Azure Devops with Powershell

I'm trying to automate the deployment of releases in Azure Devops using the Azure Devops API with Powershell. I'm able to retrieve the last release definition, but I'm not able to extract a release for a specific environment.
I'm looking to retrieve the last 'prod' release in order to obtain the 'release ID', and then deploy the release to UAT or DEV using a HTTP 'PATCH' request.
I can hit a URL manually (example below), which gives me the last release for DEV (definitionEnvironmentId=1), and I know I could easily changed the environment ID to '12' for production, but I'd like to get this via an API call.
https://{redacted url}/{redacted}/{redacted}/_apis/release/deployments?api-version=6.0&definitionId=1&definitionEnvironmentId=1&$top=1
Any ideas if this is possible as I can't find any info on it?
You were using the correct Deployments - List Rest api. The reason you didnot get any info from the api call via powershell is because you didnot escape the special character $ for $top parameter in the url.
$ is a special character in powershell. You should escape it by adding a back tick '`' in the url. See below:
#add ` to $top parameter
$url = "https://{redacted url}/_apis/release/deployments?definitionId=3&definitionEnvironmentId=5&`$top=1&api-version=6.0"
$connectionToken="PAT"
$base64AuthInfo= [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$($connectionToken)"))
$response= Invoke-RestMethod -Uri $url -Headers #{authorization = "Basic $base64AuthInfo"} -Method Get
$releaseId = $response.value[0].release.id

How to use a custom variable from build-pipeline in a release pipeline

Question:
Is there any way to define a custom-variable in a build pipeline in azure-devops, which then can be used in/exposed to the release pipeline in any way?
Scenario:
We are not using a variable group, because we need to dynamically set the variable in the build pipeline and then consume it in the release pipeline - it is not a static super global.
Checked the docs at release variables and build variables but could not find any helping information or a hint, that this is possible.
What I tried
Define a variable in variables ( in the build pipeline ) and try to access it in the release pipeline using $(name) or checking if it is in env.
Extras - Motivation
The motivation behind this is
read the latest git-tag used in a build-pipeline step and expose this into the pipeline variable VERSION ( actually, we bump patch during that )
Releasing builds is a manual step.
If a build is released, the azure-devops gui shows us all the variables of the release-pipeline, which are "settable during release - this includes the version we want to release this package with
we want this "VERSION" to be prefilled with the version of the build-pipeline as a suggestion for the next version
In the release pipeline we checkout the repo, add the VERSION as a tag and package/publish the artifact with this version
This is impossible by default, but you can use 2 extensions from the marketplace:
1) Shared variable updater - Create a variable group and during the build update there the variables dynamically with this task. you can also do it with your script, see the answers here.
2) Variable Kit for Azure DevOps Services - During a Build, save variables to a json file stored along with your build assets. During a Release, load the saved variables and use them within the release definition.
How to use a custom variable from build-pipeline in a release pipeline
You could try to use the REST API Release Definitions - Update to update the default variable in the release pipeline to use the value defined in a build pipeline.
PUT https://vsrm.dev.azure.com/{organization}/{project}/_apis/release/definitions?api-version=5.1
Details:
Define a custom-variable in a build pipeline, like TestValue, the value is 123:
Also define the same custom-variable in a Release pipeline with default value 123:
Then add a inline powershell scripts to invoke the REST API Definitions - Update to update the default value in the release pipeline:
$url = "https://vsrm.dev.azure.com/<OrganizationName>/<ProjectName>/_apis/release/definitions/<DefinitionId>?api-version=5.1"
Write-Host "URL: $url"
$pipeline = Invoke-RestMethod -Uri $url -Method Get -Headers #{
Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"
}
Write-Host "Pipeline = $($pipeline | ConvertTo-Json -Depth 100)"
# Update an existing variable named TestValue to its new value 987
$pipeline.variables.TestValue.value = "$(TestValue)"
####****************** update the modified object **************************
$json = #($pipeline) | ConvertTo-Json -Depth 99
Write-Host "URL: $json "
$updatedef = Invoke-RestMethod -Uri $url -Method Put -Body $json -ContentType "application/json" -Headers #{Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"}
write-host "=========================================================="
Write-host "The value of Varialbe 'TestValue' is updated to" $updatedef.variables.TestValue.value
In this case, we could dynamically set the variable when we queue the build pipeline, and this value will overwrite the default value in the release pipeline, so that we could use it release pipeline.
Hope this helps.

Azure DevOps integration in Sentry: Associate commits

Did someone manage to integrate Azure DevOps in Sentry (sentry.io)? I stuck on "Associate commits with a Release" (see: https://docs.sentry.io/workflow/releases/?platform=browser#associate-commits-with-a-release)
I can not figure out a way how I can tell Sentry (by API) which commit ids are associated with a current release/deploy. How can I add a task to the pipeline which will post the commit ids to Sentry API? Or is there some other way to do it?
In azure devops, the Powershell task also support curl. So, you can execute the api in powershell task of VSTS pipeline directly.
In release pipeline, there has a pre-defined release variable, it stores the commit id which is associated with the current release pipeline: $(Release.Artifacts.{alias}.SourceVersion). Here alias is the artifacts name, and you can get it by getting $(Release.PrimaryArtifactSourceAlias).
First, create variables like this:
Then you can apply the variable $(id) into that API, and execute the api in powershell task:
"refs": [{
"commit":"$(id)"
}]
Now, the commit id could be packed into the body of this api, and send to the Sentry server.
If there has multiple commits associate with this release, since the variable $(Release.Artifacts.{alias}.SourceVersion) I mentioned above only store the latest commit message, here you may need add additional scripts to get what you want by Build id.
In release pipeline, with $(Build.BuildId) you can get the corresponding buildid which associate with this release. And then, you could get the commits(changes) by using this API:
GET https://dev.azure.com/{organization}/{project}/_apis/build/changes?fromBuildId={fromBuildId}&toBuildId={toBuildId}&api-version=5.1-preview.2
You could apply these powershell script into your task without change anything because this script is universal among powershell-ise, powershell command line and powershell task in VSTS.
$token = "{PAT token}"
$url="https://dev.azure.com/{org name}/{project name}/_apis/build/changes?fromBuildId={id1}&toBuildId={id2}"
$token = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$($token)"))
$response = Invoke-RestMethod -Uri $url -Headers #{Authorization = "Basic $token"} -Method Get
Write-Host "results = $($response.value.id | ConvertTo-Json -Depth 100)"
Now, you could get the list of commits which associate with the build and corresponding release.

Azure DevOps get Deployment Agent status during release

I'm trying to get the status of agents in a deployment pool at release time.
The use case is I have 2 servers with shared disk, I want the release to run on one server only. I have two Deployment groups that run based on a custom condition:
eq(variables['DeployGroupSelector'], '1')
With a job that runs prior to those that will determine the value of the DeployGroupSelector var, essentially a case statement.
In the job that sets the var, I'm trying to reach out to the Azure DevOps REST API:
$headers = #{
Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"
}
$url = "https://dev.azure.com/$($organization)/_apis/distributedtask/pools/$($poolId)/agents?api-version=5.1"
$response = Invoke-RestMethod $url -Headers $headers -Verbose
write-host "Output: $response"
$status = ($response.value | where {$_.name -eq $($env:primaryPoolName)}).status
if($status -eq "online")
{
Write-Output("##vso[task.setvariable variable=DeployGroupSelector;]1")
}
else
{
Write-Output("##vso[task.setvariable variable=DeployGroupSelector;]2")
}
For the group containing the script above the "Allow scripts access to the OAuth token" box is checked.
When I run this powershell locally using a PAT it returns data. When I run the release in ADO, it hits the service, but returns an empty data set:
2019-10-07T14:16:18.8942915Z VERBOSE: GET https://dev.azure.com/xxxxxx/_apis/distributedtask/pools/13/agents?api-version=5.1 with 0-byte payload
2019-10-07T14:16:19.3235204Z VERBOSE: received 22-byte response of content type application/json
2019-10-07T14:16:19.9626359Z VERBOSE: Content encoding: utf-8
2019-10-07T14:16:19.9835101Z Output: #{count=0; value=System.Object[]}
I have tried giving the "Project Collection Build Service Accounts" group read access to pools and groups, I even tried bumping it up to user. I tried adding the build service accounts group to release administrators. I even tried using the old url format just in case.
Added picture of data returned from powershell:
UPDATE: Just to further rule out an issue with how I was using the token, I added a second powershell task to the task group in question. This script hits a different segment of the AzDO Rest API (below). This successfully gets a response. So the OAuth token is working, it just doesn't seem to have access to the entire API.
$headers = #{
Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"
}
$url = "https://dev.azure.com/$($organization)/$($project)/_apis/git/repositories?api-version=5.1"
$response = Invoke-RestMethod $url -Headers $headers -Verbose
write-host "Output: $($response)"
Response: Output: #{value=System.Object[]; count=10}
Since you're using the System_AccessToken variable, did you also enable the "Allow scripts to access OAuth token" checkbox in the agent job? https://learn.microsoft.com/en-us/azure/devops/pipelines/build/variables?view=azure-devops&tabs=classic#systemaccesstoken This link shows where it is in build but you'll find it at the bottom of the Agent Job panel in release. If that's not checked, that may be why you're getting an empty response.
Had exactly the same issue. Were considered two options, the same as you have been trying.
System.AccessToken
PAT
Problem was solved by putting PAT into a KeyVault and using it as a basic auth token for REST API call in a pipeline.
My suggestion is that it seems expected and right behavior. Why do I think so? From Azure DevOps point of view there are two levels in our case Organization's level and Project's level. You can notice the difference by URI that were used:
$url = "https://dev.azure.com/$($organization)/_apis/distributedtask/pools/$($poolId)/agents?api-version=5.1"
$url = "https://dev.azure.com/$($organization)/$($project)/_apis/git/repositories?api-version=5.1
From security point of view it's a bad practice, let entities from lower layers, which is project in our case, access and manipulate on the higher layer, which is organization in our case.
As a conclusion I would say that SystemToken and PAT in essence have slightly different nature, one is devoted for agent, and another for one's personal profile.

How can one automate promotion of an artifact to a feed view in Azure DevOps?

Our build artifact is an Octopus nuget package. When the build is released it lands into the QA stage where the artifact is deployed through Octopus. This octopus consumes it directly from the Azure Artifacts nuget feed.
If the deployment and the subsequent tests are successful we want to promote the artifact to the Release view of the Azure Artifacts nuget feed, because we think it gives us a different nuget URL that can be used by another Octopus serving the next stage (for historic reasons we have dedicated Octopus per stage - working to change that, but it takes time).
We can promote manually, but we want to do it automatically. How can this be done?
We are testing it on on-premises TFS 2019 RC2.
EDIT 1
The suggested plugin does not seem to install on on-premises TFS 2019 RC2:
Using PowerShell...
$organisationName = '' # Name of organisation
$projectName = '' # Name of project
$feedName = '' # Name of Azure Artifacts feed
$viewName = 'Release' # I believe this can also be Prerelease, but I've not tried it
# List of names of packages within Azure Artifacts feed to be promoted
$packagesToPromote = #('')
# Need a personal access token for this script to work
# PAT token should be assigned to Packaging (Read, Write and Manage) scopes
$azureArtifactsPAT = ''
$AzureArtifactsPAT_Base64 = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(":$($azureArtifactsPAT)"))
$restAPICallHeader = #{ Authorization = "Basic $AzureArtifactsPAT_Base64" }
$feedBaseURL = "https://feeds.dev.azure.com/$organisationName/$projectName/_apis/packaging/feeds"
$packageBaseURL = "https://pkgs.dev.azure.com/$organisationName/$projectName/_apis/packaging/feeds"
$feedIdURL = "$feedBaseURL/$feedName/?api-version=5.1-preview.1"
$feedIdResponse = (Invoke-RestMethod -Method Get -Uri $feedIdUrl -Headers $restAPICallHeader -ContentType 'application/json')
$feedId = $feedIdResponse.id
$viewIdURL = "$feedBaseURL/$feedId/views/$viewName/?api-version=5.1-preview.1"
$viewIdResponse = (Invoke-RestMethod -Method Get -Uri $viewIdUrl -Headers $restAPICallHeader -ContentType 'application/json')
$viewId = $viewIdResponse.id
$restAPICallBodyJson = #{
views = #{
op = 'add'
path = '/views/-'
value = "$viewId"
}
}
$restAPICallBody = (ConvertTo-Json $restAPICallBodyJson)
foreach ($packageName in $packagesToPromote) {
$packageQueryUrl = "$feedBaseURL/$feedId/packages?api-version=5.1-preview.1&packageNameQuery=$packageName"
$packagesResponse = (Invoke-RestMethod -Method Get -Uri $packageQueryUrl -Headers $restAPICallHeader -ContentType 'application/json')
$latestPackageVersion = ($packagesResponse.value.versions | ? { $_.isLatest -eq $True } | Select -ExpandProperty version)
$encodedPackageVersion = [System.Web.HttpUtility]::UrlEncode($latestPackageVersion)
Write-Host "Package Name: $packageName"
Write-Host "Package Version: $latestPackageVersion"
$releaseViewURL = $packageBaseURL `
+ "/$($feedId)" `
+ "/nuget/packages/$packageName" `
+ "/versions/$encodedPackageVersion" `
+ "?api-version=5.1-preview.1"
$response = Invoke-RestMethod -Method Patch -Uri $releaseViewURL -Headers $restAPICallHeader -ContentType 'application/json' -Body $restAPICallBody
Write-Host $response
}
For reference, the script above uses the following API calls:
Feed Management - Get Feed
https://learn.microsoft.com/en-us/rest/api/azure/devops/artifacts/feed%20%20management/get%20feed?view=azure-devops-rest-5.1
Feed Management - Get Feed View
https://learn.microsoft.com/en-us/rest/api/azure/devops/artifacts/feed%20%20management/get%20feed%20view?view=azure-devops-rest-5.1
Artifact Details - Get Packages
https://learn.microsoft.com/en-us/rest/api/azure/devops/artifacts/artifact%20%20details/get%20packages?view=azure-devops-rest-5.1
NuGet - Update Package Version
https://learn.microsoft.com/en-us/rest/api/azure/devops/artifactspackagetypes/nuget/update%20package%20version?view=azure-devops-rest-5.1
As per Azure DevOps documentation the marketplace task Promote package to Release View is the recommended way to accomplish this from a CI/CD pipeline.
The repository can be found on Github.
Edit:
Since you are on-prem with a version that this task doesn't support. I would say that the comments about using the REST api would be the route you need to go in something like a powershell script.
Having never used the REST Api for this task I'm not exactly sure how the body is supposed to look for the request. However, it seems to be documented here.
My understanding of the JSON Patch object is limited, but I would think you might use the replace operation.
{ "op": "replace", "path": "/view", "value": "#Release" }
This article may also be helpful, but I still don't see anything that would relate to the from identifier on the JsonPatchObject definition in the REST Api documentation.
I also recently struggled with trying to implement version using TFS. I've produced some PowerShell scripts (adapting other scripts out on the web) to do package versioning.
https://gist.github.com/oceanexplorer/6a91930419b35c1923974af265777a5f
https://gist.github.com/oceanexplorer/35e0f26962018dc8578c745060365c15
The first step is my build pipeline I use the "Update AssemblyInfo" task to set the build version which then gets embedded into the DLL's.
https://marketplace.visualstudio.com/items?itemName=sebastianlux.UpdateAssemblyInfo
Initially I embedded the above scripts with my project to get things going but eventually in my release pipeline I then have a task that deploys these build scripts via a "NuGet Install" task which effectively pulls them from a feed and unzips them.
In the release pipeline I then have a task "Version Package" which is a custom PowerShell script that calls functions defined in the two gists above, what these do is to unzip the NuGet packages that have been created from the build pipeline and placed in the artifact directory, applies the correct versioning to the package and zips it back up. I have used the following build number format in my build pipeline:
$(version.major).$(version.minor).$(version.patch).$(Date:yyyyMMdd)$(Rev:r)-CI
1.0.0.201902051-CI
This will produce a semantic build number format of:
1.0.0-alpha.201902051
I call the scripts using an inline PowerShell task
##-------------------------------------------
## Import Build Scripts
##-------------------------------------------
gci -Recurse "$(System.DefaultWorkingDirectory)\scripts\*.psm1" | ForEach-Object { Import-Module $_.FullName }
##-------------------------------------------
## Version Files
##-------------------------------------------
Expand-NugetPackages -packagesDirectory "$(artifact.directory)" -Verbose
Add-VersionToAssemblies -suffix "$(Release.EnvironmentName)" -semVer "2.0" -artifactsToApplyTo "nuspec" -isRelease $(isRelease) -Verbose
Compress-NugetPackages -packagesDirectory "$(artifact.directory)" -Verbose
Then a NuGet push task to push the package
Them another inline PowerShell script which sets the release view for the package feed:
##-------------------------------------------
## Import Build Scripts
##-------------------------------------------
gci -Recurse "$(System.DefaultWorkingDirectory)\scripts\*.psm1" | ForEach-Object { Import-Module $_.FullName }
##-------------------------------------------
## Set Package Quality
##-------------------------------------------
Set-PackageQuality -feedName "Libraries" -packageId $(nuget.packageId) -packageVersion $env:semanticVersion -packageQuality $(Release.EnvironmentName)
Something changed recently in the ADO RestApi.
What worked for me was sending a PATCH request like:
curl --location --request PATCH 'https://pkgs.dev.azure.com/YOUR_ORGANIZATION/YOUR_PROJECT/_apis/packaging/feeds/YOUR_FEEDNAME/upack/packages/YOUR_PACKAGENAME/versions/YOUR_PACKAGEVERSION?api-version=5.1-preview.1' \
--header 'Authorization: Basic YOUR_TOKEN' \
--header 'Content-Type: application/json' \
--data-raw '{
"views": {
"op": "add",
"path": "/views/-",
"value": "Release"
}
}'