I have around 80 DLL's for which builds are created and pushed to Nuget using Azure Devops.
I am trying to create release pipeline by seggrigating different builds into different release pipelines.
But before creating I would like to identify if other release pipeline exist which using these builds as artifacts?
How could I identify if a release pipelines exits which already uses above builds.
You will want to use the API for Release Pipelines and pass the artifacts expand option. I'll leave you to figure out the authentication piece (there are other answers that cover it).
Here's an example in PowerShell:
$uri = 'https://vsrm.dev.azure.com/{organization}/{project}/_apis/release/definitions?api-version=5.1&$expand=artifacts'
Invoke-RestMethod -Method get -uri $uri -UseDefaultCredentials | Select-Object -ExpandProperty value | Select-Object id, artifacts
Related
I am working on creating qnamaker based Chatbot and for this I have prepared ARM template for creating and deploying resources.
PowerShell script is being used to create knowledge base once all the resources are created.
This process works just fine when I am executing PowerShell script from local PowerShell tool.
Now I want to do CI/CD for this deployment process so that deployment process can be automated
Creating resources and deploying to azure through Pipeline is quite possible through Azure ARM Template deployment task but I am not sure how to execute PowerShell script that is responsible for creating knowledge base once resources are created based on recently created qnamaker service
Any help greatly appreciated
You can refer to Create a CI/CD pipeline for .NET with the Azure DevOps Project for step by step lab to create the pipelines.
If you need help on ARM templates (if you are stuck somewhere), you can check out A quick guide on writing your Azure Resource Manager (ARM) Templates. and get started quickly.
Please clarify that your issue is creating and training the QnA Maker KB after the actual resources have been created by ARM? There are probably multiple ways to do this, but here is what I have done. First of all, I am using the Azure CLI Task version 2.* which simplifies some of the initial steps. You could probably use regular PowerShell for some of the subsequent steps. I have selected all as PowerShell script types and inline scripts. Each of these segments I do as a separate task but you could probably do all as one. I like being able to see which step fails if something happens, though.
First you need to get the key for your cognitive service
$QNAKEY= & az cognitiveservices account keys list -g "YOUR_RESOURCE_GROUP" --name "YOUR_QNA_RESOURCE_NAME" --query key1 -o tsv
Write-Output("##vso[task.setvariable variable=QNA_KEY;]$QNAKEY")
Next you need to create the KB. I have mine seeded with information via a json file in my project repository. You can do this from wherever, or just create a blank KB (I think). Note I'm checking here to see if the KB exists; KB name does not have to be unique, so if you don't do this you will end up creating a lot of duplicate KBs.
$header = #{
"Content-Type"="application/json"
"Ocp-Apim-Subscription-Key"="$(QNA_KEY)"
}
$res = Invoke-RestMethod -Uri "https://westus.api.cognitive.microsoft.com/qnamaker/v4.0/knowledgebases" -Method 'Get' -Headers $header
$kb = $res.knowledgebases | Where-Object name -eq "YOUR_KB_NAME"
Write-Host $res.knowledgebases
Write-Host $kb
if (!$kb) {
Write-Host "KB does not exist, so creating new KB"
$body = Get-Content '$(System.DefaultWorkingDirectory)/PATH_TO_MY_MODEL.json' | Out-String
Invoke-RestMethod -Uri "https://westus.api.cognitive.microsoft.com/qnamaker/v4.0/knowledgebases/create" -Method 'Post' -Body $body -Headers $header
}
Finally, you will likely want to publish your KB. With LUIS I think you need to train it first (via separate CLI task), but QnA Maker you should be able to publish directly. I do this as a dependent stage to ensure the KB is created before I try to publish.
$QNAKEY= & az cognitiveservices account keys list -g "YOUR_RESOURCE_GROUP" --name "YOUR_QNA_RESOURCE_NAME" --query key1 -o tsv
$header = #{
"Content-Type"="application/json"
"Ocp-Apim-Subscription-Key"="$QNAKEY"
}
$res = Invoke-RestMethod -Uri "https://westus.api.cognitive.microsoft.com/qnamaker/v4.0/knowledgebases" -Method 'Get' -Headers $header
$kb = $res.knowledgebases | Where-Object name -eq "YOUR_KB_NAME"
$qnaId = $kb.id
Invoke-RestMethod -Uri "https://westus.api.cognitive.microsoft.com/qnamaker/v4.0/knowledgebases/$qnaId" -Method 'Post' -Headers $header
And that's it! At this point your QnA Maker knowledgebase should be created, published, and ready to go.
Did someone manage to integrate Azure DevOps in Sentry (sentry.io)? I stuck on "Associate commits with a Release" (see: https://docs.sentry.io/workflow/releases/?platform=browser#associate-commits-with-a-release)
I can not figure out a way how I can tell Sentry (by API) which commit ids are associated with a current release/deploy. How can I add a task to the pipeline which will post the commit ids to Sentry API? Or is there some other way to do it?
In azure devops, the Powershell task also support curl. So, you can execute the api in powershell task of VSTS pipeline directly.
In release pipeline, there has a pre-defined release variable, it stores the commit id which is associated with the current release pipeline: $(Release.Artifacts.{alias}.SourceVersion). Here alias is the artifacts name, and you can get it by getting $(Release.PrimaryArtifactSourceAlias).
First, create variables like this:
Then you can apply the variable $(id) into that API, and execute the api in powershell task:
"refs": [{
"commit":"$(id)"
}]
Now, the commit id could be packed into the body of this api, and send to the Sentry server.
If there has multiple commits associate with this release, since the variable $(Release.Artifacts.{alias}.SourceVersion) I mentioned above only store the latest commit message, here you may need add additional scripts to get what you want by Build id.
In release pipeline, with $(Build.BuildId) you can get the corresponding buildid which associate with this release. And then, you could get the commits(changes) by using this API:
GET https://dev.azure.com/{organization}/{project}/_apis/build/changes?fromBuildId={fromBuildId}&toBuildId={toBuildId}&api-version=5.1-preview.2
You could apply these powershell script into your task without change anything because this script is universal among powershell-ise, powershell command line and powershell task in VSTS.
$token = "{PAT token}"
$url="https://dev.azure.com/{org name}/{project name}/_apis/build/changes?fromBuildId={id1}&toBuildId={id2}"
$token = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$($token)"))
$response = Invoke-RestMethod -Uri $url -Headers #{Authorization = "Basic $token"} -Method Get
Write-Host "results = $($response.value.id | ConvertTo-Json -Depth 100)"
Now, you could get the list of commits which associate with the build and corresponding release.
Our Azure DevOps build agents are setup on Kubernetes. Failed pods can easily be deleted from kube, but they appear as "offline" agents from the Azure DevOps Web UI.
Overtime the list of offline agents has grown really long. Is there a way to programmatically delete them ?
$agents = Invoke-RestMethod -uri 'http://dev.azure.com/{organization}/_apis/distributedtask/pools/29/agents' -Method Get -UseDefaultCredentials
$agents.value |
Where-Object { $_.status -eq 'offline' } |
ForEach-Object {
Invoke-RestMethod -uri "http://dev.azure.com/{organization}/_apis/distributedtask/pools/29/agents/$($_.id)?api-version=4.1" -Method Delete -UseDefaultCredentials
}
Some assumptions for this solution:
You are looking for build agents
You know the id of the pool you are looking for already. You can get to this programatically also, or just loop through all pools if you want
You don't have any issues deleting any offline agents
Note: I'm using Azure DevOps Server, so replace the -UseDefaultCredentials with your authorization.
I think you would need to use a combination of these 2 api calls:
Get deployment groups
Delete agent
Or you can use some naming matrix for agents, so they are using the same "name" even if it is not the same container.
https://github.com/julioct/azure-pipelines-kubernetes-agents
I am using TFS for source and build of my project. After successful build I want to automate the tagging of build using powershell or any TFS task. And after that I wanted to use the same tag to deploy in specific environment. Can you guide me how can use this?
Assuming you are using VNext build, you can use the REST API to add tags during the build, then set a CD release to deploy the tagged artifacts.
PowerShell sample:
$buildid= $env:BUILD_BUILDID
$TagName= $env:BUILD_DEFINITIONNAME + $env:BUILD_BUILDID
$Uri = "http://mo-12r2:8080/tfs/CollectionLC/0418Scrum/_apis/build/builds/$buildid/tags/$($TagName)?&api-version=2.0"
$buildresponse = Invoke-RestMethod -Method Put -UseDefaultCredentials -ContentType application/json -Uri $Uri
Write-Host "BuildID:" $buildid
Write-Host "TagName:" $TagName
You can follow below steps to do that:
Create a PowerShell script and check in to server, See above sample
Create a build definition and add a PowerShell task as the end task
Select the PowerShell script to run
Create a Release definition with Continuous Deployment enabled
and link the build definition created in step2 as the artifacts. (You can also select any tagged build to deploy if you set manual release.)
In order to automate our deployments, I would like to rebuild an application, based on a given ChangeSetId. Once this build has completed, I want to get the artifacts of the build (the .exe), so we can deploy them. For the sake of the question I am focussing on the 'Get artifacts from build' part.
For DevOps purposes, I'd like to use PowerShell, since it should be able to access the TFS API libraries and because MS recommends using it.
Environment
I've set up Builds in our On Premise TFS 2015 server (which are working neatly) - and added a VSO task 'Publish artifacts' after this build. So far so good.
The published artifacts is are to be stored on the Server, which basically means I have to download the artifacts connected to build - every existing build will have its artifacts linked - which is better then an UNC drop in my book.
No comes my challenge; how do I programmaticaly access these artifacts, step 3?
Get Sources for ChangeSetId
MSBuild application with given configuration
Get build Artifacts using PowerShell
Deploy to environment using Release Management (Powershell as well)
TFS 2015 comes with the new REST API, and it includes the method to get the artifacts of the specific build. I would approach your challenge in the following way:
Add a "PowerShell script" build step after your "Publish artifacts" step
In that PowerShell script:
Get the ID of the current build. TFS exposes a number of predefined variables, and build ID is there. All those variables end up as environment variable, and this post can help you read the appropriate one from your PowerShell script
Next, make a web request to get build artifacts. As you can see from the API description, you'll have to provide just the build ID
Then, parse the JSON response - the downloadUrl property contains the link to download all the artifacts of the build zipped as a single archive
Finally, extract the archive and pick up those artifacts you need. Perhaps, you'd like to deploy it to your testing environment in this step as well
Hope this helps.
Okay so, like Yan Sklyarenko said, TFS 2015 (and 2013, after some update), has an excellent REST API.
I've created a very, very rough basic PowerShell script that does what i want. I cannot emphasize enough how much this code needs refactoring - i really just needed this to work as a proof of concept, and we will develop multiple scripts for different needs, but for the people that came here for a code example, you'll find it here.
Connect to TFS' Build system
List Build Definition items (for myself, Poc)
Search for some string and get Build ID
Kick off a build, using a hard coded ID 7 (because I knew this was going to work, and therefor my work was done)
Get the Artifacts (in which I incorporated the VSO build task 'Publish Artifacts Server')
Extract said received Artifacts, because TFS zips them.
From there on out i will incorporate these scripts and outputs into MS Release Management services - and be ready to migrate to VSO Release vNext when it ships for on-premise TFS 2015!
$projectId ='{ProjectIdGuid}'
$buildNr = '3945'
$username = 'username'
$password = 'password'
$zipDestination = 'C:\temp\unzip\temp.zip'
$workingFolder = ('C:\temp\unzip\' + [System.DateTime]::Now.ToString("yyyyMMddhhmmss")) #temp because of file already exist warnings... after completion we should delete the working directory content
$tfsURL = 'http://myTFS:8080/tfs/MyCollection/'+ $projectId
$cred = New-Object System.Management.Automation.PSCredential($username, (ConvertTo-SecureString -String $password -AsPlainText -Force))
#write list of build definitions (to be used later)
$allbuildDefs = (Invoke-RestMethod -Uri ($tfsURL + '/_apis/build/definitions?api-version=2.0') -Method GET -Credential $cred).value | Where-Object {$_.name -like '*buildName*'} | Out-Default | select name
Write-Host($allbuildDefs)
$buildDefs = ConvertFrom-Json($allbuildDefs)
$buildId = ($buildDefs.value).id;
#Get build Definition for what you want to build
$buildDefinitionURI = $tfsURL + '/_apis/build/requests?api-version=1.0'
#kick off build
$body = '{ "definition": { "id": '+ 7 + '}, reason: "Manual", priority: "Normal"}'
$BuildReqBodyJson = $body | ConvertTo-Json
$buildOutput = Invoke-RestMethod -Method Post -Uri $buildDefinitionURI -Credential $cred -ContentType 'application/json' -Body $body
#get buildNr
#build URI for buildNr
$BuildURI = $tfsURL + '/_apis/build/builds/' + $buildNr + '/artifacts'
#get artifact downloadPath
$downloadURL = (Invoke-RestMethod -Uri $BuildURI -Credential $cred).Value.Resource.downloadUrl
#download ZIP
Invoke-WebRequest -uri $downloadURL -Credential $cred -OutFile $zipDestination
#unzip
Add-Type -assembly 'system.io.compression.filesystem'
[io.compression.zipfile]::ExtractToDirectory($zipDestination, $workingFolder)