TFS: Best way to trigger build on server restart, or on Windows Updates installation - powershell

In short, the requirement is to verify that our latest released software can be built and then installed after the latest Windows updates and/or other patches were applied. So the build server VM(s) will be configured just for this purpose and the build only needs to run after an update.
Since such updates usually are followed with a restart, I am thinking of a server restart event triggering a build and deployment. Does such option exist in TFS 2017?
If there is no way to do it through TFS then, I guess, a PowerShell script that runs on startup should work?

No such a build-in function to achieve that. However create a PowerShell script that runs on startup should work. Just as Jessehouwing said, you can create the script with the REST API to trigger builds.
Create a script to trigger the specific build definition. (Reference below sample)
Run the script on startup:
How to run a batch file each time the computer boots
How to schedule a Batch File to run automatically in Windows
10/8/7
Param(
[string]$collectionurl = "http://server:8080/tfs/DefaultCollection",
[string]$projectName = "ProjectName",
[string]$keepForever = "true",
[string]$BuildDefinitionId = "34",
[string]$user = "username",
[string]$token = "password"
)
# Base64-encodes the Personal Access Token (PAT) appropriately
$base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f $user,$token)))
function CreateJsonBody
{
$value = #"
{
"definition": {
"id": $BuildDefinitionId
},
"parameters": "{\"system.debug\":\"true\",\"BuildConfiguration\":\"debug\",\"BuildPlatform\":\"x64\"}"
}
"#
return $value
}
$json = CreateJsonBody
$uri = "$($collectionurl)/$($projectName)/_apis/build/builds?api-version=2.0"
$result = Invoke-RestMethod -Uri $uri -Method Post -Body $json -ContentType "application/json" -Headers #{Authorization=("Basic {0}" -f $base64AuthInfo)}

There is no existing trigger that handles this, but there is a simple REST API to query and trigger builds.
It would be easy to create an on startup job in the task scheduler, use the REST API to query a list of Build Definitions based on a certain name or tag and then queue it.
List build definitions
Queue a build

Related

Using semaphore-like conditions for pipeline job queueing

we use floating licenses for our more expensive compilers/tools, so that we can do local development as well as our production builds. The license manager (flexLM) has an api we can query, so we could block the license. However, I cannot find a mechanism by which I can cause my pipeline to queue based on the state of an auxiliary variable or return value of a script or something like that.
This means that I can launch the build on any machine on which the compiler is installed but it will then fail if the license is not available and I will have to relaunch the pipeline. If I did that automatically I would effectively just block that machine until the license becomes available.
Is there anything I have missed that could achieve a "queueing until license becomes available" kind of thing?
Thank you,
Manuel
We can add the first task power shell in the pipeline definition and define new variable in the variable tab such as Value:true, then add script to check the license status, if the license is available, set the variable Value to true, if the license is not available, set the variable Value to False. Then add condition eq(variables['{variable name}'], '{variable value}') in the Second task.
After configure, if your license is available, the pipeline will run successfully.
Or we could check the license first, then call the below script to queue build pipeline.
$token = "$(pat)"
$url = "https://dev.azure.com/{Org name}/{project name}/_apis/build/builds?api-version=6.1-preview.6"
$token = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$($token)"))
$JSON = #"
{
"definition": {
"id": {Build definition ID}
}
}
"#
$response = Invoke-RestMethod -Uri $url -Headers #{Authorization = "Basic $token"} -Method Post -ContentType application/json -body $JSON
write-host $response

Azure Devops - Azure powershell task find release/deployment directory

I'm running Azure devops where i have a pipeline and a release (running on a self hosted agent), and the release is set to send up to an azure app service. The deployment works fine, the only issue is that i'd also like to be able to (based on some of my release variables) edit the web.config of the site AFTER it has already been deployed to the azure website.
I'm using the Azure Powershell task ( https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/deploy/azure-powershell?view=azure-devops ) ,and i can't find anywhere in the release variables ( https://learn.microsoft.com/en-us/azure/devops/pipelines/release/variables?view=azure-devops&tabs=batch ) that shows the directory of where the site exists. Looking through Kudu, it's pretty basic, like d:\home\site\wwwroot\ , but using that doesn't work at all.
Is this post config that i'm looking for not really possible, or should i be approaching it a different way?
I think you can use powershell task to call Kudu api to get the deployed web.config and edit it using Magic Chunks task or File Transform task in your release pipeline. Then using Kudu api to upload the changed web.config to azure website again.
1, Below script shows how to get web.config from azure website.
$srcResGroupName = "Test"
$srcWebAppName = "tstest12"
$outwebconfig="$(System.DefaultWorkingDirectory)\tempfolder\web.config"
# Get publishing profile for SOURCE application
$srcWebApp = Get-AzWebApp -Name $srcWebAppName -ResourceGroupName $srcResGroupName
[xml]$publishingProfile = Get-AzWebAppPublishingProfile -WebApp $srcWebApp
# Create Base64 authorization header
$username = $publishingProfile.publishData.publishProfile[0].userName
$password = $publishingProfile.publishData.publishProfile[0].userPWD
$base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f $username,$password)))
$apiBaseUrl = "https://$($srcWebApp.Name).scm.azurewebsites.net/api/vfs/site/wwwroot/web.config"
# Download the web.config file to $outwebconfig
Invoke-RestMethod -Uri "$apiBaseUrl" `
-Headers #{UserAgent="powershell/1.0"; `
Authorization=("Basic {0}" -f $base64AuthInfo)} `
-Method GET `
-OutFile $outwebconfig
Above script will download the web.config from the azure website and save it to $(System.DefaultWorkingDirectory)\tempfolder\web.config, where you can edit it with transform task later.
Above scripts get the username and password with scripts, you can also get them in the publish profile by going to the Overview blade on your App Service, clicking ...More at the top of the blade, and then clicking Get publish profile
2, Then you can add a config transform task to change your web.config according.
3, Last add a powershell task to upload the changed web.config to azure website
$srcResGroupName = "Test"
$srcWebAppName = "tstest12"
$webconfig="$(System.DefaultWorkingDirectory)\tempfolder\web.config"
# Get publishing profile for SOURCE application
$srcWebApp = Get-AzWebApp -Name $srcWebAppName -ResourceGroupName $srcResGroupName
[xml]$publishingProfile = Get-AzWebAppPublishingProfile -WebApp $srcWebApp
# Create Base64 authorization header
$username = $publishingProfile.publishData.publishProfile[0].userName
$password = $publishingProfile.publishData.publishProfile[0].userPWD
$base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f $username,$password)))
$apiUrl = "https://$($srcWebApp.Name).scm.azurewebsites.net/api/vfs/site/wwwroot/web.config";
Invoke-RestMethod -Uri $apiUrl -Headers #{Authorization=("Basic {0}" -f $base64AuthInfo)} -UserAgent $userAgent -Method PUT -InFile $webconfig -ContentType "application/xml";
For more usage of Kudu api you can check here.

How to add a new dynamically discoverable capability to an agent?

Updates
I am going to propose a Capabilities Provider here as an update to my post.
If you need more details please let me know.
We currently have a bunch of shipped Capabilities Providers in the agent source code:
https://github.com/microsoft/azure-pipelines-agent/tree/master/src/Microsoft.VisualStudio.Services.Agent/Capabilities
Agent
Environment
Nix
PowerShell
What is being proposed is one additional Provider named ExecutableCapabilitiesProvider.
This new ExecutableCapabilitiesProvider will probably have a config file which can be edited on the agent machine.
The format of this file could probably be:
#name,executable
pip,pip3 freeze
xyz,/usr/bin/xyz-runner
abc,sh -C "ls -l /blah/blah"
As the maintainer of the self-hosted pool, I would configure this file with entries suiting me and have the agent run it as it starts. This way I am not hard-coding any values for my capabilities but rather those be determined at the start up.
And I would go one step further and add a new API call to add capabilities which is more flexible than the current one asking for name/values. An example, would be to change the parameters to Name, Provider, Params:
efg, NixProvider, /path/to/file/efg
klm, ExecutableCapabilitiesProvider, /usr/bin/klm -a -b -c
Original Post
I'd like to make my agents report on new capabilities which are not static but rather result of a command or something similar? How can I do that?
Our agents run on linux boxes.
To be specific, I'd like to have a new capability called pip-packages and the value for that is the result of the command pip freeze executed on the shell.
If you mean to add User-defined capabilities, then you can write a script to call the REST API to update the agent capabilities.
PUT https://dev.azure.com/{organization}/_apis/distributedtask/pools/{poolid}/agents/{agentid}/usercapabilities?api-version=5.0
Request body:
{"pip-packages": "xxxx"}
For example, you can set a variable and run command pip freeze and export the response as the value of that variable, then update the agent capability by calling the REST API:
Below PowerShell sample for your reference :
Param(
[string]$collectionurl = "https://dev.azure.com/{organization}",
[string]$poolid = "14",
[string]$agentid = "16",
[string]$user = "user",
[string]$token = "PAT/Password"
)
# Base64-encodes the Personal Access Token (PAT) appropriately
$base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f $user,$token)))
# Run pip freeze command and get response as the value of the variable $pipfreeze (Just for your reference here, you need to extract the value with running the commands)
$pipfreeze = "response of pip freeze"
# Create json body with that value
$baseUri = "$collectionurl/_apis/distributedtask/pools/$poolid/agents/$agentid/usercapabilities?api-version=5.0"
function CreateJsonBody
{
$value = #"
{"pip-packages":"$pipfreeze"}
"#
return $value
}
$json = CreateJsonBody
# Update the Agent user capability
$agentcapability = Invoke-RestMethod -Uri $baseUri -Method Put -Body $json -ContentType "application/json" -Headers #{Authorization=("Basic {0}" -f $base64AuthInfo)}
write-host "=========================================================="
Write-host "userCapabilities :" $agentcapability.userCapabilities.'pip-packages'
write-host "=========================================================="

How can one automate promotion of an artifact to a feed view in Azure DevOps?

Our build artifact is an Octopus nuget package. When the build is released it lands into the QA stage where the artifact is deployed through Octopus. This octopus consumes it directly from the Azure Artifacts nuget feed.
If the deployment and the subsequent tests are successful we want to promote the artifact to the Release view of the Azure Artifacts nuget feed, because we think it gives us a different nuget URL that can be used by another Octopus serving the next stage (for historic reasons we have dedicated Octopus per stage - working to change that, but it takes time).
We can promote manually, but we want to do it automatically. How can this be done?
We are testing it on on-premises TFS 2019 RC2.
EDIT 1
The suggested plugin does not seem to install on on-premises TFS 2019 RC2:
Using PowerShell...
$organisationName = '' # Name of organisation
$projectName = '' # Name of project
$feedName = '' # Name of Azure Artifacts feed
$viewName = 'Release' # I believe this can also be Prerelease, but I've not tried it
# List of names of packages within Azure Artifacts feed to be promoted
$packagesToPromote = #('')
# Need a personal access token for this script to work
# PAT token should be assigned to Packaging (Read, Write and Manage) scopes
$azureArtifactsPAT = ''
$AzureArtifactsPAT_Base64 = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(":$($azureArtifactsPAT)"))
$restAPICallHeader = #{ Authorization = "Basic $AzureArtifactsPAT_Base64" }
$feedBaseURL = "https://feeds.dev.azure.com/$organisationName/$projectName/_apis/packaging/feeds"
$packageBaseURL = "https://pkgs.dev.azure.com/$organisationName/$projectName/_apis/packaging/feeds"
$feedIdURL = "$feedBaseURL/$feedName/?api-version=5.1-preview.1"
$feedIdResponse = (Invoke-RestMethod -Method Get -Uri $feedIdUrl -Headers $restAPICallHeader -ContentType 'application/json')
$feedId = $feedIdResponse.id
$viewIdURL = "$feedBaseURL/$feedId/views/$viewName/?api-version=5.1-preview.1"
$viewIdResponse = (Invoke-RestMethod -Method Get -Uri $viewIdUrl -Headers $restAPICallHeader -ContentType 'application/json')
$viewId = $viewIdResponse.id
$restAPICallBodyJson = #{
views = #{
op = 'add'
path = '/views/-'
value = "$viewId"
}
}
$restAPICallBody = (ConvertTo-Json $restAPICallBodyJson)
foreach ($packageName in $packagesToPromote) {
$packageQueryUrl = "$feedBaseURL/$feedId/packages?api-version=5.1-preview.1&packageNameQuery=$packageName"
$packagesResponse = (Invoke-RestMethod -Method Get -Uri $packageQueryUrl -Headers $restAPICallHeader -ContentType 'application/json')
$latestPackageVersion = ($packagesResponse.value.versions | ? { $_.isLatest -eq $True } | Select -ExpandProperty version)
$encodedPackageVersion = [System.Web.HttpUtility]::UrlEncode($latestPackageVersion)
Write-Host "Package Name: $packageName"
Write-Host "Package Version: $latestPackageVersion"
$releaseViewURL = $packageBaseURL `
+ "/$($feedId)" `
+ "/nuget/packages/$packageName" `
+ "/versions/$encodedPackageVersion" `
+ "?api-version=5.1-preview.1"
$response = Invoke-RestMethod -Method Patch -Uri $releaseViewURL -Headers $restAPICallHeader -ContentType 'application/json' -Body $restAPICallBody
Write-Host $response
}
For reference, the script above uses the following API calls:
Feed Management - Get Feed
https://learn.microsoft.com/en-us/rest/api/azure/devops/artifacts/feed%20%20management/get%20feed?view=azure-devops-rest-5.1
Feed Management - Get Feed View
https://learn.microsoft.com/en-us/rest/api/azure/devops/artifacts/feed%20%20management/get%20feed%20view?view=azure-devops-rest-5.1
Artifact Details - Get Packages
https://learn.microsoft.com/en-us/rest/api/azure/devops/artifacts/artifact%20%20details/get%20packages?view=azure-devops-rest-5.1
NuGet - Update Package Version
https://learn.microsoft.com/en-us/rest/api/azure/devops/artifactspackagetypes/nuget/update%20package%20version?view=azure-devops-rest-5.1
As per Azure DevOps documentation the marketplace task Promote package to Release View is the recommended way to accomplish this from a CI/CD pipeline.
The repository can be found on Github.
Edit:
Since you are on-prem with a version that this task doesn't support. I would say that the comments about using the REST api would be the route you need to go in something like a powershell script.
Having never used the REST Api for this task I'm not exactly sure how the body is supposed to look for the request. However, it seems to be documented here.
My understanding of the JSON Patch object is limited, but I would think you might use the replace operation.
{ "op": "replace", "path": "/view", "value": "#Release" }
This article may also be helpful, but I still don't see anything that would relate to the from identifier on the JsonPatchObject definition in the REST Api documentation.
I also recently struggled with trying to implement version using TFS. I've produced some PowerShell scripts (adapting other scripts out on the web) to do package versioning.
https://gist.github.com/oceanexplorer/6a91930419b35c1923974af265777a5f
https://gist.github.com/oceanexplorer/35e0f26962018dc8578c745060365c15
The first step is my build pipeline I use the "Update AssemblyInfo" task to set the build version which then gets embedded into the DLL's.
https://marketplace.visualstudio.com/items?itemName=sebastianlux.UpdateAssemblyInfo
Initially I embedded the above scripts with my project to get things going but eventually in my release pipeline I then have a task that deploys these build scripts via a "NuGet Install" task which effectively pulls them from a feed and unzips them.
In the release pipeline I then have a task "Version Package" which is a custom PowerShell script that calls functions defined in the two gists above, what these do is to unzip the NuGet packages that have been created from the build pipeline and placed in the artifact directory, applies the correct versioning to the package and zips it back up. I have used the following build number format in my build pipeline:
$(version.major).$(version.minor).$(version.patch).$(Date:yyyyMMdd)$(Rev:r)-CI
1.0.0.201902051-CI
This will produce a semantic build number format of:
1.0.0-alpha.201902051
I call the scripts using an inline PowerShell task
##-------------------------------------------
## Import Build Scripts
##-------------------------------------------
gci -Recurse "$(System.DefaultWorkingDirectory)\scripts\*.psm1" | ForEach-Object { Import-Module $_.FullName }
##-------------------------------------------
## Version Files
##-------------------------------------------
Expand-NugetPackages -packagesDirectory "$(artifact.directory)" -Verbose
Add-VersionToAssemblies -suffix "$(Release.EnvironmentName)" -semVer "2.0" -artifactsToApplyTo "nuspec" -isRelease $(isRelease) -Verbose
Compress-NugetPackages -packagesDirectory "$(artifact.directory)" -Verbose
Then a NuGet push task to push the package
Them another inline PowerShell script which sets the release view for the package feed:
##-------------------------------------------
## Import Build Scripts
##-------------------------------------------
gci -Recurse "$(System.DefaultWorkingDirectory)\scripts\*.psm1" | ForEach-Object { Import-Module $_.FullName }
##-------------------------------------------
## Set Package Quality
##-------------------------------------------
Set-PackageQuality -feedName "Libraries" -packageId $(nuget.packageId) -packageVersion $env:semanticVersion -packageQuality $(Release.EnvironmentName)
Something changed recently in the ADO RestApi.
What worked for me was sending a PATCH request like:
curl --location --request PATCH 'https://pkgs.dev.azure.com/YOUR_ORGANIZATION/YOUR_PROJECT/_apis/packaging/feeds/YOUR_FEEDNAME/upack/packages/YOUR_PACKAGENAME/versions/YOUR_PACKAGEVERSION?api-version=5.1-preview.1' \
--header 'Authorization: Basic YOUR_TOKEN' \
--header 'Content-Type: application/json' \
--data-raw '{
"views": {
"op": "add",
"path": "/views/-",
"value": "Release"
}
}'

Queueing TFS builds via PowerShell

TFS2012 with a one 2010 build controller with one 2010 build agent. Also have one 2012 build controller with multiple 2012 build agents.
We have multiple builds for multiple versions of our software. The builds are named according to a convention e.g. Foo_version_1_0 and Foo_version_2_0.
When I run this code on my local machine, all the builds are queued. When I run this code manually on a 2012 build agent, the builds are queued. When I run this code manually on a 2010 build agent, no builds are queued. When the code is executed as part of a triggered build in TFS (either on the 2010 or 2012 controller/agent), it doesn't queue any builds and errors out with my custom exception saying no definitions returned from TFS.
My questions:
Is the $buildServer.QueryBuildDefinitions() function an administrator function only? I.e. if a non-admin user account (like TFSService) runs it, it won't be able to get the data from the TFS api?
Is the $buildServer.QueryBuildDefinitions() a new function that is only available in 2012?
Is there another way of doing this that will work? Previously, we had all our build names hard coded - this is not a viable way forward for us.
[void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.TeamFoundation.Client")
[void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.TeamFoundation.Build.Client")
$serverName="http://tfs:8080/tfs"
$tfs = [Microsoft.TeamFoundation.Client.TeamFoundationServerFactory]::GetServer($serverName)
$buildserver = $tfs.GetService([Microsoft.TeamFoundation.Build.Client.IBuildServer])
$buildServer.QueryBuildDefinitions("FooProject") | foreach {
if ($_.EndsWith("version_1_0"))
{
echo "Queueing build: $_.Name"
$buildServer.QueueBuild($buildServer.GetBuildDefinition("FooProject",$_.Name))
}
}
}
Edit: removed $buildDefinitions = $buildServer.QueryBuildDefinitions("FooProject").Name, replaced it with $buildServer.QueryBuildDefinitions("FooProject") | foreach...
Builds are now queued programmatically.
The API hasn't changed, and I suppose that both agents are using the same account.
The line
$buildDefinitions = $buildServer.QueryBuildDefinitions("FooProject").Name
seems wrong: the Name property get will raise an exception for an empty result.
You can also utilize one of the built in APIs, to prevent downloading dll binaries.
The following will work for
TFS 2017:
https://github.com/sameer-kumar/adhoc-posh/blob/master/QueueTfsBuild.ps1
$rootTfsUri = "http://myTFS:8080/tfs"
$collectionName = "Default"
$projectName = "Project1"
$tfsUri = $rootTfsUri + "/" + $collectionName + "/" + $projectName
$buildDefinition = "DevCI-vnext"
$buildDefinitionUri = "$tfsUri/_apis/build/definitions?api-version=3.1&name=$buildDefinition"
# first get build definition id
$buildResponse = Invoke-WebRequest -Uri $buildDefinitionUri -UseDefaultCredentials -Method Get -Verbose -UseBasicParsing -ContentType "application/json"
$buildResponseAsJson = $buildResponse.Content | convertfrom-json
$buildDefinitionId = $buildResponseAsJson.value.id
# Now queue this build definition
$requestContentString = #"
{
"definition": {
"id" : "$buildDefinitionId"
}
}
"#
$buildUri = "$tfsUri/_apis/build/builds?api-version=3.1"
$buildResponse = Invoke-WebRequest -Uri $buildUri -UseDefaultCredentials -Method Post -Verbose -UseBasicParsing -ContentType "application/json" -Body $requestContentString
$buildNumber = ($buildResponse.Content | ConvertFrom-Json).buildNumber
TFS 2015
Utilizes a slightly different structure where uri definition replace with this,
$buildDefinitionUri = "$tfsUri/_apis/Build/builds?api-version=2.0&name=$buildDefinition"