we use floating licenses for our more expensive compilers/tools, so that we can do local development as well as our production builds. The license manager (flexLM) has an api we can query, so we could block the license. However, I cannot find a mechanism by which I can cause my pipeline to queue based on the state of an auxiliary variable or return value of a script or something like that.
This means that I can launch the build on any machine on which the compiler is installed but it will then fail if the license is not available and I will have to relaunch the pipeline. If I did that automatically I would effectively just block that machine until the license becomes available.
Is there anything I have missed that could achieve a "queueing until license becomes available" kind of thing?
Thank you,
Manuel
We can add the first task power shell in the pipeline definition and define new variable in the variable tab such as Value:true, then add script to check the license status, if the license is available, set the variable Value to true, if the license is not available, set the variable Value to False. Then add condition eq(variables['{variable name}'], '{variable value}') in the Second task.
After configure, if your license is available, the pipeline will run successfully.
Or we could check the license first, then call the below script to queue build pipeline.
$token = "$(pat)"
$url = "https://dev.azure.com/{Org name}/{project name}/_apis/build/builds?api-version=6.1-preview.6"
$token = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$($token)"))
$JSON = #"
{
"definition": {
"id": {Build definition ID}
}
}
"#
$response = Invoke-RestMethod -Uri $url -Headers #{Authorization = "Basic $token"} -Method Post -ContentType application/json -body $JSON
write-host $response
Related
I've been scouring for the answer to this for the last several days. There's many fields labeled "status" in the TC API and for the life of me, I can't figure out which one returns the status listed above.
I'm currently calling...
$status = (Invoke-RestMethod -Uri %teamcity.serverUrl%/httpAuth/app/rest/builds/id:%teamcity.build.id% -Method Get -Headers $header).build.status
...within PowerShell. This returns SUCCESS unless the build has an execution timeout. However, as you can see from the screenshot above, the build failed. Obviously, the call above doesn't return the right status. The failure is due to a specific build step failing. If any of the build steps fail, then the entire build fails. Not sure if that helps!
The problem doesn't seem to be in the endpoint itself. You are requesting the correct field. The same can also be done with the following requests:
GET /app/rest/builds?locator=id:%teamcity.build.id%&fields=build(status)
GET /app/rest/builds/%teamcity.build.id%/status
The first request will return you an XML or JSON structure depending on your Accept header. The second response will return you plain text, so set the headers accordingly.
This returns SUCCESS unless the build has an execution timeout. However, as you can see from the screenshot above, the build failed. Obviously, the call above doesn't return the right status. The failure is due to a specific build step failing. If any of the build steps fail, then the entire build fails.
You are requesting the build status during the build execution. It is possible, that when the build step that checks the status is running, the status of the build is still SUCCESS. Try moving the build step to the very end of this build and configure it to execute 'Always, even if build stop command was issued'
Ok, so this is what was happening. I have an build step in TeamCity that runs Katalon (our test automation software). Even though the tests were failing, TeamCity was not updating the build status until after the last step, which happened to be my PS script that called the API then sent the info over to MS Teams. I had to setup a Failure Condition in TeamCity for the Katalon build step that looks in the build log for the specific text "(Katalon) failed". This immediately fails the build, but the rest of the build steps continue to run. Now, when my last step runs, it pulls the correct status and pushes it to MS Teams. For anyone that is interested, this is the full PowerShell script for that build step.
function Get-AuthenticationHeader
{
param($User, $Password)
$pair = "$($User):$($Password)"
$encodedCreds = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes($pair))
"Basic $encodedCreds"
}
$authHeader = Get-AuthenticationHeader -User "%system.teamcity.auth.userId%" -Password "%system.teamcity.auth.password%"
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls -bor [Net.SecurityProtocolType]::Tls11 -bor [Net.SecurityProtocolType]::Tls12
$header = #{
"Authorization"="$authHeader"
}
$response = (Invoke-RestMethod -Method GET -Headers $header -ContentType "application/json" -Uri "%teamcity.serverUrl%/httpAuth/app/rest/builds/%teamcity.build.id%")
$status = $response.build.status
$name = "%system.teamcity.buildConfName%"
$teamcityAgent = "%teamcity.agent.name%"
$testResults = $response.build.webUrl
Write-Host "%teamcity.build.id%"
Write-Host $status
Write-Host $name
Write-Host $testResults
if ( $status -eq "SUCCESS" ) {
$color = "00ff00"
}
else {
$status = "FAILURE"
$color = "ff0000"
}
$body = #{
title= "API Test Automation";
text= "<pre><b><i><font color=$color>$status</font></i></b> - $name<br>Tests ran on <i>$teamcityAgent</i><br>Test results are HERE</pre>"
} | ConvertTo-Json
Invoke-WebRequest -Method POST -ContentType "application/json" -Uri "<YOUR MS TEAMS WEBHOOK HERE>" -Body $body -UseBasicParsing
I have two release definitions release-def-1 and release-def-2 in azure devops.
I want to trigger a release in second one(release-def-2) once the deployment in first(release-def-1) succeeds.
Is there a way to achieve this?
I tried to find several trigger settings in release definition triggers but could not find anything relevant to what I need.
I know I can add multiple stages in the same release definition but that not what I am looking for.
As a workaround, you can add a Trigger Azure DevOps Pipeline task at the end of the job and set the task running conditions: Only when all previous tasks have succeeded. In this way, when the first release pipeline is successfully deployed, the second release pipeline will be automatically triggered.
We could not configure it in the release trigger.
As a workaround, we could open release-def-1 add task power shell at the end of the job and call the REST API to trigger release pipeline(release-def-2).
Power shell script:
$token = "{PAT}"
$url = "https://vsrm.dev.azure.com/{Org name}/{Project name}/_apis/Release/releases?api-version=5.0"
$token = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$($token)"))
$JSON = #"
{
"definitionId": {release-def-2 definition ID}
}
"#
$response = Invoke-RestMethod -Uri $url -Headers #{Authorization = "Basic $token"} -Method Post -ContentType application/json -body $JSON
}
There is a requirement, to create Azure Devops pipeline which could support version, add file into application, train, test & publish application into Stage & production.
How it can be done through Azure Devops Pipeline. Any documentation or steps could be helpful.
Thanks
A.Prabhuram
I have done something similar through Azure CLI (PowerShell version 2.X) tasks, but it is not straight forward. I haven't done all of the steps as you mention above, but hopefully this will give you what you need to further build on it.
As a baseline, the functions you need are outlined in the LUIS Programmatic API. You will need the LUIS key and app id for most requests, which you can get via
$LUISKEY= & az cognitiveservices account keys list -g "resourceGroupName" --name "LUISauthoringKeyName" --query key1 -o tsv
$header = #{"Ocp-Apim-Subscription-Key"="$LUISKEY"}
$res = Invoke-RestMethod -Uri "https://westus.api.cognitive.microsoft.com/luis/authoring/v3.0-preview/apps/?take=1" -Method 'Get' -Headers $header
$appid = $res.id
For brevity, I'm not repeating this code. But if you have separate tasks and/or agent jobs in your pipeline (as opposed to doing this as one script, which I do not recommend), you'll need to repeat these statements for each task. Do take note of the region and modify as needed for your authoring resource.
Obviously, to update your LUIS version, you need to have the model definition. I don't do this often since we are set up to use the same LUIS app in QA and PROD. So I just add a new version to the project repo if I need to run it through DevOps. Then I add the repo as an artifact for the release pipeline. But you should be able to use the Export Application Version API to get it programmatically, though I haven't tried personally. Here is what I did to add the new version:
$body = Get-Content '$(System.DefaultWorkingDirectory)/_AveryCreek_OEM_CSC_Bot/models/luis/AveryCreek OEM_CSC Team.json' | Out-String
Invoke-RestMethod -Uri "https://westus.api.cognitive.microsoft.com/luis/authoring/v3.0-preview/apps/$appid/versions/import" -Method 'Post' -Body $body -Headers $header
Note that this version is not additive and will completely replace your previous version (though you can revert). In other words, if you have changes to the previous version that are NOT incorporated into the version you are importing, they will be lost. This is one of the main reasons we do not use separate LUIS apps per environment (sidebar - you can use separate predicition resources so you don't use production capacity while testing, but all of the endpoint utterances still go through to the one application).
Once the version is imported, you need to train and publish it. I personally don't have any testing built in, but I'm sure you could build some calls via the LUIS Prediction API and check for expected results. To train, you first need to grab the version, then call the training endpoint.
$res = Invoke-RestMethod -Uri "https://westus.api.cognitive.microsoft.com/luis/authoring/v3.0-preview/apps/$appid/versions" -Method 'Get' -Headers $header
$version = $res[0].version
Invoke-RestMethod -Uri "https://westus.api.cognitive.microsoft.com/luis/authoring/v3.0-preview/apps/$appid/versions/$version/train" -Method 'Post' -Headers $header
The next part is the most tricky. You can't publish the app until it is done training. To mitigate the risk of trying to publish before it is ready, I have created a separate agent job in my "Train and Publish" task to delay it.
This is typically enough delay, but I also have a check on training status and throw an error if it is not ready. Here is the snippet to get and check status and then publish.
$status = Invoke-RestMethod -Uri "https://westus.api.cognitive.microsoft.com/luis/authoring/v3.0-preview/apps/$appid/versions/$version/train" -Method 'Get' -Headers $header
if ($status.details[0].status -ne "Success" -and $status.details[0].status -ne "UpToDate") { throw }
Invoke-RestMethod -Uri "https://westus.api.cognitive.microsoft.com/luis/authoring/v3.0-preview/apps/$appid/publish" -Method 'Post' -Body $body -Headers $header
And that should do it! As I mentioned, take care to make sure you define things like the LUIS Key and App ID in each task, as I have not repeated all of those values here. And you can add additional tasks to programmatically export the version (make sure you get the right key for your source app) and test the model as desired.
What is the easiest way to get key rotation to work for azure storage accounts from a AzureDevOps relase task? The current plan is to re-generate the old key after release to invalidate it, and have a fresh key that can be used on next deployment. But to get that to work it seems like I at least need to store the name of the key to use in a release variable.
I had a look at he logging tasks (https://github.com/Microsoft/azure-pipelines-tasks/blob/master/docs/authoring/commands.md), but that only changes the value in the current release and does not modify the release definition.
You can use the REST API (Definitions - Update) to update the value of the release definition variable from a release task.
Go to the Agent Phase and select Allow Scripts to Access OAuth Token. See Use the OAuth token to access the REST API
Grant Project Collection Build Service (xxx) account the edit
release pipeline permission. (Select the release pipeline --> ... --> Security --> Edit release definition set to Allow)
Add a PowerShell task in your release pipeline
Run inline script: (Update the value of variable v1030 in below sample)
$url = "$($env:SYSTEM_TEAMFOUNDATIONSERVERURI)$env:SYSTEM_TEAMPROJECTID/_apis/Release/definitions/$($env:RELEASE_DEFINITIONID)?api-version=5.0-preview.3"
Write-Host "URL: $url"
$pipeline = Invoke-RestMethod -Uri $url -Headers #{
Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"
}
Write-Host "Pipeline = $($pipeline | ConvertTo-Json -Depth 100)"
# Update an existing variable named v1030 to its new value 1035
$pipeline.variables.v1030.value = "1035"
####****************** update the modified object **************************
$json = #($pipeline) | ConvertTo-Json -Depth 99
$updatedef = Invoke-RestMethod -Uri $url -Method Put -Body $json -ContentType "application/json" -Headers #{Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"}
write-host "=========================================================="
Write-host "The value of Varialbe 'v1030' is updated to" $updatedef.variables.v1030.value
write-host "=========================================================="
Here is a much cleaner and better solution that also allows for multiple builds being triggered simultaneously.
https://tsuyoshiushio.medium.com/how-to-pass-variables-with-pipeline-trigger-in-azure-pipeline-5771c5f18f91
Essentially your triggering build produces artifacts that your triggered build reads and turns into variables.
Still not at all great, but better than nothing and better than REST calls setting static global variable groups.
The other answer above talks about how to update Release pipelines.
If you would like to update a Build Pipeline's variables, here is how you do that:
Edit build pipeline
Go to the Agent Phase and select Allow Scripts to Access OAuth Token. See Use the OAuth token to access the REST API
Go to Manage Security -> Users -> Select Project Collection Build Service (YOUR TEAM NAME HERE)
Change "Edit Build Definitions" to Allow
Now add a powershell stage - 2.x - inline script called Update variables.
Script inline contents:
$api_version='5.0-preview.6'
$url = "$($env:SYSTEM_TEAMFOUNDATIONSERVERURI)$env:SYSTEM_TEAMPROJECTID/_apis/build/definitions/$(System.DefinitionId)?api-version=${api_version}"
Write-Host "URL: $url"
$pipeline = Invoke-RestMethod -Uri $url -Headers #{
Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"
}
Write-Host "Pipeline = $($pipeline | ConvertTo-Json -Depth 100)"
# Update variables as desired here:
$pipeline.variables.mavenBuildVersionPatch.value = "2401"
####****************** update the modified object **************************
$json = #($pipeline) | ConvertTo-Json -Depth 99
$updatedef = Invoke-RestMethod -Uri $url -Method Put -Body $json -ContentType "application/json" -Headers #{Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"}
write-host "=========================================================="
Write-host "The value of Variable mavenBuildVersionPatch is updated to" $updatedef.variables.mavenBuildVersionPatch.value
write-host "=========================================================="
Take note of the API version in this script is 5.0-preview.6.
If your version of Azure Devops is newer, you may need to update this in the future.
Save build pipeline.
Now when you run job, after the job completes this powershell stage, this variable will be set.
Important: If you want to update a variable then make the updated variable for other pipeline stages, then you do it with a powershell stage with the following inline script:
$mavenBuildVersionPatch = [int]"$(mavenBuildVersionPatch)"
$mavenBuildVersionPatch = $mavenBuildVersionPatch + 1
Write-Host "##vso[task.setvariable variable=mavenBuildVersionPatch;]$mavenBuildVersionPatch"
This example would take our existing patch number and increment it by 1.
This does not save the variable at the end of the job, you still need to do that with another powershell script if desired.
In short, the requirement is to verify that our latest released software can be built and then installed after the latest Windows updates and/or other patches were applied. So the build server VM(s) will be configured just for this purpose and the build only needs to run after an update.
Since such updates usually are followed with a restart, I am thinking of a server restart event triggering a build and deployment. Does such option exist in TFS 2017?
If there is no way to do it through TFS then, I guess, a PowerShell script that runs on startup should work?
No such a build-in function to achieve that. However create a PowerShell script that runs on startup should work. Just as Jessehouwing said, you can create the script with the REST API to trigger builds.
Create a script to trigger the specific build definition. (Reference below sample)
Run the script on startup:
How to run a batch file each time the computer boots
How to schedule a Batch File to run automatically in Windows
10/8/7
Param(
[string]$collectionurl = "http://server:8080/tfs/DefaultCollection",
[string]$projectName = "ProjectName",
[string]$keepForever = "true",
[string]$BuildDefinitionId = "34",
[string]$user = "username",
[string]$token = "password"
)
# Base64-encodes the Personal Access Token (PAT) appropriately
$base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f $user,$token)))
function CreateJsonBody
{
$value = #"
{
"definition": {
"id": $BuildDefinitionId
},
"parameters": "{\"system.debug\":\"true\",\"BuildConfiguration\":\"debug\",\"BuildPlatform\":\"x64\"}"
}
"#
return $value
}
$json = CreateJsonBody
$uri = "$($collectionurl)/$($projectName)/_apis/build/builds?api-version=2.0"
$result = Invoke-RestMethod -Uri $uri -Method Post -Body $json -ContentType "application/json" -Headers #{Authorization=("Basic {0}" -f $base64AuthInfo)}
There is no existing trigger that handles this, but there is a simple REST API to query and trigger builds.
It would be easy to create an on startup job in the task scheduler, use the REST API to query a list of Build Definitions based on a certain name or tag and then queue it.
List build definitions
Queue a build