I've got a very simple Azure Pipeline Release, I just want to skip a stage if there are any errors in the previous one. I've already checked https://learn.microsoft.com/en-us/azure/devops/pipelines/process/conditions?view=azure-devops&tabs=classic
Setting the Test job to run "Only when all previous jobs have succeeded" doesn't help
My main goal is to skip the test stage whenever there's a particular condition in the previous one, and passing variables between stages doesn't seem to be possible, nor using the gates, so I've got to idea to deliberately raise an error in the stage. The stages run some PS scripts, and I can't make the whole stage fail from that either
Screen shot
The stages run some PS scripts, and I can't make the whole stage fail
from that either
Are you using PowerShell task or similar tasks? Try editing the task and enabling the Fail on Standard Error option:
After that task will fail if there's any error is written to the error pipeline. Also in pre-deployment conditions of your Test stage, make sure the trigger when ... checkbox is unchecked:
From what I understood you need sth like this feature. So as you see this is not supported out of the box.
You can use REST API to get state of running release and analyze response to verify it there is any issue with issueType = Error. Then in this script you need to call exit 1. This is not ideal, but it works.
$uri = "https://vsrm.dev.azure.com/thecodemanual/Devops manual/_apis/release/releases/$(Release.ReleaseId)?api-version=5.1"
Write-Host $(Release.ReleaseId)
Write-Host $uri
# Invoke the REST call
$result = Invoke-RestMethod -Uri $uri -Method Get -Headers #{Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"}
$errors = #()
$result.environments |
ForEach-Object { $_.deploySteps |
ForEach-Object { $_.releaseDeployPhases |
ForEach-Object { $_.deploymentJobs |
ForEach-Object { $_.tasks |
ForEach-Object { $errors += $_.issues | Where-Object { $_.issueType -eq "Error" } }}}}}
Write-Host $errors.Count
if($errors.Count -gt 0) {
Write-Host Error
exit 1
}
Without step above I got this:
And with this step this:
Related
I've been scouring for the answer to this for the last several days. There's many fields labeled "status" in the TC API and for the life of me, I can't figure out which one returns the status listed above.
I'm currently calling...
$status = (Invoke-RestMethod -Uri %teamcity.serverUrl%/httpAuth/app/rest/builds/id:%teamcity.build.id% -Method Get -Headers $header).build.status
...within PowerShell. This returns SUCCESS unless the build has an execution timeout. However, as you can see from the screenshot above, the build failed. Obviously, the call above doesn't return the right status. The failure is due to a specific build step failing. If any of the build steps fail, then the entire build fails. Not sure if that helps!
The problem doesn't seem to be in the endpoint itself. You are requesting the correct field. The same can also be done with the following requests:
GET /app/rest/builds?locator=id:%teamcity.build.id%&fields=build(status)
GET /app/rest/builds/%teamcity.build.id%/status
The first request will return you an XML or JSON structure depending on your Accept header. The second response will return you plain text, so set the headers accordingly.
This returns SUCCESS unless the build has an execution timeout. However, as you can see from the screenshot above, the build failed. Obviously, the call above doesn't return the right status. The failure is due to a specific build step failing. If any of the build steps fail, then the entire build fails.
You are requesting the build status during the build execution. It is possible, that when the build step that checks the status is running, the status of the build is still SUCCESS. Try moving the build step to the very end of this build and configure it to execute 'Always, even if build stop command was issued'
Ok, so this is what was happening. I have an build step in TeamCity that runs Katalon (our test automation software). Even though the tests were failing, TeamCity was not updating the build status until after the last step, which happened to be my PS script that called the API then sent the info over to MS Teams. I had to setup a Failure Condition in TeamCity for the Katalon build step that looks in the build log for the specific text "(Katalon) failed". This immediately fails the build, but the rest of the build steps continue to run. Now, when my last step runs, it pulls the correct status and pushes it to MS Teams. For anyone that is interested, this is the full PowerShell script for that build step.
function Get-AuthenticationHeader
{
param($User, $Password)
$pair = "$($User):$($Password)"
$encodedCreds = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes($pair))
"Basic $encodedCreds"
}
$authHeader = Get-AuthenticationHeader -User "%system.teamcity.auth.userId%" -Password "%system.teamcity.auth.password%"
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls -bor [Net.SecurityProtocolType]::Tls11 -bor [Net.SecurityProtocolType]::Tls12
$header = #{
"Authorization"="$authHeader"
}
$response = (Invoke-RestMethod -Method GET -Headers $header -ContentType "application/json" -Uri "%teamcity.serverUrl%/httpAuth/app/rest/builds/%teamcity.build.id%")
$status = $response.build.status
$name = "%system.teamcity.buildConfName%"
$teamcityAgent = "%teamcity.agent.name%"
$testResults = $response.build.webUrl
Write-Host "%teamcity.build.id%"
Write-Host $status
Write-Host $name
Write-Host $testResults
if ( $status -eq "SUCCESS" ) {
$color = "00ff00"
}
else {
$status = "FAILURE"
$color = "ff0000"
}
$body = #{
title= "API Test Automation";
text= "<pre><b><i><font color=$color>$status</font></i></b> - $name<br>Tests ran on <i>$teamcityAgent</i><br>Test results are HERE</pre>"
} | ConvertTo-Json
Invoke-WebRequest -Method POST -ContentType "application/json" -Uri "<YOUR MS TEAMS WEBHOOK HERE>" -Body $body -UseBasicParsing
I have an Azure DevOps Multi Stage Pipeline with multiple agents. Generally my stages consist of:
Build and Publish Artifacts
Deploy to environment 1
UI Test environment 1
Deploy to environment 2
UI Test environment 2
and so on for several environments. How do I allow simultaneous pipelines runs (e.g. Building and Publishing Artifacts for Build #2 while simultaneously UI Testing environment 2 for Build #1), while ensuring that no two agents will ever perform a UI Test for a given environment at the same time?
trigger: batch seems close to what I want, but I believe that disallows concurrency at the pipeline level, not at the stage level.
Your
UI Test environment 1 stage (specifically deployment job which Runs the tests) should target
env1uitest environment.
UI Test environment 2 stage (specifically deployment job which Runs the tests) should target
env2uitest environment.
Then set a exclusive lock ( https://learn.microsoft.com/en-us/azure/devops/release-notes/2020/pipelines/sprint-172-update#exclusive-deployment-lock-policy) policy on these two environments.
It should address your need:
"ensuring that no two agents will ever perform a UI Test for a given environment at the same time"
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/approvals?view=azure-devops&tabs=check-pass#exclusive-lock
It seems that if somebody runs the release pipeline twice, You need it to run it one by one, not in parallel, right?
As a workaround:
Each release will start from stage 1. Thus we can add a PowerShell task as the first task for stage 1 to check if there are previous in-progress deployments.
In this PowerShell task, we can call this Rest API to check release stage status.
Power shell script:
# Base64-encodes the Personal Access Token (PAT) appropriately
$token = "$(pat)"
$base64AuthInfo= [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$($connectionToken)"))
$success = $false
$count = 0
do{
try{
$stageurl2 = "https://vsrm.dev.azure.com/{org name}/{project name}/_apis/release/deployments?definitionId={release definition ID}&deploymentStatus=inProgress&api-version=6.0"
$stageinfo2 = Invoke-RestMethod -Method Get -ContentType application/json -Uri $stageurl2 -Headers #{Authorization=("Basic {0}" -f $base64AuthInfo)}
$inprogressdeployments = $stageinfo2.value | where {($_.deploymentStatus -eq "inProgress") -and ($_.release.name -ne $ENV:RELEASE_RELEASENAME) -and ($_.releaseEnvironment.name -ne 'stop services')} | Sort-Object -Property completedOn -Descending
#write-output $inprogressdeployments
$stageurl3 = "https://vsrm.dev.azure.com/{org name}/{project name}/_apis/release/deployments?definitionId={release definition ID}&operationStatus=QueuedForAgent&api-version=6.0"
$stageinfo3 = Invoke-RestMethod -Method Get -ContentType application/json -Uri $stageurl3 -Headers #{Authorization=("Basic {0}" -f $base64AuthInfo)}
$queueddeployments = $stageinfo3.value
#write-output $queueddeployments
if($inprogressdeployments) {
Write-output "Deployment In Progress, Waiting for it to finish!"
Write-output "Next attempt in 30 seconds"
Start-sleep -Seconds 30
} else {
Write-Host "No Current Deployment in Progress"
if($queueddeployments) {
write-output "Current Queued deployments"
Write-output "if 2 - Next attempt in 30 seconds"
Start-sleep -Seconds 30
}
else{
write-output "No Queued deployments, starting release"
$success = $true
}
}
}
catch{
Write-output "catch - Next attempt in 30 seconds"
write-output "1"
Start-sleep -Seconds 30
# Put the start-sleep in the catch statemtnt so we
# don't sleep if the condition is true and waste time
}
$count++
}until($count -eq 2000 -or $success)
if(-not($success)){exit}
Result:
Stage1 will continue to check until all previous versions are complete.
We could also try it with demands, we could specify same demands. Use demands to make sure that the capabilities your pipeline needs are present on the agents that run it. It won't run unless one or more demands are met by the agent.
You put the tasks into different jobs, and specify a dependsOn property for jobs that need to wait for particular job to be finished.
If you leave out the dependsOn, it will run synchronounsly.
After reading your prompt more closely, I think you just need a build stage that builds and publishes your artifact.
then a deploy_test stage for each environment. Stages run sequentially by default (unlike jobs)
We have an Azure DevOps pipeline which uses self hosted Windows agents with Azure DevOps server 2019. The pipeline runs our front-end tests without any problems. However, occasionally our linting step will find problems that it throws as warnings (such as unused variables). This is what we want it to do but the issue is that these warnings were not being elevated. So the only way to see them was to look in the build execution.
This we were able to resolve by adding a vso formatter to the linting command: npm run nx run-many -- --target="lint" --all --skip-nx-cache=true --parallel --format=vso. So now the warnings are thrown like this:
As shown in the green box the warnings are displaying properly. However, in the red circles the status of the build, job, and linting task are success. Is there a way I can mark this build, job, and task as warning so we know to take a further look? Thank you for any help, please let me know if I can provide additional information.
You can add a powershell task at the end of the pipeline, and then run the Rest API(Timeline - Get) to traverse the warning messages in the previous task. Finally, you can use the logging command to set the pipeline status
Here is PowerShell Sample:
$token = "PAT"
$url=" https://{instance}/{collection}/{project}/_apis/build/builds/$(build.buildid)/timeline?api-version=5.0"
$token = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$($token)"))
$count = 0
$response = Invoke-RestMethod -Uri $url -Headers #{Authorization = "Basic $token"} -Method Get -ContentType application/json
ForEach( $issues in $response.records.issues )
{
if($issues.type -eq "warning")
{
echo $issues.Message
$count ++
}
}
echo $count
if($count -ne 0 )
{
Write-Host "##vso[task.complete result=SucceededWithIssues;]"
}
Result:
There is Azure DevOps Extension/Task created by Microsoft "Build Quality Checks"
So you can use it to set up an additional option to force specific build qulity.
I'm working on a release pipeline there are around 3 task in the 1 Agent. taskA,taskB,taskC
I want to run specific task based on specific task failed. I tried custom condition but it didn't satisfied my case.
I want task C to be execute when only task B is failed for that I'm using output variable as well. In this case its working only for taskB. When TaskA failed Task B will skip and output variable become null in that case my task C is execute which is not correct.
I'm trying to make a condition which fulfill both TaskA and TaskB condition.
if TaskA failed --> TaskC should not run
if TaskB failed --> TaskC should run.
Here is my condition:-> and(eq(Agent.JobStatus, 'failed'), in(variables['oneboxout.oneboxvar'],'False'))
Is there any way we can get task name only so my work would be easier.
Below are my task screenshot for your reference.
Is there any way we can get task name only so my work would be easier.
The answer is yes.
We could use the REST API Releases - Get Release to get the task name or status；
https://learn.microsoft.com/en-us/rest/api/azure/devops/release/releases/get%20release?view=azure-devops-rest-6.0
As the above REST API, we need provide the current release Id to that REST API. To resolve this, we could use the REST API Releases - List with Parameter definitionId={definitionId} and powershell parameter Select-Object -first 1 to get the current release Id.
To resolve this request, I would like use following method:
Summary:
Add powershell a task (Let's call it Get JobB task result)between JobB and JobC to invoke REST API
to get the result of the JobB with condition Even if a previous task has failed, unless the build was canceled.
Set a variable RunJobC with different value based on the result of the task JobB in above powershell task.
Set condition and(always(), eq(variables['RunJobC'], 'True')) for the JobC.
My test scripts (Check Allow scripts to access the OAuth token option in the Phase):
$url = "https://vsrm.dev.azure.com/M<YourOrganization>/<YourProject>/_apis/release/releases?definitionId=24&api-version=6.0"
$RealeasePipeline= Invoke-RestMethod -Uri $url -Headers #{
Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"
} -Method Get
$ReleaseId= $RealeasePipeline.value.id | Select-Object -first 1
Write-Host The current Release Id: $ReleaseId
$url2 = "https://vsrm.dev.azure.com/<YourOrganization>/<YourProject>/_apis/release/releases/$($ReleaseId)?api-version=6.0"
$ReleaseInfo= Invoke-RestMethod -Uri $url2 -Headers #{
Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"
} -Method Get
$TargetTask=$ReleaseInfo.environments.deploySteps.releaseDeployPhases.deploymentJobs.tasks| where { $_.Name -eq "JobB"}
Write-Host JobB task Result is: $TargetTask.status
if ($TargetTask.status -eq "succeeded"){
Write-Host ("##vso[task.setvariable variable=RunJobC]False")
}elseif($TargetTask.status -eq "Failed"){
Write-Host ("##vso[task.setvariable variable=RunJobC]True")
}
My test result:
I have a simple pipeline, in which gatling load script executes:
My pipeline
After the script is completed, we see the script execution time:
Execution time
My first question is: how can I extract value of execution time and compare it with previous execution (my script executes every night), and if execution time in last time is more than in previous time, I want to get notification.
Also, after execution generates report: Report
And my next question is: how can I open the generated report directly in Azure (or at least send it by mail as an attachment to a standard notification from Azure)?
You can use Build timeline API to get the execution time of a specific task.
I add a few more lines of scripts to my previous answer. Please check below scripts.
I added script $result.records | where {$_.name -eq "MyTaskName"} to filter the task records by its name. Note: You need to define a unique Display name for your powershell task.
$url = "$(System.TeamFoundationCollectionUri)$(System.TeamProject)/_apis/build/builds/$(Build.TriggeredBy.BuildId)/timeline?api-version=5.1"
echo $url
$result = Invoke-RestMethod -Uri $url -Headers #{authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"} -ContentType "application/json" -Method get
# filter the task's records by name
$taskResult = $result.records | where {$_.name -eq "MyTaskName"}
# calculate the totaltime of the newest build
$time = [datetime]$taskResult.finishTime - [datetime]$taskResult.startTime
$thisTaskTime= $time.TotalMinutes
# get the last build's Id
$lasturl = "$(System.TeamFoundationCollectionUri)$(System.TeamProject)/_apis/build/builds?definitions=$(Build.TriggeredBy.DefinitionId)&resultFilter=succeeded&`$top=2&api-version=5.1"
$buildResult =Invoke-RestMethod -Uri $lasturl -Headers #{authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"} -ContentType "application/json" -Method get
#extract last buildId
$lastBuildId = $buildResult.value[1].id
#get the timeline of the last build
$timeUrl = "$(System.TeamFoundationCollectionUri)$(System.TeamProject)/_apis/build/builds/$($lastBuildId)/timeline?api-version=5.1"
$lastResult =Invoke-RestMethod -Uri $timeUrl -Headers #{authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"} -ContentType "application/json" -Method get
#Caculate the totaltime of the last build task
# filter the task's records by name
$lastTaskResult = $lastResult.records | where {$_.name -eq "TaskName"}
$LastTime = [datetime]$lastTaskResult.finishTime - [datetime]$lastTaskResult.startTime
$lastTaskTime= $LastTime.TotalMinutes
#Store the result to varialbe isLonger
if($thisTaskTime -ge $lastTaskTime){ echo "##vso[task.setvariable variable=isLonger]True" }
If you want to use the azure default notification when the task execution time is greater then previous time. You can add another script task to the end of your pipeline to purposely fail your pipeline, if the time is greater. And then set up a build fails notification for your project.
if($(isLonger) -eq "True")
{
exit 1
}
For your second question, I am afraid you cannot view the report directly in azure devops.
You can use copy files task to copy out the generated report and use publish build artifacts task to upload the report to azure devops server, where you can download directly from your build history.
Copy Files task
Publish build artifact task
Then you can download the report file from the build UI.
Go to Builds under Pipelines--> Select your pipeline--> Click lastest build from the History
There are other ways to upload the report file to the build besides Publish build artifact task by using statement ##vso[task.addattachment]value, ##vso[task.uploadfile]local file path. For example run below statement in a script task to add the report file to the build.
##vso[task.addattachment type=myattachmenttype;name=myattachmentname;]$(System.DefaultWorkingDirectory)\reportTempFolder\index.html
Please check here for more information.