We have Pull Request triggers enabled for our release and the all the automatic stages are already completed. But still, the Pull request status is shown as in progress. We have branch policies and pull request triggers properly set.
Note - There are few stages with manual deployment, followed by stages with PR triggers enabled. Is this causing the problem?
PR status is overall status of release. In case of Multiple stages, if any of the stage is in 'in-progress' state, status will be 'in-progress'. Status takes into account all the stages for which "Pull request deployment" option is enabled. In your case since there are stages with Pull request deployment enabled and are not started (as preceeding stage is manual) status will be 'in-progress'. If you don't want those stages for PR validation, Please disable the option for those stages. Then status computation will not consider those stages.
Related
We have an Azure DevOps YAML multi-stage pipeline where code is built and then deployed to a sequence of environments. Deployment is achieved using Terraform.
i.e.
Builds -> Test -> Deploy to DEV -> Deploy to TEST - - ->
This pipeline is used for both CI/CD builds and also PR builds. For the PR build, the only part of the deployment stage is running terraform plan on the TF scripts.
For PRs, the pipeline is configured to cancel the ongoing build if changes are pushed to that PR.
The problem we're seeing is that when changes are pushed to the PR and the ongoing build is cancelled, sometimes that cancellation happens during the terraform plan step. This occasionally means that the blob lease taken by terraform plan is not released. From that point onwards, manual intervention is required (break the blob lease) in order for the deployment stages to run successfully.
I believe we can switch off the setting which causes the ongoing PR build to be cancelled if changes are pushed.
But I wondered if there was a way of marking a pipeline step as critical - i.e. it should run to completion if the build is cancelled?
There are other ways of cancelling a pipeline build and there must be other tasks/steps which should not be cancelled part-way through. Such a critical-task setting would cover these situations too.
Not sure if you solved this, but I had the exact same issue. Adding condition: always() to my task forced it to complete when DevOps cancelled the pipeline after additional changes were pushed.
See https://learn.microsoft.com/en-us/azure/devops/pipelines/process/conditions?view=azure-devops&tabs=yaml:
jobs:
- job: Foo
steps:
- script: echo Hello!
condition: always() # this step will always run, even if the pipeline is canceled
I'm afraid you won't be able to achieve this using just YAML. What you can do will require some additional effort (and in some cases quite big):
replace terraform script with bicep for isntance - for me syntax iq quite similar and what you get here is lack of terrafrom state
add in your very first step a check if your state is locked and break a lease if it is needed
I understood that you would like to hear something better than this but at the moment there is no way to mark a task as non-cancellable. However, this sound like a cool feature and candidate for feature request.
Our current pipeline deploys a new instance of our app for every new branch created on azure repos, just like review apps on Heroku or Gitlab.
The creation part went smooth, but I'm not sure what to do with the orphaned resources and deployment once the branch is deleted (hopefully by an accepted pr).
Deleting them manually is not an option and there I can't find a trigger in the documentation for branch deletes.
The only option I can see right now is to create a scheduled job for the master branch(since it always exists) with a bash script that compares the list of deployed apps and existing branches and cleans up the resources.
Is it my only option, or is there another way without a fairly complex, all-access, destroy machine?
So did a little investigation dumping all enviroment vars to Notepad++ and using the compare plugin i realized that when a PR is accepted 2 env variables are different.
First of the initial variable "BUILD_REASON" during a push is set to "IndividualCI", but with the "BUILD_SOURCEBRANCH" set to "refs/heads/feature/******". When a pull request is initiated the "BUILD_REASON" changes to "pullrequest" and "BUILD_SOURCEBRANCH" to "refs/pull/***".
Finally when a PR is accepted the variables change to "BUILD_REASON" = "IndividualCI" and "BUILD_SOURCEBRANCH" = "refs/heads/master".
Once i figured out this i could create stage that have the following conditions:
- stage: CleanUp
displayName: 'CleanUp'
dependsOn: Test
condition: and(succeeded(), in(variables['Build.Reason'], 'IndividualCI'),in(variables['Build.SourceBranchName'], 'master'))
The above stage will be triggered when PR is accepted so to cleanup resources created during PR :-) havnt tested all the way but seems to do the job.
You can use a webhook in Azure DevOps to watch the pull request for updates. When the pull request status changes to completed, fire a script that deletes the resources used for the PR.
I have a situation where two commits were merged to master (e.g. FIRST and SECOND) very close together (seconds apart). Both triggered the build pipeline: FIRST triggered the pipeline first and SECOND triggered it second (the builds ran in parallel). For whatever reason, the build pipeline for commit SECOND finished first, and 30 seconds later the build for commit FIRST finished.
My automatic release pipeline is configured to always get the "latest" artifact from the build pipeline. The sequence of events described above caused the SECOND change to be deployed first, and then the FIRST change was deployed next (since its pipeline finished second) and stomped on the prior release, effectively deploying old bits to the service.
Is there any way to prevent this situation? Even if a build pipeline finishes second for intermittent reasons, I don't want a release to stomp over a more recent change that happened to finish earlier.
EDIT: Thank you to those who suggested/supported the idea of batching builds but that's not an option I'm looking to enable. I still want each commit to trigger its own build (to enable easier assignment of build break cause). I'm just looking for the releases to trigger in the order of commits, not the order of builds finishing.
Thanks!
You can set batch to true in triggers, so the system waits until the build is completed. Set "Batch changes while a build is in progress" option to true in Triggers for Build Pipeline at Azure DevOps or in YAML:
trigger:
batch: true
If you use Pull request, there should be no issues as new push should cancel in-progress run. Check autoCancel in PR triggers
You may need to make the pipelines to run on the same agent. So that the newest queue will wait for the previous queue to complete.
You can follow below steps to confine your pipeline to one agent.
1, Add a custom capability to the agent you want to run the pipeline(project settings->agent pools(select an agent pool)->agents(select a agent)->capabilities)
2,Add a demand to your pipeline : # this works for both microsoft-hosted agents and self-hosted agents
I tested and found microsoft-hosted agent pool doesnot support demands for custom capabilities in yaml pipeline.
Below yaml pipeline works only for self-hosted agent pool.
pool:
name: Default
demands: Tag -equals Agent1
I have 3 steps YAML pipeline Build (1), Deploy To Development (2) and Deploy to Test (3).
My needs are very basic:
a. When some code changes in master, pipelines triggers (OK)
b. With this changes, Build and Deploy To Development stages automatically runs (OK)
c. Deploy to Test stages waits until I approve (OK)
But I've facing a problem if I don't approve the Stage (3), the pipeline never ends and always shows an in-progress icon. So whenever I check the Pipelines page, I saw all pipelines are running but actually is not.
Worse part is the whole pipeline falls in Failed status after two weeks.
My question is:
Is there any way to mark Deploy to Test stage as optional? The pipeline should be completed without this stage but optionally I want to able to execute this step manually.
For your issue, I think it is currently not supported in multi-stage yaml.
If you want to run a stage manually in yaml, you can only through creating checks for your environments. There are only two options for review: Approve or Reject. So when you don't want to deploy to this stage, the stage will remain in the waiting state, and will automatically reject until the approval timeout. Once deployed to the stage is rejected, the pipeline will show as failed, even if the previous stages were successfully deployed.
So just as Shayki said in the comment, the multi-stage needs to be improved, it should give the stage a more reasonable manual trigger. Here I created a feature request for your issue in our developercommunity forum. You can vote and comment here to improve its priority.
In addition,as a workaround ,you can deploy with release pipeline. You can create Release pipeline in Releases Page. Then you can add test stage and set Manual only trigger for it. So that the pipeline can be completed without this stage .
I have a "QA" stage that is deploying a package and after deployment I have a post-deployment approval. I only want to notify the "UAT Post-Deployment Approvers" group when the package is a Release package. I use GitVersion and Git Tags to tag a git commit. The Build Pipeline will automatically build the source code with the Git Tag event because of the branch filter "refs/tags/v*" that I've added. I have also used Artifact Filters in the Release Pipeline before on the Pre-Approval side to prevent entry and triggering of the Pre-Approvers notification to look for "refs/tags/v*", but this time I want to filter on the Post-Approval side.
Is there a way to filter AFTER a successful deployment, but BEFORE a Post-Approval so that the stage doesn't appear as a failure? Why? Because the deployment was successful and now I want the Release Pipeline to stop and not go any further. I don't want to ask for a Post-Deployment Approval. The Gate check that I currently have (Azure Function) will return a failure and thus will fail the complete Stage (what I don't want) and then not ask for Post-Deployment Approval (which I do want). OR is there a way to set the Stage to success after a failed Gate Check? Or, maybe set a Jobless Agent step to stop processing?
Is there a way to filter AFTER a successful deployment, but BEFORE a
Post-Approval so that the stage doesn't appear as a failure? OR is
there a way to set the Stage to success after a failed Gate Check?
For these issues,I am afraid that it is impossible to do this in azure devops.If you set Gates and Post-deployment approvals in the stage, stage will only show success if both conditions pass.If either of these two conditions does not pass, the stage will show as failure.
In addition,AFTER a successful deployment, but BEFORE a Post-Approval,the stage shows the status as Pending approval, not failure.