I understand that regardless of commit in the VCS , if highlighted option below is selected then build would trigger as per the scheduled event. Am I correct?
What will happen when we don’t select below option? Provided in 2 scenarios with commit & with out commit?
enter image description here
You can find an explanation here: Running even when there are no code changes
By default, your pipeline does not run as scheduled if there have been
no code changes since the last successful scheduled run. For instance,
consider that you have scheduled a pipeline to run every night at
9:00pm. During the weekdays, you push various changes to your code.
The pipeline runs as per schedule. During the weekends, you do not
make any changes to your code. If there have been no code changes
since the scheduled run on Friday, then the pipeline does not run as
scheduled during the weekend.
To configure the scheduled pipeline to build only if there has been a
change since the last build, check Only schedule builds if the source
or pipeline has changed.
You configured pipeline for scheduled runs. It will run according to your schedule, irrespectively of whether commit happened or not.
You might be interested in commits-based triggers, which are Continuous Integration (CI) triggers and configured separately
Here are some examples for CI triggers in pipeline YAML file:
# specific path build
trigger:
branches:
include:
- master
- releases/*
paths:
include:
- docs
exclude:
- docs/README.md
Related
We have an Azure DevOps YAML multi-stage pipeline where code is built and then deployed to a sequence of environments. Deployment is achieved using Terraform.
i.e.
Builds -> Test -> Deploy to DEV -> Deploy to TEST - - ->
This pipeline is used for both CI/CD builds and also PR builds. For the PR build, the only part of the deployment stage is running terraform plan on the TF scripts.
For PRs, the pipeline is configured to cancel the ongoing build if changes are pushed to that PR.
The problem we're seeing is that when changes are pushed to the PR and the ongoing build is cancelled, sometimes that cancellation happens during the terraform plan step. This occasionally means that the blob lease taken by terraform plan is not released. From that point onwards, manual intervention is required (break the blob lease) in order for the deployment stages to run successfully.
I believe we can switch off the setting which causes the ongoing PR build to be cancelled if changes are pushed.
But I wondered if there was a way of marking a pipeline step as critical - i.e. it should run to completion if the build is cancelled?
There are other ways of cancelling a pipeline build and there must be other tasks/steps which should not be cancelled part-way through. Such a critical-task setting would cover these situations too.
Not sure if you solved this, but I had the exact same issue. Adding condition: always() to my task forced it to complete when DevOps cancelled the pipeline after additional changes were pushed.
See https://learn.microsoft.com/en-us/azure/devops/pipelines/process/conditions?view=azure-devops&tabs=yaml:
jobs:
- job: Foo
steps:
- script: echo Hello!
condition: always() # this step will always run, even if the pipeline is canceled
I'm afraid you won't be able to achieve this using just YAML. What you can do will require some additional effort (and in some cases quite big):
replace terraform script with bicep for isntance - for me syntax iq quite similar and what you get here is lack of terrafrom state
add in your very first step a check if your state is locked and break a lease if it is needed
I understood that you would like to hear something better than this but at the moment there is no way to mark a task as non-cancellable. However, this sound like a cool feature and candidate for feature request.
What I want to achieve: run pipeline ONLY on cron timer and run it on timer ALWAYS ignoring previous run status
trigger: none # No CI build
pr: none # Not for pull requests
schedules:
- cron: "0 0 * * *"
displayName: Daily midnight build
branches:
include:
- integration-tests
always: true
What I currently have:
Failing pipeline works as expected
Successful pipeline stopped working after first successful run. It won't even start when manually triggered run failed.
All pipelines have correct "Scheduled runs" tab.
Every pipeline does not have any trigger/schedule configured over UI
What am I doing wrong?
You could check this FAQ:
I see the planned run in the Scheduled runs panel. However, it does not run at that time. Why?
The Scheduled runs panel shows all potential schedules. However, it may not actually run unless you have made real updates to the code. To force a schedule to always run, ensure that you have set the always property in the YAML pipeline, or checked the option to always run in a classic pipeline.
And this My YAML schedules were working fine. But, they stopped working now. How do I debug this?
According to the description, you could make a small trivial change to you YAML file, and push the update to your repository.
You could also try to create a new YAML build and check the result.
Oof, looks like I found the reason. Scheduled runs tab made everything a bit confusing to me.
So, according to description should I put cron timer into master to make this tab work?
Here you can see that it mentions default branch (master for me). I had to put the exact yml into master to make this tab display future scheduled runs. But always: true in the master branch seems to be irrelevant, it should be applied to the branch you are running pipeline from (integration-tests for me). Looks like at some point during trying to fix stuff I have put always: true into master's pipeline.yml, but not in integration-tests's
When I have put option to both yml's, everything started working as expected.
However, when I look at pipelines that have schedule binded, I become even more confused. It was mentioning default branch when there were no schedule, and now it shows refs/heads/integration-tests?? where does it take cron from?
Our current pipeline deploys a new instance of our app for every new branch created on azure repos, just like review apps on Heroku or Gitlab.
The creation part went smooth, but I'm not sure what to do with the orphaned resources and deployment once the branch is deleted (hopefully by an accepted pr).
Deleting them manually is not an option and there I can't find a trigger in the documentation for branch deletes.
The only option I can see right now is to create a scheduled job for the master branch(since it always exists) with a bash script that compares the list of deployed apps and existing branches and cleans up the resources.
Is it my only option, or is there another way without a fairly complex, all-access, destroy machine?
So did a little investigation dumping all enviroment vars to Notepad++ and using the compare plugin i realized that when a PR is accepted 2 env variables are different.
First of the initial variable "BUILD_REASON" during a push is set to "IndividualCI", but with the "BUILD_SOURCEBRANCH" set to "refs/heads/feature/******". When a pull request is initiated the "BUILD_REASON" changes to "pullrequest" and "BUILD_SOURCEBRANCH" to "refs/pull/***".
Finally when a PR is accepted the variables change to "BUILD_REASON" = "IndividualCI" and "BUILD_SOURCEBRANCH" = "refs/heads/master".
Once i figured out this i could create stage that have the following conditions:
- stage: CleanUp
displayName: 'CleanUp'
dependsOn: Test
condition: and(succeeded(), in(variables['Build.Reason'], 'IndividualCI'),in(variables['Build.SourceBranchName'], 'master'))
The above stage will be triggered when PR is accepted so to cleanup resources created during PR :-) havnt tested all the way but seems to do the job.
You can use a webhook in Azure DevOps to watch the pull request for updates. When the pull request status changes to completed, fire a script that deletes the resources used for the PR.
I have a situation where two commits were merged to master (e.g. FIRST and SECOND) very close together (seconds apart). Both triggered the build pipeline: FIRST triggered the pipeline first and SECOND triggered it second (the builds ran in parallel). For whatever reason, the build pipeline for commit SECOND finished first, and 30 seconds later the build for commit FIRST finished.
My automatic release pipeline is configured to always get the "latest" artifact from the build pipeline. The sequence of events described above caused the SECOND change to be deployed first, and then the FIRST change was deployed next (since its pipeline finished second) and stomped on the prior release, effectively deploying old bits to the service.
Is there any way to prevent this situation? Even if a build pipeline finishes second for intermittent reasons, I don't want a release to stomp over a more recent change that happened to finish earlier.
EDIT: Thank you to those who suggested/supported the idea of batching builds but that's not an option I'm looking to enable. I still want each commit to trigger its own build (to enable easier assignment of build break cause). I'm just looking for the releases to trigger in the order of commits, not the order of builds finishing.
Thanks!
You can set batch to true in triggers, so the system waits until the build is completed. Set "Batch changes while a build is in progress" option to true in Triggers for Build Pipeline at Azure DevOps or in YAML:
trigger:
batch: true
If you use Pull request, there should be no issues as new push should cancel in-progress run. Check autoCancel in PR triggers
You may need to make the pipelines to run on the same agent. So that the newest queue will wait for the previous queue to complete.
You can follow below steps to confine your pipeline to one agent.
1, Add a custom capability to the agent you want to run the pipeline(project settings->agent pools(select an agent pool)->agents(select a agent)->capabilities)
2,Add a demand to your pipeline : # this works for both microsoft-hosted agents and self-hosted agents
I tested and found microsoft-hosted agent pool doesnot support demands for custom capabilities in yaml pipeline.
Below yaml pipeline works only for self-hosted agent pool.
pool:
name: Default
demands: Tag -equals Agent1
Had a scheduled job running every 30 minutes of Azure devops, it was running fine, last scheduled build I saw was on 2019-10-02·14:00, and since then there were no changes made to azure devops or even to the repository for which the pipeline builds.
Devops is not triggering any new builds, not sure about the issue and where should I look for issues.
Tried a manual run, thinking that it could invoke the sleeping process somewhere, but it did not help
trigger:
branches:
include:
- master
schedules:
- cron: "*/30 * * * *"
displayName: Daily half-hourly build
branches:
include:
- master
always: true
....
....
For your issue ,there could be two reasons why the schedule trigger stopped scheduling new builds.
The first one : Set the schedule trigger in UI
Scheduled triggers defined using the pipeline settings UI take precedence over YAML scheduled triggers.
If your YAML pipeline has both YAML scheduled triggers and UI defined scheduled triggers, only the UI defined scheduled triggers are run. To run the YAML defined scheduled triggers in your YAML pipeline, you must remove the scheduled triggers defined in the pipeline setting UI.
The second one: Each cron schedule has a limit
Each cron schedule has a maximum of 100 pipeline runs per week.From your description: running every 30 minutes, it should exceed the limit of 100 pipelines per week. If you need more, you can split your cron schedule into multiple cron schedules that each result in 100 or less pipeline runs per week.
You can check these two points to see if this is the cause of the issue.For details please refer to this official document.