How to use artifact from other multiple workflow filtering by branch.
WorkflowCD can run triggered or manual, consuming artifact uploaded by Workflow1,Workflow2....WorfklowN. Taking artifacts coming from
resources:
pipelines:
- pipeline: workflow1
project: YY
branch: master
- pipeline: workflow2
project: xx
branch: develop
resources: # types: pipelines | builds | repositories | containers | packages
pipelines:
- pipeline: string # identifier for the resource used in pipeline resource variables
project: string # project for the source; optional for current project
source: string # name of the pipeline that produces an artifact
version: string # the pipeline run number to pick the artifact, defaults to latest pipeline successful across all stages; Used only for manual or scheduled triggers
branch: string # branch to pick the artifact, optional; defaults to all branches; Used only for manual or scheduled triggers
tags: [ string ] # list of tags required on the pipeline to pickup default artifacts, optional; Used only for manual or scheduled triggers
trigger: # triggers aren't enabled by default unless you add trigger section to the resource
branches: # branch conditions to filter the events, optional; Defaults to all branches.
include: [ string ] # branches to consider the trigger events, optional; Defaults to all branches.
exclude: [ string ] # branches to discard the trigger events, optional; Defaults to none.
tags: [ string ] # list of tags to evaluate for trigger event, optional
stages: [ string ] # list of stages to evaluate for trigger event, optional
Related
I have a CD pipeline, which uses a CI pipeline as a resource. In my CD pipeline, I have triggers that exclude some project paths that I have in the solution, in my resource I have included my main branch on the trigger.
CD pipeline:
trigger:
branches:
include:
- main
paths:
exclude:
- src/Customer.Worker.Converter
pool:
vmImage: "ubuntu-latest"
resources:
pipelines:
- pipeline: customer-ci
source: Customer-API-CI
trigger:
branches:
include:
- main
stages:
...
What I want to happen is when I push from my main branch, the CI pipeline runs and activates the CD pipeline only when what I changed is not in the paths I excluded from the trigger, in this case, when I change something in the Customer.Worker.Converter project, I don't want this pipeline to be triggered.
But even when I change something from path Customer.Worker.Converter, the CD pipeline triggers after the CI pipeline, ignoring my exclude path trigger and apparently obeying the CI resource pipeline trigger I use.
Is there any way to get what I want using pipeline resource?
As per this article, you've defined two separate triggers, but I think you only want one when the CI pipeline completes. At the moment, you will trigger changes based on the CI pipeline and when there are code-pushes.
If you only want the CD pipeline to be triggered when the CI pipeline completes, change the trigger for the CD pipeline to be none:
# ci pipeline
trigger:
branches:
include:
- main
paths:
include:
- src/Customer.Worker.Converter/*
----
# cd pipeline
resources:
pipelines:
- pipeline: customer-ci
source: customer-api-ci
trigger:
branches:
include:
- main # only trigger the CD pipeline when the CI pipeline on 'main' completes.
# do not trigger changes to this pipeline when code-changes are made
trigger: none
In theory, you can combine resource triggers:
When filters are specified, the source pipeline run must match all of the filters to trigger a run.
So one possibility would be to configure the cd pipeline to only run when the CI pipeline completes on the main branch and has a specific tag.
I have two repositories:
devops - with definitions of pipelines
src_project - with sources of project
Default when I run a Azure pipeline I can choice between branch/tag of repository with pipeline definition. I would like to add a parameter with which I can select a branch from the source project.
Is this possible in a simple way in Azure? I don't want, I cannot keep definitions of pipelines in src_project.
In Jenkins, we used an additional method to fetch these branches, using the shared libraries plugin.
You can do it with resources from type repositories:
resources:
repositories:
- repository: string # identifier (A-Z, a-z, 0-9, and underscore)
type: enum # see the following "Type" topic
name: string # repository name (format depends on `type`)
ref: string # ref name to use; defaults to 'refs/heads/master'
endpoint: string # name of the service connection to use (for types that aren't Azure Repos)
trigger: # CI trigger for this repository, no CI trigger if skipped (only works for Azure Repos)
branches:
include: [ string ] # branch names which will trigger a build
exclude: [ string ] # branch names which will not
tags:
include: [ string ] # tag names which will trigger a build
exclude: [ string ] # tag names which will not
paths:
include: [ string ] # file paths which must match to trigger a build
exclude: [ string ] # file paths which will not trigger a build
And with checkout step, for example:
resources:
repositories:
- repository: MyAzureReposGitRepository # In a different organization
endpoint: MyAzureReposGitServiceConnection
type: git
name: OtherProject/MyAzureReposGitRepo
trigger:
- main
pool:
vmImage: 'ubuntu-latest'
steps:
- checkout: self
- checkout: MyAzureReposGitRepository
#Shayki
Not exactly what I want. I use checkout step but with parameter:
- checkout: git://project/repo#{{parameters.BRANCH_NAME}}
Because of this, I cannot use the resources. And for parameters I have:
parameters:
- name: BRANCH_NAME
type: string
default: develop
This is static method. I don't know how to dynamically fetch branches for a repository and pass them as values for the parameter.
I have four YAML "release" pipelines where I use the same YAML syntax to define a continuation trigger. Here is the YAML definition for the trigger.
resources:
pipelines:
- pipeline: Build # Name of the pipeline resource
source: BuildPipeline # Name of the pipeline as registered with Azure DevOps
trigger: true
Not really sure about this syntax where I don't specify any branch but everything was working fine till recently. More recently I updated two of the YAML release pipelines and they now are not getting triggered when the build pipeline completes. All pipelines if executed manually work fine.
All release pipelines have the same YAML for the continuation trigger definition (see above) and have the same branch set for "Default branch for manual and scheduled builds".
I don't know how to investigate why some of the release pipelines are not triggered (any log available somewhere?) and I don't see them executed and failing, they simply are not being triggered. How do I investigate this issue?
For your question about investigating the logs - you can see what pipeline runs were created, but unfortunately you can't see what wasn't. So far as Azure DevOps is concerned, if "nothing occurred" to set off a trigger, then there's nothing to log.
As for the pipelines themselves not triggering, from the pipeline editor, check the trigger settings to ensure that nothing is set there - UI and YAML settings tend to cancel one another out:
Finally, if you want to specify a branch, you can use some combination of the following options:
resources:
pipelines:
- pipeline: Build # Name of the pipeline resource
source: BuildPipeline # Name of the pipeline as registered with Azure DevOps
trigger:
branches:
include: # branch names which will trigger a build
exclude: # branch names which will not
tags:
include: # tag names which will trigger a build
exclude: # tag names which will not
paths:
include: # file paths which must match to trigger a build
exclude: # file paths which will not trigger a build
I believe I found the issue and it's the removal of the following statements from my deploy pipelines
pool:
vmImage: windows-2019
I removed these statements because I transformed all jobs into deployment jobs as follows
- deployment: MyDeployJob
displayName: 'bla bla bla'
environment:
name: ${{ parameters.AzureDevopsEnv }}
resourceType: VirtualMachine
resourceName: ${{ parameters.AzureDevopsVM }}
The pipelines with no pool statement run perfectly well if started manually but I'm convinced fail at being triggered if started via the pipeline completion trigger. I do not understand this behavior but I placed the pool statement back in all deploy pipelines and all are now getting triggered as the build pipeline completes.
I found that when defining the resource pipeline (trigger) in a template that you extend in the depending pipeline, there are two things that can prevent builds from being triggered:
There are syntax errors in the template (or the parent .yaml)
The depending pipeline needs to be updated before Azure Devops realizes you made edits to the template it extends
This worked for me:
template.yaml
stages:
- stage: SomeBuildStage
displayName: Build The Project
jobs:
- job: SomeJob
displayName: Build NuGet package from Project
pool:
name: My Self-hosted Agent Pool # Using Pool here works fine for me, contrary to #whatever 's answer
steps:
- pwsh: |
echo "This template can be extended by multiple pipelines in order to define a trigger only once."
# I still use CI triggers as well here (optional)
trigger:
branches:
include:
- '*'
# This is where the triggering pipeline is defined
resources:
pipelines:
- pipeline: trigger-all-builds # This can be any string
source: trigger-all-builds # This is the name defined in the Azure Devops GUI
trigger: true
depending-pipeline.yaml
extends:
template: template.yaml
# I still use CI triggers as well here (optional)
trigger:
paths:
include:
- some/subfolder
triggering-pipeline.yaml
stages:
- stage: TriggerAllBuilds
displayName: Trigger all package builds
jobs:
- job: TriggerAllBuilds
displayName: Trigger all builds
pool:
name: My Self-hosted Agent Pool
steps:
- pwsh: |
echo "Geronimooo!"
displayName: Geronimo
trigger: none
pr: none
I currently have my YAML pipeline and my application's source code in two different branches.
I was trying find evidence that what is being checked out is indeed the source code's branch and not my pipeline's branch.
I see that the checkout call at the end of the git fetch is to a specific commit, not to the specified branch name.
This is my resources definition:
resources:
repositories:
- repository: RepoName
type: git
name: 'MyRepository' # repository in Azure DevOps
trigger:
branches:
include:
- UAT
and in one of my steps I do a checkout: RepoName.
I was expecting a git checkout UAT after pulling the source code, but as said I see a checkout of a specific commit.
How can I be sure about which branch is being checked out?
By default, the build pipeline will check out the single commit that triggered the current pipeline run. When you manual, or via other methods, trigger a pipeline run, the run will check out the latest commit on default branch or the branch you specify.
However, no matter what methods triggers the pipeline run, you can use the predefined build variable Build.SourceBranch or Build.SourceBranchName to get the branch name where the commit is checked out from.
To view more details, you can see "Use predefined variables".
Typical syntax to display is:
steps:
- bash: echo $(Build.SourceBranch)
In addition, you also can go to the root of the local repository, and execute the git branch command. This command will output the name of the current branch.
For example:
git branch --show-current
If current branch is master, the output:
master
Note that a number of people have reported that this produces no output
You need to set ref
resources:
repositories:
- repository: string # identifier (A-Z, a-z, 0-9, and underscore)
type: enum # see the following "Type" topic
name: string # repository name (format depends on `type`)
ref: string # ref name to use; defaults to 'refs/heads/master'
endpoint: string # name of the service connection to use (for types that aren't Azure Repos)
trigger: # CI trigger for this repository, no CI trigger if skipped (only works for Azure Repos)
branches:
include: [ string ] # branch names which will trigger a build
exclude: [ string ] # branch names which will not
tags:
include: [ string ] # tag names which will trigger a build
exclude: [ string ] # tag names which will not
paths:
include: [ string ] # file paths which must match to trigger a build
exclude: [ string ] # file paths which will not trigger a build
so it would be
resources:
repositories:
- repository: RepoName
type: git
name: 'MyRepository' # repository in Azure DevOps
ref: 'UAT'
trigger:
branches:
include:
- UAT
In ref property you can set like this: ref: 'refs/heads/UAT'
resources:
repositories:
- repository: RepoName
type: git
name: 'MyRepository' # repository in Azure DevOps
ref: 'refs/heads/UAT'
trigger:
branches:
include:
- UAT
Then in steps, you need to add - checkout: RepoName
Here is my sample you can refer to:
pool:
vmImage: 'windows-2019'
trigger: none
resources:
repositories:
- repository: RepoName
type: git
name: '{proName}/{repoName}' # repository in Azure DevOps
ref: 'refs/heads/dev'
steps:
- checkout: RepoName
In build summary page:
In log :
Pushing in the dev branch triggers a build.
But I want only to create a build if the commit in the dev branch has a tag with a format of '?.??.??.?'.
trigger:
- dev
In the documentation I can only find how to trigger based on a tag OR an branch. How can I have two conditions the tag and the commit in the specific branch?
As a workaround you can use multiple jobs(if your pipeline has multiple jobs you can make use of multiple stages) in your pipeline to achieve above requirement.
Add an additional job(or stage)(ie. Job A) to get the tags associated to the commit. And then use script check if the tag has a certain format, if not then fail the job. Then and set the other job dependsOn this job. Check here for more information about dependenies and conditions
So that your job(ie.Job B) which builds your project will only get executed if the additional job succeeded. Please check below example yaml:
trigger:
branches:
include:
- dev
pool:
vmImage: "ubuntu-latest"
jobs:
- job: A
steps:
- powershell: |
#get the tags associated to the commit
$tags = git describe --exact-match $(Build.SourceVersion)
# check the tag format
if($($tags) -notmatch '.\...\...\..'){exit 1}
- job: B
dependsOn: A
condition: succeeded('A')
steps:
- powershell: |
$tags = git describe --exact-match $(Build.SourceVersion)
echo $($tags)