Azure DevOps yaml pipeline : how to know which branch is being checked out? - azure-devops

I currently have my YAML pipeline and my application's source code in two different branches.
I was trying find evidence that what is being checked out is indeed the source code's branch and not my pipeline's branch.
I see that the checkout call at the end of the git fetch is to a specific commit, not to the specified branch name.
This is my resources definition:
resources:
repositories:
- repository: RepoName
type: git
name: 'MyRepository' # repository in Azure DevOps
trigger:
branches:
include:
- UAT
and in one of my steps I do a checkout: RepoName.
I was expecting a git checkout UAT after pulling the source code, but as said I see a checkout of a specific commit.
How can I be sure about which branch is being checked out?

By default, the build pipeline will check out the single commit that triggered the current pipeline run. When you manual, or via other methods, trigger a pipeline run, the run will check out the latest commit on default branch or the branch you specify.
However, no matter what methods triggers the pipeline run, you can use the predefined build variable Build.SourceBranch or Build.SourceBranchName to get the branch name where the commit is checked out from.
To view more details, you can see "Use predefined variables".
Typical syntax to display is:
steps:
- bash: echo $(Build.SourceBranch)
In addition, you also can go to the root of the local repository, and execute the git branch command. This command will output the name of the current branch.
For example:
git branch --show-current
If current branch is master, the output:
master
Note that a number of people have reported that this produces no output

You need to set ref
resources:
repositories:
- repository: string # identifier (A-Z, a-z, 0-9, and underscore)
type: enum # see the following "Type" topic
name: string # repository name (format depends on `type`)
ref: string # ref name to use; defaults to 'refs/heads/master'
endpoint: string # name of the service connection to use (for types that aren't Azure Repos)
trigger: # CI trigger for this repository, no CI trigger if skipped (only works for Azure Repos)
branches:
include: [ string ] # branch names which will trigger a build
exclude: [ string ] # branch names which will not
tags:
include: [ string ] # tag names which will trigger a build
exclude: [ string ] # tag names which will not
paths:
include: [ string ] # file paths which must match to trigger a build
exclude: [ string ] # file paths which will not trigger a build
so it would be
resources:
repositories:
- repository: RepoName
type: git
name: 'MyRepository' # repository in Azure DevOps
ref: 'UAT'
trigger:
branches:
include:
- UAT

In ref property you can set like this: ref: 'refs/heads/UAT'
resources:
repositories:
- repository: RepoName
type: git
name: 'MyRepository' # repository in Azure DevOps
ref: 'refs/heads/UAT'
trigger:
branches:
include:
- UAT
Then in steps, you need to add - checkout: RepoName
Here is my sample you can refer to:
pool:
vmImage: 'windows-2019'
trigger: none
resources:
repositories:
- repository: RepoName
type: git
name: '{proName}/{repoName}' # repository in Azure DevOps
ref: 'refs/heads/dev'
steps:
- checkout: RepoName
In build summary page:
In log :

Related

Azure Devops pipelines - clone multiple repos by same tag or branch

I have three Azure Repos, all in the same ADO Project - repo-0001, repo-0002, repo-0003
repo-0001 has my yaml pipeline which has a manual trigger, and the pipeline needs to be run on demand by either branch or tag (eg release/branch001, release/tag001)
The Pipeline checkouts out repo-0001, repo-0002 & repo-0003 at whatever ref (branch or tag) is used to start the pipeline in repo-0001 (these branches & tags are created by a different process and are always present in all repos)
I've been using $(Build.SourceBranch) to extract:
branch - refs/heads/release/branch001
tag - refs/tags/release/tag001
This yaml works for a pipeline started using a branch, but not for a tag, and it results in Could not get the latest source version for repository repo-0001 hosted on Azure Repos using refs/heads/refs/tags/release/tag001 so it appears to append the whole ref path of the tag to refs/heads
variables:
REPOSITORY_SOURCE_BRANCH: $(Build.SourceBranch)
resources:
repositories:
- repository: repo-0001
type: git
name: aks/repo-0001
ref: $(REPOSITORY_SOURCE_BRANCH)
- repository: repo-0002
type: git
name: aks/repo-0002
ref: $(REPOSITORY_SOURCE_BRANCH)
- repository: repo-0003
type: git
name: aks/repo-0003
ref: $(REPOSITORY_SOURCE_BRANCH)
I can make it work with a tag by doing something like this, or passing in as a parameter
variables:
tag: "release/tag001"
resources:
repositories:
- repository: repo-0001
type: git
name: aks/repo-0001
ref: 'refs/tags/$(tag)'
etc
But that's not what I need...
Can a tag ref be passed in from pipeline variables to make this work, and is there a way to make it flexible so that I can use the same pipeline code for tags and branches?
Thanks!
The ref field in repositories Resources doesn't support defining Pipeline variable. It only supports hardcode the ref value.
Can a tag ref be passed in from pipeline variables to make this work, and is there a way to make it flexible so that I can use the same pipeline code for tags and branches?
To meet your requirement, you can change to use the following format to checkout the repo.
- checkout: git://MyProject/MyRepo#refs/tags/$(tag)
- checkout: git://MyProject/MyRepo#$(REPOSITORY_SOURCE_BRANCH)
Here is an example:
variables:
REPOSITORY_SOURCE_BRANCH: test
steps:
- checkout: git://aks/repo-0001#$(REPOSITORY_SOURCE_BRANCH)
For more detailed info, you can refer to this doc: Checking out a specific ref

DevOps Build Pipeline - Templates - Run from the same branch

I am testing a POC build pipeline and I am trying to use templates as a sort of reference library, so I can reuse the same code in multiple pipelines, while also allowing for simpler editing/updating, if changes are required.
I want to be able to run the pipeline from a DevOps branch and refer to the same branch (or tag) when validating/running all the template pipelines in this build pipeline.
The ultimate goal is to pass multiple variables to the template (via a loop/for each) to create multiple build artifacts referencing public repositories (in this test a public terraform github repo). We could then maybe only need to maintain the variable data and not the source/reference code.
I thought DevOps did this 'OOTB', but realised that this was probably not the case. Something like this:
Repo: InfraAsData (all pipelines)
Branch: feature/sparse-clone-repo
Updated azure-pipeline.yaml and added template templates/sparse-clone.yaml (FYI - this template is untested as well)
azure-pipeline.yaml:
trigger: none
resources:
repositories:
- repository: InfraAsData #resource name to be used in the build pipeline
type: git #Azure git
name: 'IAC/InfraAsData'
refs: 'refs/heads/$(branch_name)'
parameters:
- name: branch
displayName: branch
type: string
default: $(Build.SourceBranch)
variables:
- name: branch_name
value: ${{ parameters.branch }}
- template: ./templates/variables/resource-groups.yaml#InfraAsData #variables template
pool:
vmImage: 'ubuntu-latest'
stages:
#sparse clone the target public repo blobs to build pipeline artifacts directory
- stage: Template-Sparse-Clone-To-Artifacts
displayName: 'Test Sparse Clone to Artifacts'
jobs:
- deployment: sparseCloneTfModule
displayName: 'Sparse Clone Terraform module to Artifacts'
environment: Test
strategy:
runOnce:
deploy:
steps:
- template: $(Build.SourceBranch)/templates/sparse-clone.yaml#InfraAsData #Create artifacts based on template variables
parameters: #module specific var from var template
repoUrl: '${{ variables[template.repoUrl] }}'
repoPath: '${{ variables[template.repoPath] }}'
artifactPath: '${{ variables[template.artifactPath] }}'
sparse-clone.yaml
parameters:
repoUrl: ''
repoPath: ''
artifactPath: ''
#testing
steps:
- script: |
md $(Build.BinariesDirectory)/$(parameters.artifactPath)
git clone --filter=blob:none --sparse $(parameters.repoUrl) $(parameters.artifactPath)
cd $(Build.BinariesDirectory)/$(parameters.artifactPath)
git sparse-checkout init --cone
git sparse-checkout set $(parameters.repoPath)
git checkout main
dir $(Build.BinariesDirectory)/$(parameters.artifactPath)
displayName: 'Clone Github Repo Subdirectory - filter blob none'
When I run this from the feature/sparse-clone-repo branch in DevOps I get an error that the new template does not exist in the main branch (it doesn't of course).
/build/test/azure-pipeline.yaml: File /templates/variables/resource-groups.yaml not found in repository https://dev.azure.com//IAC/_git/InfraAsData branch refs/heads/main version
I have also tried using no resources.repository, using $(Build.SourceBranch) $(Build.SourceBranchName) as refs value and as the template path reference (like in the 2nd template example).
EDIT:
If i remove the resource.repositories reference, then the correct branch is identified, but the azure-pipeline.yaml relative path is appended to the template path:
/build/test/azure-pipeline.yaml: File /build/test/$(Build.SourceBranch)/templates/variables/resource-groups.yaml not found in repository https://dev.azure.com//IAC/_git/InfraAsData branch refs/heads/feature/pipelines
So correct branch, but not the correct relative path.
EDIT 2:
I can remove the resources section and use a relative path
../../templates/variables/resource-groups.yaml
but this is not dynamic, to a 'root' reference, so would not work if the folder structure were to change (EG I moved the azure-pipeline.yaml up a level)
I'm sure I am missing something obvious or misunderstanding the docs (or pipeline templates). Any pointers gratefully received!
When I run this from the feature/sparse-clone-repo branch in DevOps I
get an error that the new template does not exist in the main branch
(it doesn't of course).
From your description, seems you thought the 'refs' of the resources determine the branch of the alias of the template part.
But it is not, the 'resources.repositories.repository' doesn't have a section named 'refs', only 'ref' is allowed.
In your situation, you used a section named 'refs' which doesn't exist. Pipeline will use the default branch of the repository to looking for the template YAML. So change the default branch of the repository or change the 'refs' to 'ref' will solve the first issue.
And I notice you were using variable in resources section, you have two mistakes.
One is '$()' can't use in compile time part. Another is even compile time variables are also not allowed in 'ref' section.
ref
string
ref name to checkout; defaults to 'refs/heads/main'. The branch
checked out by default whenever the resource trigger fires. Does not
accept variables.
You can use your pipeline like this:
trigger: none
resources:
repositories:
- repository: InfraAsData #resource name to be used in the build pipeline
type: git #Azure git
name: 'BowmanCP/template_branch'
ref: 'refs/heads/${{parameters.branch}}'
parameters:
- name: branch
displayName: branch
type: string
default: main2
pool:
vmImage: 'ubuntu-latest'
stages:
#sparse clone the target public repo blobs to build pipeline artifacts directory
- stage: Template_Sparse_Clone_To_Artifacts
displayName: 'Test Sparse Clone to Artifacts'
jobs:
- deployment: sparseCloneTfModule
displayName: 'Sparse Clone Terraform module to Artifacts'
environment: Test
strategy:
runOnce:
deploy:
steps:
- template: ./templates/sparse-clone.yaml#InfraAsData #Create artifacts based on template variables
parameters: #module specific var from var template
repoUrl: 'xxx'
repoPath: 'xxx'
artifactPath: 'xxx'
Basically, variables can't be used in your situation and there doesn't have a parameter reuse feature. That's everything.

Is there a way I can define the azure pipeline template to be imported from a git tag instead of master?

Consider a yaml based pipeline
resources:
repositories:
- repository: templates
type: git
name: My-Proj/azure-build-templates
...
stages:
- template: test_pipeline/include-build-java-sonarqube.yml#templates
parameters:
agent_pool_name: $(agentPoolName)
maven_goal: 'mvn clean package'
...
This will refer to repository azure-build-templates in branch master. Is there a way I can declare to refer a tag instead of master? like name: My-Proj/azure-build-templates#release-20211215-better-tag
Context of its utility: Recently we pushed something in master which consequently broke all the pipelines referring. As a fix we had to rollback with git operations on master branch. I wonder if there would be a way to refer from tag like jenkins and rollback would be just point from one git tag to another. (Downside on every release you have update all the pipelines, so I am up for any suggestions for templates release management). Thanks.
ref: https://learn.microsoft.com/en-us/azure/devops/pipelines/process/resources?view=azure-devops&tabs=schema#define-a-repositories-resource
You can link to any ref.
Tags are represented in git as refs/tags/myTag
resources:
repositories:
- repository: string # identifier (A-Z, a-z, 0-9, and underscore)
type: enum # see the following "Type" topic
name: string # repository name (format depends on `type`)
ref: string # ref name to use; defaults to 'refs/heads/main'
You can list your tags with: git show-ref --tags

Azure devops pipeline - get branches from other repository

I have two repositories:
devops - with definitions of pipelines
src_project - with sources of project
Default when I run a Azure pipeline I can choice between branch/tag of repository with pipeline definition. I would like to add a parameter with which I can select a branch from the source project.
Is this possible in a simple way in Azure? I don't want, I cannot keep definitions of pipelines in src_project.
In Jenkins, we used an additional method to fetch these branches, using the shared libraries plugin.
You can do it with resources from type repositories:
resources:
repositories:
- repository: string # identifier (A-Z, a-z, 0-9, and underscore)
type: enum # see the following "Type" topic
name: string # repository name (format depends on `type`)
ref: string # ref name to use; defaults to 'refs/heads/master'
endpoint: string # name of the service connection to use (for types that aren't Azure Repos)
trigger: # CI trigger for this repository, no CI trigger if skipped (only works for Azure Repos)
branches:
include: [ string ] # branch names which will trigger a build
exclude: [ string ] # branch names which will not
tags:
include: [ string ] # tag names which will trigger a build
exclude: [ string ] # tag names which will not
paths:
include: [ string ] # file paths which must match to trigger a build
exclude: [ string ] # file paths which will not trigger a build
And with checkout step, for example:
resources:
repositories:
- repository: MyAzureReposGitRepository # In a different organization
endpoint: MyAzureReposGitServiceConnection
type: git
name: OtherProject/MyAzureReposGitRepo
trigger:
- main
pool:
vmImage: 'ubuntu-latest'
steps:
- checkout: self
- checkout: MyAzureReposGitRepository
#Shayki
Not exactly what I want. I use checkout step but with parameter:
- checkout: git://project/repo#{{parameters.BRANCH_NAME}}
Because of this, I cannot use the resources. And for parameters I have:
parameters:
- name: BRANCH_NAME
type: string
default: develop
This is static method. I don't know how to dynamically fetch branches for a repository and pass them as values for the parameter.

Predefined variables for repository resource trigger

When a pipeline is triggered by a repository resource trigger I need to be able to determine metadata about the triggering repository (such as repo name and branch).
It sounds like these variables should be set during a pipeline run that was triggered by a repository resource; however, their values are blank when I do echo $(resources.triggeringAlias) or env | sort in a pipeline run that was triggered by a repository resource.
UPDATE: the predefined variables for Build.Repository.Name and Build.SourceBranchName now work as expected when used with a repository trigger. So, while I'm no longer in need of resources.triggeringAlias and resources.triggeringCategory, they still aren't working.
I need to be able to determine metadata about the triggering
repository (such as repo name and branch)
We can use $(Build.Repository.Name) and $(Build.Repository.Uri) to get repo name and repo uri.
And $(Build.SourceVersion) can be used to get CommitId, $(Build.SourceBranch) or $(Build.SourceBranchName) can be used to get branch info.
Just note we must also checkout the triggering repo to make above variables work to fetch the info of triggering repo, otherwise those variables will always represents the value of triggered repo:
steps:
- checkout: self
- checkout: TheTriggeringRepo
Some details:
I have a triggering repo PipelineA, and the triggered repo PipelineB. PipelineB's azure-pipelines.yml file:
resources:
repositories:
- repository: PipelineA
type: git
name: PipelineA
trigger:
- master
steps:
- checkout: self
- checkout: PipelineA
- task: Bash#3
inputs:
targetType: 'inline'
script: |
echo $(Build.Repository.Name)
echo $(Build.SourceBranch)
So this pipeline will be triggered by both PipelineA repo and PipelineB repo.
When PipelineB repo has changes:
When PipelineA repo has changes:
It's clear the $(Build.Repository.Name) variable can work well to output the real trigger repo if we checkout both these two repos. So just make sure you checkout those triggering repos, then my variables above would work for you.
resources:
pipelines:
- pipeline: client-qa-deployment ## Resources.TriggeringAlias will be 'client-qa-deployment' if this resource gets triggered.
source: benjose.dev-code
trigger:
stages:
- client_deploy_qa
- stage: qa_smoketest
displayName: Smoke Test on QA Environment
condition: eq(variables['Resources.TriggeringAlias'], 'client-qa-deployment')
pool:
vmImage: 'windows-latest'
jobs:
- job: TestQA
steps:
- script: echo Hello, $(Resources.TriggeringAlias)!
displayName: 'Running Smoke Test on QA Environment'
Hello, client-staging-deployment!
I believe the above snippet will help you to understand the syntax and usage of Resources.TriggeringAlias. This variable will be helpful if we have multiple resources in the same pipeline and to find out which resource got trigged.