Azure DevOps multi-repo, multi-branch-trigger, selecting the branch to build from - azure-devops

In an Azure DevOps pipeline, I had
resources:
repositories:
- repository: self
type: git
name: MyProject.Web
trigger:
- master
- repository: UiRepo
type: git
name: MyProject.Web.UI
trigger:
- main
[other stuff]
steps:
- checkout: self
- checkout: UiRepo
[other stuff]
This has worked fine: The pipeline ran when triggered by completion of a pull request to either the master branch of the "self" repo or the main branch of the UiRepo. The checkouts pulled from the master branch of "self" and the main branch of UiRepo.
Now I'm adding a branch called release/1.0 to each of the repos, and will introduce other release/x.x branches in the future. So now I have:
resources:
repositories:
- repository: self
type: git
name: MyProject.Web
trigger:
- master
- release/*
- repository: UiRepo
type: git
name: MyProject.Web.UI
trigger:
- main
- release/*
[other stuff]
steps: # ??????
- checkout: self
- checkout: UiRepo
[other stuff]
Now the following is required:
If the triggering branch's name ($Build.SourceBranchName) is either 'master' or 'main', check out master from "self" and main from UiRepo.
Otherwise, check out a branch whose value is $Build.SourceBranch from both repos.
At https://learn.microsoft.com/en-us/azure/devops/pipelines/repos/multi-repo-checkout?view=azure-devops#checking-out-a-specific-ref, I've read that I can use inline syntax in the checkout steps to include specific refs. In my case, I think this comes out as follows:
- ${{ if in(variables['Build.SourceBranchName''], 'master', 'main') }}:
- checkout: git://MyProject/MyProject.Web#refs/heads/master
- checkout: git://MyProject/MyProject.Web.UI#refs/heads/main
- ${{ else }}
- checkout: git://MyProject/MyProject.Web#$(Build.SourceBranch)
- checkout: git://MyProject/MyProject.Web.UI#$(Build.SourceBranch)
(or do I need to be using [ variables['Build.SourceBranch'] ] here instead of $(Build.SourceBranch)?)
But, by dispensing with the references in the checkout steps to "self" and UiRepo, this seems to be divorcing the connection between the checkout steps and the repos as I've already defined them. Does that matter? Is this correct anyway? If it isn't, how can I accomplish my goal?

The if/else you are using is named Conditional Insertion in DevOps concept.
Need to clarify is in this concept, it indeed can accept predefined variables, but only which is 'Available in templates'.
I notice you are using Build.Repository.SourceBranchName, this variable not even in the list of predefined variables, let alone the concept of 'Available in templates'. So the first part of your Conditional Insertion will not be process. Only the second part will be able to process.
And please do not use 'self' as the alias of the repository, this will causes ambiguity.(Your current situation use this is no problem, this is just a suggestion.)
So basically, your pipeline definition should be like this:
trigger:
- none
resources:
repositories:
- repository: self
type: git
name: Repo1
trigger:
- master
- release/*
- repository: UiRepo
type: git
name: Repo2
trigger:
- main
- release/*
pool:
vmImage: ubuntu-latest
steps:
- ${{ if in(variables['Build.SourceBranchName'], 'master', 'main') }}: #This place needs to change
- checkout: git://BowmanCP/Repo1#refs/heads/master
- checkout: git://BowmanCP/Repo2#refs/heads/main
- ${{ else }}:
- checkout: git://BowmanCP/Repo1#$(Build.SourceBranch)
- checkout: git://BowmanCP/Repo2#$(Build.SourceBranch)
Works prefect on my side:

Related

Github action runs on pr unexpectedly

I have this deveopment.yml in .github/workflows:
name: Development
on:
push:
branches:
- dev
paths-ignore:
- '**.md'
jobs:
test:
runs-on: ubuntu-latest
permissions:
id-token: write
contents: write
steps:
- name: "Checkout code"
uses: actions/checkout#v2
- name: "Test Suite"
uses: ./.github/actions/test_suite
lint:
[...lint stuff]
db_migrate:
[...db migrate stuff]
[etc]
[etc]
I would expect this workflow to only fire when I push to the branch named dev
I also have a pull_request.yml workflow. It's similar:
on:
pull_request:
branches:
- '*'
paths-ignore:
- '**.md'
jobs:
test:
runs-on: ubuntu-latest
permissions:
id-token: write
contents: write
steps:
- name: "Checkout code"
uses: actions/checkout#v2
- name: "Test Suite"
uses: ./.github/actions/test_suite
lint:
runs-on: ubuntu-latest
permissions:
id-token: write
contents: write
steps:
- name: "Checkout code"
uses: actions/checkout#v2
- name: "Lint Suite"
uses: ./.github/actions/linting
I would expect this to run on any pull request to any branch, before it merges.
So far so good. If I make a PR from a feature branch into dev, it runs test and lint. When I merge, it runs the development.yml actions like db_migrate.
My question is why when I make a PR of dev into prod, it seems to run the development.yml actions. As soon as the PR is opened, it runs db_migrate. Why is that, and how do I make it not do that?
There is a production.yml but that's set to on release, so I don't think that's relevant here
on:
release:
types: [published]
paths-ignore:
- '**.md'
I've been staring at the docs but I'm having a slow brain morning. It's a little tricky to debug this because it's tightly coupled to github and two important branches.

Azure Devops Pipeline - Repository branch on trigger

I have a pipeline using a git reposiory
resources:
repositories:
repository: myrepo
type: git
name: src/myrepo
ref: nameofbranch
trigger:
branches:
include:
- triggeringbranch
I want to be able to change repo branch (nameofbranch could be a parameter - for manual run),
but then when the pipeline is automatically triggered by changes on a branch (for example changes on triggeringbranch), I'd like of course the pipeline to use that triggeringbranch...
How to deal with it ?
Can i use some condition to set the value of ref , using Build.SourceBranch if not empty, or nameofbranch otherwise ?
Thank you
You can create a local variable and fill it using the conditions. After that, you can use the variable as input for "ref":
parameters:
- name: "branchName"
default: ""
variables:
- name: "branchName"
${{ if eq(parameters.branchName, '') }}:
value: $(Build.SourceBranch)
${{ else }}:
value: ${{ parameters.branchName }}
resources:
repositories:
- repository: myrepo
type: git
name: src/myrepo
ref: $(branchName)
trigger:
branches:
include:
- triggeringbranch

How to point sonarqube Azure Devops pipeline to scan a different branch in another repo?

I have a separate pipeline in Azure Devops for sonar analysis, which scans another repo in Azure Devops, however, it is only scanning the master branch, I would like it to scan specific branches, ideally, all feature branches or just our test branch.
I've tried a few things but unable to get this working, here is part of my sonar pipeline code (removed/changed any names/sensitive info):
resources:
repositories:
- repository: platform-code
type: git
name: platform-code
#pipelines:
# - pipeline: platform-build
# source: platform-build
# trigger:
# branches:
# - feature/awesomeaddition
#trigger:
# branches:
# include:
# - master
# - features/*
stages:
- stage: Sonarqube
variables:
- template: variables.yaml
jobs:
- job: SonarqubeScan
steps:
- checkout: self
- checkout: platform-code
- task: SonarQubePrepare#4
displayName: 'SonarQube Prepare'
inputs:
SonarQube: ${{ variables.SonarQubeServiceConnectionName }}
scannerMode: 'CLI'
configMode: 'manual'
cliProjectKey: ${{ variables.cliProjectKey }}
cliProjectName: ${{ variables.cliProjectName }}
cliSources: '$(Build.SourcesDirectory)/platform-build'
extraProperties: |
sonar.branch.name='feature/awesomeaddition'

Azure Devops pipeline can't find a checkouted template

My issue
I have an Azure devops project with a template on another repository than main one's.
I try to load the template but I get this:
/azure-pipelines.yml: File /TemplateRepository/build.yml not found in
repository
https://dev.azure.com/XXXXX/TestAzureDevops/_git/TemplateRepository
branch refs/heads/main
What I did
build.yml is my template into repository: TemplateRepository.
azure-pipelines.yml is my main file.
this is my template:
parameters:
- name: 'name'
default: 'name'
type: 'string'
steps:
- script: echo ${{parameters.name}}
This is my main:
pool:
vmimage: 'windows-latest'
parameters:
- name: contexts
type: object
default: [{
name: macOS,
pool: 'macOS-latest',
sign: false
},
{
name: Windows,
pool: 'windows-latest',
sign: true
}]
resources:
repositories:
- repository: Tuto-Ressources
ref: main
type: git
name: TemplateRepository
stages:
- stage: BUILD
jobs:
- job: test
steps:
- checkout: self
- checkout: Tuto-Ressources
- script: dir
displayName: Dir
- ${{ each context in parameters.contexts }}:
- template: .\TemplateRepository\build.yml#Tuto-Ressources
parameters:
name: ${{context.name}}
pool: ${{context.pool}}
sign: ${{context.sign}}
buildSteps:
- checkout: self
- checkout: Tuto-Ressources
- bash: echo "Test module"
What I tested
If I remove the template lines from main script so I just have script: dir I can see my checkout on the VM and build.yml into \TemplateRepository directory.
So there is no reason it can't find my template.
I checked the branch name, it's main and I have no other one. I can see the file from the Devops Portal
What I need
I would like to understand what happen and what I can do.
I know there are artifact, but in this situation I don't understand whay it is not working like this because I don't change job neither stage.
The template line should reference the template file by its path relative to TemplateRepository. If your build template resides at \build.yml in TemplateRepository, then you should just do:
- template: build.yml#Tuto-Ressources
Also, you don't need to do - checkout: Tuto-Ressources - the pipeline engine will know to fetch the template file from referenced repository.

Triggers on release branch X strategy

Question: how do you setup CI/CD in YAML pipelines for following context.
branches
master
release/{ALPHABETICAL NAME} ex. release/Albert next release is release/Bertrand and so on.
environments
accept: everything that's pushed on master
test: latest release ex. release/Bertrand
sandbox: latest release -1 (here we can test hotfixes) ex. release/Albert
live: latest release -1 (with hotfixes)
Closest solution
build: creates project artifacts
build.yml
trigger:
- master
- release/*
pool:
vmImage: 'ubuntu-latest'
steps:
- powershell: |
New-Item -Path . -Name "testfile1.txt" -ItemType "file" -Value "This is a text string."
- publish: $(Pipeline.workspace)
artifact: testArtifact
release-phase1: deploys master branch to accept
release-phase1.yml
trigger: none
resources:
pipelines:
- pipeline: pipelineId
source: build
trigger:
branches:
- master
pool:
vmImage: 'ubuntu-latest'
jobs:
- deployment: DeployWeb
environment: 'testenvironment'
strategy:
runOnce:
deploy:
steps:
- script: echo FOO
release-phase2: deploys release branch to test
release-phase2.yml
trigger: none
resources:
pipelines:
- pipeline: pipelineId
source: build
trigger:
branches:
- release/current
pool:
vmImage: 'ubuntu-latest'
jobs:
- deployment: DeployWeb
environment: 'testenvironment'
strategy:
runOnce:
deploy:
steps:
- script: echo FOO
release-phase3: deploys release-1 branch to sandbox and after manual approval to live
release-phase3.yml
trigger: none
resources:
pipelines:
- pipeline: pipelineId
source: build
trigger:
branches:
- release/previous
pool:
vmImage: 'ubuntu-latest'
jobs:
- deployment: DeployWeb
environment: 'testenvironment'
strategy:
runOnce:
deploy:
steps:
- script: echo FOO
Reasons why this solutions doesn't fulfill our needs:
the names of the release branches aren't static.
we should be able to run release-phase3.yml pipeline without running a build on this branch firts. It should download artifacts from the latest build of that branch. Which is not the case.
SHORT ON PURPOSE
Since you have multiple branches (master and releases branches), different branch is built and deploy to different environment. So you can try having the CI build yaml pipeline in each branch and put the CD deployment yaml pipeline in a template yaml in master branch.(You have to have the build yaml file in each branch to get the code in this branch built. You can check this thread).
Below is a simple example.
In master branch
There are azure-pipelines.yml and a template-deploy.yml. In azure-pipelines.yml the Environment value will be passed as a parameter to template-deploy.yml. So that the build will be deployed to its corresponding environment.
azure-pipelines.yml:
trigger:
- master
- release/*
pool:
vmImage: 'windows-latest'
resources:
repositories:
- repository: deploy
type: git
name: {project name}
jobs:
- job: Build
steps:
- script: echo "start build job"
- template: template-deploy.yml#deploy
parameters:
envir: "prod"
template-deploy.yml:
parameters:
envir: ""
jobs:
- deployment: DeployWeb
environment: '${{parameters.envir}}'
strategy:
runOnce:
deploy:
steps:
- script: echo FOO
In the release branches
You can define its individual ci build yaml like below example:
azure-pipelines.yml in release-phase2 branch:
pool:
vmImage: 'windows-latest'
resources:
repositories:
- repository: deploy
type: git
name: {project name}
jobs:
- job: Build
steps:
- script: echo "start build job"
- template: template-deploy.yml#deploy
parameters:
envir: "test"