I have a pipeline using a git reposiory
resources:
repositories:
repository: myrepo
type: git
name: src/myrepo
ref: nameofbranch
trigger:
branches:
include:
- triggeringbranch
I want to be able to change repo branch (nameofbranch could be a parameter - for manual run),
but then when the pipeline is automatically triggered by changes on a branch (for example changes on triggeringbranch), I'd like of course the pipeline to use that triggeringbranch...
How to deal with it ?
Can i use some condition to set the value of ref , using Build.SourceBranch if not empty, or nameofbranch otherwise ?
Thank you
You can create a local variable and fill it using the conditions. After that, you can use the variable as input for "ref":
parameters:
- name: "branchName"
default: ""
variables:
- name: "branchName"
${{ if eq(parameters.branchName, '') }}:
value: $(Build.SourceBranch)
${{ else }}:
value: ${{ parameters.branchName }}
resources:
repositories:
- repository: myrepo
type: git
name: src/myrepo
ref: $(branchName)
trigger:
branches:
include:
- triggeringbranch
Related
I am trying to find an alternate solution to using variable in resources/repositories/repository/ref because using a variable is technically not allowed.
resources:
resources:
repositories:
- repository: devops
name: MyProjects/devops
type: git
ref: master
The workaround of doing a git clone of the external repository will not work for me because my dependency on that repository is for referencing the templates.
Example:
- template: Build/Templates/downloadFiles.yaml#devops
Does anyone have a solution? Thank you for reading my post!
Refer to this doc: Template Expressions in Repository Resource Definition
Now, you can use template expressions to choose the branch of a repository resource.
Azure DevOps has supported to use variable in Repo Resource to set the ref. We can use template expression: ${{ variables.var }} to define the ref.
Here is an example:
variables:
branchname: main
resources:
repositories:
- repository: devops
name: 123/Repo90
type: git
ref: ${{ variables['branchname'] }}
pool:
vmImage: ubuntu-latest
steps:
- template: test.yml#devops
In an Azure DevOps pipeline, I had
resources:
repositories:
- repository: self
type: git
name: MyProject.Web
trigger:
- master
- repository: UiRepo
type: git
name: MyProject.Web.UI
trigger:
- main
[other stuff]
steps:
- checkout: self
- checkout: UiRepo
[other stuff]
This has worked fine: The pipeline ran when triggered by completion of a pull request to either the master branch of the "self" repo or the main branch of the UiRepo. The checkouts pulled from the master branch of "self" and the main branch of UiRepo.
Now I'm adding a branch called release/1.0 to each of the repos, and will introduce other release/x.x branches in the future. So now I have:
resources:
repositories:
- repository: self
type: git
name: MyProject.Web
trigger:
- master
- release/*
- repository: UiRepo
type: git
name: MyProject.Web.UI
trigger:
- main
- release/*
[other stuff]
steps: # ??????
- checkout: self
- checkout: UiRepo
[other stuff]
Now the following is required:
If the triggering branch's name ($Build.SourceBranchName) is either 'master' or 'main', check out master from "self" and main from UiRepo.
Otherwise, check out a branch whose value is $Build.SourceBranch from both repos.
At https://learn.microsoft.com/en-us/azure/devops/pipelines/repos/multi-repo-checkout?view=azure-devops#checking-out-a-specific-ref, I've read that I can use inline syntax in the checkout steps to include specific refs. In my case, I think this comes out as follows:
- ${{ if in(variables['Build.SourceBranchName''], 'master', 'main') }}:
- checkout: git://MyProject/MyProject.Web#refs/heads/master
- checkout: git://MyProject/MyProject.Web.UI#refs/heads/main
- ${{ else }}
- checkout: git://MyProject/MyProject.Web#$(Build.SourceBranch)
- checkout: git://MyProject/MyProject.Web.UI#$(Build.SourceBranch)
(or do I need to be using [ variables['Build.SourceBranch'] ] here instead of $(Build.SourceBranch)?)
But, by dispensing with the references in the checkout steps to "self" and UiRepo, this seems to be divorcing the connection between the checkout steps and the repos as I've already defined them. Does that matter? Is this correct anyway? If it isn't, how can I accomplish my goal?
The if/else you are using is named Conditional Insertion in DevOps concept.
Need to clarify is in this concept, it indeed can accept predefined variables， but only which is 'Available in templates'.
I notice you are using Build.Repository.SourceBranchName, this variable not even in the list of predefined variables, let alone the concept of 'Available in templates'. So the first part of your Conditional Insertion will not be process. Only the second part will be able to process.
And please do not use 'self' as the alias of the repository, this will causes ambiguity.(Your current situation use this is no problem, this is just a suggestion.)
So basically, your pipeline definition should be like this:
trigger:
- none
resources:
repositories:
- repository: self
type: git
name: Repo1
trigger:
- master
- release/*
- repository: UiRepo
type: git
name: Repo2
trigger:
- main
- release/*
pool:
vmImage: ubuntu-latest
steps:
- ${{ if in(variables['Build.SourceBranchName'], 'master', 'main') }}: #This place needs to change
- checkout: git://BowmanCP/Repo1#refs/heads/master
- checkout: git://BowmanCP/Repo2#refs/heads/main
- ${{ else }}:
- checkout: git://BowmanCP/Repo1#$(Build.SourceBranch)
- checkout: git://BowmanCP/Repo2#$(Build.SourceBranch)
Works prefect on my side:
My issue
I have an Azure devops project with a template on another repository than main one's.
I try to load the template but I get this:
/azure-pipelines.yml: File /TemplateRepository/build.yml not found in
repository
https://dev.azure.com/XXXXX/TestAzureDevops/_git/TemplateRepository
branch refs/heads/main
What I did
build.yml is my template into repository: TemplateRepository.
azure-pipelines.yml is my main file.
this is my template:
parameters:
- name: 'name'
default: 'name'
type: 'string'
steps:
- script: echo ${{parameters.name}}
This is my main:
pool:
vmimage: 'windows-latest'
parameters:
- name: contexts
type: object
default: [{
name: macOS,
pool: 'macOS-latest',
sign: false
},
{
name: Windows,
pool: 'windows-latest',
sign: true
}]
resources:
repositories:
- repository: Tuto-Ressources
ref: main
type: git
name: TemplateRepository
stages:
- stage: BUILD
jobs:
- job: test
steps:
- checkout: self
- checkout: Tuto-Ressources
- script: dir
displayName: Dir
- ${{ each context in parameters.contexts }}:
- template: .\TemplateRepository\build.yml#Tuto-Ressources
parameters:
name: ${{context.name}}
pool: ${{context.pool}}
sign: ${{context.sign}}
buildSteps:
- checkout: self
- checkout: Tuto-Ressources
- bash: echo "Test module"
What I tested
If I remove the template lines from main script so I just have script: dir I can see my checkout on the VM and build.yml into \TemplateRepository directory.
So there is no reason it can't find my template.
I checked the branch name, it's main and I have no other one. I can see the file from the Devops Portal
What I need
I would like to understand what happen and what I can do.
I know there are artifact, but in this situation I don't understand whay it is not working like this because I don't change job neither stage.
The template line should reference the template file by its path relative to TemplateRepository. If your build template resides at \build.yml in TemplateRepository, then you should just do:
- template: build.yml#Tuto-Ressources
Also, you don't need to do - checkout: Tuto-Ressources - the pipeline engine will know to fetch the template file from referenced repository.
I have pipeline.yml which is stored in repo - buildresources and it also have resources.json file
buildresources repo
pipeline.yml
resources.json
now I want to pass resources.json in following pipeline.yml which further extends template in other repo,how do I pass the resources.json path in buildresources repo to deploy-iac.yml in AzureIacPoC/AzureIacPoc repo
resources:
repositories:
- repository: cloud-iac # The name used to reference this repository in the checkout step
type: git
name: AzureIacPoC/AzureIacPoc
ref: refs/heads/features/azure
trigger:
branches:
include: [features/*, master]
extends:
template: deploy-iac.yml#cloud-iac
parameters:
resourceGroupName: 'anuj-test-automation'
location: 'Canada Central'
csmfile: resources.json
environment: 'sandbox'
resources:
repositories:
- repository: AnotherRepoName
type: git
name: YourProjectName/AnotherRepoName
pool:
vmImage: ubuntu-latest
steps:
- checkout: AnotherRepoName
- script: dir $(Build.SourcesDirectory)
displayName: 'Run a one-line script'
I would like to be able to pass a pipeline parameter all the way through my YAML pipeline without having to define a parameter in each and every YAML file.
Essentially I have a main YAML file which calls a stage YAML, that has multiple nested jobs YAML, which in turn calls nested steps YAML; essentially building up my pipeline as I should using templates: https://learn.microsoft.com/en-us/azure/devops/pipelines/process/templates?view=azure-devops
Here's a tree list sample folder;
E:.
├───01_stage (many files per folder)
├───02_jobs (many files per folder)
├───03_steps (many files per folder)
└───...main pipeline files
Ideally I want to run an IF condition on checking out a repository depending upon the pipeline being PROD or NON-PROD. I am fine with defining this as a parameter, but I am also open to it being defined as a variable. As far as I'm aware; you can't use IF condition on variables.
This is fine
- ${{ if eq(parameters.pester, true) }}: # or even as variables['pester']
- name: pester
value: yes
This is not fine
- ${{ if eq(variables.pester, true) }}: # or even as variables['pester']
- name: pester
value: yes
The condition I want this to run is nest far below many templates, and it would be absolutely painful to have to re-code everything to confirm to the parameters value being declared and passed down in each file.
This is where I want it:
steps:
- ${{ if eq(parameters['masterTagged'], 'true') }}: # here
- checkout: masterTagged
displayName: Repo Tagged
- ${{ if ne(parameters['masterTagged'], 'true') }}: # here
- checkout: self
displayName: Repo Self
- template: /.pipelines/03_steps/ssh_install.yml
- template: /.pipelines/03_steps/tf_install.yml
parameters:
terraformVersion: ${{ parameters['terraformVersion'] }}
- ...many more templates
Here is my main YAML pipeline file:
parameters:
- name: artifactory_base
type: boolean
default: true
# ...many more params
- name: pester
type: boolean
default: true
- name: planDeploy
type: boolean
default: true
- name: useBackupAgent
type: boolean
default: false
- name: masterTagged # key param
type: boolean
default: true
name: Team2
pr: none
resources:
repositories:
- repository: masterTagged
endpoint: nationwide-ccoe
name: my-github-org/my-github-repo
type: github
ref: refs/tags/v2.0.3
trigger: none
variables:
- template: /.pipelines/config/sub-asdfasdf.config.yml
- template: /.pipelines/config/namingstd.config.yml
- ${{ if eq(parameters.artifactory_base, true) }}:
- name: artifactory_base
value: yes
# ...many more conditions
- ${{ if eq(parameters.pester, true) }}:
- name: pester
value: yes
- ${{ if eq(parameters.planDeploy, true) }}:
- name: planDeploy
value: yes
stages:
- template: /.pipelines/01_stage/lz_deploy.yml
parameters:
${{ if eq(parameters.useBackupAgent, false) }}:
pool:
vmImage: Ubuntu 18.04
${{ if eq(parameters.useBackupAgent, true) }}:
pool:
name: backupAgents
terraformVersion: $(TERRAFORM_VERSION)
Is it possible to set this masterTagged parameter and for it to filter all the way down without having to declare it each time?
Also; is it even possible to use variables instead of parameters in this manner (I understand that parameters expand before variables):
- ${{ if eq(variables.pester, true) }}: # or even as variables['pester']
- name: pester
value: yes
...if it is, have I been doing it wrong all this time?
Note:
I do understand that you can use a standard task condition on the checkout task (shown below); however, having a 'switch' on two tasks ruins the folder path of the checked out repository. Even though we're only checking out on repository, it adds another folder level to the $SYSTEM_DEFAULTWORKINGDIRECTORY. Doing it this way would require more re-coding on the current structure of my YAML piplines.
- checkout: masterTagged
condition: eq(variables['masterTagged'], 'true')
displayName: Repo Tagged
- checkout: self
condition: ne(variables['masterTagged'], 'true')
displayName: Repo Self
If I could, but I know it's not possible (as seen by other peoples requests), I would enable a parameter or variable on the repository reference:
resources:
repositories:
- repository: masterTagged
endpoint: nationwide-ccoe
name: my-github-org/my-github-repo
type: github
ref: ${{ parameters.repoRef }} # here
Is it possible to set this masterTagged parameter and for it to filter all the way down without having to declare it each time?
No, because parameters are “scoped” to the file they are defined with. This is due to them being expanded when the pipeline is first compiled. (See > Pipeline run sequence)
You can use IF conditions on variables, however you can’t use template expressions (wrapped with {{}}) on variables inside templates as the variables do not exist/have not been populated at the point of template expansion.
One option is just using the conditions on the checkout tasks as you suggested, and dealing with the extra folder level to the default working directory. I had to do something similar a while back, our solution was to copy the contents of the repo folder up a level into the default working directory.
Your other option is to do the checkout in the top level pipeline file. This will allow you to template the checkout step/s using the parameter without having to pass it all the way through the files. This is the option I would suggest as you do not have to deal with the folder structure issues of the first option.
This would look something like this:
parameters:
- name: masterTagged
default: true
type: boolean
resources:
repositories:
- repository: masterTagged
endpoint: nationwide-ccoe
name: my-github-org/my-github-repo
type: github
ref: refs/tags/v2.0.3
steps:
- ${{ if eq(parameters.masterTagged, true) }}:
- checkout: masterTagged
- ${{ if eq(parameters.masterTagged, false) }}:
- checkout: self
- template: ./path/to/template.yml
I hope this answers your question.