Cannot trigger build via push to another repository in a YML Pipeline - azure-devops

I have the following repositories:
my-app-repo - Contains the code for the app
pipeline-repo - A collection of pipelines that build my-app-repo as well as others
I'm trying to build a yml pipeline within pipeline-repo that will be triggered on a commit to my-app-repo.
According to the official yml documentation, it sounds possible but I'm not able to get it working.
Here's what I've tried:
pipeline-repo/my-app-repo-build.yml
resources:
repositories:
- repository: target_repo
type: git
name: my-project/my-app_repo
trigger:
branches:
include:
- master
jobs:
- job:
steps:
- script: echo "Should be triggered from a push to my-app-repo!"
The build is not triggering when I push to my-app-repo. It only kicks off for commits to the source repo (pipeline-repo) which I cannot change since that holds the yml definition.
Am I missing something easy?

Edit:
I see you've added an issue to the github repository you mentioned. I agree that it seems to be broken.
While you aren't exactly trying to provide template functionality to your pipelines, you might use it as a work around until the issue you created is addressed.
I've tested this in my playground and it seems to work.
In the app repository
resources:
repositories:
- repository: templates
type: git
name: Pipeline-Templates
trigger:
branches:
include:
- master
jobs:
- template: azure-pipelines-template.yml#templates
In the pipelines repository
jobs:
- job: Get_Last_10_Commits
pool:
vmImage: 'vs2017-win2016'
steps:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
cd $(Build.SourcesDirectory)
Write-Host "Show git log (last 10):"
git log --oneline -10
One "advantage" of this work around is that you no longer need to specify that the repository to checkout is the resource repository and not the self (your template-repository) as the self repository is the app-repo.
This would allow you to restrict changes to the pipeline core by having it in a separate repo like you want, but would still trigger on the app-repo master.
Not ideal b/c you now have 2x .yml files for each build you need, but that's basically the definition of work around: not ideal.
It looks like your include syntax is wrong. Have you tried to use the simple syntax?
All of the examples (and my experience) show that you should use the wildcard syntax when your trigger has an include or exclude specification.
resources:
repositories:
- repository: myPHPApp
type: GitHub
connection: myGitHubConnection
source: ashokirla/phpApp
trigger:
branches:
include:
- features/*
exclude:
- features/experimental/*
paths:
exclude:
- README.md

Related

Azure Pipeline configurations

I am facing a problem that is confusing me a bit and I would like to find the best approach to avoid to repeat my pipelines.
in this GitHub repos I have my yaml file to build the project, and this yaml targets a folder template on which it runs the C# Build and Publish. Approximately the GitHub repo is structure as follow:
- Folder 1
- Folder 2
- Folder 3
- Azure-Pipelines(build and Publish)
- Azure-pipeline.yaml
During the pipeline run, my yaml targets the Àzure-Pipelines(Build and Publish) folder and build the project. This is my Azure-pipeline.yaml file
stages:
- stage: Build
displayName: 'Build'
jobs:
- template: Azure-Pipelines/build.yaml
parameters:
solution: 'Solution-to-build'
- stage: Publish
displayName: 'Release and Push'
jobs:
- template: Azure-Pipelines/publish.yaml
parameters:
<All the parameters configured for this yaml file>
The template and the structure of my GitHub, keeps repeating themselves, as in each gitrepo I have that Azure-pipeline folder. What I am trying to to. Is to have a GitHub repo Where I keep the build.yaml and publish.yaml. and make all the other repos refer to this folder when the pipeline runs.
Is there any way how I can achieve this?
Please if I am missing any details to make my point clear, just ask. Thank you so much in advance
Is there any way how I can achieve this?
You could try to use the Resources in YAML Pipeline.
Here are the steps:
Step1: Create a Github Service Connection in Project Settings -> Service connection .
Step2: You could try to use the following sample to use the yaml template from another Github Repo.
Example
resources:
repositories:
- repository: MyGitHubRepo # The name used to reference this repository in the checkout step
type: github
endpoint: serviceconnectionname
name: githuborg/reponame #e.g. yy/test
ref: main
pool:
vmImage: ubuntu-latest
stages:
- stage: Build
displayName: 'Build'
jobs:
- template: Azure-Pipelines/build.yml#MyGitHubRepo

Determining the triggering branch with a multi-repo CI setup in Azure Devops

In ADO, you can create a "repository resource" per this documentation. The "trigger" section allows you to define a CI trigger for any Azure repo in your space. Therefore, given the following config:
Repos:
AzureRepo1 - Contains project files that should be built
AzureRepo2 - Contains pipeline file 'pipeline.yml'
resources:
repositories:
- repository: "Azure_Repo_1"
type: git
name: AzureRepo1
ref: development
trigger:
branches:
include:
- development
- staging
- production
pool:
vmImage: 'windows-latest'
jobs:
- template: Template.yml
parameters:
service: "development"
run_tests: true
When I make a change to AzureRepo1, the pipeline triggers. At runtime, how would I determine which branch ("production", "staging", or "development") of the target repo (AzureRepo1) triggered the build? Ideally, the "service" parameter being fed into the example template would dynamically reflect which branch triggered the build.
Note: "Build.SourceBranch" and "Build.SourceBranchName" seem to pull the branch from the repo that hosts the YML file (in this case, AzureRepo2).
I was wrong. These function as intended. Use the below solution.
According to documentation here:
When an update to one of the repositories triggers a pipeline, then the following variables are set based on triggering repository:
Build.Repository.ID
Build.Repository.Name
Build.Repository.Provider
Build.Repository.Uri
Build.SourceBranch
Build.SourceBranchName
Build.SourceVersion
Build.SourceVersionMessage
For the triggering repository, the commit that triggered the pipeline determines the version of the code that is checked out. For other repositories, the ref defined in the YAML for that repository resource determines the default version that is checked out.
If triggers happens on AzureRepos1 you should have correct branch name in Build.SourceBranchName

How to create a CI Trigger on a different Azure Repo than where the YAML pipeline resides?

I want my YAML in one repo RepoA and my code to build in RepoB. How do I configure the YAML to have a CI Trigger on the code RepoB only?
Note: these repos are in the same Azure DevOps project.
The YAML is in the default branch (master) of RepoA. I've seen that people have had issues with CI triggers if the pipeline is not in the default branch.
Here is the azure-pipeline.yml contents:
trigger: none
resources:
repositories:
- repository: RepoB
type: git
name: RepoB
ref: master
trigger:
- master
pool:
vmImage: 'windows-latest'
steps:
- checkout: RepoB
I also tried removing the line
trigger: none
to see if that would work. The build will then start as soon as I save the yaml, as I would expect, but not when I make a change to RepoB master.
Then I tried the following and many more guesses, but nothing ever enabled a CI Trigger on RepoB. That is, the pipeline never ran when I would make commits to the master branch in RepoB.
resources:
repositories:
- repository: RepoB
type: git
name: RepoB
ref: master
trigger:
branches:
include:
- master
pool:
vmImage: 'windows-latest'
steps:
- checkout: RepoB
Here is what I see when I look at the pipelines triggers in the Azure Pipelines UI. Should I see a trigger for RepoB?
Update 1:
Although these were not my original settings, I have updated the settings to be as open as possible (no limits). I then tried the following:
I committed a file to the branch in RepoB. No CI trigger occurred.
Deleted the pipeline, and recreated. I committed a file to the branch in RepoB. CI Trigger finally occurred!
I believe this is a bug because none of these settings should limit this scenario for the following reasons:
Both of the repos are in the same Project.
RepoB is explicitly referenced in the pipeline.
Also, you shouldn't have to delete and recreate a pipeline in order for a setting to take effect.
UPDATE 2:
I narrowed it down to this Organization or Project level setting: Limit job authorization scope to referenced Azure DevOps repositories
The documentation of this setting does not mention CI Triggers at all, but I do not think that it should affect this scenario regardless, because the repo is referenced explicitly.
Doc References:
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/access-tokens?view=azure-devops&tabs=yaml#limit-job-authorization-scope-to-referenced-azure-devops-repositories
https://learn.microsoft.com/en-us/azure/devops/pipelines/repos/azure-repos-git?view=azure-devops&tabs=yaml#limit-job-authorization-scope-to-referenced-azure-devops-repositories
I believe this is a bug and I have reported it here:
https://developercommunity2.visualstudio.com/t/yaml-pipeline-ci-trigger-for-repository-resource-i/1314241
Azure DevOps enables some limitation to access to resources by default. Please check if this project enables below options in Project Settings page.
Testing in my side that if these options are enabled, this issue can be reproduced. Thus please disable them, and create a new yaml pipeline. The new yaml pipeline should work as expected.
See: Access repositories, artifacts, and other resources for details.
I used your yaml and all works (the only difference is that I have main not master branch)
trigger: none
resources:
repositories:
- repository: RepoB
type: git
name: azure-functions
ref: main
trigger:
- main
pool:
vmImage: 'windows-latest'
steps:
- checkout: RepoB
On this screen you have a trigger which fires for a change done on RepoB

azure pipeline pull dependency projects

I have a project which depends on 2-3 other projects, is there a way to pull them together with the master project?
When the build process starts projects will be on the file system and the master project can locate the other dependency projects?
As #Kehinde's said in comment, what you want could be achieved by the feature Multi-repo checkout.
Note:
Multi-repo checkout is the feature which only supported YAML. Because what the design logic is Checkouts from multiple repos in combination with YAML builds enable focusing the source level dependency management to one structured descriptor file in Git (the YAML biuld definition) for good visibility.
But for pipeline that configured by classic UI, you had to add those other repositories/projects as submodules, or as manual scripts to run git checkout in pipeline.
For personal, I strongly suggest you use YAML to achieve what you want.
Simple sample YAML definition:
resources:
repositories:
- repository: tools
name: Tools
type: git
steps:
- checkout: self
- checkout: tools
- script: dir $(Build.SourcesDirectory)
Here, image I have a repository called "MyCode" with a YAML pipeline, plus a second repository called "Tools".
In above third step(dir $(Build.SourcesDirectory)), it will show you two directories, "MyCode" and "Tools", in the sources directory.
Hope this helps.
For Bitbucket:
resources:
repositories:
- repository: MyBitBucketRepo
type: bitbucket
endpoint: MyBitBucketServiceConnection
name: {BitBucketOrg}/{BitBucketRepo}
trigger:
- master
pool:
vmImage: 'ubuntu-latest'
steps:
- checkout: self
- checkout: MyBitBucketRepo
- script: dir $(Build.SourcesDirectory)

how to restrict the access on build.yml file from developers in Azure DevOps repository

In Azure DevOps we have created both build and release pipeline using classic way and now we are planning this to convert to yaml file.
But it seems in yaml method, the code can be put only on the root of the repo, where we want to keep the build yaml files in a separate repo, where the developers won't have access.
How can achieve this?
You can use templates, put in the main repo only the minimal yaml that refers to a template with all the steps, the template exits in another repo.
For example, your main repo yaml:
resources:
repositories:
- repository: templates
type: git
name: Contoso/BuildTemplates
jobs:
- template: common.yml#templates # Template reference
In in the repo: Contoso/BuildTemplates put the full yaml:
# Repo: Contoso/BuildTemplates
# File: common.yml
parameters:
vmImage: 'ubuntu 16.04'
jobs:
- job: Build
pool:
vmImage: ${{ parameters.vmImage }}
steps:
- script: npm install
- script: npm test
Restrict the access to the second repo (unless the agent pipeline user).
Read here more info about the resources.
I agree that one solution could be the one proposed by #Shayki Abramczyk
but to have standalone *.yml in dedicated repository you can use 'git clone' while using 'Git Credentials' to access the other repository that contains the files you want to build by the pipeline.
If your repository dedicated for *.yml is in the same Azure Devops project then you should not have any problem with the release definition.
Please see example *.yml that works for us as described:
pool:
vmImage: 'your-preferred-image'
variables:
solution: '$(Agent.BuildDirectory)/**/YourSolution.sln'
buildPlatform: 'Any CPU'
buildConfiguration: 'Debug'
urlWithCreds: 'https://YourUser:YourPassword#dev.azure.com/YourOrganization/YourProject/
_git/YourOtherRepository'
steps:
- task: CmdLine#2
inputs:
script: |
git --version
git clone --quiet $(urlWithCreds)
git checkout master
- task: VSBuild#1
inputs:
solution: '$(solution)'
msbuildArgs: 'your build args'
platform: '$(buildPlatform)'
configuration: '$(buildConfiguration)'
You don't have to keep the YAML files in the root of the repository; ours are in a dedicated sub-folder:
That's crucial, because it means that we can add a PR policy which restricts who can approve changes to any of the pipeline YAML files.