I am in an Organization ABC, It has three Azure DevOps projects, namely A, B, and C. Everything is under Azure Repos that means we are not using GitHub.
I am trying to perform a checkout in Project C the repos from project A and B. Every project will have its ow microservices but Shared Platform Services will automate what needs to be deployed into AKS.
Problem Statement: I get error like below when I am trying to checkout repos from projects A and B in Project C. Snippet is as below:
resources:
repositories:
- repository: microservice-a
type: git
name: 'InPlaceCommunications/microservice-a'
ref: master
- repository: microservice-b
type: git
name: 'Project B/microservice-b'
ref: master
However, I get error as below:
remote: TF401019: The Git repository with name or identifier microservice-a does not exist or you do not have permissions for the operation you are attempting.
Any guidance will be very much appreciated.
I do note that I have full permissions to clone the repo myself it is just not happening on the Pipeline.
remote: TF401019: The Git repository with name or identifier microservice-a does not exist or you do not have permissions for the operation you are attempting.
Based on the error message, you need to check the following two points:
Check if the option Limit job authorization scope to current project for non-release pipelines is disable in Project C -> Project Settings -> Settings. You need to disable this option.
Check if the Service Account: Project Collection Build Service (OrganizationName) has the Read Permission in Project A/B -> Project Settings -> Repositories -> Target repos -> Security.
Related
Has the Following Feature has been implemented for Gihub Repos yet?strong text
Multi-repo triggers
You can specify multiple repositories in one YAML file and cause a pipeline to trigger by updates to any of the repositories. This feature is useful, for instance, in the following scenarios:
You consume a tool or a library from a different repository. You want to run tests for your application whenever the tool or library is updated.
You keep your YAML file in a separate repository from the application code. You want to trigger the pipeline every time an update is pushed to the application repository.
With this update, multi-repo triggers will only work for Git repositories in Azure Repos. They don't work for GitHub or Bitbucket repository resources.
SAMPLE :
trigger:
main
resources:
repositories:
- repository: tools
type: git
name: MyProject/tools
ref: main
trigger:
branches:
include:
- main
- release
As per Microsoft official sprint 173 updates 2020, this is achiveable using resources tag inside your yaml.
Here is an example that shows how you can setup an auto trigger inside your yaml pipeline based on any change in any other repos inside the same project and even other projects inside Azure DevOps as well.
Sample:
trigger:
- main
resources:
repositories:
- repository: tools
type: git
name: MyProject/tools
ref: main
trigger:
branches:
include:
- main
- release
In the above code snippet:
main branch in the self repo containing the YAML file
main or release branches in tools repo
Here is the link for official documentation form Microsoft for further details.
Hope that solution works for you.
Repository resource triggers only work for Azure Repos Git repositories in the same organization at present. They do not work for GitHub or Bitbucket repository resources.
Refer to this official doc for details: https://learn.microsoft.com/en-us/azure/devops/pipelines/repos/multi-repo-checkout?view=azure-devops#triggers
If you would like this feature to be supported, create a feature request: https://developercommunity.visualstudio.com/AzureDevOps/suggest
I'm trying to set up a pipeline that will trigger, when a commit is made in a repository that exists in a different organization.
In my own org, I've created a git repo with a yaml pipeline file in the main branch.
With the below setup, I can checkout the code from the other organization if I run the pipeline manually. But it is not triggered when a commit is pushed to that repository.
Looking at the documentation, this should be possible?
https://learn.microsoft.com/en-us/azure/devops/pipelines/repos/multi-repo-checkout?view=azure-devops#triggers
resources:
repositories:
- repository: OtherOrgRepo # In a different organization
endpoint: OtherOrgConnection
type: git
name: proj/reponame
ref: develop
trigger:
- develop
pool:
vmImage: ubuntu-latest
steps:
- checkout: OtherOrgRepo
The token used for the service connection has full access.
Is this not supported, or am I missing a step?
I guess I just need to read the big blue box:
https://learn.microsoft.com/en-us/azure/devops/pipelines/repos/multi-repo-checkout?view=azure-devops#triggers
Repository resource triggers only work for Azure Repos Git repositories in the same organization at present. They do not work for GitHub or Bitbucket repository resources.
I did however manage to trigger a classic pipeline by using a generic Git service connection, which will poll for changes at an interval.
I got 2 projects within my Azure Organisation. I have created a pipeline from one repository which it's job is to promote code from one repo in the current project to another repo in a different project. I am using YAML and have built up the service connections
# Deploy to PREPROD
resources:
repositories:
- repository: target
type: git
name: 'Other Project/ThisRepo'
trigger:
branches:
include: [
azure-pipelines
]
pool:
name: 'My Pool'
demands:
- agent.computerName -equals MYPC
steps:
- checkout: self
path: source
- checkout: target
path: target
So on the GUI, it shows the target repo and I can browse to it. However when I run the pipeline, I get the following: -
remote: TF401019: The Git repository with name or identifier ThisRepo
does not exist or you do not have permissions for the operation you
are attempting. fatal: repository
'https://dev.azure.com/myOrg/Other%20Project/_git/ThisRepo/' not found
I can't figure out why it can't access it. I've seen in the docs about if I can access it then when the pipeline is created it should be given permission... I don't understand :/
TIA
Please check if your <your project> Build Service accounts belongs to Project Collection Service Accounts on organization level.
I found similar issue solved here
Another option would be to add access only to particular repo to Build Service account from another project:
To do that go to:
Other Project
Settings
Repos -> Repositories
select ThisRepo
select Security tab and type name of the project which wants to use this repo and add Read permission
I would like to run a yaml pipeline from one project. I have a task in my yaml to scan all the source code. Using this Yaml I would like to scan all the source code in master branch for all the project and all the repository inside the same Org.
How can I get all the repo for all the project and iterate? Can someone help me ?
test.yaml
repositories:
- repository: justAnotherName
type: github
name: myGitRepo
endpoint: myGitServiceConnection
trigger:
branches:
include:
- master
steps:
- task: CredScan#2
inputs:
toolMajorVersion: 'V2'
outputFormat: 'tsv'
scanFolder: '$(Build.SourcesDirectory)'
If you're looking to pull every repo within a project, you have one of two options (see below). However, I'd advise caution before attempting this on a Microsoft-hosted agent, they have a 60-minute timeout by default. If you're using a self-hosted agent, you need not worry. I'd still advise breaking this up to avoid creating a long-running release that also consumes a large amount of disk space with each run.
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/phases?view=azure-devops&tabs=yaml#timeouts
That being said, here are the options you have:
Option 1 (Not the best)
Manually add a repository: dependency for every project and a checkout: task for every repo within the projects.
This is heavily manual and would require maintenance every time a report is added.
Option 2
You can write a custom PowerShell/bash script that uses the Azure DevOps API and git to automatically scan all projects and repos within the org and pull them onto the machine.
Start by issuing a request to get all of the projects within the org:
https://learn.microsoft.com/en-us/rest/api/azure/devops/core/projects/list?view=azure-devops-rest-6.0
Then, iterate through every project and get all repos:
https://learn.microsoft.com/en-us/rest/api/azure/devops/git/repositories/list?view=azure-devops-rest-6.0
Finally, iterate through each repo and run git clone [repository URL] to clone it onto the build agent.
NOTE: You will want to ensure to have a lot of free disk space on the agent machine and that you clean up the build space after this operation.
I am setting up CI/CD at work and there is one step I’m not sure how to do and furthermore, if it is a right thing do.
For background, I am used to develop in C# with Visual Studio, source code in TFS and deploying with basic script that copies files on the intranet.
Now, I’m requested to setup Build and Release pipelines on Dot Net Core projects in GitHub.
I have three branches on this project: DEV, RELEASE and MASTER
I created one pipeline that triggers on DEV’s commits, creates an artefact and deploy to DEV server.
Those are the pipelines that deploy all developers work to a DEV server where they run their own tests.
Next step, when we want to deploy to staging servers, we click a button in Azure, this merge the DEV branch to the RELEASE branch but I know close to nothing in GitHub, not even sure those are the appropriate words.
When the merge is done, this will trigger a build pipeline that will create a different artefact, when this artefact is updated, deploy to staging server.
Once this release is validated on Staging and Quality, we would merge RELEASE to MASTER and do the same until PROD servers. It is all on intranet and self-hosted agents.
Is that a good way of doing things? Can it be done this way? I need a PowerShell task or is there something that exists?
If you are using Azure DevOps pipeline, the pipeline should select GitHub for repository type, then we can configure the CI trigger.
a. Configure CI trigger:
Classic steps:
1.Open project setting->Service connections->select GitHub-> create a new GitHub service connection
2.Create a new build pipeline via classic editor-> Select GitHub as the source.
3.Open pipeline->select the tab Triggers-> enable the option Enable continuous integration and configure the Branch filters
b.YAML steps:
1.Open project setting->Service connections->select GitHub-> create a new GitHub service connection
2.Create a new build pipeline and select GitHub(YAML)
c.The sample of Check out GitHub repositories in your pipeline
resources:
repositories:
- repository: MyGitHubRepo # The name used to reference this repository in the checkout step
type: github
endpoint: MyGitHubServiceConnection
name: MyGitHubOrgOrUser/MyGitHubRepo
trigger:
- {branch name}
Configure CD trigger:
Please refer to this doc to configure the release trigger.
If you are using GitHub action.
Please select the correct workflow to configure the CI/CD, Please refer this doc for more details
CI sample:
on:
push:
branches:
- ' DEV'
Update1
When Dev branch is updated, it is built and deployed to Dev server by pipeline.
Create build A and release B, configure the CI build trigger, when the branch Dev is updated, it triggers the build pipeline A, and when build pipeline A is completed, it triggers the release pipeline B.
click a button to synchronize Dev branch to Release branch.
We cannot see the button, as a workaround, we can add task powershell and call the API to create pull request and complete the pull request. We also can add task cmd and publish the code via git cmd