We are having multiple projects in Azure devops.
We would like to access the pipelines created across multiple projects.
Is that possible to access pipeline across projects or the pipelines are project specific/restricted?
Thanks
Naresh Ede
If you use yaml pipelines and rely on multiple repositories which are stored in multi-projects, please refer to this doc: Check out multiple repositories in your pipeline to implement this requirement.
In addition, please disable “Limit job authorization scope to referenced Azure DevOps repositories” option in Project settings page by reference to this doc. Please note that there are also other limitation settings listed here: Job authorization scope.
If you use classic editor pipelines, please specify the authorization scope for a build job as “Project Collection”, see: Build job authorization scope for details. And you need to manually checkout other repositories in pipeline tasks.
For YAML Pipelines here is the code you'd need:
resources:
repositories:
- repository: repositoryReferenceName
type: git
name: ProjectName/RepositoryName
And to reference a template it would be
- template: templateName.yml#repositoryReferenceName
Related
Background -
I have multiple web service projects under multiple azure repos. All these projects have same structure as they are ASP.NET core web applications. I have written separate yaml pipelines and created separate yaml jobs for each project which has similar steps and placed them in separate azure repos.
Issue -
I have created separate yaml pipeline jobs (multistage - build and deploy) depending on each yaml. Is there any way to consolidate these yaml to one and place to shared azure repo and trigger the pipelines on the code commit to respective azure repos and deploy it to their related environments on azure web services?
Is there any way to consolidate these yaml to one and place to shared azure repo and trigger the pipelines on the code commit to respective azure repos and deploy it to their related environments on azure web services?
The answer is yes.
You could create a new repo with a new YAML file in it, or you could select one repo as main repo and set the YAML with Repository resource:
resources:
repositories:
- repository: A
type: git
name: MyProject/A
ref: main
trigger:
- main
- repository: B
type: git
name: MyProject/B
ref: release
trigger:
- main
- release
The best way to manage these is a separate repo as template. Essentially your template contains the pipeline steps that you want to run, and for each pipeline you need you have a yaml file that extends the template by defining what triggers you want for that pipeline and pass any pipeline dependant parameters you are using to the template.
Templates
I have Azure YAML pipelines that have been created long back and I am now looking into migrating my existing devops solutions form Azure to Github Actions.
How do I approach migrating / translating existing pipelines from Azure DevOps to Github Actions?
Manually translating each ADO Pipeline is not something that would be consider a feasible solution for the number of pipelines I have.
Reference Links:
Azure DevOps Pipelines: https://learn.microsoft.com/en-us/azure/devops/pipelines/yaml-schema/?view=azure-pipelines
Github Actions: https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions
Simply put: I need to convert my existing ADO pipelines to Github Actions
I am thinking of building a tool to serve this purpose of converting Azure DevOps pipelines to Github Actions but need ideas on how to approach this problem.
From Leo Liu's answer, May 2022:
I am afraid there is no such out of box extension or tool to batch Convert Azure Pipelines YAML to Github Actions YAML at this moment.
Update:
Introducing GitHub Actions Importer (Nov. 2022)
We’ve launched a public preview of GitHub Actions Importer, which helps you forecast, plan, and automate migrations from your current CI/CD tool to GitHub Actions.
Doing individual migrations is relatively easy in isolation.
However, for those that have a large and established CI/CD footprint, having tooling available to assist with migrations is key to their ability to adopt Actions at scale.
In the time that we’ve been developing and using GitHub Actions Importer in its private preview form, we’ve encountered numerous customers that have thousands of pipelines—even in excess of 15K—in their legacy environments that need to be migrated.
GitHub Actions Importer is designed to help when manual migration is not feasible, such as when users have a large number of teams that depend on hundreds or thousands of workflows.
To learn more, see Automating migrations with GitHub Actions Importer and the announcement post on the GitHub blog.
Sign up here to request access to the public preview.
So check it out: it does list Azure DevOps in its Supported CI platforms.
How to Batch Convert Azure Pipelines YAML to Github Actions YAML?
I am afraid there is no such out of box extension or tool to batch Convert Azure Pipelines YAML to Github Actions YAML at this moment.
There is a tool to make migrations between Azure Pipelines YAML and GitHub Actions YAML easier for one YAML file at a time:
Convert Azure Pipelines YAML to GitHub Actions YAML
But the tool for batch convert doesn't seem to exist yet.
I work in a platform team that builds and supports Azure Pipelines YAML templates and various custom scripting for these pipelines. People in my organisation use these YAML templates like:
resources:
repositories:
- repository: templates
type: git
name: PROJECT/cicd_stuff
ref: refs/heads/releases/v0.21
extends:
template: our_awesome_template.yml#templates
parameters:
...
In order to reliably match the version of our scripts to the version of the YAML templates, we now include an extra build stage in our pipeline templates that checks out the repo templates repo and puts all our scripts in an artifact to be used in that run. Using "release branches" allows us to safely put out and test new features and fixes to our pipelines: Teams can upgrade their pipeline on their own pace, with older version remaining supported for a while.
I'd like to start using Azure Artifacts for our script artifacts, but now I'm wondering "how can I determine which version of my scripts I should be downloading from Azure Artifacts?" The information included in the resources block would work for me, but I can't seem to access that with an expression or with a predefined variable. The only solution I can currently think of is to use the az pipelines cli. Any thoughts?
how can I determine which version of my scripts I should be downloading from Azure Artifacts?
If the feature verion is the target version, you could try the following yaml to get its value. See: Repository details for details.
resources:
repositories:
- repository: templates
type: git
name: PROJECT/cicd_stuff
ref: releases/v0.21
variables:
tools.ref: $[ resources.repositories['templates'].ref ]
pool:
vmImage: ubuntu-latest
steps:
- bash: |
echo "Tools version: $TOOLS_REF"
Also you could try Daniel's solution using tagref: refs/tags/v1.0 # optional ref to pin to by reference to this doc: Use other repositories and Checking out a specific ref.
Update>>Currently there is no available predefined variables regarding to the name that was used to include the repository resource ('templates' in this case). If we know the repository alias, the repository name can be parsed via $[ resources.repositories['templates'].name].
Another finding is that the repository resource will be added as the build artifacts, and we can get it from API:GET https://dev.azure.com/{organization}/{project}/_build/results?buildId={buildId}&view=artifacts&__rt=fps&type=consumedArtifacts(I grab this API from browser developer tool). And buildId can be got using variable Build.BuildId. See: Build variables (DevOps Services) for details. From the response, search consumedSources field to find below json segment, which will return all artifacts, you could find repository resource and all its detailed information.
Azure Devops pipelines provide a clone functionality. This is super useful functionality for scaffolding pipelines. My yaml pipelines have a lot of variables, and it's difficult and error prone to reassign all of these variables from scratch.
The Azure Devops UI updated in the last week or so, and there doesn't seem to be an option to clone yaml pipelines anymore. Classic pipelines still have the clone option.
Can anyone figure out how to clone a yaml pipeline with the new UI?
Hmm I haven't used the clone feature, but have you investigated using variable groups or variable templates to make your life easier? You can link a variable group to Azure Key Vault to facilitate secrets.
I've never used a cloning option in YAML. But I wouldn't use so many Azure DevOps variables which were created in the UI. You have multiple options here:
Store the variables in your YAML file (not for secrets)
Store the variables in a YAML template (not for secrets)
Store the variables in a variable group (use Azure Key Vault or the Secure flag for secrets)
The variable group can be linked to multiple pipelines. But there is currently no scope available. So, if you have multiple environments, you have to name it like "myvariablegroup-dev", "myvariablegroup-prod". Hopefully, this feature to attach a variable group to an environment is coming in the near feature.
1,You can disable the multi-stage pipelines feature in preview features
On the top right corner of the page click the user profile and choose Preview features
Then disable the Multi-stage pipelines. Then you will see the clone option appear again.
2,The other workaround is to create a new pipeline and choose to use the Existing Azure PipelineS Yaml file while following the wizard. (which has the same effect with cloning yaml, they use the same azure-pipelines.yml file)
Then choose the same yaml file with the original pipeline.
I have a project on Azure DevOps containing multiple forks of the same main repository. I created a build pipeline for that repository which unfortunately cannot be reused for the present forks since a pipeline can only be configured for a single repository.
This solution is not ideal because leads to multiple identical pipelines, one for each fork, and maintaining all of them can be difficult.
Is there a way to use one pipeline for multiple repositories?
you can create a template file and reference that file from each pipeline, that way you can edit a single file and every pipeline will change.
example how to reuse a step file from different repo
resources:
repositories:
- repository: DevOps
type: git
name: DevOps
trigger: none
jobs:
- template: vsts/yaml/build.yaml#DevOps
parameters:
solutionName: xxx
registryName: yyy
You can take a look at the official docs for more examples
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/templates?view=azure-devops
It's on the roadmap for 2019 Q3:
Multi-repository support for YAML pipelines
https://dev.azure.com/mseng/AzureDevOpsRoadmap/_workitems/edit/1454026
Update: this is now implemented:
https://learn.microsoft.com/en-us/azure/devops/pipelines/repos/multi-repo-checkout?view=azure-devops#triggers
"Triggers
You can trigger a pipeline when an update is pushed to the self repository or to any of the repositories declared as resources."