Creating a yaml build template with access to all repositories - azure-devops

Question

I have a build that uses the Azure DevOps REST api to do analysis across a collection of repositories in a single Azure DevOps project.
To speed up the build, it only checks out a single repository containing certain build utilities, such as powershell scripts - the rest of the analysis is done via querying specific information via the REST api.
NOTE: This build is running on Azure DevOps Server 2020, which still calls the setting "Limit job authorization scope to referenced Azure DevOps repositories". Looking at doc history, I believe this is equivalent to "Protect access to repositories in YAML pipeline", I use the more recent term below.
This all worked fine until "Protect access to repositories in YAML pipelines" was turned on. With that setting turned on, the REST api only returns information about the repository containing the build utilities. This is due to the reduced scope of the Job Access Token (see Protect access to repositories in YAML pipelines)
I've attempted to create a template containing the list of all repositories, so that a few select builds can continue to easily access all repositories. Previously, no explicit list of repositories was needed, but now it appears they will have to manually be listed, and I'd like to do that in a single file.
Both yaml files below are in the same repository.
Template allRepos.yaml:
parameters:
- name: steps
type: stepList
default: []
jobs:
- job:
pool: 'swimming'
uses:
repositories:
- R1
- R2
- R3
- Rnnn
steps:
- ${{ parameters.steps }}
Yaml for pipeline:
extends:
template: allRepos.yaml
parameters:
steps:
- checkout: self
- task: PowerShell#2
displayName: multiRepoAnalysis
inputs:
filePath: analysis.ps1
env:
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
It seems like this should work according to:
access tokens - Protect access to repositories in YAML pipelines)
repos - Protect access to repositories in YAML pipelines
jobs.job definition
However, attempting to run this pipeline results in the errors:
Job 'Job1' references the repository 'R1' which is not defined by the pipeline.
Job 'Job1' references the repository 'R2' which is not defined by the pipeline.
Job 'Job1' references the repository 'R3' which is not defined by the pipeline.
Job 'Job1' references the repository 'Rnnn' which is not defined by the pipeline.
How can I create a template that:
allows the access token for specific builds to access all repositories when "Protect access to repositories in YAML pipelines" is turned on
do so without checking out each repository

Answer 1

Limitations
This solution only works for a list of at most 20 repositories
- checkout: self counts toward the 20 repository limit
You must include submodule repositories in the list, which also counts toward the 20 repository limit.
Exceeding 20 repositories will still result in an error:
##[error]Tokens may be scoped to a maximum of 20 repositories. Reference fewer repositories in YAML or disable Scoped Tokens.
This limitation could be worked around by splitting the build into multiple definitions - but note if your analysis includes accessing submodule repositories, those submodule repositories must be referenced in any list of the primary repositories.
To my current knowledge, there is not a way to scope the access token for a particular pipeline with a generic "all repositories" permission.
Solution
The original templates were on the right track - but missed the fact that any uses: repositories must previously have been declared as a resources: repository.
There is a github issue to clarify the Microsoft documentation: uses: keyword is not explained, does not work as described
The working solution is:
Template allRepos.yaml:
parameters:
- name: steps
type: stepList
default: []
resources:
repositories:
- repository: R1
type: git
name: MyDevOpsProject/R1
- repository: R2
type: git
name: MyDevOpsProject/R2
- repository: R3
type: git
name: MyDevOpsProject/R3
- repository: Rnnn
type: git
name: MyDevOpsProject/Rnnn
jobs:
- job:
pool: 'swimming'
uses:
repositories:
- R1
- R2
- R3
- Rnnn
steps:
- ${{ parameters.steps }}
Yaml for pipeline:
extends:
template: allRepos.yaml
parameters:
steps:
- checkout: self
- task: PowerShell#2
displayName: multiRepoAnalysis
inputs:
filePath: analysis.ps1
env:
SYSTEM_ACCESSTOKEN: $(System.AccessToken)