I have pipeline.yml which is stored in repo - buildresources and it also have resources.json file
buildresources repo
pipeline.yml
resources.json
now I want to pass resources.json in following pipeline.yml which further extends template in other repo,how do I pass the resources.json path in buildresources repo to deploy-iac.yml in AzureIacPoC/AzureIacPoc repo
resources:
repositories:
- repository: cloud-iac # The name used to reference this repository in the checkout step
type: git
name: AzureIacPoC/AzureIacPoc
ref: refs/heads/features/azure
trigger:
branches:
include: [features/*, master]
extends:
template: deploy-iac.yml#cloud-iac
parameters:
resourceGroupName: 'anuj-test-automation'
location: 'Canada Central'
csmfile: resources.json
environment: 'sandbox'
resources:
repositories:
- repository: AnotherRepoName
type: git
name: YourProjectName/AnotherRepoName
pool:
vmImage: ubuntu-latest
steps:
- checkout: AnotherRepoName
- script: dir $(Build.SourcesDirectory)
displayName: 'Run a one-line script'
Related
I have a pipeline that runs a template pipeline. It looks like this:
resources:
repositories:
- repository: repoName
type: git
name: projectName/repoName
ref: branchName
stages:
- stage: GetLastCommitId
jobs:
- job: lastCommitId
steps:
- checkout: repoName
- bash: |
cd repoName
echo "##vso[task.setvariable variable=commitId;isOutput=true]$(git rev-parse HEAD)"
name: a
- bash: |
echo $(a.commitId)
- checkout: self
- template: templates/bicep.yml
parameters:
environment: dev
lastCommitId: $[stageDependencies.GetLastCommitId.lastCommitId.outputs['a.commitId']]
bash returns me the required ID. Everything as it is meant to be.
Now I want to pass this output value to the template as parameter - lastCommitId.
Is there a way to do this?
Here's what I tried:
- template: templates/bicep.yml
parameters:
environment: dev
lastCommitId: $(a.commitId)
Error: Empty string
- template: templates/bicep.yml
dependsOn: GetLastCommitId
parameters:
environment: dev
lastCommitId: $[stageDependencies.GetLastCommitId.lastCommitId.outputs['a.commitId']]
Error: Can not start pipeline. dependsOn not expected here
- template: templates/bicep.yml
parameters:
environment: dev
dependsOn: GetLastCommitId
lastCommitId: $[stageDependencies.GetLastCommitId.lastCommitId.outputs['a.commitId']]
Error: syntax error: invalid arithmetic operator (error token is ".GetLastCommitId.lastCommitId.outputs['a.commitId']
syntax according to microsoft doc: $[stageDependencies.A.A1.outputs['MyOutputVar.myStageVal']]
Here is documentation from Microsoft: https://learn.microsoft.com/en-us/azure/devops/pipelines/process/set-variables-scripts?view=azure-devops&tabs=bash
The value of the parameter your are trying to set is done before the actually run of the tasks. Hence, the lastCommitId is not set as the value and is resulting in your different approaches into a dissapointed result.
But, depending on what is in your bicep.yml, there is a solution!
With the azure-pipeline.yml and bicep.yml below, you are able to use your lastCommitId in the template:
trigger:
- main
pool:
vmImage: ubuntu-latest
resources:
repositories:
- repository: repoName
type: git
name: projectName/repoName
ref: branchName
stages:
- stage: GetLastCommitId
jobs:
- job: lastCommitId
steps:
- checkout: repoName
- bash: |
cd repoName
echo "##vso[task.setvariable variable=commitId;isOutput=true]$(git rev-parse HEAD)"
name: a
- bash: |
echo $(a.commitId)
- checkout: self
- template: templates/bicep.yml
And the bicep.yml:
stages:
- stage: Template
jobs:
- job: JobInTemplate
variables:
lastCommitId: $[stageDependencies.GetLastCommitId.lastCommitId.outputs['a.commitId']]
steps:
- script: echo the value is $(lastCommitId)
I have a separate pipeline in Azure Devops for sonar analysis, which scans another repo in Azure Devops, however, it is only scanning the master branch, I would like it to scan specific branches, ideally, all feature branches or just our test branch.
I've tried a few things but unable to get this working, here is part of my sonar pipeline code (removed/changed any names/sensitive info):
resources:
repositories:
- repository: platform-code
type: git
name: platform-code
#pipelines:
# - pipeline: platform-build
# source: platform-build
# trigger:
# branches:
# - feature/awesomeaddition
#trigger:
# branches:
# include:
# - master
# - features/*
stages:
- stage: Sonarqube
variables:
- template: variables.yaml
jobs:
- job: SonarqubeScan
steps:
- checkout: self
- checkout: platform-code
- task: SonarQubePrepare#4
displayName: 'SonarQube Prepare'
inputs:
SonarQube: ${{ variables.SonarQubeServiceConnectionName }}
scannerMode: 'CLI'
configMode: 'manual'
cliProjectKey: ${{ variables.cliProjectKey }}
cliProjectName: ${{ variables.cliProjectName }}
cliSources: '$(Build.SourcesDirectory)/platform-build'
extraProperties: |
sonar.branch.name='feature/awesomeaddition'
I am a newbie to Azure DevOps and want to execute code from two different repositories and perform a different operation on each repo for e.g:
Stages:
- stage: Data_Setup
jobs:
- job: Data_Setup // Want to perform this operation on repo1
timeoutInMinutes: 120
pool:
vmImage: ubuntu-20.04
continueOnError: true
steps:
- task: Gradle#2
inputs:
gradleWrapperFile: gradlew
tasks: cleanReports test aggregate
env:
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
- stage: Run_all_regression_tests // Want to perform this operation on repo2
jobs:
- job: Run_all_regression_tests
timeoutInMinutes: 100
pool:
vmImage: ubuntu-20.04
continueOnError: true
steps:
- task: Gradle#2
inputs:
gradleWrapperFile: gradlew
tasks: cleanReports createJar test aggregate
env:
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
You add multiple repositories by ,
resources:
repositories:
and when you want to use a specific repository you can checkout that in the step using "checkout" like below (Copied from Microsoft documentation)
resources:
repositories:
- repository: MyGitHubRepo # The name used to reference this repository in the checkout step
type: github
endpoint: MyGitHubServiceConnection
name: MyGitHubOrgOrUser/MyGitHubRepo
- repository: MyBitbucketRepo
type: bitbucket
endpoint: MyBitbucketServiceConnection
name: MyBitbucketOrgOrUser/MyBitbucketRepo
- repository: MyAzureReposGitRepository # In a different organization
endpoint: MyAzureReposGitServiceConnection
type: git
name: OtherProject/MyAzureReposGitRepo
trigger:
- main
pool:
vmImage: 'ubuntu-latest'
steps:
- checkout: self
- checkout: MyGitHubRepo
- checkout: MyBitbucketRepo
- checkout: MyAzureReposGitRepository
- script: dir $(Build.SourcesDirectory)
My issue
I have an Azure devops project with a template on another repository than main one's.
I try to load the template but I get this:
/azure-pipelines.yml: File /TemplateRepository/build.yml not found in
repository
https://dev.azure.com/XXXXX/TestAzureDevops/_git/TemplateRepository
branch refs/heads/main
What I did
build.yml is my template into repository: TemplateRepository.
azure-pipelines.yml is my main file.
this is my template:
parameters:
- name: 'name'
default: 'name'
type: 'string'
steps:
- script: echo ${{parameters.name}}
This is my main:
pool:
vmimage: 'windows-latest'
parameters:
- name: contexts
type: object
default: [{
name: macOS,
pool: 'macOS-latest',
sign: false
},
{
name: Windows,
pool: 'windows-latest',
sign: true
}]
resources:
repositories:
- repository: Tuto-Ressources
ref: main
type: git
name: TemplateRepository
stages:
- stage: BUILD
jobs:
- job: test
steps:
- checkout: self
- checkout: Tuto-Ressources
- script: dir
displayName: Dir
- ${{ each context in parameters.contexts }}:
- template: .\TemplateRepository\build.yml#Tuto-Ressources
parameters:
name: ${{context.name}}
pool: ${{context.pool}}
sign: ${{context.sign}}
buildSteps:
- checkout: self
- checkout: Tuto-Ressources
- bash: echo "Test module"
What I tested
If I remove the template lines from main script so I just have script: dir I can see my checkout on the VM and build.yml into \TemplateRepository directory.
So there is no reason it can't find my template.
I checked the branch name, it's main and I have no other one. I can see the file from the Devops Portal
What I need
I would like to understand what happen and what I can do.
I know there are artifact, but in this situation I don't understand whay it is not working like this because I don't change job neither stage.
The template line should reference the template file by its path relative to TemplateRepository. If your build template resides at \build.yml in TemplateRepository, then you should just do:
- template: build.yml#Tuto-Ressources
Also, you don't need to do - checkout: Tuto-Ressources - the pipeline engine will know to fetch the template file from referenced repository.
Question: how do you setup CI/CD in YAML pipelines for following context.
branches
master
release/{ALPHABETICAL NAME} ex. release/Albert next release is release/Bertrand and so on.
environments
accept: everything that's pushed on master
test: latest release ex. release/Bertrand
sandbox: latest release -1 (here we can test hotfixes) ex. release/Albert
live: latest release -1 (with hotfixes)
Closest solution
build: creates project artifacts
build.yml
trigger:
- master
- release/*
pool:
vmImage: 'ubuntu-latest'
steps:
- powershell: |
New-Item -Path . -Name "testfile1.txt" -ItemType "file" -Value "This is a text string."
- publish: $(Pipeline.workspace)
artifact: testArtifact
release-phase1: deploys master branch to accept
release-phase1.yml
trigger: none
resources:
pipelines:
- pipeline: pipelineId
source: build
trigger:
branches:
- master
pool:
vmImage: 'ubuntu-latest'
jobs:
- deployment: DeployWeb
environment: 'testenvironment'
strategy:
runOnce:
deploy:
steps:
- script: echo FOO
release-phase2: deploys release branch to test
release-phase2.yml
trigger: none
resources:
pipelines:
- pipeline: pipelineId
source: build
trigger:
branches:
- release/current
pool:
vmImage: 'ubuntu-latest'
jobs:
- deployment: DeployWeb
environment: 'testenvironment'
strategy:
runOnce:
deploy:
steps:
- script: echo FOO
release-phase3: deploys release-1 branch to sandbox and after manual approval to live
release-phase3.yml
trigger: none
resources:
pipelines:
- pipeline: pipelineId
source: build
trigger:
branches:
- release/previous
pool:
vmImage: 'ubuntu-latest'
jobs:
- deployment: DeployWeb
environment: 'testenvironment'
strategy:
runOnce:
deploy:
steps:
- script: echo FOO
Reasons why this solutions doesn't fulfill our needs:
the names of the release branches aren't static.
we should be able to run release-phase3.yml pipeline without running a build on this branch firts. It should download artifacts from the latest build of that branch. Which is not the case.
SHORT ON PURPOSE
Since you have multiple branches (master and releases branches), different branch is built and deploy to different environment. So you can try having the CI build yaml pipeline in each branch and put the CD deployment yaml pipeline in a template yaml in master branch.(You have to have the build yaml file in each branch to get the code in this branch built. You can check this thread).
Below is a simple example.
In master branch
There are azure-pipelines.yml and a template-deploy.yml. In azure-pipelines.yml the Environment value will be passed as a parameter to template-deploy.yml. So that the build will be deployed to its corresponding environment.
azure-pipelines.yml:
trigger:
- master
- release/*
pool:
vmImage: 'windows-latest'
resources:
repositories:
- repository: deploy
type: git
name: {project name}
jobs:
- job: Build
steps:
- script: echo "start build job"
- template: template-deploy.yml#deploy
parameters:
envir: "prod"
template-deploy.yml:
parameters:
envir: ""
jobs:
- deployment: DeployWeb
environment: '${{parameters.envir}}'
strategy:
runOnce:
deploy:
steps:
- script: echo FOO
In the release branches
You can define its individual ci build yaml like below example:
azure-pipelines.yml in release-phase2 branch:
pool:
vmImage: 'windows-latest'
resources:
repositories:
- repository: deploy
type: git
name: {project name}
jobs:
- job: Build
steps:
- script: echo "start build job"
- template: template-deploy.yml#deploy
parameters:
envir: "test"