My issue
I have an Azure devops project with a template on another repository than main one's.
I try to load the template but I get this:
/azure-pipelines.yml: File /TemplateRepository/build.yml not found in
repository
https://dev.azure.com/XXXXX/TestAzureDevops/_git/TemplateRepository
branch refs/heads/main
What I did
build.yml is my template into repository: TemplateRepository.
azure-pipelines.yml is my main file.
this is my template:
parameters:
- name: 'name'
default: 'name'
type: 'string'
steps:
- script: echo ${{parameters.name}}
This is my main:
pool:
vmimage: 'windows-latest'
parameters:
- name: contexts
type: object
default: [{
name: macOS,
pool: 'macOS-latest',
sign: false
},
{
name: Windows,
pool: 'windows-latest',
sign: true
}]
resources:
repositories:
- repository: Tuto-Ressources
ref: main
type: git
name: TemplateRepository
stages:
- stage: BUILD
jobs:
- job: test
steps:
- checkout: self
- checkout: Tuto-Ressources
- script: dir
displayName: Dir
- ${{ each context in parameters.contexts }}:
- template: .\TemplateRepository\build.yml#Tuto-Ressources
parameters:
name: ${{context.name}}
pool: ${{context.pool}}
sign: ${{context.sign}}
buildSteps:
- checkout: self
- checkout: Tuto-Ressources
- bash: echo "Test module"
What I tested
If I remove the template lines from main script so I just have script: dir I can see my checkout on the VM and build.yml into \TemplateRepository directory.
So there is no reason it can't find my template.
I checked the branch name, it's main and I have no other one. I can see the file from the Devops Portal
What I need
I would like to understand what happen and what I can do.
I know there are artifact, but in this situation I don't understand whay it is not working like this because I don't change job neither stage.
The template line should reference the template file by its path relative to TemplateRepository. If your build template resides at \build.yml in TemplateRepository, then you should just do:
- template: build.yml#Tuto-Ressources
Also, you don't need to do - checkout: Tuto-Ressources - the pipeline engine will know to fetch the template file from referenced repository.
Related
I have a pipeline that runs a template pipeline. It looks like this:
resources:
repositories:
- repository: repoName
type: git
name: projectName/repoName
ref: branchName
stages:
- stage: GetLastCommitId
jobs:
- job: lastCommitId
steps:
- checkout: repoName
- bash: |
cd repoName
echo "##vso[task.setvariable variable=commitId;isOutput=true]$(git rev-parse HEAD)"
name: a
- bash: |
echo $(a.commitId)
- checkout: self
- template: templates/bicep.yml
parameters:
environment: dev
lastCommitId: $[stageDependencies.GetLastCommitId.lastCommitId.outputs['a.commitId']]
bash returns me the required ID. Everything as it is meant to be.
Now I want to pass this output value to the template as parameter - lastCommitId.
Is there a way to do this?
Here's what I tried:
- template: templates/bicep.yml
parameters:
environment: dev
lastCommitId: $(a.commitId)
Error: Empty string
- template: templates/bicep.yml
dependsOn: GetLastCommitId
parameters:
environment: dev
lastCommitId: $[stageDependencies.GetLastCommitId.lastCommitId.outputs['a.commitId']]
Error: Can not start pipeline. dependsOn not expected here
- template: templates/bicep.yml
parameters:
environment: dev
dependsOn: GetLastCommitId
lastCommitId: $[stageDependencies.GetLastCommitId.lastCommitId.outputs['a.commitId']]
Error: syntax error: invalid arithmetic operator (error token is ".GetLastCommitId.lastCommitId.outputs['a.commitId']
syntax according to microsoft doc: $[stageDependencies.A.A1.outputs['MyOutputVar.myStageVal']]
Here is documentation from Microsoft: https://learn.microsoft.com/en-us/azure/devops/pipelines/process/set-variables-scripts?view=azure-devops&tabs=bash
The value of the parameter your are trying to set is done before the actually run of the tasks. Hence, the lastCommitId is not set as the value and is resulting in your different approaches into a dissapointed result.
But, depending on what is in your bicep.yml, there is a solution!
With the azure-pipeline.yml and bicep.yml below, you are able to use your lastCommitId in the template:
trigger:
- main
pool:
vmImage: ubuntu-latest
resources:
repositories:
- repository: repoName
type: git
name: projectName/repoName
ref: branchName
stages:
- stage: GetLastCommitId
jobs:
- job: lastCommitId
steps:
- checkout: repoName
- bash: |
cd repoName
echo "##vso[task.setvariable variable=commitId;isOutput=true]$(git rev-parse HEAD)"
name: a
- bash: |
echo $(a.commitId)
- checkout: self
- template: templates/bicep.yml
And the bicep.yml:
stages:
- stage: Template
jobs:
- job: JobInTemplate
variables:
lastCommitId: $[stageDependencies.GetLastCommitId.lastCommitId.outputs['a.commitId']]
steps:
- script: echo the value is $(lastCommitId)
I have main yml file for my pipeline. Is it possible to call/use another yml after condition is met in a task of main yaml?
- task: VSTest#2
conditions: eq(${{ parameters.Update }}, 'true')
## use pipeline2.yaml to continue
See https://learn.microsoft.com/en-us/azure/devops/pipelines/process/templates?view=azure-devops
Copied from the link:
# File: simple-param.yml
parameters:
- name: yesNo # name of the parameter; required
type: boolean # data type of the parameter; required
default: false
steps:
- script: echo ${{ parameters.yesNo }}
and
# File: azure-pipelines.yml
trigger:
- main
extends:
template: simple-param.yml
parameters:
yesNo: false # set to a non-boolean value to have the build fail
Also note you don't have to extend a template, you can just insert a template:
# File: templates/include-npm-steps.yml
steps:
- script: npm install
- script: yarn install
- script: npm run compile
and
# File: azure-pipelines.yml
jobs:
- job: Linux
pool:
vmImage: 'ubuntu-latest'
steps:
- template: templates/include-npm-steps.yml # Template reference
- job: Windows
pool:
vmImage: 'windows-latest'
steps:
- template: templates/include-npm-steps.yml # Template reference
I have pipeline.yml which is stored in repo - buildresources and it also have resources.json file
buildresources repo
pipeline.yml
resources.json
now I want to pass resources.json in following pipeline.yml which further extends template in other repo,how do I pass the resources.json path in buildresources repo to deploy-iac.yml in AzureIacPoC/AzureIacPoc repo
resources:
repositories:
- repository: cloud-iac # The name used to reference this repository in the checkout step
type: git
name: AzureIacPoC/AzureIacPoc
ref: refs/heads/features/azure
trigger:
branches:
include: [features/*, master]
extends:
template: deploy-iac.yml#cloud-iac
parameters:
resourceGroupName: 'anuj-test-automation'
location: 'Canada Central'
csmfile: resources.json
environment: 'sandbox'
resources:
repositories:
- repository: AnotherRepoName
type: git
name: YourProjectName/AnotherRepoName
pool:
vmImage: ubuntu-latest
steps:
- checkout: AnotherRepoName
- script: dir $(Build.SourcesDirectory)
displayName: 'Run a one-line script'
I'm trying to create a deploy pipeline YAML template for all environments/stages. I've set up the Environments on Azure DevOps so that I can add checks and approvals on the Test and Prod environments before they get deployed. I've set up a library group for each stage and each one of them has a variable called 'env' which defines the current stage running in the pipeline. For some reason, the environment property under the deployment job (see code snippet below) doesn't read that variable.
Has anyone faced this issue before, or is there a reason why the variable won't be read for that specific property?
Note: I've tested the variables and they do work, for example, the stage property outputs as 'deploy-dev/test/prod' (depending on the environment)
- stage: deploy-$(env)
jobs:
- deployment: DeployWeb
displayName: deploy Web App
pool:
vmImage: 'Ubuntu-latest'
# creates an environment if it doesn't exist
environment: 'smarthotel-$(env)'
strategy:
runOnce:
deploy:
steps:
- script: echo Hello world
You can't do this because it have to be know at compilation phase.
But you can try this (lets name this file deploy.yml):
parameters:
- name: env
type: string
default: 'dev'
stages:
- stage: deploy-${{ parameters.env }}
jobs:
- deployment: DeployWeb
displayName: deploy Web App
pool:
vmImage: 'Ubuntu-latest'
# creates an environment if it doesn't exist
environment: 'smarthotel-${{ parameters.env }}'
strategy:
runOnce:
deploy:
steps:
- script: echo Hello world
and then you need to run as follows (in build.yml file):
stages:
- template: deploy.yml
parameters:
env: dev
- template: deploy.yml
parameters:
env: qa
- template: deploy.yml
parameters:
env: prod
Question: how do you setup CI/CD in YAML pipelines for following context.
branches
master
release/{ALPHABETICAL NAME} ex. release/Albert next release is release/Bertrand and so on.
environments
accept: everything that's pushed on master
test: latest release ex. release/Bertrand
sandbox: latest release -1 (here we can test hotfixes) ex. release/Albert
live: latest release -1 (with hotfixes)
Closest solution
build: creates project artifacts
build.yml
trigger:
- master
- release/*
pool:
vmImage: 'ubuntu-latest'
steps:
- powershell: |
New-Item -Path . -Name "testfile1.txt" -ItemType "file" -Value "This is a text string."
- publish: $(Pipeline.workspace)
artifact: testArtifact
release-phase1: deploys master branch to accept
release-phase1.yml
trigger: none
resources:
pipelines:
- pipeline: pipelineId
source: build
trigger:
branches:
- master
pool:
vmImage: 'ubuntu-latest'
jobs:
- deployment: DeployWeb
environment: 'testenvironment'
strategy:
runOnce:
deploy:
steps:
- script: echo FOO
release-phase2: deploys release branch to test
release-phase2.yml
trigger: none
resources:
pipelines:
- pipeline: pipelineId
source: build
trigger:
branches:
- release/current
pool:
vmImage: 'ubuntu-latest'
jobs:
- deployment: DeployWeb
environment: 'testenvironment'
strategy:
runOnce:
deploy:
steps:
- script: echo FOO
release-phase3: deploys release-1 branch to sandbox and after manual approval to live
release-phase3.yml
trigger: none
resources:
pipelines:
- pipeline: pipelineId
source: build
trigger:
branches:
- release/previous
pool:
vmImage: 'ubuntu-latest'
jobs:
- deployment: DeployWeb
environment: 'testenvironment'
strategy:
runOnce:
deploy:
steps:
- script: echo FOO
Reasons why this solutions doesn't fulfill our needs:
the names of the release branches aren't static.
we should be able to run release-phase3.yml pipeline without running a build on this branch firts. It should download artifacts from the latest build of that branch. Which is not the case.
SHORT ON PURPOSE
Since you have multiple branches (master and releases branches), different branch is built and deploy to different environment. So you can try having the CI build yaml pipeline in each branch and put the CD deployment yaml pipeline in a template yaml in master branch.(You have to have the build yaml file in each branch to get the code in this branch built. You can check this thread).
Below is a simple example.
In master branch
There are azure-pipelines.yml and a template-deploy.yml. In azure-pipelines.yml the Environment value will be passed as a parameter to template-deploy.yml. So that the build will be deployed to its corresponding environment.
azure-pipelines.yml:
trigger:
- master
- release/*
pool:
vmImage: 'windows-latest'
resources:
repositories:
- repository: deploy
type: git
name: {project name}
jobs:
- job: Build
steps:
- script: echo "start build job"
- template: template-deploy.yml#deploy
parameters:
envir: "prod"
template-deploy.yml:
parameters:
envir: ""
jobs:
- deployment: DeployWeb
environment: '${{parameters.envir}}'
strategy:
runOnce:
deploy:
steps:
- script: echo FOO
In the release branches
You can define its individual ci build yaml like below example:
azure-pipelines.yml in release-phase2 branch:
pool:
vmImage: 'windows-latest'
resources:
repositories:
- repository: deploy
type: git
name: {project name}
jobs:
- job: Build
steps:
- script: echo "start build job"
- template: template-deploy.yml#deploy
parameters:
envir: "test"