Manually triggering Devops pipeline with pipeline resource should use latest resource pipeline run for that branch - azure-devops

I have 2 pipelines in the same repo:
Build
Deploy
The Build pipeline is declared as a pipeline resource in the Deploy pipeline:
resources:
pipelines:
- pipeline: Build
source: BuildPipelineName
trigger: true
When I run the Build pipeline, the Deploy pipeline is correctly triggered on the same branch. However, when I run the Deploy pipeline manually, it does not use the latest pipeline run from same branch.
I tried adding a couple of variations of the line below to the to the pipeline resource, but the variable does not expand:
branch: ${{ variables.Build.SourceBranchName }}
Is there any way to make this work?

Workaround that achieves the result I am looking for, but is not very elegant:
- ${{ if ne(variables['Build.Reason'], 'ResourceTrigger') }}:
- task: DeleteFiles#1
displayName: 'Remove downloaded artifacts from pipeline resource'
inputs:
SourceFolder: $(Pipeline.Workspace)
- task: DownloadPipelineArtifact#2
displayName: 'Download artifacts for branch'
inputs:
source: 'specific'
project: 'myProject'
pipeline: <BuildPipelineId>
runVersion: 'latestFromBranch'
runBranch: $(Build.SourceBranch)

For example, if I have a build pipeline named 'BuildPipelineAndDeployPipeline',
then the below YAML definition can get the latest build pipeline run from a specific branch:
resources:
pipelines:
- pipeline: BuildPipelineAndDeployPipeline
project: xxx
source: BuildPipelineAndDeployPipeline
trigger:
branches:
- main
pool:
vmImage: 'windows-latest'
steps:
- task: CmdLine#2
inputs:
script: |
echo Write your commands here
echo Hello world
echo $(resources.pipeline.BuildPipelineAndDeployPipeline.runID)

Related

Can an Azure pipeline have multiple pipeline resources referencing the same source?

I'm using the pipeline resource to trigger a second pipeline from a first pipeline. The two pipelines are in different repositories. They may even be in different projects.
This is the pipeline resource definition in the second pipeline.
resources:
pipelines:
- pipeline: xyz_build
source: company.xyz_source
trigger:
enabled: true
branches:
include:
- develop
- release/*
I want the second pipeline to take different actions, based on whether it was triggered by a build on the develop branch or a release branch. If an Azure pipeline can handle multiple pipeline resources which reference the same source, then I can rewrite my pipeline resources like this, and then choose different execution paths based on the value of $(Resources.TriggeringAlias).
resources:
pipelines:
- pipeline: xyz_develop_build
source: company.xyz_source
trigger:
enabled: true
branches:
include:
- develop
- pipeline: xyz_release_build
source: company.xyz_source
trigger:
enabled: true
branches:
include:
- release/*
Does this work in Azure Pipelines? Does Azure Pipelines support this?
EDITED TO ADD: It runs when I trigger it manually. I guess we'll find out what happens when somebody runs a build on xyz.source.
Update:
You can still capture the branch of the source resource from the different repo or even different project. But you can't set conditions at compile time to erase unnecessary tasks at the beginning of compilation(in the same repo, you can).
You can determine conditions at runtime to achieve the purpose of 'skip' tasks.
I provide an example here which have pipelines from two different projects, one for ProjectA and the other for ProjectB:
ProjectA
trigger:
- none
pool:
vmImage: ubuntu-latest
steps:
- script: echo Hello, world!
displayName: 'Run a one-line script'
ProjectB
resources:
pipelines:
- pipeline: ProjectA
project: ProjectA
source: ProjectA
trigger:
enabled: true
branches:
include:
- develop
- release
trigger:
- none
pool:
vmImage: ubuntu-latest
jobs:
- job:
displayName: Handle ProjectA develop branch
condition: eq(variables['resources.pipeline.ProjectA.sourceBranch'],'refs/heads/develop')
steps:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
# Write your PowerShell commands here.
Write-Host "Hello World"
Write-Host $(resources.pipeline.ProjectA.sourceBranch)
- job:
displayName: Handle ProjectA release branch
condition: eq(variables['resources.pipeline.ProjectA.sourceBranch'],'refs/heads/release')
steps:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
# Write your PowerShell commands here.
Write-Host "Hello World"
Write-Host $(resources.pipeline.ProjectA.sourceBranch)
Original Answer:
For example, pipeline A trigger PipelineB
PipelineA
trigger:
- none
pool:
vmImage: ubuntu-latest
steps:
- script: echo Hello, world!
displayName: 'Run a one-line script'
PipelineB
resources:
pipelines:
- pipeline: PipelineA
project: xxx
source: PipelineA
trigger:
enabled: true
branches:
include:
- develop
- release
trigger:
- none
pool:
vmImage: ubuntu-latest
steps:
- ${{ if eq(variables['Build.SourceBranch'],'refs/heads/develop') }}:
- script: echo "this is develop branch"
- ${{ if eq(variables['Build.SourceBranch'],'refs/heads/release') }}:
- script: echo "this is release branch"
Please make sure both branches have the above YML file(Pipeline will looking for the same name YML file in every branches).

Extending my Azure DevOps Pipeline to include Build validation for all Pull Requests

I have 30 or so Java Microservices that run of the same ci and cd template. i.e. Each of my Microservices has a build pipeline as follows and as shown below it runs automatically on a merge to master:
name: 0.0.$(Rev:r)
trigger:
- master
pr: none
variables:
- group: MyCompany
resources:
repositories:
- repository: templates
type: git
name: <id>/yaml-templates
stages:
- stage: Build
jobs:
- job: build
displayName: Build
steps:
- template: my-templates/ci-build-template.yaml#templates
- stage: PushToContainerRegistry
dependsOn: Build
condition: succeeded()
jobs:
- job: PushToContainerRegistry
displayName: PushToContainerRegistry
steps:
- template: my-templates/ci-template.yaml#templates
Where ci-build-template.yaml contains...
steps:
- checkout: self
path: s
- task: PowerShell#2
- task: Gradle#2
displayName: 'Gradle Build'
- task: SonarQubePrepare#4
displayName: SonarQube Analysis
- task: CopyFiles#2
displayName: Copy build/docker to Staging Directory
I would like to implement pr build validation wherever someone raises a pr to merge into master. In the PR build only the Build stage should run and from the build template only some tasks within ci-build-template.yaml should run.
A few questions for my learning:
How can i uplift the yaml pipeline above to make the "PushToContainerRegistry" skip if it is a pr build?
How can i uplift ci-build-template.yaml to make the "SonarQubePrepare#4" and "CopyFiles#2" skip if it is a pr build?
And lastly how can i uplift the yaml pipeline above to enable build validation for all pr's that have a target branch of master?
Whilst doing my own research i know you can do this via clickops but I am keep on learning on how to implement via yaml.
thanks
How can i uplift the yaml pipeline above to make the
"PushToContainerRegistry" skip if it is a pr build?
How can i uplift ci-build-template.yaml to make the
"SonarQubePrepare#4" and "CopyFiles#2" skip if it is a pr build?
Just need to use condition of task:
For example,
pool:
name: Azure Pipelines
steps:
- script: |
echo Write your commands here
echo Hello world
echo $(Build.Reason)
displayName: 'Command Line Script'
condition: and(succeeded(), ne(variables['Build.Reason'], 'PullRequest'))
Above definition will skip the step if Pull request trigger the pipeline.
Please refer to these documents:
https://learn.microsoft.com/en-us/azure/devops/pipelines/repos/azure-repos-git?view=azure-devops&tabs=yaml#using-the-trigger-type-in-conditions
https://learn.microsoft.com/en-us/azure/devops/pipelines/build/variables?view=azure-devops&tabs=yaml#build-variables-devops-services
And lastly how can i uplift the yaml pipeline above to enable build
validation for all pr's that have a target branch of master?
You can use this expression in the condition:
eq(variables['System.PullRequest.TargetBranch'], 'refs/heads/master')
If you are based on DevOps git repo, then just need to add branch policy is ok:
https://learn.microsoft.com/en-us/azure/devops/repos/git/branch-policies?view=azure-devops&tabs=browser#configure-branch-policies

How to make correct my azure deployment issue by rearranging my azure-pipelines.yml?

I am trying to establish a pipeline by using azure cloud and devops. But I got an error below while deploying from succeeded building. How can I solve this issue?
I read an article it is awesome "http://www.alessandromoura.com.br/2020/04/23/azure-devops-publish-and-download-artifacts/"
I applied your rule sets but I got error always below :
Error: No package found with specified pattern: D:\a\r1\a***.zip
Check if the package mentioned in the task is published as an artifact in the build or a previous stage and downloaded in the current job.
azure-pipelines.yml :
# Docker
# Build and push an image to Azure Container Registry
# https://learn.microsoft.com/azure/devops/pipelines/languages/docker
trigger:
- main
resources:
- repo: self
variables:
# Container registry service connection established during pipeline creation
dockerRegistryServiceConnection: 'xxxxx'
imageRepository: 'xxxhelloaspnetcore'
containerRegistry: 'xxxcontainer01.azurecr.io'
dockerfilePath: '$(Build.SourcesDirectory)/Dockerfile'
tag: '$(Build.BuildId)'
# Agent VM image name
vmImageName: 'ubuntu-latest'
stages:
- stage: Build
displayName: Build and push stage
jobs:
- job: Build
displayName: Build
pool:
vmImage: $(vmImageName)
steps:
- task: Docker#2
displayName: Build and push an image to container registry
inputs:
command: buildAndPush
repository: $(imageRepository)
dockerfile: $(dockerfilePath)
containerRegistry: $(dockerRegistryServiceConnection)
tags: |
$(tag)
- download: none
- task: DownloadPipelineArtifact#2
displayName: 'Download Build Artifacts'
inputs:
patterns: '**/*.zip'
path: '$(Build.ArtifactStagingDirectory)'
- task: PowerShell#2
displayName: 'Degug parameters'
inputs:
targetType: Inline
script: |
Write-Host "$(Build.ArtifactStagingDirectory)"
Write-Host "$(System.DefaultWorkingDirectory)"
Write-Host "$(System.ArtifactsDirectory)"
Write-Host "$(Pipeline.Workspace)"
Write-Host "$(System.ArtifactsDirectory)"
Your pipeline creates and pushes image to container registry so you don't have there pipeline artifacts. This is why DownloadPipelineArtifact throws error.
DownloadPipelineArtifact makes sens only if you use PublishPipelineArtifact before. This doc - Publish and download artifacts in Azure Pipelines describe it very well.
There is also a way to download artifacts from another pipeline - but it requires to use resource, but you don't have defined pipeline resource in your pipeline.
So all works as expected. Can you explain what actually do you want to download and what achieve by that?

Unable to resolve $(Release.ReleaseId) in Azure DevOps yml pipelines

When using the Releases tab in Azure DevOps web console, to create release definitions, the tasks can resolve $(Release.ReleaseId) inside of a bash task.
But if I instead do my deployment in the azure-pipelines.yml file and do echo $(Release.ReleaseId), I get null because the variable doesn't exist. How come?
Here is part of the yml file
- stage: Deploy
dependsOn: BuildAndPublishArtifact
condition: succeeded('BuildAndPublishArtifact')
jobs:
- deployment: DeployToAWSDev
displayName: My display name
pool:
vmImage: 'Ubuntu-16.04'
environment: 'dev'
strategy:
runOnce:
deploy:
steps:
- download: current
artifact: MyArtifact
- task: Base#3
inputs:
targetType: 'inline'
script: |
echo $(Release.ReleaseId) # Nothing
Thanks for any help to point in the right direction of how I can retrieve my release id.
Refer to the documentation on variables. There's no differentiation of "build" vs "release" in a YAML pipeline. Thus, Build.BuildId would be the run's ID.

Using ARM Templates from external repository

I'm working with azure multistage pipelines, using deployment jobs with templates in a separate repo. I'm currently starting to use ARM templates in my deployment process and want to run ARM templates that are located in a different repository as well. This is where I get a little stuck, any help/advice appreciated.
To illustrate my setup:
Repo A -> Source code that has to be build and deployed to azure
Repo B -> Azure pipeline templates (only consists of yml files)
Repo C -> ARM templates
So what I want to accomplish: A uses B uses C.
REPO A: Documentation build and release yml
resources:
repositories:
- repository: templates
type: git
name: <ACCOUNT>/Azure.Pipelines.Templates
ref: refs/tags/2.2.40
stages:
- stage: Build
jobs:
- template: src/jobs/doc-build.yml#templates
- stage: DEV
jobs:
- template: src/deployment-jobs/doc.yml#templates
....
REPO B: Documentation deployment
parameters:
webAppName: ''
connectedServiceName: 'DEV'
jobs:
- deployment: doc_deploy
pool:
name: 'DOC'
environment: 'doc'
strategy:
runOnce:
deploy:
steps:
- template: ../deployment/arm-template.yml
parameters:
connectedServiceName: ${{ parameters.connectedServiceName }}
resourceGroupName: 'documentation'
templateFile: $(Build.SourcesDirectory)/Azure.ARM.Templates/src/web-app/documentation.jsonc
paramFile: $(Build.SourcesDirectory)/Azure.ARM.Templates/src/web-app/documentation-params.json
parameters: -name ${{ parameters.webAppName }}
...
REPO C: contains arm template + param file
The issue I'm facing is that I can't seem to be able to get to the files of repo c. I tried adding another repository entry on multiple levels but it does not seem to clone the dependent repo at all.
My current workaround/solution:
Use a powershell script to manually clone repo C and directly reference the file on disk.
Related github issue: https://github.com/microsoft/azure-pipelines-yaml/issues/103
I've also stumbled upon this issue, having to load arm templates from another repo into the current build. What I did was setting up a build on the arm-template-containing repo, producing a build artifact with following azure-pipelines.yml: (this would be your repo c)
trigger:
- master
steps:
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: '$(System.DefaultWorkingDirectory)/templates'
ArtifactName: 'templates'
publishLocation: 'Container'
Afterwards I could add following step into the actual pipeline:
- task: DownloadPipelineArtifact#2
displayName: 'Get ARM Templates'
inputs:
buildType: 'specific'
project: <YOUR-PROJECT-ID>'
definition: '<ARM-BUILD-DEFINITION-ID>'
buildVersionToDownload: 'latest'
artifactName: 'scripts'
targetPath: '$(Pipeline.Workspace)/templates'
and I was able to access the files as follows:
- task: AzureResourceGroupDeployment#2
displayName: 'Create Queues $(ResourceGroup.Name) '
inputs:
azureSubscription: '<YOUR-SUBSCRIPTION>'
resourceGroupName: '$(ResourceGroup.Name)'
location: '$(ResourceGroup.Location)'
csmFile: '$(Pipeline.Workspace)/templates/servicebus.json'
For more information about the Download Pipeline Artifact task check out following link:
Download Pipeline Artifact task