How to capture and retain the artifact package version for Universal artifacts in azure pipelines for cd - azure-devops

I have this azure devops ci/cd pipeline using yaml. My yaml has two stages CI and CD. My CI stage has one job called BuildandDeploy. The CD stage has one deployment job. I am using universal artifacts to publish and downloading the same. In the CD phase I am using UniversalPackages devops task to download the artifact. The task has a input variable called vstsPackageVersion which is the package version that is shown in universal artifacts. I have known of two other variables that could be used $(Build.BuildId) and $(Build.BuildNumber). As a temporary work around I am hard coding the package version for the universal artifact.
I wasn't able to download the artifact with either of the built-in variables. Since the CI and CD are in the same pipeline, is there any way to store and retrieve the package version of the artifact? Is there a identifier like latest that I could use to get the latest artifact from universal package.
# specific branch build with batching
trigger:
batch: true
branches:
include:
- master
stages:
- stage: CI
jobs:
- job: BuildAndPublish
pool:
vmImage: 'Ubuntu-16.04'
steps:
-
script: |
docker build -t $(dockerId).azurecr.io/$(imageName):$(version) .
docker login -u $(dockerId) -p $(pswd) $(dockerId).azurecr.io
docker push $(dockerId).azurecr.io/$(imageName):$(version)
- task: Bash#3
displayName: Initialize Helm Client - create local repo
inputs:
targetType: 'inline'
script: '
helm init --client-only
'
- task: HelmDeploy#0
displayName: Package helm chart
inputs:
connectionType: 'Kubernetes Service Connection'
command: 'package'
chartPath: 'my-helm-dir'
- task: UniversalPackages#0
displayName: Publish helm package to my-company-artifacts
inputs:
command: 'publish'
publishDirectory: '$(Build.ArtifactStagingDirectory)'
feedsToUsePublish: 'internal'
vstsFeedPublish: '$(my-feed-guid)'
vstsFeedPackagePublish: 'my-artifact-name'
versionOption: patch
packagePublishDescription: 'My helm package descrition'
- stage: CD
jobs:
- deployment: DeployJob
displayName: Deploy Job
pool:
vmImage: Ubuntu-16.04
environment: dev
strategy:
runOnce:
deploy:
steps:
- task: UniversalPackages#0
displayName: 'Universal download'
inputs:
command: download
vstsFeed: '$(my-feed-name)'
vstsFeedPackage: 'my-artifact-name'
vstsPackageVersion: 0.0.32
- task: ExtractFiles#1
displayName: 'Extract files '
inputs:
archiveFilePatterns: '*.tgz'
destinationFolder: 'my-folder'
cleanDestinationFolder: true

The Universal Packages task based on az artifacts universal cli that not support "latest version", but only specific version (by the way, this cli is on preview).
As workaround, you can use the Rest API to retrieve the latest version and set a new variable, then, in the download task use this variable.
For example, add a PowerShell task that get the version number and set the variable:
- powershell: |
$head = #{ Authorization = "Bearer $env:TOKEN" }
$url = "https://feeds.dev.azure.com/{organization}/_apis/packaging/Feeds/{feed-name}/packages/{package-guid}?api-version=5.0-preview.1"
$package = Invoke-RestMethod -Uri $url -Method Get -Headers $head -ContentType application/json
$latestVersion = ($package.versions.Where({ $_.isLatest -eq $True })).version
Write-Host "The latest version is $latestVersion"
Write-Host "##vso[task.setvariable variable=latestVersion]$latestVersion"
env:
TOKEN: $(system.accesstoken)
Now, in the download task use it:
vstsPackageVersion: $(latestVersion)

Related

Azure devops deploy to function app with access restrictions

I am trying to deploy a function app via an Azure DevOps pipeline, however I am receiving the following error:
##[error]Failed to deploy web package to App Service.
##[error]To debug further please check Kudu stack trace URL : $URL_REMOVED
##[error]Error: Error: Failed to deploy web package to App Service. Ip Forbidden (CODE: 403)
From some googling a suggested solution seems to be to whitelist agent IP before the deployment, and then remove it after. I have added this to my pipeline, and I can see the agent IP get added to access restrictions, however the deployment still fails.
Here is my pipeline file:
# Node.js Function App to Linux on Azure
# Build a Node.js function app and deploy it to Azure as a Linux function app.
# Add steps that analyze code, save build artifacts, deploy, and more:
# https://learn.microsoft.com/azure/devops/pipelines/languages/javascript
trigger:
- main
variables:
# Azure Resource Manager connection created during pipeline creation
azureSubscription: 'xxx'
# Function app name
functionAppName: 'xxx'
# Environment name
environmentName: 'xxx'
# Agent VM image name
vmImageName: 'ubuntu-latest'
stages:
- stage: Build
displayName: Build stage
jobs:
- job: Build
displayName: Build
pool:
vmImage: $(vmImageName)
steps:
- task: NodeTool#0
inputs:
versionSpec: '10.x'
displayName: 'Install Node.js'
- script: |
if [ -f extensions.csproj ]
then
dotnet build extensions.csproj --runtime ubuntu.16.04-x64 --output ./bin
fi
displayName: 'Build extensions'
- script: |
npm install
npm run build --if-present
npm run test --if-present
displayName: 'Prepare binaries'
- task: ArchiveFiles#2
displayName: 'Archive files'
inputs:
rootFolderOrFile: '$(System.DefaultWorkingDirectory)'
includeRootFolder: false
archiveType: zip
archiveFile: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
replaceExistingArchive: true
- upload: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
artifact: drop
- stage: Deploy
displayName: Deploy stage
dependsOn: Build
condition: succeeded()
jobs:
- deployment: Deploy
displayName: Deploy
environment: $(environmentName)
pool:
vmImage: $(vmImageName)
strategy:
runOnce:
deploy:
steps:
- task: AzureCLI#2
inputs:
azureSubscription: '$(azureSubscription)'
scriptType: 'bash'
scriptLocation: 'inlineScript'
inlineScript: |
agentIP=$(curl -s https://api.ipify.org/)
az functionapp config access-restriction add -g xxx -n xxx --action Allow --ip-address $agentIP --priority 200
- task: AzureFunctionApp#1
displayName: 'Azure Functions App Deploy: xxx'
inputs:
azureSubscription: '$(azureSubscription)'
appType: functionAppLinux
appName: $(functionAppName)
package: '$(Pipeline.Workspace)/drop/$(Build.BuildId).zip'
Is anyone able to advise where I am going wrong?
I've had a simmilar issue while adding the agent IP to the network restrictions of an storage account (using Powershell but you'll understand the idea), we added a 60s sleep to be sure that the setting are taken into account by Azure.
$sa_name = "sapricer$env_prefix"
if ($null -ne (Get-AzStorageAccount -ResourceGroupName $sa_rg -AccountName $sa_name -ErrorAction Ignore)) {
Write-Output "Storage account '$sa_name' exists"
if ($enable) {
Write-Output "Add ip rule for $current_ip on $sa_name..."
Add-AzStorageAccountNetworkRule -ResourceGroupName $sa_rg -AccountName $sa_name -IPAddressOrRange $current_ip
}
else {
Write-Output "Remove ip rule for $current_ip on $sa_name..."
Remove-AzStorageAccountNetworkRule -ResourceGroupName $sa_rg -AccountName $sa_name -IPAddressOrRange $current_ip
}
}
Start-Sleep -Seconds 60
I found the solution to this.
Function Apps have two IP Restriction sections, one for the App and one for the SCM site. The SCM site is the one that requires the IP to be whitelisted in order for the deployment to work:
az functionapp config access-restriction add --scm-site true -g xxx -n xxx --action Allow --ip-address $agentIP --priority 200
You can deploy Azure function app to azure devops pipeline using azure function app task from Azure devops pipeline tasks
Here is the sample snippet for deploying azure function app
variables:
azureSubscription: Contoso
# To ignore SSL error, uncomment the below variable
# VSTS_ARM_REST_IGNORE_SSL_ERRORS: true
steps:
- task: AzureFunctionApp#1
displayName: Azure Function App Deploy
inputs:
azureSubscription: $(azureSubscription)
appName: samplefunctionapp
package: $(System.DefaultWorkingDirectory)/**/*.zip
Here is the Microsoft Document for deploying azure function app.

How to make correct my azure deployment issue by rearranging my azure-pipelines.yml?

I am trying to establish a pipeline by using azure cloud and devops. But I got an error below while deploying from succeeded building. How can I solve this issue?
I read an article it is awesome "http://www.alessandromoura.com.br/2020/04/23/azure-devops-publish-and-download-artifacts/"
I applied your rule sets but I got error always below :
Error: No package found with specified pattern: D:\a\r1\a***.zip
Check if the package mentioned in the task is published as an artifact in the build or a previous stage and downloaded in the current job.
azure-pipelines.yml :
# Docker
# Build and push an image to Azure Container Registry
# https://learn.microsoft.com/azure/devops/pipelines/languages/docker
trigger:
- main
resources:
- repo: self
variables:
# Container registry service connection established during pipeline creation
dockerRegistryServiceConnection: 'xxxxx'
imageRepository: 'xxxhelloaspnetcore'
containerRegistry: 'xxxcontainer01.azurecr.io'
dockerfilePath: '$(Build.SourcesDirectory)/Dockerfile'
tag: '$(Build.BuildId)'
# Agent VM image name
vmImageName: 'ubuntu-latest'
stages:
- stage: Build
displayName: Build and push stage
jobs:
- job: Build
displayName: Build
pool:
vmImage: $(vmImageName)
steps:
- task: Docker#2
displayName: Build and push an image to container registry
inputs:
command: buildAndPush
repository: $(imageRepository)
dockerfile: $(dockerfilePath)
containerRegistry: $(dockerRegistryServiceConnection)
tags: |
$(tag)
- download: none
- task: DownloadPipelineArtifact#2
displayName: 'Download Build Artifacts'
inputs:
patterns: '**/*.zip'
path: '$(Build.ArtifactStagingDirectory)'
- task: PowerShell#2
displayName: 'Degug parameters'
inputs:
targetType: Inline
script: |
Write-Host "$(Build.ArtifactStagingDirectory)"
Write-Host "$(System.DefaultWorkingDirectory)"
Write-Host "$(System.ArtifactsDirectory)"
Write-Host "$(Pipeline.Workspace)"
Write-Host "$(System.ArtifactsDirectory)"
Your pipeline creates and pushes image to container registry so you don't have there pipeline artifacts. This is why DownloadPipelineArtifact throws error.
DownloadPipelineArtifact makes sens only if you use PublishPipelineArtifact before. This doc - Publish and download artifacts in Azure Pipelines describe it very well.
There is also a way to download artifacts from another pipeline - but it requires to use resource, but you don't have defined pipeline resource in your pipeline.
So all works as expected. Can you explain what actually do you want to download and what achieve by that?

Packages failed to publish due to TypeError: Cannot read property 'toLowerCase' of null

In my azure devops pipeline. A conda package is build firstly and published via UniversalPackages#0 task.
The build and upload code snippet is shown below
- stage: build_upload_package
displayName: Build and upload package
pool:
name: Private Agent Eve
jobs:
- job: build_upload_package
displayName: Build and Upload Package
condition: and(succeeded(), eq(variables['build.sourceBranch'], 'refs/heads/release'))
container:
image: <azureacr_repo.io>/conda:latest //modified to remove the information
endpoint: NameOfTheServiceConnection //modified to remove the information
steps:
- bash: |
echo "##vso[task.prependpath]$CONDA/bin"
conda build . --output-folder $(Build.ArtifactStagingDirectory)
displayName: Build Package
- bash: echo "##vso[task.setvariable variable=packageName]$(basename $(gitRepoName))"
displayName: Set Package Name
- task: UniversalPackages#0 //Error appears here
displayName: Upload to Azure Universal Packages
inputs:
command: publish
publishDirectory: '$(Build.ArtifactStagingDirectory)'
vstsFeedPublish: 'project_1/feed_1' //modified to remove the information
vstsFeedPackagePublish: '$(packageName)'
versionOption: patch
packagePublishDescription: 'upload a Conda Package'
The error for Universal UniversalPackages task is show below
2020-11-24T17:54:48.9628862Z ##[section]Starting: Upload to Azure Universal Packages
2020-11-24T17:54:48.9645737Z ==============================================================================
2020-11-24T17:54:48.9646470Z Task : Universal packages
2020-11-24T17:54:48.9647068Z Description : Download or publish Universal Packages
2020-11-24T17:54:48.9647636Z Version : 0.175.0
2020-11-24T17:54:48.9648154Z Author : Microsoft Corporation
2020-11-24T17:54:48.9648809Z Help : https://learn.microsoft.com/azure/devops/pipelines/tasks
2020-11-24T17:54:48.9649526Z ==============================================================================
2020-11-24T17:54:49.4896147Z SYSTEMVSSCONNECTION exists true
2020-11-24T17:54:50.1267511Z Downloading: https://08wvsblobprodsu6weus73.vsblob.vsassets.io/artifacttool/artifacttool-linux-x64-Release_0.2.172.zip?sv=2019-02-02&sr=b&sig=FvnCFjIg7ZBmIapFcCcWDC1%2F7pajvzvgo0fR2Y3YYeU%3D&spr=https&se=2020-11-24T18%3A54%3A51Z&sp=r&P1=1606243791&P2=11&P3=2&P4=1nPT99OOlNXc9xNzoCS6LpVATo%2bW4fIskTJ8u6olWMk%3d
2020-11-24T17:54:56.4979070Z Caching tool: ArtifactTool 0.2.172 x64
2020-11-24T17:54:56.7183832Z SYSTEMVSSCONNECTION exists true
2020-11-24T17:54:58.7313555Z ##[error]TypeError: Cannot read property 'toLowerCase' of null
2020-11-24T17:54:58.7323845Z ##[error]Packages failed to publish
2020-11-24T17:54:58.7641267Z ##[section]Finishing: Upload to Azure Universal Packages
Question:
How can I solve this issue?
I think I found the error, gitRepoName is a template parameter, it should be accessed as ${{ gitRepoName }} not the $(gitRepoName)
I did some tests on this task:UniversalPackages#0. I noticed the following reminder in the task:
Universal package names must be one or more lowercase alphanumeric segments separated by a dash, dot or underscore. The package name must be under 256 characters.
When your packageName contains capital letters or other special characters, this task will fail.
You could try the following scripts:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
$name = $(basename $(gitRepoName)).ToLower()
Write-Host "##vso[task.setvariable variable=packageName]$name"
- task: UniversalPackages#0
inputs:
command: 'publish'
publishDirectory: '$(Build.ArtifactStagingDirectory)'
vstsFeedPublish: 'project_1/feed_1'
vstsFeedPackagePublish: '$(packageName)'
versionOption: 'patch'
The Powershell task could Lowercase the RepoName.

How to deploy multiple apps (monorepo) with Azure and NX

I'm using NX tools to manage a monorepo with multiple apps and I'm struggling to understand how to deploy using Azure and release pipelines.
Disclaimer : I'm very new to Azure and devops in general.
My understanding is this :
I create a pipeline (not a release pipeline, just a "regular one" if that make any sense) and plug a yml to it. Also, the pipeline is linked to a repo on Azure Repos, which means that every time I push to this repo, it will trigger the pipeline and run the yaml commands.
On this commands, I run lint, test and builds.
This is what I can do and can understand, the following become more obscure :
The build job is supposed to create an artifact if I'm pusing/merging on master which I can conditioned.
Now I can create a release pipeline which will be trigger when to repo it is linked to will create an artifact. This release pipeline can then send this artifact to an app service which is a slot where an app will live.
Okay, but I'm using a monorepo, which means the build will produce multiple applications and each one of these apps should be deploy to the correct app service.
After some research, I found that the general idea is to create one release pipeline for each app. These release pipelines are all linked to the same monorepo, but they have a filter which is a build tag. The build tag is added while building the apps using the yml file.
So these are basically my understanding of all of this. Now here are the questions :
What exactly is a build tag and where does it live? Is it somehow linked to an artifact?
The idea is to create one build tag per artifact, right?
I failed to create a build tag, how can I do so?
What is the correct way to zip and publish the artifacts?
Here is the yaml I'm using :
jobs:
- job: Lint
steps:
- task: NodeTool#0
inputs:
versionSpec: '12.x'
displayName: 'Install Node.js'
- task: Npm#1
displayName: 'Npm install'
- pwsh: 'npm run nx affected -- --target=lint --parallel --base=origin/master --maxParallel=4'
displayName: 'Running lint'
- job: Test
steps:
- task: NodeTool#0
inputs:
versionSpec: '12.x'
displayName: 'Install Node.js'
- task: Npm#1
displayName: 'npm install'
- pwsh: 'npm run nx affected -- --target=test --parallel --code-coverage --base=origin/master --maxParallel=4'
displayName: 'Running tests'
- job: Build
steps:
- task: NodeTool#0
inputs:
versionSpec: '12.x'
displayName: 'Install Node.js'
- task: Npm#1
displayName: 'npm install'
- pwsh: 'npm run nx affected -- --target=build --parallel --base=origin/master --prod'
displayName: 'Running build'
- pwsh: |
npm run nx affected:apps -- --base=HEAD~1 --head=HEAD | grep -E '( - )(\w|-|\d|_)+' | sed -E 's/ - /##vso[build.addbuildtag]/g'
displayName: 'Adding build tags'
When running this, test, lint and builds are working, but I don't think it adds a build tag, here is the log :
It appears like nothing happens... How can I correctly add the tag and make the release pipeline to be triggered?
I've also found this snippet to zip and publish the artifact, but I don't know if I can use that since in a monorepo we should - I think - create multiple artifacts.
5) So the last question is : how can I create multiple artifacts and is it even the good thing to do?
Many many thanks in advance for the help, I know this is a long boring post and helping a noob can be boring but I'm stuck with this for way to long...
1, You can add tags for a build from the UI page(Build summary page, see below) or using Rest api. It can be used to filter between builds. If your build pipeline generated multiple artifacts, build tags will not be able to filter the artifacts which generated in one build.
2, So you can think about how to separate the build artifacts for each app in your pipeline.
You can use Archive files task to zip your build artifacts and publish using publish build artifacts task.
See below yaml example: I used two archive files task to package the build artifacts for app1 and app2 separately. And save the zipped artifacts in folder $(Build.ArtifactStagingDirectory).
Then the publish build artifacts task will publish the artifacts to azure devops cloud. (Release pipeline will download the artifacts to deploy to your app server)
- task: ArchiveFiles#2
inputs:
rootFolderOrFile: $(system.defaultworkingdirectory)/app1/dist
archiveType: 'zip'
archiveFile: '$(Build.ArtifactStagingDirectory)/app1/dist1.zip'
includeRootFolder: false
enabled: true
- task: ArchiveFiles#2
inputs:
rootFolderOrFile: $(system.defaultworkingdirectory)/app2/dist
archiveType: 'zip'
archiveFile: '$(Build.ArtifactStagingDirectory)/app2/dist2.zip'
includeRootFolder: false
enabled: true
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: $(Build.ArtifactStagingDirectory)/
artifactName: build
3, Then you can use multiple stages in your release pipeline and use Azure App Service Deploy task. For below example:
In stage one add Azure App Service Deploy task and set the package to $(System.DefaultWorkingDirectory)/**/app1/dist1.zip to deploy app1. And in stage two set it to $(System.DefaultWorkingDirectory)/**/app2/dist2.zip to deploy app2.
Another workaround to deploy multiple apps in monorepo is to create multiple build/release pipeline, one for each app. And in the build pipeline using path filter to let the build pipeline only be triggered when it corresponding app is updated.
trigger:
paths:
include:
- root/app1/*
Hope above helps!
#levi Lu-MSFT answered helped me a ton and is basically how we are doing it, but they didn't really give full yaml code.
The way we have the release pipeline set up is one release pipeline per application, looking at the build tags for the release trigger. We are currently bundling all apps together inside one artifact, so the downside is when an app's main branch is triggered, the release process will download all the apps.
The way I added my build tags dynamically started with this nx command:
npx nx affected:apps --base=origin/main --plain
which prints out a list of apps that have changed. From there, I loop through the apps and create the build tags dynamically.
Here I have two jobs, one that runs on PR's and one that runs on main branch.
trigger:
branches:
include:
- main
pool:
name: My-Pool
demands: Agent.OS -equals Windows_NT
jobs:
- job: NX_AFFECTED_PR
pool:
name: WBOmega-Pool
demands: Agent.OS -equals Windows_NT
condition: eq(variables['Build.Reason'], 'PullRequest')
steps:
- task: Npm#1
displayName: 'npm install'
inputs:
command: 'install'
verbose: true
- powershell: |
npx nx affected:apps --base=origin/main --plain | Tee-Object -Variable output
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls -bor [Net.SecurityProtocolType]::Tls11 -bor [Net.SecurityProtocolType]::Tls12
foreach ($appName in ($output -replace '\s+', ' ').Split()) {
Write-Host "this is the app-name: $appName"
if($appName.Trim() -eq ""){
Write-Host "App name is blank"
} else {
$url="https://dev.azure.com/YOUR_ORG/$(System.TeamProject)/_apis/build/builds/$(build.buildid)/tags/" + $appName + "?api-version=6.0"
Write-Host $url
$result = Invoke-RestMethod -Uri $url -Headers #{Authorization = "Bearer $(System.AccessToken)"} -Method Put
}
}
name: set_build_tags
- powershell: |
npx nx affected --target=build --base=origin/main --parallel --max-parallel=3
name: build
- powershell: |
npx nx affected --target=lint --base=origin/main --parallel --max-parallel=3
name: lint
- powershell: |
npx nx affected --target=test --base=origin/main --parallel --max-parallel=3
name: test
- job: NX_AFFECTED_MAIN
pool:
name: My-Pool
demands: Agent.OS -equals Windows_NT
condition: ne(variables['Build.Reason'], 'PullRequest')
steps:
- checkout: self
persistCredentials: true
- powershell: |
npx nx affected:apps --base=HEAD~1 --plain | Tee-Object -Variable output
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls -bor [Net.SecurityProtocolType]::Tls11 -bor [Net.SecurityProtocolType]::Tls12
foreach ($appName in ($output -replace '\s+', ' ').Split()) {
Write-Host "this is the app-name: $appName"
if($appName.Trim() -eq ""){
Write-Host "App name is blank"
} else {
$url="https://dev.azure.com/YOUR_ORG/$(System.TeamProject)/_apis/build/builds/$(build.buildid)/tags/" + $appName + "?api-version=6.0"
Write-Host $url
$result = Invoke-RestMethod -Uri $url -Headers #{Authorization = "Bearer $(System.AccessToken)"} -Method Put
}
}
name: set_build_tags
- powershell: |
npx nx affected --target=build --base=HEAD~1 --parallel --max-parallel=3
name: build
- powershell: |
npx nx affected --target=lint --base=HEAD~1 --parallel --max-parallel=3
name: lint
- powershell: |
npx nx affected --target=test --base=HEAD~1 --parallel --max-parallel=3
name: test
- task: CopyFiles#2
displayName: 'Copy Files to: $(build.artifactstagingdirectory)\apps\'
inputs:
SourceFolder: '$(Build.SourcesDirectory)\dist\apps\'
TargetFolder: '$(build.artifactstagingdirectory)\apps\'
CleanTargetFolder: true
OverWrite: true
- task: PublishBuildArtifacts#1
displayName: 'Publish Artifact'
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)\apps'
ArtifactName: 'Apps'

Download same pipeline artifact latestFromBranch over multiple jobs

I currently write a Azure pipeline YAML which downloads the latest development artifacts and executes different tests in different environments.
Since every kind of test is modeled as its own Job, the first step of the job is downloading the artifact (documentation).
- task: DownloadPipelineArtifact#2
inputs:
buildType: 'specific'
project: '[hidden]'
definition: '[hidden]'
buildVersionToDownload: 'latestFromBranch'
branchName: 'refs/heads/development'
How can I achieve that every job takes the same artifact, also if a newer is created during the pipeline run?
My first approach was downloading the latest artifact and extracting the concrete version. But I wasn't able to get the build id which is required as pipelineId for downloading a specific version.
But I wasn't able to get the build id which is required as pipelineId for downloading a specific version.
We could use the REST API to get the pipeline build id before we use the task DownloadPipelineArtifact. Add the parameter definitions to get the build id for specify definitions:
https://dev.azure.com/{organization}/{project}/_apis/build/builds?definitions={definitions}&api-version=5.1
Then I got all the build IDs, use Select-Object -first 1 get the latest build ID and set it as environment variable, so my powershell task should be like:
$url = "https://dev.azure.com/{organization}/{project}/_apis/build/builds?definitions={definitions}&api-version=5.1"
$buildPipeline= Invoke-RestMethod -Uri $url -Headers #{
Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"
} -Method Get
$LatestBuildID= $buildPipeline.value.id | Select-Object -first 1
Write-Host This is Latest Build ID: $LatestBuildID
Write-Host "##vso[task.setvariable variable=LatestBuildID;]$LatestBuildID"
Next, we set the variable $(LatestBuildID) in the Build option:
Now, we could get the latest build id which is required as pipelineId for downloading a specific version.
How can I achieve that every job takes the same artifact, also if a
newer is created during the pipeline run?
For this issue, we need set the Build completion for build or Continuous deployment trigger for release, so that there is a new artifact created, the pipeline of DownloadPipelineArtifact should be executed.
Hope this helps.
I implemented a solution as suggested by #LeoLiu-MSFT.
Find the last successful master build and store the buildId
Provide a download task with the buildId as parameter
Use the download task in all test jobs
The following snippets show a concrete example.
# 1. steps/find-latest-build.yml
steps:
- task: Bash#3
name: LatestArtifactsBuild
displayName: 'Find latest successful master build'
env:
system_accesstoken: $(System.AccessToken)
inputs:
targetType: 'inline'
script: |
definitionId=<pipeline>
url="https://dev.azure.com/<organization>/<project>/_apis/build/builds?api-version=5.1"
url+="&definitions=$definitionId"
url+="&resultFilter=succeeded"
url+="&branchName=refs/heads/master"
url+="&\$top=1"
json=$(curl -sL -H "Authorization: Bearer $system_accesstoken" -H 'Content-Type: application/json' $url)
buildId="$(echo $json | jq '.value[0].id' )"
echo "##vso[task.setvariable variable=id;isOutput=true]$buildId"
# 2. steps/download-artifacts-from-pipeline-build.yml
parameters:
definitionId: <pipeline>
buildId: $(Build.BuildId)
artifactName: ''
steps:
- task: DownloadPipelineArtifact#2
displayName: 'Download build artifact archives'
inputs:
buildType: 'specific'
project: $(System.TeamProjectId)
definition: ${{ parameters.definitionId }}
buildVersionToDownload: 'specific'
pipelineId: ${{ parameters.buildId }}
artifactName: ${{ parameters.artifactName }}
# 3. examples/reuse-build-artifacts.yml
trigger: none
jobs:
- job: PrepareBuild
displayName: 'Provide build information'
steps:
- checkout: none
- template: ../steps/find-latest-build.yml
- job: DownloadAllArchivesOfSpecificBuild
dependsOn: PrepareBuild
variables:
download_build_id: $[ dependencies.PrepareBuild.outputs['LatestArtifactsBuild.id'] ]
steps:
- checkout: none
- template: ../steps/download-artifacts-from-pipeline-build.yml
parameters:
buildId: $(download_build_id)