I have CI and CD pipelines using Azure DevOps for a frontend angular project. Both are separate pipelines.
Here goes the YAML file for the CI pipeline which produces published artifact: output_final.zip. The below pipeline leverages Azure Pipelines for generating the published artifact.
# Node.js with Angular
# Build a Node.js project that uses Angular.
# Add steps that analyze code, save build artifacts, deploy, and more:
# https://learn.microsoft.com/azure/devops/pipelines/languages/javascript
trigger:
- integration
pool:
vmImage: ubuntu-latest
steps:
- task: NodeTool#0
inputs:
versionSpec: '14.x'
displayName: 'Install Node.js'
- powershell: |
$buildNumber="$(Build.BuildNumber)"
echo $buildNumber > src/version.txt
- script: |
npm install -g #angular/cli
npm install
ng build --prod
displayName: 'npm install and build'
- task: CopyFiles#2
displayName: 'Copy Files of UI'
inputs:
SourceFolder: 'dist/source'
TargetFolder: '$(Build.ArtifactStagingDirectory)/output'
OverWrite: true
- task: ArchiveFiles#2
displayName: 'Archive'
inputs:
rootFolderOrFile: '$(Build.ArtifactStagingDirectory)/output/'
includeRootFolder: false
archiveFile: '$(Build.ArtifactStagingDirectory)/output/output_final.zip'
- task: PublishBuildArtifacts#1
displayName: 'Publish Artifact'
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)/output/output_final.zip'
ArtifactName: drop
Now I have a separate CD pipeline which leverages self hosted private agent. In this CD pipeline, I want to consume artifacts published by the CI pipeline in the CD pipeline
Can anyone help me to know how to consume artifacts published by the CI pipeline and use it in the CD pipeline with some sample YAML example.
My suggestion would be to use the Publish Pipeline Artifact task to publish the CI artifact. Then, in your CD pipeline, use the Download Pipeline Artifact to consume it.
You can configure the download task to get the artifact from the current run or from a specific run. In your case, it sounds like you want a specific run since the CI pipeline is separate.
Here's an example of what the yaml tasks might look like:
#CI Task
- task: PublishPipelineArtifact#1
displayName: 'Publish pipeline Artifact'
inputs:
targetPath: '$(Pipeline.Workspace)'
artifact: '<Some Artifact Name>'
publishLocation: 'pipeline'
#CD Task example
- task: DownloadPipelineArtifact#2
inputs:
buildType: 'specific'
project: ''
definition: ''
specificBuildWithTriggering: true
buildVersionToDownload: 'latest'
targetPath: '$(Pipeline.Workspace)'
Related
I have an ASP.NET Core project in Azure DevOps repository and it gets built OK using the DevOps build pipeline. However, the release over that builds always fails with this error:
No package found with specified pattern.Check if the package mentioned in the task is published as an artifact in the build or a previous stage and downloaded in the current job.
More Details:
# ASP.NET Core
# Build and test ASP.NET Core projects targeting .NET Core.
# Add steps that run tests, create a NuGet package, deploy, and more:
# https://learn.microsoft.com/azure/devops/pipelines/languages/dotnet-core
trigger:
branches:
include:
- release/*
pool:
vmImage: 'windows-2022'
variables:
solution: '**/*.sln'
buildPlatform: 'Any CPU'
buildConfiguration: 'Release'
projectName: '**/F11.Web.csproj'
runtime: 'win-x64'
steps:
- task: UseDotNet#2
displayName: 'Use .NET 5 SDK (preview)'
inputs:
packageType: 'sdk'
version: '5.0.100-rc.1.20452.10'
vsVersion: '16.8.0'
includePreviewVersions: true
- task: DotNetCoreCLI#2
inputs:
command: 'restore'
projects: '$(projectName)'
feedsToUse: 'select'
- task: DotNetCoreCLI#2
displayName: Build
inputs:
projects: '$(projectName)'
arguments: '--no-restore'
- task: DotNetCoreCLI#2
displayName: Test
inputs:
command: test
projects: '$(projectName)'
arguments: '-l "console;verbosity=detailed"'
- task: DotNetCoreCLI#2
displayName: 'Publish WebApi'
inputs:
command: publish
publishWebProjects: false
projects: '$(projectName)'
arguments: '--configuration $(BuildConfiguration) --output $(Build.ArtifactStagingDirectory) --runtime -r $(runtime)'
- task: CopyFiles#2
inputs:
Contents: '**'
TargetFolder: '$(Build.ArtifactStagingDirectory)'
- task: PublishPipelineArtifact#1
inputs:
targetPath: '$(Pipeline.Workspace)'
artifactName: 'PublishBuildArtifact'
Azure DevOps pipeline release Error: No package found with specified pattern:
First, you could make sure you have generated the artifact for your build pipeline, check it from the build history in the build pipeline.
Second, when you using the release pipeline to deploy the artifact, you need make sure you have select the correct build pipeline.
Third, set the correct configuration for the task IIS web app deploy:
If above not help you, please share the configuration for the task IIS web app deploy with image in your question.
I have a build pipeline that publishes a release artifact for consumption by the release pipeline.
pool:
name: Azure Pipelines
demands:
- npm
- msbuild
steps:
- task: Cache#2
displayName: 'npm Cache'
inputs:
key: 'npm | "$(Agent.OS)" | $(System.DefaultWorkingDirectory)/ABCDashboard/Angular/package-lock.json'
path: '$(System.DefaultWorkingDirectory)/ABCDashboard/Angular/node_modules'
cacheHitVar: 'CACHE_RESTORED'
restoreKeys: 'npm | "$(Agent.OS)"'
- task: Npm#1
displayName: 'npm ci'
inputs:
command: ci
workingDir: ABCDashboard/Angular
verbose: false
condition: eq(variables['CACHE_RESTORED'],False)
- task: Npm#1
displayName: 'npm custom: angular build'
inputs:
command: custom
workingDir: ABCDashboard/Angular
verbose: false
customCommand: 'run-script build -- --prod'
- task: NuGetCommand#2
displayName: 'NuGet restore'
- task: MSBuild#1
displayName: '.Net build | Build solution (No need to build test as well)'
inputs:
solution: 'ABCDashboard/*.csproj'
msbuildArchitecture: x64
configuration: Release
msbuildArguments: '/p:OutputPath=$(Build.ArtifactStagingDirectory)'
clean: true
- task: Snyk.snyk-security-scan.custom-build-release-task.SnykSecurityScan#0
displayName: 'Snyk scan for open source vulnerabilities'
inputs:
serviceConnectionEndpoint: SnykAuthFreeTier
failOnIssues: false
testDirectory: ABCDashboard
additionalArguments: '-d --all-projects'
- task: CopyFiles#2
displayName: 'Copy Files to: $(Build.ArtifactStagingDirectory)/_PublishedWebsites/ABCDashboard'
inputs:
SourceFolder: ABCDashboard
Contents: |
Bundles/**
!(packages.config|phantomjs-license.txt)
TargetFolder: '$(Build.ArtifactStagingDirectory)/_PublishedWebsites/ABCDashboard'
- script: |
cd $(Build.ArtifactStagingDirectory)/_PublishedWebsites/ABCDashboard
ls -ltr
displayName: 'List artifacts under $(Build.ArtifactStagingDirectory)/_PublishedWebsites/ABCDashboard'
enabled: false
- task: CopyFiles#2
displayName: 'Copy Files to: $(Build.ArtifactStagingDirectory)/Release'
inputs:
SourceFolder: '$(Build.ArtifactStagingDirectory)/_PublishedWebsites/ABCDashboard'
Contents: |
assets/**
bin/**
Bundles/**
Content/**
Scripts/**
Views/**
Web.config
Web.Dev.config
Web.Test.config
Web.Prod.config
Web.Beta.config
Web.Sandbox.config
ApplicationInsights.config
*.asax
*.ico
*.txt
TargetFolder: '$(Build.ArtifactStagingDirectory)/Release'
- task: ArchiveFiles#2
displayName: 'Archive $(Build.ArtifactStagingDirectory)/Release'
inputs:
rootFolderOrFile: '$(Build.ArtifactStagingDirectory)/Release'
includeRootFolder: false
archiveFile: '$(Build.ArtifactStagingDirectory)/Release/$(Build.BuildId).zip'
verbose: true
- task: PublishBuildArtifacts#1
displayName: 'Publish Artifact: Release'
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)/Release'
ArtifactName: Release
I am looking into Azure Devops Artifacts, and from what I understand, we can create feeds to store those published build artifacts?
From my understanding, those kind of build artifacts my pipeline publishes are considered "Universal Packages"? and If so, I would have to add a Universal Package task like so in order for the build artifact (e.g. Release) to be stored in the Artifacts feed?
- task: UniversalPackages#0
displayName: 'Universal publish'
inputs:
command: publish
vstsFeedPublish: 'feedname'
vstsFeedPackagePublish: test
versionOption: custom
versionPublish: 1.1.0
I had already gone ahead and created a feed, do I just wait for next time I run a pipeline for a Release artifact to show up in the feed? or do I have to add "upstream sources" first? If upstream sources is a required step, I selected "Azure Artifacts feed in this organization" as the upstream source type and then when going over the configuration setting, for some reason, it's complaining that "This feed has no available views to be used as an upstream source" despite defined views as shown here:
Based on your requirement, you can use feed to store Build artifacts.
From my understanding, those kind of build artifacts my pipeline publishes are considered "Universal Packages"?
Your understanding is correct. You can use Universal Package Publish task to upload the build artifacts to Azure Feed.
Do i just wait for next time I run a pipeline for a Release artifact to show up in the feed? or do i have to add "upstream sources" first?
You don't need to add upstream sources for the feed.
To publish the Universal Package, you can directly use Universal Package Publish task in Pipeline.
For example:
- task: UniversalPackages#0
displayName: 'Universal publish'
inputs:
command: publish
vstsFeedPublish: 'name'
vstsFeedPackagePublish: kevintest
Result:
To use the published Universal Package, you can directly use Universal Package download task.
For example:
- task: UniversalPackages#0
displayName: 'Universal download'
inputs:
command: download
vstsFeed: 'name'
vstsFeedPackage: 'name'
vstsPackageVersion: 0.0.1
I have searched around for a while, but not been able to find a good way to create a "installation bundle" for web service to run on IIS.
My problem is that we need to manually copy files to the on premises servers due to security, and thus not allowed to automate this job. But as far as I can see all the IIS Deployment templates uses Deployment group to distribute the releases to registered servers.
Is there a way to create a Release pipeline for IIS that produce a zip file/artifact that can be downloaded manually from DevOps or from a drop folder instead?
I have made one Release pipeline using tasks CopyFiles#2, FileTransform#1 and UniversalPackages#0 to copy the build artifact, transform the appsettings.json file and publish the package, but this does not add the nesessary files for IIS, such as the web.config file.
Thanx for any responses :-)
This is the build pipeline yml file:
trigger:
- release*
pool:
vmImage: 'windows-latest'
variables:
solution: '**/*.sln'
buildPlatform: 'Any CPU'
buildConfiguration: 'Release'
steps:
- task: DotNetCoreCLI#2
displayName: 'dotnet build'
inputs:
command: build
projects: '**/*.sln'
configuration: '$(buildConfiguration)'
- task: DotNetCoreCLI#2
displayName: 'dotnet test'
inputs:
command: test
projects: '**/*Tests.csproj'
arguments: '--configuration $(buildConfiguration) --no-restore --collect "Code coverage"'
testRunTitle: 'Unit and Integrtion Tests'
- task: DotNetCoreCLI#2
displayName: 'dotnet publish'
inputs:
command: publish
publishWebProjects: true
arguments: '--configuration $(buildConfiguration) --output $(build.artifactStagingDirectory)'
zipAfterPublish: true
- task: PublishPipelineArtifact#1
displayName: 'publish pipeline artifact'
inputs:
targetPath: '$(build.artifactStagingDirectory)'
artifactName: 'WebAPI'
And the Release pipeline:
Release pipeline
You could directly use a release pipeline which associated with a build pipeline.
Add the related build pipeline under Artifacts, also, remember to add the Publish build artifacts Task under the build pipeline, which will help the release catch the final zip folder.
Then, you could create a Release under Stages, use Manage IISWebsite Task and Deploy IIS Webiste/App Task.
That is enough.
After som twists and turns I ended up with this solution, using a template yml file for Dev and Release.
If anyone have any suggestions for improvments, please comment below.
Hope this can be of help :-)
Pipelines:
azure-pipelines-build-template.yml:
variables:
buildPlatform: 'Any CPU'
buildConfiguration: 'Release'
artifactPath: '$(build.artifactStagingDirectory)'
steps:
- task: DotNetCoreCLI#2
displayName: 'dotnet build solution'
inputs:
command: build
projects: '**/*.sln'
configuration: '$(buildConfiguration)'
- task: DotNetCoreCLI#2
displayName: 'dotnet test solution'
inputs:
command: test
projects: '**/*Tests.csproj'
arguments: '--configuration $(buildConfiguration) --no-restore'
testRunTitle: 'Unit and Integrtion Tests'
- task: DotNetCoreCLI#2
displayName: 'dotnet publish to pipeline'
inputs:
command: publish
publishWebProjects: true
arguments: '--configuration $(buildConfiguration) --output $(artifactPath)'
zipAfterPublish: true
- task: PublishPipelineArtifact#1
displayName: 'publish pipeline artifact Flyttavle'
inputs:
targetPath: '$(artifactPath)'
artifactName: 'WebAPI'
azure-pipelines-dev-build.yml:
trigger:
- dev
pool:
vmImage: 'windows-latest'
extends:
template: azure-pipelines-build-template.yml
azure-pipelines-release-build.yml:
trigger:
- release*
pool:
vmImage: 'windows-latest'
extends:
template: azure-pipelines-build-template.yml
Releases:
I have created 4 releases, Azure_Dev_Deploy and Azure_Release_Deploy which deploys to Azure, and On_Prem_Prod_Deploy and On_Prem_Test_Deploy that creates packages to download for on premise deployment.
Dev_Deploy and Release_Deploy:
Difference here are:
Trigger is set up for changes respectively on dev and release branches
Deployment points to the app service in Azure for respectively dev and release app services
Variables points to Variable groups respectively for dev and release variable substitutions (defined in Library)
Release Pipeline and Tasks for Dev_Deploy and Release_Deploy
(same setup for both, but picture shows the Dev deployment)
And the yaml view of the task:
Deploy Azure App Service:
steps:
- task: AzureRmWebAppDeployment#4
displayName: 'Deploy Azure App Service'
inputs:
azureSubscription: '$(Parameters.ConnectedServiceName)'
appType: '$(Parameters.WebAppKind)'
WebAppName: '$(Parameters.WebAppName)'
packageForLinux: '$(System.DefaultWorkingDirectory)/_BEDevBuild/WebAPI.zip'
JSONFiles: '**/appsettings*.json'
On_Prem_Test_Deploy and On_Prem_Prod_Deploy:
Difference here are:
Creates Artifacts respectively for on-prem-test and on-prem-prod
Variables points to Variable groups respectively for on-prem-test and on-prem-prod variable substitutions (defined in Library)
Release Pipeline and Tasks for On_Prem_Test_Deploy and On_Prem_Prod_Deploy
(same setup for both, but picture shows the Test deployment)
And the yaml view of the tasks:
File Transform:
steps:
- task: FileTransform#1
displayName: 'File Transform: on-prem/prod'
inputs:
folderPath: '$(System.ArtifactsDirectory)/_BEReleaseBuild/*.zip'
fileType: json
targetFiles: '**/appsettings*.json'
Universal publish:
steps:
- task: UniversalPackages#0
displayName: 'Universal publish'
inputs:
command: publish
publishDirectory: '$(System.ArtifactsDirectory)/_BEReleaseBuild'
vstsFeedPublish: '********-****-****-****-************/********-****-****-****-************'
vstsFeedPackagePublish: 'on-prem-prod'
packagePublishDescription: 'Package for on premise production'
Download Artifacts:
Then to download the artifacts in the Artifacts feed, I use the following statement from Powershell locally.
PS! As for today it is not possible to download from a REST call for the universal package which ruined my grand plan of creating a download link in the Wiki, but hopefully this will be supported in the future.
on-prem-test:
az artifacts universal download --organization "https://dev.azure.com/[my org name]/" --project "[DevOps project id]" --scope project --feed "on-premise" --name "on-prem-test" --version "*" --path .
on-prem-prod:
az artifacts universal download --organization "https://dev.azure.com/[my org name]/" --project "[DevOps project id]" --scope project --feed "on-premise" --name "on-prem-prod" --version "*" --path .
Tip on parameters:
--version "*" will allways get latest version in the feed for the given package name
--path . downloads the package to where you execute the command in Powershell
Within our company we build and release a lot through DevOps for .NET Core applications.
For these application our setup is to have a single pipeline that builds the artifact and then the release pipeline manages pushing this same artifact out through the various stages (test / uat / staging / live)
The advantages we see here is that the package is the same and it is just environment variables on the deployment target that allow for variation in how it runs such as different 3rd party endpoints, different database etc.
We are now looking to move a Vue.js application which is built using Webpack into DevOps for automating the builds and deployments too, but here is where we are in a conundrum.
We need to encapsulate the same variation in our solutions (different api, configs, etc), this is currently managed by doing npm run build:uat, npm run build:live etc.
This works fine, but means we would need to setup different builds for each environment and loose the reassurance the the package we have put out for UAT is consistent with the release we put out for Live.
Are there any best practises around managing builds like this?
Possible options I can see, although open to others:
Build test / uat / live at the same time and then selectively copy the right code for the right environment
Different builds for each release
Can we have a single artifact build and then swap out the configuration variable in some other way?
Any support or advice would be appreciated.
Option 2 should be the simplest solution to understand and operate.
Do not need to keep using a single artifact. A release can be configured to trigger off of multiple artifact sources (builds). Then simply combine multiple builds in one release.
You could create a task which add a label to the current build if there is no other build running/queued.
Additionally add a tag condition for the release trigger.
More details please take a look at boindiil's answer in this link: Combine multiple builds into one release
We ended up going down the following route. We would ideally have had a single build but as the build for each environment is done at the same time from the same npm install install etc I'm happy with this as a compromise.
trigger:
- release/*
variables:
- template: config/app-names.yml
pool:
vmImage: 'ubuntu-latest'
steps:
- task: NodeTool#0
inputs:
versionSpec: '12.x'
displayName: 'Install Node.js'
- script:
npm install
displayName: 'npm install'
- script:
npm run build:test -- --vn "$(Build.SourceBranchName)" --n "$(Build.RequestedFor)"
displayName: 'npm build test'
- task: ArchiveFiles#2
displayName: 'Archive test'
inputs:
rootFolderOrFile: 'dist/$(TestAppName)'
includeRootFolder: true
archiveType: 'zip'
archiveFile: 'dist/test/$(TestAppName).zip'
replaceExistingArchive: true
- script:
npm run build:acc -- --vn "$(Build.SourceBranchName)" --n "$(Build.RequestedFor)"
displayName: 'npm build acc'
- task: ArchiveFiles#2
displayName: 'Archive acc'
inputs:
rootFolderOrFile: 'dist/$(AccAppName)'
includeRootFolder: true
archiveType: 'zip'
archiveFile: 'dist/acc/$(AccAppName).zip'
replaceExistingArchive: true
- script:
npm run build:live -- --vn "$(Build.SourceBranchName)" --n "$(Build.RequestedFor)"
displayName: 'npm build live'
- task: ArchiveFiles#2
displayName: 'Archive live'
inputs:
rootFolderOrFile: 'dist/$(LiveAppName)'
includeRootFolder: true
archiveType: 'zip'
archiveFile: 'dist/live/$(LiveAppName).zip'
replaceExistingArchive: true
- task: PublishBuildArtifacts#1
displayName: 'Publish test build'
inputs:
PathtoPublish: 'dist/test/'
ArtifactName: 'test'
publishLocation: 'Container'
- task: PublishBuildArtifacts#1
displayName: 'Publish acc build'
inputs:
PathtoPublish: 'dist/acc/'
ArtifactName: 'acc'
publishLocation: 'Container'
- task: PublishBuildArtifacts#1
displayName: 'Publish live build'
inputs:
PathtoPublish: 'dist/live/'
ArtifactName: 'live'
publishLocation: 'Container'
There is a duplication of a dist folder in the artifacts produced by the AzureDevOps -> Pipelines, the duplication is the /dist folder and also /drop/dist folder. EDIT: Full azure-pipeline.yml file
# Node.js with Angular
# Build a Node.js project that uses Angular.
# Add steps that analyze code, save build artifacts, deploy, and more:
# https://learn.microsoft.com/azure/devops/pipelines/languages/javascript
# Major modification referencing
# https://dev.to/thisdotmedia/continuously-integrating-angular-with-azure-devops-2k9l
trigger:
- master
pool:
vmImage: 'ubuntu-latest'
steps:
- task: NodeTool#0
inputs:
versionSpec: '10.x'
displayName: 'Install Node.js'
# Build angular app area
- script: npm install
displayName: 'npm install'
- script: npx ng build --prod
displayName: 'npm build'
# Testing area
- script: npm install puppeteer --save-dev
displayName: 'Installing puppeteer (Headless browser for testing)'
- script: npx ng test --watch=false --codeCoverage=true
displayName: 'Running Tests'
- task: PublishTestResults#2
condition: succeededOrFailed()
inputs:
testResultsFormat: 'JUnit'
testResultsFiles: '**/TEST-*.xml'
displayName: 'Publish Test Results'
# Publishing items
# deploy.psl (Powershell script to deploy)
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: 'deploy.ps1'
ArtifactName: 'drop'
publishLocation: 'Container'
# Firebase.json
- task: PublishBuildArtifacts#1
displayName: 'Publish Artifact: Firebase.json'
inputs:
PathtoPublish: 'firebase.json'
ArtifactName: 'drop'
publishLocation: 'Container'
# App
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: 'dist'
ArtifactName: 'drop/dist'
publishLocation: 'Container'
displayName: 'Publish Artifacts'
# Code Coverage Results
- task: PublishCodeCoverageResults#1
condition: succeededOrFailed()
inputs:
codeCoverageTool: 'Cobertura'
summaryFileLocation: '$(Build.SourcesDirectory)/coverage/ng-azure-devops/cobertura-coverage.xml'
displayName: 'Publish Code Coverage Results'
- script: npx ng lint
displayName: 'Code Analysis'
I've tried the using 'drop' as the ArtifactName, which will NOT produce a duplicate folder artifact anywhere. I am very confused on why 'drop/dist' will produce another '/dist' artifact
AzureDevOps Duplicate dist folder in pipelines build ? Why?
I could reproduce this issue on my side.
When we use the publish the artifacts dist folder with ArtifactName: drop/dist, Azure Devops will create a new folder drop first, then publish the artifacts dist folder to that folder drop.
You can get this message from the build log:
Upload '/home/vsts/work/1/s/dist' to file container:
'#/3620698/drop/dist'
However, the drop folder is already present by default. When we publish the dist folder with with ArtifactName: drop/dist, there are two drop folder, then Azure devops will publish dist folder to those two drop folders:
In order to understand this problem more clearly, you could disable the Multi-stage pipelines in the Preview features, then you will get the output:
Obviously, there are two drop folders here, that is the reason why you get the Duplicate dist folder in pipelines build.
So, to resolve this issue, we could change the ArtifactName: drop/dist to ArtifactName: dropTest/dist:
Now, the duplicate dist folder disappears.
Hope this helps.