Managing consistent Webpack builds in DevOps - azure-devops

Within our company we build and release a lot through DevOps for .NET Core applications.
For these application our setup is to have a single pipeline that builds the artifact and then the release pipeline manages pushing this same artifact out through the various stages (test / uat / staging / live)
The advantages we see here is that the package is the same and it is just environment variables on the deployment target that allow for variation in how it runs such as different 3rd party endpoints, different database etc.
We are now looking to move a Vue.js application which is built using Webpack into DevOps for automating the builds and deployments too, but here is where we are in a conundrum.
We need to encapsulate the same variation in our solutions (different api, configs, etc), this is currently managed by doing npm run build:uat, npm run build:live etc.
This works fine, but means we would need to setup different builds for each environment and loose the reassurance the the package we have put out for UAT is consistent with the release we put out for Live.
Are there any best practises around managing builds like this?
Possible options I can see, although open to others:
Build test / uat / live at the same time and then selectively copy the right code for the right environment
Different builds for each release
Can we have a single artifact build and then swap out the configuration variable in some other way?
Any support or advice would be appreciated.

Option 2 should be the simplest solution to understand and operate.
Do not need to keep using a single artifact. A release can be configured to trigger off of multiple artifact sources (builds). Then simply combine multiple builds in one release.
You could create a task which add a label to the current build if there is no other build running/queued.
Additionally add a tag condition for the release trigger.
More details please take a look at boindiil's answer in this link: Combine multiple builds into one release

We ended up going down the following route. We would ideally have had a single build but as the build for each environment is done at the same time from the same npm install install etc I'm happy with this as a compromise.
trigger:
- release/*
variables:
- template: config/app-names.yml
pool:
vmImage: 'ubuntu-latest'
steps:
- task: NodeTool#0
inputs:
versionSpec: '12.x'
displayName: 'Install Node.js'
- script:
npm install
displayName: 'npm install'
- script:
npm run build:test -- --vn "$(Build.SourceBranchName)" --n "$(Build.RequestedFor)"
displayName: 'npm build test'
- task: ArchiveFiles#2
displayName: 'Archive test'
inputs:
rootFolderOrFile: 'dist/$(TestAppName)'
includeRootFolder: true
archiveType: 'zip'
archiveFile: 'dist/test/$(TestAppName).zip'
replaceExistingArchive: true
- script:
npm run build:acc -- --vn "$(Build.SourceBranchName)" --n "$(Build.RequestedFor)"
displayName: 'npm build acc'
- task: ArchiveFiles#2
displayName: 'Archive acc'
inputs:
rootFolderOrFile: 'dist/$(AccAppName)'
includeRootFolder: true
archiveType: 'zip'
archiveFile: 'dist/acc/$(AccAppName).zip'
replaceExistingArchive: true
- script:
npm run build:live -- --vn "$(Build.SourceBranchName)" --n "$(Build.RequestedFor)"
displayName: 'npm build live'
- task: ArchiveFiles#2
displayName: 'Archive live'
inputs:
rootFolderOrFile: 'dist/$(LiveAppName)'
includeRootFolder: true
archiveType: 'zip'
archiveFile: 'dist/live/$(LiveAppName).zip'
replaceExistingArchive: true
- task: PublishBuildArtifacts#1
displayName: 'Publish test build'
inputs:
PathtoPublish: 'dist/test/'
ArtifactName: 'test'
publishLocation: 'Container'
- task: PublishBuildArtifacts#1
displayName: 'Publish acc build'
inputs:
PathtoPublish: 'dist/acc/'
ArtifactName: 'acc'
publishLocation: 'Container'
- task: PublishBuildArtifacts#1
displayName: 'Publish live build'
inputs:
PathtoPublish: 'dist/live/'
ArtifactName: 'live'
publishLocation: 'Container'

Related

How to consume artifacts published by CI pipeline in CD pipeline

I have CI and CD pipelines using Azure DevOps for a frontend angular project. Both are separate pipelines.
Here goes the YAML file for the CI pipeline which produces published artifact: output_final.zip. The below pipeline leverages Azure Pipelines for generating the published artifact.
# Node.js with Angular
# Build a Node.js project that uses Angular.
# Add steps that analyze code, save build artifacts, deploy, and more:
# https://learn.microsoft.com/azure/devops/pipelines/languages/javascript
trigger:
- integration
pool:
vmImage: ubuntu-latest
steps:
- task: NodeTool#0
inputs:
versionSpec: '14.x'
displayName: 'Install Node.js'
- powershell: |
$buildNumber="$(Build.BuildNumber)"
echo $buildNumber > src/version.txt
- script: |
npm install -g #angular/cli
npm install
ng build --prod
displayName: 'npm install and build'
- task: CopyFiles#2
displayName: 'Copy Files of UI'
inputs:
SourceFolder: 'dist/source'
TargetFolder: '$(Build.ArtifactStagingDirectory)/output'
OverWrite: true
- task: ArchiveFiles#2
displayName: 'Archive'
inputs:
rootFolderOrFile: '$(Build.ArtifactStagingDirectory)/output/'
includeRootFolder: false
archiveFile: '$(Build.ArtifactStagingDirectory)/output/output_final.zip'
- task: PublishBuildArtifacts#1
displayName: 'Publish Artifact'
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)/output/output_final.zip'
ArtifactName: drop
Now I have a separate CD pipeline which leverages self hosted private agent. In this CD pipeline, I want to consume artifacts published by the CI pipeline in the CD pipeline
Can anyone help me to know how to consume artifacts published by the CI pipeline and use it in the CD pipeline with some sample YAML example.
My suggestion would be to use the Publish Pipeline Artifact task to publish the CI artifact. Then, in your CD pipeline, use the Download Pipeline Artifact to consume it.
You can configure the download task to get the artifact from the current run or from a specific run. In your case, it sounds like you want a specific run since the CI pipeline is separate.
Here's an example of what the yaml tasks might look like:
#CI Task
- task: PublishPipelineArtifact#1
displayName: 'Publish pipeline Artifact'
inputs:
targetPath: '$(Pipeline.Workspace)'
artifact: '<Some Artifact Name>'
publishLocation: 'pipeline'
#CD Task example
- task: DownloadPipelineArtifact#2
inputs:
buildType: 'specific'
project: ''
definition: ''
specificBuildWithTriggering: true
buildVersionToDownload: 'latest'
targetPath: '$(Pipeline.Workspace)'

How to create a deployment to an IIS server out of sight from DevOps Release pipeline

I have searched around for a while, but not been able to find a good way to create a "installation bundle" for web service to run on IIS.
My problem is that we need to manually copy files to the on premises servers due to security, and thus not allowed to automate this job. But as far as I can see all the IIS Deployment templates uses Deployment group to distribute the releases to registered servers.
Is there a way to create a Release pipeline for IIS that produce a zip file/artifact that can be downloaded manually from DevOps or from a drop folder instead?
I have made one Release pipeline using tasks CopyFiles#2, FileTransform#1 and UniversalPackages#0 to copy the build artifact, transform the appsettings.json file and publish the package, but this does not add the nesessary files for IIS, such as the web.config file.
Thanx for any responses :-)
This is the build pipeline yml file:
trigger:
- release*
pool:
vmImage: 'windows-latest'
variables:
solution: '**/*.sln'
buildPlatform: 'Any CPU'
buildConfiguration: 'Release'
steps:
- task: DotNetCoreCLI#2
displayName: 'dotnet build'
inputs:
command: build
projects: '**/*.sln'
configuration: '$(buildConfiguration)'
- task: DotNetCoreCLI#2
displayName: 'dotnet test'
inputs:
command: test
projects: '**/*Tests.csproj'
arguments: '--configuration $(buildConfiguration) --no-restore --collect "Code coverage"'
testRunTitle: 'Unit and Integrtion Tests'
- task: DotNetCoreCLI#2
displayName: 'dotnet publish'
inputs:
command: publish
publishWebProjects: true
arguments: '--configuration $(buildConfiguration) --output $(build.artifactStagingDirectory)'
zipAfterPublish: true
- task: PublishPipelineArtifact#1
displayName: 'publish pipeline artifact'
inputs:
targetPath: '$(build.artifactStagingDirectory)'
artifactName: 'WebAPI'
And the Release pipeline:
Release pipeline
You could directly use a release pipeline which associated with a build pipeline.
Add the related build pipeline under Artifacts, also, remember to add the Publish build artifacts Task under the build pipeline, which will help the release catch the final zip folder.
Then, you could create a Release under Stages, use Manage IISWebsite Task and Deploy IIS Webiste/App Task.
That is enough.
After som twists and turns I ended up with this solution, using a template yml file for Dev and Release.
If anyone have any suggestions for improvments, please comment below.
Hope this can be of help :-)
Pipelines:
azure-pipelines-build-template.yml:
variables:
buildPlatform: 'Any CPU'
buildConfiguration: 'Release'
artifactPath: '$(build.artifactStagingDirectory)'
steps:
- task: DotNetCoreCLI#2
displayName: 'dotnet build solution'
inputs:
command: build
projects: '**/*.sln'
configuration: '$(buildConfiguration)'
- task: DotNetCoreCLI#2
displayName: 'dotnet test solution'
inputs:
command: test
projects: '**/*Tests.csproj'
arguments: '--configuration $(buildConfiguration) --no-restore'
testRunTitle: 'Unit and Integrtion Tests'
- task: DotNetCoreCLI#2
displayName: 'dotnet publish to pipeline'
inputs:
command: publish
publishWebProjects: true
arguments: '--configuration $(buildConfiguration) --output $(artifactPath)'
zipAfterPublish: true
- task: PublishPipelineArtifact#1
displayName: 'publish pipeline artifact Flyttavle'
inputs:
targetPath: '$(artifactPath)'
artifactName: 'WebAPI'
azure-pipelines-dev-build.yml:
trigger:
- dev
pool:
vmImage: 'windows-latest'
extends:
template: azure-pipelines-build-template.yml
azure-pipelines-release-build.yml:
trigger:
- release*
pool:
vmImage: 'windows-latest'
extends:
template: azure-pipelines-build-template.yml
Releases:
I have created 4 releases, Azure_Dev_Deploy and Azure_Release_Deploy which deploys to Azure, and On_Prem_Prod_Deploy and On_Prem_Test_Deploy that creates packages to download for on premise deployment.
Dev_Deploy and Release_Deploy:
Difference here are:
Trigger is set up for changes respectively on dev and release branches
Deployment points to the app service in Azure for respectively dev and release app services
Variables points to Variable groups respectively for dev and release variable substitutions (defined in Library)
Release Pipeline and Tasks for Dev_Deploy and Release_Deploy
(same setup for both, but picture shows the Dev deployment)
And the yaml view of the task:
Deploy Azure App Service:
steps:
- task: AzureRmWebAppDeployment#4
displayName: 'Deploy Azure App Service'
inputs:
azureSubscription: '$(Parameters.ConnectedServiceName)'
appType: '$(Parameters.WebAppKind)'
WebAppName: '$(Parameters.WebAppName)'
packageForLinux: '$(System.DefaultWorkingDirectory)/_BEDevBuild/WebAPI.zip'
JSONFiles: '**/appsettings*.json'
On_Prem_Test_Deploy and On_Prem_Prod_Deploy:
Difference here are:
Creates Artifacts respectively for on-prem-test and on-prem-prod
Variables points to Variable groups respectively for on-prem-test and on-prem-prod variable substitutions (defined in Library)
Release Pipeline and Tasks for On_Prem_Test_Deploy and On_Prem_Prod_Deploy
(same setup for both, but picture shows the Test deployment)
And the yaml view of the tasks:
File Transform:
steps:
- task: FileTransform#1
displayName: 'File Transform: on-prem/prod'
inputs:
folderPath: '$(System.ArtifactsDirectory)/_BEReleaseBuild/*.zip'
fileType: json
targetFiles: '**/appsettings*.json'
Universal publish:
steps:
- task: UniversalPackages#0
displayName: 'Universal publish'
inputs:
command: publish
publishDirectory: '$(System.ArtifactsDirectory)/_BEReleaseBuild'
vstsFeedPublish: '********-****-****-****-************/********-****-****-****-************'
vstsFeedPackagePublish: 'on-prem-prod'
packagePublishDescription: 'Package for on premise production'
Download Artifacts:
Then to download the artifacts in the Artifacts feed, I use the following statement from Powershell locally.
PS! As for today it is not possible to download from a REST call for the universal package which ruined my grand plan of creating a download link in the Wiki, but hopefully this will be supported in the future.
on-prem-test:
az artifacts universal download --organization "https://dev.azure.com/[my org name]/" --project "[DevOps project id]" --scope project --feed "on-premise" --name "on-prem-test" --version "*" --path .
on-prem-prod:
az artifacts universal download --organization "https://dev.azure.com/[my org name]/" --project "[DevOps project id]" --scope project --feed "on-premise" --name "on-prem-prod" --version "*" --path .
Tip on parameters:
--version "*" will allways get latest version in the feed for the given package name
--path . downloads the package to where you execute the command in Powershell

React JS Azure DevOps Web app won't run but files are present in server and no errors in pipelines

I am doing a simple Azure DevOps CICD deployment. I am following all the steps. First up, here is my YAML file. I have kept the comments as it is, just in case, I am making more mistakes than I am aware of.
trigger:
- master
pool:
vmImage: 'ubuntu-latest'
# Set variables
variables:
directory: ReactJSRecipeApp
steps:
- task: NodeTool#0
inputs:
versionSpec: '12.13.0'
displayName: 'Install Node.js'
- script:
npm install
displayName: 'npm install'
- script:
npm run build
displayName: 'npm build'
- task: CopyFiles#2
displayName: 'Copy files'
inputs:
sourceFolder: 'build'
Contents: '**/*'
TargetFolder: '$(Build.ArtifactStagingDirectory)'
cleanTargetFolder: true
- task: ArchiveFiles#2
displayName: 'Archive files'
inputs:
rootFolderOrFile: '$(Build.ArtifactStagingDirectory)'
includeRootFolder: false
archiveType: zip
archiveFile: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
replaceExistingArchive: true
- task: PublishBuildArtifacts#1
displayName: 'Publish Build Artifacts'
inputs:
pathtoPublish: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
ArtifactName: 'www' # output artifact named www
- task: AzureWebApp#1
displayName: 'Deploy to App Service'
inputs:
azureSubscription: 'ReactJSRecipeAppConnection'
appName: 'reactjsrecipeappwebapp2'
appType: 'webApp'
package: '$(System.ArtifactsDirectory)/$(Build.BuildId).zip'
customWebConfig: '-Handler iisnode -NodeStartFile server.js -appType node'
So, when I run this, I get no errors in the Pipeline.
Further, I want to point out the following.
If I download the artifacts zip folder, I am able to run that folder locally and get my react app running in a localhost server just fine.
I check my Azure web app via Kudo Tools, I see all the files inside wwwroot, with timestamps that match the zip file from the artifact folder. So, I am assuming that the files are indeed getting pushed and to the correct spot in the web server.
Before I run the CICD trigger, these azure web apps were created brand new, and I get the standard azure welcome/landing page. So, the web apps themselves are fine.
After all this, the website itself does not serve the pages. I get a 404. I have tried two different web apps on Azure but the same results.
Any advise, where I am going wrong?
Update 1
I decided to manually check the files on Filezilla. But, its empty!!!
But, KUDO shows files. I dont understand!
Update 2
So, I did a direct deploy from visual studio code with the artifact publish folder. the web app runs fine. So, did this step to make sure that the web app is configured correctly.
Alright, so, it looks like my YAML file was not correct. I finally got it to work.
I am posting it here if someone comes around looking for a ready to use React YAML file (because the Azure DevOps Documentation is not that useful in its current form)
trigger:
- master
pool:
vmImage: 'ubuntu-latest'
steps:
- task: NodeTool#0
inputs:
versionSpec: '10.x'
displayName: 'Install Node.js'
- script: |
npm install
npm run build
displayName: 'npm install and build'
- task: CopyFiles#2
inputs:
Contents: 'build/**' # Pull the build directory (React)
TargetFolder: '$(Build.ArtifactStagingDirectory)'
- task: PublishBuildArtifacts#1
inputs:
pathtoPublish: $(Build.ArtifactStagingDirectory) # dist or build files
ArtifactName: 'www' # output artifact named www
# Default value: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
- task: ArchiveFiles#2
inputs:
rootFolderOrFile: '$(Build.ArtifactStagingDirectory)/build/'
includeRootFolder: false
- task: AzureWebApp#1
inputs:
azureSubscription: 'ReactJSRecipeAppConnection'
appName: 'ReactJSRecipeApp4'
package: '$(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip'
The full repository with simple react code including the YAML is here.
https://github.com/Jay-study-nildana/ReactJSRecipeApp

How to launch a build pipeline on pull request

I would like to have a pull request launch a build and that build must succeed before merging into the target branch is allowed.
I have master and develop branch and create feature branches from develop. work is completed and then a pull request initiated to merge back into develop.
I have created a build pipeline that completes successfully when run manually.
I have specified in the branch policies for the develop branch that the build must be run and complete successfully.
Now when I create a pull request, it says that in order for the pull request to be approved, the build must run but it does not run the build. What am I doing wrong?
This is a c# .net framework app.
Should I be saving the yaml in the develop branch or master? That part confuses me. Is the trigger correct?
This is my yaml for the build pipeline:
trigger:
- develop
pool:
vmImage: 'windows-latest'
name: '1.0.1.$(Rev:r)'
variables:
solution: '**/*.sln'
buildPlatform: 'Any CPU'
buildConfiguration: 'Release'
steps:
- checkout: git://OvaFlow/Utilities#develop
- task: NuGetToolInstaller#0
displayName: 'Use NuGet 4.4.1'
inputs:
versionSpec: 4.4.1
- task: VersionAssemblies#2
inputs:
Path: '$(Build.SourcesDirectory)'
VersionNumber: '$(Build.BuildNumber)'
InjectVersion: true
FilenamePattern: 'AssemblyInfo.*'
OutputVersion: 'OutputedVersion'
- task: NuGetCommand#2
displayName: 'NuGet restore'
inputs:
restoreSolution: '$(Build.SourcesDirectory)/Utilities/packages.config'
feedsToUse: config
nugetConfigPath: '$(Build.SourcesDirectory)/NuGet.config'
restoreDirectory: '$(Build.SourcesDirectory)/packages'
- task: NuGetCommand#2
displayName: 'NuGet restore'
inputs:
restoreSolution: '$(Build.SourcesDirectory)/UtilitiesTests/packages.config'
feedsToUse: config
nugetConfigPath: '$(Build.SourcesDirectory)/NuGet.config'
restoreDirectory: '$(Build.SourcesDirectory)/packages'
- task: VSBuild#1
displayName: 'Build solution **\*.sln'
inputs:
solution: '$(Build.SourcesDirectory)/Utilities.sln'
platform: '$(BuildPlatform)'
configuration: '$(BuildConfiguration)'
- task: VSTest#2
displayName: 'VsTest - testAssemblies'
inputs:
testAssemblyVer2: |
**\$(BuildConfiguration)\*test*.dll
!**\obj\**
codeCoverageEnabled: true
platform: '$(BuildPlatform)'
configuration: '$(BuildConfiguration)'
continueOnError: true
- task: PublishSymbols#2
displayName: 'Publish symbols path'
inputs:
SearchPattern: '**\bin\**\*.pdb'
PublishSymbols: false
continueOnError: true
- task: CopyFiles#2
displayName: 'Copy Files to: $(build.artifactstagingdirectory)'
inputs:
SourceFolder: '$(system.defaultworkingdirectory)'
Contents: '**\bin\$(BuildConfiguration)\**'
TargetFolder: '$(build.artifactstagingdirectory)'
condition: succeededOrFailed()
- task: PublishBuildArtifacts#1
displayName: 'Publish Artifact: drop'
inputs:
PathtoPublish: '$(build.artifactstagingdirectory)'
condition: succeededOrFailed()
Should I be saving the yaml in the develop branch or master? That part confuses me. Is the trigger correct?
The answer is yes! In fact, you have already found the answer.
As we know from the Build validation:
Set a policy requiring changes in a pull request to build successfully
with the protected branch before the pull request can be completed.
Build policies reduce breaks and keep your test results passing. Build
policies help even if you're using continuous integration (CI) on your
development branches to catch problems early.
So, the build validation used for the branches which you want to protected, the .yml should saved in the branch which you set the build policy.
As test, if I set the .yml file under the Test branch, then pull request for the dev branch, the build will not be triggered automatically:
Then, I moved the .yml file to the dev branch with a new build pipeline, it works fine:
Hope this helps.
If build is not configured to run automatically on push, you have to run it manually, here:
Or you can configure build to trigger automatic on branch update:

AzureDevOps Duplicate dist folder in pipelines build ? Why?

There is a duplication of a dist folder in the artifacts produced by the AzureDevOps -> Pipelines, the duplication is the /dist folder and also /drop/dist folder. EDIT: Full azure-pipeline.yml file
# Node.js with Angular
# Build a Node.js project that uses Angular.
# Add steps that analyze code, save build artifacts, deploy, and more:
# https://learn.microsoft.com/azure/devops/pipelines/languages/javascript
# Major modification referencing
# https://dev.to/thisdotmedia/continuously-integrating-angular-with-azure-devops-2k9l
trigger:
- master
pool:
vmImage: 'ubuntu-latest'
steps:
- task: NodeTool#0
inputs:
versionSpec: '10.x'
displayName: 'Install Node.js'
# Build angular app area
- script: npm install
displayName: 'npm install'
- script: npx ng build --prod
displayName: 'npm build'
# Testing area
- script: npm install puppeteer --save-dev
displayName: 'Installing puppeteer (Headless browser for testing)'
- script: npx ng test --watch=false --codeCoverage=true
displayName: 'Running Tests'
- task: PublishTestResults#2
condition: succeededOrFailed()
inputs:
testResultsFormat: 'JUnit'
testResultsFiles: '**/TEST-*.xml'
displayName: 'Publish Test Results'
# Publishing items
# deploy.psl (Powershell script to deploy)
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: 'deploy.ps1'
ArtifactName: 'drop'
publishLocation: 'Container'
# Firebase.json
- task: PublishBuildArtifacts#1
displayName: 'Publish Artifact: Firebase.json'
inputs:
PathtoPublish: 'firebase.json'
ArtifactName: 'drop'
publishLocation: 'Container'
# App
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: 'dist'
ArtifactName: 'drop/dist'
publishLocation: 'Container'
displayName: 'Publish Artifacts'
# Code Coverage Results
- task: PublishCodeCoverageResults#1
condition: succeededOrFailed()
inputs:
codeCoverageTool: 'Cobertura'
summaryFileLocation: '$(Build.SourcesDirectory)/coverage/ng-azure-devops/cobertura-coverage.xml'
displayName: 'Publish Code Coverage Results'
- script: npx ng lint
displayName: 'Code Analysis'
I've tried the using 'drop' as the ArtifactName, which will NOT produce a duplicate folder artifact anywhere. I am very confused on why 'drop/dist' will produce another '/dist' artifact
AzureDevOps Duplicate dist folder in pipelines build ? Why?
I could reproduce this issue on my side.
When we use the publish the artifacts dist folder with ArtifactName: drop/dist, Azure Devops will create a new folder drop first, then publish the artifacts dist folder to that folder drop.
You can get this message from the build log:
Upload '/home/vsts/work/1/s/dist' to file container:
'#/3620698/drop/dist'
However, the drop folder is already present by default. When we publish the dist folder with with ArtifactName: drop/dist, there are two drop folder, then Azure devops will publish dist folder to those two drop folders:
In order to understand this problem more clearly, you could disable the Multi-stage pipelines in the Preview features, then you will get the output:
Obviously, there are two drop folders here, that is the reason why you get the Duplicate dist folder in pipelines build.
So, to resolve this issue, we could change the ArtifactName: drop/dist to ArtifactName: dropTest/dist:
Now, the duplicate dist folder disappears.
Hope this helps.