Azure DevOps yaml pipeline, downloaded artifact is empty - azure-devops

Converting a pipeline from classic to YAML (so I know it works).
I build and my publish artifact task looks like this:
- task: PublishPipelineArtifact#1
displayName: 'Publish Pipeline Artifact'
inputs:
targetPath: '$(Build.ArtifactStagingDirectory)/MyArtifact'
artifact: MyArtifact
The result when build looks like this:
I then want to deploy this to a webapp
YAML:
- stage: Dev
jobs:
- job: DeployApp
displayName: Deploy Web App
pool:
name: myPrivate
demands: msbuild
steps:
- task: DownloadPipelineArtifact#2
inputs:
artifact: MyArtifact
- task: AzureRmWebAppDeployment#4
displayName: 'Deploy App'
inputs:
WebAppKind: 'Web App On Windows'
azureSubscription: 'edited out...'
WebAppName: webAppName
package: '$(System.ArtifactsDirectory)'
enableXmlTransform: true
Which results in this output in Azure DevOps:
A difference in chunks but since size match very exactly I'm happy.
I then get error on the deployment so I started look in Kudo and can see that in SitePackages all my packages are 1 kb (or empty...). I expected 61.2 MB.
I took my working classic deployment, and there i had this row:
packageForLinux:'$(System.DefaultWorkingDirectory)/_MyArtifact/MyArtifact/MyApplication.zip'
But that path wasn't even found. And packageForLinux felt wrong but doesn't matter as I understand it.
So can someone please me in the right direction how to deploy a web app (on windows) using YAML?

The folder that's used is $(Pipeline.Workspace), not $(System.DefaultWorkingDirectory), the default folder is different for each task type, so you can't always trust they extract to the default working directory.

I've wasted a lot of time deciphering these folder paths. I'm adding this info to help anyone else with this issue. Obviously my artifact was empty because my staging folder was empty.
After a few hours of digging I discovered that $(Build.SourcesDirectory) represents the root of where my dacpac files are built to
In my case I'm using the CopyFiles task as it supports wildcards. So whatever dacpac files are generated under the bin subfolder will be picked up and copied to the artifacts staging directory
# Build and publish the database solution
trigger:
- master
# this builds on a self hosted pool rather than the MS cloud agent
pool:
name: MySelfHostedPool
demands:
- MSBuild
- SqlPackage
steps:
- task: MSBuild#1
displayName: Build DB solution
inputs:
solution: MemberDB.sln
logFileVerbosity: diagnostic
# Copy build output to artifact staging directory so they can be published as artifacts in the next step
- task: CopyFiles#2
displayName: Copy dacpac to artifacts staging
inputs:
SourceFolder: '$(Build.SourcesDirectory)'
flattenFolders: true
Contents: '**\bin\**\*.dacpac'
TargetFolder: '$(Build.ArtifactStagingDirectory)'
- task: PublishPipelineArtifact#1
displayName: Publish dacpac
inputs:
artifactname: 'drop'
targetPath: $(Build.ArtifactStagingDirectory)

Related

How to consume artifacts published by CI pipeline in CD pipeline

I have CI and CD pipelines using Azure DevOps for a frontend angular project. Both are separate pipelines.
Here goes the YAML file for the CI pipeline which produces published artifact: output_final.zip. The below pipeline leverages Azure Pipelines for generating the published artifact.
# Node.js with Angular
# Build a Node.js project that uses Angular.
# Add steps that analyze code, save build artifacts, deploy, and more:
# https://learn.microsoft.com/azure/devops/pipelines/languages/javascript
trigger:
- integration
pool:
vmImage: ubuntu-latest
steps:
- task: NodeTool#0
inputs:
versionSpec: '14.x'
displayName: 'Install Node.js'
- powershell: |
$buildNumber="$(Build.BuildNumber)"
echo $buildNumber > src/version.txt
- script: |
npm install -g #angular/cli
npm install
ng build --prod
displayName: 'npm install and build'
- task: CopyFiles#2
displayName: 'Copy Files of UI'
inputs:
SourceFolder: 'dist/source'
TargetFolder: '$(Build.ArtifactStagingDirectory)/output'
OverWrite: true
- task: ArchiveFiles#2
displayName: 'Archive'
inputs:
rootFolderOrFile: '$(Build.ArtifactStagingDirectory)/output/'
includeRootFolder: false
archiveFile: '$(Build.ArtifactStagingDirectory)/output/output_final.zip'
- task: PublishBuildArtifacts#1
displayName: 'Publish Artifact'
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)/output/output_final.zip'
ArtifactName: drop
Now I have a separate CD pipeline which leverages self hosted private agent. In this CD pipeline, I want to consume artifacts published by the CI pipeline in the CD pipeline
Can anyone help me to know how to consume artifacts published by the CI pipeline and use it in the CD pipeline with some sample YAML example.
My suggestion would be to use the Publish Pipeline Artifact task to publish the CI artifact. Then, in your CD pipeline, use the Download Pipeline Artifact to consume it.
You can configure the download task to get the artifact from the current run or from a specific run. In your case, it sounds like you want a specific run since the CI pipeline is separate.
Here's an example of what the yaml tasks might look like:
#CI Task
- task: PublishPipelineArtifact#1
displayName: 'Publish pipeline Artifact'
inputs:
targetPath: '$(Pipeline.Workspace)'
artifact: '<Some Artifact Name>'
publishLocation: 'pipeline'
#CD Task example
- task: DownloadPipelineArtifact#2
inputs:
buildType: 'specific'
project: ''
definition: ''
specificBuildWithTriggering: true
buildVersionToDownload: 'latest'
targetPath: '$(Pipeline.Workspace)'

React JS Azure DevOps Web app won't run but files are present in server and no errors in pipelines

I am doing a simple Azure DevOps CICD deployment. I am following all the steps. First up, here is my YAML file. I have kept the comments as it is, just in case, I am making more mistakes than I am aware of.
trigger:
- master
pool:
vmImage: 'ubuntu-latest'
# Set variables
variables:
directory: ReactJSRecipeApp
steps:
- task: NodeTool#0
inputs:
versionSpec: '12.13.0'
displayName: 'Install Node.js'
- script:
npm install
displayName: 'npm install'
- script:
npm run build
displayName: 'npm build'
- task: CopyFiles#2
displayName: 'Copy files'
inputs:
sourceFolder: 'build'
Contents: '**/*'
TargetFolder: '$(Build.ArtifactStagingDirectory)'
cleanTargetFolder: true
- task: ArchiveFiles#2
displayName: 'Archive files'
inputs:
rootFolderOrFile: '$(Build.ArtifactStagingDirectory)'
includeRootFolder: false
archiveType: zip
archiveFile: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
replaceExistingArchive: true
- task: PublishBuildArtifacts#1
displayName: 'Publish Build Artifacts'
inputs:
pathtoPublish: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
ArtifactName: 'www' # output artifact named www
- task: AzureWebApp#1
displayName: 'Deploy to App Service'
inputs:
azureSubscription: 'ReactJSRecipeAppConnection'
appName: 'reactjsrecipeappwebapp2'
appType: 'webApp'
package: '$(System.ArtifactsDirectory)/$(Build.BuildId).zip'
customWebConfig: '-Handler iisnode -NodeStartFile server.js -appType node'
So, when I run this, I get no errors in the Pipeline.
Further, I want to point out the following.
If I download the artifacts zip folder, I am able to run that folder locally and get my react app running in a localhost server just fine.
I check my Azure web app via Kudo Tools, I see all the files inside wwwroot, with timestamps that match the zip file from the artifact folder. So, I am assuming that the files are indeed getting pushed and to the correct spot in the web server.
Before I run the CICD trigger, these azure web apps were created brand new, and I get the standard azure welcome/landing page. So, the web apps themselves are fine.
After all this, the website itself does not serve the pages. I get a 404. I have tried two different web apps on Azure but the same results.
Any advise, where I am going wrong?
Update 1
I decided to manually check the files on Filezilla. But, its empty!!!
But, KUDO shows files. I dont understand!
Update 2
So, I did a direct deploy from visual studio code with the artifact publish folder. the web app runs fine. So, did this step to make sure that the web app is configured correctly.
Alright, so, it looks like my YAML file was not correct. I finally got it to work.
I am posting it here if someone comes around looking for a ready to use React YAML file (because the Azure DevOps Documentation is not that useful in its current form)
trigger:
- master
pool:
vmImage: 'ubuntu-latest'
steps:
- task: NodeTool#0
inputs:
versionSpec: '10.x'
displayName: 'Install Node.js'
- script: |
npm install
npm run build
displayName: 'npm install and build'
- task: CopyFiles#2
inputs:
Contents: 'build/**' # Pull the build directory (React)
TargetFolder: '$(Build.ArtifactStagingDirectory)'
- task: PublishBuildArtifacts#1
inputs:
pathtoPublish: $(Build.ArtifactStagingDirectory) # dist or build files
ArtifactName: 'www' # output artifact named www
# Default value: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
- task: ArchiveFiles#2
inputs:
rootFolderOrFile: '$(Build.ArtifactStagingDirectory)/build/'
includeRootFolder: false
- task: AzureWebApp#1
inputs:
azureSubscription: 'ReactJSRecipeAppConnection'
appName: 'ReactJSRecipeApp4'
package: '$(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip'
The full repository with simple react code including the YAML is here.
https://github.com/Jay-study-nildana/ReactJSRecipeApp

Deploy Nuxt.js SSR app to Azure App Service via Azure DevOps Pipelines

I'm having trouble deploying a SSR Nuxt.js app from Azure DevOps Pipelines to Azure App Service.
Disclaimers:
✅I have setup an Azure App Service with a Linux container.
✅I have added this to nuxt.config.js
server: {
host: '0.0.0.0'
}
In Azure DevOps Pipelines I copy over these files after the build:
.nuxt/**
static/**
package.json
nuxt.config.js
then I zip them up as package.zip
in the logs it shows it as successfully zipping the files but when I unzip the package none of the .nuxt files are present. I'm also confused that for some reason in the package.zip it puts all the build files in a folder named simply a/
You can see in the photo that it's creating the 'a' folder at the top and that it looks like all the files are being added to the archive.
When I unzip the package.zip the only files that are in there are:
* package.json
* nuxt.config.js
* /static
this is what my YAML file looks like:
trigger:
- master
pool:
vmImage: 'ubuntu-latest'
steps:
- task: NodeTool#0
inputs:
versionSpec: '12.x'
displayName: 'Install Node.js'
- script: |
cd src/Web.Nuxt
npm install
npm run build
displayName: 'npm install and build'
- task: CopyFiles#2
inputs:
SourceFolder: '$(Build.SourcesDirectory)/src/Web.Nuxt'
Contents: |
.nuxt/**
static/**
package.json
nuxt.config.js
TargetFolder: '$(Build.ArtifactStagingDirectory)'
- task: ArchiveFiles#2
inputs:
rootFolderOrFile: '$(Build.ArtifactStagingDirectory)'
archiveType: 'zip'
archiveFile: '$(Build.ArtifactStagingDirectory)/package.zip'
replaceExistingArchive: true
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)/package.zip'
ArtifactName: 'drop'
publishLocation: 'Container'
Any help would be greatly appreciated that can set me back on course to deploying a SSR Next.app with Azure DevOps Pipelines, Azure App Service, and a Nuxt SSR app. This has been tough to troubleshoot because in Azure DevOps Pipelines and Releases it says that everything was a success.
Nothing wrong with this process. The .nuxt folder is just hidden folder in Linux environment. It's hidden instead of missing ~
For Linux: The folder whose name starts with .(dot) is considered as one hidden folder. There's many discussions online about this behavior, you can try this one.
If you download that package.zip file and unzip it in Windows environment, you can easily find the .nuxt folder:

Can you copy files from one VSTS build agent to another?

Is it possible to copy files from one build agent to another and kick it off as a part of the pipeline task?
One build agent is Linux but I need to continue my work on Windows agent.
Following Hanna's solution, attached more detailed working solution:
Agent-1 and Agent-2 are two different machines from a different agent pool.
Agent-1 does 2 steps:
CopyFiles - writes the file 'livne.txt' (can be any pattern) from the default working directory into the artifact's staging directory
PublishPipelineArtifact - publishes an artifact called: 'PROJECT_NAME' contains all the files copied into the default working directory.
Agent-2 does one main task:
DownloadPipelineArtifact - downloads the file 'livne.txt' (can be any pattern) from the 'PROJECT_NAME' artifact into the current agent's working directory.
simple bash script to make sure 'livne.txt' indeed exists on the working directory.
pool:
name: Agent-1
- task: CopyFiles#2
displayName: 'Copy txt file'
inputs:
SourceFolder: '$(system.DefaultWorkingDirectory)'
Contents: livne.txt
TargetFolder: '$(build.ArtifactStagingDirectory)'
- task: PublishPipelineArtifact#1
displayName: 'Publish Pipeline Artifact'
inputs:
targetPath: '$(build.ArtifactStagingDirectory)'
artifact: 'PROJECT_NAME'
dependsOn: Job_1
pool:
name: Self Hosted Ubuntu for Docker Multiplatform
steps:
- task: DownloadPipelineArtifact#2
displayName: 'Download Pipeline Artifact'
inputs:
artifactName: 'PROJECT_NAME'
itemPattern: livne.txt
targetPath: '$(build.ArtifactStagingDirectory)'
- bash: |
ls $(build.ArtifactStagingDirectory)
cat $(build.ArtifactStagingDirectory)/livne.txt
displayName: 'Bash Script'
I think the best way to do this is typically to publish the files as artefacts of the pipeline and then download those artefacts again on the second agent. I have done this in projects before when one machine is consuming test results from the test agent to build reports.
You might imagine your pipeline would look something like this:
- job: Build
displayName: Build on Linux
steps:
...
- task: PublishPipelineArtifact#1
displayName: Publish Built binaries from Linux
inputs:
path: $(Build.SourcesDirectory)/bin/
artifact: Binaries
- job: Additional
displayName: Do something with the binaries on windows
steps:
- task: DownloadPipelineArtifact#2
inputs:
artifact: Binaries
targetPath: $(Pipeline.Workspace)/Binaries
...
I hope this helps! :)

PublishPipelineArtifact .artifact ignore not recognized

I am using Azure DevOps build pipeline to send a artifact over to a release pipeline. My build pipeline is as follows:
Install Node JS with NodeTool#0
Run a Script Command that runs npm install and then runs the test suite.
Post the code coverage results via PublishCodeCoverageResults#1
Publish artifact with PublishPipelineArtifact#1 and targetPath: '$(Pipeline.Workspace)' as the artifact directory.
When I originally did this without using the code coverage part, IE not installing node, the files sent over were 257 and was a little over 25MB. Completely workable and quick.
However when I installed node and ran code coverage the files exploded to 750MB, which is understandable - but is way too much to transfer as an artifact IMHO.
So, looking at a solution I found this: https://learn.microsoft.com/en-us/azure/devops/artifacts/reference/artifactignore?view=azure-devops
And, I added this into the root of the repo.
However, this file seems to have not made a difference using the PublishPipelineArtifact#1 at all. No matter what I did, all the files from the Workspace were used as the artifact.
What did work though, was physically deleting the files in the build process.
I realize that is a lot of detail for a question, but I would like to know if the .artifactignore file actually works with this build pipeline?
Is there a way I can use the artifact ignore file properly, and not worry about deleting files before creating the artifact?
Tried to change the target directory in the PublishPipelineArtifact#1 task.
Here is the current working YAML:
# Node.js with Angular
# Build a Node.js project that uses Angular.
# Add steps that analyze code, save build artifacts, deploy, and more:
# https://learn.microsoft.com/azure/devops/pipelines/languages/javascript
trigger:
- master
pool:
vmImage: 'ubuntu-latest'
steps:
- task: NodeTool#0
inputs:
versionSpec: '10.x'
displayName: 'Install Node.js'
- script: |
npm install
npm run test:devops
displayName: 'npm install and test'
- task: PublishCodeCoverageResults#1
displayName: 'publish ng code coverage results'
condition: succeededOrFailed()
inputs:
codeCoverageTool: 'Cobertura'
summaryFileLocation: 'coverage/cobertura-coverage.xml'
reportDirectory: coverage
failIfCoverageEmpty: true
- task: DeleteFiles#1
inputs:
SourceFolder: '$(Pipeline.Workspace)/s'
Contents: |
node_modules
coverage
- task: PublishPipelineArtifact#1
inputs:
targetPath: '$(Pipeline.Workspace)'
artifact: 'my-artifact'
The expected results would be a smaller artifact file that still posts code coverage and installs node.
The .artifactignore should be in the same directory as defined as target path of the publish pipelines artifact task (not only in the root of the repo like azure-pipelines.yml).
Is not in the docs and there is an issue about it here. also check here.
Ok, so I used the information from Shayki Abramczyk - which is the correct answer - but wanted to put a working yaml together for everyone. I had to copy my .artifactignore from the repo, which is: $(Pipeline.Workspace)/s to the root which is $(Pipeline.Workspace)
As soon as I did this - it showed results. As soon as I adjusted the artifact ignore file, the output reduced from 750+MB to 2.3MB and processed in 8 seconds.
Working YAML:
- task: NodeTool#0
inputs:
versionSpec: '10.x'
displayName: 'Install Node.js'
- script: |
npm install
npm run test
displayName: 'npm install and test'
- task: PublishCodeCoverageResults#1
displayName: 'publish ng code coverage results'
condition: succeededOrFailed()
inputs:
codeCoverageTool: 'Cobertura'
summaryFileLocation: 'coverage/cobertura-coverage.xml'
reportDirectory: coverage
failIfCoverageEmpty: true
- task: CopyFiles#2
inputs:
SourceFolder: '$(Pipeline.Workspace)/s'
Contents: '**/.artifactignore'
TargetFolder: '$(Pipeline.Workspace)'
- task: PublishPipelineArtifact#1
inputs:
targetPath: '$(Pipeline.Workspace)'
artifact: 'my-artifact'
FYI - this is being built for angular, so this is the artifact ignore file I used. Your mileage will vary based on your project.
**/dist
**/node_modules
**/coverage
**/.git