How do I add additional files to the Azure DevOps artifacts? - azure-devops

My Azure DevOps pipeline is successfully packing up my web service into a zip file with the following task:
- task: DotNetCoreCLI#2
displayName: Pack Artifacts
inputs:
command: 'publish'
publishWebProjects: false
arguments: '"Web Service" --configuration $(BuildConfiguration) --output $(Build.ArtifactStagingDirectory)'
This deposits the entire contents of the project in a zip file in 'drop\a.zip'. Now, I have a script (setup.ps1) that I want to run to open up the firewall and an SSL certificate that needs to be installed. I want to add them to the same zip file that was the target for the 'publish' operation. I tried this:
- task: CopyFiles#2
inputs:
SourceFolder: 'Additional Files'
Contents: '*'
TargetFolder: '$(Build.ArtifactStagingDirectory)'
flattenFolders: true
But it just added the files in the 'Additional Files' directory to the "\drop" directory alongside the a.zip file:
drop
a.zip
setup.ps1
www_mysite_com.pfx
How would I go about adding the additional files to the zip file that's created as part of the build and publish steps. This must be a common problem. What's the common solution?

You have three options:
unpack archive, copy file and pack again
add file to archive - like it is shown here
add your file to solution and make dotnet publish responsible for putting file in archive like it is shown here

Your artifacts directory should be
drop
a.zip
And now, you want to add setup.ps1 and www_mysite_com.pfx to the folder a, right?
YAML sample:
- task: DotNetCoreCLI#2
displayName: Pack Artifacts
inputs:
command: 'publish'
publishWebProjects: false
arguments: '"Web Service" --configuration $(BuildConfiguration) --output $(Build.ArtifactStagingDirectory)'
#Extract a.zip file to $(Build.ArtifactStagingDirectory)/drop/a folder
- task: ExtractFiles#1
inputs:
archiveFilePatterns: '$(Build.ArtifactStagingDirectory)/drop/a.zip'
destinationFolder: '$(Build.ArtifactStagingDirectory)/drop/a'
cleanDestinationFolder: false
overwriteExistingFiles: false
#Delete PrescQIPPWebApp.zip file
- task: PowerShell#2
inputs:
targetType: 'inline'
script: 'Remove-Item ''$(Build.ArtifactStagingDirectory)/drop/a.zip'''
#copy these files to the folder a
- task: CopyFiles#2
inputs:
SourceFolder: 'Additional Files'
Contents: |
setup.ps1
www_mysite_com.pfx
TargetFolder: '$(Build.ArtifactStagingDirectory)/drop/a'
OverWrite: true
flattenFolders: true
#Archive folder a to a.zip
- task: ArchiveFiles#2
inputs:
rootFolderOrFile: '$(Build.ArtifactStagingDirectory)/drop/a'
includeRootFolder: true
archiveType: 'zip'
archiveFile: '$(Build.ArtifactStagingDirectory)/drop/a.zip'
replaceExistingArchive: true
#Delete a folder
- task: PowerShell#2
inputs:
targetType: 'inline'
script: 'Remove-Item -path ''$(Build.ArtifactStagingDirectory)/drop/a'' -Recurse -Force -EA SilentlyContinue -Verbose'
#And now, we could the a.zip contain the file setup.ps1 and www_mysite_com.pfx.
We could also check this Thread Adding additional files to Azure Build Pipeline

Related

Azure DevOps: Publishing a file created on a previous task wont work

My pipeline looks like this:
- task: PythonScript#0
displayName: "Create Excel sheet"
inputs:
scriptSource: 'filePath'
failOnStderr: false
scriptPath: /create_excel.py
- task: CopyFiles#2
inputs:
contents: '_buildOutput/**'
targetFolder: $(Build.ArtifactStagingDirectory)
- task: PublishBuildArtifacts#1
inputs:
pathToPublish: $(Build.ArtifactStagingDirectory)
artifactName: MyBuildOutputs
And the last line of the python code that it runs looks like this, it returns an excel sheet:
def create_excel():
...
return rn_df.to_excel(fr'rn{rel_ver}.xlsx', sheet_name=f"RN {rel_ver}", index=False)
But I see nothing is copied nor uploaded. What is the problem?
From the YAML sample, the default working directory of the python task is $(build.sourcesdirectory).
When the task runs the python file, it will create the excel sheet in the working directory.
To solve this issue, you can change the $(Build.ArtifactStagingDirectory) to $(build.sourcesdirectory) in the next tasks.
For example:
- task: PythonScript#0
displayName: "Create Excel sheet"
inputs:
scriptSource: 'filePath'
failOnStderr: false
scriptPath: /create_excel.py
- task: CopyFiles#2
inputs:
contents: '_buildOutput/**'
targetFolder: $(build.sourcesdirectory)
- task: PublishBuildArtifacts#1
inputs:
pathToPublish: $(build.sourcesdirectory)
artifactName: MyBuildOutputs

Azure Pipeline Copy Secure file into build folder

I've a vite/svelte project which uses .env files for environment settings. I also have an Azure Pipeline which contains a secure file .env.staging this is on the .gitignore list of the associated repo. I'd like to download this secure file, copy it to my build directory and then have it's contents read when I run vite build --mode staging (well, npm run build:staging which includes vite build...)
When run locally from my machine npm run build:staging works as expected and reads the .env.staging file, however it seems to get ignored when used in the pipeline, am I doing anything wrong?
Here's my yml.
trigger:
- main
pool:
vmImage: 'ubuntu-latest'
steps:
- task: DownloadSecureFile#1
name: "dotenvStaging"
inputs:
secureFile: '.env.staging'
displayName: "Download .env.staging"
- task: NodeTool#0
inputs:
versionSpec: 14.15.4
displayName: "Install Node.JS"
- task: CopyFiles#2
inputs:
contents: "$(Agent.TempDirectory)/.env.staging"
targetFolder: "$(Agent.BuildDirectory)"
displayName: "Import .env.staging"
- script: npm install
displayName: "npm install"
- script: npm run build:staging
displayName: "npm run build:staging"
- task: ArchiveFiles#2
inputs:
rootFolderOrFile: 'dist'
archiveType: 'zip'
archiveFile: '$(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip'
#replaceExistingArchive: true
#verbose: # Optional
#quiet: # Optional
displayName: "Create archive"
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip'
ArtifactName: 'drop'
publishLocation: 'Container'
displayName: "Publish archive"
I'm not sure if CopyFiles#2 is doing what I expect or not as it just matches the content parameter to copy whatever files match, which could be 0 if I'm writing it wrong...
Another note, I also tried using $(dotenvStaging.secureFilePath) as the content parameter, but it doesn't seem to do anything either.
Naturally I figured it out as soon as I posted, I needed to update the CopyFiles part to specify sourceFolder, clearly it didn't like my absolute file path for content.
- task: CopyFiles#2
inputs:
sourceFolder: "$(Agent.TempDirectory)"
contents: ".env.staging"
targetFolder: "$(Agent.BuildDirectory)"
displayName: "Import .env.staging"

How to copy specific folders/files to artifact?

I have the following build pipeline:
pool:
name: Azure Pipelines
demands:
- npm
- msbuild
steps:
- task: Npm#1
displayName: 'npm install'
inputs:
workingDir: Project123/Angular
verbose: false
steps:
- task: Npm#1
displayName: 'npm custom: angular build'
inputs:
command: custom
workingDir: Project123/Angular
verbose: false
customCommand: 'run-script build --prod --extractCss'
steps:
- task: NuGetCommand#2
displayName: 'NuGet restore'
steps:
- task: MSBuild#1
displayName: '.Net build'
inputs:
solution: 'Project123/*.csproj'
msbuildArchitecture: x64
configuration: Release
msbuildArguments: '/p:OutputPath=$(Build.ArtifactStagingDirectory)'
- task: CopyFiles#2
displayName: 'Copy Files to: $(Build.ArtifactStagingDirectory)'
inputs:
SourceFolder: Project123/Bundles
TargetFolder: '$(Build.ArtifactStagingDirectory)'
- task: PublishBuildArtifacts#1
displayName: 'Publish Artifact: Release'
inputs:
ArtifactName: Release
When we build the project from VS manually, this is the (expected) artifact we deploy:
However, the YAML I have generates this artifact instead:
How do I accomplish the expected artifact?
I am actually surprised that AngularOutput was copied in the root of the artifact, and not Bundles...I specified in the copy task to copy the Bundles folder , which would contain the AngularOutput...
Since I do not know the .csproj configuration, just provide a workaround based on the screenshot.
Don't need this Angular folder in this artifact
These Web.Debug and Web.Release are unneeded in the artifact
Add task power shell and delete the folder and files via below script
#Delete Angular folder and files
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
Remove-Item ''$(Build.ArtifactStagingDirectory)/_PublishedWebsites/Project123/Angular''
Remove-Item ''$(Build.ArtifactStagingDirectory)/_PublishedWebsites/Project123/Web.Debug.configure''
Remove-Item ''$(Build.ArtifactStagingDirectory)/_PublishedWebsites/Project123/Web.Release.configure''
Those dlls, .pdb, .xml and .configure files are not needed here.
There needs to be a 'Bundles' directory here which is generated by the angular build task
Check this Copy files task, these files in the folder Project123/Bundles, right?
- task: CopyFiles#2
displayName: 'Copy Files to: $(Build.ArtifactStagingDirectory)'
inputs:
SourceFolder: Project123/Bundles
TargetFolder: '$(Build.ArtifactStagingDirectory)'
We need to change the Copy file task as below:
- task: CopyFiles#2
displayName: 'Copy Files to: $(Build.ArtifactStagingDirectory)'
inputs:
SourceFolder: Project123/Bundles
TargetFolder: '$(Build.ArtifactStagingDirectory)/_PublishedWebsites/Project123/Bundles'
It will save these files in the folder Bundles instead of root.
This AngularOutput folder needs to be located under 'Bundles' directory inside 'Project123' folder above
We could copy the folder to project123 folder and then publish the artifact.
- task: CopyFiles#2
inputs:
SourceFolder: '$(Build.ArtifactStagingDirectory)/AngularOutput'
Contents: '**'
TargetFolder: '$(Build.ArtifactStagingDirectory)/_PublishedWebsites/Project123/AngularOutput'
You could run below YAML build and check the result.
pool:
name: Azure Pipelines
demands:
- npm
- msbuild
steps:
- task: Npm#1
displayName: 'npm install'
inputs:
workingDir: Project123/Angular
verbose: false
- task: Npm#1
displayName: 'npm custom: angular build'
inputs:
command: custom
workingDir: Project123/Angular
verbose: false
customCommand: 'run-script build --prod --extractCss'
- task: NuGetCommand#2
displayName: 'NuGet restore'
- task: MSBuild#1
displayName: '.Net build'
inputs:
solution: 'Project123/*.csproj'
msbuildArchitecture: x64
configuration: Release
msbuildArguments: '/p:OutputPath=$(Build.ArtifactStagingDirectory)'
- task: CopyFiles#2
displayName: 'Copy Files to: $(Build.ArtifactStagingDirectory)'
inputs:
SourceFolder: Project123/Bundles
TargetFolder: '$(Build.ArtifactStagingDirectory)/_PublishedWebsites/Project123/Bundles'
#Delete Angular folder and files
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
Remove-Item ''$(Build.ArtifactStagingDirectory)/_PublishedWebsites/Project123/Angular''
Remove-Item ''$(Build.ArtifactStagingDirectory)/_PublishedWebsites/Project123/Web.Debug.configure''
Remove-Item ''$(Build.ArtifactStagingDirectory)/_PublishedWebsites/Project123/Web.Release.configure''
- task: CopyFiles#2
inputs:
SourceFolder: '$(Build.ArtifactStagingDirectory)/AngularOutput'
Contents: '**'
TargetFolder: '$(Build.ArtifactStagingDirectory)/_PublishedWebsites/Project123/AngularOutput'
- task: PublishBuildArtifacts#1
displayName: 'Publish Artifact: Release'
inputs:
ArtifactName: Release
Update1
If the issue is delete the folder and files, please update the power shell script as below and try it again.
#Delete Angular folder and files
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
Remove-Item -Path $(Build.ArtifactStagingDirectory)/_PublishedWebsites/Project123/Angular -Recurse -Force
Remove-Item -Path $(Build.ArtifactStagingDirectory)/_PublishedWebsites/Project123/Web.Debug.configure -Recurse -Force
Remove-Item -Path $(Build.ArtifactStagingDirectory)/_PublishedWebsites/Project123/Web.Release.configure -Recurse -Force
And the test result:

Adding If condition in Yaml pipeline for a copy task

In Azure Devops pipeline, I have a Copy task like below which copies the file in 3 different folder as it run for 3 different agent(MAC, LINUX and WINDOWS). I want the below Copy task to be run Only for windows . How can I do it as I don't want the below file in MAC and Linux Folder.
- task: CopyFiles#2
displayName: 'Copy ONNX to bin folder'
inputs:
SourceFolder: '$(Build.SourcesDirectory)/packages/ONNXRuntime/runtimes/win-x64/native'
Contents: onnxruntime.dll
TargetFolder: '$(Build.SourcesDirectory)/ProjectName_x64-$(osSuffix)/bin'
flattenFolders: true
you can use a condition parameter to drive the task at runtime:
- task: CopyFiles#2
displayName: 'Copy ONNX to bin folder'
condition: $[eq(variables['osSuffix'], 'WINDOWS')]
inputs:
SourceFolder: '$(Build.SourcesDirectory)/packages/ONNXRuntime/runtimes/win-x64/native'
Contents: onnxruntime.dll
TargetFolder: '$(Build.SourcesDirectory)/ProjectName_x64-$(osSuffix)/bin'
flattenFolders: true

How do you prevent FtpUpload#2 from creating extra root folders on the ftp server when using Azure DevOps?

FtpUpload#2 is creating an extra folder on my ftp server when it uploads. It creates a folder with the name of the project instead of creating just the root file structure
e.g. instead of creating /wwwroot/... its creating /projectname/wwwroot/... on the ftp server. So when I look in the folder I see (image below) a folder with the project name, where I was expecting to see a wwwroot folder and some files.
So for example I end up with
http://domain/projectname/page.html
When I actually want
http://domain/page.html
The directory tree underneath is correct but I don't need/want the extra folder.
What settings can I use to stop this, please? My YAML is below
- task: DotNetCoreCLI#2
displayName: Publish
inputs:
command: publish
publishWebProjects: True
arguments: '--configuration $(BuildConfiguration) --output $(build.artifactstagingdirectory)'
zipAfterPublish: false
- task: FtpUpload#2
displayName: 'FTP Upload'
inputs:
credentialsOption: inputs
serverUrl: 'ftp://ftp.xxxxxxx.com'
username: 'xxxxxxxxx'
password: 'xxxxxxxxx'
remoteDirectory: '/'
trustSSL: true
cleanContents: true
rootDirectory: $(Build.artifactstagingdirectory)
preservePaths: true
I cannot reproduce above issue. The FtpUpload task works fine for me. It looks like the wwwroot folder doesnot reside directly in the $(Build.artifactstagingdirectory) folder. It is most likely its path is $(Build.artifactstagingdirectory)/projectname/wwwroot.
Or you might mistakenly set the physical path for your ftp server to include the projectname folder like this c:\site\projectname
You can add a powershell task to ls your $(Build.artifactstagingdirectory) directory to check your $(Build.artifactstagingdirectory) folder structure. And change the rootDirectory field of FtpUpload task accordingly.
- powershell: |
ls $(Build.artifactstagingdirectory)
I had a test on my pipeline. It worked all fine. See below:
1, Below is the folder structure of the $(Build.artifactstagingdirectory)
2, My FtpUpload task has the exactly same configuration.
- task: FtpUpload#2
displayName: 'FTP Upload'
inputs:
credentialsOption: inputs
serverUrl: '..'
username: '..'
password: '..'
rootDirectory: '$(Build.artifactstagingdirectory)'
remoteDirectory: /
cleanContents: true
preservePaths: true
3, See below result screenshot. No extract folder is created.
Update:
I found you used dotnet publish task to publish your app. There is a modifyOutputPath field for dotnet publish task. Its default value is true which will have the project file name prefixed to their folder names when output path is specified explicitly in arguments. See here.
You can either change the dotnet publish task to disable the modifyOutputPath field modifyOutputPath: false,
- task: DotNetCoreCLI#2
displayName: Publish
inputs:
command: publish
publishWebProjects: True
arguments: '--configuration $(BuildConfiguration) --output $(build.artifactstagingdirectory)'
zipAfterPublish: false
modifyOutputPath: false #add this field
- task: FtpUpload#2
displayName: 'FTP Upload'
inputs:
....
cleanContents: true
rootDirectory: $(Build.artifactstagingdirectory)
preservePaths: true
Or change the rootDirectory field for FTPupload task to $(Build.artifactstagingdirectory)/projectname
- task: DotNetCoreCLI#2
displayName: Publish
inputs:
command: publish
publishWebProjects: True
arguments: '--configuration $(BuildConfiguration) --output $(build.artifactstagingdirectory)'
zipAfterPublish: false
- task: FtpUpload#2
displayName: 'FTP Upload'
inputs:
....
cleanContents: true
rootDirectory: $(Build.artifactstagingdirectory)/projectname
preservePaths: true