I have quite a bit of experience with Azure DevOps on-premise and know where the build artifacts are stored when I run a build pipeline. However, we are moving to Microsoft-hosted build agents, and won't have any on-prem build server to store the artifacts from the build.
My question is, how do I get the build artifacts created from a build pipeline processed by a Microsoft-hosted build agent? Ultimately, I would like to download those artifacts to a file share that we have on-prem. Is this something that can be done?
I created a build pipeline in Azure DevOps (not on-prem) and ran the build. I added the marketplace extension Publish Build Artifact to the build pipeline and I expected that an artifact would be published so that I could download it to our company server. No artifact was produced.
In Azure yaml based pipelines, you can either use the pipeline artifacts method in this the artifacts are stored and associated with the pipeline or the universal method required additional steps to setup that uses the internal feed to store the artifacts. For pipeline artifacts you necessarily don't need to version the artifact since it is contained inside the pipeline. For universal unless you are overwriting the artifact you need to uniquely version them by any version mechanism you want to use, either to leverage off from app source code versioning or your own custom versioning.. like timestamp etc.
Below is an example of an dot. net build with both publish methods done in yaml pipeline.
# To run and store your build
- task: DotNetCoreCLI#2
displayName: "Build Project"
enabled: true
inputs:
command: "build"
projects: $(SolutionPath)
arguments: "--configuration Release --output $(Build.ArtifactStagingDirectory)"
# Here we start the postbuild steps
# This is to archive the build artifacts, stored by file number.
- task: ArchiveFiles#2
displayName: "Archive Artifacts"
inputs:
rootFolderOrFile: "$(Build.ArtifactStagingDirectory)"
includeRootFolder: false
archiveType: "zip"
archiveFile: "$(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip"
replaceExistingArchive: true
# This is to publish the artifact inside the pipeline, stored by file number.
- task: PublishBuildArtifacts#1
displayName: "Publish Artifacts"
inputs:
PathtoPublish: "$(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip"
ArtifactName: "drop"
publishLocation: "Container"
# This is to publish the artifact inside the universal artifacts feeds, stored by date/timestamp variable passed below as filename from another step.
- task: UniversalPackages#0
inputs:
command: 'publish'
publishDirectory: '$(Build.ArtifactStagingDirectory)'
feedsToUsePublish: 'internal'
vstsFeedPublish: 'your feed'
vstsFeedPackagePublish: 'azure-pipeline-dotnetapi'
versionOption: 'custom'
versionPublish: '$(setBuildValues.ApplicationVersion)'
packagePublishDescription: '$(setBuildValues.appVersion)'
For download of artifacts to a path you can use the below task and pass your network location as the path for download, i am not sure if it can use UNC paths might have to map the network path.
steps:
- task: DownloadBuildArtifacts#1
displayName: 'Download Build Artifacts'
inputs:
buildType: "current"
downloadType: "single"
downloadPath: ${{ parameters.DownloadPath }} #INSERT YOUR DOWNLOAD PATH HERE.
Related
I have tried 2 different attempts to reference the correct pipeline when downloading specific artifacts via two different built in steps. and cannot seem to get it to work.
this first way, will error in the oddest way i have ever seen. the pipeline will show 'pending' when selecting the run itself, but when going back one level to view the pipeline as a whole it will show a red x as if the run has failed, but when trying to dig into it, i can get no output.
#attempt 1
- download: $(PIPELINE_NAME)
and. this second way will successfully download the package in <1sec.... meaning it isnt actually downloading anything. the next step will fail as no package is found.
- task: DownloadPipelineArtifact#2
name:
displayName: 'Download artifact test'
inputs:
buildtype: specific
project: <project name here>
pipeline: '$(PIPELINE_NAME)' #this just doesnt work for some reason
runVersion: specific
runId: $(resources.pipeline.$(PIPELINE_NAME).runID)
downloadPath: $(Pipeline.Workspace)
Is this just not meant to work? neither of these will resolve the variables and will fail. Anyone have any other suggestions? do these NEED to be hardcoded? or am i missing some syntax? We have several nearly identical micro-services that i would like to just share the same template between them if i could.
According to the arguments document of Download Pipeline Artifacts task, the value of pipeline parameter should be the definition ID of the build pipeline, not the source name. So this may be the reason that your parameter doesn't work.
To download artifact from another pipeline, you don't need to specify pipeline resources. This is my method of downloading artifacts from another pipeline:
Firstly, in my previous pipeline, I have a Publish pipeline artifact task:
- task: PublishPipelineArtifact#1
inputs:
targetPath: '$(Build.ArtifactStagingDirectory)'
artifact: 'drop'
publishLocation: 'pipeline'
And then, in another pipeline, I have a Download Pipeline Artifacts task:
- task: DownloadPipelineArtifact#2
inputs:
buildType: 'specific'
project: '$(project)'
definition: '$(pipelineID)'
buildVersionToDownload: 'specific'
pipelineId: '$(buildID)'
artifactName: 'drop'
targetPath: '$(Pipeline.Workspace)'
Tip:
You can click on "settings" above the task and fill in the information with UI if you don't need to use variables. UI will produce picklists of projects, pipelines, ... that you can choose.
Need to create a YAML based azure build pipeline i.e, need to run particular tasks like only build step when it is PR automated and when the same pipeline manually run it should run build task along with archive and publish artifact tasks
You can distinguish with the Build.Reason https://learn.microsoft.com/en-US/azure/devops/pipelines/build/variables?view=azure-devops&tabs=yaml#build-variables-devops-services in the yaml pipeline why a build was triggered. This you might set as condition in your build stages/jobs/steps.
See the following example from one of our build pipelines:
- task: DotNetCoreCLI#2
displayName: "Publish NuGet"
condition: and(succeeded(), ne(variables['Build.Reason'], 'Schedule'))
inputs:
command: 'push'
searchPatternPush: '$(Build.SourcesDirectory)/source/**/*.nupkg'
nuGetFeedType: 'internal'
feedPublish: 'MyFeed'
So I am trying to copy file and publish it via Azure yaml pipeline task PublishBuildArtifacts#1 so that I can use that file in other pipeline by downloading build artifact.
- script: |
cat ./pipelines/config/abc.config
mkdir $(Build.ArtifactStagingDirectory)/abc-config
cp -R ./pipelines/config/abc.config $(Build.ArtifactStagingDirectory)/abc-config
displayName: "Archiving runtime config file"
# Publish Config Build Artifacts
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: "$(Build.ArtifactStagingDirectory)/abc-config/"
ArtifactName: "abc-config"
publishLocation: "Container"
when I use the task DownloadBuildArtifacts#0 and download that file to a specific path, it supposed to store that file to ./pipelines/config/abc.config
but when I type ls to see if I get this file, for some reason I couldn't able to get that file.
- task: DownloadBuildArtifacts#0
inputs:
buildType: 'current'
project: '$(System.TeamProjectId)'
pipeline: '$(System.DefinitionId)'
downloadType: 'single'
artifactName: 'abc-config'
downloadPath: '$(System.ArtifactsDirectory)/pipelines/config'
- bash: |
# input file and remove empty lines and/or lines with crlf
ls ./pipelines/config
pipeline Logs:
logs show that it's trying to download that file by adding artifact name before that file. I don't understand this behavior of adding artifact name before the file name.
Downloading abc-config/runtime.config to /home/vsts/work/1/a/pipelines/abc-config/runtime.config
Downloaded abc-config/runtime.config to /home/vsts/work/1/a/pipelines/abc-config/runtime.config
What should I need to do to store that file to pipelines/config/runtime.config?
Documentation of task: DownloadBuildArtifacts#0 doesn't mention anything about this behavior. so I am not sure what am I missing here. Any help would be highly appreciated.
Thank you.
When you publish build artifacts, the artifacts will be saved in the artifactName folder. So when you download them using DownloadBuildArtifacts task. The artifactName folder and all the artifacts contents will be downloaded. This is the default behavior.
As a workaround, you can change the artifactName to config for PublishBuildArtifacts task. See below:
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: "$(Build.ArtifactStagingDirectory)/abc-config/"
ArtifactName: "config"
publishLocation: "Container"
Then change the downloadPath to $(System.ArtifactsDirectory)/pipelines for DownloadBuildArtifacts task. See below:
- task: DownloadBuildArtifacts#0
inputs:
buildType: 'current'
project: '$(System.TeamProjectId)'
pipeline: '$(System.DefinitionId)'
downloadType: 'single'
artifactName: 'config'
downloadPath: '$(System.ArtifactsDirectory)/pipelines'
The following code is for archive, but how to upload the archived tar.bz2 to a universal packages from Azure Artifacts
- task: ArchiveFiles#2
inputs:
#rootFolderOrFile: '$(Build.BinariesDirectory)'
#includeRootFolder: true
#archiveType: 'tar'
#tarCompression: 'bz2'
#archiveFile: '$(Build.ArtifactStagingDirectory)/$(fileName)_$(Build.BuildId).tar.bz2'
#replaceExistingArchive: true
#verbose: # Optional
#quiet: # Optional
To publish a Universal Package to your feed, add the following snippet to your azure-pipelines.yml file.
- task: UniversalPackages#0
displayName: Universal Publish
inputs:
command: publish
publishDirectory: '$(Build.ArtifactStagingDirectory)'
vstsFeedPublish: '<projectName>/<feedName>'
vstsFeedPackagePublish: '<Package name>'
versionOption: custom
versionPublish: '<Package version>'
packagePublishDescription: '<Package description>'
By default, the Universal Packages task will publish all files in$(Build.ArtifactStagingDirectory). To prepare your Universal Package for publishing, either configure preceding tasks to place output files in that directory, or use the Copy Files utility task to assemble the files that you want to publish.
For details ,please refer to this official document.
We're in the process of moving our product to an azure web app. We have an extensive existing pipeline containing multiple parallel jobs. One of these jobs compiles a asp.net web application. Some others compile a vue.js website. Currently, the results of the web application and the vue projects are combined in a separate stage, using a powershell script.
Now I can convert the publish step of the web application to generate a deployment package for azure. But what is the best way of also adding the vue outputs into this package so I can deploy it to azure correctly, without losing the parallel jobs? I cannot include the is output files in my project, because they don't exist within the web application build job
You can use publish build artifact task to upload the build results of the web application and vue projects to azure devops server as #Krzysztof mentioned. And you can add a new job to download the artifacts.
Please check below simple example in yaml.
In order to combine the build results, you can use extract file task to extract the zipped artifacts and published the unpacked artifacts in Build_web job. And in the Combine job you can use copy file task to copy the results of vue artifacts to the web artifacts folder. And then you can use archive file task to pack the artifacts which now contains the results of web and vue application.
Combine job should dependsOn Build_web and Build_vue jobs
jobs:
- job: Build_Web
pool:
vmImage: "windows-latest"
steps:
- task: ExtractFiles#1
inputs:
archiveFilePatterns: '*.zip'
destinationFolder: '$(Build.ArtifactStagingDirectory)\unzip'
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)\unzip'
artifactName: webapp
- job: Build_Vue
pool:
vmImage: "windows-latest"
steps:
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: 'path to the build results'
artifactName: vueapp
- job: Combine
dependsOn:
- Build_Web
- Build_Vue
pool:
vmImage: "windows-latest"
steps:
- task: DownloadBuildArtifacts#0
inputs:
buildType: 'current'
artifactName: webapp
downloadPath: "$(System.ArtifactsDirectory)"
- task: DownloadBuildArtifacts#0
inputs:
buildType: 'current'
artifactName: vueapp
downloadPath: "$(System.ArtifactsDirectory)"
- task: CopyFiles#2
inputs:
SourceFolder: '$(System.ArtifactsDirectory)\vueapp'
TargetFolder: 'path to web application result folder' #eg. $(System.ArtifactsDirectory)\webapp\Content\d_C\a\1\s\AboutSite\AboutSite\obj\Release\netcoreapp2.0\PubTmp\Out\
- task: ArchiveFiles#2
inputs:
rootFolderOrFile: $(System.ArtifactsDirectory)\webapp
archiveType: 'zip'
archiveFile: '$(Build.ArtifactStagingDirectory)/webapplication.zip'
Above example only shows a general idea. You can aslo move the ExtractFile task to Combine job. In either way, you will have to use extract file, copy file and archive file task.
For TargetFolder parameter in copy file task, you can check the download build artifacts log for webapp artifact to get the full path. For example as below screenshot shows.
You can use PublishPipelineArtifact#1 to create artifacts for your projects and then in a separate job DownloadPipelineArtifact#2. By defining path you may compose your final artifact (if this mixing many projects is not more complicated than putting one inside another). And publish your artifact as build or pipeline artifact depending how you have roganized your release.
# Download an artifact named 'WebApp' to 'bin' in $(Build.SourcesDirectory)
- task: DownloadPipelineArtifact#2
inputs:
artifact: 'WebApp'
path: $(Build.SourcesDirectory)/bin
Here you have more info about publishing and downloding artifacts.