Azure SQL - DacPac Deployment - azure-devops

I am currently trying to test my DacPac deplyoment pipelines. However, I get error message all the time:
##[error]No files were found to deploy with search pattern C:\agent\_work\5\a\sqlproj_artifacts_1\DWH\DWH\bin\Release\database.dacpacCheck out how to troubleshoot failures at
I can't find what the problem is. Here the code, idea why it does not work? Have done before and worked perfectly....
steps:
- task: VSBuild#1
inputs:
solution: '$(solution)'
platform: '$(buildPlatform)'
configuration: '$(buildConfiguration)'
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: '$(Build.SourcesDirectory)'
ArtifactName: 'sqlproj_artifacts_$(System.JobAttempt)'
publishLocation: 'Container'
steps:
- task: DownloadBuildArtifacts#0
inputs:
buildType: 'current'
downloadType: 'single'
artifactName: 'sqlproj_artifacts_$(System.JobAttempt)'
downloadPath: '$(System.ArtifactsDirectory)'
deployType: 'DacpacTask'
DeploymentAction: 'Publish'
DacpacFile: '$(System.ArtifactsDirectory)/sqlproj_artifacts_$(System.JobAttempt)/DWH/DWH/bin/Release/$(azureSqlDBName).dacpac'
I'm using AuthenticationType: 'server'
And this is the local path where project is:
source\repos\DataWarehouse\DWH\DWH
Tried to adjust the path, but it still doesn't work.
Hope you can help me, thanks!

You can check the path of the artifact you are looking for on the build screen when you click on artifacts:
Here the $(azureSqlDBName).dacpac file name looks suspicious to me, as in if the dacpac is produced from a sql project, then probably the file will have the project name, not the azure sql db name.
In addition to this, I also would try to publish separate build artifacts for separate deployment, so instead of creating an artifact for the whole source artifact directory, I would copy the dacpac from the source to staging directory and then publish the dacpac artifact:
- task: CopyFiles#2
displayName: 'Copy DacPac Files'
inputs:
SourceFolder: '$(Build.SourcesDirectory)'
Contents: '**/*.dacpac'
TargetFolder: '$(Build.ArtifactStagingDirectory)/db'
CleanTargetFolder: true
OverWrite: true
flattenFolders: true
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)/db'
ArtifactName: 'sqlproj_artifacts_$(System.JobAttempt)'
publishLocation: 'Container'
Then it is easier to find what you are looking for when you need to use the artifact. Please note the flattenFolders flag which comes in handy when you are dealing with a single package within the artifact.
One more thing is the use of pipeline artifacts instead of build artifacts, since you are already working with yaml pipelines, and possibly multi-stage yamls, you can utilize pipeline artifacts which are automatically downloaded to the default workspace folder.
Hope it helps,

Related

Get Build Artifacts from Azure DevOps Microsoft-hosted Build Pipeline

I have quite a bit of experience with Azure DevOps on-premise and know where the build artifacts are stored when I run a build pipeline. However, we are moving to Microsoft-hosted build agents, and won't have any on-prem build server to store the artifacts from the build.
My question is, how do I get the build artifacts created from a build pipeline processed by a Microsoft-hosted build agent? Ultimately, I would like to download those artifacts to a file share that we have on-prem. Is this something that can be done?
I created a build pipeline in Azure DevOps (not on-prem) and ran the build. I added the marketplace extension Publish Build Artifact to the build pipeline and I expected that an artifact would be published so that I could download it to our company server. No artifact was produced.
In Azure yaml based pipelines, you can either use the pipeline artifacts method in this the artifacts are stored and associated with the pipeline or the universal method required additional steps to setup that uses the internal feed to store the artifacts. For pipeline artifacts you necessarily don't need to version the artifact since it is contained inside the pipeline. For universal unless you are overwriting the artifact you need to uniquely version them by any version mechanism you want to use, either to leverage off from app source code versioning or your own custom versioning.. like timestamp etc.
Below is an example of an dot. net build with both publish methods done in yaml pipeline.
# To run and store your build
- task: DotNetCoreCLI#2
displayName: "Build Project"
enabled: true
inputs:
command: "build"
projects: $(SolutionPath)
arguments: "--configuration Release --output $(Build.ArtifactStagingDirectory)"
# Here we start the postbuild steps
# This is to archive the build artifacts, stored by file number.
- task: ArchiveFiles#2
displayName: "Archive Artifacts"
inputs:
rootFolderOrFile: "$(Build.ArtifactStagingDirectory)"
includeRootFolder: false
archiveType: "zip"
archiveFile: "$(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip"
replaceExistingArchive: true
# This is to publish the artifact inside the pipeline, stored by file number.
- task: PublishBuildArtifacts#1
displayName: "Publish Artifacts"
inputs:
PathtoPublish: "$(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip"
ArtifactName: "drop"
publishLocation: "Container"
# This is to publish the artifact inside the universal artifacts feeds, stored by date/timestamp variable passed below as filename from another step.
- task: UniversalPackages#0
inputs:
command: 'publish'
publishDirectory: '$(Build.ArtifactStagingDirectory)'
feedsToUsePublish: 'internal'
vstsFeedPublish: 'your feed'
vstsFeedPackagePublish: 'azure-pipeline-dotnetapi'
versionOption: 'custom'
versionPublish: '$(setBuildValues.ApplicationVersion)'
packagePublishDescription: '$(setBuildValues.appVersion)'
For download of artifacts to a path you can use the below task and pass your network location as the path for download, i am not sure if it can use UNC paths might have to map the network path.
steps:
- task: DownloadBuildArtifacts#1
displayName: 'Download Build Artifacts'
inputs:
buildType: "current"
downloadType: "single"
downloadPath: ${{ parameters.DownloadPath }} #INSERT YOUR DOWNLOAD PATH HERE.

Azure publish and download build artifact yaml task not working

So I am trying to copy file and publish it via Azure yaml pipeline task PublishBuildArtifacts#1 so that I can use that file in other pipeline by downloading build artifact.
- script: |
cat ./pipelines/config/abc.config
mkdir $(Build.ArtifactStagingDirectory)/abc-config
cp -R ./pipelines/config/abc.config $(Build.ArtifactStagingDirectory)/abc-config
displayName: "Archiving runtime config file"
# Publish Config Build Artifacts
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: "$(Build.ArtifactStagingDirectory)/abc-config/"
ArtifactName: "abc-config"
publishLocation: "Container"
when I use the task DownloadBuildArtifacts#0 and download that file to a specific path, it supposed to store that file to ./pipelines/config/abc.config
but when I type ls to see if I get this file, for some reason I couldn't able to get that file.
- task: DownloadBuildArtifacts#0
inputs:
buildType: 'current'
project: '$(System.TeamProjectId)'
pipeline: '$(System.DefinitionId)'
downloadType: 'single'
artifactName: 'abc-config'
downloadPath: '$(System.ArtifactsDirectory)/pipelines/config'
- bash: |
# input file and remove empty lines and/or lines with crlf
ls ./pipelines/config
pipeline Logs:
logs show that it's trying to download that file by adding artifact name before that file. I don't understand this behavior of adding artifact name before the file name.
Downloading abc-config/runtime.config to /home/vsts/work/1/a/pipelines/abc-config/runtime.config
Downloaded abc-config/runtime.config to /home/vsts/work/1/a/pipelines/abc-config/runtime.config
What should I need to do to store that file to pipelines/config/runtime.config?
Documentation of task: DownloadBuildArtifacts#0 doesn't mention anything about this behavior. so I am not sure what am I missing here. Any help would be highly appreciated.
Thank you.
When you publish build artifacts, the artifacts will be saved in the artifactName folder. So when you download them using DownloadBuildArtifacts task. The artifactName folder and all the artifacts contents will be downloaded. This is the default behavior.
As a workaround, you can change the artifactName to config for PublishBuildArtifacts task. See below:
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: "$(Build.ArtifactStagingDirectory)/abc-config/"
ArtifactName: "config"
publishLocation: "Container"
Then change the downloadPath to $(System.ArtifactsDirectory)/pipelines for DownloadBuildArtifacts task. See below:
- task: DownloadBuildArtifacts#0
inputs:
buildType: 'current'
project: '$(System.TeamProjectId)'
pipeline: '$(System.DefinitionId)'
downloadType: 'single'
artifactName: 'config'
downloadPath: '$(System.ArtifactsDirectory)/pipelines'

Azure Pipeline - Build one project from solution with project dependencies

I have a solution that has a few projects.
App.Client.Website
App.Client.WindowsService
App.Client.Database
App.Client.Connectors
App.Shared
The App.Client.Database and App.Client.Connectors are both dependencies of App.Client.Website and App.Client.WindowsService, and App.Shared is a dependency of all the projects.
I am wanting to create 2 pipelines: One for the website and one for the windows service.
The issue that I have is that when I set my solution to be App.Client.Website/*.csproj, it can't find any of my project references to build - which makes a lot of sense to me, but I would obviously like to have it build my project references as well.
I tried solving this by creating pipelines for each of the dependency projects that would pack a nuget package and then I would reference that nuget packages from the feed in my website and window service projects, but this stopped me from being able to debug the code and make changes in those project on the fly.
DevOps is something that I am only just starting to pick up. I have gotten to the point where I am needing to add build numbers to my website application and wanting to stop manually incrementing patch numbers. I am using YAML instead of the Classic pipeline builder.
Any and all help is appreciated.
YAML
Here is the YAML that I am using. It is just the basic YAML with the solution targeting my Website project.
# ASP.NET
# Build and test ASP.NET projects.
# Add steps that publish symbols, save build artifacts, deploy, and more:
# https://learn.microsoft.com/azure/devops/pipelines/apps/aspnet/build-aspnet-4
trigger:
- master
pool:
vmImage: 'windows-latest'
variables:
solution: '**/*.Client.Website/*.csproj'
buildPlatform: 'Any CPU'
buildConfiguration: 'Release'
steps:
- task: NuGetToolInstaller#1
- task: NuGetCommand#2
inputs:
command: 'restore'
restoreSolution: '$(solution)'
feedsToUse: 'config'
nugetConfigPath: 'Pipelines/nuget.config'
externalFeedCredentials: 'External Feed'
- task: VSBuild#1
inputs:
solution: '$(solution)'
msbuildArgs: '/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:PackageLocation="$(build.artifactStagingDirectory)"'
platform: '$(buildPlatform)'
configuration: '$(buildConfiguration)'
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)'
ArtifactName: 'drop'
publishLocation: 'Container'
The repo folder structure has the project folders at the root of repo.
My project references in my Website cs project file are as follows:
<ItemGroup>
<ProjectReference Include="..\App.Client.Database\App.Client.Database.csproj" />
<ProjectReference Include="..\App.Client.Connectors\App.Client.Connectors.csproj" />
<ProjectReference Include="..\App.Shared\App.Shared.csproj" />
</ItemGroup>
Here is an error from the build task:
##[error]C:\Program Files\dotnet\sdk\3.1.403\Sdks\Microsoft.NET.Sdk\targets\Microsoft.PackageDependencyResolution.targets(241,5): Error NETSDK1004: Assets file 'D:\a\1\s\App.Shared\obj\project.assets.json' not found. Run a NuGet package restore to generate this file.
The issue is related to the NuGet Restore task, We could try to restore the .sln file to restore all NuGet packages, and then build the .csproj file. It should work.

DevOp Pipeline: Error: More than one package matched with specified pattern: D:\a\r1\a\**\*.zip

##[error]Error: More than one package matched with specified pattern: D:\a\r1\a***.zip. Please restrain the search pattern.
I am new to DevOp. I am not quite know how to configure the yaml in the pipeline. By default, there was a template created and i added archives and publish task inside like this.
trigger:
- master
pool:
vmImage: 'windows-latest'
variables:
solution: '**/*.sln'
buildPlatform: 'Any CPU'
buildConfiguration: 'Release'
steps:
- task: NuGetToolInstaller#1
- task: NuGetCommand#2
inputs:
restoreSolution: '$(solution)'
- task: VSBuild#1
inputs:
solution: '$(solution)'
msbuildArgs: '/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:DesktopBuildPackageLocation="$(build.artifactStagingDirectory)\WebApp.zip" /p:DeployIisAppPath="Default Web Site"'
platform: '$(buildPlatform)'
configuration: '$(buildConfiguration)'
- task: VSTest#2
inputs:
platform: '$(buildPlatform)'
configuration: '$(buildConfiguration)'
- task: ArchiveFiles#2
inputs:
rootFolderOrFile: '$(Build.BinariesDirectory)'
includeRootFolder: true
archiveType: 'zip'
archiveFile: '$(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip'
replaceExistingArchive: true
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)'
ArtifactName: 'drop'
publishLocation: 'Container'
I guess there is something to do with
$(System.DefaultWorkingDirectory)/**/*.zip
Unsure what i need to configure.
##[error]Error: More than one package matched with specified pattern: D:\a\r1\a***.zip. Please restrain the search pattern.
This error message indicates that you have more than one .zip files in $(Build.ArtifactStagingDirectory).
In your YAML file, both
/p:DesktopBuildPackageLocation="$(build.artifactStagingDirectory)\WebApp.zip"
and
archiveFile: '$(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip'
generate .zip file in $(Build.ArtifactStagingDirectory).
You need to specify which file you want to deploy, just like the following:
$(System.DefaultWorkingDirectory)/**/WebApp.zip
In my case the .sln and .csproj files were in the same folder.
I created a folder for the .csproj and moved the related files there.
It solved the problem.

Combine results of several pipeline jobs into azure web app package

We're in the process of moving our product to an azure web app. We have an extensive existing pipeline containing multiple parallel jobs. One of these jobs compiles a asp.net web application. Some others compile a vue.js website. Currently, the results of the web application and the vue projects are combined in a separate stage, using a powershell script.
Now I can convert the publish step of the web application to generate a deployment package for azure. But what is the best way of also adding the vue outputs into this package so I can deploy it to azure correctly, without losing the parallel jobs? I cannot include the is output files in my project, because they don't exist within the web application build job
You can use publish build artifact task to upload the build results of the web application and vue projects to azure devops server as #Krzysztof mentioned. And you can add a new job to download the artifacts.
Please check below simple example in yaml.
In order to combine the build results, you can use extract file task to extract the zipped artifacts and published the unpacked artifacts in Build_web job. And in the Combine job you can use copy file task to copy the results of vue artifacts to the web artifacts folder. And then you can use archive file task to pack the artifacts which now contains the results of web and vue application.
Combine job should dependsOn Build_web and Build_vue jobs
jobs:
- job: Build_Web
pool:
vmImage: "windows-latest"
steps:
- task: ExtractFiles#1
inputs:
archiveFilePatterns: '*.zip'
destinationFolder: '$(Build.ArtifactStagingDirectory)\unzip'
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)\unzip'
artifactName: webapp
- job: Build_Vue
pool:
vmImage: "windows-latest"
steps:
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: 'path to the build results'
artifactName: vueapp
- job: Combine
dependsOn:
- Build_Web
- Build_Vue
pool:
vmImage: "windows-latest"
steps:
- task: DownloadBuildArtifacts#0
inputs:
buildType: 'current'
artifactName: webapp
downloadPath: "$(System.ArtifactsDirectory)"
- task: DownloadBuildArtifacts#0
inputs:
buildType: 'current'
artifactName: vueapp
downloadPath: "$(System.ArtifactsDirectory)"
- task: CopyFiles#2
inputs:
SourceFolder: '$(System.ArtifactsDirectory)\vueapp'
TargetFolder: 'path to web application result folder' #eg. $(System.ArtifactsDirectory)\webapp\Content\d_C\a\1\s\AboutSite\AboutSite\obj\Release\netcoreapp2.0\PubTmp\Out\
- task: ArchiveFiles#2
inputs:
rootFolderOrFile: $(System.ArtifactsDirectory)\webapp
archiveType: 'zip'
archiveFile: '$(Build.ArtifactStagingDirectory)/webapplication.zip'
Above example only shows a general idea. You can aslo move the ExtractFile task to Combine job. In either way, you will have to use extract file, copy file and archive file task.
For TargetFolder parameter in copy file task, you can check the download build artifacts log for webapp artifact to get the full path. For example as below screenshot shows.
You can use PublishPipelineArtifact#1 to create artifacts for your projects and then in a separate job DownloadPipelineArtifact#2. By defining path you may compose your final artifact (if this mixing many projects is not more complicated than putting one inside another). And publish your artifact as build or pipeline artifact depending how you have roganized your release.
# Download an artifact named 'WebApp' to 'bin' in $(Build.SourcesDirectory)
- task: DownloadPipelineArtifact#2
inputs:
artifact: 'WebApp'
path: $(Build.SourcesDirectory)/bin
Here you have more info about publishing and downloding artifacts.