DevOps Pipeline - can't find packages during build step - azure-devops

I'm trying to shift an existing ASP.NET MVC web application away from TeamCity and on to Azure DevOps however I can't seem to get my pipeline to find the packages that it restored from the NuGet package restore task.
The solution has multiple projects and they use a large number of NuGet packages. I've created a multi-step pipeline which currently has a NuGet Restore task and a VS Build task.
The solution structure is like this:
\source
..\SolutionFolder
...Solution.sln
..\..\Project1
....Project1.csproj
..\..\Project2
....Project2.csproj
..\..\Packages
azure-pipelines.yaml
NuGet.config
The pipeline YAML looks like this:
trigger:
- PipelineBranch
pool:
vmImage: 'windows-latest'
stages:
## NuGet Restore ##
- stage: nuget_restore
displayName: 'Restore nuget packages'
jobs:
- job: nuget_package_restore
steps:
- task: NuGetCommand#2
displayName: 'NuGet restore'
inputs:
command: 'restore'
restoreSolution: 'source/SolutionFolder/Solution.sln'
feedsToUse: 'config'
nugetConfigPath: 'NuGet.Config'
restoreDirectory: '$(Build.SourcesDirectory)\source\SolutionFolder\packages'
## Build ##
- stage: build_for_int
displayName: 'Build to int)'
jobs:
- job: run_build
pool:
vmImage: 'windows-latest'
steps:
- task: VSBuild#1
displayName: 'Build Solution.sln (int)'
inputs:
solution: 'source/SolutionFolder/Solution.sln'
msbuildArgs: '
/p:DeployOnBuild=true
/p:WebPublishMethod=Package
/p:PackageAsSingleFile=true
/p:SkipInvalidConfigurations=true
/p:PackageLocation="$(Build.ArtifactStagingDirectory)"'
platform: '$(BuildPlatform)'
configuration: '$(BuildConfiguration)'
When the pipeline runs the restore task succeeds with the following outputs:
<example package>
Added package 'postal.1.2.2' to folder 'd:\a\1\s\source\SolutionFolder\packages'
<loads more packages>
Installed:
190 package(s) to packages.config projects
When the build task runs on the solution it fails with the following error on the first project it encounters:
[warning]C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\MSBuild\Current\Bin\Microsoft.Common.CurrentVersion.targets(2106,5):
Warning MSB3245: Could not resolve this reference. Could not locate the assembly "postal, Version=1.2.14706.0, Culture=neutral, processorArchitecture=MSIL". Check to make sure the assembly exists on disk. If this reference is required by your code, you may get compilation errors.
...
Considered "d:\a\1\s\source\SolutionFolder\packages\postal.1.2.2\lib\postal.dll", but it didn't exist.
The same error occurs for each package dependency. It looks like the package restore task restores packages to here:
d:\a\1\s\source\SolutionFolder\packages\
And according to the build log it is looking for them here in the hierarchical package folder structure e.g:
d:\a\1\s\source\SolutionFolder\packages\<package>\<lib>\<dll>
I've tried a number of things including removing the restoreDirectory parameter from the restore task, running the MSBuild command locally (it works) and stripping out some of the packages however I'm still seeing the same build errors.
Some of the project files also contain hintpaths; for example in the format:
<ItemGroup>
<Reference Include="<some package>">
<HintPath>..\packages\<package>\lib\<dll></HintPath>
</Reference>
Would anyone have any insights? Thanks.

You should include tasks VSBuild#1 and NuGetCommand#2 under the one same job.
I tested on my local hosted agent and found the second job which runs vsbuild task will clear the working folder and refetch the source code, which removes the packages folder created by the first job task nugetcommand. This might be the reason that causes the error.
So the pipeline yaml should like this
trigger:
- PipelineBranch
pool:
vmImage: 'windows-latest'
stages:
## NuGet Restore ##
- stage: nuget_restore
displayName: 'Restore nuget packages'
jobs:
- job: nuget_package_restore
steps:
- task: NuGetCommand#2
displayName: 'NuGet restore'
inputs:
command: 'restore'
restoreSolution: 'source/SolutionFolder/Solution.sln'
feedsToUse: 'config'
nugetConfigPath: 'NuGet.Config'
restoreDirectory: '$(Build.SourcesDirectory)\source\SolutionFolder\packages'
- task: VSBuild#1
displayName: 'Build Solution.sln (int)'
inputs:
solution: 'source/SolutionFolder/Solution.sln'
msbuildArgs: '
/p:DeployOnBuild=true
/p:WebPublishMethod=Package
/p:PackageAsSingleFile=true
/p:SkipInvalidConfigurations=true
/p:PackageLocation="$(Build.ArtifactStagingDirectory)"'
platform: '$(BuildPlatform)'
configuration: '$(BuildConfiguration)'

Related

getting error in AzureDevOps Release pipeline running due to artifact

Hello I am having a problem. Suppose I am having a simple web app and pushed it to Azure Repos. Now I set up a build pipeline using yaml and then I set up a release pipeline which will require an artifact from the build pipeline. So I am using PublishPipelineArtifact#1. The build pipeline runs fine but my release pipeline fails with error Deployment of msBuild generated package is not supported. Change package format or use Azure App Service Deploy task. D:\a\r1\a_ReleasePipelines\drop\WebApp.zip.
So how to provide artifact required by release pipeline in correct naming sequence?
Below is my yaml for build pipeline
`
# ASP.NET Core (.NET Framework)
# Build and test ASP.NET Core projects targeting the full .NET Framework.
# Add steps that publish symbols, save build artifacts, and more:
# https://docs.microsoft.com/azure/devops/pipelines/languages/dotnet-core
trigger:
- master
pool:
vmImage: 'windows-latest'
variables:
solution: '**/*.sln'
buildPlatform: 'Any CPU'
buildConfiguration: 'Release'
steps:
- task: NuGetToolInstaller#1
- task: NuGetCommand#2
inputs:
restoreSolution: '$(solution)'
- task: VSBuild#1
inputs:
solution: '$(solution)'
msbuildArgs: '/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:DesktopBuildPackageLocation="$(build.artifactStagingDirectory)\WebApp.zip" /p:DeployIisAppPath="Default Web Site"'
platform: '$(buildPlatform)'
configuration: '$(buildConfiguration)'
- task: PublishPipelineArtifact#1
inputs:
targetPath: '$(build.artifactStagingDirectory)'
publishLocation: 'pipeline'
`
Next I am setting up a simple release pipeline using empty job and there in Add Artifact I am supplying the project name and source.
From the MS build arguments in your YAML, it is for Web Deploy method.
If you would like to change the method to Zip deploy, you could follow this official link to change the Arguments: https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/reference/azure-rm-web-app-deployment-v4?view=azure-pipelines&viewFallbackFrom=azure-devops#error-publish-using-zip-deploy-option-is-not-supported-for-msbuild-package-type
Publish build artifacts is recommended: https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/reference/publish-build-artifacts-v1?view=azure-pipelines&viewFallbackFrom=azure-devops
Here is a sample,
steps:
- task: PublishBuildArtifacts#1
displayName: 'Publish Artifact: drop'
inputs:
PathtoPublish: '$(build.artifactstagingdirectory)'
condition: succeededOrFailed()
In your release pipeline, you need to specify the pipeline artifacts from build :
And use Azure App Service Deploy task for the deployment.
Build pipeline sample:
Release pipeline sample:

Could not be resolved because it has an indirect dependency on the assembly - Azure DevOps Build Pipeline

I have two projects. Project 1 and Project 2. Project 1 refers to Project 2 in its csproj file. I have recently upgraded the Project 2 to a higher version of the .NET. When I did the same for Project 1 and ran my CI Pipeline, I have been getting quite a lot of errors, mostly like below. When I run the CI pipeline for Proj 2 seperately, it is building successfully.
I checked all possible places to see if I have any old references, but I am unable to find where exactly is the problem. Appreciate your help here.
Done Building Project "D:\a\1\s\dia-praj2\src\Proj-2\Proj-2.csproj" (default targets).
##[warning]D:\a\1\s\dia-proj1\proj1.metaproj(0,0): Warning MSB3274: The primary reference "D:\a\1\s\dia-praj2\src\Proj-2\bin\Release\Proj-2.dll" could not be resolved because it was built against the ".NETFramework,Version=v4.7.2"
##[warning]D:\a\1\s\dia-proj1\proj1.metaproj(0,0): Warning MSB3275: The primary reference "D:\a\1\s\dia-praj2\src\Proj-2\bin\Release\Proj-2.dll" could not be resolved because it has an indirect dependency on the assembly "CsvHelper, Version=27.0.0.0, Culture=neutral, PublicKeyToken=8c4959082be5c823" which was built against the ".NETFramework,Version=v4.7" framework. This is a higher version than the currently targeted framework ".NETFramework,Version=v4.5".
##[warning]D:\a\1\s\dia-proj1\proj1.metaproj(0,0): Warning MSB3275: The primary reference "D:\a\1\s\dia-praj2\src\Proj-2\bin\Release\Proj-2.dll" could not be resolved because it has an indirect dependency on the assembly "Azure.Core, Version=1.22.0.0, Culture=neutral, PublicKeyToken=92742159e12e44c8" which was built against the ".NETFramework,Version=v4.6.1" framework. This is a higher version than the currently targeted framework ".NETFramework,Version=v4.5".
##[warning]D:\a\1\s\dia-proj1\proj1.metaproj(0,0): Warning MSB3275: The primary reference "D:\a\1\s\dia-praj2\src\Proj-2\bin\Release\Proj-2.dll" could not be resolved because it has an indirect dependency on the assembly "Microsoft.Bcl.AsyncInterfaces, Version=5.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51" which was built against the ".NETFramework,Version=v4.6.1" framework. This is a higher version than the currently targeted framework ".NETFramework,Version=v4.5".
My Build Pipeline
a# ASP.NET
# Build and test ASP.NET projects.
# Add steps that publish symbols, save build artifacts, deploy, and more:
# https://learn.microsoft.com/azure/devops/pipelines/apps/aspnet/build-aspnet-4
pool:
vmImage: 'windows-latest'
variables:
solution: '**/Proj-1.sln'
buildPlatform: 'Any CPU'
buildConfiguration: 'Release'
name: $(Build.DefinitionName)_$(SourceBranchName)_$(Date:yyyyMMdd)$(Rev:.r)
resources:
repositories:
- repository: Proj-2
name: Proj-2
type: git
ref: feature/Proj-2
# this is being defined in app-ci pipeline
steps:
- script: echo '$(Build.BuildNumber)'
displayName: "echo build number"
- checkout: Proj-2
displayName: 'checkout Proj-2'
- checkout: self
displayName: 'checkout Proj-1'
- task: NuGetToolInstaller#1
displayName: Install Nuget Package Manager
- task: NuGetCommand#2
displayName: Restore Nuget packages
inputs:
command: 'restore'
restoreSolution: '$(solution)'
restoreDirectory: 'c:\packages'
- task: VSBuild#1
displayName: Build dia-ui solution
inputs:
solution: '$(solution)'
vsVersion: '16.0'
msbuildArgs: '/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:PackageLocation="$(build.artifactStagingDirectory)"'
platform: '$(buildPlatform)'
configuration: '$(buildConfiguration)'
clean: true
- task: CopyFiles#2
displayName: Copy everything to the Binaries dir
inputs:
Contents: '**/Prj1/abc/**'
TargetFolder: $(Build.BinariesDirectory)
- task: ArchiveFiles#2
displayName: Zip the binaries dir into a file called 'drop.zip'
inputs:
rootFolderOrFile: '$(Build.BinariesDirectory)/Prj1/abc/'
includeRootFolder: false
archiveType: 'zip'
archiveFile: '$(Build.ArtifactStagingDirectory)/drop.zip'
replaceExistingArchive: true
- task: PublishBuildArtifacts#1
displayName: Publish 'drop.zip' as an artifact
inputs:
PathtoPublish: '$(build.artifactstagingdirectory)/drop.zip'
ArtifactName: 'drop'
The dependencies of your project do not match the .NET version your project is on now. I had the same issue and reinstalled the packages using the method mentioned here. In short: in the package manager console run
update-package <package name> -reinstall
Do this for all packages installed in the project where you changed the .NET version.

.NET Standard Pipeline: Error NETSDK1004 project.assets.json' not found. Nuget Restore

This is my first time working with pipelines. I have a library repo that is a .NET Standard library intended to be shared with others. I would like a pipeline that builds it and runs the unit test sln.
Currently I have this after selecting ASP.NET from the predefined options for configuring a pipeline.
- main
pool:
vmImage: 'windows-latest'
variables:
solution: '**/MyLibrary.sln'
buildPlatform: 'Any CPU'
buildConfiguration: 'Release'
steps:
- task: NuGetToolInstaller#1
- task: VSBuild#1
inputs:
solution: '**\My.sln'
vsVersion: '15.0'
restoreNugetPackages: true
- task: VSTest#2 # NOT DONE YET
inputs:
platform: '$(buildPlatform)'
And I realize now that restoreNugetPackages is deprecated. But when I build I get the error NETSDK1004 project.assets.json not found which is related to not using restore.
When I select the predefined option for restore, I'm curious what feed should I be selecting from Azure Artifacts or should I be selecting my nuget.config. How would I build a nuget.config? I don't have one currently in the repo.
Below are my current NuGet packages for the sln if this helps
Microsoft.Bcl.AsyncInterfaces
NETStandard.Library
Newtonsoft.Json
System.Runtime.CompilerServices.uNSafe
System.Threading.Tasks.Extensions
Add NuGet task to restore nuget packages before running the build:
- task: NuGetCommand#2
inputs:
restoreSolution: '$(solution)'
- task: VSBuild#1
As similar pattern of Cace's answer, the below code resolved the problem in my environment.
azure-pipelines.yml:
- task: NuGetCommand#2
inputs:
command: 'restore'
restoreSolution: '**/*.sln'

Publishing Pipeline Artifacts for use in another pipeline

I'm trying to publish a couple of pipeline artifacts during a build so that I can use them in a build of another solution.
My first build builds and tests the soltuion using the following yaml
- stage: build_test_release
displayName: Build
pool:
vmImage: 'windows-latest'
variables:
solution: '**/*.sln'
buildPlatform: 'Any CPU'
buildConfiguration: 'Release'
jobs:
- job: build
steps:
- task: NuGetToolInstaller#1
- task: NuGetCommand#2
inputs:
restoreSolution: '$(solution)'
- task: VSBuild#1
inputs:
solution: '$(solution)'
msbuildArgs: '/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:PackageLocation="$(build.artifactStagingDirectory)"'
platform: '$(buildPlatform)'
configuration: '$(buildConfiguration)'
- task: VSTest#2
inputs:
platform: '$(buildPlatform)'
configuration: '$(buildConfiguration)'
This all works great and I've been able to do deployments using this yaml in the same file
- task: AzureRmWebAppDeployment#4
inputs:
ConnectionType: 'AzureRM'
azureSubscription: 'azuresubscription'
appType: 'webApp'
WebAppName: 'webappname'
deployToSlotOrASE: true
ResourceGroupName: 'resourcegroupname'
SlotName: 'staging'
packageForLinux: '$(build.artifactStagingDirectory)/**/projectToPublish.zip'
I'm now trying to publish some of the projects which were build as Pipeline Artifacts so that I can use them in the build of another solution which references them. The following publish task gives me the error:
"##[error]Path does not exist: D:\a\1\a**\projectToPublish.zip"
- task: PublishPipelineArtifact#1
inputs:
targetPath: '$(build.artifactStagingDirectory)/**/projectToPublish.zip'
artifact: 'artifactname'
publishLocation: 'pipeline'
What am I missing here? I've been thinking of moving the projects which are referenced by both soltuions into their own solutions and adding them as nuget packages or something similar. The projects I am trying to publish as artifacts to be used in the second solution are both WCF Client projects.
Thanks!
The issue is that the task you're using, Publish Pipeline Artifacts, does not support wildcards for the 'File or directory path' argument. For this task $(build.artifactStagingDirectory) is replaced with D:\a\1\a\, in the second directory named a Azure DevOps is looking for a sub directory named ** which does not exist and causes the shown error. The other task AzureRmWebAppDeployment#4 you're using does support wildcards.
See the image below, where Azure DevOps shows in the the UI that the PublishPipelineArtifact#1 task does not support wildcards in targetPath:
Second I'm wondering why you're using a wildcard, because the task VSBuild#1 will just drop the package in the build.artifactStagingDirectory which should make the path $(build.artifactStagingDirectory)/projectToPublish.zip to locate the package.
In AzureRmWebAppDeployment#4 you have
'$(build.artifactStagingDirectory)/**/WebProjectToPublish.zip'
And in PublishPipelineArtifact#1 you have a different name
'$(build.artifactStagingDirectory)/**/projectToPublish.zip'

How to automatically add string, e.g., -dev, to NuGet packages generated by DevOps Pipeline

Our DevOps Pipeline has versioningScheme set to off in the nuget pack command (options for that are documented here), and we're using the AssemblyVersion attribute in the AssemblyInfo.cs file to specify NuGet package versions. E.g.,:
[assembly: AssemblyVersion("1.9.14")]
We have a DevOps Pipeline setup that, when we merge code into our Git "develop" branch, will automatically perform a Debug build and publish to our private NuGet repository. The NuGet package version will match the AssemblyVersion attribute's value.
We have another DevOps pipeline that will do the same for the Master branch, but the build will be in Release mode.
We would like to have the Pipeline for the "develop" branch automatically append a string, e.g., "-beta" or "-dev", so that we can easily distinguish our pre-release builds, that are built in Debug mode, from the release builds. I've reviewed the YAML for our pipeline and I have been unable to figure out how to do that when you're using the AssemblyVersion (or AssemblyInformationalVersion) attribute. Is this possible? Or would we need to switch to a different technique for specifying versions?
YAML for our current Pipeline is below:
# ASP.NET
# Build and test ASP.NET projects.
# Add steps that publish symbols, save build artifacts, deploy, and more:
# https://learn.microsoft.com/azure/devops/pipelines/apps/aspnet/build-aspnet-4
trigger:
- develop
pool:
vmImage: 'windows-latest'
variables:
solution: '**/*.sln'
buildPlatform: 'Any CPU'
buildConfiguration: 'Debug'
steps:
- task: NuGetToolInstaller#1
- task: NuGetCommand#2
inputs:
restoreSolution: '$(solution)'
- task: VSBuild#1
inputs:
solution: '$(solution)'
msbuildArgs: '/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:PackageLocation="$(build.artifactStagingDirectory)"'
platform: '$(buildPlatform)'
configuration: '$(buildConfiguration)'
- task: VSTest#2
inputs:
platform: '$(buildPlatform)'
configuration: '$(buildConfiguration)'
- task: NuGetCommand#2
inputs:
command: 'pack'
packagesToPack: 'DataAccessLayer/DataAccessLayer.csproj'
versioningScheme: 'off'
- task: NuGetCommand#2
inputs:
command: 'push'
packagesToPush: '$(Build.ArtifactStagingDirectory)/**/*.nupkg;!$(Build.ArtifactStagingDirectory)/**/*.symbols.nupkg'
nuGetFeedType: 'internal'
publishVstsFeed: '2ba68ed5-dcd7-40e3-8b6b-06972c41ec5e'
versioningScheme: 'off' in the pack step is the issue, it means there is no way to override the package version passed to the nuget pack command.
Rather than setting the version number explicitly in the assembly info file, an alternative solution is to just specify the major minor version and let Azure Devops calculate the next patch version with a counter expression, then setting the build number to major.minor.path:
variables:
majorMinor: 1.10
patch: $[counter(variables['majorMinor'])]
steps:
- task: PowerShell#2
displayName: 'echo patch (debug)'
inputs:
targetType: inline
script: echo $(patch)
- task: PowerShell#2
displayName: 'update build number to major.minor.patch'
inputs:
targetType: inline
script: |
if ("$(Build.SourceBranchName)" -eq "master") {
echo "##vso[build.updatebuildnumber]$(majorMinor).$(patch)"
} else {
echo "##vso[build.updatebuildnumber]$(majorMinor).$(patch)-alpha"
}
You can then make use of the BUILD_BUILDNUMBER environment variable in your nuget pack step:
- task: NuGetCommand#2
inputs:
command: 'pack'
packagesToPack: 'DataAccessLayer/DataAccessLayer.csproj'
versioningScheme: byEnvVar
versionEnvVar: BUILD_BUILDNUMBER