Inconsistent output in YAML pipeline vs default Azure Pipeline - azure-devops

I'm trying to run a "dotnet tool install" to install cli tool on the build agent in my pipeline and the behavior is inconsistent. Sometimes it executes the command successfully but it also fails sometimes with below error and I cannot figure out why however the same step executes successfully when i use azure default pipeline instead of YAML based.
error NU1212: Invalid project-package combination for project 1.0.0. DotnetToolReference project style can only contain references of the DotnetTool type The tool package could not be restored. Tool 'project' failed to install. This failure may have been caused by:
* You are attempting to install a preview release and did not use the --version option to specify the version.
* A package by this name was found, but it was not a .NET Core tool.
* The required NuGet feed cannot be accessed, perhaps because of an Internet connection problem.
* You mistyped the name of the tool.
Here is the task:
- task: DotNetCoreCLI#2
displayName: 'Install project'
inputs:
command: custom
custom: tool
arguments: 'install -g --version 1.0.0 --add-source https://sample.jfrog.io/artifactory/project project'
Command:
dotnet tool install -g --version 1.0.0 --add-source https://sample.jfrog.io/artifactory/project project
These tasks are running on Windows-latest image on Microsoft Hosted agents

Related

Alternatives for gnu tar when using cache#2 task in Azure DevOps

I'm having a hard time getting cache#2 task working as it seems when the build agent is windows OS, it requires to have GNU Tar installed in the machine and this seems it doesnt seem to be an option right now. This needs to run in the windows machine as its part of a stage and requires to collect the build output to produce the artifact.
Somewhere I read this could be an option but it doesn't work and its not on Microsoft's documentation: tool: '7zip'
- task: Cache#2
displayName: Restore from cache
inputs:
key: 'npm | "$(Agent.OS)" | $(projectDir)package-lock.json'
#restoreKeys: |
# npm | "$(Agent.OS)"
path: $(Pipeline.Workspace)/.npm
cacheHitVar: CACHE_RESTORED
verbose: true
- script: |
npm install
displayName: Install dependencies #, Build and Lint
workingDirectory: $(projectDir)
condition: ne(variables.CACHE_RESTORED, 'true')
The log says: ##[error]Failed to start the required dependency 'tar'. Please verify the correct version is installed and available on the path. This is accurate but I was wondering if there's any workaround or if I just can have the cache implemented and I have to install all dependencies on each run.
Btw, I'm using the default pool and its using windows v10.0.14393 and agent version 2.206.1
Thanks!
If it helps to anyone, I had to split my workflow into API and FE stages. The angular stage would run in the ubuntu agent that would have the tar installed. That fixed my problem and I also had the 2 stages running in parallel.

Azure Pipelines: Where is the output of my script?

I tried to build my Angular 13 app on a self-hosted agent and created the following YAML snippet for this:
- task: NodeTool#0
displayName: 'Install Node.js'
inputs:
versionSpec: '14.x'
- script: |
npm install -g #angular/cli
npm install
ng build --configuration production --aot
displayName: 'npm install and build'
workingDirectory: '$(Build.SourcesDirectory)/src'
I can observe the /s directory of the agent _work-directory and after my task was running, there is no node_modules folder or dist folder inside.
But also no console output.
If I remove the line "npm install -g #angular/cli" from the line, a node_modules folder gets created, but no dist-folder.
I am pretty sure that the installation of angular cli fails, but I do not get any error output in my window.
It just looks like this:
How can I get more logs to find out why the angular cli is not installing correctly? I saw that the "script" file that is executed on the agent puts an #echo off by default in front of the script.
Why is that?
How can I get some output to find my problem?
To get more detailed log from the pipeline you can add the variable system.debug and set the value to true in your pipeline.
For YAML pipelines, you can select Variables in the upper right corner of the YAML edit page.
Add a new variable with the name System.Debug and value true.
For more info about logs, please refer to Review logs to diagnose pipeline issues.

Azure CI Pipeline - DotNetCoreCLI#2 Task error

I developed WebAPI project using .NET Core 3.1.0 and integration tests using XUnit.
I added the below task in Azure DevOps CI Pipeline (azure-pipelines.yaml) to run the integration tests project.
- task: DotNetCoreCLI#2
displayName: 'Run API integration tests - $(buildConfiguration)'
inputs:
command: 'test'
arguments: '--configuration $(buildConfiguration)'
publishTestResults: true
projects: '**/IntegrationTests/IntegrationTests.csproj'
I got the below error during pipeline execution. How to resolve this error?
##[error]Error: The process '/usr/bin/dotnet' failed with exit code 1
##[warning].NET 5 has some compatibility issues with older Nuget versions(<=5.7), so if you are using an older Nuget version(and not dotnet cli) to restore, then the dotnet cli commands (e.g. dotnet build) which rely on such restored packages might fail. To mitigate such error, you can either: (1) - Use dotnet cli to restore, (2) - Use Nuget version 5.8 to restore, (3) - Use global.json using an older sdk version(<=3) to build
Info: Azure Pipelines hosted agents have been updated and now contain .Net 5.x SDK/Runtime along with the older .Net Core version which are currently lts. Unless you have locked down a SDK version for your project(s), 5.x SDK might be picked up which might have breaking behavior as compared to previous versions. You can learn more about the breaking changes here: https://learn.microsoft.com/en-us/dotnet/core/tools/ and https://learn.microsoft.com/en-us/dotnet/core/compatibility/ . To learn about more such changes and troubleshoot, refer here: https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/build/dotnet-core-cli?view=azure-devops#troubleshooting
##[error]Dotnet command failed with non-zero exit code on the following projects : /home/vsts/work/1/s/src/IntegrationTests/IntegrationTests.csproj
I've had a smiller issue but with .netcore 2.2. The problem was that the test tries to build before the test starts restoring the packages for the test, thus fails before the test runs or builds. One thinge that help me overcome this problem was this FAQ:
Most dotnet commands, including build, publish, and test include an implicit restore step. This will fail against authenticated feeds, even if you ran a successful dotnet restore in an earlier step, because the earlier step will have cleaned up the credentials it used.
To fix this issue, add the --no-restore flag to the Arguments textbox.
I've also read that the DotNetCLI had some issues when it came to tests like this one here
So I ended up using a script to solve this and other issues related to package restore.
- script: dotnet test '**/IntegrationTests/IntegrationTests.csproj' --configuration $(buildConfiguration) --logger trx;LogFileName=C:\temp\results
displayName: 'Run API integration tests - $(buildConfiguration)'
I hope that will help you or anyone who has similar issues.
I've had exactly the same problem, with the difference that my solution consisted of .net5 apps as well as .netcore3.1 apps.
I was able to solve this problem by specifying the newer dotnet runtime in the azure pipeline:
- task: UseDotNet#2
inputs:
version: '5.0.x'
packageType: runtime

Random failures in build pipelines when running dotnet-ef

As part of build I need to generate db migration script. I'm using Microsoft provided build agent
(only interesting part below)
pool:
vmImage: 'windows-2019'
- task: DotNetCoreCLI#2
displayName: Install dotnet-ef
inputs:
command: 'custom'
custom: 'tool'
arguments: 'install dotnet-ef -g --version 5.0.0-preview.8.20407.4'
- task: DotNetCoreCLI#2
displayName: Generate migrations sql script
inputs:
command: 'custom'
custom: 'ef'
arguments: 'migrations script --project Web/Dal --startup-project Web/WebApi --configuration $(buildConfiguration) --context EmailContext --no-build --output $(Build.ArtifactStagingDirectory)/emailcontext-migrations.sql --idempotent --verbose'
dotnet-ef installation seems to work fine:
Tool 'dotnet-ef' (version '5.0.0-preview.8.20407.4') was successfully installed.
but it still fails from time to time with (more often recently) :
"C:\Program Files\dotnet\dotnet.exe" ef migrations script --project Web/Dal --startup-project Web/WebApi --configuration Release --context EmailContext --no-build --output D:\a\1\a/emailcontext-migrations.sql --idempotent --verbose
Could not execute because the specified command or file was not found.
Is there a problem with my build pipeline configuration?
If it fails from time to time I would rather say that this can be an issue with preview version.
Please add an next step after installing to list all globally installed tools:
dotnet tool list -g
You may also show us a log of installing tool for case when your pipeline doesn't work. To verify if you have this:
(We simply don't know it, since we can't check your logs).
And if it still happens I would encourage you to create an issue on GitHub.
From your description, this is an intermittent issue. So your pipeline configuration could be correct.
Could not execute because the specified command or file was not found.
This issue seems to be related to the dotnet-ef package installed.
As Krzysztof Madej's suggestion, this package version could cause this issue.
You could try to use the latest version: 5.0.0-rc.1.20451.13 or latest stable version: 3.1.8.
Here is a GitHub ticket with the same issue( Can't find the file after global installing dotnet-ef). You could follow it and check the update.
On the other hand, you could try to use the Command Line Task to install the dotnet-ef.
For example:
- task: CmdLine#2
inputs:
script: 'dotnet tool install --global dotnet-ef --version xxxx'

Code Coverage Screen hangs in Azure Pipeline Build

Im currently setting up an azure pipeline for my repository. Currently it builds correctly and runs the unit tests. However the code coverage tab just spins infinitely. Any idea on what would cause this?
Details:
The artifact directory looks like this:
The console shows this error:
Error: Could not find route for route id
ms.vss-tfs-web.project-overview-route. Ensure that the requested route
is added to routes shared data.
This is how the test results are ran and generated:
dotnet tool install dotnet-reportgenerator-globaltool --tool-path .
dotnet test $(Build.SourcesDirectory)\RulesMadeEasy.Tests -c debug --logger trx --no-restore /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura --results-directory $(Build.SourcesDirectory)\TestResults\ /p:CoverletOutput=$(Build.SourcesDirectory)\TestResults\
.\reportgenerator -reports:$(Build.SourcesDirectory)\TestResults\coverage.cobertura.xml -targetdir:$(Build.SourcesDirectory)\TestResults\ -reporttypes:"HTMLInline_AzurePipelines;Badges" --version 4.0.0-rc4
The code coverage results are published using the PublishCodeCoverageResults#1 task with the following inputs
inputs:
codeCoverageTool: Cobertura
summaryFileLocation: '$(Build.SourcesDirectory)\TestResults\coverage.cobertura.xml'
reportDirectory: '$(Build.SourcesDirectory)\TestResults'
You have to enable the Boards service in you Azure DevOps project to let the error disappear.