I want to rerun failed tests in Azure Pipelines via YAML.
Project structure:
RerunTestsDemo.sln
+-- ClassLibrary1.csproj
+-- UnitTestProject1.csproj
+-- UnitTestProject2.csproj
azure-pipelines.yml:
trigger:
- automation/infra
pool: '$(AZURE_AGENT_POOL)'
steps:
- task: DotNetCoreCLI#2
inputs:
command: 'build'
projects: '**/MyUnitTestProject.csproj'
- task: VisualStudioTestPlatformInstaller#1
inputs:
packageFeedSelector: 'nugetOrg'
versionSelector: 'latestStable'
- task: VSTest#2
inputs:
testSelector: 'testAssemblies'
testAssemblyVer2: |
**\*test*.dll
!**\*TestAdapter.dll
!**\obj\**
searchFolder: '$(System.DefaultWorkingDirectory)'
publishRunAttachments: true
rerunFailedTests: true
rerunType: 'basedOnTestFailurePercentage'
rerunFailedThreshold: '30'
rerunMaxAttempts: '2'
env:
# All env vars correctly configured in pipeline
But it is not finding any test:
Please try the following to see if it can work.
steps:
- task: DotNetCoreCLI#2
displayName: 'dotnet build $(buildConfiguration)'
inputs:
projects: |
**/ClassLib1.csproj
**/ClassLib2.csproj
arguments: '--configuration $(buildConfiguration)'
- task: DotNetCoreCLI#2
displayName: 'dotnet test $(buildConfiguration)'
inputs:
command: test
projects: '**/MyUnitTestProject.csproj'
arguments: '--configuration $(buildConfiguration)'
[UPDATE]
Here is a sample as reference:
steps:
. . .
- task: VSBuild#1
displayName: 'Build solution **\*.sln'
inputs:
solution: '$(Parameters.solution)'
platform: '$(BuildPlatform)'
configuration: '$(BuildConfiguration)'
- task: VSTest#2
inputs:
testSelector: 'testAssemblies'
testAssemblyVer2: |
**\*test*.dll
!**\*TestAdapter.dll
!**\obj\**
searchFolder: '$(System.DefaultWorkingDirectory)'
publishRunAttachments: true
rerunFailedTests: true
rerunType: 'basedOnTestFailurePercentage'
rerunFailedThreshold: '30'
rerunMaxAttempts: '2'
I eventually figured it out. The test assembly task was not matching the project pattern.
This is the barebones working YAML:
trigger:
- master
pool:
vmImage: 'windows-latest'
variables:
solution: '**/*Tests*.sln'
steps:
- task: NuGetToolInstaller#1
- task: NuGetCommand#2
inputs:
restoreSolution: '$(solution)'
- task: VSBuild#1
inputs:
solution: '$(solution)'
- task: VisualStudioTestPlatformInstaller#1
inputs:
packageFeedSelector: 'nugetOrg'
versionSelector: 'latestStable'
- task: VSTest#2
inputs:
testSelector: 'testAssemblies'
testAssemblyVer2: |
**\UnitTest*.dll
!**\*TestAdapter.dll
!**\obj\**
searchFolder: '$(System.DefaultWorkingDirectory)'
testFiltercriteria: 'UnitTest1'
publishRunAttachments: True
rerunFailedTests: True
rerunType: 'basedOnTestFailureCount'
rerunFailedTestCasesMaxLimit: '5'
rerunMaxAttempts: '2'
Output:
Related
I'm trying to setup a build on azure devops to suppress flaky tests and continue going through the the tasks, but the test section that I got from the documentation fails regardless of what I try and do
I setup my test section based off of this document
https://learn.microsoft.com/en-us/azure/devops/pipelines/ecosystems/dotnet-core?view=azure-devops&tabs=dotnetfive
specifically lifting this block
steps:
# ...
# do this after your tests have run
- script: dotnet test <test-project> --logger trx
- task: PublishTestResults#2
condition: succeededOrFailed()
inputs:
testRunner: VSTest
testResultsFiles: '**/*.trx'
I get this error message
##[error]Bash exited with code '1'.
and my azure pipeline looks like this
parameters:
- name: solution
type: string
- name: buildPlatform
type: string
- name: buildConfiguration
type: string
steps:
- task: UseDotNet#2
inputs:
version: '5.0.x'
packageType: 'sdk'
- task: DotNetCoreCLI#2
displayName: 'dotnet restore'
inputs:
command: restore
projects: ${{ parameters.solution }}
feedsToUse: select
- task: DotNetCoreCLI#2
displayName: 'dotnet build'
inputs:
command: build
includeNuGetOrg: true
projects: ${{ parameters.solution }}
arguments: '--configuration ${{ parameters.buildConfiguration }}'
- script: dotnet test ${{ parameters.solution }} --logger trx
- task: PublishTestResults#2
condition: succeededOrFailed()
inputs:
testRunner: VSTest
testResultsFiles: '**/*.trx'
- task: CopyFiles#2
displayName: 'copy build'
inputs:
sourceFolder: '$(Build.SourcesDirectory)'
Contents: |
**/*
!.git/**/*
targetFolder: '$(Build.ArtifactStagingDirectory)'
- task: PublishBuildArtifacts#1
displayName: 'publish artifact'
inputs:
pathToPublish: '$(Build.ArtifactStagingDirectory)'
artifactName: drop
- task: ArchiveFiles#2
displayName: 'zip nuget packages'
inputs:
rootFolderOrFile: '$(Build.ArtifactStagingDirectory)/_nupkgs'
includeRootFolder: false
archiveType: 'zip'
archiveFile: '$(Build.ArtifactStagingDirectory)/NugetPackages.zip'
replaceExistingArchive: true
- task: PublishBuildArtifacts#1
displayName: 'publish nuget packages'
inputs:
pathToPublish: '$(Build.ArtifactStagingDirectory)/NugetPackages.zip'
artifactName: nugetPackages
i build a pipeline, that should run on changes on master-branch. at the moment, it runs right, but the code is not changed. If i delete and recreate the pipeline, the update is correct installed. Where is my Error? The server is actually a F1 Plan from Azure (a linux maschine).
parameters:
- name: artifactName
type: string
default: drop
pool:
vmImage: 'windows-latest'
variables:
solution: '**/*.sln'
buildPlatform: 'Any CPU'
buildConfiguration: 'Release'
steps:
- task: NuGetToolInstaller#1
displayName: 'Use NuGet 5.11.0'
inputs:
versionSpec: 5.11.0
checkLatest: true
- task: NuGetCommand#2
displayName: 'NuGet restore'
inputs:
restoreSolution: '$(Parameters.solution)'
- task: DotNetCoreCLI#2
displayName: 'dotnet publish'
inputs:
command: publish
publishWebProjects: false
projects: '**/*.csproj'
arguments: '-r linux-x64 --output $(build.artifactstagingdirectory)'
- task: PublishBuildArtifacts#1
displayName: 'Publish Artifact'
inputs:
PathtoPublish: '$(build.artifactstagingdirectory)'
ArtifactName: '$(Parameters.ArtifactName)'
condition: succeededOrFailed()
- task: AzureRmWebAppDeployment#4
inputs:
ConnectionType: 'AzureRM'
azureSubscription: '...nameOfSubscription'
appType: 'webAppLinux'
WebAppName: '...nameOfWebApp'
packageForLinux: '$(build.artifactstagingdirectory)/*.zip'
RuntimeStack: 'DOTNETCORE|5.0'
You simply didn't deploy application to Azure. What you did is create a package for deployment.
- task: PublishBuildArtifacts#1
displayName: 'Publish Artifact'
inputs:
PathtoPublish: '$(build.artifactstagingdirectory)'
ArtifactName: '$(Parameters.ArtifactName)'
condition: succeededOrFailed()
It publishes artifact, but it doesn't publish your app to Azure.
Please take a look on documentation here
If you want to do this in the same job it would be like:
- task: AzureWebApp#1
inputs:
azureSubscription: '<Azure service connection>'
appType: 'webAppLinux'
appName: '<Name of web app>'
package: '$(build.artifactstagingdirectory)/*.zip'
Please remeber to provide all needed details.
I am trying to get the Nuget file generated in the drop folder to have a name like this - Project.1.0.1-prerelease-2021-05-10.nupkg. But the output of this YAML in the build pipeline is Project.0.1.0.nupkg. What is not correct here? I have tried too many combinations but it keeps generating this in the drop folder.
The csproj file does have a version prefix field like this though
YAML
# ASP.NET Core
# Build and test ASP.NET Core projects targeting .NET Core.
# Add steps that run tests, create a NuGet package, deploy, and more:
# https://learn.microsoft.com/azure/devops/pipelines/languages/dotnet-core
name: $(BuildDefinitionName)_$(Year:yyyy).$(Month).$(DayOfMonth)$(Rev:.r)
trigger:
- main
pool:
vmImage: windows-latest
variables:
buildConfiguration: 'Release'
steps:
- task: DotNetCoreCLI#2
displayName: 'Restore Dependencies'
inputs:
command: 'restore'
projects: '**/*.csproj'
vstsFeed: '9fc2d633-cc8b-49be-a58d-abb9adf6a2c9'
- task: DotNetCoreCLI#2
displayName: 'Debug Build'
inputs:
command: 'build'
projects: '**/*.csproj'
configuration: debug
- task: DotNetCoreCLI#2
displayName: 'Pack Pre Release'
inputs:
command: 'pack'
packagesToPack: '**/Project.csproj'
includesymbols: true
includesource: true
packDirectory: '$(build.artifactstagingdirectory)/release'
arguments: '--configuration debug'
versioningScheme: byPrereleaseNumber
majorVersion: 1
minorVersion: 0
patchVersion: 1
- task: CopyFiles#2
displayName: 'Copy Artifacts'
inputs:
Contents: '**\*.nupkg'
TargetFolder: '$(build.artifactstagingdirectory)'
- task: PublishBuildArtifacts#1
displayName: 'Publish artifacts'
inputs:
PathtoPublish: '$(build.artifactstagingdirectory)'
Ok so got it working by changing this
- task: DotNetCoreCLI#2
displayName: 'Pack Pre Release'
inputs:
command: 'pack'
packagesToPack: '**/Project.csproj'
includesymbols: true
includesource: true
packDirectory: '$(build.artifactstagingdirectory)/release'
arguments: '--configuration debug'
versioningScheme: byPrereleaseNumber
majorVersion: 1
minorVersion: 0
patchVersion: 1
to this
- task: DotNetCoreCLI#2
displayName: 'Pack pre-release package'
inputs:
command: 'custom'
projects: '**/Project.csproj'
custom: 'pack'
arguments: '--version-suffix "pre-release-"$(Build.BuildNumber) --output $(build.artifactstagingdirectory)/debug'
And also right at the top, I changed the name variable to
name: $(Year:yyyy).$(Month).$(DayOfMonth)$(Rev:.r)
And that gives me this package.
The following is my YAML:
trigger:
- master
pool:
vmImage: 'windows-latest'
variables:
solution: '**/*.sln'
buildPlatform: 'Any CPU'
buildConfiguration: 'Release'
steps:
- task: NuGetToolInstaller#1
- task: NuGetCommand#2
inputs:
restoreSolution: '$(solution)'
- task: VSBuild#1
inputs:
solution: '$(solution)'
msbuildArgs: '/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:PackageLocation="$(build.artifactStagingDirectory)"'
platform: '$(buildPlatform)'
configuration: '$(buildConfiguration)'
- task: VSTest#2
inputs:
platform: '$(buildPlatform)'
configuration: '$(buildConfiguration)'
- task: PublishBuildArtifacts#1
inputs:
pathtoPublish: '$(Build.ArtifactStagingDirectory)'
artifactName: 'myWebsiteName'
- task: AzureWebApp#1
inputs:
azureSubscription: 'mySubscription'
appType: 'webApp'
appName: 'mySiteName'
package: '$(System.DefaultWorkingDirectory)/**/*.zip'
deploymentMethod: 'auto'
##[error]Error: No package found with specified pattern: D:\a\1\s\**\*.zip<br/>Check if the package mentioned in the task is published as an artifact in the build or a previous stage and downloaded in the current job.
I've even tried changing $(System.DefaultWorkingDirectory) to $(Build.ArtifactStagingDirectory) in the AzureWebApp task to no avail.
In your current situation, I have some suggestions might help you solve this issue:
Please check the 'Enable system diagnostics' option when you run
your pipeline then check your build task and check if the zip file
has been created successfully and find the zip file folder.
Please go to the pipeline UI page and check if you can find the
artifact file on the page, also please make sure the zip file has
been included.
If you can find the zip file in your artifact. Then we can modify your yaml file like this:
trigger:
- master
pool:
vmImage: 'windows-latest'
variables:
solution: '**/*.sln'
buildPlatform: 'Any CPU'
buildConfiguration: 'Release'
stages:
- stage: generate
jobs:
- job: Job_1
displayName: job_Create_Zip
steps:
- task: NuGetToolInstaller#1
- task: NuGetCommand#2
inputs:
restoreSolution: '$(solution)'
- task: VSBuild#1
inputs:
solution: '$(solution)'
msbuildArgs: '/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:PackageLocation="$(build.artifactStagingDirectory)"'
platform: '$(buildPlatform)'
configuration: '$(buildConfiguration)'
- task: VSTest#2
inputs:
platform: '$(buildPlatform)'
configuration: '$(buildConfiguration)'
- task: PublishBuildArtifacts#1
inputs:
pathtoPublish: '$(Build.ArtifactStagingDirectory)'
artifactName: 'myWebsiteName'
- stage:
jobs:
- job: Job_2
displayName: Deploy_zip
steps:
- task: DownloadPipelineArtifact#2
inputs:
buildType: 'current'
artifactName: 'myWebsiteName'
itemPattern: '**/*.zip'
targetPath: '$(System.DefaultWorkingDirectory)'
- task: AzureWebApp#1
inputs:
azureSubscription: 'mySubscription'
appType: 'webApp'
appName: 'mySiteName'
package: '$(System.DefaultWorkingDirectory)/**/*.zip'
deploymentMethod: 'auto'
In the VSBuild task output, you should see a line saying:
Package "{your_project_name}.zip" is successfully created as single file at the following location:
file:///D:/a/1/a
D:/a/1/a in the above above example comes from /p:PackageLocation="$(build.artifactStagingDirectory)" parameter that you pass to msbuild (which is different than DefaultWorkingDirectory: D:\a\1\s\).
So your AzureWebApp task has to look inside $(build.artifactStagingDirectory). Make sure these paths match.
I am trying to push private nuget package via azure pipeline. There is no error but i cant see artifact being published in my feed. Here is my yml file. What am i doing wrong here or what can i add?
name: $(BuildDefinitionName)_$(Year:yyyy).$(Month).$(DayOfMonth)$(Rev:.r)
trigger:
- master
pool:
vmImage: 'windows-latest'
variables:
buildConfiguration: 'Release'
version.MajorMinor: '4.0'
version.Revision: $[counter(variables['version.MajorMinor'], 0)]
versionNumber: '$(version.MajorMinor).$(version.Revision)'
steps:
- task: CmdLine#2
inputs:
script: 'dir'
displayName: 'dotnet build $(buildConfiguration)'
- task: DotNetCoreCLI#2
displayName: 'dotnet build'
inputs:
command: 'build'
arguments: '--configuration $(buildConfiguration)'
projects: 'MyProj.csproj'
- task: PowerShell#2
displayName: Set the name of the build (i.e. the Build.BuildNumber)
inputs:
targetType: 'inline'
script: |
[string] $buildName = "$(Build.SourceBranchName)_$(versionNumber)"
Write-Host "Setting the name of the build to '$buildName'."
Write-Host "##vso[build.updatebuildnumber]$buildName"
- task: DotNetCoreCLI#2
displayName: "dotnet pack"
inputs:
command: 'pack'
packagesToPack: 'MyProj.csproj'
nobuild: true
versioningScheme: 'byEnvVar'
versionEnvVar: 'versionNumber'
packDestination: '$(Build.ArtifactStagingDirectory)'
- task: NuGetCommand#2
inputs:
command: 'push'
packagesToPush: '$(Build.ArtifactStagingDirectory)/**/*.nupkg;!$(Build.ArtifactStagingDirectory)/**/*.symbols.nupkg'
nuGetFeedType: 'internal'
publishVstsFeed: 'XXXXXXXXX/YYYYYYYYYYYY'
allowPackageConflicts: true
To publish to an Azure Artifacts feed, set the Project Collection Build Service identity to be a Contributor on the feed. And add the following snippet to your azure-pipelines.yml file.
steps:
- task: NuGetAuthenticate#0
displayName: 'NuGet Authenticate'
- task: NuGetCommand#2
displayName: 'NuGet push'
inputs:
command: push
publishVstsFeed: '<projectName>/<feed>'
allowPackageConflicts: true
Refer to the documentation here:
https://learn.microsoft.com/en-us/azure/devops/pipelines/artifacts/nuget?view=azure-devops&tabs=yaml