I'm trying to add a 'Publish Test Results' task into my Azure DevOps release pipeline, however, it looks like when I try to publish the .trx file, it's not looking in the correct folder and tries to parse an invalid file format (I'm assuming this is because it can't find the TestResults folder). I can see my file being created here in the Visual Studio Test task logs:
2020-02-19T21:28:57.6557476Z Vstest.console.exe exited with code 0.
2020-02-19T21:28:57.6557819Z:Completed test execution
2020-02-19T21:28:57.6640479Z Test results files: C:\agents\vm1-1_work_temp\TestResults\vm1$_vm1_2020-02-19_21_28_31.trx
However, in my Publish Test Results task logs, I get this error:
2020-02-19T21:28:59.7249456Z [command]C:\agents\vm1-1_work_tasks\PublishTestResults_0b0f01ed-7dde-43ff-9cbb-e48954daf9b1\2.164.3\modules\TestResultsPublisher.exe #C:\agents\vm1-1_work_temp\d793cb80-535e-11ea-9ab4-8f1d738f183d.txt
2020-02-19T21:29:00.5600855Z ##[warning]Failed to parse result files: Invalid file format.
This is what my Test Run and Publish Test Results tasks look like....can anyone point out what I am doing wrong here? I've seen people say that I don't need to add this task, but I am unable to see the Test Results tab in the pipeline, I can only see the .trx file attached to the task (see below).
TestResultsFormat is an alias for the testRunner input name. It seems you are using VSTest testRunner, so in Publish Test Results task, you should choose VSTest in TestResultsFormat.
Related
I have an .Net Core application along with unit test cases. For that I have configured the Build pipeline in Azure DevOps. In that pipeline I have integrated SonarQube tasks (prepare analysis, run code analysis, and publish quality gate results).
I can see the report in SonarQube server after successful run. But in that report, I didn’t see the Code Coverage Results and Unit test results. Even though I used Cobertura for unit tests.
Your pipeline seems to contain the right steps, so there can be two issues:
1. Code coverage file is not generated (correctly)
The easiest way to validate if the code coverage file is generated correctly, is by publishing it as an artifact. Now check what format the output file is. If there is no output file, please check if you did include /p:CollectCoverage=true --logger trx to the test command. If you are running the build pipeline on Linux, you should also add /p:CoverletOutputFormat=opencover and install the coverlet.collector NuGet package in the .NET Test Project.
2. Code coverage file is not sent to Sonarqube
If you configured step 1 correctly, it is still possible that the generated files are not sent to Sonarqube. The best way to see what is going wrong, is by checking the build logs of the Run Code Analysis and Publish Quality Gate Results steps.
The most common issue, is that Sonarscanner is checking the wrong directory. In the prepare step, please specify where the files are located, like:
sonar.cs.opencover.reportsPaths=$(Build.SourcesDirectory)/**/coverage.opencover.xml
sonar.cs.vstest.reportsPaths=$(Agent.TempDirectory)/*.trx
Build pipeline generates code coverage files would it be coverlet or opencover, the problem is that Azure Devops pipeline creates the reports outside the working directory on the build agent, it uses the _temp folder inside _work whereas soanrquube searches inside the working directory. I am facing this issue with coverlet as well as Visual studio coverage tool. You can read the following 2 threads
https://github.com/coverlet-coverage/coverlet/issues/1399
https://github.com/microsoft/azure-pipelines-tasks/issues/11536.
I posted an issue on snoarqube community site but did not get any response so far. The solution I think is either make the test step in the build pipeline change the directory or configure somewher sonarqube to search the agent build directory _temp which is above the working directory
I have an Azure DevOps pipeline that runs some Postman tests with newman, then publishes the test results to a JUnit XML file. I have this step:
Followed by this step (all other steps have been disabled):
However, when running the pipeline I get this error. Note line 10:
But as shown here, the file exists in that location:
What is going on? Why can't Azure Devops find the file?
I have a TFS build process that incorporates asynchronous remote functional test executions. When the tests have finished executing, I want to publish the generated TRX file in the originating build summary and update the build status (if required).
I've been searching for awhile now and have so far been unsuccessful in finding exactly what I'm looking for: is it possible to publish the TRX file to the build summary via a PowerShell scripted REST API call?
I am afraid there is not a rest api available to publish the TRX file to the build summary page.
During the pipeline execution, the tasks consume the trx file and read the test result to generate a report which you see on the build summary page. So even if you managed to upload the trx file afterwards, the trx file will not be processed and you cannot get test report on the build summary page.
If your pipeline waits for the test execution to complete, you can try using scripts to copy the generated TRX file back to the local agent machine and published via publish tesk results task.
If the pipeline finished before the test execution completed. You can create new pipeline to publish the trx file as workaround. But this will end up showing the test result in a different pipeline build summary page. If it is acceptable to you, you can copy the trx file back to the local agent machine and trigger the new pipeline via Build Queue rest api. You can also consider publishing the trx file to a git repo and add this git repo to the new pipeline as git source.
I have a YAML task that runs tests on .NET solutions in Azure pipeline.
It is meant to run after a build step and execute Unit-tests on that assembly.
The output is a simple XML file with test results that needs to be shown after each build run in the summary tab.
How can I make it recognizable by azure?
For example: MsBuild step is recognized and shown in the summary menu as Build Artifacts and have the option to download them from Azure UI. How can I make Azure recognize my task and show it's artifacts and info too? How many tests ran and info in the title and when I click drop it will show artifacts
Summary menu after build run on Azure DevOps server
You can use ##vso[task.uploadsummary]local file path to upload summary to the build summary. However you may need to write the contents of xml file into md file. please check here to learn more upload tasks.
- powershell: '##vso[task.uploadsummary]path to test result'
However test results isnot intended to be displayed on the summary page.
You must have noticed that there is a Test tab beside Summary tab. Usually the test results will be automatically published by Vstest tasks and the test results will be displayed in Test tab.
You can also publish your test results using task PublishTestResults as mentioned by #4c74356b41. You test results will then be displayed in Test tab.
If you are asking how to make your test results visible in the build summary, there is task meant for that specifically.
- task: PublishTestResults#2
inputs:
testResultsFormat: 'JUnit' # Options: JUnit, NUnit, VSTest, xUnit, cTest
testResultsFiles: '**/TEST-*.xml'
your test results should be compatible with one of the supported versions
I have a azure pipeline build that runs the following command.
pytest tests --junitxml=junit/test-results.xml --cov --cov-report=xml --cov-report=html
I know this generates the correct coverage report as azure produces it as an artifact which i can download. The problem is that azure seems to have an issue with the coverage report. i get the following warnings:
##[warning]Invalid results file. Make sure the result format of the file '/home/vsts/work/1/s/test-cov.xml' matches 'JUnit' test results format.
##[warning]No code coverage results were found to publish.
In the code coverage tab after the pipeline has finished the html output is there but is is incorrectly displayed.
How do i fix this?