I am in the process of trying to figure out if we can use the Allure plugin for TeamCity to show code coverage, and completed tests in our build pipeline.
All of the tests for our web services are written in CoffeeScript using Hapijs Lab.
When we run the tests now we use 'npm test' and once the tests complete a coverage.html document is generated.
Does anyone know of a way that I can convert this coverage.html report into a format that Allure would understand? From what I have read it seems that Allure is expecting XML. Does anyone have experience in converting an existing report into XML?
When you run your lab tests you can add the flag:
lab tests.js --reporter junit
And you will get a junit style xml output.
Then you need to make sure you have the JUnit adapter installed for allure. You can find info about that here:
https://github.com/allure-framework/allure-core/wiki/JUnit
Related
Hi guys i have a problem with my code coverage of my azure pipeline. When the CI gets trigged for the master branch the agent has a job to test the .net core. When i use this test i have to configure the route and also enable the code coverage. After the CI has been done it ended succesfully and i can download a file with the code coverage. The problem i have is i cant open this file, not with visual studio or anything else. Aynone that knows how i can fix this problem and maby i didnt configure it right?
Picture of my settings:
picture of the file
picture of the error https://i.stack.imgur.com/fxqdI.png
Seems you are using the wrong configuration. I'm afraid --collect"Code coverage" will not work.
You could add a .NET core test task and add /p:CollectCoverage=true argument then use report generate task for adding code coverage reports.
More detail steps, please refer below tutorials:
ASP.NET Core code coverage reports on Azure DevOps
Azure DevOps, unit testing and code coverage with .Net Core
To generate code coverage please follow this steps
Restore nuget packages
Build the app
Run unit tests (You need to add nuegte package to test projects -> coverlet.collector to use 'XPlat Code Coverage')
Install report generator
Run report generator
Publish code coverage
I am using the ReportGenerator task in my Azure DevOps build to generate a code coverage report. The problem is that the generated report is empty.
In my VsTest task that runs the unit tests I have checked the option Code coverage enabled. The .coverage file is correctly generated and the unit tests are all successfully run. However, the generated code coverage report shows that no code was covered by any of the unit tests.
Here is a screenshot of my code coverage Report Generator task.
And here is the output from the code coverage report.
Here is a screenshot of the Azure DevOps log file for the task showing that it executed correctly without any errors.
Interestingly the log does show the following
Analyzing 0 classes
Is this because ReportGenerator cannnot find the assemblies / classes? I've tried specififying a value for Source directories but this has made no difference.
Why is the code coverage report showing no code coverage?
ReportGenerator can not handle .coverage files.
You can find the supported formats here: https://github.com/danielpalme/ReportGenerator#supported-input-and-output-file-formats
You can use a tool like OpenCover or coverlet to generate a coverage report in a format that ReportGenerator is able to parse.
Or you can convert the .coverage file format. Here you find instructions how this can be done: https://github.com/danielpalme/ReportGenerator/wiki/Visual-Studio-Coverage-Tools
I have a CakeBuild build script which compiles a Visual Studio 2015 solution then uses the Cake add in for NUnit 3 to run the unit tests and generate the TestResult.xml file. The tests pass successfully when run locally and the report is generated.
This same build script is being run by TeamCity (version 10.0.2), with the build agent also successfully running the tests (all passing) and the TestReport.xml file being generated.
The TeamCity build configuration has been configured via the XML Report Processing Build Feature to import the NUnit report. The build log suggests this has also occurred successfully. Example of the log:
[10:06:41]NUnit report watcher
[10:06:41][NUnit report watcher] 1 report found for paths:
[10:06:41][NUnit report watcher] **/TestResult.xml
[10:06:41][NUnit report watcher] Successfully parsed
[10:06:41][Successfully parsed] 1 report
[10:06:41][Successfully parsed] build\TestResult.xml
However, there is no reference to any of the tests in the build run - the Test tab is missing completely and no other references to the tests at all.
Is there anything else that needs to be configured to get the test results to show up?
We've had the same issue and found the solution in https://github.com/nunit/teamcity-event-listener/issues/39
Main point is to make sure to include the package NUnit.Extension.TeamCityEventListener which reports the results to TeamCity.
In addition the Nunit Runner Settings should include
NUnit3(path, new NUnit3Settings {
TeamCity = true
});
I am planning to implement Jacoco in my project. So just wanted to know that is it mandatory that i have to write Junit test cases or i can use existing regression test cases in other tools such as selenium or soap-Ui Pro.
In another words i wanted to know whether we can get code coverage using black box test cases. And the answer is yes. To get coverage for black box test cases using jacoco, you will have to inject jacoco agent in you AUT shell/bat application start up file. Which in turn creates a jacoco.exec file containing information about coverage.Pass this file to jacoco API ReportGenerator and you can create html reports about coverage.
I remember doing this in an older version of Team City in which the NUnit Runner failed, so I tried running NUnit using the Command Line Runner instead.
I think there was an option in the Command Line Runner settings which let you specify the kind of output file: NUnit result.xml was one of them. Doing this, you could see the test result breakdown in the Web GUI, just like when the NUnit Runner worked.
But I don't see this in the latest Team City (6.5.5). Can it still be done?
PS: Before anyone asks, I can't use Team City's NUnit Runner because NUnit has a bug which will be fixed in the soon to come 2.6.0 version. Said bug does not occurr when running nunit-console
This could be done using importdata TeamCity message, this is really helpful when you are running NUnit tests manually executing nunit-console.exe from an MSBuild script, for instance using NUnit MSBuild Community Task, but keep in mind that TeamCity provides built in NUnit tests runner so you do not need this low level TeamCity scripting. Anyway you can import report in this way:
<Message
Text="##teamcity[importData type='nunit' path='...\TestResults.xml']"
Importance="High"/>
See Importing XML Reports
If you prefer to run code coverage, code inspection, test tools or
duplicate finders directly from build script, not as a build runner,
you can use the importData service messages to import generated xml
reports into TeamCity.
The functionality was still there, found it by accident when looking at another build configuration. Importing an XML is now a "Build Feature", see: