I use Gitlab CI t report Code Coverage. The CI script prints the output to the console which is then fetched by GitLab via regular expression.
As the result, I get the following coverage description in every merge request:
The question is - how can I track the trend (or change) of this value over a period of time.
I'd like to show the chart with coverage change for e.g. last month, so I'd see, whether the project's code quality is improved or decreased. Looking to something similar to this image:
Which GitLab plugins or open source tools allow to get such a picture / report? Is it possible to achieve a similar result by using just gitlab alone?
Related
I'm using flutter for this project and when I do the tests I can see the coverage but in the gitlab badge I can't see it.
Gitlab badge
here is my cicd:
cicd with test job
here is my cicd configuration:
Test coverage parsing
Here is my badge configuration
Badge image URL
In the output of test job this is the result:
Test job output
Can someone help me with this please?
Thank you!
Try changing your test coverage parsing regular expression to lines\.*: (\d+.\d+)% the parenthesis (capture group) are important to know what number to pick up.
I have a pipeline that runs on pull requests and executes tests , it also produces an xml execution report (it is a node project and tests run via jest) , and coverage report that I use for sonarcloud. However , the developers want to be able to see the tests results on each pr and see what tests failed in particular, so my question is whether it is possible to somehow export and display those results in github or sonarcloud
Please refer to the following two methods to get the test cases metric in sonarcloud:
You can add sonar.testExecutionReportPaths in Additional Properties option in Prepare Analysis Configuration task. About sonar.coverageReportPaths
For SonarQube to pick up and analyse the content of the report file, you need to inform the sonar scanner. About the location of the XML file. To do so, edit the sonarqube.properties file. Please refer to this.
coverage reporting:
sonar.testExecutionReportPaths=reports/xxx/xxx.xml
In addition, you can also refer to this similar case.
I am going to add code coverage to a repo, so when a PR is created the CI is going to generate code coverage reports with lcov.
What is the best way to show this metric in a PR to the dev?
In the end, I had to make a python script to:
1- Look for packages with tests.
2- For each language have a different strategy (I had Python 2, Python 3 and C++ tests in the same repository).
3- Combine all the coverage results in one folder.
4- With regex look for the coverage percentages.
5- Build a nice table with the data and show it via a PR comment.
Graphs and more fancy output can also be done after the regex step.
I have a Python project hosted on GitHub, which I test using Azure Pipelines (here is a link to the configuration).
I am running unit tests with pytest and get coverage results, which I upload in my pipeline. I get an overall code coverage percentage in the pipeline execution summary (example), but I don't see a "Code Coverage" tab that contains the details of which lines are covered. The build artifacts contain the HTML code coverage report, which contains this information, but it'd be nicer to see it in the pipeline run page itself.
Here is the invocation of pytest:
pytest tests --doctest-modules --junitxml=junit/test-results.xml
--cov=callgraph --cov-report=xml --cov-report=html
What am I missing?
that doesnt work like that, unfortunately. at least that was the case 6 months ago.
https://github.com/Microsoft/azure-pipelines-tasks/issues/7703
I have a requirement of settings code coverage percentage for my unit test cases running in VSTS through the build.yml file.
Right now the build is passing for any code coverage percentage that's coming. I want to restrict this and fail the build if the percentage is not 70%.
Can you help me with this?
You need to publish the test results (and code coverage is in that) and then use another task or release gate to check those results. You can do this with SonarQube Quality gate (needs a SonarQube server) or you can try the marketplace.
For example this task: https://marketplace.visualstudio.com/items?itemName=mspremier.BuildQualityChecks
You can also check it yourself (using PowerShell for example) via the API: https://learn.microsoft.com/en-us/rest/api/vsts/test/code%20coverage?view=vsts-rest-5.0