I am going to add code coverage to a repo, so when a PR is created the CI is going to generate code coverage reports with lcov.
What is the best way to show this metric in a PR to the dev?
In the end, I had to make a python script to:
1- Look for packages with tests.
2- For each language have a different strategy (I had Python 2, Python 3 and C++ tests in the same repository).
3- Combine all the coverage results in one folder.
4- With regex look for the coverage percentages.
5- Build a nice table with the data and show it via a PR comment.
Graphs and more fancy output can also be done after the regex step.
Related
Just completed a GitHub workflow using more of them are actions, but also with one bash script.
When writing the workflow, it seems much quicker use bash script than actions.(since some actions are just do one thing. ) Why are the some reasons that we just need GitHub actions rather than bash script or python script trigger?
Or we are just supposed to use script languages for most part, then use GitHub actions for small portion of the whole workflow?
Interesting but not easy to answer with more information about what your goal is. The right answer might depend on your use case.
I have not used GitHub actions yet. Let me try to explain it anyway, starting pretty high level. Unfortunately, there's no option to add a table of contents ;) Please let me know if this helps.
1. What are GitHub Actions for?
From this "What is GitHub Actions? Benefits and examples" PDF file
GitHub Actions is a CI/CD tool for the GitHub flow. You can use it to integrate and deploy code changes to a third-party cloud application platform as well as test, track, and manage code changes. GitHub Actions also supports third-party CI/CD tools, the container platform Docker, and other automation platforms.
From docs.github.com
GitHub Actions is a continuous integration and continuous delivery (CI/CD) platform that allows you to automate your build, test, and deployment pipeline. You can create workflows that build and test every pull request to your repository or deploy merged pull requests to production. [...]
GitHub Actions goes beyond just DevOps and lets you run workflows when other events happen in your repository.
2. Continuous Integration/Continuous Deployment (CI/CD)
Usually, people run CI/CD tools to build, deploy, test, and run other tasks while doing that. We use another 3rd party CI/CD pipeline using Rake to build, test, and check links. Our pipeline invokes these small scripts you mention.
3. GitHub actions and scripts
From Essential features of GitHub Actions
If your job generates files that you want to share with another job in the same workflow, or if you want to save the files for later reference, you can store them in GitHub as artifacts. Artifacts are the files created when you build and test your code. For example, artifacts might include binary or package files, test results, screenshots, or log files. Artifacts are associated with the workflow run where they were created and can be used by another job. All actions and workflows called within a run have write access to that run's artifacts.
Here's the key point, I guess. You can really do a lot of crazy stuff within a workflow. All is related/specific to GitHub. Workflows are event-driven, meaning that you can run a series of commands after a specified event has occurred. For example, every time someone creates a pull request, you can automatically run a command that executes a test or other script.
4. GitHub action workflow and scripts
You can include different scripts in your workflow, e.g. using
Javascript: https://github.com/actions/github-script
Python: https://github.com/marketplace/actions/run-python-script
5. (Complex) Examples
You can check out the repository for docs.github.com for some more complex examples, see action-scripts and workflow folders. GitHub themselves seems to use it pretty heavily.
6. Advantages/Disadvantages of GitHub actions
OR: Differences to other CI tools
It took some time to find something not marketing-ish. Key points are:
beginner-friendly using YAML config files
no need to set up your own CI pipeline
You can check out this SO post from 2019 for a list of what's good and bad about GitHub actions.
In short - for readability and the DRY ("Don't repeat yourself") principle.
It's more or less the same as using functions in programming.
I can agree that some trivial actions are useless.
But "actions/checkout" for example is priceless!
I have a Node.js project, which includes Istanbul, a tool that can generate a code coverage report. I would like to see which lines are not covered and which are covered by tests in my GitHub Pull Requests for this project.
Istanbul allows me to see the coverage locally in the coverage/lcov-report directory, but I would like to see this the "Files Changed" tab of my Pull Requests in GitHub.
There are some tools that provide visual line-by-line coverage:
CodeCov does this, but requires that I upload a coverage report to their servers.
Covaralls does also, but requires that you give them access to your repository.
Jest coverage report uses GitHub actions to add annotations, but cannot add visual indications (e.g. highlighting) of line-by-line test coverage.
Is there any other way to visualize line-by-line test coverage in a GitHub Pull Request?
I am actually using circle ci to launch my behat tests through docker.
I want to migrate to github actions for cost reasons but i can't find any clue on how i can split my files on github actions when using parallel build.
Here is the command i am using to split my files on circleCI: circleci tests glob "features/**/*.feature" | circleci tests split --split-by=filesize.
Is there any existing command to do it appart from using knapsack pro which i can't use.
Thanks!
Try using split_tests, a tool similar to circleci tests glob.
It has the same set of features: split files according to filename, lines count or timing of a previous test result.
You'll probably need to configure your Github-Actions ci.yml to:
Download a binary release of split_tests
Create a matrix with a number of nodes,
For each node, run the tests using the split_tests partial files list,
(Optionally) Save the JUnit test report to a cache, to re-use it in the next run for timing.
I have a Python project hosted on GitHub, which I test using Azure Pipelines (here is a link to the configuration).
I am running unit tests with pytest and get coverage results, which I upload in my pipeline. I get an overall code coverage percentage in the pipeline execution summary (example), but I don't see a "Code Coverage" tab that contains the details of which lines are covered. The build artifacts contain the HTML code coverage report, which contains this information, but it'd be nicer to see it in the pipeline run page itself.
Here is the invocation of pytest:
pytest tests --doctest-modules --junitxml=junit/test-results.xml
--cov=callgraph --cov-report=xml --cov-report=html
What am I missing?
that doesnt work like that, unfortunately. at least that was the case 6 months ago.
https://github.com/Microsoft/azure-pipelines-tasks/issues/7703
I used to be developer long ago but for last 10 years working on system ops. I am planning to move into devops and trying to sharpen my saw. However, when it comes to jenkins and specially static code analysis, code coverage, automated test and code review, I get so much confused.
Lets start from automated test ( for simplicity take unit test). I understand that we write a separate class file for unit test. But how does that test is carried out? Will jenkins create a jvm where the newly build artifact is deployed and the tests are run against it? or will the test be run against code ( I do not think but still want to clarify)?
I downloaded one example application with maven and codertura from github and build the project. When the build was completed, it publishes code coverage report.
I have not done any post build, for deploying the artifact. So, I am not sure how it works, and what did it do and how?
Thanks
J
Here is a common flow that you can follow to achieve your requirement.
Work with code --> Push to gerrit for review --> Jenkins gerrit trigger plugin get triggered --> The corresponding job will checkout the code you committed and do the compile, package, unit test, deploy to artifactory --> Execute the sonar build to analysis the code quality, static analysis, code coverage...
Br,
Tim