I want to run coverage for all files in a directory.
For instance, I have the following directory structure:
root_dir/
tests/
test1.py
test2.py
code_dir/
There are some python files in tests directory. I want to run them together using coverage run and generate a report.
Individually, I can do like this:
coverage run tests/test1.py
coverage run tests/test2.py
and generate a report.
How can I do this with a single command?
Thanks.
You should use a test runner to find and run those tests. Either pytest, or python -m unittest discover will do that for you.
Related
I want to add codecov to this project. Yet, codecov says here that it can not process my coverage.xml file that I created with this command: pytest python/tests -v --junitxml=coverage.xml in the Travis CI script.
Everything prior to that like providing my token seems to work as suggested in the TravisCI build here.
I thought this could perhaps be a problem with the paths but I included a potential fix in the codecov.yml and nothing changed.
Therefore, I do not think that the script codecov.yml, travis.yml, and utils/travis_runner.py are part of the problem.
The --junitxml option is for generating reports in JUnit format. Use the option --cov-report to generate coverage reports. pytest-cov allows passing --cov-report multiple times to generate reports in different formats. Example:
$ pip install pytest pytest-cov
$ pytest --cov=mypkg --cov-report term --cov-report xml:coverage.xml
will print the coverage table and generate the Cobertura XML report which is CodeCov-compatible.
I am currently running tests through Visual Studio interface, and that provides easy option to run only certain test. I want to do same, but through console / powershell.
Right now if I do dotnet test that will run all discovered tests. Is it possible to do something similar like dotnet test MyTest which would only run specific test?
Command dotnet test has switch --filter that allows you to filter tests and get only those that you want to run. So you can do:
dotnet test -t
To see full list of tests available, and then in order to just run specific ones you can do:
dotnet test --filter "FullyQualifiedName~ShouldInterconnect"
Which will execute only the tests containing ShouldInterconnect in the name.
I'm looking at moving from unittest to pytest. One thing I like to do, is to do a setup.py install and then run the tests from the installed modules, rather than directly from the source code. This means that I pick up any files I've forgotten to include in MANIFEST.in.
With unittest, I can get the test runner to do test discovery by specifying the root test module. e.g. python -m unittest myproj.tests
Is there a way to do this with pytest?
I'm using the following hack, but I wish there was a built in cleaner way.
pytest $(python -c 'import myproj.tests; print(myproj.tests.__path__[0])')
The Tests as part of application section of pytest good practices says if your tests are available at myproj.tests, run:
py.test --pyargs myproj.tests
With pytest you can instead specify the path to the root test directory. It will run all the tests that pytest is able to discover. You can find more detail in the pytest good practices
I use "newman" to run API tests on Jenkins server. It's very easy for me, I write test scripts in "Postman" and run my collection in "newman" but I can't provide good reports for my manager. I found "allure report" and I like it. Is there any chance to create allure report if I use "Newman". Does allure support newman?
Looks like no, it's impossible.
Why now (after writed tests) you are looking for report tool? It's activity happen in start automation process when qa team analyze test tools are could be used for automation.
I have a look on https://github.com/postmanlabs/newman and think could you try parse commandline output to text file? And use this output to generate simple-report for manager.
Yes, you can. Follow below steps:
we can generate nice and clean report using Allure-js framework.
1. Installation
$ npm install -g newman-reporter-allure
2. Run the newman cli command to generate Allure results, specify allure in Newman's -r or --reporters option.
$ newman run <Collection> -e <Environment> -r allure
3. Allure results will be generated under folder "allure-results" in the root location. Use allure-commandline to serve the report locally.
$ allure serve
4. To generate the static report web-application folder using allure-commandline
$ allure generate --clean
Report will be generated under folder "allure-report" in the root location
.
Try to use my repository, here's tricky solution, which covers everything:
And add couple more things:
Add 2 files: collection *.json and env *.json
Add +x permissions to start.sh file with chmod command, like chmod +x start.sh
And run script ./start.sh your_collection.json your_env.json
Finally, you will get 2 reports:
HTML report
Allure report
I am trying to use several Dist::Zilla plugins to generate and run author tests. The only problem is that they are generated by dzil build and not run with dzil test. As an example, here are some lines in my dist.ini:
[Test::Compile] ; Create t/00-compile.t
[Test::UnusedVars] ; Create xt/unused-vars.t
[RunExtraTests] ; run the xt/ tests when dzil test is run.
Running dzil test however, only runs one of the created test files- 00-compile.t. unused-vars.t isn't even created until I run dzil build. In order to test all of the author tests created like this, I need to first run dzil build, cd into the new directory, then run dzil test. Then, when something fails, I have to work with the original copy, not the copy created by the build command. Rinse and repeat.
Is there a way to get the author tests to be generated at build time so that I can run them with dzil test without changing directories?
dzil test --release is what you're looking for.
And I would also strongly recommend you to try Dist::Zilla::PluginBundle::TestingMania