I have broken my head trying to figure out how --reuse-db. I have a super-simple Django project with one model Student and the following test
import pytest
from main.models import Student
#pytest.mark.django_db
def test_1():
Student.objects.create(name=1)
assert Student.objects.all().count() == 1
When I run it for the first time with command pytest --reuse-db, the test passes - and I am not surprised.
But when I run the pytest --reuse-db for the second time, I expect that the db is not destroyed and the test fails, because I expect that Student.objects.all().count() == 2.
I am misunderstanding the --reuse-db flag ?
--reuse-db means to reuse the database between N tests within the same test run.
This flag has no bearing on running pytest twice.
Related
TLTR:
The question is maximally easy: Please look at the code base case. Pytest just ignoring this class. How I should run tests on a such class?
I just started switching from a simple python tests (with just assert) to testing with pytest and come across with this problem. Most of my tests is are classes that extending a real classes with test methods. One of my classes inherit from collections.UserDict. Pytest just ignoring this class. How I should run tests on a such class?
# Inheritance from object are ok, Inheritance from dict are not ok. Need dict :(
class TestFoo(dict):
def test_foo(self):
assert 1
output:
/home/david/PycharmProjects/proj/venv/bin/python /snap/pycharm-professional/302/plugins/python/helpers/pydev/pydevd.py --multiprocess --qt-support=auto --client 127.0.0.1 --port 44145 --file /snap/pycharm-professional/302/plugins/python/helpers/pycharm/_jb_pytest_runner.py --path /home/david/PycharmProjects/proj/tests/unit_tests_2.py
Testing started at 11:07 ...
Launching pytest with arguments /home/david/PycharmProjects/proj/tests/unit_tests_2.py --no-header --no-summary -q in /home/david/PycharmProjects/proj/tests
============================= test session starts ==============================
collecting ... collected 0 items
============================= 2 warnings in 0.03s ==============================
Process finished with exit code 5
Empty suite
UPD Thanks for #Teejay Bruno, running tests from pycharm hiding a warning from me:
PytestCollectionWarning: cannot collect test class 'TestFoo' because it has a __init__ constructor
The warning tells you the problem:
PytestCollectionWarning: cannot collect test class 'TestFoo' because it has a __init__ constructor
If I understand what you're trying to do, why not just pass the object as a fixture?
import pytest
#pytest.fixture
def my_dict():
return dict()
class TestFoo:
def test_foo(self, my_dict):
assert len(my_dict) == 0
I have 2 tests. I want to run the only one:
pipenv run pytest -s tmp_test.py::test_my_var
But pytest executes both functions in #pytest.mark.parametrize (in both tests)
How can I force Pytest to execute the only get_my_var() function if I run the only test_my_var?
If I run the whole file:
pipenv run pytest -s tmp_test.py
I want Pytest to execute the code in the following manner:
get_my_var()
test_my_var()
get_my_var_1()
test_my_var_1()
Actually, my functions in #pytest.mark.parametrize make some data preparation and both tests use the same entities. So each function in #pytest.mark.parametrize changes the state of the same test data.
That's why I strongly need the sequential order of running parametrization functions just before corresponding test.
def get_my_var():
with open('my var', 'w') as f:
f.write('my var')
return 'my var'
def get_my_var_1():
with open('my var_1', 'w') as f:
f.write('my var_1')
return 'my var_1'
#pytest.mark.parametrize('my_var', get_my_var())
def test_my_var(my_var):
pass
#pytest.mark.parametrize('my_var_1', get_my_var_1())
def test_my_var_1(my_var_1):
pass
Or how can I achive the same goal with any other options?
For example, with fixtures. I could use fixtures for data preparation but I need to use the same fixture in different tests because the preparation is the same. So I cannot use scope='session'.
At the same time scope='function' results in fixture runs for every instance of parameterized test.
Is there a way to run fixture (or any other function) the only one time for parameterized test before runs of all parameterized instances?
It looks like that only something like that can resolved the issue.
import pytest
current_test = None
#pytest.fixture()
def one_time_per_test_init(request):
test_name = request.node.originalname
global current_test
if current_test != test_name:
current_test = test_name
init, kwargs = request.param
init(**kwargs)
I'm running into a problem like this. I'm currently using pytest to run test cases, and reducing execution time using xdist to run tests in parallel and publishing tests results to TestRail. The issue is when using xdist, pytest-testrail plugin creates Test-Run for each xdist workers and then publishes test cases like Untested.
I tried this hook pytest_terminal_summary to prevent pytest_sessionfinish plugin hook from being call multiple times.
I expect only one test run is created, but still multiple test runs are created.
I ran in to the same problem, but found a kind of workaround with duct tape.
I found that all results are collecting properly in test run, if we run the tests with --tr-run-id key.
If you are using jenkins jobs to automate processes, you can do following:
1) create testrun using testrail API
2) get ID of this test run
3) run the tests with --tr-run-id=$TEST_RUN_ID
I used these docs:
http://docs.gurock.com/testrail-api2/bindings-python
http://docs.gurock.com/testrail-api2/reference-runs
from testrail import *
import sys
client = APIClient('URL')
client.user = 'login'
client.password = 'password'
result = client.send_post('add_run/1', {"name": sys.argv[1], "assignedto_id": 1}).get("id")
print(result)
then in jenkins shell
RUN_ID=`python3 testrail_run.py $BUILD_TAG`
and then
python3 -m pytest -n 3 --testrail --tr-run-id=$RUN_ID --tr-config=testrail.cfg ...
Need to run same test on different devices. Used fixture to give ip addresses of the devices, and all tests run for the IPs provided by fixtures as requests. But at the same time, need to append the test name with the IP address to quickly analyze results. pytest results have test name as same for all params, only in the log or statement we could see the parameter used, is there anyway to change the testname by appending the param to the test name based on the fixture params ?
class TestClass:
def test1():
pass
def test2():
pass
We need to run the whole test class for every device, all test methods in sequence for each device. We can not run each test with paramter cycle, we need to run the whole test class in a parameter cycle. This we achieved by a fixture implementation, but we couldn't rename the tests.
You can read my answer: How to customize the pytest name
I could change the pytest name, by creating a hook in a conftest.py file.
However, I had to use pytest private variables, so my solution could stop working when you upgrade pytest
You don't need to change the test name. The use case you're describing is exactly what parametrized fixtures are for.
Per the pytest docs, here's output from an example test run. Notice how the fixture values are included in the failure output right after the name of the test. This makes it obvious which test cases are failing.
$ pytest
======= test session starts ========
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y
rootdir: $REGENDOC_TMPDIR, inifile:
collected 3 items
test_expectation.py ..F
======= FAILURES ========
_______ test_eval[6*9-42] ________
test_input = '6*9', expected = 42
#pytest.mark.parametrize("test_input,expected", [
("3+5", 8),
("2+4", 6),
("6*9", 42),
])
def test_eval(test_input, expected):
> assert eval(test_input) == expected
E AssertionError: assert 54 == 42
E + where 54 = eval('6*9')
test_expectation.py:8: AssertionError
======= 1 failed, 2 passed in 0.12 seconds ========
I always thought that imperative and declarative usage of xfail/skip in py.test should work in the same way. In the meantime I've noticed that if I write a test that contains an imperative skip the result of the test will always be "xfail" even it the test passes.
Here's some code:
import pytest
def test_should_fail():
pytest.xfail("reason")
#pytest.mark.xfail(reason="reason")
def test_should_fail_2():
assert 1
Running these tests will always result in:
============================= test session starts ==============================
platform win32 -- Python 2.7.3 -- pytest-2.3.5 -- C:\Python27\python.exe
collecting ... collected 2 items
test_xfail.py:3: test_should_fail xfail
test_xfail.py:6: test_should_fail_2 XPASS
===================== 1 xfailed, 1 xpassed in 0.02 seconds =====================
If I understand correctly what is written in the user manual, both test should be "XPASS'ed".
Is this a bug in py.test or am I getting something wrong?
When using the pytest.xfail() helper function you are effectively raising an exception in the test function. Only when you are using the marker it is possible for py.test to execute the test fully and give you an XPASS.