I am very new to pytest.
There is a test_conf dir which has several test config files.
test_conf
test_conf1
test_conf2
Here is my test function.
conf_files function gets all test_conf files from that dir and returns a list.
#pytest.mark.parametrize('test_conf', conf_files())
def test_performance_scenario_1(fixture1, test_conf):
do_some_thing(test_conf)
The fixture1 does setup and teardown for this test.
My proposal is for each test conf file from test_conf, we run test function to do some test against it.
My question is how to pass each element of test_conf to fixture1, since I need to do some initialization in fixture1 setup step which needs the test conf file.
Any help is appreciated.
I think what you're looking for is parameterised fixtures. So instead of passing the conf files to your test, you pass it to the fixture, which does the setup/teardown with the particular conf file.
definition of your fixture1:
#pytest.fixture(scope="module", params=conf_files())
def fixture1(request):
# The pytest fixture `request` gives you access to the params defined in the annotation.
conf_file = request.param
# setup / teardown logic
...
Then in your test, it suffices to only pass the fixture:
def test_performance_scenario_1(fixture1):
# do_some_things
Related
I have 2 tests. I want to run the only one:
pipenv run pytest -s tmp_test.py::test_my_var
But pytest executes both functions in #pytest.mark.parametrize (in both tests)
How can I force Pytest to execute the only get_my_var() function if I run the only test_my_var?
If I run the whole file:
pipenv run pytest -s tmp_test.py
I want Pytest to execute the code in the following manner:
get_my_var()
test_my_var()
get_my_var_1()
test_my_var_1()
Actually, my functions in #pytest.mark.parametrize make some data preparation and both tests use the same entities. So each function in #pytest.mark.parametrize changes the state of the same test data.
That's why I strongly need the sequential order of running parametrization functions just before corresponding test.
def get_my_var():
with open('my var', 'w') as f:
f.write('my var')
return 'my var'
def get_my_var_1():
with open('my var_1', 'w') as f:
f.write('my var_1')
return 'my var_1'
#pytest.mark.parametrize('my_var', get_my_var())
def test_my_var(my_var):
pass
#pytest.mark.parametrize('my_var_1', get_my_var_1())
def test_my_var_1(my_var_1):
pass
Or how can I achive the same goal with any other options?
For example, with fixtures. I could use fixtures for data preparation but I need to use the same fixture in different tests because the preparation is the same. So I cannot use scope='session'.
At the same time scope='function' results in fixture runs for every instance of parameterized test.
Is there a way to run fixture (or any other function) the only one time for parameterized test before runs of all parameterized instances?
It looks like that only something like that can resolved the issue.
import pytest
current_test = None
#pytest.fixture()
def one_time_per_test_init(request):
test_name = request.node.originalname
global current_test
if current_test != test_name:
current_test = test_name
init, kwargs = request.param
init(**kwargs)
With PyTest, you can limit the scope of test collection by passing directories/files/nodeids as command line arguments, e.g., pytest tests, pytest tests/my_tests.py and pytest tests/my_tests.py::test_1. Is it possible to override this behavior from within a plugin, i.e., to set them to something else programmatically?
So far I've attempted setting file_or_dir to my own list within config.option and config.known_args_namespace from the pytest_configure hook, but this appears to have no effect on anything.
You are probably looking for config.args:
# conftest.py
def pytest_configure(config):
config.args = ['foo', 'bar/baz.py::test_spam']
Running pytest now will be essentially the same as running pytest foo bar/baz.py::test_spam. However, putting stuff in pytest.ini would be IMO a better solution:
# pytest.ini
[pytest]
addopts = foo bar/baz.py::test_spam
I'm wrapping pytests in a python program that does some setup and builds the argument list to invoke pytest.main.
arg_list = [ ... ] //build arg_list
pytest.main(args=arg_list)
I also need to pass a configuration object from this wrapper to the tests run by pytest. I was thinking creating a fixture called conf and reference it the test functions
#pytest.fixture
def conf(request):
# Obtain configuration object
def test_mytest(conf):
#use configuration
However, I haven't found a way to pass an arbitrary object to fixtures (only options from the pytest arguments list).
Maybe using a hook? or a plugin injected or initialized from the wrapper?
You can create a module that is shared between your wrapper and your tests or serialize the object first.
Pickle the object and load it before tests
This solution keeps your wrapper and tests mostly independent. You could still execute the tests directly and pass the configuration object from the command line if you want to reproduce the test output for a certain object.
It does not work for all objects, because not all objects can be pickled. See "What can be pickled and unpickled?" for more details. This solution respects the scope of the fixture, because the object is reloaded from disk when the fixture is created.
Add a command line option for the path of the pickled file in conftest.py
import pickle
import pytest
def pytest_addoption(parser):
parser.addoption("--cfg-obj", help="path to the pickled configuration object")
#pytest.fixture
def conf(request):
path = request.config.getoption("--cfg-obj")
with open(path, 'rb') as fp:
return pickle.load(fp)
Pickle the object in wrapper.py and save it in a temporary file.
import pickle
import tempfile
import pytest
config_obj = {"answer": 42}
with tempfile.NamedTemporaryFile(delete=False) as fp:
pickle.dump(config_obj, fp)
fp.close()
args_list = ["tests.py", "--cfg-obj", fp.name]
pytest.main(args=args_list)
Use the conf fixture in tests.py
def test_something(conf):
assert conf == {'answer': 42}
Share the object between the wrapper and the tests
This solution does not seem very "clean" to me, because the tests can't be executed without the wrapper anymore (unless you add a fallback if the object is not set), but it has the advantage that the wrapper and the tests access the same object. This will work for arbitrary objects. It also introduces a possible dependency between your tests if you modify the state of the object, because the scope parameter of the fixture decorator has no effect here (it always loads the same object).
Create a shared.py module which is imported by the tests and the wrapper. It provides a setter and getter for the shared object.
_cfg_obj = None
def set_config_obj(obj):
global _cfg_obj
_cfg_obj = obj
def get_config_obj():
global _cfg_obj
return _cfg_obj
Set the shared object in wrapper.py
import pytest
from shared import set_config_obj
set_config_obj({"answer": 42})
args_list = ["tests.py"]
pytest.main(args=args_list)
Load the shared object in your conf fixture
import pytest
from shared import get_config_obj
#pytest.fixture
def conf():
return get_config_obj()
def test_something(conf):
assert conf == {"answer": 42}
Note that the shared.py module does not have to be outside your tests directory. If you turn the tests directory into a package by adding __init__.py files and add the shared object there, then you can import the tests package from your wrapper and set it with tests.set_config_obj(...).
I would like to create a separate log file for each test method. And i would like to do this in the conftest.py file and pass the logfile instance to the test method. This way, whenever i log something in a test method it would log to a separate log file and will be very easy to analyse.
I tried the following.
Inside conftest.py file i added this:
logs_dir = pkg_resources.resource_filename("test_results", "logs")
def pytest_runtest_setup(item):
test_method_name = item.name
testpath = item.parent.name.strip('.py')
path = '%s/%s' % (logs_dir, testpath)
if not os.path.exists(path):
os.makedirs(path)
log = logger.make_logger(test_method_name, path) # Make logger takes care of creating the logfile and returns the python logging object.
The problem here is that pytest_runtest_setup does not have the ability to return anything to the test method. Atleast, i am not aware of it.
So, i thought of creating a fixture method inside the conftest.py file with scope="function" and call this fixture from the test methods. But, the fixture method does not know about the the Pytest.Item object. In case of pytest_runtest_setup method, it receives the item parameter and using that we are able to find out the test method name and test method path.
Please help!
I found this solution by researching further upon webh's answer. I tried to use pytest-logger but their file structure is very rigid and it was not really useful for me. I found this code working without any plugin. It is based on set_log_path, which is an experimental feature.
Pytest 6.1.1 and Python 3.8.4
# conftest.py
# Required modules
import pytest
from pathlib import Path
# Configure logging
#pytest.hookimpl(hookwrapper=True,tryfirst=True)
def pytest_runtest_setup(item):
config=item.config
logging_plugin=config.pluginmanager.get_plugin("logging-plugin")
filename=Path('pytest-logs', item._request.node.name+".log")
logging_plugin.set_log_path(str(filename))
yield
Notice that the use of Path can be substituted by os.path.join. Moreover, different tests can be set up in different folders and keep a record of all tests done historically by using a timestamp on the filename. One could use the following filename for example:
# conftest.py
# Required modules
import pytest
import datetime
from pathlib import Path
# Configure logging
#pytest.hookimpl(hookwrapper=True,tryfirst=True)
def pytest_runtest_setup(item):
...
filename=Path(
'pytest-logs',
item._request.node.name,
f"{datetime.datetime.now().strftime('%Y%m%dT%H%M%S')}.log"
)
...
Additionally, if one would like to modify the log format, one can change it in pytest configuration file as described in the documentation.
# pytest.ini
[pytest]
log_file_level = INFO
log_file_format = %(name)s [%(levelname)s]: %(message)
My first stackoverflow answer!
I found the answer i was looking for.
I was able to achieve it using the function scoped fixture like this:
#pytest.fixture(scope="function")
def log(request):
test_path = request.node.parent.name.strip(".py")
test_name = request.node.name
node_id = request.node.nodeid
log_file_path = '%s/%s' % (logs_dir, test_path)
if not os.path.exists(log_file_path):
os.makedirs(log_file_path)
logger_obj = logger.make_logger(test_name, log_file_path, node_id)
yield logger_obj
handlers = logger_obj.handlers
for handler in handlers:
handler.close()
logger_obj.removeHandler(handler)
In newer pytest version this can be achieved with set_log_path.
#pytest.fixture
def manage_logs(request, autouse=True):
"""Set log file name same as test name"""
request.config.pluginmanager.get_plugin("logging-plugin")\
.set_log_path(os.path.join('log', request.node.name + '.log'))
I want to get the test name and test result during runtime.
I have setup and tearDown methods in my script. In setup, I need to get the test name, and in tearDown I need to get the test result and test execution time.
Is there a way I can do this?
You can, using a hook.
I have these files in my test directory:
./rest/
├── conftest.py
├── __init__.py
└── test_rest_author.py
In test_rest_author.py I have three functions, startup, teardown and test_tc15, but I only want to show the result and name for test_tc15.
Create a conftest.py file if you don't have one yet and add this:
import pytest
from _pytest.runner import runtestprotocol
def pytest_runtest_protocol(item, nextitem):
reports = runtestprotocol(item, nextitem=nextitem)
for report in reports:
if report.when == 'call':
print '\n%s --- %s' % (item.name, report.outcome)
return True
The hook pytest_runtest_protocol implements the runtest_setup/call/teardown protocol for the given test item, including capturing exceptions and calling reporting hooks. It is called when any test finishes (like startup or teardown or your test).
If you run your script you can see the result and name of the test:
$ py.test ./rest/test_rest_author.py
====== test session starts ======
/test_rest_author.py::TestREST::test_tc15 PASSED
test_tc15 --- passed
======== 1 passed in 1.47 seconds =======
See also the docs on pytest hooks and conftest.py.
unittest.TestCase.id() this will return the complete Details including class name , method name .
From this we can extract test method name.
Getting the results during can be achieved by checking if there any exceptions in executing the test.
If the test fails then there wil be an exception if sys.exc_info() returns None then test is pass else test will be fail.
Using pytest_runtest_protocol as suggested with fixture marker solved my problem. In my case it was enough just to use reports = runtestprotocol(item, nextitem=nextitem) within my pytest html fixture. So to finalize the item element contains the information you need.
Many Thanks.