What is the proper way of testing simple CRUD operations with Pytest to ensure correct reads and writes? Function object instead of value is returned - pytest

Using Pytest, I am trying to test a simple database read function and an update function.
The update function needs to be run first to ensure the test database has the values we will test for in our read function.
In /src there is my main script, here are the read, update, delete functions:
async def read(self, collection, query=None, mods=None):
Table = self._cache.classes[collection]
async with self._session() as session:
matches = set(Table.__mapper__.columns.keys()) & set(mods['fields'])
for m in matches:
q = select(getattr(Table, m))
q = q.filter_by(**query)
result = await session.execute(q)
return [row for row in result]
async def update(self, collection, query, data):
Table = self._cache.classes[collection]
async with self._session() as s:
q = sa_update(Table).filter_by(**query)
q = q.values(**data)
await s.execute(q)
await s.commit()
async def delete_collection(self, table_name, syncro_engine=sync_engine):
table = self.retrieve_table_obj(table_name)
async with self._engine.begin() as conn:
await conn.run_sync(Base.metadata.drop_all(syncro_engine, [table], checkfirst=True))
When running read the output is [('Wonderland',)] so this is what I am testing in assert in Pytest.
In /test I have a conftest.py file that has these three functions:
#pytest.fixture(scope='function')
def test_update():
def get_update(collection, query, data):
return asyncio.run(DB.update(collection, query, data))
return get_update
#pytest.fixture(scope='function')
def test_read():
def get_read(collection, query, mods):
return asyncio.run(DB.read(collection, query, mods))
return get_read
#pytest.fixture(scope='function')
def drop_table():
def get_delete_tables(table_name, sync_engine):
return asyncio.run(DB.delete_collection(table_name, sync_engine))
return get_delete_tables
Then the tests get run in test_db.py:
# Tests 'update' by updating user 'Alice' with 'org' 'Wonderland'
# Doesn't return anything, using it to ensure the correct values are in test_db to test the read function
#pytest.mark.parametrize('function_input, expected',
[("'user', {'name': 'Alice'}, {'org': 'Wonderland'})", "[('Wonderland',)]")])
def test_update(test_update, function_input, expected):
test_update
# Tests 'read' by reading the just-updated values
#pytest.mark.parametrize('function_input, expected',
[("'user', query={'name': 'Alice'}, mods={'fields': ['name', 'org']})", "[('Wonderland',)]")])
def test_read(test_read, function_input, expected):
assert test_read == expected
# Tests that tables are dropped cleanly by dropping a given table then checking for that table name
#pytest.mark.parametrize('fn_input1, fn_input2, expected',
[('image', sync_engine, 'image')])
def test_drop_table(drop_table, collection_names, fn_input1, fn_input2, expected):
# First drop the table
drop_table(fn_input1, fn_input2)
# Now check that it no longer exists
with pytest.raises(KeyError) as error_info:
assert fn_input1 in collection_names
raise KeyError
assert error_info.type is KeyError
Running test_drop_table does not drop anything from test_db even though it's being passed the correct database test_db and its counterpart in the main script in /src works properly.
When I run this test_read test it fails:
> assert test_read == expected
E assert <function test_read.<locals>.get_read at 0x7f1fa6beed08> == "[('Wonderland',)]"
How can I access the actual returned value to assert against, i.e. [('Wonderland',)], rather than the <function object>?

Related

R1703:The if statement can be replaced with 'return bool(test)'

I write a function to check if the file exists, but pylint throws a message:"R1703:The if statement can be replaced with 'return bool(test)". What the message means? In addition, how to write a pytest script to test my code below?
def is_file_valid(file):
"""
check if input file exits
:param file: file
:return: True/False
"""
if os.path.exists(file):
return True
else:
return False
I've tried if ...==1: but it seems not work.
def is_file_valid(file):
"""
check if input file exits
:param file: file
:return: True/False
"""
if os.path.exists(file)==1:
return True
else:
return False
For pytest script, I write...
file_test = 'test.txt' # actually this file does not exist.
def test_is_file_valid(file_test):
# test is_file_valid()
assert os.path.exists(file_test) is False, "No this file"
os.remove(file_test)
pytest only shows the following message:
ERROR test/unit/test_io_utils.py::test_is_file_valid
Do you guys have any ideas how to figure it out?
The suggestion means that your function could be rewritten to return a boolean result without the need for an if statement. In your case os.path.exists already returns a boolean so it's as simple as returning its result directly.
def is_file_valid(file):
"""
check if input file exits
:param file: file
:return: True/False
"""
return os.path.exists(file)
However, whether the function in this state actually makes sense, is in my opinion questionable because I don't see any "added value" compared to using os.path.exists(file) directly.
As for how to test it... create (or not) a file in Pytest's temporary folder:
def test_is_file_valid_true(tmp_path):
file = tmp_path / 'file.txt'
file.touch()
assert is_file_valid(file)
def test_is_file_valid_false(tmp_path):
file = tmp_path / 'file.txt'
assert not is_file_valid(file)

How to repeat each test with a delay if a particular Exception happens (pytest)

I have a load of test which I want to rerun if there is a particular exception. The reason for this is that I am running real API calls to a server and sometimes I hit the rate limit for the API, in which case I want to wait and try again.
However, I am also using a pytest fixture to make each test is run several times, because I am sending requests to different servers (the actual use case is different cryptocurrency exchanges).
Using pytest-rerunfailures comes very close to what I need...apart from that I can't see how to look at the exception of the last test run in the condition.
Below is some code which shows what I am trying to achieve, but I don't want to write code like this for every test obviously.
#pytest_asyncio.fixture(
params=EXCHANGE_NAMES,
)
async def client(request):
exchange_name = request.param
exchange_client = get_exchange_client(exchange_name)
return exchange_client
def test_something(client):
test_something.count += 1
### This block is the code I want to
try:
result = client.do_something()
except RateLimitException:
test_something.count
if test_something.count <= 3:
sleep_duration = get_sleep_duration(client)
time.sleep(sleep_duration)
# run the same test again
test_something()
else:
raise
expected = [1,2,3]
assert result == expected
You can use the retry library to wrap your actual code in:
#pytest_asyncio.fixture(
params=EXCHANGE_NAMES,
autouse=True,
)
async def client(request):
exchange_name = request.param
exchange_client = get_exchange_client(exchange_name)
return exchange_client
def test_something(client):
actual_test_something(client)
#retry(RateLimitException, tries=3, delay=2)
def actual_test_something(client):
'''Retry on RateLimitException, raise error after 3 attempts, sleep 2 seconds between attempts.'''
result = client.do_something()
expected = [1,2,3]
assert result == expected
The code looks much cleaner this way.

how to use a pytest function to test different site using a different set of test data for each site such as staging/production

I have a set of pytest functions to test APIs, and test data is in a json file loaded by the pytest.mark.parametrize. Because the staging, production, and pre_production have different data but are similar, I want to save the test data in a different folder and use the same file name, in order to keep the python function clean. Site information is a new option from the command line of pytest. It doesn't work, pytest.mark.parametrize can't get the right folder to collect the test data.
This is in the conftest.py
#pytest.fixture(autouse=True)
def setup(request, site):
request.cls.site = site
yield
def pytest_addoption(parser):
parser.addoption("--site", action="store", default="staging")
#pytest.fixture(scope="session", autouse=True)
def site(request):
return request.config.getoption("--site")
This is in the test cases file:
#pytest.mark.usefixtures("setup")
class TestAAA:
#pytest.fixture(autouse=True)
def class_setup(self):
self.endpoint = read_data_from_file("endpoint.json")["AAA"][self.site]
if self.site == "production":
self.test_data_folder = "SourcesV2/production/"
else: // staging
self.test_data_folder = "SourcesV2/"
testdata.set_data_folder(self.test_data_folder)
#pytest.mark.parametrize("test_data", testdata.read_data_from_json_file(r"get_source_information.json"))
def test_get_source_information(self, test_data):
request_url = self.endpoint + f"/AAA/sources/{test_data['sourceID']}"
response = requests.get(request_url)
print(response)
I can use pytest.skip to skip the test data which is not for the current site.
if test_data["site"] != self.site:
pytest.skip("this test case is for " + test_data["site"] + ", skiping...")
But it will need to put all the test data in one file for staging/production/pre-production, and there will be a lot of skipped tests in the report, which is not my favorite.
Do you have any idea to solve this? How to pass a different file name to the parametrize according to the site?
Or, at least, how to let the skipped test not write logs in the report?
Thanks
The parametrize decorator is evaluated at load time, not at run time, so you will not be able to use it directly for this. You need to do the parametrization at runtime instead. This can be done using the pytest_generate_tests hook:
def pytest_generate_tests(metafunc):
if "test_data" in metafunc.fixturenames:
site = metafunc.config.getoption("--site")
if site == "production":
test_data_folder = "SourcesV2/production"
else:
test_data_folder = "SourcesV2"
# this is just for illustration, your test data may be loaded differently
with open(os.path.join(test_data_folder, "test_data.json")) as f:
test_data = json.load(f)
metafunc.parametrize("test_data", test_data)
class TestAAA:
def test_get_source_information(self, test_data):
...
If loading the test data is expansive, you could also cache it to avoid reading it for each test.

H2 database content is not persisting on insert and update

I am using h2 database to test my postgres slick functionality.
I created a below h2DbComponent:
trait H2DBComponent extends DbComponent {
val driver = slick.jdbc.H2Profile
import driver.api._
val h2Url = "jdbc:h2:mem:test;MODE=PostgreSQL;DB_CLOSE_DELAY=-1;DATABASE_TO_UPPER=false;INIT=runscript from './test/resources/schema.sql'\\;runscript from './test/resources/schemadata.sql'"
val logger = LoggerFactory.getLogger(this.getClass)
val db: Database = {
logger.info("Creating test connection ..................................")
Database.forURL(url = h2Url, driver = "org.h2.Driver")
}
}
In the above snippet i am creating my tables using schema.sql and inserting a single row(record) with schemadata.sql.
Then i am trying to insert a record into the table as below using my test case:
class RequestRepoTest extends FunSuite with RequestRepo with H2DBComponent {
test("Add new Request") {
val response = insertRequest(Request("XYZ","tk", "DM", "RUNNING", "0.1", "l1", "file1",
Timestamp.valueOf("2016-06-22 19:10:25"), Some(Timestamp.valueOf("2016-06-22 19:10:25")), Some("scienceType")))
val actualResult=Await.result(response,10 seconds)
assert(actualResult===1)
val response2 = getAllRequest()
assert(Await.result(response2, 5 seconds).size === 2)
}
}
The above assert of insert works fine stating that the record is inserted. But the getAllRequest() assert fails as the output still contains the single row(as inserted by schemadata.sql) => which means the insertRequest change is not persisted. However the below statements states that the record is inserted as the insert returned 1 stating one record inserted.
val response = insertRequest(Request("CMP_XYZ","tesco_uk", "DM", "RUNNING", "0.1", "l1", "file1",
Timestamp.valueOf("2016-06-22 19:10:25"), Some(Timestamp.valueOf("2016-06-22 19:10:25")),
Some("scienceType")))
val actualResult=Await.result(response,10 seconds)
Below is my definition of insertRequest:
def insertRequest(request: Request):Future[Int]= {
db.run { requestTableQuery += request }
}
I am unable to figure out how can i see the inserted record. Is there any property/config which i need to add?
But the getAllRequest() assert fails as the output still contains the single row(as inserted by schemadata.sql) => which means the insertRequest change is not persisted
I would double-check that the assert(Await.result(response2, 5 seconds).size === 2) line is failing because of a size difference. Could it be failing for some other general failure?
For example, as INIT is run on each connection it could be that you are re-creating the database for each connection. Unless you're careful with the SQL, that could produce an error such as "table already exists". Adding TRACE_LEVEL_SYSTEM_OUT=2; to your H2 URL can be helpful in tracking what H2 is doing.
A couple of suggestions.
First, you could ensure your SQL only runs as needed. For example, your schema.sql could add checks to avoid trying to create the table twice:
CREATE TABLE IF NOT EXISTS my_table( my_column VARCHAR NULL );
And likewise for your schemadata.sql:
MERGE INTO my_table KEY(my_column) VALUES ('a') ;
Alternatively, you could establish schema and test data around your tests (e.g., possibly in Scala code, using Slick). Your test framework probably has a way to ensure something is run before and after a test or test suit.

pytest overall result 'Pass' when all tests are skipped

Currently pytest returns 0 when all tests are skipped. Is it possible to configure pytest return value to 'fail' when all tests are skipped? Or is it possible to get total number passed/failed tests in pytest at the end of execution?
There is possibly a more idiomatic solution, but the best I could come up with so far is this.
Modify this example of the documentation to save the results somewhere.
# content of conftest.py
import pytest
TEST_RESULTS = []
#pytest.mark.tryfirst
def pytest_runtest_makereport(item, call, __multicall__):
rep = __multicall__.execute()
if rep.when == "call":
TEST_RESULTS.append(rep.outcome)
return rep
If you want to make the session fail on a certain condition, then you can just write a session scoped fixture-teardown to do that for you:
# conftest.py continues...
#pytest.yield_fixture(scope="session", autouse=True)
def _skipped_checker(request):
yield
if not [tr for tr in TEST_RESULTS if tr != "skipped"]:
pytest.failed("All tests were skipped")
Unfortunatelly the fail (Error actually) from this will be associated to the last testcase in the session.
If you want to change the return value then you can write a hook:
# still conftest.py
def pytest_sessionfinish(session):
if not [tr for tr in TEST_RESULTS if tr != "skipped"]:
session.exitstatus = 10
Or just call through pytest.main() then access that variable and do you own post session checks.
import pytest
return_code = pytest.main()
import conftest
if not [tr for tr in conftest.TEST_RESULTS if tr != "skipped"]:
sys.exit(10)
sys.exit(return_code)