For example,
I pass a list of data into a function. How do I pytest on get_message, foo_function, and yield?
I figure I can pytest on the foo_function on it's own but struggling to find examples of how I would test an iterative function generally and one that uses yield specifically.
def get_message(records):
for record in records:
data = foo_function(record)
if data['message'] == 'BAR':
yield {
'result': 'OK',
'message': data['message']
}
Here is an example providing some hints.
yield output is a generator, the next method with a default value (None in this case) can be used to retrieve and test the value.
pytest.mark.parametrize can be used to test different combinations
In the example I have written a dumb foo_function to return some test data. According to your use case you can choose to mock it.
import pytest
def foo_function(record):
data = {"OK": {"message": "BAR"}, "NOK": {"message": "FOO"}}
return data.get(record)
def get_message(records):
for record in records:
data = foo_function(record)
if data["message"] == "BAR":
yield {"result": "OK", "message": data["message"]}
#pytest.mark.parametrize(
"record,result", [("OK", {"result": "OK", "message": "BAR"}), ("NOK", None)]
)
def test_get_message(record, result):
assert next(get_message([record]), None) == result
Test execution
pytest -rA test_mock.py
# ...
# PASSED test_mock.py::test_get_message[OK-result0]
# PASSED test_mock.py::test_get_message[NOK-None]
Related
I am writing a unit test where I check if an object can be found after being inserted in a mongodb, my unit test looks like this:
class TestReviewCRUD:
app = FastAPI()
config = dotenv_values("../.env")
app.include_router(review_router, tags=["reviews"], prefix="/review")
def setup_method(self):
self.app.db_client = MongoClient(f'mongodb://{self.config["DB_USER"]}:{self.config["DB_PASSWORD"]}#localhost:27017/')
self.app.db = self.app.db_client[self.config['TEST_DB_NAME']]
def teardown_method(self):
self.app.db_client.close()
def test_get_review(self):
with TestClient(self.app) as client:
response = self.given_a_new_review(client)
assert response.status_code == 201 # <- this works
new_review = client.get(f'/review/{response.json().get("_id")}')
assert new_review.status_code == 200 # <- this doesn't work
The element seems to be added to the database (per the 201 http code) and if I go into the docker container, I can see it in the mongo database, but running that get keeps failing, I'm not that versed in python so maybe I am missing something? My get method is structured as:
#router.get("/{id}", response_description="Get a single review by id", response_model=Review)
def find_review(id: str, request: Request):
review = request.app.db["my_db"].find_one({"_id": ObjectId(id)})
if review is not None:
return review
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=f"Review with ID {id} not found")
If I look for an existing ID, it works, it is failing when I insert a new object and immediately look for it
Could someone shed a light, please?
I receive an error RuntimeError: Event loop is closed each time when i try to make more than one async call function inside my test. I already tried to use all suggestions on stackoverflow to rewrite event_loop fixture but nothing works. I wonder what i'm missing
Run test command: python -m pytest tests/ --asyncio-mode=auto
requirements.txt
pytest==7.1.2
pytest-asyncio==0.18.3
pytest-html==3.1.1
pytest-metadata==2.0.1
test.py
async def test_user(test_client_fast_api):
assert 200 == 200
request_first = test_client_fast_api.post( # works fine
"/first_route",
)
request_second = test_client_fast_api.post( # recieve RuntimeError: Event loop is closed
"/second_route",
)
conftest.py
#pytest.fixture()
def event_loop():
try:
loop = asyncio.get_running_loop()
except RuntimeError:
loop = asyncio.new_event_loop()
yield loop
loop.close()
It took me all afternoon to solve this problem.
I also try to succeed from other people's code, here is my code.
Add a file conftest.py to the directory where the test script is placed.
And write the following code.
import pytest
from main import app
from httpx import AsyncClient
#pytest.fixture(scope="session")
def anyio_backend():
return "asyncio"
#pytest.fixture(scope="session")
async def client():
async with AsyncClient(app=app, base_url="http://test") as client:
print("Client is ready")
yield client
And then write a test script test_xxx.py.
import pytest
from httpx import AsyncClient
#pytest.mark.anyio
async def test_run_not_exists_schedule(client: AsyncClient):
response = await client.get("/schedule/list")
assert response.status_code == 200
schedules = response.json()["data"]["schedules"]
schedules_exists = [i["id"] for i in schedules]
not_exists_id = max(schedules_exists) + 1
request_body = {"id": not_exists_id}
response = await client.put("/schedule/run_cycle", data=request_body)
assert response.status_code != 200
#pytest.mark.anyio
async def test_run_adfasdfw(client: AsyncClient):
response = await client.get("/schedule/list")
assert response.status_code == 200
schedules = response.json()["data"]["schedules"]
schedules_exists = [i["id"] for i in schedules]
not_exists_id = max(schedules_exists) + 1
request_body = {"id": not_exists_id}
response = await client.put("/schedule/run_cycle", data=request_body)
assert response.status_code != 200
This is the real test code for my own project. You can change it to your own.Finally, run in the project's terminal python -m pytest.If all goes well, it should be ok's.This may involve libraries that need to be installed.
pytest
httpx
Yeah wow I had a similar afternoon to your experience #Bai Jinge
This is the event loop fixture and TestClient pattern that worked for me:
from asyncio import get_event_loop
from unittest import TestCase
from async_asgi_testclient import TestClient
#pytest.fixture(scope="module")
def event_loop():
loop = get_event_loop()
yield loop
#pytest.mark.asyncio
async def test_example_test_case(self):
async with TestClient(app) as async_client:
response = await async_client.get(
"/api/v1/example",
query_string={"example": "param"},
)
assert response.status_code == HTTP_200_OK
Ref to relevant GitHub issue: https://github.com/tiangolo/fastapi/issues/2006#issuecomment-689611040
Please note - I could NOT figure our how to use Class based tests. Neither unittest.TestCase or asynctest.case.TestCase would work for me. pytest-asyncio docs (here) state that:
Test classes subclassing the standard unittest library are not supported, users are recommended to use unittest.IsolatedAsyncioTestCase or an async framework such as asynctest.
#scenario('../features/config.feature', 'Loading a valid config')
def test_config():
pass
#given("I have provided a valid pylon config",target_fixture="pylon_config")
def pylon_config():
input_data = {
}
return input_data
#when("The configuration is loaded")
def excute_pylon_config(pylon_config):
createFilterPattern(pylon_config)
#then("It should be enriched with the expected FilterPatterns")
def no_error_message(pylon_config):
test_data1= {
}
test_data_2 = {
}
result = pylon_config
#scenario('../features/config.feature', 'Missing usagePlan section')
def test_missing_usageplan():
pass
#given("I have provided a pylon config with a missing key",target_fixture="pylon_config_missing_usageplan")
def pylon_config_missing_usageplan():
input_data = {
'metricFilters': {
'defaults': {
'xyz': []
}
}
}
return input_data
#when("The configuration is loaded")
def excute_pylon_config_missing_usageplan(pylon_config_missing_usageplan):
try:
createFilterPattern(pylon_config_missing_usageplan)
except KeyError:
pass
#then("I should receive an exception")
def error_message_pylon_config_missing_usageplan(pylon_config_missing_usageplan):
print(pylon_config_missing_usageplan)
I have written multiple test case with specifying target_fixture in both #given scenario.
While running the test case it's throwing an error with
fixture 'pylon_config_missing_usageplan' not found
available fixtures: cache, capfd, capfdbinary, caplog, capsys, capsysbinary, doctest_namespace, monkeypatch, pylon_config, pytestbdd_given_I have provided a pylon config with a missing key, pytestbdd_given_I have provided a valid pylon config, pytestbdd_given_trace, pytestbdd_then_I should receive an exception, pytestbdd_then_It should be enriched with the expected FilterPatterns, pytestbdd_then_trace, pytestbdd_when_The configuration is loaded, pytestbdd_when_trace, pytestconfig, record_property, record_testsuite_property, record_xml_attribute, recwarn, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory
use 'pytest --fixtures [testpath]' for help on them.
Can anyone help me over here?
The issue is, that the step When The configuration is loaded has two different implementations in code:
#when("The configuration is loaded")
def excute_pylon_config(pylon_config):
createFilterPattern(pylon_config)
#when("The configuration is loaded")
def excute_pylon_config_missing_usageplan(pylon_config_missing_usageplan):
try:
createFilterPattern(pylon_config_missing_usageplan)
except KeyError:
pass
The function excute_pylon_config_missing_usageplan overrides the step implementation of excute_pylon_config and therefore if you try to load the pylon config in scenario Loading a valid config, pytest-bdd actually tries to execute the function excute_pylon_config_missing_usageplan, which is expecting the fixture pylon_config_missing_usageplan (which is not available in this scenario ...)
Solutions
Have two distinct steps for loading a valid/invalid config, e.g. When The configuration is loaded and When The invalid configuration is loaded (I would recommend this approach, since it is simpler and easier to read than solution 2)
Add a variable to the step for loading the configuration which contains the type of the configuration
Example of variable in configuration loading step:
#when(parsers.parse("The {config_type} configuration is loaded"))
def excute_pylon_config(request, config_type):
if config_type == 'valid':
# Retrieve fixture dynamically by name
pylon_config = request.getfixturevalue('pylon_config')
createFilterPattern(pylon_config)
else:
# Retrieve fixture dynamically by name
pylon_config_missing_usageplan = request.getfixturevalue('pylon_config_missing_usageplan')
try:
createFilterPattern(pylon_config_missing_usageplan)
except KeyError:
pass
Coming from a Java background, I have been trying to teach myself Scala for some time now. As part of that, I am doing a small pet project that exposes a HTTP endpoint that saves the registration numberof a vehicle against the owner and returns the status.
To give more context, I am using Slick as FRM which performs DB operations asynchronously and returns a Future.
Based on the output of this Future, I want to set the status variable to return back to the client.
Here, is the code
def addVehicleOwner(vehicle: Vehicle): String = {
var status = ""
val addFuture = db.run((vehicles returning vehicles.map(_.id)) += vehicle)
addFuture onComplete {
case Success(id) => {
BotLogger.info(LOGTAG, s"Vehicle registered at $id ")
status = String.format("Registration number - '%s' mapped to owner '%s' successfully", vehicle.registration,
vehicle.owner)
println(s"status inside success $status") //--------- (1)
}
case Failure(e: SQLException) if e.getMessage.contains("SQLITE_CONSTRAINT") => {
status = updateVehicleOwner(vehicle)
BotLogger.info(LOGTAG, s"Updated owner='${vehicle.owner}' for '${vehicle.registration}'")
}
case Failure(e) => {
BotLogger.error(LOGTAG, e)
status = "Sorry, unable to add now!"
}
}
exec(addFuture)
println(s"Status=$status") //--------- (2)
status
}
// Helper method for running a query in this example file:
def exec[T](sqlFuture: Future[T]):T = Await.result(sqlFuture, 1 seconds)
This was fairly simple in Java. With Scala, I am facing the following problems:
The expected value gets printed at (1), but (2) always prints empty string and same is what method returns. Can someone explain why?
I even tried marking the var status as #volatile var status, it still evaluates to empty string.
I know, that the above is not the functional way of doing things as I am muting state. What is the clean way of writing code for such cases.
Almost all the examples I could find described how to map the result of Success or handle Failure by doing a println. I want to do more than that.
What are some good references of small projects that I can refer to? Specially, that follow TDD.
Instead of relying on status to complete inside the closure, you can recover over the Future[T] which handle the exception if they occur, and always returns the result you want. This is taking advantage of the nature of expressions in Scala:
val addFuture =
db.run((vehicles returning vehicles.map(_.id)) += vehicle)
.recover {
case e: SQLException if e.getMessage.contains("SQLITE_CONSTRAINT") => {
val status = updateVehicleOwner(vehicle)
BotLogger.info(
LOGTAG,
s"Updated owner='${vehicle.owner}' for '${vehicle.registration}'"
)
status
}
case e => {
BotLogger.error(LOGTAG, e)
val status = "Sorry, unable to add now!"
status
}
}
val result: String = exec(addFuture)
println(s"Status = $result")
result
Note that Await.result should not be used in any production environment as it synchronously blocks on the Future, which is exactly the opposite of what you actually want. If you're already using a Future to delegate work, you want it to complete asynchronously. I'm assuming your exec method was simply for testing purposes.
I've a MongoDB collection where I store User documents like this:
{
"_id" : ObjectId("52d14842ed0000ed0017cceb"),
"email": "joe#gmail.com",
"firstName": "Joe"
...
}
Users must be unique by email address, so I added an index for the email field:
collection.indexesManager.ensure(
Index(List("email" -> IndexType.Ascending), unique = true)
)
And here is how I insert a new document:
def insert(user: User): Future[User] = {
val json = user.asJson.transform(generateId andThen copyKey(publicIdPath, privateIdPath) andThen publicIdPath.json.prune).get
collection.insert(json).map { lastError =>
User(json.transform(copyKey(privateIdPath, publicIdPath) andThen privateIdPath.json.prune).get).get
}.recover {
throw new IllegalArgumentException(s"an user with email ${user.email} already exists")
}
}
In case of error, the code above throws an IllegalArgumentException and the caller is able to handle it accordingly. BUT if I modify the recover section like this...
def insert(user: User): Future[User] = {
val json = user.asJson.transform(generateId andThen copyKey(publicIdPath, privateIdPath) andThen publicIdPath.json.prune).get
collection.insert(json).map { lastError =>
User(json.transform(copyKey(privateIdPath, publicIdPath) andThen privateIdPath.json.prune).get).get
}.recover {
case e: Throwable => throw new IllegalArgumentException(s"an user with email ${user.email} already exists")
}
}
... I no longer get an IllegalArgumentException, but I get something like this:
play.api.Application$$anon$1: Execution exception[[IllegalArgumentException: DatabaseException['E11000 duplicate key error index: gokillo.users.$email_1 dup key: { : "giuseppe.greco#agamura.com" }' (code = 11000)]]]
... and the caller is no longer able to handle the exception as it should. Now the real questions are:
How do I handle the diverse error types (i.e. the ones provided by LastError) in the recover section?
How do I ensure the caller gets the expected exceptions (e.g. IllegalArgumentException)?
Finally I was able to manage things correctly. Here below is how to insert an user and handle possible exceptions with ReactiveMongo:
val idPath = __ \ 'id
val oidPath = __ \ '_id
/**
* Generates a BSON object id.
*/
protected val generateId = __.json.update(
(oidPath \ '$oid).json.put(JsString(BSONObjectID.generate.stringify))
)
/**
* Converts the current JSON into an internal representation to be used
* to interact with Mongo.
*/
protected val toInternal = (__.json.update((oidPath \ '$oid).json.copyFrom(idPath.json.pick))
andThen idPath.json.prune
)
/**
* Converts the current JSON into an external representation to be used
* to interact with the rest of the world.
*/
protected val toExternal = (__.json.update(idPath.json.copyFrom((oidPath \ '$oid).json.pick))
andThen oidPath.json.prune
)
...
def insert(user: User): Future[User] = {
val json = user.asJson.transform(idPath.json.prune andThen generateId).get
collection.insert(json).transform(
success => User(json.transform(toExternal).get).get,
failure => DaoServiceException(failure.getMessage)
)
}
The user parameter is a POJO-like instance with an internal representation in JSON – User instances always contain valid JSON since it is generated and validated in the constructor and I no longer need to check whether user.asJson.transform fails.
The first transform ensures there is no id already in the user and then generates a brand new Mongo ObjectID. Then, the new object is inserted in the database, and finally the result converted back to the external representation (i.e. _id => id). In case of failure, I just create a custom exception with the current error message. I hope that helps.
My experience is more with the pure java driver, so I can only comment on your strategy for working with mongo in general -
It seems to me that all you're accomplishing by doing the query beforehand is duplicating mongos uniqueness check. Even with that, you still have to percolate an exception upwards because of possible failure. Not only is this slower, but it's vulnerable to a race condition because the combination of your query + insert is not atomic. In that case you'd have
request 1: try to insert. email exists? false - Proceed with insert
request 2: try to insert. email exists? false - Proceed with insert
request 1: succeed
request 2: mongo will throw the database exception.
Wouldn't it be simpler to just let mongo throw the db exception and throw your own illegal argument if that happens?
Also, pretty sure the id will be generated for you if you omit it, and that there's a simpler query for doing your uniqueness check, if that's still the way you want to code it. At least in the java driver you can just do
collection.findOne(new BasicDBObject("email",someemailAddress))
Take a look at upsert mode of the update method (section "Insert a New Document if No Match Exists (Upsert)"): http://docs.mongodb.org/manual/reference/method/db.collection.update/#insert-a-new-document-if-no-match-exists-upsert
I asked a similar question a while back on reactivemongo's google group. You can have another case inside the recovery block to match a LastError object, query its error code, and handle the error appropriately. Here's the original question:
https://groups.google.com/forum/#!searchin/reactivemongo/alvaro$20naranjo/reactivemongo/FYUm9x8AMVo/LKyK01e9VEMJ