I have a fixture that I want to apply to every test function, where I extract metadata from the tests. Something like
#pytest.fixture(autouse=True)
def extract_metadata(request):
func_name = request.function.__name__
# etc.
...
I also want to extract the parametrize values here. But I can't figure out how to extract the current parameter values from the request object. The only place I see that they are indicated at all is in the test id inside of the request.node.name, but I'd prefer to extract the actual values rather than parsing them out of the id in the string.
The parameters can be accessed with request.node.callspec.params, which is a dict mapping parameter name to parameter value.
Related
I'm trying to run the same test for a series of arguments using #pytest.mark.parametrize. The test data must be computed dynamically, which I attempted as follows:
data = [("1", "2")]
#pytest.fixture(scope="class")
def make_data():
global data
data.append(("3", "4"))
#pytest.mark.usefixtures("make_data")
class Tester:
#pytest.mark.parametrize("arg0, arg1", data)
def test_data(self, arg0, arg1):
print(arg0, arg1)
print(data)
assert 0
I'm creating the data in the class scope fixture and then using it as the parameter set for test_data. I expect test_data to run twice, with arguments 1, 2 and 3, 4, respectively. However, what I get is a single test with arguments 1, 2 and the following stdout:
1 2
[('1', '2'), ('3', '4')]
The value of data is obviously [('1', '2'), ('3', '4')], which means that the class scoped fixture initialized it as I wanted. But somehow it appears that parametrization already happened before this.
Is there a cleaner way to achieve what I want? I could simply run a loop within the test_data method, but I feel like this defies the purpose of parametrization.
Is there a way to return data in the make_data fixture and use the fixture in #pytest.mark.parametrize? When using #pytest.mark.parametrize("arg0, arg1", make_data) I get TypeError: 'function' object is not iterable. make_data must be a fixture, because in the real test case it relies on other fixtures.
I am new to pytest and would be grateful for any hints. Thank you.
EDIT
To provide an explanation on why I'm doing what I'm doing: the way I understand, #pytest.mark.parametrize("arg0, arg1", data) allows parametrization with a hard-coded data set. What if my test data is not hard-coded? What if I need to pre-process it, like I tried in the make_data method? Specifically, what if I need to read it from a file or url? Let's say I have 1000 data samples for which to run the test case, how can I be expected to hard-code them?
Can I somehow use a function to generate the data argument in #pytest.mark.parametrize("arg0, arg1", data)? Something like:
def obtain_data():
data = []
# read 1000 samples
# pre-process
return data
#pytest.mark.parametrize("arg0, arg1", obtain_data())
This produces an error.
As it turns out, the pytest-cases provides the option to define cases for the parametrization by functions, which helped a great deal. Hope this helps everyone who's looking for something similar
Straight forward way is to use pytest global variables:
You can assign them in the conftest.py:
def pytest_configure(config):
pytest.data = foo()
And in the test case
#pytest.mark.parametrize("data", pytest.data)
def test_000():
...
But I agree with previous comment, it's not the best practice
I am trying to seize a given number of resources dynamically, but I can't figure out the syntax. In the Resource Sets Dynamic Assignment, each unit is represented by the name of the resource set it belongs to. In the picture, the seize block will seize 3 resources of the set "resourcePool".
I need to seize a specific number of resources for each individual agent. Then I tried creating ArrayList of Resource Pool objects and passing it in the dynamic assignment but it doesn't work as the type doesn't match.
For example, let's say I have an agent which requires 4 resources, so the expression needed is: { resourcePool, resourcePool, resourcePool, resourcePool }. How can I assign this expression in a variable or collection of the agent such that it can be used in the Resource Sets Dynamic Assignment box? I think I should finally get something like:
{{agent.resourceSetsToSeize}}
So how to define "resourceSetsToSeize"?
You were so close. The only issue is that the parameters inside the agent must be of type ResourcePool[][], an array of arrays. To convert an array list, in your case resourceSetsToSeize to array you need to call toArray() but with the argument of the specific array you want to convert it to.
So your code should have looked like
{agent.resourceSetsToSeize.toArray(new ResourcePool[resourceSetsToSeize.size()]}
(Assuming that resourceSetsToSeize is a List object
The code can be a bit confusing, see another example below of how to rather use an array as the parameter and then use that directly without converting.
Here is an agent with the parameter of type ResourcePool[][]
When you create the agent you then create this array and put it in the constructor. As you can see you don't need to use the empty constructor and then assign it, you can make use of your parameterized constructor.
And then in the seize object you can simply access the parameter.
I am trying to pass text with dynamic content as a parameter into a pipeline (execute pipeline activity).
As a super simple example, I want the input to my pipeline to be a timestamp, utcnow(). Here are my results:
I've noticed:
If I put #utcnow() in a set variable activity and set the execute pipeline parameter to that variable it works.
If I put #utcnow() (or #{utcnow()}) in the main parameter and set the execute pipeline parameter to that parameter it does not work. I get that string "utcnow()" as the result.
Is there anything that I am missing here? I definitely feel like I've done this successfully before.
If I understand your question correctly, the issue is caused by the main parameter(pipeline parameter) doesn't support expression or functions.
For example, we could pass the value from variable to pipeline active parameter, and it works well, because variable support expression/functions:
When the main pipeline only contains an Execute Pipeline active, we pass the value from main parameter(pipeline parameter) to the Execute Pipeline parameter:
When we debug the pipeline, we need pass the value of main parameter:
The value of pipeline parameter only support the String value, then function utcNow() or #{utcnow() will considered as the String.
Suppose I have a foreach inside of a pipe:
I'd like to iterate through the following:
#split(split(item().name,'_')[4],'-')[1]
However, I'd like to pass this formula in through a parameter.
I've defined a parameter myExpression with the desired value in the pipeline, and attempting to reference it like so:
Note that the full expression would be: {#pipeline().parameters.myExpression}
However, data factory does not execute that expression, rather it just accepts it as a verbatim string:
{#pipeline().parameters.myExpression}
How do we pass in an expression from parameters from within the pipeline?
When you define a parameter like this and pass the value, what you are doing is is send a string input, as the textbox doesn't accept expression. The only way to pass expression to a parameter is to pass it from another pipeline. Another issue we have is one ADF limitation - there can not be nested iterations. Calling a second pipeline solves both the issues.
Split your flow in two pipelines.
First (parent) pipeline - Keep all steps up until generating the array over which iteration has to happen.
#split(split(item().name,'_')[4],'-')[1]
Then, inside a for each loop, invoke an "Execute pipeline" activity. In there, pass the expression that you desire in a similar fashion-
In the child pipeline, define a string parameter to absorb the passed value. Then, use #pipeline().parameters.ParamName to use it in there.
HTH
your description lacks a lot of context of what are you trying to do. I can only presume that you generate array in one pipeline and you want to iterate it in another. Looking at your print screen it looks like you typed in your value, therefore output is a plain text. you should hit dynamic context
so it would look like this:
For example, take OneHotEncoderModel but you could take anything from pyspark.ml.feature package. When you use OneHotEncoderEstimator you have the option to set the inputCols. In face you must use the inputCols and outputCols in the constructor.
After you create the corresponding model from the estimator, you cannot retrieve the value for inputCols anymore. There is no method like getInputCols() to give you that from the given model. If you use getParam("inputCols") it will just give you the Param description and not its value.
If you look at the serialized model (the metadata file) the value for this param (inputCols) is actually written out. See example below:
{"class":"org.apache.spark.ml.feature.OneHotEncoderModel","timestamp":1548215172466,"sparkVersion":"2.4.0","uid":"OneHotEncoderEstimator_c5fcbebe4045","paramMap":{"inputCols":["workclass-tmp"],"outputCols":["workclass-encoded"]},"defaultParamMap":{"handleInvalid":"error","dropLast":true}}
However I'm looking for a way to get that from the API.
Correction to my earlier answer:
the right method is called getOrDefault. For instance:
model.getOrDefault("inputCols")
It looks like there is this undocumented way of getting to those values:
model._paramMap[model.inputCols]
or
model._paramMap[model.params["inputCols"]]