I would like to test datasets with a variable number of values. Each value should be tested and I would like to have a standardized output that I can read in afterward again. My used framework is Matlab.
Example:
The use case would be a dataset which includes, e.g., 14 values that need to be testet. The comparison is already completely handled by my implementation. So I have 14 values, which I would like to compare against some tolerance or similar and get an output like
1..14
ok value1
ok value2
not ok value3
...
ok value14
Current solution:
I try to use the unit-testing framework and the according TAPPlugin that would produce exactly such an output (tap), one for every unit-test. My main problem is that the unit-testing framework does not take any input parameters. I already read about parametrization, but I do not know how this helps me. I could put the values as a list into the parameter, but how to I pass them there? Afaik the unit-test class does not allow additional parameters during initialization, so I cannot include this in the program the way I want.
I would like to avoid to need to format the TAP output on my own, because it is already there, but only for unit-test objects. Unfortunately, I cannot see how to implement this wisely.
How can I implement the output of a test anything protocol where I have a variable amount of comparisons (values) in Matlab?
If you are using class based unit tests you could access its properties from outside the test.
So let's say you have following unit test:
classdef MyTestCase < matlab.unittest.TestCase
properties
property1 = false;
end
methods(Test)
function test1(testCase)
verifyTrue(testCase,testCase.property1)
end
end
You could access and change properties from outside:
test=MyTestCase;
MyTestCase.property1 = true;
MyTestCase.run;
This should no succeed since you changed from false to true. If you want to have a more flexible way, you could have a list with the variables and a list with the requirements and then cycle through both in one of the test functions.
properties
variables = [];
requirements = [];
end
methods(Test)
function test1(testCase)
for i = 1:length(variables):
verifyEqual(testCase,testCase.variables[i],testCase.requirements [i])
end
end
end
Now you would set variables and requirements:
test=MyTestCase;
MyTestCase.variables = [1,2,3,4,5,6];
MyTestCase.requirements = [1,3,4,5,5,6];
MyTestCase.run;
Please note, that in theory you should not have multiple assert statements in one test.
Related
I am writing a class-based test suite in MATLAB for a timeseries handling package. The first test in my suite needs to check whether a connection exists to a Haver database on a network drive. If the connection does not exist, then the first test should abort the rest of the suite using one of the fatalAssert methods.
One complicating factor, which I have excluded from the exposition below, but I will mention now is that I need to use an anonymous function to check the connection to Haver (unless someone has a better idea). My package handles data from multiple sources, Haver only being one of them.
I have a parent-class test suite that performs general tests for all of the sources. I then inherit this parent-class into specific child-class test suites and set specific parameters in their respective TestMethodSetup method. One of these parameters is an anonymous function, connfun, and a location, connloc, which I use in the parent-class to test the connection. The reason I do this is because the parent tests are executed first, so I would have to wait for all of those to end if I wanted to test the connection in the child class.
This also complicates the order of execution. If I want to assign the connfun in the child class, then I have to use either the TestMethodSetup or TestClassSetup of the child class (open to recommendations on which is best here) and put this connection test in the Test method of the parent class. I noticed the if I put checkConn in the TestMethodSetup and TestClassSetup of the parent class was running before that of the child class, I was unable to pass the anonymous function and the test would be incomplete.
Putting the previous point aside for a moment, this was my first attempt at writing the test in the parent-class (note that I used a fatalAssertEqual instead of a fatalAssertTrue because isconnection() does not return a logical):
methods (Test)
function checkConn(testCase)
connloc = 'pathToHaverDatabase';
connfun = #(x) isconnection(haver(x));
testCase.fatalAssertEqual(connfun(connloc), 1);
end
end
The above works when there is a connection, but the problem that I bumped into with this is that when I cannot access connloc, an error ocurrs during the call to haver(). So instead of returning a 1 or 0 from the isconnection() call that I can fatalAssertEqual on, all of checkConn errors out due to haver(). This then leads to the rest of the tests running (and failing, which is exactly what I want to avoid).
My next idea works for both cases, but it feels like bad code, and does not have the anonymous function specification described above.
methods (Test)
function checkConn(testCase)
connloc = 'pathToHaverDatabase';
connfun = #(x) isconnection(haver(x));
try
isconn = connfun(connloc);
catch
isconn = 0;
end
testCase.fatalAssertEqual(isconn, 1)
end
end
When I wrote this, I did not necessarily want to distinguish between not having access to the network drive, not being able to call the haver() function, and getting an isconnection equal to 0 because the last case covers all three. But I realized that if I did differentiate them, then it would be a bit more robust, but it's still missing the anonymous function taht I could pass from child to parent.
properties
connloc = 'pathToHaverDatabase';
end
methods (Test)
function checkDrive(testCase)
isfound = fillattrib(testCase.connloc);
testCase.fatalAssertTrue(isfound);
end
function checkHaver(testCase)
try
hav = haver(testCase.connloc);
ishaver = ~isempty(hav);
catch
ishaver = false;
end
testCase.fatalAssertTrue(ishaver);
end
function checkConn(testCase)
connfun = #(x) isconnection(haver(x));
testCase.fatalAssertEqual(connfun(testCase.connloc), 1);
end
end
Ideally, what I would want is a fatalAssert method (or something similar) that ends the test suite when its input is an error. Something that would perhaps be called fatalAssertNotError, but I don't think that exists. If it did, the last line of my first function would simply be testCase.fatalAssertNotError(connfun(connloc)) and I would not have to worry about all the cases.
I'm very open to dynamic rewrite of this whole test setup, so any specific comments or general advice are welcome!
First of all, I think the fatalAssert case is a strong use case to provide something like fatalAssertNotError. One reason why it is not part of the package is because many/most times people don't want to check whether something doesn't error, they just want to call the code, and if it errors, it fails for the test author automatically and it is much simpler. However, other qualification types like fatal assertions and assumptions perhaps point to the need to provide this so you can choose the outcome of the test in the presence of an error, in cases where you don't want it to fail (like with assumptions) or you want it to fail "more strongly" like with fatal assertions.
That being said, I am still not convinced that you can't achieve what you are ultimately trying to do without it. The question I have centers around why you can't use TestClassSetup. It is not clear to me exactly why you weren't able to customize in the derived test class to get the behavior you want. For example, does something like this work?
classdef BaseTest < matlab.unittest.TestCase
properties(Abstract)
connloc
connfun
end
methods(TestClassSetup)
function validateConnection(testCase)
% If this errors it behaves like an assertion (not fatal assertion)
% and fails all tests in the test class. If it doesn't error but
% doesn't return 1 then the assertion failure will occur.
testCase.assertEqual(connfun(connloc), 1,
'Could not establish a connection to the database');
end
end
end
classdef DerivedTest < BaseTest
properties
connloc = 'pathToHaverDatabase';
connfun = #(x) isconnection(haver(x));
end
methods(Test)
function testSomething(testCase)
% Have at least one test method to test it out
end
end
end
Hope that helps!
If you really want to use a function you can define a nested one like this:
methods (Test)
function checkConn(testCase)
connloc = 'pathToHaverDatabase';
function res = connfun(x)
try
res = isconnection(haver(x));
catch
res = false
end
end
testCase.fatalAssertEqual(connfun(connloc), 1);
end
end
Nested functions can be a bit confusing to me because of the way they share data with the parent function. There really is no difference between an anonymous function and a nested function.
The alternative is to put the function at the end of the file, outside the classdef block:
classdef ...
%...
methods (Test)
function checkConn(testCase)
connloc = 'pathToHaverDatabase';
function res = connfun(x)
try
res = isconnection(haver(x));
catch
res = false
end
end
testCase.fatalAssertEqual(connfun(connloc), 1);
end
end
%...
end
function res = connfun(x)
try
res = isconnection(haver(x));
catch
res = false
end
end
But I honestly don't understand why you need to have a function call within fatalAssertEqual. The code you have seems perfectly fine to me.
Hi Modelica Community,
I would like to run two models in parallel in JModelica but I'm not sure how to pass variables between the models. One model is a python model and the other is an EnergyPlusToFMU model.
The examples in the JModelica documentation has the full simulation period inputs defined prior to the simulation of the model. I don't understand how one would configure a model that pauses for inputs, which is a key feature of FMUs and co-simulation.
Can someone provide me with an example or piece of code that shows how this could be implemented in JModelica?
Do I put the simulate command in a loop? If so, how do I handle warm up periods and initialization without losing data at prior timesteps?
Thank you for your time,
Justin
Late answer, but in case it is picked up by others...
You can indeed put the simulation into a loop, you just need to keep track of the state of your system, such that you can re-init it at every iteration. Consider the following example:
Ts = 100
x_k = x_0
for k in range(100):
# Do whatever you need to get your input here
u_k = ...
FMU.reset()
FMU.set(x_k.keys(), x_k.values())
sim_res = FMU.simulate(
start_time=k*Ts,
final_time=(k+1)*Ts,
input=u_k
)
x_k = get_state(sim_res)
Now, I have written a small function to grab the state, x_k, of the system:
# Get state names and their values at given index
def get_state(fmu, results, index):
# Identify states as variables with a _start_ value
identifier = "_start_"
keys = fmu.get_model_variables(filter=identifier + "*").keys()
# Now, loop through all states, get their value and put it in x
x = {}
for name in keys:
x[name] = results[name[len(identifier):]][index]
# Return state
return x
This relies on setting "state_initial_equations": True compile option.
A program I am working on performs calculations that involve objects that can only have several possible sets of values. These parameter sets are read from a catalogue file.
As an example say the objects are representing cars and the catalogue contains a value set {id: (name, color, power, etc.)} for each model. There are however many of these catalogues.
I use Matlab's unittest package to test if the calculations fail for any of the property combinations listed in the catalogues. I want to use this package, because it provides a nice list of entries that failed. I already have a test that generates a cell array of all ids for a (hardcoded) catalogue file and uses it for parameterized tests.
For now I need to create a new class for each catalogue file. I would like to set the catalogue file name as a class parameter and the entries in it as method parameters (which are generated for all class parameters), but I cannot find a way to pass the current class parameter to the local method for creating the method parameter list.
How can I make this work?
In case it is important: I am using Matlab 2014a, 2015b or 2016a.
I have a couple thoughts.
Short answer is no this can't be done currently because the TestParameters are defined as Constant properties and so can't change across each ClassSetupParameter value.
However, to me creating a separate class for each catalogue doesn't seem like a bad idea. Where does that workflow fallover for you? If desired you can still share code across these files by using a test base class with your content and an abstract property for the catalogue file.
classdef CatalogueTest < matlab.unittest.TestCase
properties(Abstract)
Catalogue;
end
properties(Abstract, TestParameter)
catalogueValue
end
methods(Static)
function cellOfValues = getValuesFor(catalog)
% Takes a catalog and returns the values applicable to
% that catalog.
end
end
methods(Test)
function testSomething(testCase, catalogueValue)
% do stuff with the catalogue value
end
function testAnotherThing(testCase, catalogueValue)
% do more stuff with the catalogue value
end
end
end
classdef CarModel1Test < CatalogueTest
properties
% If the catalog is not needed elsewhere in the test then
% maybe the Catalogue abstract property is not needed and you
% only need the abstract TestParameter.
Catalogue = 'Model1';
end
properties(TestParameter)
% Note call a function that lives next to these tests
catalogueValue = CatalogueTest.getValuesFor('Model1');
end
end
Does that work for what you are trying to do?
When you say method parameters I assume you mean "TestParameters" as opposed to "MethodSetupParameters" correct? If I am reading your question right I am not sure this applies in your case, but I wanted to mention that you can get the data from your ClassSetupParameters/MethodSetupParameters into your test methods by creating another property on your class to hold the values in Test[Method|Class]Setup and then referencing those values inside your Test method.
Like so:
classdef TestMethodUsesSetupParamsTest < matlab.unittest.TestCase
properties(ClassSetupParameter)
classParam = {'data'};
end
properties
ThisClassParam
end
methods(TestClassSetup)
function storeClassSetupParam(testCase, classParam)
testCase.ThisClassParam = classParam;
end
end
methods(Test)
function testSomethingAgainstClassParam(testCase)
testCase.ThisClassParam
end
end
end
Of course, in this example you should just use a TestParameter, but there may be some cases where this might be useful. Not sure if its useful here or not.
I have made many functions where I do not always want to require input, so the first lines in many of them look like:
function something = thisIsMyFunction(OptionalStruct)
if(exist('OptionalStruct')
building on existing struct;
else
build a new struct;
end
end
According to the docs, the exist() search goes much faster when I can pass in a type along with the option, so it will only search for that type. I want to know if in this case (or ever) a struct is a variable and I can say:
if(exist('OptionalStruct', 'var')
You could use use isstruct to check if the variable is a structure.
if(exist('OptionalStruct')
if(isstruct(OptionalStruct))
building on existing struct;
else
build a new struct;
end
Whatever is assigned in a statement like
varName = ...;
is a variable and will make exist('varName', 'var') true.
It is totally irrelevant what type of value that variables holds or refers to.
And, seconding natan's comment:
It should take less time to test a thing like than this, than to post this question on SO.
I've come into ownership of a bunch of MATLAB code and have noticed a bunch of "magic numbers" scattered about the code. Typically, I like to make those constants in languages like C, Ruby, PHP, etc. When Googling this problem, I found that the "official" way of having constants is to define functions that return the constant value. Seems kludgey, especially because MATLAB can be finicky when allowing more than one function per file.
Is this really the best option?
I'm tempted to use / make something like the C Preprocessor to do this for me. (I found that something called mpp was made by someone else in a similar predicament, but it looks abandoned. The code doesn't compile, and I'm not sure if it would meet my needs.)
Matlab has constants now. The newer (R2008a+) "classdef" style of Matlab OOP lets you define constant class properties. This is probably the best option if you don't require back-compatibility to old Matlabs. (Or, conversely, is a good reason to abandon back-compatibility.)
Define them in a class.
classdef MyConstants
properties (Constant = true)
SECONDS_PER_HOUR = 60*60;
DISTANCE_TO_MOON_KM = 384403;
end
end
Then reference them from any other code using dot-qualification.
>> disp(MyConstants.SECONDS_PER_HOUR)
3600
See the Matlab documentation for "Object-Oriented Programming" under "User Guide" for all the details.
There are a couple minor gotchas. If code accidentally tries to write to a constant, instead of getting an error, it will create a local struct that masks the constants class.
>> MyConstants.SECONDS_PER_HOUR
ans =
3600
>> MyConstants.SECONDS_PER_HOUR = 42
MyConstants =
SECONDS_PER_HOUR: 42
>> whos
Name Size Bytes Class Attributes
MyConstants 1x1 132 struct
ans 1x1 8 double
But the damage is local. And if you want to be thorough, you can protect against it by calling the MyConstants() constructor at the beginning of a function, which forces Matlab to parse it as a class name in that scope. (IMHO this is overkill, but it's there if you want it.)
function broken_constant_use
MyConstants(); % "import" to protect assignment
MyConstants.SECONDS_PER_HOUR = 42 % this bug is a syntax error now
The other gotcha is that classdef properties and methods, especially statics like this, are slow. On my machine, reading this constant is about 100x slower than calling a plain function (22 usec vs. 0.2 usec, see this question). If you're using a constant inside a loop, copy it to a local variable before entering the loop. If for some reason you must use direct access of constants, go with a plain function that returns the value.
For the sake of your sanity, stay away from the preprocessor stuff. Getting that to work inside the Matlab IDE and debugger (which are very useful) would require deep and terrible hacks.
I usually just define a variable with UPPER_CASE and place near the top of the file. But you have to take the responsibly of not changing its value.
Otherwise you can use MATLAB classes to define named constants.
MATLAB doesn't have an exact const equivalent. I recommend NOT using global for constants - for one thing, you need to make sure they are declared everywhere you want to use them. I would create a function that returns the value(s) you want. You might check out this blog post for some ideas.
You might some of these answers How do I create enumerated types in MATLAB? useful. But in short, no there is not a "one-line" way of specifying variables whose value shouldn't change after initial setting in MATLAB.
Any way you do it, it will still be somewhat of a kludge. In past projects, my approach to this was to define all the constants as global variables in one script file, invoke the script at the beginning of program execution to initialize the variables, and include "global MYCONST;" statements at the beginning of any function that needed to use MYCONST. Whether or not this approach is superior to the "official" way of defining a function to return a constant value is a matter of opinion that one could argue either way. Neither way is ideal.
My way of dealing with constants that I want to pass to other functions is to use a struct:
% Define constants
params.PI = 3.1416;
params.SQRT2 = 1.414;
% Call a function which needs one or more of the constants
myFunction( params );
It's not as clean as C header files, but it does the job and avoids MATLAB globals. If you wanted the constants all defined in a separate file (e.g., getConstants.m), that would also be easy:
params = getConstants();
Don't call a constant using myClass.myconst without creating an instance first! Unless speed is not an issue. I was under the impression that the first call to a constant property would create an instance and then all future calls would reference that instance, (Properties with Constant Values), but I no longer believe that to be the case. I created a very basic test function of the form:
tic;
for n = 1:N
a = myObj.field;
end
t = toc;
With classes defined like:
classdef TestObj
properties
field = 10;
end
end
or:
classdef TestHandleObj < handle
properties
field = 10;
end
end
or:
classdef TestConstant
properties (Constant)
field = 10;
end
end
For different cases of objects, handle-objects, nested objects etc (as well as assignment operations). Note that these were all scalars; I didn't investigate arrays, cells or chars. For N = 1,000,000 my results (for total elapsed time) were:
Access(s) Assign(s) Type of object/call
0.0034 0.0042 'myObj.field'
0.0033 0.0042 'myStruct.field'
0.0034 0.0033 'myVar' //Plain old workspace evaluation
0.0033 0.0042 'myNestedObj.obj.field'
0.1581 0.3066 'myHandleObj.field'
0.1694 0.3124 'myNestedHandleObj.handleObj.field'
29.2161 - 'TestConstant.const' //Call directly to class(supposed to be faster)
0.0034 - 'myTestConstant.const' //Create an instance of TestConstant
0.0051 0.0078 'TestObj > methods' //This calls get and set methods that loop internally
0.1574 0.3053 'TestHandleObj > methods' //get and set methods (internal loop)
I also created a Java class and ran a similar test:
12.18 17.53 'jObj.field > in matlab for loop'
0.0043 0.0039 'jObj.get and jObj.set loop N times internally'
The overhead in calling the Java object is high, but within the object, simple access and assign operations happen as fast as regular matlab objects. If you want reference behavior to boot, Java may be the way to go. I did not investigate object calls within nested functions, but I've seen some weird things. Also, the profiler is garbage when it comes to a lot of this stuff, which is why I switched to manually saving the times.
For reference, the Java class used:
public class JtestObj {
public double field = 10;
public double getMe() {
double N = 1000000;
double val = 0;
for (int i = 1; i < N; i++) {
val = this.field;
}
return val;
}
public void setMe(double val) {
double N = 1000000;
for (int i = 1; i < N; i++){
this.field = val;
}
}
}
On a related note, here's a link to a table of NIST constants: ascii table and a matlab function that returns a struct with those listed values: Matlab FileExchange
I use a script with simple constants in capitals and include teh script in other scripts tr=that beed them.
LEFT = 1;
DOWN = 2;
RIGHT = 3; etc.
I do not mind about these being not constant. If I write "LEFT=3" then I wupold be plain stupid and there is no cure against stupidity anyway, so I do not bother.
But I really hate the fact that this method clutters up my workspace with variables that I would never have to inspect. And I also do not like to use sothing like "turn(MyConstants.LEFT)" because this makes longer statements like a zillion chars wide, making my code unreadible.
What I would need is not a variable but a possibility to have real pre-compiler constants. That is: strings that are replaced by values just before executing the code. That is how it should be. A constant should not have to be a variable. It is only meant to make your code more readible and maintainable. MathWorks: PLEASE, PLEASE, PLEASE. It can't be that hard to implement this. . .