Is it possible to retrieve the number of verilog assertions failures in test from specman code? Is it possible to change the printing at the end of the test to include this data as well? For example:
Checking is complete - 0 DUT errors, 0 DUT warnings, 1 Assertion failures.
Thanks, Alex
In theory, you can get the verilog assertion count using external C-DPI via UCIS into the Specman world. In practice it is easy to merge the simulation log file into the Specman log file and post-process it.
Related
I have a Matlab model and lots of unit tests, based on Matlab's own class-based unit test framework (matlab.unittest.TestCase and matlab.unittest.TestRunner). Tests produce quite a lot of warnings, some of which are serious from my pov. I would like the framework to report a test case failure if some of the specific warnings pop up.
A test runner can be easily configured to fail on warnings. But then it will fail on any warning:
import matlab.unittest.TestRunner;
import matlab.unittest.plugins.FailOnWarningsPlugin;
runner = TestRunner.withNoPlugins;
runner.addPlugin(FailOnWarningsPlugin);
A test runner can be also configured to ignore specific warnings, for example:
runner.addPlugin(FailOnWarningsPlugin('Ignoring',{'MATLAB:singularMatrix'}));
Here is the documentation:
https://se.mathworks.com/help/matlab/ref/matlab.unittest.plugins.failonwarningsplugin-class.html
Using 'Ignoring' flag and listing lots of warnings seems troublesome.
Is there a way to do it the other way around? That is, to force my test cases to fail only on certain warnings and ignore others?
You can temporarily set warning to be reported as errors:
s= warning('error', 'MATLAB:DELETE:FileNotFound'); % set warning as an error
warn(s) % restore the warning to non-error
Reference: https://undocumentedmatlab.com/blog/trapping-warnings-efficiently
See the Question in the title:
Runtime error 201 at $0000000100001D42 $0000000100001D42
Is there a way to show the faulty code line instead of the memory address?
Thanks for the Help
Normally, when a run-time error occurs, you are presented with a list of addresses that represent the call stack backtrace, i.e. the addresses of all procedures that were invoked when the run-time error occurred.
This list is not very informative, so there exists a unit that generates the file names and line numbers of the called procedures using the addresses of the stack backtrace. This unit is called lineinfo.
You can use this unit by giving the -gl option to the compiler. The unit will be automatically included. It is also possible to use the unit explicitly in your uses clause, but you must make sure that you compile your program with debug info.
When compiled with -gl, the following output is generated:
Runtime error 255 at 0x0040BDE5
0x0040BDE5 GENERATEERROR255, line 6 of testline.pp
0x0040BDF0 GENERATEANERROR, line 13 of testline.pp
0x0040BE0C main, line 17 of testline.pp
0x0040B7B1
This is more understandable than the normal message. Make sure that all units you use are compiled with debug info, because if they are not, no line number and filename can be found.
What is the point of the REQUIRE_NOTHROW assertion? If I just put a statement and don't wrap it in any assertion macro it will fail if it throws anyway?
It's the difference between the TEST_CASE failing and an individual assertion failing. The REQUIRE macros ensure that the next lines aren't executed if they fail. Conversely, the CHECK macros can mark the test case as a failure but continue.
Consider this example:
REQUIRE_NOTHROW(parseInput(validInput));
REQUIRE_THROWS(parseInput(errorInput));
REQUIRE_THROWS(parseInput(NULL));
So we're explicitly requesting that passing valid input does not cause an exception, but bad input does. If we didn't use the REQUIRE_NOTHROW() macro, then the test would fail but then we'd need to decipher where it failed - an exception could have come from other lines of test code.
I have to write the following concurrent assertion in my monitor:
assert property (vif.cos == 1 |-> vif_out.y == vif.xi/sqrt(2));
I tried to put it in the run_phase, but I got the following error:
Concurrent assertions are not allowed in tasks/class methods.
Where should I put it?
Please refer to the IEEE Std 1800-2012, Section 14.16 (Concurrent assertions).
A concurrent assertion statement may be specified in any of the following:
An always procedure or initial procedure as a statement, wherever these procedures may appear
A module
An interface
A program
A generate block
A checker
Personally, I would just move that piece of code into the interface file.
Concurrent assertions are not allowed in tasks/class methods. You have to put them in a module or similar (interface/program/checker).
When my simulation, using Omnet++, starts, there is a .ned file describing the initial scenario (in my case it shows a certain type of a network configuration). During the simulation this scenario changes, come times it changes very much. Is there a way to get a .ned file describing the final scenario after the simulation has ended? So that I can analyze it with a script...
thnaks
Not the NED file, because that describes the initial network structure in an abstract way (i.e. it can contain vectors, loops, conditional statements etc.)
What you need is a simple dump of all or selected objects. You should use the 'snapshot feature' for this. It produces a nicely formatted XML output. You can read more about it in the manual:
http://www.omnetpp.org/doc/omnetpp/manual/usman.html#sec299