I'm looking for some tutorials showing how I could test C programs by writing Perl programs to automate testing.
Basically I want to learn automation testing with Perl programs. Can anyone kindly share such tutorials or any experiences of yours which can help me kick-start this process?
Perl tests usually use TAP. There are a several C libraries for TAP. Watch this Perl testing presentation.
If you want to start learning how to use Perl to test external programs, start with learning to use Perl to test Perl bits. The Test::More module is a good place to start. Once you understand that, look at all of the other Test::* modules on CPAN to see if one of those modules does the sort of thing you need to do.
If you have a specific question, ask about that. This question is really too broad for anyone to provide a useful answer.
You should look at Perl Testing: A Developer's Notebook by chromatic and Ian Langworth.
I keep meaning to buy a copy but as yet I've just skimmed it at perlmongers meetings. But it seems to be spot on what you're looking for.
UPDATE:
Hm, and this shows I should read the question - testing C programs with Perl, not testing Perl programs with Perl.
The book may still be useful (in that you should probably be writing test scripts and using Test::More and friends) but you will need to write a set of Perl functions to control your C if you take that approach. Basically
sub run_my_c_program {
my #args=#_;
#Set up test environment according to #args
system "my-c-program";
# Turns restults into a $rv data structure
return $rv;
}
and then check $rv in the same way as a normal Perl test:
is_deeply(run_my_c_program(...),
{ .. what I think it returns ..},
".. description of what I'm testing ..");
I don't have a tutorial but I used to be on a test team that tested a C++ compiler. The test harness (home brewed) we used was written in perl and it worked for us for many years. Perl was ideal because we could easily use it to invoke the tools to build the program, capture the output for later insertion into our test logs (using the "backticks" to run the compiler. For example: $compilerOutput = `cl -?`;
), then, if the build was successful, run the test programs and capture their output, again for insertion into our test logs.
Other benefits we got from using Perl:
For pass/fail detection, Perl's regular expression support was helpful.
Our program (the compiler) was ported during the course of its life to many different hardware architectures. Since Perl source was available and in C, we could get perl running on a new architecture as soon as a C compiler was available (which is usually the first language implemented in a new environment after an assembler).
Lots of documentation, books, help available.
-Ron
I wrote an article about Testing C with Libtap that uses Perl's Test::Harness to test C programs. Here's an example project: https://github.com/stig/libggtl/tree/master/t -- might not build any more, as the esoteric build system probably doesn't exist, but you should be able to figure out how it works :-)
Related
I am trying to better understand logic and flow of exceptions. So i got to state that i really feeled lack of understanding how Perl interpretes and runs programs, which phases are involved and what happens on every phase.
For example, I'd like to understand, when are binded STD* IO and when released, what is happening with $SIG{*} things, how they are depended with execepions, how program dies, etc. I'd like to have better insight of internals mechanics.
I am looking for links or books. I prefer some material which has also visual charts involved but this is not mandatory. I'd like to see some "big picture" of whole process, then i have already possibilities to dig further if i find it necessary.
I found Chapter 18th in Programming Perl gives overview of compiling phase and i try to work it trough, but i appreciate other good sources too.
Some alternative sources (there are not very many):
Mannning's Extending and Embedding Perl, which is the go-to reference on Perl's internals outside of the source
The chapter on the Perl internals in Advanced Perl Programming, which may be exactly what you want
Simon Cozens's Perl internals FAQ
Those may be more focused to what you're looking for. I'm not sure any of them explicitly spells out the interpreter's runtime execution order, though. The first one is a better "I want to work with this stuff" book; the second two are probably good introductory references.
Some of the questions you ask are not, as far as I know, explicitly documented - the I/O question being one I can't think of a good source for in particular. Exception handling is documented very well in Try::Tiny's documentation, and it's what we use for exceptions. Signal handling is messy, but perlipc documents it pretty well. With threads, you may be stuck with unsafe signals - I generally avoid threads in favor of multiple processes unless I must have shared memory.
You might start with these topics accessible via the perldoc program:
Internals and C Language Interface
perlembed Perl ways to embed perl in your C or C++ application
perldebguts Perl debugging guts and tips
perlxstut Perl XS tutorial
perlxs Perl XS application programming interface
perlxstypemap Perl XS C/Perl type conversion tools
perlclib Internal replacements for standard C library functions
perlguts Perl internal functions for those doing extensions
perlcall Perl calling conventions from C
perlmroapi Perl method resolution plugin interface
perlreapi Perl regular expression plugin interface
perlreguts Perl regular expression engine internals
perlapi Perl API listing (autogenerated)
perlintern Perl internal functions (autogenerated)
perliol C API for Perl's implementation of IO in Layers
perlapio Perl internal IO abstraction interface
perlhack Perl hackers guide
perlsource Guide to the Perl source tree
perlinterp Overview of the Perl interpreter source and how it works
perlhacktut Walk through the creation of a simple C code patch
perlhacktips Tips for Perl core C code hacking
perlpolicy Perl development policies
perlgit Using git with the Perl repository
I'm using the following command to test my perl code:
perl -MB::Lint::StrictOO -MO=Lint,all,oo -M-circular::require -M-indirect -Mwarnings::method -Mwarnings::unused -c $file
On a system with a perl version less than 5.10 I am also using uninit.
I am also using Perl::Critic and Perl::Tidy and have set up the appropriate rc files to my liking.
These modules have done a great job in helping me break some bad habits I learned when first learning perl.
Are there any more modules or pragmas that will kick me back on the straight and narrow when I mess up?
Using tests, and the Test::* family of modules and some good books have been pointed out. This new information has caused me to reconsider some assumptions about the relationship between testing and code skill building. These are all appreciated and already being researched and put to use.
It seems to me that these are two separate parts of a whole. 'perl -c', Perl::Critic and Perl::Tidy all help during the process of writing code and before execution of code. Devel::Cover, Devel::NYTProf and Tests happen during and after execution of code.
Good development dictates an iterative process, so tests will be run, and code developed over and over, but we still have this separation.
It appears to me that the focus in the answers have been on the 'during and after execution' of code. Again, this is very appreciated. Can I assume that I have the 'writing and pre-execution' part down pretty well then? At least, insomuch as the pragmas, modules and utilities are concerned.
I'm a little worried that you're using Perl 5.9. For two reasons.
Firstly it's a little old. 5.9.0 was released in 2003 and 5.9.5 (the last version in the 5.9.x series) was released in 2007. There have been several high quality versions of Perl since then.
Secondly (and most importantly), 5.9 is an unstable development version of Perl. 5.9 is basically the series of experiments that eventually led to Perl 5.10.0. The only reason to use it is to test that 5.10 will be a stable version of Perl. No-one should be using it at all now.
You don't appear to be testing your code, merely checking that it will compile. I suggest that you look at Test::More (which makes writing actual tests nice and easy), Test::Class (which makes dealing with very large test suites easier), and Devel::Cover (to see which bits of your code are covered by your tests and which aren't).
I have a bunch of scripts that I wrote at times when I did not realize how use v1.2.3; can be useful. So some of them may be using features from later versions of perl, some of them may be OK with, say, perl 5.8.
Now I would like to get that into some order and add proper uses where there is need for them, just to be able to sleep better. :-)
How should I do that? Is there any tool that could help me make an educated guess?
Perl::MinimumVersion
Find a minimum required version of perl for Perl code
The most reliable way is 1) to write a decent test suite, then 2) to run your tests using each version of Perl.
You've surely already done the first part (!), and the second part is actually pretty easy to do using perlbrew.
The docs I find around the ’net and the book I have, Perl Testing, either say or suggest that unit-testing for Perl is usually done when creating modules.
Is this true? Is there no way to unit-test actual programs using Test::More and cousins?
Of course you can test scripts using Test::More. It's just harder, because most scripts would need to be run as a separate process from which you capture the output, and then test it against expected output.
This is why modulinos (see chapter 17 in: brian d foy, Mastering Perl, second edition, O'Reilly, 2014) were developed. A modulino is a script that can also be used as a module. This makes it easier to test, as you can load the modulino into your test script and then test its functions like you would a regular module.
The key feature of a modulino is this:
#! /usr/bin/perl
package App::MyName; # put it in a package
run() unless caller; # Run program unless loaded as a module
sub run {
... # your program here
}
The function doesn't have to be called run; you could use main if you're a C programmer. You'd also normally have additional subroutines that run calls as needed.
Then your test scripts can use require "path/to/script" to load your modulino and exercise its functions. Since many scripts involve writing output, and it's often easier to print as you go instead of doing print sub_that_returns_big_string(), you may find Test::Output useful.
It's not an easiest way to test your code, but you can test the script directly. You can run the script with specific parameters using system (or better IPC::Run3), capture output and compare it with expected result.
But this will be a top level test. It'll be hard to tell which part of your code caused problem.
Unit-tests are used to test individual modules. This makes it easier to see where the problem came from. Also testing functions individually is much easier, because you only need to think about what can happen in smaller piece of code.
The way of testing is depend on your project size. You can, of cause, put everything into single file, but putting your code into module (or even splitting it into different modules) will give you benefits in future: code can be easier reused and tested.
I would like to know which is the current status of the statistical modules in CPAN, does any one know any recent review or could comment about its likes/dislikes with those modules?
I have used the clasical: Statistics::Descriptive, Statistics::Distributions, and some others contained in Bundle::Math::Statistics
Some of the modules has not been updated for long time. I don't know if this is because they are rock solid or has been overtaken by better modules.
Does someone know any current review similar to this old one:
Using Perl for Statistics: Data Processing and Statistical Computing
NB (for the people that will suggest to use R ;-)):
All my code is mainly in perl, but I use R a lot for statistics and plotting. I usually prepare the dataframes with perl and write the R script in the perl modules as templates and save to a file and execute them from perl. But sometimes you have small data sets where efficiency is not an issue (well I am using perl insn't it ;-)) and you want to add some statistics and histograms to your report produced with perl.
PDL, the Perl Data Language is alive and thriving so its worth taking a look at that.
And I think the other stats modules you mention are OK. For eg. Statistics::Descriptive is up-to-date and has been used in answers to a few questions here on Stackoverflow.
NB. There is also a Perl to R bridge called Statistics::R which looks interesting.
/I3az/