What is the preferred unit testing framework for Perl? - perl

Question

I'm sort of new to Perl and I'm wondering if there a prefered unit testing framework?
Google is showing me some nice results, but since I'm new to this, I don't know if there is a clear preference within the community.

Answer 1

Perl has a MASSIVE set of great testing tools that come with it! The Perl core has several tens of thousands of automated checks for it, and for the most part they all use use these standard Perl frameworks. They're all tied together using TAP - the Test Anything Protocol.
The standard way of creating TAP tests in Perl is using the Test::More family of packages, including Test::Simple for getting started. Here's a quick example:
use 5.012;
use warnings;
use Test::More tests => 3;
my $foo = 5;
my $bar = 6;
ok $foo == 5, 'Foo was assigned 5.';
ok $bar == 6, 'Bar was assigned 6.';
ok $foo + $bar == 11, 'Addition works correctly.';
And the output would be:
ok 1 - Foo was assigned 5.
ok 2 - Bar was assigned 6.
ok 3 - Addition works correctly.
Essentially, to get started, all you need to do is put pass a boolean value and a string explaining what should occur!
Once you get past that step, Test::More has a large number of other functions to make testing other things easier (string, regex compares, deep structure compares) and there's the Test::Harness back end that will let you test large groups of individual test scripts together.
On top of that, as Schwern pointed out, almost all of the modern Test:: modules work together. That means you can use Test::Class (as pointed out by Markus) with all of the great modules listed in rjh's answer. In fact, because Test::Builder--the tool that Test::More and others are built on (and currently maintained by Schwern...thanks Schwern!)--you can, if needed, build your OWN test subroutines from the ground up that will work with all the other test frameworks. That alone makes Perl's TAP system one of the nicest out there in my opinion: everything works together, everyone uses the same tool, and you can add on to the framework to suit your needs with very little additional work.

Answer 2

Perl's most popular test 'framework' is a test results format known as TAP (Test Anything Protocol) which is a set of strings that look like:
ok 1 - Imported correctly
ok 2 - foo() takes two arguments
not ok 3 - foo() throws an error if passed no arguments
Any script that can generate these strings counts as a Perl test. You can use Test::More to generate TAP for various conditions - checking if a variable is equal to a value, checking if a module imported correctly, or if two structures (arrays/hashes) are identical. But in true Perl spirit, there's more than one way to do it, and there are other approaches (e.g. Test::Class, which looks a bit like JUnit!)
A simple example of a test script (they usually end in .t, e.g. foo.t)
use strict;
use warnings;
use Test::More tests => 3; # Tell Test::More you intend to do 3 tests
my $foo = 3;
ok(defined $foo, 'foo is defined');
is($foo, 3, 'foo is 3');
$foo++;
is($foo, 4, 'incremented foo');
You can use Test::Harness (commonly invoked as prove from the shell) to run a series of tests in sequence, and get a summary of which ones passed or failed.
Test::More can also do some more complex stuff, like mark tests as TODO (don't expect them to pass, but run them just in case) or SKIP (these tests are broken/optional, don't run them). You can declare the number of tests you expect to run, so if your test script dies half-way, this can be detected.
Once you begin to do more complex testing, you might find some other CPAN modules useful - here are a few examples, but there are many (many) more:
Test::Exception - test that your code throws an error/doesn't throw any errors
Test::Warn - test that your code does/doesn't generate warnings
Test::Deep - deeply compare objects. They don't have to be identical - you can ignore array ordering, use regexes, ignore classes of objects etc.
Test::Pod - make sure your script has POD (documentation), and that it is valid
Test::Pod::Coverage - make sure that your POD documents all the methods/functions in your modules
Test::DBUnit - test database interactions
Test::MockObject - make pretend objects to control the environment of your tests

Answer 3

Definitely start with this page: http://perldoc.perl.org/Test/Simple.html and follow the reference to Test::Tutorial.

Answer 4

If you practice TDD, you will notice that your set of unit tests are changing A LOT. Test::Class follows the xUnit patterns (http://en.wikipedia.org/wiki/XUnit).
For me, the main benefit with xUnit is the encapsulation of each test in methods. The framework names each assertion by the name of the test method, and adds the possibility to run setup- and teardown methods before and after each test.
I have tried the "perl-ish" way for unit testing also (just using Test::More), but I find it kind of old-fashioned and cumbersome.

Answer 5

Some anti-recommendations may be in order:
Anti-recommendation:
Do NOT use the Test::Unit family of test packages for Perl, such as Test::Unit::Assert and Test::Unit::TestCases.
Reason: Test::Unit appears to be abandoned.
Test::Unit, Test::Unit::TestCases, Test::Unit::Assert work, pretty well (when I used them 2015-2016). Test::Unit is supposedly not integrated with Perl's Test Anything Protocol (TAP), although I found that easy to fix.
But Test::Unit is frustrating because so many of the other Perl test packages, mostly built using Test::Builder, like Test::More, Test::Most, Test::Exception, Test::Differences, Test::Deep, Test::Warn, etc., do NOT interact well with the object oriented testing approach of Test::Unit.
You can mix Test::Unit tests and Test::Builder tests once you have adapted Test::Unit to work with Test::More and the TAP; but the good features of these other packages are not available for OO extension. Which is much of the reason to use an xUnit-style test anyway.
Supposedly CPAN's Test::Class allows to "Easily create test classes in an xUnit/JUnit style" -- but I am not sure that I can recommend this. It certainly doesn't look like xUnit to me - not OO, idiosyncratic names like is(VAL1,VAL2,TESTNAME) instead of xUnit style names like $test_object->assert_equals(VAL1,VAL2,TEST_ERR_MSG). Test::Class does have the pleasant feature of auto-detecting all tests annotated :Test, comparable to xUnit and TEST::Unit::TestCase's approach of using introspection to run all functions named test_*.
However, the underlying package Test::Builder is object oriented, and hence much more xUnit style. Don't be scared away by the name - it's not a factory, it's mostly a suite with test assert methods. Although most people inherit from it, you can call it directly if you wish, e.g. $test_object->is(VAL1,VAL2,TESTNAME), and often you can use Test::Builder calls to work around the limitations of procedural packages like Test::More that are built on top of Test::Builder - like fixing the callstack level at which an error is reported.
Test::Builder is usually used singleton style, but you can create multiple objects. I am unsure as to whether these behave as one would expect from an xUnit family test.
So far, no easy way to work around limitations such as Perl TAP tests use TEST_NAMES, per assert, without hierarchy, and without distinguishing TEST_NAMES from TEST_ERROR_MESSAGES. (Err reporting level helps with that lack.)
It may be possible to create an adapter that makes Test::Builder and TAP style tests more object oriented, so that you can rebase on something other than TAP (that records more useful info than TAP - supposedly like ANT's XML protocol). I think to adapt the names and/or the missing concepts will either involve going into Test::Builder, or introspection.