I'm using Test::More to test my application. I have a single script, run_tests.pl, that runs all the tests. Now I want to split this into run_tests_component_A.pl and B, and run both test suites from run_tests.pl. What is the proper way of doing this, does Test::More have any helpful methods?
I'm not using any build system.
Instead of running the creating a run_tests.pl to run the test suite, the standard practice is to use prove.
Say you have
t/foo.t
t/bar.t
Then,
prove is short for prove t.
prove t runs the entire test suite (both t/foo.t and t/bar.t).
prove t/foo.t runs that specific test file.
perl t/foo.t runs that specific test file, and you get the raw output. Easier for debugging.
perl -d t/foo.t even allows you to run the test in the debugger.
Each file is a self-standing program. If you need to share code between test programs, you can create t/lib/Test/Utils.pm (or whatever) and use the following in your test files:
use FindBin qw( $RealBin );
use lib "$RealBin/lib";
use Test::Utils;
prove executes the files in alphabetical order, so it's common to name the files
00_baseline.t
01_basic_tests.t
02_more_basic_tests.t
03_advanced_tests.t
The 00 test tests if the modules can be loaded and that's it. It usually outputs the versions of loaded modules to help with dependency problems. Then you have your more basic tests. The stuff that's like "if this doesn't work, you have major problems". There's no point in testing the more complex features if the basics don't work.
Related
I have been tasked with cleaning up some legacy code which is poorly written, but has an astonishingly large number of tests. Some of these tests run code in files like this:
fcgi/*.fcgi
I would very much like to include those in my coverage reports. In fact, I'd love to ensure that I can include everything (regardless of extension) in lib/, fcgi/, and utils/ and nothing in any other directories.
This is one of my many attempts:
HARNESS_PERL_SWITCHES=-MDevel::Cover=+inc,fcgi,+inc,lib,+inc,util prove -rl t
FAIL!
I've also tried creating simple Build.PL or Makefile.PL scripts and keep getting "No tests defined" when I run things like 'cover -test' or './Build testcover'.
This is Devel::Cover 0.88 and perl version 5.12.2
Alright, i took your example, fiddled a bit with it and it seems to work fine for me with a minimal Makefile.PL and cover -test. Please clone this: git://gist.github.com/3061026.git
The README file contains what happens on my system.
Specifically, in one of my test files, I added:
use 5.12.0;
use Test::More;
use lib 'lib';
use Foo;
require 'fcgi/foo.fcgi'; # <====== pull in an fcgi file
is craptastic(), 'This is craptastic',
'We have run an fcgi/*fcgi test';
I'm working on a personal Perl module to build a basic script framework and to help me learn more about the language. I've created a new module called "AWSTools::Framework" with ExtUtils::ModuleMaker via the command line tool modulemaker. I'm trying to figure out the appropriate way to test it during development.
The directory structure that was created includes the following:
./AWSTOOLS/Framework/lib/AWSTools/Framework.pm
./AWSTOOLS/Framework/t/001_load.t
The autogenerated 001_load.t file looks like this:
# -*- perl -*-
# t/001_load.t - check module loading and create testing directory
use Test::More tests => 2;
BEGIN { use_ok( 'AWSTools::Framework' ); }
my $object = AWSTools::Framework->new ();
isa_ok ($object, 'AWSTools::Framework');
If I try to run the script directly (either from the command line or inside my TextMate editor), it fails with:
Can't locate AWSTools/Framework.pm in #INC....
If I try to run prove in the ./AWSTOOLS/Framework directory, it fails as well.
The question is: What is the proper way to run the tests on Perl modules while developing them?
If you want to run a single test file, you need to tell perl where to find your modules just like you would for any other program. I use the blib to automatically add the right paths:
$ perl Makefile.PL; make; perl -Mblib t/some_test.t
You can also use prove to do the same thing. I don't use prove, but you can read its documentation to figure it out. The -b switch should do that, but I've had problems with it not doing the right thing (could just be my own idiocy).
If you're using the typical toolchain (ExtUtils::MakeMaker) it will be perl Makefile.PL to generate a makefile, then make test every time afterward. Those commands should be run from the root directory of the module. See http://search.cpan.org/perldoc?ExtUtils::MakeMaker#make_test
Edit: and don't do it all manually, or you will come to hate testing. (Well, more than usual.) You will also want to look at least briefly at Test::Tutorial and https://www.socialtext.net/perl5/testing
You may also want to ask the friendly* people in #perl or related channels on your preferred IRC networks.
*Not actually friendly
I actually think that Dist::Zilla is sufficiently flexible enough to allow you to use it for all development. If you aren't uploading to CPAN, just make sure you don't have [UploadToCPAN] in your dist.ini. Also make sure to [#Filter] it out of any plugin bundles which provide it.
Dist::Zilla may be too much to install for only one quick module that you aren't going to touch very often. If you have more than one dist in development then it is definitely worth a look.
You can easily interface it with your VCS using plugins. (Including Git)
You can create a plugin to deploy onto your server. Which would allow you to make sure that all your test files pass before allowing you to deploy ([TestRelease]).
If you don't like tabs in your source files, you can test for that without writing the test yourself ([NoTabsTests]).
Minimal dist.ini for non-CPAN dist
name = Your-Library
author = E. Xavier Ample <example#example.org>
license = Perl_5
copyright_holder = E. Xavier Ample <example#example.org>
copyright_year = 2012
version = 0.001
[GatherDir]
[PruneCruft]
[PruneFiles]
filename = dist.ini
filename = TODO.txt
match = ^.*[.]te?mp$
[NoTabsTests]
[TestRelease]
[CheckExtraTests]
[ModuleBuild]
[FakeRelease]
Test the dist:
dzil test
dzil xtest
If at a later date, you decide to upload it to CPAN:
Replace [FakeRelease] with [UploadToCPAN].
Get a PAUSE id, and set ~/.pause.
user YOUR-PAUSE-ID
password YOUR-PAUSE-PASSWORD
Run dzil release
DONE
In a quick attempt to help you, I would recommend looking at Testing Files and Test Modules.
Continuing to dig around and experiment, I've found the following two things which work for me:
Use prove -l in the './AWSTOOLS/Framework' directory. According to the prove perldoc page, it adds the "lib" directory to the path when Perl runs all the tests in the "t" directory.
To run the script individually/directly, I'm adding the following to the start of the script above the use Test::More line:
use FindBin qw($Bin);
use lib "$Bin/../lib";
This let's me run the script directly via the commad line and in my editor (TextMate). This is based off this page from the Programming Perl book.
Using the -l flag for prove seems very much like the correct thing to do.
As for the "use lib" solution, I doubt that's actually a best practice. If it was, I would expect that modulemaker would have created the 001_load.t test file with that to begin with.
When Module::Starter initializes a project, it creates a test called manifest.t.
#!perl -T
use strict;
use warnings;
use Test::More;
unless ( $ENV{RELEASE_TESTING} ) {
plan( skip_all => "Author tests not required for installation" );
}
eval "use Test::CheckManifest 0.9";
plan skip_all => "Test::CheckManifest 0.9 required" if $#;
ok_manifest();
When you run tests with Build test, here's part of the output:
t\00-load.t ....... ok
t\boilerplate.t ... ok
t\manifest.t ...... skipped: Author tests not required for installation
I understand the outcome in a narrow sense ($ENV{RELEASE_TESTING} is not set, so the tests are skipped), but I don't fully grasp the big picture. What's the intended development process? I assume it's a good idea to run tests to confirm that my module's manifest is accurate. Should I be setting that environment variable? If so, at what point during the development process?
Many module distributions have tests that check not whether the code works, but whether the distribution is in a suitable state for releasing. Things like the MANIFEST being up to date, whether all functions have been documented in POD, etc.
In order to save time, these tests may be written to skip themselves unless the RELEASE_TESTING environment variable is set. This is an informal standard. That way, these tests don't get run when people install the module, nor do they run when the author is just checking to see if a code change broke anything.
You should run RELEASE_TESTING=1 make test (or the Build equivalent) before releasing your dist. If you use Dist::Zilla (which I highly recommend), you can run release tests with dzil test --release. That flag is also set automatically by the TestRelease plugin, which you should definitely use if you use dzil.
Other environment variables commonly used to control testing are AUTOMATED_TESTING and AUTHOR_TESTING. AUTOMATED_TESTING is set by CPAN testers running automated smoke tests.
So I've made a few modules for my own use, and I'm wondering if there is any documentation about how to write tests for perl modules using Module::Build.
The specific problem I'm running into is that I've got three modules, HomeBrew::IO, HomeBrew::Stats, and HomeBrew::Bio, and I can't figure out how to build or test them separately the way the files are arranged.
The three module files are located in the same directory .../HomeBrew/lib/HomeBrew/, and I've got three Build.PL files located in the .../HomeBrew/ directory (named IO-Build.PL, etc), and three .t files in .../HomeBrew/t/ (named HomeBrew-IO.t etc).
What seems to happen is that the three Build.PL files don't seem to know that they're only supposed to build one module at a time. I'll show you one of these:
#!/usr/bin/perl
use warnings;
use strict;
use Module::Build;
my $build = Module::Build->new
(
module_name => "HomeBrew::IO",
dist_author => "George Locke",
dist_abstract => "Various utilities for reading files",
build_requires => {
'Test::More' => '0.10',
'POSIX' => '0', # for tmpname()
'Test::Exception' => '0', # to test that checkExist dies
},
);
$build->create_build_script();
(I should probably be using File::Temp instead of POSIX, but this is only used in testing so it's not a high priority)
In the future, I would like to change my test scripts to have one for each subroutine so I can say Build test checkExist and check just one at sub at a time.
So,
how do I make sure that create_build_script() doesn't include every single .pm file in the lib/HomeBrew. I'd prefer to keep all the HomeBrew module files within the .../HomeBrew directory, but do I have to separate out each one into different directories?
How do I make tests for each subroutine such that Module::Build knows how to test a whole module or just one part of it?
right now, when I say ./Build test it tests all three modules at once (and ./Build install installs all three at once).
It sounds to me like the issue is with your premise, that you need to build and test them separately. If you want to do that, they should be separate installations, in their own directories, with their own Build.PL files etc. If, however, they should be distributed together, they should be built and tested together.
As for testing things separately, you could test each sub in its own test file, or split up the tests in any other way you like. You don't have to invoke your tests with make test or ./Build test -- you can always just explicitly run the test(s) you care about: perl -Ilib t/<name of test>.
You say in your comment to Ether that you don't always want to run every test. In that case, you probably want to use App::Prove to run just the tests that you want to see. It has many other features to manage the information you track and see.
This question is related to this question I asked before. I have multiple test files (A.t, B.t, C.t etc) created to test their respective module A, B, C & so on. But when I do a make test, it runs all the tests. But, when I'm working on a specific module say B, I'd like to run unit tests for that module. After I'm done with my changes, I'll run the whole suite.
So is there any way to do like make test B, which will run only the tests using B.t? And when I say some thing like "make test all" it runs all the tests under the "t" dir? Thanks.
I just run the test that I want to run:
% make; perl -Mblib t/B.t
You can do the same thing with prove, too.
That -Mblib loads the module blib which merely adds blib/lib (and various special directories under it) to #INC for you. It comes with Perl. prove should do the same thing with the -b switch.
My command is really two parts: the make (or ./Build for Module::Build). This builds the source and moves Perl modules and other files into the "build library", or blib, as an intermediate step in the full installation. Normally make test works against the versions in blib and refreshes that for me. Since I'm testing on my own, I ensure that I refresh blib myself and include it in Perl's module search path.
Despite the fact that I know all this, you might be surprised that I often forget to do one of those steps and end up testing against the wrong version of things, whether the fully installed old version (forgot -Mblib) or the old development sources (forgot make). This leads me to debugging statements such as:
print "No really, this is the Foo version. kthnxbye\n";
Executing tests directly through perl works. The prove command was designed to simplify the process, save keystrokes, and shorten the test-debug-test cycle.
prove was designed to work on multiple tests at a time, which you can't do invoking via perl. For instance, you can do:
prove t/*.t
prove t/ # Same as t/*.t
prove t/dev/ # Run only tests in t/dev/
prove -r t/ # Runs all .t files in t/ and any directories below.
The modern prove also has many features for handling suites of tests and modifying how the TAP output from the tests is displayed.
prove --help will show you all prove's options, and prove --man will show you the manual page.