Is it possible to configure CPAN to run author tests as well if they are appropriate for my OS/Arch? I am more concerned about installing a package that is out of sync with its own test suite without realizing it than taking more time to install new packages.
"Author tests" are usually run or skipped based on an env var, so it's just a question of setting that env var. For example, I use DEVEL_TESTS, so the following would run all of WWW-Kickstarter's tests, including the one that makes sure all references to the distro's version are consistent:
DEVEL_TESTS=1 cpan WWW::Kickstarter
According to the Lancaster Consensus AUTHOR_TESTING is the env var that distribution authors should be using for this kind of testing. In practice, there are other var names out in the wild, but people should probably standardize on this one.
AUTHOR_TESTING=1 cpan Module::NAME
Related
I am writing tests that require external software (Amazon's local DynamoDB server) to be installed and running. Is there some way to tell CPAN Testers what to do?
Or should I just download the server and start it myself in the test setup? That would require Java 6.x or newer to be installed. So I think I am back to the first question.
In case people don't know, CPAN Testers is a group of people who test all of CPAN using automated scripts called smokers.
Further background:
Right now, CPAN Testers shows 227 machines pass all tests for Amazon::DynamoDB, but that is misleading since only one of the over seven thousand tests is currently being run: use_ok( 'Amazon::DynamoDB' );. The rest are hidden behind unless statements:
unless ( $ENV{'AMAZON_DYNAMODB_EXPENSIVE_TESTS'} ) {
plan skip_all => 'Testing this module for real costs money.';
}
And a significant number of the tests do not pass. I have fixed that, but testing requires either the setting of three environment variables in the tester's environment and money (the current way):
AMAZON_DYNAMODB_EXPENSIVE_TESTS=1
EC2_ACCESS_KEY=<user's AWS access key>
EC2_SECRET_KEY=<user's AWS secret key>
or the installation of the local version of Amazon DynamoDB. If this module is released as is, it will appear broken on all machines it runs on that don't have the prerequisite environment setup (ie it will erroneously appear broken rather than erroneously appear to be working).
CPAN Testers run the same tests that your module will run upon installation. Should your tests install other software on the machine? Probably not. Instead, the tests should fail loudly when its prerequisites are not met.
You should also draw a distinction between author tests and installation tests. There is no expectation that the installation tests verify all the functionality. Expensive tests (in this case, tests that literally cost money) shouldn't be part of that. You can run them yourself before you release. However, it might be better to put them in xt/ and guard them with the EXTENDED_TESTING variable instead of a non-standard environment variable. See also the Lancaster Consensus for a discussion of various environment variables during testing of Perl projects.
You can also consider using a different provider for your more thorough tests than the donated CPAN Testers capacity, e.g. by setting up Travis CI for your project. Since they give you a container to play around, you can install extra software. You can also securely provide credentials to your tests. In contrast, the main advantage of CPAN Testers is the diverse range of operating systems, i.e. the lack of control over the testing environment.
Call die from Makefile.PL or Build.PL if the prerequisites for building your module cannot be satisfied. On CPANTesters, aborting from the Makefile will give you an NA test result instead of a FAIL test result, and does not reflect poorly on your module and your build process.
# Makefile.PL
...
if ($ENV{AUTOMATED_TESTING}) {
if (!$ENV{AMAZON_DYNAMODB_EXPENSIVE_TESTS} ||
!$ENV{EC2_ACCESS_KEY} ||
!$ENV{EC2_SECRET_KET}) {
die "To test this module, you must set the environment\n",
"variables EC2_ACCESS_KEY, EC2_SECRET_KEY, and\n",
"AMAZON_DYNAMODB_EXPENSIVE_TESTS. Be advised that\n",
"running these test will result in charges against\n",
"your AWS account.";
}
}
...
Is there some way to tell CPAN Testers what to do?
This is more of a social problem than a technical one.
You can ask the regulars on cpan-testers-discuss to manually set up the requirements; there's precedent for doing so. Not everyone will oblige, of course.
Another possibility is to reach out to your module's users and ask them to become ad-hoc test reporters via Task::CPAN::Reporter/cpanm-reporter or similar.
Look at what other CPAN modules that have external dependencies do, and do something like that.
For example, look at the DBI drivers for various databases. While File and SQLite come with their own prereqs, the same is not true for others like Oracle and DB2. Or look at wxGTK which, IIRC, uses an Alien package to install Wx.
In your case, I would suggest more along the lines of the DBD drivers than embedding through Alien, but you have to make that choice.
I have a module I'd like to release to CPAN, and I like using dzil to do the packaging and releasing. However, the module relies on an external application, and while I know where it is installed on my machine, I'd like to ask users to input where it is installed on their machine. Having read Prompt user during unit test in Perl I see ExtUtils::MakeMaker::prompt does just what I want to do.
How would I incorporate that (or something similar) when using dzil?
The standard MakeMaker dzil plugin has no support for anything but a basic Makefile.PL. (Well, it can use File::ShareDir::Install, but that's its limit.) If you need more complex install-time behavior, you'll need to use something else.
I recommend my MakeMaker::Custom plugin. You write your own Makefile.PL, which can do anything that ExtUtils::MakeMaker is capable of, including prompt for information. You can still have dzil add things like your prerequisites at dzil build time, so you can still use AutoPrereqs. (Actually, I recommend ModuleBuild::Custom instead, but if you want to stick with MakeMaker, that's ok.)
Note: You should also allow the information you're prompting for to be supplied on the command line. This will help people who are trying to package your distribution using automated build tools. But that's a MakeMaker issue, not a Dist::Zilla one.
The user should not be installing via Dist::Zilla at all. It is an author-tool only, as its documentation explicitly says. Dist::Zilla is meant to build a distribution that is installed via EUMM or M::B.
Edit: Given your comment, I would instead say, it sounds like your build process isn't a good candidate for using Dist::Zilla, at least consistently. I would suggest using it to build it once more and then move to using the EUMM or M::B that it builds, modify it to your purposes and keep developing that.
If you're using ExtUtil::MakeMaker to install your distribution, then you can use the dzil plugin Dist::Zilla::Plugin::MakeMaker::Runner (that's a mouthful) to bundle a custom Makefile.PL with your dist instead of generating the default one.
That will allow you to use prompt to gather custom information from within the Makefile.PL if you need it.
I have an app that I pack into "binary" form using PerlApp for distribution. Since my clients want a simple install for their Win32 systems, this works very nicely.
Now a client has decided that they need to run all unit tests, like in a standard install. However, they still will not install a normal Perl.
So, I find myself in need of a way to package my unit tests for operation on my client's systems.
My first thought was that I could pack up prove in one file and pack each of my tests separately. Then ship a zip file with the appropriate structure.
A bit of research showed that Test::Harness::Straps invokes perl from the command line.
Is there an existing tool that helps with this process?
Perhaps I could use PAR::Packer's parl tool to handle invocation of my test scripts.
I'm interested in thoughts on how to apply either PAR or PerlApp, as well as any thoughts on how to approach overriding Test::Harness and friends.
Thanks.
Update: I don't have my heart set on PAR or PerlApp. Those are just the tools I am familiar with. If you have an idea or a solution that requires a different packager (such as Cava Packager), I would love to hear about it.
Update 2: tsee pointed out a great new feature in PAR that gets me close. Are there any TAP experts out there that can supply some ideas or pointers on where to look in the new Test::Harness distribution?
I'm probably not breaking big news if I tell you that PAR (and probably also perlapp) aren't meant to package the whole test suite and plethora of CPAN-module build byproducts. They're intended to package stand-alone applications or binary JAR-like module libraries.
This being said, you can add arbitrary files to a PAR archive (both to .par libraries and stand-alone .exe's) using pp's -a switch. In case of the stand-alone executable, the contents will be extracted to $ENV{PAR_TEMP}."/inc" at run-time.
That leaves you with the problem of reusing the PAR-packaged executable to run the test harness (and letting that run your executable as a "perl"). Now, I have no ready and done solution for that, but I've recently worked on making PAR-packaged executables re-useable as more-or-less general purpose perl interpreters. Two gotchas before I explain how you can use that:
Your application isn't going to magically be called "perl" and add itself to your $PATH.
The "reuse" of the application as a general purpose perl requires special options and does not currently support the normal perl options (those in perlrun). It can simply run an external perl script of your choice.
Unfortunately, the latter problem is what may kill this approach for you. Support for perl command line options is something I've been thinking about, but won't implement any time soon.
Here's the recipe how you get PAR with "reusable exe" support:
Install the newest version of PAR from CPAN.
Install the newest, developer version of PAR::Packer from CPAN (0.992_02 or 03).
Add the "--reusable" option to your pp command line.
Run your executable with the following options to run an external script "foo.pl":
./myapp --par-options --reuse foo.pl FOO-PL-OPTIONS-HERE
Unfortunately, how you're going to teach Test::Harness that "./myapp --par-options --reuse" is a perl interpreter is beyond me.
Cava Packager allows you to package test scripts with your packaged executables. This is primarily to allow you to run tests against the packaged code before distribution. However the option is there to also distribute the tests and test capability to your end users.
Note: As indicated by my name, I am affiliated with Cava Packager.
I want another developer to run a Perl script I have written. The script uses many CPAN modules that have to be installed before the script can be run. Is it possible to make the script (or the perl binary) to dump a list of all the missing modules? Perl prints out the missing modules’ names when I attempt to run the script, but this is verbose and does not list all the missing modules at once. I’d like to do something like:
$ cpan -i `said-script --list-deps`
Or even:
$ list-deps said-script > required-modules # on my machine
$ cpan -i `cat required-modules` # on his machine
Is there a simple way to do it? This is not a show stopper, but I would like to make the other developer’s life easier. (The required modules are sprinkled across several files, so that it’s not easy for me to make the list by hand without missing anything. I know about PAR, but it seems a bit too complicated for what I want.)
Update: Thanks, Manni, that will do. I did not know about %INC, I only knew about #INC. I settled with something like this:
print join("\n", map { s|/|::|g; s|\.pm$||; $_ } keys %INC);
Which prints out:
Moose::Meta::TypeConstraint::Registry
Moose::Meta::Role::Application::ToClass
Class::C3
List::Util
Imager::Color
…
Looks like this will work.
Check out Module::ScanDeps and the "scandeps.pl" utility that comes with it. It can do a static (and recursive) analysis of your code for dependencies as well as the %INC dump either after compiling or running the program.
Please note that the static source scanning always errs on the side of including too many dependencies. (It is the dependency scanner used by PAR and aims at being easiest on the end-user.)
Finally, you could choose to distribute your script as a CPAN distribution. That sounds much more complicated than it really is. You can use something like Module::Starter to set up a basic skeleton of a tentative App::YourScript distribution. Put your script in the bin/ subdirectory and edit the Makefile.PL to reference all of your direct dependencies. Then, for distribution you do:
perl Makefile.PL
make
make dist
The last step generates a nice App-YourScript-VERSION.tar.gz
Now, when the client wants to install all dependencies, he does the following:
Set up the CPAN client correctly. Simply run it and answer the questions. But you're requiring that already anyway.
"tar -xz App-YourScript-VERSION.tar.gz && cd App-YourScript-VERSION"
Run "cpan ."
The CPAN client will now install all direct dependencies and the dependencies of those distributions automatically. Depending on how you set it up, it will either follow the prerequisites recursively automatically or prompt with a y/n each time.
As an example of this, you might check out a few of the App::* distributions on CPAN. I would think App::Ack is a good example. Maybe one of the App::* distributions from my CPAN directory (SMUELLER).
You could dump %INC at the end of your script. It will contain all used and required modules. But of course, this will only be helpful if you don't require modules conditionally (require Foo if $bar).
For quick-and-dirty, infrequent use, the %INC is the best way to go. If you have to do this with continuous integration testing or something more robust, there are some other tools to help.
Steffen already mentioned the Module::ScanDeps.
The code in Test::Prereq does this, but it has an additional layer that ensures that your Makefile.PL or Build.PL lists them as a dependency. If you make your scripts look like a normal Perl distribution, that makes it fairly easy to check for new dependencies; just run the test suite again.
Aside from that, you might use a tool such as Module::Extract::Use, which parses the static code looking for use and require statements (although it won't find them in string evals). That gets you just the modules you told your script to load.
Also, once you know which modules you loaded, you can combine that with David Cantrell's CPANdeps tool that has already created the dependency tree for most CPAN modules.
Note that you also have to think about optional features too. Your code in this case my not have them, but sometimes you don't load a module until you need it:
sub foo
{
require Bar; # don't load until we need to use it
....
}
If you don't exercise that feature in your trial run or test, you won't see that you need Bar for that feature. A similar problem comes up when a module loads a different set of dependency modules in a different environment (say, mod_perl or Windows, and so on).
There's not a good, automated way of testing optional features like that so you can get their dependencies. However, I think that should be on my To Do list since it sounds like an interesting problem.
Another tool in this area, which is used by Dist::Zilla and its AutoPrereqs plugin, is Perl::PrereqScanner. It installs a scan-perl-prereqs program that will use PPI and a few plugins to search for most kinds of prereq declaration, using the minimum versions you define. In general, I suggest this over scanning %INC, which can bring in bogus requirements and ignores versions.
Today I develop my Perl apps as CPAN-like distributions using Dist::Zilla that can take care of the dependencies through the AutoPrereq plugin. Another interesting piece of code in this area is carton.
I'm tasked with replicating a production environment to create many test/sit environments.
One of the things I need to do is build up Perl, with all the modules which have been installed (including internal and external modules) over the years. I could just use CPAN.pm autobundle, but this will result in the test environment having much newer versions of the external modules that production has.
What is the easiest/best way to get and install (a lot of) version specific Perl modules.
bdfoy has the best large scale solution, but if you just want to install a few modules you can ask the CPAN shell to install a specific distribution by referencing a path to a tarball (relative to the top of the CPAN tree).
cpan> install MSCHWERN/Test-Simple-0.62.tar.gz
Throw a URL to BackPAN into your URL list and you can install any older version.
cpan> o conf urllist push http://backpan.perl.org/
This is in the CPAN.pm FAQ under "how do I install a 'DEVELOPER RELEASE' of a module?"
cpan install App::cpanminus
cpanm Your::Module#1.23
(Carton, as referenced in other answers, uses cpanm underneath to resolve explicit version requirements.)
Make your own CPAN mirror with exactly what you want. Stratopan.com, a service, and Pinto, tools that's built on top of, can help you do that.
The CPAN tools only install the latest version of any distribution because PAUSE only indexes the latest version. However, you can create your own, private CPAN that has exactly the distributions that you want. Once you have your own CPAN mirror with only what you want, you point your CPAN tools at only that mirror so it only installs those versions. More on that in a minute.
Now, you want to have several versions of that. You can create as many mirrors as you like, and you can also put the mirrors in source control so you can check out any version of the mirror that you like.
Tools such as CPAN::Mini::Inject can help you set up your own CPAN. Check out my talks on Slideshare for the basic examples, and some of my videos on Vimeo for some of the demonstrations. Look at anything that has "CPAN" or "BackPAN" in the title. I think I might have some stuff about it in The Perl Review too, or should by the next issue. :)
Lately, I've been working on a program called dpan (for DarkPAN) that can look at random directories, find Perl distributions in them, and create the structure and index files that you need. You run dpan, you get a URL to point your CPAN client toward, and off you go. It's part of my MyCPAN-Indexer project, which is in Github. It's not quite ready for unsupervised public use because I mostly work with corporate clients to customize their setup. If you're interested in that, feel free to ask me questions though.
Also, I recently released CPAN::PackageDetails that can help you build the right index file. It's still a bit young too, but again, if you need something special, just ask.
[It's almost five years on and this is a well-asked and well-answered question that has had a lot of views. Since this page must still come up in Google searches, an update can't hurt.]
Carton is worth mentioning here. Carton is a relatively recent tool in the same style as App::cpanminus, App::cpanoutdated, perlbrew, et. al. The author (Miyagawa) calls it "alpha" quality, but even in its current state carton helps simplify the maintenance of multiple environments of version tuned modules across machines.
Pinto too is another recent tool relevant to some of the responses (in fact one of the respondents is a contributor).
Stratopan.com is another alternative. Stratopan provides private CPANs in the cloud. You can fill your Stratopan repository with specific versions of modules (and their dependencies) and then install them using the standard Perl tool chain. The repository changes only when you decide to change it, so you'll get always get the versions of the modules that you want.
Disclaimer: I operate Stratopan.
It seems that creating a cpanfile listing all your modules and desired versions (using the == <version> syntax to lock it to a specific release) could serve well here, too. That would mean using Carton or cpanm for installing the modules.
Doing this would have the benefit of being able to quickly/easily tweak the file to test upgrading specific modules in a dev or staging environment - something that a private CPAN mirror wouldn't let you do (without creating multiple mirrors).