How can I run Perl test suite automatically when files change? - perl

Is there a tool that would watch file changes in a directory tree of a Perl application and re-run the test suite every time I save changes to some module? Something akin to masak’s tote.

Have a look at Test::Continuous
Some Test::Continuous links:
Video presentation with slides from YAPC::Asia 2008
Run continuous tests on remote host
Github repo

The old school unix solution would be to write up a Makefile and trigger it regularly through a cron job (as frequently as once a minute), and have it mail you the results if something broke.
Alternatively if you use a revision control system such as svn, you could use a commit hook to kick off a build/test cycle when you commit a file.
One other thing you could do is write a wrapper script around your editor (such that when you close or save a file, the build/test cycle is triggered).

Win32::FileNotify can help with monitoring changes to the file system if you are on Windows.

The basic way to do this is through source control and an continuous integration package, say, like Smolder. When you check in files, the CI server notices the changes and runs the test suite for you.
Many CI products collect cross-run information for you and can show you trends in your tests, test cover, and so on.

Ironically I just ported stakeout.rb to PHP this week.
http://joshribakoff.com/?p=106

I don't know of any generic filesystem monitoring widget, but here's the Perl specific half.
sub run_tests {
my $prove_out = `prove -lr`;
my $tests_passed = $? == 0;
return "" if $tests_passed;
return $prove_out;
}
This uses the prove utility that comes with Test::Harness 3. It exits non-zero on test failure. Plug that into your filesystem monitoring thing and you're set.

I haven't managed to get Test::Continuous installed successfully on Windows, I use the following script, and it works pretty well for me:
use File::ChangeNotify;
$| = 1;
my $watcher = File::ChangeNotify->instantiate_watcher(
directories => [ 't', 'lib' ],
filter => qr/\.t|\.pl|\.pm/,
);
while (my #events = $watcher->wait_for_events) {
print `prove -l -r -t --timer`;
}

Related

How to test a single failing test when building perl

This issue usually is encountered when trying to run make test and sees one test fails. The README describes one can run each test individually, didn't clearly specifies how to do so.
make test uses the script called TEST in the test directory (t). To replicate make test of a single file, one would use this script as follows:
[.../perl/t]$ ./perl -I../lib TEST op/array.t
t/op/array ... ok
All tests successful.
Elapsed: 0 sec
u=0.01 s=0.00 cu=0.03 cs=0.02 scripts=1 tests=194
If you want to see the raw output of the test script, you can run perl as follows:
[.../perl/t]$ ./perl -I../lib op/array.t
1..194
ok 1
ok 2
ok 3
...
ok 192 - holes passed to sub do not lose their position (multideref, mg)
ok 193 - holes passed to sub do not lose their position (aelem)
ok 194 - holes passed to sub do not lose their position (aelem, mg)
The above information and more is found in perlhack.
This document explains how Perl development works. It includes details about the Perl 5 Porters email list, the Perl repository, the Perlbug bug tracker, patch guidelines, and commentary on Perl development philosophy.
Note that you need to run make test_prep before the above commands work. (If you've run make test, you've effectively run make test_prep already.)
Run ./perl harness ../foo/boo.t in the t directory, with foo/boo the name of the failing test.
To run a single test script, use perl, or better, prove. Assuming you are in the module's base directory:
prove -lv t/some-test-script.t
This will run the test script against the libraries in ./lib, with fallback to the libraries available to your install of Perl.
If you want to use the build libraries built by make, then this:
prove -bv t/some-test-script.t
Now the test script will be run against the libraries in ./blib, falling back to libraries installed for your Perl.
The test scripts are typically just Perl scripts that live in a t/ or xt/ or some similar path within the distribution's directory structure. So you can also run them just with Perl:
perl -Iblib t/some-test-script.t
But prove produces nicer test summary information and color coding.
That is about as granular as you can get unless tests are written to allow for targeting specific segments within a test script. If you need to target a specific test within a test script you'll usually have to dig into the test code itself.

How can I run a test perl script designed for prove form inside TextMate?

I'm using TextMate 1.5.10 (Mac OSX 10.7.2) to write a perl modulino application. To verify the functionality, I'm using test scripts designed to be run with the prove command line tool.
An example of the directory structure I'm using looks like this:
text_mate_test/MyModule.pm
text_mate_test/t/001_load_test.t
The 001_load_test.t file looks like this:
#!/usr/bin/perl
use Modern::Perl;
use Test::More;
use MyModule;
my $testObj = new_ok("MyModule", undef, "Initial load test.");
done_testing();
When I run prove or prove -v in the "text_mate_test" directory, everything passes as expected.
I'd like to be able to setup a hotkey in TextMate that allows me to run the test file without having to jump over to the terminal. Currently, if I run "001_load_test.t" directly from inside TextMate with Cmd+R, it chokes saying "Can't locate MyModule.pm in #INC". That's expected since the test script isn't designed to run directly. (I'm still pretty new to writing test files, but I believe that's the proper way to set them up.)
Running off the assumption that I don't want to change the test file itself, is there a way to setup a hotkey so I can run the file accurately from inside TextMate?
I've figured out an even better way to do this.
In the TextMate Bundle Editor (Menubar -> Bundles -> Bundle Editor -> Show Bundle Editor), I've updated the default "Perl -> Run Script" bundle to this:
#!/usr/bin/env ruby
require "#{ENV["TM_SUPPORT_PATH"]}/lib/tm/executor"
require "#{ENV["TM_SUPPORT_PATH"]}/lib/tm/save_current_document"
TextMate.save_current_document
TextMate::Executor.make_project_master_current_document
### If it's a ".t" test script in a "t" directory, run prove
if ( ENV["TM_FILEPATH"] =~ /^.*\/(t\/[^\/]+)$/ )
### Grab the relative file path for more legible output
relative_file_path = $1
### Jump up one directory so prove will work
Dir.chdir("../");
### Call prove with args to run only the file you are working on.
TextMate::Executor.run("prove", :script_args => ["-v", relative_file_path]);
### Otherwise, run with perl
else
TextMate::Executor.run(ENV["TM_PERL"] || "perl", "-I#{ENV["TM_BUNDLE_SUPPORT"]}",
"-Mexception_handler", ENV["TM_FILEPATH"],
:version_args => ["-e", 'printf "Perl v%vd", $^V;'])
end
Here's a screenshot of how it looks in the Bundle Editor.
The benefit of this is that you can use the same hot key (Cmd+r by default) to run your normal scripts with perl and your test scripts with prove.
This is what I was looking for.
UPDATED: When I first developed this, I only had one test script in the "t" directory. I didn't notice until I added other test scripts that the code in the original version of this answer would run prove across all the scripts. Not just the one being worked on. To get back to the expected behavior, I've update the bundle code so that prove will only run on the active script.
I've come up with a solution. Create a new Perl bundle called "Run Script with prove" and associate it with Shift-Cmd-R. The code for the bundle is:
#!/usr/bin/env ruby
require "#{ENV["TM_SUPPORT_PATH"]}/lib/tm/executor"
require "#{ENV["TM_SUPPORT_PATH"]}/lib/tm/save_current_document"
TextMate.save_current_document
TextMate::Executor.make_project_master_current_document
### If it's a ".t" test script in a "t" directory, run prove
if ( ENV["TM_FILEPATH"] =~ /^.*\/(t\/[^\/]+)$/ )
### Use the relative file path for more legible output
relative_file_path = $1
### Jump up one directory so prove will work
Dir.chdir("../");
### Call prove with args to run only the file you are working on.
TextMate::Executor.run("prove", :script_args => ["-v", relative_file_path]);
else
error_string = "This script's filepath doesn't end with /t/.*\.t\n"
error_string += "That is required for the 'Perl -> Run Script with prove' bundle to work.\n"
TextMate::Executor.run("echo", :script_args => [error_string]);
end
Note: This is the results of a bunch of trial and error hacking. I don't know if it's the "right" to do it, but this works for me. Everything but the last two lines is a copy from the original "Run Script" bundle that comes with TextMate. Based on that, it seems like this should be pretty safe.
UPDATE: When I first built this I only had one test file in the "t" directory. When I added more, I discovered that the original version of the bundle was running all the test files. This code represents an update to the expected behavior of only running the test script you working on. Because of the way I ended up doing that, it also became necessary to add in a fallback. If you try to run a script that doesn't match the standard test file path format, it gives an error message.
It will enable the program to find your module if you add
use lib '..';
to the top of your code (before the use MyModule). This will add the text_mate_test directory to #INC and enable Perl to find the module, though you may come across other problems with running the program directly.

Export PATH using NET::SSH:PERL

I am writing a shell script to automate some of the tedious tasks that we perform. I need to ssh to a server and change the PATH variable remotely, have that variable persist for the next commands to be executed. Code below;
sub ab_tier{
my $ssh=Net::SSH::Perl->new($host);
$ssh->login($user2,$user2);
my $PATH;
my($stdout,$stderr,$exit)=$ssh->cmd("export
PATH=/usr/bin/sudo:/local/perl-5.6.1/bin:$PATH");
my($PATH, $stderr, $exit)=$ssh->cmd("echo $PATH");
print $PATH; # Check the path for correctness : does not change
}
However the PATH does not change. Is there another way to implement this or am I doing something wrong. I need to automate tasks so dont think $ssh->shell would help here. Please suggest.
I made changes as per suggestions and everything works fine. However I am noticing another issue, which is occurring when trying to display environment variables.
my $cmd_ref_pri={
cmd0=>"echo $ENV{'HOME'}",
cmd1=>"chmod 777 $ENV{'COMMON_TOP'}/temp"
};
Now I am connecting to a remote server using Net::SSH::Perl and the value returned by $ENV{"HOME"} is the value of the my home directory and not of the remote server. However if I add a command as in :
my $cmd_ref_pri={
cmd0=>"whoami ; echo $ENV{'HOME'}",
cmd1=>"chmod 777 $ENV{'COMMON_TOP'}/temp"
};
Then the user id displayed is of the user using which I ssh to the remote server. I do not have other modules installed and the only one available is Net:SSh:perl hence I am forced to use this.
routine for executing command
sub ssh_cmd{
#$cmd_sub - contains command, $ssh contains object ref
my ($cmd_sub,$ssh)=#_;
my($stdout, $stderr, $exit)=$ssh->cmd("bash",$cmd_sub);
if( $exit !=0){
print $stdout;
print "ERROR-> $stderr";
exit 1;
}
return 0;
}
Any suggestions as to why this could happen ?
cmd() is not passing your commands into one shell. It executes them in separate shells (or without any shell - manual is not clear about it). As soon as you finish your export PATH the shell exits and the new PATH is lost.
Looks like it is possible to pass all the relevant commands to a single shell process as separate lines of $stdin?
my $stdin='export A=B
echo $A
';
$ssh->cmd("bash",$stdin);
This would work just like on interactive login (but without terminal control, so commands that talk directly to terminal would likely fail).
Anyway Net::SSH::Perl does not look like the best tool for the job. I would rather use expect for automation.
Set PATH on every command call:
$ssh->cmd('PATH=/usr/bin/sudo:/local/perl-5.6.1/bin:$PATH echo $PATH');
And BTW, Net::SSH::Perl is not being maintained anymore, nowadays Net::SSH2 and Net::OpenSSH are better alternatives.
Write commands to a remote temp file, then execute that one. Or, skip the $PATH thing and use the full path for subsequent commands (assuming you know it).

Why doesn't my shell script work when I run it from Perl?

I have this command that I load (example.sh) works well in the unix command line.
However, if I execute it in Perl using the system or ` syntax, it doesn't work.
I am guessing certain settings like environment variables and other external sh files weren't loaded.
Is there an example coding to ensure it will work?
More Updates on coding execution failure (I have been trying with different codes):
push (#JOBSTORUN, "cd $a/$b/$c/$d; loadproject cats; sleep 60;");
...
my $pm = new Parallel::ForkManager(3);
foreach my $job (#JOBSTORUN) {
$pm->start and next;
print(`$job`);
$pm->finish;
}
print "\n\n[DONE] FINISHED EXECUTING JOBS\n";
Output Messages:
sh: loadproject: command not found
Can you show us what you have tried so far? How are you running this program?
My first suspicion wouldn't be the environment if you are running it from a login shell. any Perl script you start (well, any program, really) inherits the same environment. However, if you are running the program through cron, then that's a different story.
The other mistakes I usually make in these situations is specifying the relative paths incorrectly. The paths are fine from the command line, but my Perl script has some other current working directory.
For general advice, see Interact with the system when Perl isn't enough. There's also a chapter in Learning Perl about this.
That's about the best advice you can hope for given the very limited information you've shared with us.

How can I pause Perl processing without hard-coding the duration?

I have a Perl script that contains this code snippet, which calls the system shell to get some files by SFTP and unzip them with WinZip:
# Run script to get files from remote server
system "exec_SFTP.vbs";
# Unzip any files that were retrieved
foreach $zipFile (<*.zip>) {
system "wzunzip $zipFile";
}
Even if some files are retrieved, they are never unzipped, because by the time the files are retrieved and the SFTP connection is closed, the Perl script has already completed the unzip step, with the result that it doesn't find anything to unzip.
My short-term fix is to insert
sleep(60);
before the unzip step, but that assumes that the SFTP connection will finish within 60 seconds, which may sometimes be a gross over-estimate, and other times an under-estimate.
Is there a more sound way to cause Perl to pause until the SFTP connection is closed before proceeding with the unzip step?
Edit: Responders have questioned (and reasonably so) the use of a VB script rather than having Perl do the file transfer. It has to do with security -- the VB script is maintained by others and is authorized to do the SFTP.
Check the code in your *.vbs file. The system function waits for the child process to finish before execution continues. It appears that your *.vbs file is forking a background task to do the FTP and returning immediately.
In a perfect world your script would be rewritten to use Net::SFTP::Foreign and Archive::Extract..
An ugly quick-hackish kind of way might be to create a touch-file before your first system call, alter your sftp-fetching script to delete the file once it is done and have a while like so
while(-e 'touch.file') {
sleep 5;
}
# foreach [...]
Of course, you would need to take care if your .vbs fails and leaves the touchfile undeleted and many other bad side effects. This would be for a quick solution (if none of the other suggestions work) until you get the time to rewrite without system() calls.
You need a way for Perl to wait until the SFTP transfer is done, but as your script is currently written, Perl has no way of knowing this. (It looks like you're combining at least two scripting languages and a (GUI?) SFTP client; this can work, but it's not exactly reliable or robust. Why use VBscript to start the SFTP transfer?)
I can think of four options:
Your Perl script could do the SFTP transfer itself, using something like CPAN's Net::SFTP module, rather than spawning an external job whose status it cannot track.
Your Perl script could spawn a command-line SFTP utility (like PSFTP) that doesn't return until the transfer is done.
Or change exec_SFTP.vbs script to not return until the transfer is done.
If you're currently using a graphical SFTP client and can't switch for whatever reason, I'd recommend using a scripting language like AutoIt instead of Perl. AutoIt has features to wait for windows to change state and so on, so it could more easily monitor for an activity's completion.
Options 1 or 2 would be the most robust and reliable.
The best I can suggest is modifying exec_SFTP.vbs to exit only after the file transfer is complete. system waits for the program it called to complete, so that should solve your problem:
system LIST
system PROGRAM LIST
Does exactly the same thing as "exec LIST", except
that a fork is done first, and the parent process
waits for the child process to complete.
If you can't modify the vbs script to stay alive until it terminates, you may be able to track subprocess creation. If you get subprocess ids, you can monitor them thereby know when the vbs' various offspring terminate.
Win32::Process::Info lets you get a subprocess ids from a running process.
Maybe this is a dumb question, but why not just use the Net::SFTP and Archive::Extract Perl modules to download and unzip the files?
system will not return until the shell it's running the command in has returned; this may be wrong for launching graphical programs and file associations.
See if any of the following help?
system('cscript exec_SFTP.vbs');
use Win32::Process;
use Win32;
Win32::Process::Create(my $proc, 'wscript.exe',
'wscript exec_SFTP.vbs', 0, NORMAL_PRIORITY_CLASS, '.');
$proc->Wait(INFINITE);
Have a look at IPC::Open3
IPC::Open3 - open a process for reading, writing, and error handling using open3()