I am trying to write some unit tests for this perl module function, but am having some issues with environment variables. I'll list the files first, then explain the issue in greater detail.
processBuildSubs.pm
package processBuildSubs;
use strict;
use warnings;
use LWP::UserAgent;
use HTTP::Request::Common;
use HTTP::Status;
# Declare environment variables used in this package that are needed
use constant URI_BASE => $ENV {"URI_BASE"};
use constant URI_RESOURCE => $ENV {"URI_RESOURCE"};
# Shell Environment Related Constants visible.
# Make visible.
our $URI_BASE = URI_BASE;
our $URI_RESOURCE = URI_RESOURCE;
sub populatePartitions
{
# Define locals
my $url;
my $ua = new LWP::UserAgent;
$url = "$URI_BASE"."$URI_RESOURCE"."/some/path";
# Make a request to the $url
$res = $ua->request (GET $url);
if ($res->code() != HTTP::Status->RC_OK() )
{
# The request didn't return 200 OK so it's in here now.
}
else
{
# The request returned 200 OK, so now it's here.
}
}
I want to be able to unit test both the if path and the else path, however, it would be best for me if I don't need to change the processBuildSubs.pm code at all. It's an external file that I don't currently have control over. I am just tasked in unit testing it (although I do understand it could be tested more efficiently if we could also change the source code).
So in order to test both paths, we need the environment variables URI_BASE and URI_RESOURCE to be set accordingly, so that the request fails once, and succeeds another time. (I am interested in learning how to stub out this call at a future time, but that's reserved for another question.)
Here's my test file:
processBuildSubs.t
use strict;
use Test::More qw(no_plan);
BEGIN { use_ok('processBuildSubs') };
# Test 1 of populatePartitions() function
my $processBuildProdsCall = processBuildSubs::populatePartitions();
is( $populatePartitionsCall, 0, "populatePartitions() Test for 0 Val Passed" );
# Test 2 of populatePartitions() function
# I need to change some environment variables that processBuildSubs depends on here.
my $processBuildProdsCall = processBuildSubs::populatePartitions();
is( $populatePartitionsCall, 0, "populatePartitions() Test for 0 Val Passed" );
The best attempt we have right now at changing the environment variables is using an external shell script like so (But it would be ideal to change them between the my calls in the file above):
run_tests.sh
#!/bin/bash
# Run the tests once
perl ./BuildProcess.pl
perl ./Build testcover # Ultimately calls the processBuildSubs.t test file
# Now export some variables so the other test passes.
export URI_BASE="https://some-alias/"
export URI_RESOURCE="some-resource"
# Then run the test suite again with the env set so the else condition passes.
perl ./BuildProcess.pl
perl ./Build testcover
As you can see, this would be a bad way of doing things, as we run the entire test suite with different environments each time. Ideally we'd like to setup our environment in the processBuildSubs.t file if possible, between tests.
Please let me know if I can provide any further information.
Are you averse to having separate scripts for separate test environments?
# processBuildSubs.t
BEGIN {
#ENV{"URI_BASE","URI_RESOURCE"} = ("https://some-alias/","some-resource");
}
use Test::More;
... tests go here ...
# processBuildSubs-env2.t
BEGIN {
#ENV{"URI_BASE","URI_RESOURCE"} = ("https://another-alias/","another-resource");
}
use Test::More;
... tests go here ...
By setting %ENV in a BEGIN block, before any other modules are loaded, you make the different environment variables available to your other modules at compile-time.
Related
ActiveState Perl 5.12 on WinXP
I've recently become a convert to Log4perl and used it successfully where I defined a specific logger for each module that makes up my application. Example:
Upload.pm-----
my $logger = get_logger("Upload"); # specific logger
my $layout = Log::Log4perl::Layout::PatternLayout->new( "%d %p> %F{1}:%L %M - %m%n");
my $appender = Log::Log4perl::Appender->new( "Log::Dispatch::File",
filename => "Upload.log",
mode => "append"
);
$appender->layout($layout);
$logger->level($OFF); # or $INFO
$logger->add_appender($appender)
....
This works but is hard to track program flow across numerous log files ie Upload.log , Parse.log , FileRead.log etc. Quick-n-dirty solution: use same filename in all loggers.
This works much better, but only as long as the program is used serially and in sequence.
Now for the wrinkle - suppose module Upload.pm is used by several progams ie readMyFiles.pl, readHerFiles.pl and dumpAllFiles.pl . When running readMyFiles.pl I want the logger in Upload.pm to write to readMyFiles.pl.log and when runnng dumpAllFiles.pl want the logger in Upload.pm to write to dumpAllFiles.pl.log
One method might be to declare an our variable $logFileName in my .pl files and use it in all my modules like so:
filename => $logFileName,
Another might be to remove all the loggers from my .pm's and define them only in the .pl's - but then how would I reference $logger in the .pm's ?
All thoughts and suggestions are appreciated.
Still-learning Steve
Configure your logging settings in the caller, not in the module. If somebody else uses your module, they might not want to log things to the same place that you do. They might also want to format log messages differently, or use a different type of appender, or...the list goes on.
Your module should only get a logger and write messages to it:
MyModule.pm
#!/usr/bin/perl
package MyModule;
use strict;
use Log::Log4perl qw(get_logger);
sub foo {
my $logger = get_logger("Foo");
$logger->debug("Hello from MyModule");
}
1;
Your main program(s) should configure and initialize logging:
logtest
#!/usr/bin/perl
use strict;
use warnings;
use Log::Log4perl qw(get_logger);
use MyModule;
Log::Log4perl->init("log4perl.cfg");
my $logger = get_logger("Foo");
$logger->debug("Hello from main");
MyModule::foo();
I prefer to use a separate config file for Log4perl settings:
log4perl.cfg
log4perl.logger.Foo=DEBUG, Screen
log4perl.appender.Screen=Log::Dispatch::Screen
log4perl.appender.Screen.Threshold=DEBUG
log4perl.appender.Screen.layout=Log::Log4perl::Layout::PatternLayout
log4perl.appender.Screen.layout.ConversionPattern=[%r] %F %L %c - %m%n
Output:
[0] logtest 12 Foo - Hello from main
[0] MyModule.pm 11 Foo - Hello from MyModule
Note that you must initialize logging with Log::Log4perl->init in the caller before trying to get_logger in the module. If you don't, log messages from the module will be ignored and you'll get the following warning:
Log4perl: Seems like no initialization happened.
Forgot to call init()?
See the documentation for details.
I'm trying to switch out 'Ben' with an environment variable using Perl/Plack. If the environment variable is set (e.g. 'Dave'), then it will print out "Powered by Dave". But it if is not set, it will default to 'Ben'.
Currently, the following program works when using plackup -r
#!/usr/bin/env plackup
my $app = sub {
my $env = shift;
return [
200,
['Content-Type', 'text/plain'],
['Powered by Ben'],
];
};
Currently, my cpanfile contains the following
requires 'Plack' => '1.0028';
requires 'CGI::Emulate::PSGI' => '0.15';
requires 'CGI::Compile' => '0.16';
By Environment Varaible do you mean, the operating system's environment or the plack request enviroment and thus an HTTP request parameter? (they're completely different).
If the former, you can access it via the %ENV hash.
If the latter, you should read the documentation for Plack::Request.
(note: I've never used plack; I'm only going by the docs)
I am using perl Rover module version 3 to login to the Linux/Unix server and run the script. In the ruleset if I add the full path name it copies the script to the remote server, not able to substitute the environment variable.
eg.
This works:
copy:{
put_file "/home/u1/find.sh" "/tmp/"
};
This didn't work:
copy:{
put_file "$HOME/find.sh" "/tmp/"
};
used $ENV{'HOME'}, this also didn't work.
How can I pass the environment variable?
Rover module document.
http://rover.sourceforge.net/QuickStart/Rover-QuickStart-3.html#ss3.2
http://rover.sourceforge.net/
After reviewing the source code for rover, which I never used, I determined it was not possible from the existing code.
I created a new extension for you, that has that functionality, it supports the ~ and ${HOME} syntax, (which are bash extensions and not part of the OS directly, that is why perl does not support them).
code is here:
https://github.com/h4ck3rm1k3/perl-rover/commit/2c78aefb97e819956bb665b04056763f8df1b242
I have had a hard time testing it because I never used rover before, and rover does not seem to support scp.(I read it is supported,but could not test it yet.) Anyway, let me know if you like it. I will put more work into it if reasonably requested.
Update
Here is my example ruleset :
example ruleset
[rulesets]
test:
{
put_file_from_home put_file "~/find2.sh" "/tmp/"
put_file_from_home put_file "${HOME}/find3.sh" "/tmp/"
}, ;
example output
Here is the example output, I cannot get rover to work. See the test case below.
Test output
perl -I lib t/example2.t
Local was ~/find2.sh and home was /home/mdupont at lib/Rover/CoreExtension.pm line 19.
Local now /home/mdupont/find2.sh at lib/Rover/CoreExtension.pm line 22.
Local was ${HOME}/find3.sh and home was /home/mdupont at lib/Rover/CoreExtension.pm line 19.
Local now /home/mdupont/find3.sh at lib//Rover/CoreExtension.pm line 22.
new config option for the new sshport option
[hosts]
someexample:{
os linux
username myusername
description 'myhost'
sshport 12345
ftp_method_used sftp
};
update2
Dont use quotes around the name, use a comma between the args,
To git#github.com:h4ck3rm1k3/perl-rover.git
2207417..7637741 CoreExtension -> CoreExtension
[rulesets]
test: { put_file_from_home ~/find2.sh,/tmp/ }, ;
[hosts]
localhost:{
os linux
username mdupont
description 'localhost'
ftp_methods sftp
ftp_method_used sftp };
mike
Old question but new answer, since your using Rover v3 you can just extend the Rover::Core modules by overloading it.
Add this to your ~/.rover/contrib directory:
CoreVariables.pm:
package CoreVariables;
use strict;
use Exporter;
our #ISA = qw( Exporter );
our #EXPORT = qw( put_file );
sub put_file {
my ($self, $host, $command) = #_;
$command =~ s/(\$[\w{}]+)/$1/eeg;
return Rover::Core::put_file($self, $host, $command);
}
And add the following to your ~/.rover/config [modules] section (must be after Rover::Core):
CoreVariables:{
};
And then you can store environment variables in your rover config when using put_file. Add other routines if you wish, this only extends put_file.
And since this is such an easy task I will add it to the requested feature list and include it in the next release (I am the Rover author).
The better place to ask Rover questions is on the sourceforge website of course: http://sourceforge.net/projects/rover/
I'm building a perl application to archive simple web pages (i.e. static pages with no query strings involved). I'd like to write tests to verify the functionality of the module that will be accessing the remote files. To make the tests self-reliant, I'm looking for a simple, self-contained web server that the test scripts can use locally.
Below is an example which outlines what I'm trying to do. I've cut it down to a minimum with the following directory structure:
./MirrorPage.pm
./t/001_get_url.t
./t/test-docroot/test-1.json
Contents of "./MirrorPage.pm":
package MirrorPage;
use Moose;
use LWP::Simple;
use namespace::autoclean;
sub get_url {
my ($self, $url_to_get) = #_;
### grab the contents of the url
my $url_data = get($url_to_get);
### return the contents.
return $url_data;
}
__PACKAGE__->meta->make_immutable;
1;
Contents of "./t/001_get_url.t":
#!/usr/bin/perl
use Modern::Perl;
use Test::More;
use MirrorPage;
### Start test www server on port 8123 here ###
my $t = new_ok('MirrorPage', undef, 'Create MirrorPage');
is(
$t->get_url("http://localhost:8123/test-1.json"),
'{ testkey: "testvalue" }',
"Verify the data."
);
### Kill test www server here ###
done_testing();
Contents of "./t/test-docroot/test-1.json":
{ testkey: "testvalue" }
The goal is to start and kill a self-contained web server at the corresponding comment locations in "./t/001_get_url.t". The web server needs to serve the contents of the "./t/test-docroot" directory as its document root.
Given all that: What is the best/simplest way to setup a self-contained web server to provide static files for testing in perl?
I would mock the HTTP call near the top of your .t file (if you're only wanting to test MirrorPage.pm):
my $mock = new Test::MockObject();
$mock->fake_module( 'LWP::Simple', get => sub { return '{ testkey: "testvalue" }' } );
LWP can fetch files, so you can rewrite $url_to_get from http://... to file://....
Perhaps:
At the top, fork and do a simple static file server using HTTP::Server::Simple::Static, then at the bottom terminate the child process.
I can highly recommend Mojolicious as a great way to produce a test server for a client. Since Mojolicious simply maps a URL into a subroutine call, it's very easy to have very fine control over what the server does, and therefore you can easily test things like "does my client fail properly if the server returns a bad response/bad content/times out". And since it is very simple to set up and tear down a server, a little cleverness with fork() makes it possible to have the test and the server setup live in the same test file.
Here's what I've come up with using Net::HTTPServer. Based on the idea that "It’s OK to Ask and Answer Your Own Questions", I'm posting it here for comment/consideration. What I've done is the following:
First, create a new module at: "./t/TestServer.pm". The contents of this file are:
package TestServer;
use Moose;
use Net::HTTPServer;
use namespace::autoclean;
has 'server' => (
is => "rw",
isa => "Net::HTTPServer",
default => sub {
Net::HTTPServer->new (
port => 8123,
docroot => "t/test-docroot"
)
},
);
sub BUILD {
my $self = shift;
### Spin up the server.
$self->server->Start();
$self->server->Process();
}
### Close up the Moose package.
__PACKAGE__->meta->make_immutable;
1;
Then, update the test "./t/001_get_url.t" file to use it via a fork:
#!/usr/bin/perl
use Modern::Perl;
use Test::More;
use MirrorPage;
### Fork for the server
my $pid = fork();
### Parent process. Holds the tests.
if($pid) {
### Make sure the server has a moment to startup
sleep(2);
my $t = new_ok('MirrorPage', undef, 'Create MirrorPage');
is(
$t->get_url("http://localhost:8123/test-1.json"),
'{ testkey: "testvalue" }',
"Verify the data."
);
}
### Child process. Holds the server.
elsif(defined($pid)) {
use lib "t/";
use TestServer;
my $svr = TestServer->new();
exit; # Should never get here.
}
### Error out if necessary.
else {
die "Can not fork child process.";
}
### Kill the server fork.
kill 1, $pid;
done_testing();
This is working well for me.
Although it's OS specific, I'm sure Apache is your answer.
Has anyone ever experienced a unit test that fails and when they tried to debug it to find out where the failure was occurring, the unit test succeeds when running the code in the debugger?
I'm using Eclipse 3.5.1 with EPIC 0.6.35 and ActiveState ActivePerl 5.10.0. I wrote module A and module B both with multiple routines. A routine in module B calls a bunch of routines from module A. I'm adding mock objects to my module B unit test file to try to get more complete code coverage on module B where the code in module B tests to see if all the calls to module As routines fail or succeed. So I added some mock objects to my unit test to force some of the module A routines to return failures, but I was not getting the failures as expected. When I debugged my unit test file, the calls to the module A routine did fail as expected (and my unit test succeeds). When I run the unit test file as normal without debugging, the call to the mocked Module A routine does not fail as expected (and my unit test fails).
What could be going on here? I'll try to post a working example of my problem if I can get it to fail using a small set of simple code.
ADDENDUM: I got my code whittled down to a bare minimum set that demonstrates my problem. Details and a working example of the problem follows:
My Eclipse project contains a "lib" directory with two modules ... MainModule.pm and UtilityModule.pm. My Eclipse project also contains at the top level a unit test file named MainModuleTest.t and a text file called input_file.txt which just contains some garbage text.
EclipseProject/
MainModuleTest.t
input_file.txt
lib/
MainModule.pm
UtilityModule.pm
Contents of the MainModuleTest.t file:
use Test::More qw(no_plan);
use Test::MockModule;
use MainModule qw( mainModuleRoutine );
$testName = "force the Utility Module call to fail";
# set up mock utility routine that fails
my $mocked = new Test::MockModule('UtilityModule');
$mocked->mock( 'slurpFile', undef );
# call the routine under test
my $return_value = mainModuleRoutine( 'input_file.txt' );
if ( defined($return_value) ) {
# failure; actually expected undefined return value
fail($testName);
}
else {
# this is what we expect to occur
pass($testName);
}
Contents of the MainModule.pm file:
package MainModule;
use strict;
use warnings;
use Exporter;
use base qw(Exporter);
use UtilityModule qw( slurpFile );
our #EXPORT_OK = qw( mainModuleRoutine );
sub mainModuleRoutine {
my ( $file_name ) = #_;
my $file_contents = slurpFile($file_name);
if( !defined($file_contents) ) {
# failure
print STDERR "slurpFile() encountered a problem!\n";
return;
}
print "slurpFile() was successful!\n";
return $file_contents;
}
1;
Contents of the UtilityModule.pm file:
package UtilityModule;
use strict;
use warnings;
use Exporter;
use base qw(Exporter);
our #EXPORT_OK = qw( slurpFile );
sub slurpFile {
my ( $file_name ) = #_;
my $filehandle;
my $file_contents = "";
if ( open( $filehandle, '<', $file_name ) ) {
local $/=undef;
$file_contents = <$filehandle>;
local $/='\n';
close( $filehandle );
}
else {
print STDERR "Unable to open $file_name for read: $!";
return;
}
return $file_contents;
}
1;
When I right-click on MainModuleTest.t in Eclipse and select Run As | Perl Local, it gives me the following output:
slurpFile() was successful!
not ok 1 - force the Utility Module call to fail
1..1
# Failed test 'force the Utility Module call to fail'
# at D:/Documents and Settings/[SNIP]/MainModuleTest.t line 13.
# Looks like you failed 1 test of 1.
When I right click on the same unit test file and select Debug As | Perl Local, it gives me the following output:
slurpFile() encountered a problem!
ok 1 - force the Utility Module call to fail
1..1
So, this is obviously a problem. Run As and Debug As should give the same results, right?!?!?
Both Exporter and Test::MockModule work by manipulating the symbol table. Things that do that don't always play nicely together. In this case, Test::MockModule is installing the mocked version of slurpFile into UtilityModule after Exporter has already exported it to MainModule. The alias that MainModule is using still points to the original version.
To fix it, change MainModule to use the fully qualified subroutine name:
my $file_contents = UtilityModule::slurpFile($file_name);
The reason this works in the debugger is that the debugger also uses symbol table manipulation to install hooks. Those hooks must be getting installed in the right way and at the right time to avoid the mismatch that occurs normally.
It's arguable that it's a bug (in the debugger) any time the code behaves differently there than it does when run outside the debugger, but when you have three modules all mucking with the symbol table it's not surprising that things might behave oddly.
Does your mocking manipulate the symbol table? I've seen a bug in the debugger that interferes with symbol table munging. Although in my case the problem was reversed; the code broke under the debugger but worked when run normally.