Using root logger across multiple modules - perl

ActiveState Perl 5.12 on WinXP
I've recently become a convert to Log4perl and used it successfully where I defined a specific logger for each module that makes up my application. Example:
Upload.pm-----
my $logger = get_logger("Upload"); # specific logger
my $layout = Log::Log4perl::Layout::PatternLayout->new( "%d %p> %F{1}:%L %M - %m%n");
my $appender = Log::Log4perl::Appender->new( "Log::Dispatch::File",
filename => "Upload.log",
mode => "append"
);
$appender->layout($layout);
$logger->level($OFF); # or $INFO
$logger->add_appender($appender)
....
This works but is hard to track program flow across numerous log files ie Upload.log , Parse.log , FileRead.log etc. Quick-n-dirty solution: use same filename in all loggers.
This works much better, but only as long as the program is used serially and in sequence.
Now for the wrinkle - suppose module Upload.pm is used by several progams ie readMyFiles.pl, readHerFiles.pl and dumpAllFiles.pl . When running readMyFiles.pl I want the logger in Upload.pm to write to readMyFiles.pl.log and when runnng dumpAllFiles.pl want the logger in Upload.pm to write to dumpAllFiles.pl.log
One method might be to declare an our variable $logFileName in my .pl files and use it in all my modules like so:
filename => $logFileName,
Another might be to remove all the loggers from my .pm's and define them only in the .pl's - but then how would I reference $logger in the .pm's ?
All thoughts and suggestions are appreciated.
Still-learning Steve

Configure your logging settings in the caller, not in the module. If somebody else uses your module, they might not want to log things to the same place that you do. They might also want to format log messages differently, or use a different type of appender, or...the list goes on.
Your module should only get a logger and write messages to it:
MyModule.pm
#!/usr/bin/perl
package MyModule;
use strict;
use Log::Log4perl qw(get_logger);
sub foo {
my $logger = get_logger("Foo");
$logger->debug("Hello from MyModule");
}
1;
Your main program(s) should configure and initialize logging:
logtest
#!/usr/bin/perl
use strict;
use warnings;
use Log::Log4perl qw(get_logger);
use MyModule;
Log::Log4perl->init("log4perl.cfg");
my $logger = get_logger("Foo");
$logger->debug("Hello from main");
MyModule::foo();
I prefer to use a separate config file for Log4perl settings:
log4perl.cfg
log4perl.logger.Foo=DEBUG, Screen
log4perl.appender.Screen=Log::Dispatch::Screen
log4perl.appender.Screen.Threshold=DEBUG
log4perl.appender.Screen.layout=Log::Log4perl::Layout::PatternLayout
log4perl.appender.Screen.layout.ConversionPattern=[%r] %F %L %c - %m%n
Output:
[0] logtest 12 Foo - Hello from main
[0] MyModule.pm 11 Foo - Hello from MyModule
Note that you must initialize logging with Log::Log4perl->init in the caller before trying to get_logger in the module. If you don't, log messages from the module will be ignored and you'll get the following warning:
Log4perl: Seems like no initialization happened.
Forgot to call init()?
See the documentation for details.

Related

Change Environment Variables During Perl Unit Testing

I am trying to write some unit tests for this perl module function, but am having some issues with environment variables. I'll list the files first, then explain the issue in greater detail.
processBuildSubs.pm
package processBuildSubs;
use strict;
use warnings;
use LWP::UserAgent;
use HTTP::Request::Common;
use HTTP::Status;
# Declare environment variables used in this package that are needed
use constant URI_BASE => $ENV {"URI_BASE"};
use constant URI_RESOURCE => $ENV {"URI_RESOURCE"};
# Shell Environment Related Constants visible.
# Make visible.
our $URI_BASE = URI_BASE;
our $URI_RESOURCE = URI_RESOURCE;
sub populatePartitions
{
# Define locals
my $url;
my $ua = new LWP::UserAgent;
$url = "$URI_BASE"."$URI_RESOURCE"."/some/path";
# Make a request to the $url
$res = $ua->request (GET $url);
if ($res->code() != HTTP::Status->RC_OK() )
{
# The request didn't return 200 OK so it's in here now.
}
else
{
# The request returned 200 OK, so now it's here.
}
}
I want to be able to unit test both the if path and the else path, however, it would be best for me if I don't need to change the processBuildSubs.pm code at all. It's an external file that I don't currently have control over. I am just tasked in unit testing it (although I do understand it could be tested more efficiently if we could also change the source code).
So in order to test both paths, we need the environment variables URI_BASE and URI_RESOURCE to be set accordingly, so that the request fails once, and succeeds another time. (I am interested in learning how to stub out this call at a future time, but that's reserved for another question.)
Here's my test file:
processBuildSubs.t
use strict;
use Test::More qw(no_plan);
BEGIN { use_ok('processBuildSubs') };
# Test 1 of populatePartitions() function
my $processBuildProdsCall = processBuildSubs::populatePartitions();
is( $populatePartitionsCall, 0, "populatePartitions() Test for 0 Val Passed" );
# Test 2 of populatePartitions() function
# I need to change some environment variables that processBuildSubs depends on here.
my $processBuildProdsCall = processBuildSubs::populatePartitions();
is( $populatePartitionsCall, 0, "populatePartitions() Test for 0 Val Passed" );
The best attempt we have right now at changing the environment variables is using an external shell script like so (But it would be ideal to change them between the my calls in the file above):
run_tests.sh
#!/bin/bash
# Run the tests once
perl ./BuildProcess.pl
perl ./Build testcover # Ultimately calls the processBuildSubs.t test file
# Now export some variables so the other test passes.
export URI_BASE="https://some-alias/"
export URI_RESOURCE="some-resource"
# Then run the test suite again with the env set so the else condition passes.
perl ./BuildProcess.pl
perl ./Build testcover
As you can see, this would be a bad way of doing things, as we run the entire test suite with different environments each time. Ideally we'd like to setup our environment in the processBuildSubs.t file if possible, between tests.
Please let me know if I can provide any further information.
Are you averse to having separate scripts for separate test environments?
# processBuildSubs.t
BEGIN {
#ENV{"URI_BASE","URI_RESOURCE"} = ("https://some-alias/","some-resource");
}
use Test::More;
... tests go here ...
# processBuildSubs-env2.t
BEGIN {
#ENV{"URI_BASE","URI_RESOURCE"} = ("https://another-alias/","another-resource");
}
use Test::More;
... tests go here ...
By setting %ENV in a BEGIN block, before any other modules are loaded, you make the different environment variables available to your other modules at compile-time.

How to pass environment variable to an AutoLoaded mod_perl handler, to be used at module load time?

I have a HTTP Request Handler for mod_perl which needs to read an environment variable, from %ENV, at module load time. The environment variable is passed from the Apache config into mod_perl using the PerlSetEnv directive.
This worked fine, until we changed the Apache configuration to AutoLoad the handler at startup time, for performance reasons. When the module is AutoLoaded like this, thePerlSetEnv does not take effect at module load time, and the variable we need is only available from %ENV at request time inside the handler method.
Is there a way to continue using AutoLoad, but still set an environment variable in the Apache config which is available in Perl's %ENV at module load time?
Minimal example:
Here's a stripped down test-case to illustrate the problem.
The Apache config without autoload enabled:
PerlSwitches -I/home/day/modperl
<Location /perl>
SetHandler modperl
PerlSetEnv TEST_PERLSETENV 'Does it work?'
PerlResponseHandler ModPerl::Test
Allow from all
</Location>
Contents of /home/day/modperl/ModPerl/Test.pm:
package ModPerl::Test;
use strict;
use warnings;
use Apache2::RequestRec ();
use Apache2::RequestIO ();
use Apache2::Const qw(OK);
my %ENV_AT_MODULE_LOAD = %ENV; # Take a copy
sub handler {
my $r = shift;
$r->content_type('text/plain');
$r->print("ENV:\n");
foreach my $key (sort keys %ENV) {
$r->print(" $key: $ENV{$key}\n");
}
$r->print("ENV_AT_MODULE_LOAD:\n");
foreach my $key (sort keys %ENV_AT_MODULE_LOAD) {
$r->print(" $key: $ENV_AT_MODULE_LOAD{$key}\n");
}
return OK;
}
1;
When localhost/perl is viewed in the browser, I see this:
ENV:
MOD_PERL: mod_perl/2.0.5
MOD_PERL_API_VERSION: 2
PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
TEST_PERLSETENV: Does it work?
ENV_AT_MODULE_LOAD:
MOD_PERL: mod_perl/2.0.5
MOD_PERL_API_VERSION: 2
PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
TEST_PERLSETENV: Does it work?
Hooray! TEST_PERLSETENV is available at module load time, as we want.
But when we change the Apache config to enable Autoload (by using + in the PerlResponseHandler like so):
PerlResponseHandler +ModPerl::Test
I get the following output instead:
ENV:
MOD_PERL: mod_perl/2.0.5
MOD_PERL_API_VERSION: 2
PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
TEST_PERLSETENV: Does it work?
ENV_AT_MODULE_LOAD:
MOD_PERL: mod_perl/2.0.5
MOD_PERL_API_VERSION: 2
PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
Boo! TEST_PERLSETENV is no longer available at module load time :( How can I get it back while keeping the AutoLoad behaviour?
Argh, 30 seconds after posting this question, I found the answer. Thank you rubber duck.
Move the PerlSetEnv to before the <Location> block which contains the PerlResponseHandler directive, and it works again!
i.e. like this:
PerlSwitches -I/home/dbarr/modperl
PerlSetEnv TEST_PERLSETENV 'Does it work?'
<Location /perl>
SetHandler modperl
PerlResponseHandler +ModPerl::Test
Allow from all
</Location>

Perl Rover v3 pass environment variable to in the Rulesets

I am using perl Rover module version 3 to login to the Linux/Unix server and run the script. In the ruleset if I add the full path name it copies the script to the remote server, not able to substitute the environment variable.
eg.
This works:
copy:{
put_file "/home/u1/find.sh" "/tmp/"
};
This didn't work:
copy:{
put_file "$HOME/find.sh" "/tmp/"
};
used $ENV{'HOME'}, this also didn't work.
How can I pass the environment variable?
Rover module document.
http://rover.sourceforge.net/QuickStart/Rover-QuickStart-3.html#ss3.2
http://rover.sourceforge.net/
After reviewing the source code for rover, which I never used, I determined it was not possible from the existing code.
I created a new extension for you, that has that functionality, it supports the ~ and ${HOME} syntax, (which are bash extensions and not part of the OS directly, that is why perl does not support them).
code is here:
https://github.com/h4ck3rm1k3/perl-rover/commit/2c78aefb97e819956bb665b04056763f8df1b242
I have had a hard time testing it because I never used rover before, and rover does not seem to support scp.(I read it is supported,but could not test it yet.) Anyway, let me know if you like it. I will put more work into it if reasonably requested.
Update
Here is my example ruleset :
example ruleset
[rulesets]
test:
{
put_file_from_home put_file "~/find2.sh" "/tmp/"
put_file_from_home put_file "${HOME}/find3.sh" "/tmp/"
}, ;
example output
Here is the example output, I cannot get rover to work. See the test case below.
Test output
perl -I lib t/example2.t
Local was ~/find2.sh and home was /home/mdupont at lib/Rover/CoreExtension.pm line 19.
Local now /home/mdupont/find2.sh at lib/Rover/CoreExtension.pm line 22.
Local was ${HOME}/find3.sh and home was /home/mdupont at lib/Rover/CoreExtension.pm line 19.
Local now /home/mdupont/find3.sh at lib//Rover/CoreExtension.pm line 22.
new config option for the new sshport option
[hosts]
someexample:{
os linux
username myusername
description 'myhost'
sshport 12345
ftp_method_used sftp
};
update2
Dont use quotes around the name, use a comma between the args,
To git#github.com:h4ck3rm1k3/perl-rover.git
2207417..7637741 CoreExtension -> CoreExtension
[rulesets]
test: { put_file_from_home ~/find2.sh,/tmp/ }, ;
[hosts]
localhost:{
os linux
username mdupont
description 'localhost'
ftp_methods sftp
ftp_method_used sftp };
mike
Old question but new answer, since your using Rover v3 you can just extend the Rover::Core modules by overloading it.
Add this to your ~/.rover/contrib directory:
CoreVariables.pm:
package CoreVariables;
use strict;
use Exporter;
our #ISA = qw( Exporter );
our #EXPORT = qw( put_file );
sub put_file {
my ($self, $host, $command) = #_;
$command =~ s/(\$[\w{}]+)/$1/eeg;
return Rover::Core::put_file($self, $host, $command);
}
And add the following to your ~/.rover/config [modules] section (must be after Rover::Core):
CoreVariables:{
};
And then you can store environment variables in your rover config when using put_file. Add other routines if you wish, this only extends put_file.
And since this is such an easy task I will add it to the requested feature list and include it in the next release (I am the Rover author).
The better place to ask Rover questions is on the sourceforge website of course: http://sourceforge.net/projects/rover/

How to access a simple SOAP Service in Perl

I am currently dabbling around with perl and SOAP, using SOAP::Lite.
I have a simple SOAP server that appears to run fine:
#!perl -w
use SOAP::Transport::HTTP;
use Demo;
# don't want to die on 'Broken pipe' or Ctrl-C
$SIG{PIPE} = $SIG{INT} = 'IGNORE';
my $daemon = SOAP::Transport::HTTP::Daemon
-> new (LocalPort => 801)
-> dispatch_to('/home/soaplite/modules')
;
print "Contact to SOAP server at ", $daemon->url, "\n";
$daemon->handle;
It includes a small class called Demo, which simply retrieves the systems total memory:
Demo.py
#!/usr/bin/perl
use Sys::MemInfo qw(totalmem freemem totalswap);
print "total memory: ".(&totalmem / 1024)."\n";
I have an example of a SOAP client below written in PERL, although I am unsure how to communicate with the server (since the tutorial I am following here goes of on a tangent e.g. retrieve the result of the Demo.py class from the client:
#!perl -w
use SOAP::Lite;
# Frontier http://www.userland.com/
$s = SOAP::Lite
-> uri('/examples')
-> on_action(sub { sprintf '"%s"', shift })
-> proxy('http://superhonker.userland.com/')
;
print $s->getStateName(SOAP::Data->name(statenum => 25))->result;
Any help would be greatly appreciated :)
For the server script, the dispatch_to method takes the path to the package to load, and the name of the package itself. If you pass a third parameter, it will limit the names of the methods made visible by the server. (e.g. 2 methods named memory and time, passing Demo::time as the 3rd param will make memory invisible to the client service.)
File server.pl
my $daemon = SOAP::Transport::HTTP::Daemon
-> new (LocalPort => 801)
-> dispatch_to('/home/soaplite/modules', 'Demo')
;
Your Demo package should be a package with methods that return the values. I couldn't get Sys::MemInfo compiled on my system, so I just used localtime instead. I'm not sure why you named your package Demo.py, but Perl packages must have the extension pm, otherwise they won't be properly loaded.
File Demo.pm
#!/usr/bin/perl
package Demo;
#use Sys::MemInfo qw(totalmem freemem totalswap);
sub memory {
#print "total memory: ".(&totalmem / 1024)."\n";
return "Can't load Sys::MemInfo, sorry";
}
sub time {
my $time = localtime;
return $time;
}
1;
For the client code, there's 2 important pieces that must be properly specified to work, the proxy and the uri. The proxy is the url path to the soap web service. Since you are running the server script as a daemon process, your path is just the web site's url. My computer doesn't have a url, so I used http://localhost:801/. The 801 is the port you specified above. If you were running as a cgi script inside of a different web server (such as Apache), then you would need to specify the cgi script to call (e.g. http://localhost/cgi-bin/server.pl, changing the package in server.pl to SOAP::Transport::HTTP::CGI.
uri is probably the most confusing, but it's the namespace of the xml files returned by the web service. Turn on +trace => 'debug' to see the xml file returned by the web service. The uri should just be the name of the server. Even if you switch ports or to a cgi dispatch method, this uri stays the same.
File test.pl
#!perl -w
use SOAP::Lite +trace => 'debug';
# Frontier http://www.userland.com/
$s = SOAP::Lite->new(proxy => 'http://superhonker.userland.com:801/',
uri => 'http://superhonker.userland.com/');
#might be http://www.userland.com/
#but I could not test sub-domains
print $s->time()->result;
I'll recycle these two answers for tips:
Client of web service in Perl
Remote function call using SOAP::Lite

Why does my Perl unit test fail in EPIC but work in the debugger?

Has anyone ever experienced a unit test that fails and when they tried to debug it to find out where the failure was occurring, the unit test succeeds when running the code in the debugger?
I'm using Eclipse 3.5.1 with EPIC 0.6.35 and ActiveState ActivePerl 5.10.0. I wrote module A and module B both with multiple routines. A routine in module B calls a bunch of routines from module A. I'm adding mock objects to my module B unit test file to try to get more complete code coverage on module B where the code in module B tests to see if all the calls to module As routines fail or succeed. So I added some mock objects to my unit test to force some of the module A routines to return failures, but I was not getting the failures as expected. When I debugged my unit test file, the calls to the module A routine did fail as expected (and my unit test succeeds). When I run the unit test file as normal without debugging, the call to the mocked Module A routine does not fail as expected (and my unit test fails).
What could be going on here? I'll try to post a working example of my problem if I can get it to fail using a small set of simple code.
ADDENDUM: I got my code whittled down to a bare minimum set that demonstrates my problem. Details and a working example of the problem follows:
My Eclipse project contains a "lib" directory with two modules ... MainModule.pm and UtilityModule.pm. My Eclipse project also contains at the top level a unit test file named MainModuleTest.t and a text file called input_file.txt which just contains some garbage text.
EclipseProject/
MainModuleTest.t
input_file.txt
lib/
MainModule.pm
UtilityModule.pm
Contents of the MainModuleTest.t file:
use Test::More qw(no_plan);
use Test::MockModule;
use MainModule qw( mainModuleRoutine );
$testName = "force the Utility Module call to fail";
# set up mock utility routine that fails
my $mocked = new Test::MockModule('UtilityModule');
$mocked->mock( 'slurpFile', undef );
# call the routine under test
my $return_value = mainModuleRoutine( 'input_file.txt' );
if ( defined($return_value) ) {
# failure; actually expected undefined return value
fail($testName);
}
else {
# this is what we expect to occur
pass($testName);
}
Contents of the MainModule.pm file:
package MainModule;
use strict;
use warnings;
use Exporter;
use base qw(Exporter);
use UtilityModule qw( slurpFile );
our #EXPORT_OK = qw( mainModuleRoutine );
sub mainModuleRoutine {
my ( $file_name ) = #_;
my $file_contents = slurpFile($file_name);
if( !defined($file_contents) ) {
# failure
print STDERR "slurpFile() encountered a problem!\n";
return;
}
print "slurpFile() was successful!\n";
return $file_contents;
}
1;
Contents of the UtilityModule.pm file:
package UtilityModule;
use strict;
use warnings;
use Exporter;
use base qw(Exporter);
our #EXPORT_OK = qw( slurpFile );
sub slurpFile {
my ( $file_name ) = #_;
my $filehandle;
my $file_contents = "";
if ( open( $filehandle, '<', $file_name ) ) {
local $/=undef;
$file_contents = <$filehandle>;
local $/='\n';
close( $filehandle );
}
else {
print STDERR "Unable to open $file_name for read: $!";
return;
}
return $file_contents;
}
1;
When I right-click on MainModuleTest.t in Eclipse and select Run As | Perl Local, it gives me the following output:
slurpFile() was successful!
not ok 1 - force the Utility Module call to fail
1..1
# Failed test 'force the Utility Module call to fail'
# at D:/Documents and Settings/[SNIP]/MainModuleTest.t line 13.
# Looks like you failed 1 test of 1.
When I right click on the same unit test file and select Debug As | Perl Local, it gives me the following output:
slurpFile() encountered a problem!
ok 1 - force the Utility Module call to fail
1..1
So, this is obviously a problem. Run As and Debug As should give the same results, right?!?!?
Both Exporter and Test::MockModule work by manipulating the symbol table. Things that do that don't always play nicely together. In this case, Test::MockModule is installing the mocked version of slurpFile into UtilityModule after Exporter has already exported it to MainModule. The alias that MainModule is using still points to the original version.
To fix it, change MainModule to use the fully qualified subroutine name:
my $file_contents = UtilityModule::slurpFile($file_name);
The reason this works in the debugger is that the debugger also uses symbol table manipulation to install hooks. Those hooks must be getting installed in the right way and at the right time to avoid the mismatch that occurs normally.
It's arguable that it's a bug (in the debugger) any time the code behaves differently there than it does when run outside the debugger, but when you have three modules all mucking with the symbol table it's not surprising that things might behave oddly.
Does your mocking manipulate the symbol table? I've seen a bug in the debugger that interferes with symbol table munging. Although in my case the problem was reversed; the code broke under the debugger but worked when run normally.