How can one use the Environment Modules system* in Perl?
Running
system("load module <module>");
does not work, presumably because it forks to another environment.
* Not to be confused with Perl modules. According to the Wikipedia entry:
The Environment Modules system is a tool to help users manage their Unix or Linux shell environment, by allowing groups of related environment-variable settings to be made or removed dynamically.
It looks like the Perl module Env::Modulecmd will do what you want. From the documentation:
Env::Modulecmd provides an automated interface to modulecmd from Perl. The most straightforward use of Env::Modulecmd is for loading and unloading modules at compile time, although many other uses are provided.
Example usage:
use Env::Modulecmd { load => 'foo/1.0' };
Alternately, to do it less perl-namespace like and more environment module shell-like, you can source the Environment Modules initialization perl code like the other shells:
do( '/usr/share/Modules/init/perl');
module('load use.own');
print module('list');
For a one-line example:
perl -e "do ('/usr/share/Modules/init/perl');print module('list');"
(This problem, "source perl environment module" uses such generic words, that it is almost un-searchable.)
system("load module foo ; foo bar");
or, if that doesn't work, then
system("load module foo\nfoo bar");
I'm guessing it makes changes to the environment variables. To change Perl's environment variables, it would have to be executed within the Perl process. That's not going to work since it was surely only designed to be integrated into the shell. (It might not be too hard to port it, though.)
If you are ok with restarting the script after loading the module, you can use the following workaround:
use String::ShellQuote qw( shell_quote );
BEGIN {
if (!#ARGV || $ARGV[0] ne '!!foo_loaded!!') {
my $perl_cmd = shell_quote($^X, '--', $0, '!!foo_loaded!!', #ARGV);
exec("load module foo ; $perl_cmd")
or die $!;
}
shift(#ARGV);
}
Related
Is there any way to report the missing modules used in the Perl file beforehand instead of getting as an error.
I have something like use Digest::MD5, use File::DosGlob modules in my Perl program. Whenever the users run the script they are getting an error if there is no specific module installed in their system. They could not understand the default error message given by #INC. So I would like to clearly tell them that these modules need to be installed to run the script.
You could build your own verification by using a BEGIN block. Those are run at compile time, just like use is. Keep in mind that use Foo is essentially nothing else as this:
BEGIN {
require Foo;
Foo->import;
}
The following code will replace all use statements with a single BEGIN and place them inside an eval. That's essentially like a try/catch mechanism.
We need the string eval (which is considered evil around here) because require only converts from package names with colons :: to paths if the argument is a bareword. But because we have the name in $module, it's a string, so we need to place it into an eval according to require's docs.
If that string eval fails, we die. That's caught by the outer eval block and $# is set. We can then check if it contains our module name, in which case we naively assume the failure was because that module is not installed. This check could be a bit more elaborate.
We keep track of any failures in $fails, and if there were any, we stop.
#!/usr/bin/perl
use strict;
use warnings;
# all our use statements go here
BEGIN {
my $fails;
foreach my $module ( qw/Digest::MD5 File::DosGlob ASDF/ ) {
eval {
eval "require $module" or die; # because $module is not a bareword
$module->import;
};
if ($# && $# =~ /$module/) {
warn "You need to install the $module module";
$fails++;
}
}
exit if $fails;
}
# ...
Above I included ASDF, which I don't have, so when run it will say
You need to install the ASDF module at /home/code/scratch.pl line 1335.
You might want to make that message a bit more verbose. If your users are not able to understand the default error message that Perl gives when it cannot find a module, it might be wise to include a guide on how to install stuff right there.
Note that both modules you listed have been included with Perl for a while (read: since March 2002). So why would you want to do this for those modules?
$ corelist Digest::MD5
Data for 2014-09-14
Digest::MD5 was first released with perl v5.7.3
$ corelist File::DosGlob
Data for 2014-09-14
File::DosGlob was first released with perl 5.00405
A better way would be ship your program as a distribution that can be installed, and include a Makefile or a cpanfile or something similar that lists dependencies. There is a guide in perlnewmod on how to start a new module. You'd not want to upload to CPAN obviously, but the basics are the same.
With this, your users would get all dependencies installed automatically.
You could use Devel::Modlist, it will list all the required module for your program.
perl -d:Modlist test.pl
There's another module Module::ScanDeps which comes with a utility scandeps.pl which you can use on your script as:
scandeps.pl test.pl
Note that sanity checking your Perl code using perl -c is dangerous, so use it carefully.
Your question isn't really clear about what "beforehand" means. To check if a Perl program's syntax is correct and directly included modules are resolvable, use
perl -c <perl-program.pl>
This checks the syntax of your file and ensures that any modules used by your code exist. However, it does not transitively check the entire dependency tree, only those mentioned in perl-program.pl.
I'm running a development environment for (mostly) a CGI app with Linux (openSuse) and Apache. Many of the apps we run used to have CGI::Carp 'fatalsToBrowser' all the time, and of course we do not want that in production. So this is about doing it on one machine, but not on another. Thus the dreadful CGI::Carp needs to vanish from the code.
I know I can load modules with -M from the command line. That one is a no-brainer, but doesn't help me. I don't want command line, I want CGI with Apache.
I've learned on perlmonks that there is something called sitecustomize.pl wich is described in perlrun. It lets you add code that is executed very early during startup. I could probably use it to add a check for the CGI environment and to load the module.
The doc says I have to check $Config{usesitecustomize} to see if my Perl can do that. Unfortunately, perl -e 'print $Config{usesitecustomize}' does not yield anything. I've go the feeling that it doesn't even knof $Config since that is empty when I Data::Dumper it.
I will have a sysadmin just put something there to check, but that may take some time.
Is there any other way to have Perl load the CGI::Carp module and import fatalsToBrowser every time it is started by Apache?
The perlrun documentation that you linked to states that the -f command line flag disables the execution of the usesitecuistomize option if the perl in question was compiled to support that (-Dusesitecustomize).
This option seems to be meant to do rather global changes to a perl configuration, not to alter the behaviour of perl on a per-site-basis.
You can always decide to conditionally load and import a module, e.g.
# CGI::Carp is no pragma, so this works without scoping issues:
BEGIN { eval q{ use CGI::Carp qw/fatalsToBrowser/ } if DEBUG_VERSION };
where DEBUG_VERSION is set to false on deployment.
I ended up adding a switch to the vhosts.conf file for the Apache configuration that tells Perl to always load the module. Since that only affects the development box, it solved the problem:
PERL5OPT="-MCGI::Carp=fatalsToBrowser"
The PERL5OPT variable is described in perlrun.
I cannot use %ENV var on my Perl script to use Oracle libs.
BEGIN {
$ORACLE_HOME = "/usr/lib/oracle/10.2.0.3/client64";
$LD_LIBRARY_PATH = "$ORACLE_HOME/lib";
$ORACLE_SID="prod";
$ENV{ORACLE_SID}=$ORACLE_SID;
$ENV{ORACLE_HOME}= $ORACLE_HOME;
$ENV{LD_LIBRARY_PATH}= $LD_LIBRARY_PATH;
};
If I print $ENV{'ORACLE_HOME'} and $ENV{'LD_LIBRARY_PATH'} all seems ok but, when I run my script I have the error:
install_driver(Oracle) failed: Can't load '/usr/local/lib64/perl5/auto/DBD/Oracle/Oracle.so' for module DBD::Oracle: libclntsh.so.10.1: cannot open shared object file: No such file or directory at /usr/lib64/perl5/DynaLoader.pm line 200.
at (eval 3) line 3
Compilation failed in require at (eval 3) line 3.
Perhaps a required shared library or dll isn't installed where expected
at persistence.perl line 22
Searching on web I saw that the correct way to set env vars on Perl is to use %ENV hash.
Exporting ORACLE_HOME and LD_LIBRARY_PATH through unix shell (export LD_LIBRARY_PATH=...) it works correctly. Any advice?
The LD_LIBRARY_PATH environment variable has to be set before your program starts — before perl itself is loaded. Changing it in BEGIN{} will affect new programs that you start, but it won't affect the loading of shared libraries — in this case (although I've never used the DBD::Oracle) you're loading an Oracle .so into the already-running program, so it's “too late” to change the LD_LIBRARY_PATH. The dynamic linker /lib/ld.so (or so) is started before perl, so by the time your script is compiled and BEGIN{} runs, it's already set up.
You could try to re-exec your script as its own successor or something*, but a short shell script is almost certainly going to be the simplest solution:
#!/bin/sh
export LD_LIBRARY_PATH=/usr/lib/oracle/10.2.0.3/client64/lib
export ORACLE_SID=prod
exec /usr/local/bin/your-db-program "$#"
*- this would be kinda crazy, but TIMTOWTDI:
eval {
use DBD::Oracle foo bar baz; …
};
if ($# =~ /install_driver\(Oracle\) failed/) {
$ENV{LD_LIBRARY_PATH} .= ':/usr/lib/oracle/10.2.0.3/client64/lib';
$ENV{ORACLE_SID} = 'prod';
warn "Restarting with LD_LIBRARY_PATH reset:\n\n$#\n";
exec { $0 } $0 => #ARGV;
}
I wrote a few test scripts to verify that the environment is being set when you change %ENV:
use strict;
use warnings;
use feature qw(say);
BEGIN {
my $foo = "bar-bar";
$ENV{FOO} = "$foo";
}
system qq(/bin/echo printing out \$FOO);
This prints out:
printing out bar-bar
which is what I expected.
I then tried this:
use strict;
use warnings;
use feature qw(say);
BEGIN {
my $foo = "bar-bar";
$ENV{FOO} = "$foo";
}
system qq(./test.sh);
and created a test.sh program that looks like this:
#! /bin/sh
echo This is what I got: $FOO;
In this case, my Perl script is running test.sh which prints out the value of the $FOO environment variable that was set in my Perl script. Running test.pl I get:
This is what I got bar-bar
This shows that not only is Perl setting the environment variables, but that it is also exporting those variables, so called shell scripts have access to them.
You can try a similar technique to verify that both LD_LIBRARY_PATH and ORACLE_HOME are being set before they're used. I suspect you'll find that this is indeed happening, but that your program still isn't working when you set %ENV.
This points to one conclusion: Setting the environment for LD_LIBRARY_PATH and ORACLE_HOME might be occurring too late by the time your Perl script starts. I believe the operating system examines LD_LIBRARY_PATH before Perl starts. I found this doing a search on LD_LIBRARY_PATH:
LD_LIBRARY_PATH is an environment variable you set to give the run-time shared library loader (ld.so) an extra set of directories to look for when searching for shared libraries. Multiple directories can be listed, separated with a colon (:). This list is prepended to the existing list of compiled-in loader paths for a given executable, and any system default loader paths.
So, LD_LIBRARY_PATH is for the ld.so runtime shared library loader, If ld.so has already been loaded, changing LD_LIBRARY_PATH won't do anything.
I found a similar discussion on Perl Monks. I noticed someone found rerunning env seemed to work.
One solution is to modify /etc/ld.so.conf
On CentOS/RHEL 6.4, you could create etc/ld.so.conf.d/oracle with this:
/oracle/sw/product/11.2.0/dbhome_1/lib
Obviously, modify as suits your ORACLE_HOME.
Then run
ldconfig -v
You could put the export commands into the start up script for your unix shell which you should have permission to edit. That way, the environment variables will be set whenever you start a new shell and all scripts and programs that use Oracle will pick them up.
I just went through something similar. I had to make sure that the Oracle environment was setup before anything else called it. Make sure the BEGIN block is before any other "use" statements. In my case, something was being called in Apache's httpd.conf file, so I had to setup my environment there instead of in my package.
I am facing the following problem:
I am working on a perl project consisting of a number of modules and scripts. The project must run on two different machines.
Throughout the project i call external programs, but the paths are different on both machines, so I would like to define them once globally for all files and then only change this definition when i switch machines.
Since I am fairly new to perl I ask you what would be a common way to accomplish this.
Should I use "use define" or global variables or something else?
Thanks in advance!
If I were you, I'd definitely do my best to avoid global variables - they are a sign of weak coding style (in any language) and offer you a maintenance hell.
Instead, you could create and use configuration files - one for each of your machines. Being on Perl, you have plenty of options for free, ready to use CPAN modules:
Config::Auto
Config::JSON
Config::YAML
And many many other
Rather than defining globals which may or may not work, why not use a subroutine to find a working executable?
my $program = program_finder();
sub program_finder {
-x && return $_ for qw( /bin/perl /usr/bin/perl /usr/local/bin/perl );
die "Could not find a perl executable";
}
Create a module to hold your configuration information.
In file My/Config.pm in your perl library path:
package My::Config;
use warnings;
use strict;
use Carp ();
my %setup = (
one => {path => '/some/path'},
two => {path => '/other/path'},
);
my $config = $setup{ $ENV{MYCONFIG} }
or Carp::croak "environment variable MYCONFIG must be set to one of: "
.(join ' ' => keys %setup)."\n";
sub AUTOLOAD {
my ($key) = our $AUTOLOAD =~ /([^:]+)$/;
exists $$config{$key} or Carp::croak "no config for '$key'";
$$config{$key}
}
And then in your files:
use My::Config;
my $path = My::Config->path;
And of course on your machines, set the environment variable MYCONFIG to one of the keys in %setup.
How can I check if a Perl module is part of the core - i.e. it is part of the standard installation?
I'm looking for:
a command-line command:
a Perl subroutine/function to check within code
Perhaps the question should be: How can I tell what modules were originally provided with the specific Perl installation on a machine? (Actually, it is now asked as How can I tell what modules were originally provided with the specific Perl installation on a machine?.)
Given that there now appears to not to be an overall Perl standard installation, at least the answer to this new question will tell me what I originally had in the installation when it was first installed.
With that knowledge and if I keep the original installer image/package OR know how to get the exact thing again online, then I have a repeatable Perl installation for several machines with the knowledge of what modules will be present and what modules will not.
To clarify further: I am looking at what came with the installation originally, what modules were provided as part of that installation, and what was built-in. NOT what has been installed since then.
And I want to be able to do this on the machine that has the installation. So for this I would be relying upon the installation to have a record in some form as to what it has originally.
I asked spin-off question:
How can I tell what modules were originally provided with the specific Perl installation on a machine? (How can I tell what modules were originally provided with the specific Perl installation on a machine?)
The corelist command from the Module::CoreList module will determine if a module is Core or not.
> corelist Carp
Carp was first release with perl 5
> corelist XML::Twig
XML::Twig was not in CORE (or so I think)
Here is one way to use it in a script. The Module::CoreList POD is too terse -- you have to go hunting through the source code to find what methods to call:
use strict;
use warnings;
use Module::CoreList;
my $mod = 'Carp';
#my $mod = 'XML::Twig';
my #ms = Module::CoreList->find_modules(qr/^$mod$/);
if (#ms) {
print "$mod in core\n";
}
else {
print "$mod not in core\n";
}
__END__
Carp in core
You could check perlmodlib in a sub:
my %_stdmod;
sub is_standard_module {
my($module) = #_;
unless (keys %_stdmod) {
chomp(my $perlmodlib = `perldoc -l perlmodlib`);
die "cannot locate perlmodlib\n" unless $perlmodlib;
open my $fh, "<", $perlmodlib
or die "$0: open $perlmodlib: $!\n";
while (<$fh>) {
next unless /^=head\d\s+Pragmatic\s+Modules/ ..
/^=head\d\s+CPAN/;
if (/^=item\s+(\w+(::\w+)*)/) {
++$_stdmod{ lc $1 };
}
}
}
exists $_stdmod{ lc $module } ? $module : ();
}
Example usage:
die "Usage: $0 module..\n" unless #ARGV;
foreach my $mod (#ARGV) {
my $stdmod = is_standard_module $mod;
print "$0: $mod is ", ($stdmod ? "" : "not "), "standard\n";
}
Output:
$ ./isstdmod threads::shared AnyDBM_File CGI LWP::Simple
./isstdmod: threads::shared is standard
./isstdmod: AnyDBM_File is standard
./isstdmod: CGI is standard
./isstdmod: LWP::Simple is not standard
perldoc is most definitely part of the Perl's true core and standard installation. The source distribution for perl-5.10.1, for example, contains
perldoc.PL, generates perldoc as part of the standard installation
perlmodlib.PL, generates perlmodlib.pod as part of the standard installation
This is not a new addition. Perl-5.6.0, about ten years old, had perlmodlib as part of its true-core, standard installation.
Installations that do not contain these items are non-standard. Yes, I appreciate that it may seem academic from your perspective, but your vendor's packaging permitted a non-standard installation that breaks otherwise working programs.
With Debian's package manager, you can get the standard Perl installation with
$ apt-get --install-recommends install perl
There really is no such thing as "core" any more. There used to be a standard Perl distribution, but a lot of people don't have a standard Perl distribution. Operating system distributions modify it by either adding or removing modules, changing modules, and so on. You can't rely on the standard distribution being actually standard. Some Linux distributions don't even include the Perl documentation as part of the base Perl installation.
You mention that you can't use Module::CoreList because it isn't core, but if you can create files, you can install the module. You can even pretend that you wrote it yourself.
For the really lazy, there's the Core Modules list on the perldoc.perl.org website.
You can use (for example, search for Net::FTP):
perl -MNet::FTP -e 1
If it doesn't have output, then it's installed.
Other resources
perldoc perlmodlib
perldoc perllocal
A node from perlmonks
In a response to a comment of Gbacon's, you say that you want the answer to be platform neutral. I don't know of such a solution, but I wonder if it's even the right way to go.
If your goal is to find out on specific machines, I would use the tools that come with the platform. On Debian, that would include dpkg (pre-installed on any Debian system) or apt-file (not pre-installed necessarily) or other APT tools.
As an example, take a look at the output of this:
dpkg-query -L perl | less
You would obviously need to parse the output, but it strikes me as a start precisely because it is specific to the machine in question.
From the command-line:
Let's say that you want to know
whether module Tie::Hash is
installed.
To find out, execute
the following from the command line:
perl -MTie::Hash -e 1
If you don't get any output from the above command then the module is installed; if you get an error, it's not installed.
For making this check from within the script you can make use of Module::Load::Conditional.