Perl can't find module when run from cron.daily - perl

I have perl programs that use Net::Finger and have run successfully from cron.daily in Fedora 11.
I just upgraded a server to Fedora 18 and these same perl programs no longer run from cron but run fine from command line when logged in as root.
The error is:
Can't locate Net/Finger.pm in #INC (#INC contains: /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .)
The path to the module is /root/perl5/lib/perl5/Net/Finger.pm but I can't figure out how to add the path without causing more errors.
Thanks in advance.

See perlfaq8.
Here are three ways to add arbitrary directories to Perl's module search path.
Set the PERL5LIB environment variable
15 15 * * 1-5 PERL5LIB=/root/perl5/lib/perl5 /usr/local/bin/perl my_script.pl
Use the -I command line switch
15 15 * * 1-5 /usr/local/bin/perl -I/root/perl5/lib/perl5 my_script.pl
Use the lib pragma inside your perl script
#! /usr/local/bin/perl
# my_script.pl: the script that does my thing
use lib '/root/perl5/lib/perl5';
use Net::Finger;
...
Also note that the environment of a cron job is much sparser than the environment of your command line, and in particular the cron environment's $PATH variable might not be what you expect. If you're not specifying the full path to the Perl executable, verify what $PATH the cron environment is using and make sure you are running the right version of perl.

What happens if you add this to the top of the script?
#!/usr/bin/perl
use lib "/root/perl5/lib/perl5/Net";
Was this previously set in the .profile?
This is from a ubuntu thread, but may be related:
https://askubuntu.com/questions/23009/reasons-why-crontab-does-not-work
*Cron passes a minimal set of environment variables to your jobs. To see the difference, add a dummy job like this:
* * * * * env > /tmp/env.output*
I don't have a linux box handy at the moment, but I would try to see what's different in the #INC by running the following logged in as root and trying to set up a job to see what the crontab has. The env output above may indicate a different shell is being used by crontab, than what you may have set when logging in as root.
log in as root and run:
perl -le 'print for #INC'
set up a cron job to see the differences. I'm not sure if this syntax will work, but you could put the command in a simple pl script.
* * * * * perl -le 'print for #INC' > /tmp/inc.output
do the same for env

Related

How do I call a Perl script in an SBATCH script for SLURM submissions?

I received a Perl script that apparently is called from an SBATCH script to be submitted as a job to a computer cluster managed by SLURM. The script is old and I am yet to become more familiar with Perl. Additionally, the Perl script is being used as wrapper to call an executable with mpiexec_mpt.
But whenever I do sbatch sbatch_submission, the Perl script is executed by the computer node but I don't obtain any output or execution of the system() method - or I do but I don't know where it is.
I know Perl is executed by SBATCH because I got an error that it couldn't find a module so I manually pointed Perl to the library path using the -l flag as shown below. But after that I don't get any output.
The SBATCH script and the perl script are below:
SBATCH SCRIPT
1 #!/bin/bash
2 #SBATCH --job-name=job_submission
3 #SBATCH --output=output_perl.run
4 #SBATCH --error=error_perl.run
5 #SBATCH -n 2 # request 2 cores
6 #SBATCH --constraint=intel
7
8 # Load Needed Modules:
9 module load mpt
10
11 # Set-up environment for perl:
12
13
14
15 # Running perl script:
16 echo "Calling simple hello_world.c with perl (sbatch)"
17
18 perl input_perl.pl 1> perl_in.stdout 2> perl_in.stderr # edit after
# suggestions
19 echo "Done with perl script (sbatch)"
20
PERL INPUT
1 #!/usr/bin/perl -w
2 use strict;
3 use warnings;
4 use diagnostics;
5 use List::MoreUtils qw(indexes); ## edit after suggestions
6 system("echo this is your hostname:");
7 system("hostname");
8 system("mpiexec_mpt -np 2 hello_world");
9 print "Done executing hello world! from within perl script!\n"
OUTPUT FROM STDERR
1 Can't locate List/MoreUtils.pm in #INC (#INC contains: /usr/lib64/perl5/vendor_perl/List /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl / usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .) at input_perl.pl line 5.
Aside from the output above the output files: perl.output and output_perl.run are empty.
I suspect I am missing something regarding the applicability of the system() method in Perl, as well as how to tell Perl where to send it's output when working with slurm. I have also tried generating a .txt file with the Perl script, but when I run it with SBATCH the .txt file is not generated. I have no issues running the perl_input.pl without using the SBATCH script as wrapper: e.g: perl perl_input.pl.
Additional info, the hello_world executable has been written in C and I have tested it independently and it runs. It is a simple MPI program that lists ranks and size. I don't think that's the issue though.
Independently and running locally the Perl and C scripts run, it's when I use SBATCH that the issues arise.
I am not sure this is a solution but this is what worked for me.
Ended up re-installing perl using the instructions here:
https://learn.perl.org/installing/unix_linux.html
I work from a computer cluster so there might be something going on with the perl installation I was using. The module call that was causing the problem seems to be installed but perl can't find it when I call the script from SBATCH.
I installed the module I needed: List::MoreUtils as follows:
perl -MCPAN -Mlocal::lib -e 'install List::MoreUtils'
Link: https://www.perlmonks.org/?node_id=1117597
Logged out and then started a new session. The new perl installation sources a bashrc file that automatically updates the perl version to the one that was just installed. So whenever you run a perl script you don't use the old installation.
After that, I set the PERL5LIB variable to the path where MoreUtils is located and everything worked.

Can't locate Text/Soundex.pm in #INC

I was installing repeatsmasker and it apparently seems to work because it shows "Congratulations! RepeatMasker is now ready to use."
But when I run it it reports "Can't locate Text/Soundex.pm...". so I installed the module by "sudo cpan Text::Soundex", and by the end it tells me "Text::Soundex is up to date (3.05)." It seems the module is already installed, but RepeatMasker still has the same problem, as I'll show you in this code:
fragua#picci:~/RM/RepeatMasker$ sudo cpan Text::Soundex
Loading internal null logger. Install Log::Log4perl for logging messages
Reading '/home/fragua/.cpan/Metadata'
Database was generated on Fri, 19 Apr 2019 22:17:03 GMT
Text::Soundex is up to date (3.05).
fragua#picci:~/RM/RepeatMasker$ ./RepeatMasker -s -lib /home/fragua/RepeatScout-1.0.5/ObiINK5k_repeats_filtered1.fasta /home/fragua/Documenti/Workdirectory/ObiINC5k.fa
Can't locate Text/Soundex.pm in #INC (you may need to install the Text::Soundex module) (#INC contains: /home/fragua/RM/RepeatMasker /home/fragua/perl5/lib/perl5 /home/fragua/anaconda/lib/site_perl/5.26.2/x86_64-linux-thread-multi /home/fragua/anaconda/lib/site_perl/5.26.2 /home/fragua/anaconda/lib/5.26.2/x86_64-linux-thread-multi /home/fragua/anaconda/lib/5.26.2 .) at /home/fragua/RM/RepeatMasker/Taxonomy.pm line 80.
BEGIN failed--compilation aborted at /home/fragua/RM/RepeatMasker/Taxonomy.pm line 80.
Compilation failed in require at ./RepeatMasker line 310.
BEGIN failed--compilation aborted at ./RepeatMasker line 310.
I installed RepeatMasker in another computer without problems, but I don't know why now I encontered this problem
You have two builds of Perl installed:
/usr/bin/perl
/home/fragua/anaconda/bin/perl.
/home/fragua/anaconda/bin/perl is first in your PATH. This means that programs with the following shebang (#!) line will use /home/fragua/anaconda/bin/perl:
#!/usr/bin/env perl
RepeatMasker appears to be such a program.
All of this is fine.
The Problem
/home/fragua/anaconda/bin contains the scripts installed by /home/fragua/anaconda/bin/perl. As part of the installation process of these scripts, the shebang line of these scripts should have been rewritten to specify /home/fragua/anaconda/bin/perl.
However, the shebang line of /home/fragua/anaconda/bin/cpan references /usr/bin/perl. This means that using /home/fragua/anaconda/bin/cpan would install modules for /usr/bin/perl, not /home/fragua/anaconda/bin/perl.
The Workaround
You could avoid relying on the shebang line and explicitly specify the correct perl.
/home/fragua/anaconda/bin/perl /home/fragua/anaconda/bin/cpan Text::Soundex
Or, given your $PATH,
perl /home/fragua/anaconda/bin/cpan Text::Soundex
The Fix
To fix this problem in an ongoing manner requires changing the shebang lines of the scripts to be what they should be. In every file in /home/fragua/anaconda/bin (and in particular for cpan), replace
#!/usr/bin/perl
with
#!/home/fragua/anaconda/bin/perl
You could do this use the following (which does a backup of the files it changes):
perl -0777ne'print "$ARGV\n" if m{^#!\s*/usr/bin/perl\b}' /home/fragua/anaconda/bin/* \
| xargs perl -i~ -0777pe's{^#!\s*/usr/bin/perl\b}{#!/home/fragua/anaconda/bin/perl}'

Can't locate URI/Escape.pm in #INC on AWS lambda

I have an AWS Lamba function in python that generates a url using a shell script.
The input parameters to the shell script is provided by the python code.
The shell script uses perl to URL encode the required URL.
This script runs fine when I run in locally;however on AWS lamba looks like the perl MURI is not installed. Would appreciate if there is alternative way to do this or any other suggestions to go about it.I am doing URL encode by the following commands. The error I see in lambda logs is:
Can't locate URI/Escape.pm in #INC (#INC contains: /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .).
BEGIN failed--compilation aborted.
Code in the script is
cred=$(perl -MURI::Escape -e 'print uri_escape($ARGV[0]);' "$PARM1/$DATE/$PUBLIC_REGION/$SERVICE_NAME/request")
FULLDT=$(perl -MURI::Escape -e 'print uri_escape($ARGV[0]);' "$FULLDT")
SESSION_TOKEN=$(perl -MURI::Escape -e 'print uri_escape($ARGV[0]);' "$SESSION_TOKEN")
however on AWS lamba looks like the perl MURI is not installed
You're almost right. the -M is a command-line option telling Perl to load a module. The complete module name is "URI::Escape".
And, yes, that module isn't part of a standard Perl installation, so you'll need to install it separately as part of whatever process you use to set up the environment that your lambda runs in. The simplest approach is probably to run:
cpan URI::Escape
But I'm not sure how that would translate to the lambda environment.

Can't locate XML/XPath.pm in #INC

I have a .pl script in which starts by:
#!/usr/bin/perl
use XML::XPath;
use Getopt::Long;
I can't seem to run that via perl myScript.pl, having this error:
(#INC contains: /usr/share/ /usr/local/lib64/perl5 /usr/local/share/perl5/usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .) at most_generic_wrapper.pl line 3.
BEGIN failed--compilation aborted at myScript.pl line 3.
1- I tried to locate the XPath.pm file and export that as:
export PERL5LIB=/usr/share/perl5/XML/Twig
and
export PERL5LIB=/usr/share/perl5/XML
2- Installed perl -MCPAN -e 'install XML::Parser'
3- Used -I to explicitly define the path as:
perl -I perl -MCPAN -e 'install XML::Parser' myScript.pl
4- changing the line 3 to use XML::Twig::XPath; led to:
cannot use XML::Twig::XPath: neither XML::XPathEngine 0.09+ nor XML::XPath are available at /usr/share/perl5/XML/Twig/XPath.pm line 11.
BEGIN failed--compilation aborted at /usr/share/perl5/XML/Twig/XPath.pm line 13.`
But none of them solved the issue and I keep receiving the same error at line.3.
P.S: Running on CentOS 6.2 with the kernel 2.6.32-358 and perl --version=v5.10.1 (*) built for x86_64-linux-thread-multi
Any helps would be appreciated,
Your title says XML::XPath can't be found, but your question indicates you tried to install XML::Parser. Did you try to install XML::XPath?
From man perlrun: "If PERL5LIB is not defined, PERLLIB". You seem to have tried setting PERLIB5 (notice the spelling difference: the var is PERL5LIB (or PERLLIB), not PERLIB5).
From man perlrun: "PERL5LIB -- A list of directories in which to look for Perl library files before looking in the standard library and the current directory." You seem to have tried setting it to the full path to a .pm file, rather than a directory.
The file you assigned would be XML::Twig::XPath, not XML::XPath; those are two different Perl modules.
Edit: After looking at your revised question:
I'm not sure if your script requires XML::Twig::XPath or XML::XPath, or if either one can provide the API you need. However, XML::Twig::XPath seems to depend on XML::XPath so you will need XML::XPath no matter what, and it looks like XML::XPath is not installed on your system. I think that's probably the main problem. Please try to install XML::XPath using CPAN.
The value of the PERL5LIB variable (or the argument to the -I option) should be the directory that sits at the base of the package-qualified module file. For example, if XML::XPath is located at ~/perl_custom_modules/XML/XPath.pm, then you need to set PERL5LIB (or the -I argument) to ~/perl_custom_modules. The XML directory is part of the package qualification of the module, so does not need to be included in the include path.

Can't locate Connection.pm in #INC (#INC contains: /usr/local/lib/perl/5.14.2 /etc/perl /usr/local/share/perl/5.14.2 ....) at ./select.cgi at line

I am trying my hands for the first time in -> Apache - Perl (CGI) on Ubuntu.
The Apache server is working fine in the folder /var/www for the default index.html file. But since I want to run a CGI script, I also installed (just in case) DBI, as suggested by some forums. I set the permissions for complete access to my cgi-bin folder in which the select.cgi script resides. I tried tweaking the select.cgi script multiple times redirecting the library to the Perl library, but to no avail. I modified the httpd.conf file and set the directory path to the select.cgi folder. That didn't work. I also defined ScriptAlias, and set it to the working directory. That didn't work either.
Does anyone have any helpful pointers. Thanks.
This is not exact solution that you want but the overall idea is like below.
How to resolve Cant locate xxxx.pm in #INC path problem
By default perl looks for modules in the standard library path and the current directory.
Sometimes you need to use some modules that are installed in non standard locations; there are several ways to deal with this situation:
To check whether your module is in #INC path use.
Example:
perl -e 'use SOAP::Lite;'
perl -e 'use Error;'
If you run these commands on a system that has SOAP::Lite and Error installed, Perl will simply return from these commands without printing any output.
To check current standard library path use:
perl -le 'print foreach #INC'
If you have the administrative privileges, then best solution is to install the module in any of the system defined library path.
Set the environment variable PERL5LIB
Perl will look for modules in the directories specified in PERL5LIB environment variable before looking in the standard library and current directory.
So you can set this variable to locate your modules.
Example:
# For unix like systems
PERL5LIB=/home/path/lib:/usr/another/path/lib; export PERL5LIB
Note: separate the directories with colons on unix and with a semicolon on Windows.
IF you are running your code from command line then use -I parameter. The syntax should be something like.
perl -I /home/path/lib -I /usr/another/lib script.pl
And you can also Add the library path in your script
The command for including the path in your script is: use lib path.
Example:
#!/usr/bin/perl
use lib "/home/path/lib";
use lib "/usr/another/lib";
use MyCustomModule;
In a comment, you gave information that contradicts what you gave originally. The latter information appears to be more reliable, so I'll use it.
I believe you said the path to the module is
/usr/lib/perl5/vendor_perl/5.8.5/foo/bar/Connection.pm
and that the error message you got (with line breaks added) is:
Can't locate foo/bar/Connection.pm in #INC (#INC contains:
/etc/perl
/usr/local/lib/perl/5.14.2
/usr/local/share/perl/5.14.2
/usr/lib/perl5
/usr/share/perl5
/usr/lib/perl/5.14
/usr/share/perl/5.14
/usr/local/lib/site_perl
.
/usr/lib/perl5/vendor_perl/5.8.5/foo/bar/
) at ...
You did something like
use foo::bar::Connection;
or
require "foo/bar/Connection.pm";
Perl looked for
/etc/perl/foo/bar/Connection.pm
/usr/local/lib/perl/5.14.2/foo/bar/Connection.pm
...
/usr/lib/perl5/vendor_perl/5.8.5/foo/bar/foo/bar/Connection.pm
But none of those are
/usr/lib/perl5/vendor_perl/5.8.5/foo/bar/Connection.pm
It's simple to fix. Add the following to your script:
use lib '/usr/lib/perl5/vendor_perl/5.8.5';
The other possible fix is to use
use Connection;
instead of
use foo::bar::Connection;
Which fix is the correct fix depends on what that package line in side the module looks like. If you find package foo::bar::Connection;, you need to modify #INC (as shown with use lib, for example). If you findpackage Connection;, you need to change theuse` directive.