I have a perl script written for version 5.6.1 and which has dependencies on Oracle packages, DBI packages and more stuff. Everything is correctly installed and works.
Recently perl version 5.8.4 got installed and because those dependencies are not correctly set so the script fails.
'perl' command points to /program/perl_v5.8.4/bin/perl -- latest version
So, when I have to run my perl script I have to manually specify in command prompt
/program/perl_v5.6.1/bin/perl scriptName.pl
I tried adding following lines in script:
/program/perl_v5.6.1/bin/perl
use v5.6.1;
But, this means script has to take Perl version > 5.6.1
I found couple of related question which suggested:
To export path. But, I want the script for all the users without have to export path
Specify version greater than. But, I want to use just one specific version for this script.
use/require commands. Same issue as above.
My question: How to specify in the script to only use a specific version of perl?
The problem is that the interpreter specified in the shebang line (#!/some/path/to/perl) is not used if perl script is called like this:
perl some_script.pl
... as the 'default' (to simplify) perl is chosen then. One should use the raw power of shebang instead, by executing the file itself:
./some_script.pl
Of course, that means this file should be made executable (with chmod a+x, for example).
I have in my code:
our $LEVEL = '5.10.1';
our $BRACKETLEVEL = sprintf "%d.%03d%03d", split/\./, $LEVEL;
if ($] != $currentperl::BRACKETLEVEL)
{
die sprintf "Must use perl %s, this is %vd!\n", $LEVEL, $^V;
}
These are actually two different modules, but that's the basic idea. I simply "use correctlevel" at the top of my script instead of use 5.10.1; and I get this die if a developer tries using the wrong level of perl for that product. It does not, however, do anything else that use 5.10.1; would do (enable strict, enable features like say, switch, etc.).
Related
Is there any way to report the missing modules used in the Perl file beforehand instead of getting as an error.
I have something like use Digest::MD5, use File::DosGlob modules in my Perl program. Whenever the users run the script they are getting an error if there is no specific module installed in their system. They could not understand the default error message given by #INC. So I would like to clearly tell them that these modules need to be installed to run the script.
You could build your own verification by using a BEGIN block. Those are run at compile time, just like use is. Keep in mind that use Foo is essentially nothing else as this:
BEGIN {
require Foo;
Foo->import;
}
The following code will replace all use statements with a single BEGIN and place them inside an eval. That's essentially like a try/catch mechanism.
We need the string eval (which is considered evil around here) because require only converts from package names with colons :: to paths if the argument is a bareword. But because we have the name in $module, it's a string, so we need to place it into an eval according to require's docs.
If that string eval fails, we die. That's caught by the outer eval block and $# is set. We can then check if it contains our module name, in which case we naively assume the failure was because that module is not installed. This check could be a bit more elaborate.
We keep track of any failures in $fails, and if there were any, we stop.
#!/usr/bin/perl
use strict;
use warnings;
# all our use statements go here
BEGIN {
my $fails;
foreach my $module ( qw/Digest::MD5 File::DosGlob ASDF/ ) {
eval {
eval "require $module" or die; # because $module is not a bareword
$module->import;
};
if ($# && $# =~ /$module/) {
warn "You need to install the $module module";
$fails++;
}
}
exit if $fails;
}
# ...
Above I included ASDF, which I don't have, so when run it will say
You need to install the ASDF module at /home/code/scratch.pl line 1335.
You might want to make that message a bit more verbose. If your users are not able to understand the default error message that Perl gives when it cannot find a module, it might be wise to include a guide on how to install stuff right there.
Note that both modules you listed have been included with Perl for a while (read: since March 2002). So why would you want to do this for those modules?
$ corelist Digest::MD5
Data for 2014-09-14
Digest::MD5 was first released with perl v5.7.3
$ corelist File::DosGlob
Data for 2014-09-14
File::DosGlob was first released with perl 5.00405
A better way would be ship your program as a distribution that can be installed, and include a Makefile or a cpanfile or something similar that lists dependencies. There is a guide in perlnewmod on how to start a new module. You'd not want to upload to CPAN obviously, but the basics are the same.
With this, your users would get all dependencies installed automatically.
You could use Devel::Modlist, it will list all the required module for your program.
perl -d:Modlist test.pl
There's another module Module::ScanDeps which comes with a utility scandeps.pl which you can use on your script as:
scandeps.pl test.pl
Note that sanity checking your Perl code using perl -c is dangerous, so use it carefully.
Your question isn't really clear about what "beforehand" means. To check if a Perl program's syntax is correct and directly included modules are resolvable, use
perl -c <perl-program.pl>
This checks the syntax of your file and ensures that any modules used by your code exist. However, it does not transitively check the entire dependency tree, only those mentioned in perl-program.pl.
I was looking through my CPAN distributions and realized that I had various inconsistent things at the top of my .t scripts, based on where I'd cargo-culted them from. This of course offends me.
So, what's the "best" first line of a Perl test (.t) script? A non-scientific survey of my .cpanm sources showed me:
3429 use strict;
3211 #!/usr/bin/perl
1344 #!/usr/bin/env perl
937 #!perl
909 #!/usr/bin/perl -w
801 #!perl -w
596
539 #!perl -T
Related to What should I use for a Perl script's shebang line?, but here I'm wondering if the shebang is necessary/useful at all, if tests are always expected to be called from prove.
use strict; should be in EVERY Perl program (as well as use warnings; and probably a few others (depending who you talk to). It doesn't have to be the first line.
The rest of the stuff you present are merely multiple versions of the shebang. On Unix and Unix based systems, the shebang is used to tell the interpreter to use. If the shebang isn't there, the current shell will be used. (That is, if your shell supports the shebang1.).
If you are on a Windows system, that shebang line is useless. Windows uses the suffix of the file to figure out how to execute the file.
The question is whether you really even need a shebang line in scripts that you run through a test harness. I don't normally execute these test scripts individually, so there may not really be a need for the shebang at all. In the rare event you do want to execute one of these scripts, you could always run them as an argument to the perl command itself. The same is true with Perl modules.
So, why the shebang? Mainly habit. I put one too on my test scripts and on my modules. If nothing else, it makes it easy to run modules and test scripts independently for testing purposes. Plus, the shebang lets people know this is a Perl script and not a shell script or a Python script.
There's a little issue with the shebang though, and that has to do with portability. If I put #! /usr/bin/perl on my first line, and the Perl interpreter is in #! /usr/local/bin/perl, I'll get an error. Thus, many of the shebangs reflect the location of the Perl interpreter for that developer.
There's also another issue: I use Perlbrew which allows me to install multiple versions of Perl. The version of Perl I currently have in my shell is at /Users/David/perl5/perlbrew/perls/perl-5.18.0/bin/perl. That's not good if I pass that program to another user on another system. Or, if I decide I want to test my Perl script under 5.10.
The #! perl variant is an attempt to solve this. On SunOS (I believe), this would execute the Perl that's in your $PATH (since it could be /usr/local/bin/perl or /usr/share/bin/perl or /usr/bin/perl depending upon the system). However, this does not work on most Unix boxes. For example, on my Mac, I would get a bad interpreter error.
I use #! /usr/bin/env perl. The env program takes the argument perl, sees where perl is on my $PATH, and then executes the full path of wherever Perl happens to be on my $PATH. This is the most universal way to do the shebang and the way I recommend. It allows those of use to use Perlbrew without problem.
Note, I said almost universal, and not universal. On almost all systems, env is located in /usr/bin, but there apparently some systems out there where env is either on /bin, located elsewhere, or not on the system at all.
The -w turns on warnings when executing Perl. Before Perl 5.6, you used this to turn on warnings. After Perl 5.6, you can use the more flexible use warnings; and the -w is now considered deprecated2. Programs may use it because they were originally written pre 5.6, and never modified or the developer simply did it out of habit.
The -T has to do with taint mode. Taint mode is normally used for CGI scripting. Basically, in Taint mode, data from outside a program is considered tainted and cannot be used until untainted. Taint mode also affects #INC and PERLLIB.
The real answer is that there is no set answer. I'd put #! /usr/bin/env perl as my first line out of habit -- even if I never expect to execute that file from a Unix command line. There's no real need for it for these types of scripts that are almost always executed as part of the install and not directly from the command line. What you see is really the result of 30 years of Perl habits and cruft.
1. Your shell supports the shebang unless you're using a 30 year old version of the Bourne shell or a very old version of the C shell.
2. I actually don't know if -w has ever been officially deprecated, but there's no reason to use it.
.t files are still regular Perl script. Use whatever you need for this particular script on first line.
I cannot use %ENV var on my Perl script to use Oracle libs.
BEGIN {
$ORACLE_HOME = "/usr/lib/oracle/10.2.0.3/client64";
$LD_LIBRARY_PATH = "$ORACLE_HOME/lib";
$ORACLE_SID="prod";
$ENV{ORACLE_SID}=$ORACLE_SID;
$ENV{ORACLE_HOME}= $ORACLE_HOME;
$ENV{LD_LIBRARY_PATH}= $LD_LIBRARY_PATH;
};
If I print $ENV{'ORACLE_HOME'} and $ENV{'LD_LIBRARY_PATH'} all seems ok but, when I run my script I have the error:
install_driver(Oracle) failed: Can't load '/usr/local/lib64/perl5/auto/DBD/Oracle/Oracle.so' for module DBD::Oracle: libclntsh.so.10.1: cannot open shared object file: No such file or directory at /usr/lib64/perl5/DynaLoader.pm line 200.
at (eval 3) line 3
Compilation failed in require at (eval 3) line 3.
Perhaps a required shared library or dll isn't installed where expected
at persistence.perl line 22
Searching on web I saw that the correct way to set env vars on Perl is to use %ENV hash.
Exporting ORACLE_HOME and LD_LIBRARY_PATH through unix shell (export LD_LIBRARY_PATH=...) it works correctly. Any advice?
The LD_LIBRARY_PATH environment variable has to be set before your program starts — before perl itself is loaded. Changing it in BEGIN{} will affect new programs that you start, but it won't affect the loading of shared libraries — in this case (although I've never used the DBD::Oracle) you're loading an Oracle .so into the already-running program, so it's “too late” to change the LD_LIBRARY_PATH. The dynamic linker /lib/ld.so (or so) is started before perl, so by the time your script is compiled and BEGIN{} runs, it's already set up.
You could try to re-exec your script as its own successor or something*, but a short shell script is almost certainly going to be the simplest solution:
#!/bin/sh
export LD_LIBRARY_PATH=/usr/lib/oracle/10.2.0.3/client64/lib
export ORACLE_SID=prod
exec /usr/local/bin/your-db-program "$#"
*- this would be kinda crazy, but TIMTOWTDI:
eval {
use DBD::Oracle foo bar baz; …
};
if ($# =~ /install_driver\(Oracle\) failed/) {
$ENV{LD_LIBRARY_PATH} .= ':/usr/lib/oracle/10.2.0.3/client64/lib';
$ENV{ORACLE_SID} = 'prod';
warn "Restarting with LD_LIBRARY_PATH reset:\n\n$#\n";
exec { $0 } $0 => #ARGV;
}
I wrote a few test scripts to verify that the environment is being set when you change %ENV:
use strict;
use warnings;
use feature qw(say);
BEGIN {
my $foo = "bar-bar";
$ENV{FOO} = "$foo";
}
system qq(/bin/echo printing out \$FOO);
This prints out:
printing out bar-bar
which is what I expected.
I then tried this:
use strict;
use warnings;
use feature qw(say);
BEGIN {
my $foo = "bar-bar";
$ENV{FOO} = "$foo";
}
system qq(./test.sh);
and created a test.sh program that looks like this:
#! /bin/sh
echo This is what I got: $FOO;
In this case, my Perl script is running test.sh which prints out the value of the $FOO environment variable that was set in my Perl script. Running test.pl I get:
This is what I got bar-bar
This shows that not only is Perl setting the environment variables, but that it is also exporting those variables, so called shell scripts have access to them.
You can try a similar technique to verify that both LD_LIBRARY_PATH and ORACLE_HOME are being set before they're used. I suspect you'll find that this is indeed happening, but that your program still isn't working when you set %ENV.
This points to one conclusion: Setting the environment for LD_LIBRARY_PATH and ORACLE_HOME might be occurring too late by the time your Perl script starts. I believe the operating system examines LD_LIBRARY_PATH before Perl starts. I found this doing a search on LD_LIBRARY_PATH:
LD_LIBRARY_PATH is an environment variable you set to give the run-time shared library loader (ld.so) an extra set of directories to look for when searching for shared libraries. Multiple directories can be listed, separated with a colon (:). This list is prepended to the existing list of compiled-in loader paths for a given executable, and any system default loader paths.
So, LD_LIBRARY_PATH is for the ld.so runtime shared library loader, If ld.so has already been loaded, changing LD_LIBRARY_PATH won't do anything.
I found a similar discussion on Perl Monks. I noticed someone found rerunning env seemed to work.
One solution is to modify /etc/ld.so.conf
On CentOS/RHEL 6.4, you could create etc/ld.so.conf.d/oracle with this:
/oracle/sw/product/11.2.0/dbhome_1/lib
Obviously, modify as suits your ORACLE_HOME.
Then run
ldconfig -v
You could put the export commands into the start up script for your unix shell which you should have permission to edit. That way, the environment variables will be set whenever you start a new shell and all scripts and programs that use Oracle will pick them up.
I just went through something similar. I had to make sure that the Oracle environment was setup before anything else called it. Make sure the BEGIN block is before any other "use" statements. In my case, something was being called in Apache's httpd.conf file, so I had to setup my environment there instead of in my package.
I have perl scripts starting with #!/usr/bin/perl or #!/usr/bin/env perl
First, what does the second version mean?
Second, I use Ubuntu. All the scripts are set as executables. When I try to run a script by simply invoking it's name (e.g. ./script.pl) I get : No such file or directory. when I invoke by perl ./script.pl it runs fine.
Why?
The #!/usr/bin/env perl uses the standard POSIX tool env to work around the "problem" that UNIX doesn't support relative paths in shebang lines (AFAIK). The env tool can be used to start a program (in this case perl) after modifying environment variables. In this case, no variables are modified and env then searches the PATH for Perl and runs it. Thus a script with that particular shebang line will work even when Perl is not installed in /usr/bin but in some other path (which must be in the PATH variable).
Then, you problem with ./script.pl not working: you said it has the executable bit(s) set, like with chmod +x script.pl ? But does it also start with a shebang (#!) line ? That is, the very first two bytes must be #! and it must be followed by a file path (to perl). That is necessary to tell the kernel with which program to run this script. If you have done so, is the path correct ? You want to try the #!/usr/bin/env perl variant ;-)
Using #!/usr/bin/env perl gets around the problem of perl not necessarily being in /usr/bin on every system; it's just there to make the script more portable
On a related note, for your second problem, is there a /usr/bin/perl and/or /usr/bin/env? If not, that would explain why running the scripts directly doesn't work; the shebang isn't handled if you run the script as an argument to perl
Is it possible to run Perl script (vas.pl) with shell sript inside (date.sh & backlog.sh) in cron or vice versa?
Thanks.
0 19 * * * /opt/perl/bin/perl /reports/daily/scripts/vas_rpt/vasCIO.pl 2> /reports/daily/scripts/vas_rpt/vasCIO.err
Error encountered:
date.sh: not found
backlog.sh: not found
Perl script:
#!/opt/perl/bin/perl
system("sh date.sh");
open(FH,"/reports/daily/scripts/vas_rpt/date.txt");
#date = <FH>;
close FH;
open(FH,"/reports/daily/scripts/vas_rpt/$cat1.txt");
#array = <FH>;
system("sh backlog.sh $date[0] $array[0]");
close FH;
cron runs your perl script in a different working directory than your current working directory. Use the full path of your script file:
# I'm assuming your shell script reside in the same
# dir as your perl script:
system("sh /reports/daily/scripts/date.sh");
Or if your're allergic to hardcoding paths like I am you can use the FindBin package from CPAN:
use FindBin qw($Bin);
system("sh $Bin/date.sh");
If your shell script also needs to start in the correct path then it's probably better to first change your working directory:
use FindBin qw($Bin);
chdir $Bin;
system("sh date.sh");
You can do what you want as long as you are careful.
The first thing to remember with cron jobs is that you get almost no environment set.
The chances are, the current directory is / or perhaps $HOME. And the value of $PATH is minimal - your profile has not been run, for example.
So, your script didn't find 'date.sh' because it wasn't in the correct directory.
To get the data from the shell script into your program, you need to pipe it there - or arrange for the 'date.sh' to dump the data into the file successfully. Of course, Perl has built-in date and time handling, so you don't need to use the shell for it.
You also did not run with use warnings; or use strict; which would also help you. For example, $cat1 is not a defined variable.
Personally, I run a simple shell script from cron and let it deal with all the complexities; I don't use I/O redirection in the crontab file. That's partly a legacy of working on ancient systems - but it also leads to portable and reliable running of cron jobs.
It's possible. Just keep in mind that your working directory when running under cron may not be what you think it is - it's the value in your HOME environment variable, or that specified in the /etc/passwd file. Consider fully qualifying the path to your .shes.
There are a lot of things that need care in your script, and I talk about most of them in the "Secure Programming Techniques" chapter of Mastering Perl. You can also find some of it in perlsec/
Since you are taking external data and passing them to other external programs, you should use taint checking to ensure that the data are what you expect. What if someone were able to sneak something extra into those files?
When you want to pass data to external programs, use system in the list form so the shell doesn't get a chance to interpret possible meta-characters.
Instead of relying on the PATH to find the programs that you expect to run, specify their full paths explicitly to ensure you are at least running the file you think you are (and not something someone snuck into a directory that is earlier in PATH). If you were really paranoid (like taint checking is), you might also check that those files and directories had suitable permissions (e.g., not world-writeable).
Just as a bonus note, if you only want one line from a filehandle, you can use the line-input operator in scalar context:
my $date = <$fh>;
You probably want to chomp the data too to get rid of possible ending newlines. Even if you don't think a terminating newline should be there because another program created the file, someone looking at the file with a text editor might add it.
Good luck, :)