Is it possible to run Perl script (vas.pl) with shell sript inside (date.sh & backlog.sh) in cron or vice versa?
Thanks.
0 19 * * * /opt/perl/bin/perl /reports/daily/scripts/vas_rpt/vasCIO.pl 2> /reports/daily/scripts/vas_rpt/vasCIO.err
Error encountered:
date.sh: not found
backlog.sh: not found
Perl script:
#!/opt/perl/bin/perl
system("sh date.sh");
open(FH,"/reports/daily/scripts/vas_rpt/date.txt");
#date = <FH>;
close FH;
open(FH,"/reports/daily/scripts/vas_rpt/$cat1.txt");
#array = <FH>;
system("sh backlog.sh $date[0] $array[0]");
close FH;
cron runs your perl script in a different working directory than your current working directory. Use the full path of your script file:
# I'm assuming your shell script reside in the same
# dir as your perl script:
system("sh /reports/daily/scripts/date.sh");
Or if your're allergic to hardcoding paths like I am you can use the FindBin package from CPAN:
use FindBin qw($Bin);
system("sh $Bin/date.sh");
If your shell script also needs to start in the correct path then it's probably better to first change your working directory:
use FindBin qw($Bin);
chdir $Bin;
system("sh date.sh");
You can do what you want as long as you are careful.
The first thing to remember with cron jobs is that you get almost no environment set.
The chances are, the current directory is / or perhaps $HOME. And the value of $PATH is minimal - your profile has not been run, for example.
So, your script didn't find 'date.sh' because it wasn't in the correct directory.
To get the data from the shell script into your program, you need to pipe it there - or arrange for the 'date.sh' to dump the data into the file successfully. Of course, Perl has built-in date and time handling, so you don't need to use the shell for it.
You also did not run with use warnings; or use strict; which would also help you. For example, $cat1 is not a defined variable.
Personally, I run a simple shell script from cron and let it deal with all the complexities; I don't use I/O redirection in the crontab file. That's partly a legacy of working on ancient systems - but it also leads to portable and reliable running of cron jobs.
It's possible. Just keep in mind that your working directory when running under cron may not be what you think it is - it's the value in your HOME environment variable, or that specified in the /etc/passwd file. Consider fully qualifying the path to your .shes.
There are a lot of things that need care in your script, and I talk about most of them in the "Secure Programming Techniques" chapter of Mastering Perl. You can also find some of it in perlsec/
Since you are taking external data and passing them to other external programs, you should use taint checking to ensure that the data are what you expect. What if someone were able to sneak something extra into those files?
When you want to pass data to external programs, use system in the list form so the shell doesn't get a chance to interpret possible meta-characters.
Instead of relying on the PATH to find the programs that you expect to run, specify their full paths explicitly to ensure you are at least running the file you think you are (and not something someone snuck into a directory that is earlier in PATH). If you were really paranoid (like taint checking is), you might also check that those files and directories had suitable permissions (e.g., not world-writeable).
Just as a bonus note, if you only want one line from a filehandle, you can use the line-input operator in scalar context:
my $date = <$fh>;
You probably want to chomp the data too to get rid of possible ending newlines. Even if you don't think a terminating newline should be there because another program created the file, someone looking at the file with a text editor might add it.
Good luck, :)
Related
I am making a few calls to the system, mainly cd commands, as certain functions need to called from certain directories on my system. However, I have noticed that once a call is finished, the effects of that call are lost.
For example, lets say that I start in /home/project and then call:
system("setenv home/project/env/NeededEnvironment");
system("make cfile.o");
The second system call doesn’t know about the first call setting the environment needed for the file to compile. I have tried putting them into one system call separated by ; as well, but have the same problem. Is there anyway to get the effect of the first call to be saved?
That is how system works: it creates a subshell to execute your command, and when the command is complete, the subshell exits leaving your perl process unaffected.
Section 8 of the Perl FAQ also answers this question.
I {changed directory, modified my environment} in a perl script. How come the change disappeared when I exited the script? How do I get my changes to be visible?
Unix
In the strictest sense, it can't be done—the script executes as a different process from the shell it was started from. Changes to a process are not reflected in its parent—only in any children created after the change. There is shell magic that may allow you to fake it by eval()ing the script's output in your shell; check out the comp.unix.questions FAQ for details.
You want code along the lines of
system("cd /home/project/env/NeededEnvironment && make cfile.o") == 0
or warn "$0: make failed";
or use the -C option to make and avoid shell argument parsing as in
system("make", "-C", "/home/project/env/NeededEnvironment", "cfile.o") == 0
or warn "$0: make failed";
If you are writing a Perl script, use Perl itself and shell-out as rarely as possible.
If you need to change your directory:
chdir 'some/other/dir';
If you need to set an environment variable:
$ENV{ SOME_VAR } = 'Some value';
Update
Here are some more commands where the shell equivalent should not be used:
mkdir
unlink
rmdir
Modules everyone should know about:
File::Copy
File::Path
File::Basename
File::Spec
I googled about #!/usr/bin/perl, but I could not find any satisfactory answer. I know it’s a pretty basic thing, but still, could explain me what is the significance of #!/usr/bin/perl in Perl? Moreover, what does -w or -T signify in #!/usr/bin/perl? I am a newbie to Perl, so please be patient.
The #! is commonly called a "shebang" and it tells the computer how to run a script. You'll also see lots of shell-scripts with #!/bin/sh or #!/bin/bash.
So, /usr/bin/perl is your Perl interpreter and it is run and given the file to execute.
The rest of the line are options for Perl. The "-T" is tainting (it means input is marked as "not trusted" until you check it's format). The "-w" turns warnings on.
You can find out more by running perldoc perlrun (perldoc is Perl's documentation reader, might be installed, might be in its own package).
For scripts you write I would recommend starting them with:
#!/usr/bin/perl
use warnings;
use strict;
This turns on lots of warnings and extra checks - especially useful while you are learning (I'm still learning and I've been using Perl for more than 10 years now).
Both -w and -T are sort of "foolproof" flags.
-w is the same as use warning statement in your code, and it's an equivalent of warning option in many compilers. A simplest example would be a warning about using uninitialized variable:
#!/usr/bin/perl -w
print "$A\n";
print "Hello, world!\n";
Will print:
Name "main::A" used only once: possible typo at ./perl-warnings line 3.
Use of uninitialized value $A in concatenation (.) or string at
./perl-warnings line 3.
Hello, world!
The -T flag means that any value that came from the outside world (as opposite to being calculated inside the program) is considered potential threat, and disallows usage of such values in system-related operations, like writing files, executing system command, etc. (That's why Perl would activate the "taint" mode when the script is running under setuid/setgid.)
The "tainted" mode is "enforcing" you to double-check the value inside the script.
E.g., the code:
#!/usr/bin/perl -T
$A = shift;
open FILE, ">$A";
print "$A\n";
close FILE;
Will produce a fatal error (terminating the program):
$ ./perl-tainted jkjk
Insecure dependency in open while running with -T switch at
./perl-tainted line 3.
And that's only because the argument value came from "outside" and was not "double-checked". The "taint" mode is drawing your attention to that fact. Of course, it's easy to fool it, e.g.:
#!/usr/bin/perl -T
$A = shift;
$A = $1 if $A =~ /(^.*$)/;
open FILE, ">$A";
print "$A\n";
close FILE;
In this case everything worked fine. You "fooled" the "taint mode". Well, the assumption is that programer's intentions are to make the program safer, so the programmer wouldn't just work around the error, but would rather take some security measures. One of Perl's nicknames is "the glue and the duct tape of system administrators". It's not unlikely that system administrator would create Perl script for his own needs and would run it with root permissions. Think of this script doing something normal users are not allowed to do... you probably want to double-check things which are not part of the program itself, and you want Perl to remind you about them.
Hope it helps.
about Taint Mode(-T):
require and use statements change when taint mode is turned on.
The path to load libraries/modules no longer contains . (the current directory) from its path.
So if you load any libraries or modules relative to the current working directory without explicitly specifying the path, your script will break under taint mode.
For ex: Consider perl_taint_ex.pl
#!/usr/bin/perl -T
require "abc.pl";
print "Done";
would fail like this
D:\perlex>perl perl_taint_ex.pl
"-T" is on the #! line, it must also be used on the command line
at perl_taint_ex.pl line 1.
D:\perlex>perl -T perl_taint_ex.pl
Can't locate abc.pl in #INC (#INC contains: C:/Perl/site/lib C:/Perl/lib)
at perl_taint_ex.pl line 3.
So when taint mode is on, you must tell the require statement explicitly where to load the library since . is removed during taint mode from the #INC array.
#INC contains a list of valid paths to read library files and modules from.
If taint mode is on, you would simply do the following:
D:\perlex>perl -ID:\perlex -T perl_taint_ex.pl
Done
-ID:\perlex will include directory D:\perlex in #INC.
You can try other ways for adding path to #INC,this is just one example.
It's called a shebang. On Unix based systems (OSX, Linux, etc...) that line indicates the path to the language interpreter when the script is run from the command line. In the case of perl /usr/bin/perl is the path to the perl interpreter. If the hashbang is left out the *nix systems won't know how to parse the script when invoked as an executable. It will instead try to interpret the script in whatever shell the user happens to be running (probably bash) and break the script.
http://en.wikipedia.org/wiki/Hashbang
The -W and -T are arguments that controll the way the perl interpreter operates. They are the same arguments that you could invoke when calling perl interpreter directly from the command line.
-W shows warnings (aka debuging information).
-T turns on taint / security checking.
I have a folder full of script files. When I run them, a program to which they are native is opened, it does some stuff and a CSV file is generated. I wrote some code that I want to run each script file and produce a bunch of CSV files, one for each script.
What happens is the following: when my Perl application is executed, the software is launched and the first of the scripts is run successfully (a CSV file is created). However, at this step the Perl application waits for me to close the software before I continue. It does this for every script. What can I do to avoid this from happening?
use strict;
use warnings;
use Cwd;
my $dir = cwd();
opendir(DIR, $dir);
my #files= grep(/\.acs$/,readdir(DIR));
$dir=~s/\//\\/g;
chdir $dir;
foreach (#files)
{
print "$_\n";
system ("$_")
}
I think you'll want to fork and exec and waitpid - i.e. set up and run your own process and wait for it to finish on your own.
http://larc.ee.nthu.edu.tw/~cthuang/courses/ee2320/12_process.html
This isn't easy, but unfortunately, you're doing this on an OS that isn't script-friendly. Doing this under Linux or OS X you wouldn't have any troubles like this.
You'll need to figure out if these commands are even available under Windows. You may have to find some similar things that are available under windows if there is no posix compatibility library.
your best choice is to ask the application nicely to close itself after it is done.
for example, the cmd.exe command have parameter /C that does exactly that.
try to run the application with /?, and see if anything useful comes up.
failing that, you can use Win32::Process to create the process and then kill it after you are sure it is done. see the documentation for that.
Is there a way to load a profile inside Perl?
Like in shell files:
. ~user/.profile
What specifically do you mean by "profile"?
If you mean "retrieve the values of the shell environmental variables that your shell set via .profile", then yes - you do it through a special %ENV hash.
If you mean "read the actual variables set in .profile" like the shell itself does, it's possible but doing it "right" requires either parsing an arbitrary shell script and scrubbing anything that's not an environmental variable assignment, OR executing ". ~/.profile; env`"` and parsing the output.
If you mean "supply a generic configuration to any Perl program that runs via a separate configuration file", you need to add code to those Perl programs to read this configuration file (there are a number of CPAN modules for reading various config files).
If you mean "supply a generic configuration to any Perl program that runs without any special code in those Perl programs to read a separate configuration file, sort of like any shell script gets the stuff from .profile thanks to shell", then the asnwer is "may be". You can leverage PERLOPT environmental variable to supply options which would load up a special module (via -I) containing the configuration that gets set via its "import()". While somewhat doable, it seems like an awful hack that I would strongly recommend against using.
Env::Sourced should do what you need.
use Env::Sourced qw(~/user/profile);
print $ENV{VARAIBLE};
If you have bash configuration values in (or other configuration set up by) a shell script and want that to take effect only for the duration of one execution of a program, you can use a subshell:
( source ~/my_bash_file.sh; perl my_perl_script.pl )
You can access shell environment variables in Perl using the %ENV hash (see the index of Perl's special variables, perldoc perlvar):
my $user = $ENV{USER};
my $home_dir = $ENV{HOME};
Example:
my_bash_file.sh:
#!/bin/bash
export HOME="/home/nowhere"
my_perl_script.pl:
#!/usr/bin/perl
print "the value of HOME is $ENV{HOME}\n";
When executed as perl my_perl_script.pl, my_perl_script.pl prints:
the value of HOME is /home/ether
When executed as ( source ~/my_bash_file.sh; perl my_perl_script.pl ), the output is:
the value of HOME is /home/nowhere
$u=`echo -n ~user`;
open F, "<$u/.profile" || die;
while(<F>)
{print}
Do you think changing directories inside bash or Perl scripts is acceptable? Or should one avoid doing this at all costs?
What is the best practice for this issue?
Like Hugo said, you can't effect your parent process's cwd so there's no problem.
Where the question is more applicable is if you don't control the whole process, like in a subroutine or module. In those cases you want to exit the subroutine in the same directory as you entered, otherwise subtle action-at-a-distance creeps in which causes bugs.
You can to this by hand...
use Cwd;
sub foo {
my $orig_cwd = cwd;
chdir "some/dir";
...do some work...
chdir $orig_cwd;
}
but that has problems. If the subroutine returns early or dies (and the exception is trapped) your code will still be in some/dir. Also, the chdirs might fail and you have to remember to check each use. Bleh.
Fortunately, there's a couple modules to make this easier. File::pushd is one, but I prefer File::chdir.
use File::chdir;
sub foo {
local $CWD = 'some/dir';
...do some work...
}
File::chdir makes changing directories into assigning to $CWD. And you can localize $CWD so it will reset at the end of your scope, no matter what. It also automatically checks if the chdir succeeds and throws an exception otherwise. Sometimes it use it in scripts because it's just so convenient.
The current working directory is local to the executing shell, so you can't affect the user unless he is "dotting" (running it in the current shell, as opposed to running it normally creating a new shell process) your script.
A very good way of doing this is to use subshells, which i often do in aliases.
alias build-product1='(cd $working-copy/delivery; mvn package;)'
The paranthesis will make sure that the command is executed from a sub-shell, and thus will not affect the working directory of my shell. Also it will not affect the last-working-directory, so cd -; works as expected.
I don't do this often, but sometimes it can save quite a bit of headache. Just be sure that if you change directories, you always change back to the directory you started from. Otherwise, changing code paths could leave the application somewhere it should not be.
For Perl, you have the File::pushd module from CPAN which makes locally changing the working directory quite elegant. Quoting the synopsis:
use File::pushd;
chdir $ENV{HOME};
# change directory again for a limited scope
{
my $dir = pushd( '/tmp' );
# working directory changed to /tmp
}
# working directory has reverted to $ENV{HOME}
# tempd() is equivalent to pushd( File::Temp::tempdir )
{
my $dir = tempd();
}
# object stringifies naturally as an absolute path
{
my $dir = pushd( '/tmp' );
my $filename = File::Spec->catfile( $dir, "somefile.txt" );
# gives /tmp/somefile.txt
}
I'll second Schwern's and Hugo's comments above. Note Schwern's caution about returning to the original directory in the event of an unexpected exit. He provided appropriate Perl code to handle that. I'll point out the shell (Bash, Korn, Bourne) trap command.
trap "cd $saved_dir" 0
will return to saved_dir on subshell exit (if you're .'ing the file).
mike
Consider also that Unix and Windows have a built in directory stack: pushd and popd. It’s extremely easy to use.
Is it at all feasible to try and use fully-quantified paths, and not make any assumptions on which directory you're currently in? e.g.
use FileHandle;
use FindBin qw($Bin);
# ...
my $file = new FileHandle("< $Bin/somefile");
rather than
use FileHandle;
# ...
my $file = new FileHandle("< somefile");
This will probably be easier in the long run, as you don't have to worry about weird things happening (your script dying or being killed before it could put the current working directory back to where it was), and is quite possibly more portable.