I have a program with lots of system commands to handle searching for, examining, and killing processes:
system qq(kill $pid);
and
for my $pid ( qx( pgrep -f "$pgrep_re") ) {
chomp $pid;
...
}
and
my $command_line = qx(ps -o command="" $pid);
chomp $command_line;
....
Not only is this system specific, but I'm depending upon the user to have these particular commands in their path the correct way, leaving me with a system security issue (like some joker setting alias ps="rm -rf *").
I would like to do this stuff in a nice, Perl way which would be less dependent upon all of these system commands and be a bit more platform independent1.
Is there a Perl module? Bonus points for one that does it in a nice object-oriented way and doesn't depend externally with these very same commands.
1. A lot of this deals with using ssh and setting up tunnels, so since Windows doesn't have ssh as a native command, I'm willing to exclude it as long as this works well for other Unix/Linux systems.
kill: use the builtin kill (perldoc -f kill)
ps: use search.cpan.org, there is UNIX::Process. In linux you could also scan through /proc/
pgrep: combine ps with perl pattern matching
Related
I have a question about this answer, quoted below, by friedo to another question here. (I don't have permission to comment on it, so I am asking this as a question.)
"You can use File::Tee.
use File::Tee qw(tee);
tee STDOUT, '>>', 'some_file.out';
print "w00p w00p";
If File::Tee is unavailable, it is easily simulated with a pipeline:
open my $tee, "|-", "tee some_file.out";
print $tee "w00p w00p";
close $tee;
Are both of these tees the same? Or is one from Perl and the other from Linux/Unix?
They're mostly the same, but the implementation details differ.
Opening a pipe to tee some_file.out forks a new process and runs the Unix / Linux utility program tee(1) in it. This program reads its standard input (i.e. anything you write to the pipe) and writes it both to some_file.out as well as to stdout (which it inherits from your program).
Obviously, this will not work under Windows, or on any other system that doesn't provide a Unix-style tee command.
The File::Tee module, on the other hand, is implemented in pure Perl, and doesn't depend on any external programs. However, according to its documentation, it also works by forking a new process and running what is essentially a Perl reimplementation of the Unix tee command under it. This does have some advantages, as the documentation states:
"It is implemeted around fork, creating a new process for every tee'ed stream. That way, there are no problems handling the output generated by external programs run with system or by XS modules that don't go through perlio."
On the other hand, the use of fork has its down sides as well:
"BUGS
Does not work on Windows (patches welcome)."
If you do want a pure Perl implementation of the tee functionality that works on all platforms, consider using IO::Tee instead. Unlike File::Tee, this module is implemented using PerlIO and does not use fork.
Alas, this also means that it may not correctly capture the output of external programs executed with system or XS modules that bypass PerlIO.
I am writing a code in perl with embedded shell script in it:
#!/usr/bin/perl
use strict;
use warnings;
our sub main {
my $n;
my $n2=0;
$n=chdir("/home/directory/");
if($n){
print "change directory successful $n \n";
$n2 = system("cd", "section");
system("ls");
print "$n2 \n";
}
else {
print "no success $n \n";
}
print "$n\n";
}
main();
But it doesn't work. When I do the ls. The ls doesn't show new files. Anyone knows another way of doing it. I know I can use chdir(), but that is not the only problem, as I have other commands which I have created, which are simply shell commands put together. So does anyone know how to exactly use cli in perl, so that my compiler will keep the shell script attached to the same process rather than making a new process for each system ... I really don't know what to do.
The edits have been used to improve the question. Please don't mind the edits if the question is clear.
edits: good point made by mob that the system is a single process so it dies everytime. But, What I am trying to do is create a perl script which follows an algorithm which decides the flow of control of the shell script. So how do I make all these shell commands to follow the same process?
system spawns a new process, and any changes made to the environment of the new process are lost when the process exits. So calling system("cd foo") will change the directory to foo inside of a very short-lived process, but won't have any effect on the current process or any future subprocesses.
To do what you want to do (*), combine your commands into a single system call.
$n2 = system("cd section; ls");
You can use Perl's rich quoting features to pass longer sequences of commands, if necessary.
$n2 = system q{cd section
if ls foo ; then
echo we found foo in section
./process foo
else
echo we did not find foo\!
./create_foo > foo
fi};
$n2 = system << "EOSH";
cd section
./process bar > /tmp/log
cd ../sekshun
./process foo >> /tmp/log
if grep -q Error /tmp/log ; then
echo there were errors ...
fi
EOSH
(*) of course there are easy ways to do this entirely in Perl, but let's assume that the OP eventually will need some function only available in an external program
system("cd", "section"); attempts to execute the program cd, but there is no such program on your machine.
There is no such program because each process has its own current work directory, and one process cannot change another process's current work directory. (Programs would malfunction left and right if it was possible.)
It looks like you are attempting to have a Perl program execute a shell script. That requires recreating the shell in Perl. (More specific goals might have simpler solutions.)
What I am trying to do is create a perl script which follows an algorithm which decides the flow of control of the shell script.
Minimal change:
Create a shell script that prompts for instructions/commands. Have your Perl script launch the shell script using Expect and feed it answers/commands.
I am trying to write a Perl script to bring up a Postgres client once and send several files through the client, captures the output separately for each file.
If I do something like:
system ("cat $query1.sql | psql -p 2070 super &> $HOME/Results1.txt");
system ("cat $query2.sql | psql -p 2070 super &> $HOME/Results2.txt");
then Perl will start up the client for each query. As I'll be running hundreds and maybe thousands of queries I want to skip the overhead for starting up all but the first client.
I think I should be able to bring up the Postgres client via Open2, but it hangs when I try. I'm doing this on SUSE Linux machine with Perl 5.10.0.
Here's my code:
use IPC::Open2;
use IO::Handle;
our $pid = open2(*CHILDOUT, *CHILDINT, '../installdir/bin/psql -p 2070 super');
print STDOUT $pid;
print CHILDINT "cat $dumpR5KScript";
print STDOUT 'Sent the commands';
$output = <CHILDOUT>;
close(CHILDIN);
close(CHILDOUT);
It appears to be hanging with "open2" because I never see the pid.
Can someone point out what I am doing wrong so my call to open2 doesn't hang?
And if anyone has advice on the larger issue of the best way of bringing up a Postgres client and running queries through it I would be grateful.
You have already been told to use DBI in the comments to your post, and it will be a good thing if you do. Formatting is much easier than fiddling with the IPC and stitching together a sort of an API between Perl and a command line database client, parsing the output and formatting the input.
Regarding your problem, however:
it should be \*CHILDIN instead of *CHILDIN (reference to typeglob rather than typeglob)
and anyways in such a case you should use variables instead of typeglobs and ancient idioms:
my ( $childout, $childin ) ;
our $pid = open2( $childout, $childin, '../installdir/bin/psql -p 2070 super');
print STDOUT $pid;
Please read the documentation for IPC::Open2.
Also, better to use open3 to handle STDERR as well
Finally, I do not know the postgress client, but the possibility of a deadlock (which you are experiencing) is very real with open2:
This whole affair is quite dangerous, as you may block forever. It assumes it's going to talk to something like bc, both writing to it and reading from it. This is presumably safe because you "know" that commands like bc will read a line at a time and output a line at a time. Programs like sort that read their entire input stream first, however, are quite apt to cause deadlock.
The big problem with this approach is that if you don't have control over source code being run in the child process, you can't control what it does with pipe buffering. Thus you can't just open a pipe to cat -v and continually read and write a line from it.
The IO::Pty and Expect modules from CPAN can help with this, as they provide a real tty (well, a pseudo-tty, actually), which gets you back to line buffering in the invoked command again.
In a nutshell: wrote a Perl script using flock(). On Linux, it behaves as expected. On AIX, flock() always returns 1, even though another instance of the script, using flock(), should be holding an exclusive lock on the lockfile.
We ship a Bash script to restart our program, relying on flock(1) to prevent simultaneous restarts from making multiple processes. Recently we deployed on AIX, where flock(1) doesn't come by default and won't be provided by the admins. Hoping to keep things simple, I wrote a Perl script called flock, like this:
#!/usr/bin/perl
use Fcntl ':flock';
use Getopt::Std 'getopts';
getopts("nu:x:");
%switches = (LOCK_EX => $opt_x, LOCK_UN => $opt_u, LOCK_NB => $opt_n);
my $lockFlags = 0;
foreach $key (keys %switches) {
if($switches{$key}) {$lockFlags |= eval($key)};
}
$fileDesc = $opt_x || $opt_u;
open(my $lockFile, ">&=$fileDesc") || die "Can't open file descriptor: $!";
flock($lockFile, $lockFlags) || die "Can't change lock - $!\n";;
I tested the script by running (flock -n -x 200; sleep 60)200>lockfile twice, nearly simultaneously, from two terminal tabs.
On Linux, the second run dies with "Resource temporarily unavailable", as expected.
On AIX, the second run acquires the lock, with flock() returning 1, as most definitely not expected.
I understand the flock() is implemented differently on the two systems, the Linux version using flock(1) and the AIX one using, I think, fcntl(1). I don't have enough expertise to understand how this causes my problem, and how to solve it.
Many thanks for any advice.
This isn't anything to do with AIX, the open() call in your script is incorrect.
Should should be something like:
open (my $lockfile, ">>", $fileDesc) # for LOCK_EX, must be write
You were using the "dup() previously opened file handle" syntax with >&=, but the script had not opened any files to duplicate, nor should it.
My quick tests shows the correct behavior (debugging added)
first window:
$ ./flock.pl -n -x lockfile
opened lockfile
locked
second window:
$./flock.pl -n -x lockfile
opened lockfile
Can't change lock - Resource temporarily unavailable
$
It's not about different commands, I suppose; it's more about global differences between AIX and Linux.
In POSIX systems, file locks are advisory: each program could check the file's state and then reconsider what has to be done with it. No explicit checks = no locking.
In Linux systems, however, one can try to enforce a mandatory lock, although the doc itself states that it would be unwise to rely on it: implementation is (and probably will ever be) buggy.
Therefore, I suggest implementing such checks of advisory flags within the script itself.
More about it: man 2 fcntl, man 2 flock.
In Perl, is it possible to make 'exec', 'system', and 'qx' use a shell other than /bin/sh (without using a construct like 'exec "$SHELL -c ..."', and without recompiling perl)?
EDIT: The motivation for this question is a bash script that does 'export -f foo' and then uses perl in a subshell to invoke the function directly via 'system "foo"'. I am not sure that this technique will work with all sh, and although 'system "/bin/bash -c foo"' may work in that scenario, I wouldn't expect the exported function to propagate through all variants of /bin/sh. But mostly I was just curious, and am now curious about how to extend the solution to qx. Also, since I know nothing about non-unix platforms, I'd like to avoid hard coding the path to an alternate shell in the solution.
You can override exec and system. See perldoc perlsub for the details, but here is roughly what you want (modulo some quoting bugs I don't feel like trying to fix):
#!/usr/bin/perl
use strict;
use warnings;
use subs qw/system/;
sub system {
#handle one arg version:
if (#_ == 1) {
return CORE::system "$ENV{SHELL} -c $_[0]";
}
#handle the multi argument version
return CORE::system #_;
}
print "normal system:\n";
system "perl", "-e", q{system q/ps -ef | grep $$/};
print "overloaded system:\n";
system 'ps -ef | grep $$';
exec and system will use the shell (which will likely not be /bin/sh on non-UNIX systems) if you only pass one argument to it. (Details are described in perlfunc)
You may want to have a look at IPC::Run3 as an alternative to system
Why don't you want to use 'exec "$SHELL -c ..."'? If you don't want see that code every time you call exec or system, just hide it in a subroutine. That's what they're there for. :)
sub my_exec {
exec $ENV{SHELL}, '-c', #_;
}
If you want to do that, however, I suggest somehow sanitizing $ENV{SHELL} so that people don't do odd things to your script by setting weird values. You might want to ensure that the shell is listed in /etc/shells or whatever way your system lists approved login shells. You also need to do a bit more work to make this taint-clean, which you should probably do if you are going to send data to another process.
exec doesn't use /bin/sh
It just execs the program you specify. No shells.
If you want it to go through a shell you have to do that yourself.