Apologies if some of the terminology may be slighlty off here. Feel free to correct me if I use a wrong term for something.
Is it possible to use Perl as an "advanced shell" for running "batch" scripts? (on Windows)
The problem I face when replacing a .bat/.cmd script that's getting too complicated with a perl script is that I can't easily run sub processes as a shell does.
That is, I would like to do the same thing from my perl script as a shell does when invoking a child process, that is, fully "connecting" STDIN, STDOUT and STDERR.
Example:
foo.bat -
#echo off
echo Hello, this is a simple script.
set PARAM_X=really-simple
:: The next line will allow me to simply "use" the tool on the shell I have open,
:: that is STDOUT + STDERR of the tool are displayed on my STDOUT + STDERR and if
:: I enter something on the keyboard it is sent to the tools STDIN
interactive_commandline_tool.exe %PARAM_X%
echo The tool returned %ERROLEVEL%
However, I have no clue how to fully implement this in perl (is it possible at all?):
foo.pl -
print "Hello, this is a not so simple script.\n";
my $param_x = get_more_complicated_parameter();
# Magic: This sub executes the process and hooks up my STDIN/OUT/ERR and
# returns the process error code when done
my $errlvl = run_executable("interactive_commandline_tool.exe", $param_x);
print "The tool returned $errlvl\n";
How can I achieve this in perl? I played around with IPC::Open3 but it seems this doesn't do the trick ...
Probably you'll find IPC::Run3 useful. It allow you to capture both STDOUT and STDERR (but not pipe them in real time). Command error level will be returned in $?.
Why not this way:
print "Hello, this is a not so simple script.\n";
my $param_x = get_more_complicated_parameter();
system('cmd.exe', $param_x);
my $errorlevel = $? >> 8;
print "The tool returned $errorlevel\n";
sub get_more_complicated_parameter { 42 }
I don't have your interactive program, but the shell executed allowed me to enter commands, it has inherited environment defined in perl, etc.
I am using perl as replacement for more complicated shell scripts on Windows for long time and so far everything I needed was possible.
Related
I have a Perl CGI program. myperlcgi.pl
Within this program I have the following:
my $retval = system('extprogram')
extprogram has a print statement within.
The output from extprogram is being included within myperlcgi.pl
I tried adding
> output.txt 2>&1
to system call but did not change anything.
How do I prevent output form extprogram being used in myperlcgi.pl.
Surprised that stdout from system call is being used in myperlcgi.pl
The system command just doesn’t give you complete control over capturing STDOUT and STDERR of the executed command.
Use backticks or open to execute the command instead. That will capture the STDOUT of the command’s execution. If the command also outputs to STDERR, then you can append 2>&1 to redirect STDERR to STDOUT for capture in backticks, like so:
my $output = `$command 2>&1`;
If you really need the native return status of the executed command, you can get that information using $? or ${^CHILD_ERROR_NATIVE}. See perldoc perlvar for details.
Another option is to use the IPC::Open3 Perl library, but I find that method to be overkill for most situations.
I need to write a perl script that calls a c-shell script that calls yet another perl script. I cannot change the c-shell script or the perl script it calls. One of the args that needs to be passed is a quotes string with spaces. If I use backticks to call the c-shell, and I run the c-shell with tcsh, the quoted string is respected as a single entity. However, if I run the c-shell with source, it is not.
I feel that I need to use 'source' because when the c-shell is called by users from the command line, it is called through an alias that sources the c-shell. E.g.
alias top "source top.csh"
Consider these...
topmost.pl
#!/usr/bin/env perl
use strict;
print "Try with tcsh...\n";
my $msg = `tcsh ./top.csh -arg1 "this line has spaces"`;
print "$msg\n";
print "Try with source...\n";
my $msg = `source ./top.csh -arg1 "this line has spaces"`;
print "$msg\n";
exit;
top.csh is simply....
perl ./subperl.pl $*:q
exit
And subperl.pl is...
#!/usr/bin/env perl
use strict;
print "In subperl.pl\n";
foreach $x (#ARGV) {
print "$x\n";
}
print "The End\n";
exit;
When I run the topmost.pl script, I get...
Try with tcsh...
In subperl.pl
-arg1
this line has spaces
The End
Try with source...
In subperl.pl
-arg1
this
line
has
spaces:q
The End
Why does the "sourced" call to the top.csh script fail to respect the quotes ?
#Kaz has the answer as to why your code isn't working. This answer is about how to avoid this class of problems entirely.
First, if you can, add a #!/bin/tcsh to top.csh and make it executable (ie. chmod +x). Now it can be executed as top.csh without needing to know what shell to use.
Then you'll want to avoid using `` for anything but very simple commands. This is because `` is interpreted by the shell and now you need to worry about shell special characters and escapes and spaces... it's a mess. What you need is a way to call external programs without invoking a shell.
You can do this by passing a list to system, but system cannot capture the output.
system "tcsh", "./top.csh", "-arg1", "this line has spaces";
While you can cobble something together with open and pipes, it's better to use a pre-existing library such as IPC::System::Simple.
use IPC::System::Simple qw(capturex);
# Or capturex("./top.csh", ...) if you add a #! to top.csh.
my $msg = capturex("tcsh", "./top.csh", "-arg1", "this line has spaces");
For more involved interactions with executables, look into System::Command or IPC::Run.
Needless to say, Perl scripts which call shell scripts which call Perl scripts is a bit of a nightmare to maintain. Rather than do that, it is better to scoop the guts of subperl.pl out into a Perl library and have both subperl.pl and your code use that library.
The command in backticks is being interpreted by your system interpreter (invoked via /bin/sh), which I'm guessing might be GNU Bash. Or, in any case, it seems to be some shell which understands the source command, and almost certainly a POSIX-like shell. That source command quite probably tells that shell to read a script written in that shell's own language. So, for instance, if that shell happens to be Bash, it will treat that as a Bash script1, not as a Tcsh script.
The only way both could work is if the script is a "polyglot": a program which can be interpreted by either tcsh or the system shell that is used by perl to implement backticks.
(An easy example of a C Shell + POSIX shell polyglot is a script that contains nothing but a sequence of trivial commands consisting of space separated words like cp from to.)
Your script isn't a polyglot; only Tcsh understands the :q syntax, not the other shell.
More precisely, if /bin/sh is Bash, the original source ... command as well as the contents of the sourced top.csh script will be treated as a POSIX-mode Bash script, since when Bash is invoked as /bin/sh, it turns off its POSIX-incompatible behaviors. So even if Bash's pathname expansion supported the Tcsh :q mechanism, it would almost certainly be turned off under POSIX mode because $*:q already has a firm meaning in POSIX.
I am writing a code in perl with embedded shell script in it:
#!/usr/bin/perl
use strict;
use warnings;
our sub main {
my $n;
my $n2=0;
$n=chdir("/home/directory/");
if($n){
print "change directory successful $n \n";
$n2 = system("cd", "section");
system("ls");
print "$n2 \n";
}
else {
print "no success $n \n";
}
print "$n\n";
}
main();
But it doesn't work. When I do the ls. The ls doesn't show new files. Anyone knows another way of doing it. I know I can use chdir(), but that is not the only problem, as I have other commands which I have created, which are simply shell commands put together. So does anyone know how to exactly use cli in perl, so that my compiler will keep the shell script attached to the same process rather than making a new process for each system ... I really don't know what to do.
The edits have been used to improve the question. Please don't mind the edits if the question is clear.
edits: good point made by mob that the system is a single process so it dies everytime. But, What I am trying to do is create a perl script which follows an algorithm which decides the flow of control of the shell script. So how do I make all these shell commands to follow the same process?
system spawns a new process, and any changes made to the environment of the new process are lost when the process exits. So calling system("cd foo") will change the directory to foo inside of a very short-lived process, but won't have any effect on the current process or any future subprocesses.
To do what you want to do (*), combine your commands into a single system call.
$n2 = system("cd section; ls");
You can use Perl's rich quoting features to pass longer sequences of commands, if necessary.
$n2 = system q{cd section
if ls foo ; then
echo we found foo in section
./process foo
else
echo we did not find foo\!
./create_foo > foo
fi};
$n2 = system << "EOSH";
cd section
./process bar > /tmp/log
cd ../sekshun
./process foo >> /tmp/log
if grep -q Error /tmp/log ; then
echo there were errors ...
fi
EOSH
(*) of course there are easy ways to do this entirely in Perl, but let's assume that the OP eventually will need some function only available in an external program
system("cd", "section"); attempts to execute the program cd, but there is no such program on your machine.
There is no such program because each process has its own current work directory, and one process cannot change another process's current work directory. (Programs would malfunction left and right if it was possible.)
It looks like you are attempting to have a Perl program execute a shell script. That requires recreating the shell in Perl. (More specific goals might have simpler solutions.)
What I am trying to do is create a perl script which follows an algorithm which decides the flow of control of the shell script.
Minimal change:
Create a shell script that prompts for instructions/commands. Have your Perl script launch the shell script using Expect and feed it answers/commands.
Synopsis
I execute a shell command from Perl and when run from the command line it works, but when run in the debugger it does not work. Running it as a Win32::Daemon shows the same behaviour.
The Source Code
I execute a command either with backticks
print `$cmd`
or like this:
open FH, "$cmd |" or die "Couldn't execute $cmd: $!\n";
while(defined(my $line = <FH>)) {
chomp($line);
print "$line\n";
}
close FH;
The command reads like this:
$cmd = '"C:\path\to\sscep.exe" getca -f "C:\path\to\config\capi_sscep.cnf"'
Even creating a small test script that just executes this command does only work if run from command line.
The System
Windows x64
Active Perl v5.16.0, MSWin32-x64-multi-thread
Eclipse Juno 20120614-1722
What works
I works to open an administrator prompt (necessary for script execution) and to:
perl script.pl
Output gets printed to screen, $? is 0.
What does not work
Starting Eclipse and running a debug session with the same perl script.pl call.
Also not working is adding a service and executing the command (created with Win32::Daemon). The daemon itself is working perfectly fine and starting the perl script as expected. Only the command does not get executed. $? is 13568 or 53 if shifted with $? >> 8, no output gets printed. The exit code does not belong to the program.
Further Details
The tool I am calling is called sscep and is extended by me. It uses the OpenSSL API and loads the capi engine (Windows CryptoAPI). But the command itself does at least print output before any serious action starts. I can happily provide the source code for this, but I doubt it will help.
I was able to narrow this further down: The problem only exists in the combination of the Perl program (CertNanny) and the binary (sscep). Calling dir inside CertNanny works, calling sscep in a test Perl script works, too. So what could possibly be done in Perl to break a single binary from being called...?
Any ideas where this problem might originate from or how I can possibly narrow it down?
Here is what I believe the problem to be: when you run your program on the command line, the system() command goes through the shell (cmd.exe); when you run your program elsewhere, it does not. Unfortunately, the two methods handle command line arguments differently. Here is an article that seems like it should help you solve the problem.
In my experience, this sort of thing is a mess in Windows. I have had trouble with this issue in Perl, also.
I'm writing a perl script that mimics gcc. This my script needs to process some stdout from gcc. The part for processing is done, but I can't get the simple part working: how can I forward all the command line parameters as is to the next process (gcc in my case). Command lines sent to gcc tend to be very long and can potentially contain lots of escape sequences and I don't want now to play that game with escaping and I know that it's tricky to get it right on windows in complicated cases.
Basically,
gcc.pl some crazies\ t\\ "command line\"" and that gcc.pl has to forward that same command line to real gcc.exe (I use windows).
I do it like that: open("gcc.exe $cmdline 2>&1 |") so that stderr from gcc is fed to stdout and I my perl script processes stdout. The problem is that I can't find anywhere how to construct that $cmdline.
I would use AnyEvent::Subprocess:
use AnyEvent::Subprocess;
my $process_line = sub { say "got line: $_[0]" };
my $gcc = AnyEvent::Subprocess->new(
code => ['gcc.exe', #ARGV],
delegates => [ 'CompletionCondvar', 'StandardHandles', {
MonitorHandle => {
handle => 'stdout',
callback => $process_line,
}}, {
MonitorHandle => {
handle => 'stderr',
callback => $process_line,
}},
],
);
my $running = $gcc->run;
my $done = $running->recv;
$done->is_success or die "OH NOES";
say "it worked";
The MonitorHandle delegate works like redirection, except you have the option of using a separate filter for each of stdout and stderr. The "code" arg is an arrayref representing a command to run.
"Safe Pipe Opens" in the perlipc documentation describes how to get another command's output without having to worry about how the shell will parse it. The technique is typically used for securely handling untrusted inputs, but it also spares you the error-prone task of correctly escaping all the arguments.
Because it sidesteps the shell, you'll need to create the effect of 2>&1 yourself, but as you'll see below, it's straightforward to do.
#! /usr/bin/perl
use warnings;
use strict;
my $pid = open my $fromgcc, "-|";
die "$0: fork: $!" unless defined $pid;
if ($pid) {
while (<$fromgcc>) {
print "got: $_";
}
}
else {
# 2>&1
open STDERR, ">&STDOUT" or warn "$0: dup STDERR: $!";
no warnings "exec"; # so we can write our own message
exec "gcc", #ARGV or die "$0: exec: $!";
}
Windows proper does not support open FH, "-|", but Cygwin does so happily:
$ ./gcc.pl foo.c
got: gcc: foo.c: No such file or directory
got: gcc: no input files
Read up on the exec function and the system function in Perl.
If you provide either of these with an array of arguments (rather than a single string), it invokes the Unix execve() function or a close relative directly, without letting the shell interpret anything, exactly as you need it to do.
Thanks for answers, I came to conclusion that I made a big mistake that I touched perl again: hours of time wasted to find out that it can't be done properly.
Perl uses different way to split command line parameters than all other apps that use MS stdlib (which is standard on win32).
Because of that some commandline parameters that were meant to be interpreted as a signle commandline argument, by perl can be interpreted as more than one argument. That means that all what I'm trying to do is waste of time because of that buggy behavior in perl. It's impossible to get this task done correctly if I 1) can't access original command line as is and 2) perl doesn't split command line arguments correctly.
as a simple test:
script.pl """test |test"
on win32 will incorrectly interpret command line as:
ARGV=['"test', '|test']
Whereas, the correct "answer" on windows has to be
ARGV=['"test |test']
I used activestate perl, I tried also latest version of strawberry perl: both suck. It appears that perl that comes with msys works properly, most likely because it was built against mingw instead of cygwin runtime?..
The problem and reason with perl is that it has buggy cmd line parser and it won't work on windows NO MATTER WHAT cygwin supports or not.
I have a simple case where an environment variable (which I cannot control) expands to
perl gcc.pl -c "-IC:\ffmpeg\lib_avutil\" rest of args
Perl sees that I have two args only: -c and '-IC:\ffmpeg\lib_avutil" rest of args'
whereas any conforming windows implementation receives second cmd line arg as: '-IC:\ffmpeg\lib_avutil\', that mean that perl is a huge pile of junk for my simple case, because it doesn't provide adequate means to access cmd line arguments. I'm better off using boost::regex and do all my parsing in c++ directly, at least I won't ever make dumb mistakes like ne and != for comparing strings etc. Windows's escaping rules for command line arguments are quite strange, but they are standard on windows and perl for some strange reason doesn't want to follow OS's rules.