I am trying to write a script that uses the bash line editor to recall a previously entered command. Here's the simple session I'm trying to automate.
$ bash --norc --noprofile
bash4.4$ echo hi
hi
bash4.4$
then type '^P^M'
bash4.4$ echo hi
hi
Here is my first attempt at scripting this using the Expect.pm module off CPAN. The two sleep 1s are in there to guard against the possibility of race conditions when invoking the line editor since I'm not sure what perl sees when that happens.
#!/usr/bin/env perl
use strict;
use warnings FATAL => 'all';
use autodie;
use Expect;
my $timeout = 10;
my $exp = Expect->new();
$exp->spawn('bash --norc --noprofile');
# wait for first prompt
$exp->expect($timeout, '$ ');
# send echo hi
$exp->send("echo hi\n");
# wait for prompt again
$exp->expect($timeout, '$ ');
# use history recall ^P, then send ^M
sleep 1;
$exp->send("\cp\cm");
sleep 1;
print "okay done!\n";
It works up until I hit "\cp\cm" (I've also tried "\cP\cM"). According to the perl documentation, \cX introduces an ASCII control character. (http://perldoc.perl.org/perlrebackslash.html#Character-Escapes)
Control characters \c is used to denote a control character; the
character following \c determines the value of the construct. For
example the value of \cA is chr(1), and the value of \cb is chr(2),
etc. The gory details are in Regexp Quote-Like Operators in perlop. A
complete list of what chr(1), etc. means for ASCII and EBCDIC
platforms is in OPERATOR DIFFERENCES in perlebcdic.
This is what I actually get when I run the script, which strongly suggests that the control characters are not getting passed to the subprocess properly and possibly aren't passed at all.
% perl bash.pl
bash-4.4$ echo hi
hi
bash-4.4$ okay done!
What's going on here? How do I pass a control character to a process with Expect.pm?
The characters escapes you are searching are specific to Perl regex. For your purpose, I would suggest you to pass the hex value for ctrl-M and ctrl-P.
$exp->send("\x10"); # ctrl+P
$exp->send("\x0D"); # ctrl+M
Update(tested):
$exp->send("\x10"); # ctrl+P
$exp->send("\n"); # send newline
sleep 2;
# wait for prompt
$exp->expect($timeout, '$ ');
$exp->send("\x0D"); # ctrl+M
$exp->send("\n"); # send newline
Related
What would be an example of how I can call a shell command, say 'ls -a' in a Perl script and the way to retrieve the output of the command as well?
How to run a shell script from a Perl program
1. Using system system($command, #arguments);
For example:
system("sh", "script.sh", "--help" );
system("sh script.sh --help");
System will execute the $command with
#arguments and return to your script when finished. You may check $!
for certain errors passed to the OS by the external application. Read
the documentation for system for the nuances of how various
invocations are slightly different.
2. Using exec
This is very similar to the use of system, but it will
terminate your script upon execution. Again, read the documentation
for exec for more.
3. Using backticks or qx//
my $output = `script.sh --option`;
my $output = qx/script.sh --option/;
The backtick operator and it's equivalent qx//, excute the command and options inside the operator and return that commands output to STDOUT when it finishes.
There are also ways to run external applications through creative use of open, but this is advanced use; read the documentation for more.
From Perl HowTo, the most common ways to execute external commands from Perl are:
my $files = `ls -la` — captures the output of the command in $files
system "touch ~/foo" — if you don't want to capture the command's output
exec "vim ~/foo" — if you don't want to return to the script after executing the command
open(my $file, '|-', "grep foo"); print $file "foo\nbar" — if you want to pipe input into the command
Examples
`ls -l`;
system("ls -l");
exec("ls -l");
Look at the open function in Perl - especially the variants using a '|' (pipe) in the arguments. Done correctly, you'll get a file handle that you can use to read the output of the command. The back tick operators also do this.
You might also want to review whether Perl has access to the C functions that the command itself uses. For example, for ls -a, you could use the opendir function, and then read the file names with the readdir function, and finally close the directory with (surprise) the closedir function. This has a number of benefits - precision probably being more important than speed. Using these functions, you can get the correct data even if the file names contain odd characters like newline.
As you become more experienced with using Perl, you'll find that there are fewer and fewer occasions when you need to run shell commands. For example, one way to get a list of files is to use Perl's built-in glob function. If you want the list in sorted order you could combine it with the built-in sort function. If you want details about each file, you can use the stat function. Here's an example:
#!/usr/bin/perl
use strict;
use warnings;
foreach my $file ( sort glob('/home/grant/*') ) {
my($dev,$ino,$mode,$nlink,$uid,$gid,$rdev,$size,$atime,$mtime,$ctime,$blksize,$blocks)
= stat($file);
printf("%-40s %8u bytes\n", $file, $size);
}
There are a lot of ways you can call a shell command from a Perl script, such as:
back tick
ls which captures the output and gives back to you.
system
system('ls');
open
Refer #17 here: Perl programming tips
You might want to look into open2 and open3 in case you need bidirectional communication.
I have been using system and qq to run linux programs inside perl. And it has worked well.
#!/usr/bin/perl # A hashbang line in perl
use strict; # It can save you a lot of time and headache
use warnings; # It helps you find typing mistakes
# my keyword in Perl declares the listed variable
my $adduser = '/usr/sbin/adduser';
my $edquota = '/usr/sbin/edquota';
my $chage = '/usr/bin/chage';
my $quota = '/usr/bin/quota';
my $nomeinteiro;
my $username;
my $home;
# system() function executes a system shell command
# qq() can be used in place of double quotes
system qq($adduser --home $home --gecos "$fullname" $username);
system qq($edquota -p john $username);
system qq($chage -E \$(date -d +180days +%Y-%m-%d) $username);
system qq($chage -l $username);
system qq($quota -s $username);
I know this is incorrect. I just want to know how perl parses this.
So, I'm playing around with perl, what I wanted was perl -ne what I typed was perl -ie the behavior was kind of interesting, and I'd like to know what happened.
$ echo 1 | perl -ie'next unless /g/i'
So perl Aborted (core dumped) on that. Reading perl --help I see -i takes an extension for backups.
-i[extension] edit <> files in place (makes backup if extension supplied)
For those that don't know -e is just eval. So I'm thinking one of three things could have happened either it was parsed as
perl -i -e'next unless /g/i' i gets undef, the rest goes as argument to e
perl -ie 'next unless /g/i' i gets the argument e, the rest is hanging like a file name
perl -i"-e'next unless /g/i'" whole thing as an argument to i
When I run
$ echo 1 | perl -i -e'next unless /g/i'
The program doesn't abort. This leads me to believe that 'next unless /g/i' is not being parsed as a literal argument to -e. Unambiguously the above would be parsed that way and it has a different result.
So what is it? Well playing around with a little more, I got
$ echo 1 | perl -ie'foo bar'
Unrecognized switch: -bar (-h will show valid options).
$ echo 1 | perl -ie'foo w w w'
... works fine guess it reads it as `perl -ie'foo' -w -w -w`
Playing around with the above, I try this...
$ echo 1 | perl -ie'foo e eval q[warn "bar"]'
bar at (eval 1) line 1.
Now I'm really confused.. So how is Perl parsing this? Lastly, it seems you can actually get a Perl eval command from within just -i. Does this have security implications?
$ perl -i'foo e eval "warn q[bar]" '
Quick answer
Shell quote-processing is collapsing and concatenating what it thinks is all one argument. Your invocation is equivalent to
$ perl '-ienext unless /g/i'
It aborts immediately because perl parses this argument as containing -u, which triggers a core dump where execution of your code would begin. This is an old feature that was once used for creating pseudo-executables, but it is vestigial in nature these days.
What appears to be a call to eval is the misparse of -e 'ss /g/i'.
First clue
B::Deparse can your friend, provided you happen to be running on a system without dump support.
$ echo 1 | perl -MO=Deparse,-p -ie'next unless /g/i'
dump is not supported.
BEGIN { $^I = "enext"; }
BEGIN { $/ = "\n"; $\ = "\n"; }
LINE: while (defined(($_ = <ARGV>))) {
chomp($_);
(('ss' / 'g') / 'i');
}
So why does unle disappear? If you’re running Linux, you may not have even gotten as far as I did. The output above is from Perl on Cygwin, and the error about dump being unsupported is a clue.
Next clue
Of note from the perlrun documentation:
-u
This switch causes Perl to dump core after compiling your program. You can then in theory take this core dump and turn it into an executable file by using the undump program (not supplied). This speeds startup at the expense of some disk space (which you can minimize by stripping the executable). (Still, a "hello world" executable comes out to about 200K on my machine.) If you want to execute a portion of your program before dumping, use the dump operator instead. Note: availability of undump is platform specific and may not be available for a specific port of Perl.
Working hypothesis and confirmation
Perl’s argument processing sees the entire chunk as a single cluster of options because it begins with a dash. The -i option consumes the next word (enext), as we can see in the implementation for -i processing.
case 'i':
Safefree(PL_inplace);
[Cygwin-specific code elided -geb]
{
const char * const start = ++s;
while (*s && !isSPACE(*s))
++s;
PL_inplace = savepvn(start, s - start);
}
if (*s) {
++s;
if (*s == '-') /* Additional switches on #! line. */
s++;
}
return s;
For the backup file’s extension, the code above from perl.c consumes up to the first whitespace character or end-of-string, whichever is first. If characters remain, the first must be whitespace, then skip it, and if the next is a dash then skip it also. In Perl, you might write this logic as
if ($$s =~ s/i(\S+)(?:\s-)//) {
my $extension = $1;
return $extension;
}
Then, all of -u, -n, -l, and -e are valid Perl options, so argument processing eats them and leaves the nonsensical
ss /g/i
as the argument to -e, which perl parses as a series of divisions. But before execution can even begin, the archaic -u causes perl to dump core.
Unintended behavior
An even stranger bit is if you put two spaces between next and unless
$ perl -ie'next unless /g/i'
the program attempts to run. Back in the main option-processing loop we see
case '*':
case ' ':
while( *s == ' ' )
++s;
if (s[0] == '-') /* Additional switches on #! line. */
return s+1;
break;
The extra space terminates option parsing for that argument. Witness:
$ perl -ie'next nonsense -garbage --foo' -e die
Died at -e line 1.
but without the extra space we see
$ perl -ie'next nonsense -garbage --foo' -e die
Unrecognized switch: -onsense -garbage --foo (-h will show valid options).
With an extra space and dash, however,
$ perl -ie'next -unless /g/i'
dump is not supported.
Design motivation
As the comments indicate, the logic is there for the sake of harsh shebang (#!) line constraints, which perl does its best to work around.
Interpreter scripts
An interpreter script is a text file that has execute permission enabled and whose first line is of the form:
#! interpreter [optional-arg]
The interpreter must be a valid pathname for an executable which is not itself a script. If the filename argument of execve specifies an interpreter script, then interpreter will be invoked with the following arguments:
interpreter [optional-arg] filename arg...
where arg... is the series of words pointed to by the argv argument of execve.
For portable use, optional-arg should either be absent, or be specified as a single word (i.e., it should not contain white space) …
Three things to know:
'-x y' means -xy to Perl (for some arbitrary options "x" and "y").
-xy, as common for unix tools, is a "bundle" representing -x -y.
-i, like -e absorbs the rest of the argument. Unlike -e, it considers a space to be the end of the argument (as per #1 above).
That means
-ie'next unless /g/i'
which is just a fancy way of writing
'-ienext unless /g/i'
unbundles to
-ienext -u -n -l '-ess /g/i'
^^^^^ ^^^^^^^
---------- ----------
val for -i val for -e
perlrun documents -u as:
This switch causes Perl to dump core after compiling your program. You can then in theory take this core dump and turn it into an executable file by using the undump program (not supplied). This speeds startup at the expense of some disk space (which you can minimize by stripping the executable). (Still, a "hello world" executable comes out to about 200K on my machine.) If you want to execute a portion of your program before dumping, use the dump() operator instead. Note: availability of undump is platform specific and may not be available for a specific port of Perl.
I'm writing a Perl script that allows me to change the tile of my terminal tabs, however, everytime I run it, a "-n -e" gets printed to my terminal line. If I leave those options out, just a blank line gets printed. Is there any way I can execute the system command inside perl, and not have any code remnants show up on my terminal?
1 #!/usr/bin/perl
2 #sets the title on terminal tabs (mac OSX)
3
4 #use strict;
5 use warnings;
6
7 #sets title of term window to input from user.
8 my $textline="\'echo -n -e \"\\033]0;";
9 $textline = $textline . "#ARGV" . '\007"\'';
10
11 system "\'$textline\'";
12
Thanks
I imagine -n and -e get echoed because your version of echo doesn't recognize those options and treats them as literal text to echo.
But you know what, Perl has its own tool for echoing text: print.
print("\033]0;$title\007");
You might need to flush the buffer afterwards:
use IO::Handle qw( );
STDOUT->flush();
Or just the following if you don't mind turning off buffering:
$| = 1;
PS - Your quotes (''echo ...'') makes no sense, but they collapse into nothingness, so they don't cause a problem.
You can redirect the outputs by '>' and '2>' or you can use qx.
For e.g in qx case you can write
my $output = qx( $your_command );
What would be an example of how I can call a shell command, say 'ls -a' in a Perl script and the way to retrieve the output of the command as well?
How to run a shell script from a Perl program
1. Using system system($command, #arguments);
For example:
system("sh", "script.sh", "--help" );
system("sh script.sh --help");
System will execute the $command with
#arguments and return to your script when finished. You may check $!
for certain errors passed to the OS by the external application. Read
the documentation for system for the nuances of how various
invocations are slightly different.
2. Using exec
This is very similar to the use of system, but it will
terminate your script upon execution. Again, read the documentation
for exec for more.
3. Using backticks or qx//
my $output = `script.sh --option`;
my $output = qx/script.sh --option/;
The backtick operator and it's equivalent qx//, excute the command and options inside the operator and return that commands output to STDOUT when it finishes.
There are also ways to run external applications through creative use of open, but this is advanced use; read the documentation for more.
From Perl HowTo, the most common ways to execute external commands from Perl are:
my $files = `ls -la` — captures the output of the command in $files
system "touch ~/foo" — if you don't want to capture the command's output
exec "vim ~/foo" — if you don't want to return to the script after executing the command
open(my $file, '|-', "grep foo"); print $file "foo\nbar" — if you want to pipe input into the command
Examples
`ls -l`;
system("ls -l");
exec("ls -l");
Look at the open function in Perl - especially the variants using a '|' (pipe) in the arguments. Done correctly, you'll get a file handle that you can use to read the output of the command. The back tick operators also do this.
You might also want to review whether Perl has access to the C functions that the command itself uses. For example, for ls -a, you could use the opendir function, and then read the file names with the readdir function, and finally close the directory with (surprise) the closedir function. This has a number of benefits - precision probably being more important than speed. Using these functions, you can get the correct data even if the file names contain odd characters like newline.
As you become more experienced with using Perl, you'll find that there are fewer and fewer occasions when you need to run shell commands. For example, one way to get a list of files is to use Perl's built-in glob function. If you want the list in sorted order you could combine it with the built-in sort function. If you want details about each file, you can use the stat function. Here's an example:
#!/usr/bin/perl
use strict;
use warnings;
foreach my $file ( sort glob('/home/grant/*') ) {
my($dev,$ino,$mode,$nlink,$uid,$gid,$rdev,$size,$atime,$mtime,$ctime,$blksize,$blocks)
= stat($file);
printf("%-40s %8u bytes\n", $file, $size);
}
There are a lot of ways you can call a shell command from a Perl script, such as:
back tick
ls which captures the output and gives back to you.
system
system('ls');
open
Refer #17 here: Perl programming tips
You might want to look into open2 and open3 in case you need bidirectional communication.
I have been using system and qq to run linux programs inside perl. And it has worked well.
#!/usr/bin/perl # A hashbang line in perl
use strict; # It can save you a lot of time and headache
use warnings; # It helps you find typing mistakes
# my keyword in Perl declares the listed variable
my $adduser = '/usr/sbin/adduser';
my $edquota = '/usr/sbin/edquota';
my $chage = '/usr/bin/chage';
my $quota = '/usr/bin/quota';
my $nomeinteiro;
my $username;
my $home;
# system() function executes a system shell command
# qq() can be used in place of double quotes
system qq($adduser --home $home --gecos "$fullname" $username);
system qq($edquota -p john $username);
system qq($chage -E \$(date -d +180days +%Y-%m-%d) $username);
system qq($chage -l $username);
system qq($quota -s $username);
As I understand (Perl is new to me) Perl can be used to script against a Unix command line. What I want to do is run (hardcoded) command line calls, and search the output of these calls for RegEx matches. Is there a way to do this simply in Perl? How?
EDIT: Sequence here is:
-Call another program.
-Run a regex against its output.
my $command = "ls -l /";
my #output = `$command`;
for (#output) {
print if /^d/;
}
The qx// quasi-quoting operator (for which backticks are a shortcut) is stolen from shell syntax: run the string as a command in a new shell, and return its output (as a string or a list, depending on context). See perlop for details.
You can also open a pipe:
open my $pipe, "$command |";
while (<$pipe>) {
# do stuff
}
close $pipe;
This allows you to (a) avoid gathering the entire command's output into memory at once, and (b) gives you finer control over running the command. For example, you can avoid having the command be parsed by the shell:
open my $pipe, '-|', #command, '< single argument not mangled by shell >';
See perlipc for more details on that.
You might be able to get away without Perl, as others have mentioned. However, if there is some Perl feature you need, such as extended regex features or additional text manipulation, you can pipe your output to perl then do what you need. Perl's -e switch let's you specify the Perl program on the command line:
command | perl -ne 'print if /.../'
There are several other switches you can pass to perl to make it very powerful on the command line. These are documented in perlrun. Also check out some of the articles in Randal Schwartz's Unix Review column, especially his first article for them. You can also google for Perl one liners to find lots of examples.
Do you need Perl at all? How about
command -I use | grep "myregexp" && dosomething
right in the shell?
#!/usr/bin/perl
sub my_action() {
print "Implement some action here\n";
}
open PROG, "/path/to/your/command|" or die $!;
while (<PROG>) {
/your_regexp_here/ and my_action();
print $_;
}
close PROG;
This will scan output from your command, match regexps and do some action (which now is printing the line)
In Perl you can use backticks to execute commands on the shell. Here is a document on using backticks. I'm not sure about how to capture the output, but I'm sure there's more than a way to do it.
You indeed use a one-liner in a case like this. I recently coded up one that I use, among other ways, to produce output which lists the directory structure present in a .zip archive (one dir entry per line). So using that output as an example of command output that we'd like to filter, we could put a pipe in and then use perl with the -n -e flags to filter the incoming data (and/or do other things with it):
[command_producing_text_output] | perl -MFile::Path -n -e \
"BEGIN{#PTM=()} if (m{^perl/(bin|lib(?!/site))}) {chomp;push #PTM,$_}" ^
-e "END{#WDD=mkpath (\#PTM,1);" ^
-e "printf qq/Created %u dirs to reflect part of structure present in the .ZIP file\n/, scalar(#WDD);}"
the shell syntax used, including: quoting of perl code and escaping of newlines, reflects CMD.exe usage in Windows NT-like consoles. If you need to, mentally replace
"^" with "\" and " with ' in the appropriate places.
The one-liner above adds only the directory names that start with "perl/bin" or
"perl/lib (not followed by "/site"); it then creates those directories. You wind
up with a (empty) tree that you can use for whatever evil purposes you desire.
The main point is to illustrate that there are flags available (-n, -p) to
allow perl to loop over each input record (line), and that what you can do is unlimited in terms of complexity.