Hi I'm wondering if there is something for Perl similar to Rstudio? That is to ability to run commands, retain all variables in memory without exiting the script.
For example say I execute this command my $temp = 83; then instead of ending the script I change the value $temp = 22; print "$temp \n"; and so on, but I don't end the script and continue to work on it. This will be extremely helpful when dealing with a large datasets and general workflow.
The closest thing I came across is Visual Studio Code using a plugin whereby I can execute specific chunks of code in my script. However I did not find a way to keep the variable persistently in memory.
thanks!
You want a REPL.
Take a look at Devel::REPL. It brings a script called re.pl that you can run.
$ re.pl
$ my $foo = 123;
123$ use feature 'say';
$ $foo + 1;
124$
A newer alternative is Reply with its reply script.
$ reply
0> my $foo = 123;
$res[0] = 123
1> $foo + 2
$res[1] = 125
2>
For a comparison, you can read this blog post by Matt Trout.
Related
I've been tasked with understanding and debugging some legacy code written in Perl 5. The comments aren't great, and I have almost zero experience with the language (I'm a Python guy; some shell scripting experience). Basically, I need to understand it to the point where I can decide if it will be worth it to rewrite it in Python to match the rest of our code base and be more readable to those who aren't familiar with Perl's less obvious syntax (at least, to the uninitated...) or just leave it as is and do our best to make small changes as necessary despite having an incomplete understanding of what's going on.
Perl is being invoked to process some data from within a script in our organization's internal scripting language (which is used to interact with our proprietary hardware). The part written in the in the proprietary scripting language is similar to a shellscript in that every line sent to the interpreter will be written in the log, so while not as easy as debugging something like Python in a modern IDE, it's possible to figure out what's going on by checking the logs. But perl, as a programming language, only prints/logs what you tell it to. So as of right now, it's kind of a black box.
I looked at the Perl docs and saw that there is a command line option to launch the debugger -d, along with -Dtls to configure debugger behavior (those are the recommend options to "watch how perl executes your program"). But when I tried running that in the script, i got the following...warning? error?
Recompile perl with -DDEBUGGING to use -D switch (did you mean -d ?)
Because it's being called inside a proprietary-scripting langauge script, if this debugger is just an interactive shell, I don't think it will suit this purpose (because I can't send things to stdin while the proprietary-scripting langauge script is running). But if it's not interactive, adding a 2nd perl installation to the server for debugging wouldn't be out of the question, so if anyone has experience with this debugger mode, and options, I would appreciate some feedback about that.
I'm quite familiar with Python, so I know alot of tricks to log everything or to set up a debug environment to use the VS code debugger, but with Perl, I'm out of my comfort zone.
My question is: is there some sort of easy way (a flag or something) to call Perl in such a way where every command sent to the interpreter is written to the console/stdout/ or a logfile in the same way as a shell script? Or is there another good way to debug perl scripts (other than using the interactive debug shell)? Or do I have no better option than to take the time to go through this rather massive script and put print statements everywhere?
Thanks for reading a lengthy question.
You can use the Perl debugger non-interactively to print each statement as they are executed. Here is an example: Consider you have a Perl script p.pl, for example:
use feature qw(say);
use strict;
use warnings;
$DB::trace=1; # <-- turn on AutoTrace from this point..
{
my $bar = "xyz";
$bar =~ s/y//;
say "bar = $bar";
func1();
}
sub func1 {
for (1..3) {
say "1 : $_";
func2($_);
}
say "Done 1";
}
sub func2 {
my $x = shift;
my $bar = $x ** 2;
say "2: $bar";
}
Then execute p.pl like this:
$ PERLDB_OPTS="NonStop=1 AutoTrace=0" perl -d p.pl
main::(p.pl:7): my $bar = "xyz";
main::(p.pl:7): my $bar = "xyz";
main::(p.pl:8): $bar =~ s/y//;
main::(p.pl:9): say "bar = $bar";
bar = xz
main::(p.pl:10): func1();
main::func1(p.pl:14): for (1..3) {
main::func1(p.pl:15): say "1 : $_";
1 : 1
main::func1(p.pl:16): func2($_);
main::func2(p.pl:22): my $x = shift;
main::func2(p.pl:23): my $bar = $x ** 2;
main::func2(p.pl:24): say "2: $bar";
2: 1
main::func1(p.pl:15): say "1 : $_";
1 : 2
main::func1(p.pl:16): func2($_);
main::func2(p.pl:22): my $x = shift;
main::func2(p.pl:23): my $bar = $x ** 2;
main::func2(p.pl:24): say "2: $bar";
2: 4
main::func1(p.pl:15): say "1 : $_";
1 : 3
main::func1(p.pl:16): func2($_);
main::func2(p.pl:22): my $x = shift;
main::func2(p.pl:23): my $bar = $x ** 2;
main::func2(p.pl:24): say "2: $bar";
2: 9
main::func1(p.pl:18): say "Done 1";
Done 1
You can emulate bash -x ... in a Perl script with the Devel::DumpTrace module.
#!/usr/bin/perl
# demo.pl: a demonstration of Devel::DumpTrace
$a = 1;
$b = 3;
$c = 2 * $a + 7 * $b;
#d = ($a, $b, $c + $b);
$ perl -d:DumpTrace demo.pl
>>>>> demo.pl:3: $a:1 = 1;
>>>>> demo.pl:4: $b:3 = 3;
>>>>> demo.pl:5: $c:23 = 2 * $a:1 + 7 * $b:3;
>>>>> demo.pl:6: #d:(1,3,26) = ($a:1, $b:3, $c:23 + $b:3);
There are settings to produce more or less output, and ways to just turn tracing on for interesting sections of code. It works best if you also have PPI installed, but it also works without PPI.
For something lighter weight and which may already be installed on your system, also see Devel::Trace.
I need to run a shell command within a perl script(It needs to be a shell command) but I need to use a variable in a perl loop
Ex. doing something like
#!/usr/bin/perl
for my $i (0..9) {
`echo $i`;
}
should produce the output
0
1
2
3
4
5
6
7
8
9
You can do it in several ways, depending on what exactly is needed. Generally, this involves use of system, backticks (with operator form qx), or open. There is also a number of great modules for it.
The qx returns its STDOUT to the program. If you need it assign it to a variable.
for my $i (0..9) {
my $cmd = "external-command $i";
my $cmdout = qx($cmd); # needs error checking ($?)
print $cmdout; # process as needed
}
The output can also be assigned to an array, when each line becomes one array element. Depending on what your command is you may have to pay close attention to how to quote it correctly, and String::ShellQuote can help a lot. If you want to get the command's STDERR as well use
my $cmd = "external-command $i 2>&1";
See qx in perlop (or readpipe), which also discusses error checking; also see $? in pervar.
If you don't need the command's output in your script, or specifically want it to wind up together with your script's other output, use system. Commands run by it inherit the script's STDOUT and STDERR so what they print to standard streams will go where your script's output goes (unless it is redirected by the command invocation).
for my $i (0..9) {
my $cmd = "external-command $i";
my $ret = system($cmd);
if ($ret != 0) {
warn "Error running $cmd: $ret";
# interrogate further, see documentation for 'system'
}
}
The error handling above is rudimentary, see docs for more. Finally, there are various forms of open that run a process. For example, see it in perlfaq8, along with a full review of what you are asking.
And then there are modules, that make this kind of work far easier and better. From simple to more capable, some of well-known ones are IPC::System::Simple, Capture::Tiny, IPC::Run3, and IPC::Run. Also see links (examples) assembled in this post, for example.
A command can be executed via a shell or not (but via execvp) depending on how exactly it is invoked, as discussed in the docs. The phrase "shell command" strictly refers to a facility provided by the shell itself, in which case you of course want to run it that way. However, it is also often used for a program that we normally execute by typing in a terminal, so it gets executed by the shell. In this case you may (or may not) want to bypass the shell.
Also note that the above all block (do not return control before completing), so the script continues only after they're done. This seems to be what you want.
If you just want to pass through the shell output, simply do
print STDOUT `echo $i`;
I want to take date as input from User & pass that input to shell command below.
for e.g.
$date = ARGV[0];
`cd xyz $date`
will this variable be interpolated in perl?
You have a couple of problems; first of all, cd only takes one parameter. Perhaps you meant something like cd xyz$date? Second, backticks start a shell that executes the command you give, which will do the change directory command and then immediately exit, having no effect. (The parent perl process's current directory is left unchanged.) You might be looking for chdir.
But yes, the interpolation will work; to disable interpolation in backticks, you either escape special characters (echo xyz \$date) or use qx with single quote delimiters (qx'echo xyz $date').
This is trivial to test:
Input:
use v5.16;
use strict;
use warnings;
my $foo = 123;
say `echo $foo`;
Output:
123
So yes.
Beware of variables containing characters with special meaning in shell though.
Yes, it will interpolate the variable. However, I highly recommend two things:
Use qx/../ instead of back ticks.
Set the command you're running first in a Perl variable, and then interpolate that variable.
The qx/../ is nice because it makes it more obvious what you're doing:
my $variable = qx(ls $directory);
You could use a wide variety of characters for qx:
my $variable = qx(ls $directory);
my $variable = qx/ls $directory/;
my $variable = qx^ls $directory^;
my $variable = qx#ls $directory#;
The character after the qx will be used as quotes. If you use parentheses, square brackets, or curly braces, you use them in pairs with the opening one and closing one.
This means that you can avoid issues where a particular character in your command might confuse things.
The other thing is to build your command into a Perl variable and then execute it when you try to do interpolations. This gives you more control over what you're doing:
my $HOME;
my $command;
$HOME = "bin";
$command = "ls $HOME";
print qx($command); #Lists the directory bin
$command = 'ls $HOME';
print qx($command); #List user's home directory
In both of these examples, I'm doing qx($command). However, in the first example, I allow Perl to substitute the value of $HOME. In the second example, I use single quotes, so Perl doesn't substitute the value of$HOME. Instead, the string$HOME` is just part of my command I'm using. I'm letting the shell interpolates it.
I usualy am leery of any program that uses qx/.../. In most cases, it's used to run a command that could be done in Perl itself. For example, in early Perl programs, you'd see things like this:
$date = `date +%M/%Y/%D`
chop $date; #Yeah, I said EARLY Perl programs
Because it was simply a lot easier to run the Unix command rather than trying to do it in a pure Perl way. However, doing it the Perl (i.e. the correct) way means you're no longer dependent upon the OS's behavior which is not entirely under your control.
If you need the output of the command for use in your Perl script, you should use open to execute your command, and treat the output of the command as a file.
my $command = "ls $HOME";
open my command_fh, "|-", $command or die qq(Couldn't execute "$command");
If you simply need to execute the command, use the system command:
my $command = "xyz $date";
my $error = system $command;
if ( $error ) {
#Something terrible happened...
}
Note that if you send only a single scalar argument to the system command, and it contains possible shell meta characters, it will execute the command via the OS shell. If you send the system command a list to execute, or there are no shell metacharacters, Perl will call the command executor to execute the command directly without any shell interpolations:
my #command = qw(ls $HOME);
system #command; #Will print out "No such directory '$HOME'
Perldoc has the answers, as ever.
Specifically, http://perldoc.perl.org/perlop.html#Quote-and-Quote-like-Operators has "Yes" in the Interpolates? column (with a proviso you don't need to worry about here), so yep, it's safe to assume your variable will be interpolated.
You can get the same document by running 'perldoc perlop' at your local commandline, often.
I know I can add watches for variables as well as expressions with the w command, is it possible to tell it to simply parse each command and stop if a certain piece of text is in that command?
Something like w m/bad command/ where $_ is a string containing the next command that is being executed.
I'm working with Komodo on Windows doing remote CGI debugging right now, but I can also use perl -d on a linux shell.
Update:
I'll try to make this clearer. There's a file with data on disk being deleted, and I don't know where. The program I'm debugging consists of a lot of .pl files being required into each other, and the complete codebase is even larger. I could of course grep through this for either the part of the data files' name (I know the extension), or for unlink. But since I don't know which code files are being used by my program, that does not get me very far.
Let's consider this piece of code, which is going to be debugged:
if ($foo == 1) {
unlink 'filename.example';
}
Now when I'm debugging this, I'd like the debugger to stop if the next perl command (from the script) which is about to be executed contains /example/. I want kind of a hook that is run before the execution of the next command.
That way, I don't need to know where (as in which line or file) the thing I'd like to break at is, nor do I need a variable name (as there is none).
I know this is weird to explain, but I thought it might be a common problem. Maybe there's another approach altogether?
Watch points are set with the 'w' command:
BD<1> w $cannonballs
will break the program every time $cannonballs changes.
BD<1> w $i == 19
will break when the value at $i is set to 19. For example
$ cat foo.pl
foreach $i ( 0..100 ) {
print "$i\n";
}
$ perl -d foo.pl
Loading DB routines from perl5db.pl version 1.28 Editor support available.
Enter h or `h h' for help, or `man perldebug' for more help.
main::(foo.pl:2): foreach $i ( 0..100 ) {
DB<1> w $i == 3
DB<2> c
0
1
2
Watchpoint 0: $i == 3 changed:
old value: ''
new value: '1'
main::(foo.pl:3): print "$i\n";
DB<2> p $i
3
DB<3> q
Watchpoint 0: $i == 3 changed:
old value: '1'
new value: ''
DB<3> q
$
You'll probably want to use the full namespace of the variable, otherwise you'll get breaks any time any variable with the same name come into or goes out of scope.
I am unaware of a module that does exactly what you want, but Devel::Trace is a very small module (20 lines of code) that prints every line before it is executed.
You could create a subclass of that, and modify the DB::DB subroutine to set $DB::single = 1 if the code matches your desired string.
See brian d foy's post about Devel::Trace for a more thorough description, with examples.
The debugger already can do that (well, almost). It is not called a watch expression, though, it is called a breakpoint.
You can set a breakpoint to a line and condition:
b 33 /pattern/i
You have to specify the line number, though. See Perl debugging.
You can set a break point in code using
$DB::single = 1;
To use a break point in a watch configuration, enclose the break in a conditional that matches what you want to watch.
In your case:
if ( $command =~ /example/ ) {
$DB::single = 1;
}
should serve.
Check out Tie::Trace, which can alert you of the locations that a variable's value is changed. I suppose you could even hack it with $DB::single=1 expressions to break in the debugger every time a variable's value changes.
I have a bash script which produces different integer values. When I run that script, the output looks like this:
12
34
34
67
6
This script runs on a Solaris server. In order to provide other users in the network with these values, I decided to write a Perl script which can:
run the bash file
read its output
build a tiny html page with a table in which the bash values are stored
Thats a hard job for me because I have almost no experience with Perl. I know I can use system to execute unix commands (and bash files) but I cannot get the output. I also heared about qx which sounds very useful for my case.
But I must admit I have no clue how do start... Could you give me a few hints how to solve that?
With a question like this it's a little hard to know where to begin.
The qx to which you are referring is a feature of Perl. The "q*" or "Quote and Quote-like Operators" are documented in the Perl "operators" man page (normally you'd use man perlop to read that on systems with a conventional installation of Perl).
Specifically qx is the "quoted-execution of a command" ... which is essentially an alternative form of the ` (back tick or "command substitution") operator in Perl.
In other words if you execute a command like:
perl -e '$foo = qx{ls}; print "\n###\n$foo\n###\n";'
... on a system with Perl installed then it should run Perl, which should evaluate (-e) the expression you've provided (quoted). In other words we're writing a small program right on the command line. This program starts by creating a variable whose contents will be a "scalar" (which is Perl terminology for a string or number). We're assigning (the =, or assignment, operator) the output which is captured by executing the ls command back to this variable ($foo). After that we're printing the contents of our variable (whatever the ls command would have printed) with ### lines preceding and following those contents..
A quirk of Perl's qx operator (and the various other q* operators) is that it allows you to delimit the command with just about any characters you like. For example perl -e '$bar = qx/pwd/;' would capture the output of the pwd command. When you use any of the characters that are normally used as delimiters around text parentheses, braces, brackets, etc) then the qx command will look for the appropriate matching delimiter. If you use any other punctuation (or non-alpha-numeric character?) then that same character will be the terminating delimiter as well. This later behavior is similar to, and was inspired by, a feature in "substitution" command from the old sed utility and ed line editors; while the matching of parentheses, braces, etc. are a Perl novelty.
So that's the basics of how to capture your shell script's output. To print the numbers in an HTML table you'd have to split the captured output into separate lines (saving them into a list or array) then print your HTML prologue (the <table> and <th> (header) tags, and so on) ... them loop over a series of <tr> rows, interpolating your numbers into <td>> (table data) containers) and then finally print your HTML epilogue (with the closing tags).
For that you'll want to read up on the Perl print function and about "interpolation" in Perl. That's a fairly complex topic.
This is all extremely crude and there are tools around which allow you to approach the generation of HTML at a much higher level. It's also rather dubious that you want to wrap the execution of your shell script in a Perl script since it seems likely that you could modify the shell script to directly output HTML (perhaps as an option controlled by a command line switch or environment variable) or that you could re-write the shell script in Perl. This could potentially eliminate the extra work of parsing the output (splitting it into lines and separating the values out of those lines into an array because you can capture the data directly into the array (or possibly print out your HTML rows) directly as you are generating them (however your existing shell script is doing that).
To capture the output of your bash file, you can use the backtick operator:
use strict;
my $e = `ls`;
print $e;
Many, many thanks to you! With your great help. I was able to build a perl script which does a big part of the job.
This is what I have created so far:
#!/usr/bin/perl -w
use strict;
use CGI qw(:standard);
#some variables
my $message = "please wait, loading data...\n";
#First build the web page
print header;
print start_html('Hello World');
print "<H1>we need love, peace and harmony</H1>\n";
print "<p>$message</p>\n";
#Establish a pipeline between the bash and my script.
my $bash_command = '/love/peace/harmony/./lovepeace.bash';
open(my $pipe, '-|', $bash_command) or die $!;
while (my $line = <$pipe>){
# Do something with each line.
print "<p>$line</p>\n";
}
#job done, now refresh page?
print end_html;
When I call that .pl script in my browser, everything works nice :-) But a few questions are still on my mind:
When I call this website, it is busy loading the values from the pipe. Since there are about 10 Values its rather
quick (2-4 seconds) But if I have 100+ Values the user has to wait a while. Since I cannot have a progress bar, I
should give an information to the user. Like:
"Loading data, please wait..."
And when the job is done, this message should say: "Job done" or something similar.
But how do I realize if the process is finnished?
can I reload the page if the job is done ?
Is there any chance of using my own stylesheet wihtin this perl-CGI
Regards,
JJ
Why only perl:
you can use awk for that in side your shell script itself.
I have done this earlier.
if you have the out put values in a variable then use the below method:
echo $SUBSCRIBERS|awk 'BEGIN {
print "<?xml version=\"1.0\" encoding=\"UTF-8\"?><GenTransactionHandler xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"><EntityToPublish>\n<Entitytype=\"C\" typeDesc=\"Subscriber level\"><TargetApplCode>UHUNLD</TargetApplCode><TrxName>GET_SUBSCR_DATA</TrxName>"
}
{for(i=1;i<NF+1;i++) printf("<value>%d</value>\n",$i)}
END{
print "</Entity>\n</EntityToPublish></GenTransactionHandler>"}' >XML_SUB_NUM`date +%Y%m%d%H%M%S`.xml
in $SUBSCRIBERS the values should eb tab separated.