I don't know perl at all, but I need to change an exitsting program to give a clearer debug output.
This is the statement:
print $lfh "$ts\t$tid\t$msg\n";
where $msg will be created by joining the arguments of the function like this:
my $msg = join( "\t", #_ );
Somewhere in $msg I would like to add one of the command line arguments the user has supplied when calling the program. Is that risk for an exploit if printed to stdout?
Note also that $lfh will be taken from an environment variable (set by the script itself earlier on) if it is to write to a file, like this:
open my $lfh, ">>", $ENV{GL_LOGFILE}
The info I could find on perl security says nothing about the print statement, so maybe I'm just completely paranoid, but better safe than sorry...
You can pass arbitrary data to print, and it won't break. However:
printing to a file handle may go through various IO layers like encodings. Not all data may be valid for these layers. E.g. encoding layers will either use a substitution character or terminate the script when handed invalid data.
Perl's print doesn't care about control characters like NULs, backspaces, or carriage returns and just passes them through. Depending on how the output is viewed, it might look jumbled or could break the viewing application. Example: User input is equal to the string "\rHAHA". When you print "something$userinput", you might see HAHAthing when displayed on the console, because the carriage returns places the cursor in the 1st column. If the user input contains newlines, your output records could be broken up, e.g. "foo${userinput}bar" might become the two lines foo and bar.
If the input is condidential information, the output will be confidential as well, so you should make sure that the output can't be viewed by anyone. E.g. displaying the output including debug information on a web page could allow attackers to obtain more information about your system.
Related
I am wondering why the perl creators chose an unusual syntax for printing to a filehandle:
print filehandle list
with no comma after filehandle. I see that it's to distinguish between "print list" and "print filehandle list", but why was the ad-hoc syntax preferred over creating two functions - one to print to stdout and one to print to given filehandle?
In my searches, I came across the explanation that this is an indirect object syntax, but didn't the print function exist in perl 4 and before, whereas the object-oriented features came into perl relatively late? Is anyone familiar with the history of print in perl?
Since the comma is already used as the list constructor, you can't use it to separate semantically different arguments to print.
open my $fh, ...;
print $fh, $foo, $bar
would just look like you were trying to print the values of 3 variables. There's no way for the parser, which operates at compile time, to tell that $fh is going to refer to a file handle at run time. So you need a different character to syntactically (not semantically) distinguish between the optional file handle and the values to actually print to that file handle.
At this point, it's no more work for the parser to recognize that the first argument is separated from the second argument by blank space than it would be if it were separated by any other character.
If Perl had used the comma to make print look more like a function, the filehandle would always have to be included if you are including anything to print besides $_. That is the way functions work: If you pass in a second parameter, the first parameter must also be included. There isn't one function I can think of in Perl where the first parameter is optional when the second parameter exists. Take a look at split. It can be written using zero to four parameters. However, if you want to specify a <limit>, you have to specify the first three parameters too.
If you look at other languages, they all include two different ways ways to print: One if you want STDOUT, and another if you're printing to something besides STDOUT. Thus, Python has both print and write. C has both printf and fprintf. However, Perl can do this with just a single statement.
Let's look at the print statement a bit more closely -- thinking back to 1987 when Perl was first written.
You can think of the print syntax as really being:
print <filehandle> <list_to_print>
To print to OUTFILE, you would say:
To print to this file, you would say:
print OUTFILE "This is being printed to myfile.txt\n";
The syntax is almost English like (PRINT to OUTFILE the string "This is being printed to myfile.txt\n"
You can also do the same with thing with STDOUT:
print STDOUT "This is being printed to your console";
print STDOUT " unless you redirected the output.\n";
As a shortcut, if the filehandle was not given, it would print to STDOUT or whatever filehandle the select was set to.
print "This is being printed to your console";
print " unless you redirected the output.\n";
select OUTFILE;
print "This is being printed to whatever the filehandle OUTFILE is pointing to\n";
Now, we see the thinking behind this syntax.
Imagine I have a program that normally prints to the console. However, my boss now wants some of that output printed to various files when required instead of STDOUT. In Perl, I could easily add a few select statements, and my problems will be solved. In Python, Java, or C, I would have to modify each of my print statements, and either have some logic to use a file write to STDOUT (which may involve some conniptions in file opening and dupping to STDOUT.
Remember that Perl wasn't written to be a full fledge language. It was written to do the quick and dirty job of parsing text files more easily and flexibly than awk did. Over the years, people used it because of its flexibility and new concepts were added on top of the old ones. For example, before Perl 5, there was no such things as references which meant there was no such thing as object oriented programming. If we, back in the days of Perl 3 or Perl 4 needed something more complex than the simple list, hash, scalar variable, we had to munge it ourselves. It's not like complex data structures were unheard of. C had struct since its initial beginnings. Heck, even Pascal had the concept with records back in 1969 when people thought bellbottoms were cool. (We plead insanity. We were all on drugs.) However, since neither Bourne shell nor awk had complex data structures, so why would Perl need them?
Answer to "why" is probably subjective and something close to "Larry liked it".
Do note however, that indirect object notation is not a feature of print, but a general notation that can be used with any object or class and method. For example with LWP::UserAgent.
use strict;
use warnings;
use LWP::UserAgent;
my $ua = new LWP::UserAgent;
my $response = get $ua "http://www.google.com";
my $response_content = decoded_content $response;
print $response_content;
Any time you write method object, it means exactly the same as object->method. Note also that parser seems to only reliably work as long as you don't nest such notations or do not use complex expressions to get object, so unless you want to have lots of fun with brackets and quoting, I'd recommend against using it anywhere except common cases of print, close and rest of IO methods.
Why not? it's concise and it works, in perl's DWIM spirit.
Most likely it's that way because Larry Wall liked it that way.
I am somewhat a beginner at Perl (compared to people here). I know enough to be able to write programs to do many things with through the command prompt. At one point, I decided to write a command prompt game that constructed a maze and let me solve it. Besides quality graphics, the only thing that it was missing was the ability for me to use the WASD controls without pressing enter after every move I made in the maze.
To make my game work, I want to be able to have Perl take a single character as input from STDIN, without requiring me to use something to separate my input, like the default \n. How would I accomplish this?
I have tried searching for a simple answer online and in a book that I have, but I didn't seem to find anything. I tried setting $/="", but that seemed to bypass all input. I think that there may be a really simple answer to my question, but I am also afraid that it might be impossible.
Also, does $/="" actually bypass input, or does it take input so quickly that it assumes there isn't any input if I'm not already pressing the key?
IO::Prompt can be used:
#!/usr/bin/env perl
use strict;
use warnings;
use IO::Prompt;
my $key = prompt '', -1;
print "\nPressed key: $key\n";
Relevant excerpt from perldoc -v '$/' related to setting $/ = '':
The input record separator, newline by default. This influences Perl's
idea of what a "line" is. Works like awk's RS variable, including
treating empty lines as a terminator if set to the null string (an empty line cannot contain any spaces or tabs).
The shortest way to achieve your goal is to use this special construct:
$/ = \1;
This tells perl to read one character at a time. The next time you read from any stream (not just STDIN)
my $char = <STREAM>;
it will read 1 character per assignment. From perlvar "Setting $/ to a reference to an integer, scalar containing an integer, or scalar that's convertible to an integer will attempt to read records instead of lines, with the maximum record size being the referenced integer number of characters."
If you are using *nix, you will find Curses useful.
It has a getch method that does what you want.
Term::TermKey also looks like a potential solution.
IO::Prompt is no longer maintained but IO::Prompter
has a nice example (quoted from that site):
use IO::Prompter;
# This call has no automatically added options...
my $assent = prompt "Do you wish to take the test?", -yn;
{
use IO::Prompter [-yesno, -single, -style=>'bold'];
# These three calls all have: -yesno, -single, -style=>'bold' options
my $ready = prompt 'Are you ready to begin?';
my $prev = prompt 'Have you taken this test before?';
my $hints = prompt 'Do you want hints as we go?';
}
# This call has no automatically added options...
scalar prompt 'Type any key to start...', -single;
I have a bash script which produces different integer values. When I run that script, the output looks like this:
12
34
34
67
6
This script runs on a Solaris server. In order to provide other users in the network with these values, I decided to write a Perl script which can:
run the bash file
read its output
build a tiny html page with a table in which the bash values are stored
Thats a hard job for me because I have almost no experience with Perl. I know I can use system to execute unix commands (and bash files) but I cannot get the output. I also heared about qx which sounds very useful for my case.
But I must admit I have no clue how do start... Could you give me a few hints how to solve that?
With a question like this it's a little hard to know where to begin.
The qx to which you are referring is a feature of Perl. The "q*" or "Quote and Quote-like Operators" are documented in the Perl "operators" man page (normally you'd use man perlop to read that on systems with a conventional installation of Perl).
Specifically qx is the "quoted-execution of a command" ... which is essentially an alternative form of the ` (back tick or "command substitution") operator in Perl.
In other words if you execute a command like:
perl -e '$foo = qx{ls}; print "\n###\n$foo\n###\n";'
... on a system with Perl installed then it should run Perl, which should evaluate (-e) the expression you've provided (quoted). In other words we're writing a small program right on the command line. This program starts by creating a variable whose contents will be a "scalar" (which is Perl terminology for a string or number). We're assigning (the =, or assignment, operator) the output which is captured by executing the ls command back to this variable ($foo). After that we're printing the contents of our variable (whatever the ls command would have printed) with ### lines preceding and following those contents..
A quirk of Perl's qx operator (and the various other q* operators) is that it allows you to delimit the command with just about any characters you like. For example perl -e '$bar = qx/pwd/;' would capture the output of the pwd command. When you use any of the characters that are normally used as delimiters around text parentheses, braces, brackets, etc) then the qx command will look for the appropriate matching delimiter. If you use any other punctuation (or non-alpha-numeric character?) then that same character will be the terminating delimiter as well. This later behavior is similar to, and was inspired by, a feature in "substitution" command from the old sed utility and ed line editors; while the matching of parentheses, braces, etc. are a Perl novelty.
So that's the basics of how to capture your shell script's output. To print the numbers in an HTML table you'd have to split the captured output into separate lines (saving them into a list or array) then print your HTML prologue (the <table> and <th> (header) tags, and so on) ... them loop over a series of <tr> rows, interpolating your numbers into <td>> (table data) containers) and then finally print your HTML epilogue (with the closing tags).
For that you'll want to read up on the Perl print function and about "interpolation" in Perl. That's a fairly complex topic.
This is all extremely crude and there are tools around which allow you to approach the generation of HTML at a much higher level. It's also rather dubious that you want to wrap the execution of your shell script in a Perl script since it seems likely that you could modify the shell script to directly output HTML (perhaps as an option controlled by a command line switch or environment variable) or that you could re-write the shell script in Perl. This could potentially eliminate the extra work of parsing the output (splitting it into lines and separating the values out of those lines into an array because you can capture the data directly into the array (or possibly print out your HTML rows) directly as you are generating them (however your existing shell script is doing that).
To capture the output of your bash file, you can use the backtick operator:
use strict;
my $e = `ls`;
print $e;
Many, many thanks to you! With your great help. I was able to build a perl script which does a big part of the job.
This is what I have created so far:
#!/usr/bin/perl -w
use strict;
use CGI qw(:standard);
#some variables
my $message = "please wait, loading data...\n";
#First build the web page
print header;
print start_html('Hello World');
print "<H1>we need love, peace and harmony</H1>\n";
print "<p>$message</p>\n";
#Establish a pipeline between the bash and my script.
my $bash_command = '/love/peace/harmony/./lovepeace.bash';
open(my $pipe, '-|', $bash_command) or die $!;
while (my $line = <$pipe>){
# Do something with each line.
print "<p>$line</p>\n";
}
#job done, now refresh page?
print end_html;
When I call that .pl script in my browser, everything works nice :-) But a few questions are still on my mind:
When I call this website, it is busy loading the values from the pipe. Since there are about 10 Values its rather
quick (2-4 seconds) But if I have 100+ Values the user has to wait a while. Since I cannot have a progress bar, I
should give an information to the user. Like:
"Loading data, please wait..."
And when the job is done, this message should say: "Job done" or something similar.
But how do I realize if the process is finnished?
can I reload the page if the job is done ?
Is there any chance of using my own stylesheet wihtin this perl-CGI
Regards,
JJ
Why only perl:
you can use awk for that in side your shell script itself.
I have done this earlier.
if you have the out put values in a variable then use the below method:
echo $SUBSCRIBERS|awk 'BEGIN {
print "<?xml version=\"1.0\" encoding=\"UTF-8\"?><GenTransactionHandler xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"><EntityToPublish>\n<Entitytype=\"C\" typeDesc=\"Subscriber level\"><TargetApplCode>UHUNLD</TargetApplCode><TrxName>GET_SUBSCR_DATA</TrxName>"
}
{for(i=1;i<NF+1;i++) printf("<value>%d</value>\n",$i)}
END{
print "</Entity>\n</EntityToPublish></GenTransactionHandler>"}' >XML_SUB_NUM`date +%Y%m%d%H%M%S`.xml
in $SUBSCRIBERS the values should eb tab separated.
I have encountered a weird situation while updating/upgrading some legacy code.
I have a variable which contains HTML. Before I can output it, it has to be filled with lots of data. In essence, I have the following:
for my $line (#lines) {
$output = loadstuff($line, $output);
}
Inside of loadstuff(), there is the following
sub loadstuff {
my ($line, $output) = #_;
# here the process is simplified for better understanding.
my $stuff = getOtherStuff($line);
my $result = $output.$stuff;
return $result;
}
This function builds a page which consists of different areas. All area is loaded up independently, that's why there is a for-loop.
Trouble starts right about here. When I load the page from ground up (click on a link, Perl executes and delivers HTML), everything is loaded fine. Whenever I load a second page via AJAX for comparison, that HTML has broken encoding.
I tracked down the problem to this line my $result = $output.$stuff. Before the concatenation, $output and $stuff are fine. But afterward, the encoding in $result is messed up.
Does somebody have a clue why concatenation messes up my encoding? While we are on the subject, why does it only happen when the call is done via AJAX?
Edit 1
The Perl and the AJAX call both execute the very same functions for building up a page. So, whenever I fix it for AJAX, it is broken for freshly reloaded pages. It really seems to happen only if AJAX starts the call.
The only difference in this particular case is that the current values for the page are compared with an older one (it is a backup/restore function). From here, everything is the same. The encoding in the variables (as far as I can tell) are ok. I even tried the Encode functions only on the values loaded from AJAX, but to no avail. The files themselves seem to be utf8 according to "Kate".
Besides that, I have a another function with the same behavior which uses the EXACT same functions, values and files. When the call is started from Perl/Apache, the encoding is ok. Via AJAX, again, it is messed up.
I have been examinating the AJAX Request (jQuery) and could not find anything odd. The encoding seems to be utf8 too.
Perl has a “utf8” flag for every scalar value, which may be “on” or “off”. “On” state of the flag tells perl to treat the value as a string of Unicode characters.
If you take a string with utf8 flag off and concatenate it with a string that has utf8 flag on, perl converts the first one to Unicode. This is the usual source of problems.
You need to either convert both variables to bytes with Encode::encode() or to perl's internal format with Encode::decode() before concatenation.
See perldoc Encode.
Expanding on the previous answer, here's a little more information that I found useful when I started messing with character encodings in Perl.
This is an excellent introduction to Unicode in perl: http://perldoc.perl.org/perluniintro.html. The section "Perl's Unicode Model" is particularly relevant to the issue you're seeing.
A good rule to use in Perl is to decode data to Perl characters on it's way in and encode it into bytes on it's way out. You can do this explicitly using Encode::encode and Encode::decode. If you're reading from/writing to a file handle you can specify an encoding on the filehandle by using binmode and setting layer: perldoc -f binmode
You can tell which of the strings in your example has been decoded into Perl characters using Encode::is_utf8:
use Encode qw( is_utf8 );
print is_utf8($stuff) ? 'characters' : 'bytes';
A colleague of mine found the answer to this problem. It really had something to do with the fact that AJAX started the call.
The file structure is as follows:
1 Handler, accessed by Apache
1 Handler, accessed by Apache but who only contains AJAX responders. We call it the AJAX-Handler
1 package, which contains functions relevant for the entire software, who access yet other packages from our own Framework
Inside of the AJAX-Handler, we print the result as such
sub handler {
my $r = shift;
# processing output
$r->print($output);
return Apache2::Const::OK;
}
Now, when I replace $r->print($output); by print($output);, the problem disappears! I know that this is not the recommended way to print stuff in mod_perl, but this seems to work.
Still, any ideas how to do this the proper way are welcome.
I can setup a telnet connection in Perl no problems, and have just discovered Curses, and am wondering if I can use the two together to scrape the output from the telnet session.
I can view on a row, column basis the contents of STDOUT using the simple script below:
use Curses;
my $win = new Curses;
$win->addstr(10, 10, 'foo');
$win->refresh;
my $thischar=$win->inch(10,10);
print "Char $thischar\n";
And using the below I can open a telnet connection and send \ receive commands with no problem:
use net::telnet;
my $telnet = new Net::Telnet (Timeout => 9999,);
$telnet->open($ipaddress) or die "telnet open failed\n";
$telnet->login($user,$pass);
my $output = $telnet->cmd("command string");
... But what I would really like to do is get the telnet response (which will include terminal control characters) and then search on a row \ column basis using curses. Does anyone know of a way I can connect the two together? It seems to me that curses can only operate on STDOUT
Curses does the opposite. It is a C library for optimising screen updates from a program writing to a terminal, originally designed to be used over a slow serial connection. It has no ability to scrape a layout from a sequence of control characters.
A better bet would be a terminal emulator that has an API with the ability to do this type of screen scraping. Off the top of my head I'm not sure if any Open-source terminal emulators do this, but there are certainly commercial ones available that can.
If you are interacting purely with plain-text commands and responses, you can use Expect to script that, otherwise, you can use Term::VT102, which lets you screen scrape (read specific parts of the screen, send text, handle events on scrolling, cursor movement, screen content changes, and others) applications using VT102 escape sequences for screen control (e.g., an application using the curses library).
You probably want something like Expect
use strict;
use warnings;
use Expect;
my $exp = Expect->spawn("telnet google.com 80");
$exp->expect(15, #timeout
[
qr/^Escape character.*$/,
sub {
$exp->send("GET / HTTP/1.0\n\n");
exp_continue;
}
]
);
You're looking for Term::VT102, which emulates a VT102 terminal (converting the terminal control characters back into a virtual screen state). There's an example showing how to use it with Net::Telnet in VT102/examples/telnet-usage.pl (the examples directory is inside the VT102 directory for some reason).
It's been about 7 years since I used this (the system I was automating switched to a web-based interface), but it used to work.
Or you could use the script command for this.
From the Solaris man-page:
DESCRIPTION
The script utility makes a record of everything printed
on your screen. The record is written to filename. If no file name
is given, the record is saved in the file typescript...
The script command forks and creates a
sub-shell, according to the value of
$SHELL, and records the text from this
session. The script ends when the
forked shell exits or when
Control-d is typed.
I would vote also for the Expect answer. I had to do something similar from a gui'ish application. The trick (albeit tedious) to get around the control characters was to strip all the misc characters from the returned strings. It kind of depends on how messy the screen scrape ends up being.
Here is my function from that script as an example:
# Trim out the curses crap
sub trim {
my #out = #_;
for (#out) {
s/\x1b7//g;
s/\x1b8//g;
s/\x1b//g; # remove escapes
s/\W\w\W//g;
s/\[\d\d\;\d\dH//g; #
s/\[\?25h//g;
s/\[\?25l//g;
s/\[\dm//g;
s/qq//g;
s/Recall//g;
s/\357//g;
s/[^0-9:a-zA-Z-\s,\"]/ /g;
s/\s+/ /g; # Extra spaces
}
return wantarray ? #out : $out[0];
}