I am having a strange problem, maybe something to do with Apache more than Perl. In my code when I write:
print "content-type: text/html\n\n";
it prints that along with the code. But when I type:
$c = new CGI; $c->header();
it works fine and displays the HTML rendered output.
What could the problem be?
Thank you
Is the "content-type: text/html" the very first thing that's output? Use wget or similar to verify the actual output; don't trust your eyes or the browser's view source.
Also note that under mod_perl, CGI calls the request's send_cgi_header method instead of just printing the headers.
Is it possible you are using CGI's header() in more than one place? That's harmless, but replacing just one header() call with explicitly printing the header would give the results you see.
Keep in mind, HTTP RFC specifies that
\r\n
is used for line delimiters, not
\n
so you want to be emitting
print "Content-Type: text/html\r\n\r\n";
Instead.
You should just use ->headers though to do this for you.
Also, note I used Camel-Case instead of lower-case. Although both should work, the Camel-Case is the notation used in the spec, so that is preferred and more likely to work on weird UA's.
Related
I have the following cgi bin script:
#! /usr/bin/perl
#
use utf8;
use CGI;
my $q = CGI->new();
my %params = $q->Vars;
print $q->header('text/html');
$w = $params{"words"};
print "$w\n";
I want to be able to call it as cgi-bin/script.pl?words=É for example, but when I do that, what's printed is not UTF-8, but instead garbled:
É
Is there any way to use cgi-bin with utf8?
Your line use utf8 doesn't do anything for you, other than allowing UTF-8 characters in the source file itself. You must make sure that the output handles (on STDOUT as well as any files) are set to utf8. One easy way to handle this is the utf8::all module. Also, make sure you are sending the correct headers, and use the -utf8 CGI pragma to treat incoming parameters as UTF-8. Finally, as always, be sure to use strict and warnings.
The following should get you started:
#!/usr/bin/perl
use strict;
use warnings;
use utf8::all;
use CGI qw(-utf8);
my $q = CGI->new;
print $q->header("text/html;charset=UTF-8");
print $q->param("words");
exit;
I have been having this problem of intermittent failure of utf8 encoding with my CGI script.
I tried everything but couldn't reliably repeat the problem.
I finally discovered that is is absolutely critical to be consistent with you use of the utf8 pragma throughout every module that uses CGI
use CGI qw(-utf8);
What seems to happen is that modperl invokes the CGI module just once per requests. If there is inconsistent including of the CGI module - say for some utility function that is just using a redirect function and you haven't bothered to set the utf8 pragma. Then this invocation can be the one that modperl decides to use to decode requests.
You will save yourself a lot of pain in the long run if you start out by reading the perlunitut and perlunicode documentation pages. They will give you the basics on exactly what Unicode and character encodings are, and how to work with them in Perl.
Also, what you're asking for is more complex than you think. There are many layers hidden in the phrase "use cgi-bin with utf8", starting with your interface to whatever tool you're using to send requests to the web server and ending with that tool having parsed a response and presenting it to you. You need to understand all those layers well enough to at least be able to tell if the problem lies in your CGI script or not. For example, it doesn't help if your script works perfectly if the problem is that bash and curl don't agree on the encoding of your command line arguments.
When I am trying to execute a shell script through perl-CGI it is providing me the result but all in one line. But when I run the shell script it is showing the result in proper format. Please guide me how to make it proper.
Below is the perl code
#!/usr/bin/perl
use CGI qw(:standard);
exec("sh /tmp/att.sh -t v2 --help");
print end_html
Please help me in resolving it.
Add this before your exec:
print "Content-type: text/plain\n\n";
You can safely remove the print end_html; since it is never executed anyway.
In fact, anything after your exec is never executed. If you want to execute more Perl after your shell call look at system or some other method (see this link).
If you want to embed your output in HTML you have to get a little more sophisticated and use backticks or open a pipe since you will want to capture the output and HTML escape before print.
use CGI qw/escapeHTML/;
print "Content-type: text/html\n\n";
printf "<pre>%s</pre>\n", escapeHTML(`sh /tmp/att.sh -t v2 --help`);
EDIT (per request by #ysth): what is <pre>
Per W3C documentation on pre
The PRE element tells visual user agents that the enclosed text is "preformatted". When handling preformatted text, visual user agents:
May leave white space intact.
May render text with a fixed-pitch font.
May disable automatic word wrap.
Must not disable bidirectional processing.
in the html file:
<!--#exec cgi="/cgi-bin/test.pl"-->
the perl script:
#!/usr/bin/perl
print "Content-Type: text/html\n\n";
print "<input type=\"hidden\" name=\"aname\" value=\"avalue\">\n";
print "<img src=\"/cgi-bin/script.pl\" />";
This does not give me an 'error processing directive' error, nor does it output my HTML inplace of the tag. I'll also add that the ssi tag gets replaced with nothing.
Are you sure the script is executing? If you print something to STDERR does it show up in th error log?
Beyond that I have a few comments:
I'm pretty sure printing the Content-Type is redundant, you (well, Apache anyway) have already done that by serving the HTML file that contains the SSI.
reference
exec is really meant for running commands like 'ls -l'. You should use include virtual instead. It also allows you to add arguments to the url. e.g.
<!--#include virtual="/cgi-bin/example.cgi?argument=value" --\>
do yourself a favor and use qq[] instead of the double-quotes. You won't have to escape everything then... e.g.
print qq[< input type="hidden" name="aname" value="avalue"\b];
I have encountered a weird situation while updating/upgrading some legacy code.
I have a variable which contains HTML. Before I can output it, it has to be filled with lots of data. In essence, I have the following:
for my $line (#lines) {
$output = loadstuff($line, $output);
}
Inside of loadstuff(), there is the following
sub loadstuff {
my ($line, $output) = #_;
# here the process is simplified for better understanding.
my $stuff = getOtherStuff($line);
my $result = $output.$stuff;
return $result;
}
This function builds a page which consists of different areas. All area is loaded up independently, that's why there is a for-loop.
Trouble starts right about here. When I load the page from ground up (click on a link, Perl executes and delivers HTML), everything is loaded fine. Whenever I load a second page via AJAX for comparison, that HTML has broken encoding.
I tracked down the problem to this line my $result = $output.$stuff. Before the concatenation, $output and $stuff are fine. But afterward, the encoding in $result is messed up.
Does somebody have a clue why concatenation messes up my encoding? While we are on the subject, why does it only happen when the call is done via AJAX?
Edit 1
The Perl and the AJAX call both execute the very same functions for building up a page. So, whenever I fix it for AJAX, it is broken for freshly reloaded pages. It really seems to happen only if AJAX starts the call.
The only difference in this particular case is that the current values for the page are compared with an older one (it is a backup/restore function). From here, everything is the same. The encoding in the variables (as far as I can tell) are ok. I even tried the Encode functions only on the values loaded from AJAX, but to no avail. The files themselves seem to be utf8 according to "Kate".
Besides that, I have a another function with the same behavior which uses the EXACT same functions, values and files. When the call is started from Perl/Apache, the encoding is ok. Via AJAX, again, it is messed up.
I have been examinating the AJAX Request (jQuery) and could not find anything odd. The encoding seems to be utf8 too.
Perl has a “utf8” flag for every scalar value, which may be “on” or “off”. “On” state of the flag tells perl to treat the value as a string of Unicode characters.
If you take a string with utf8 flag off and concatenate it with a string that has utf8 flag on, perl converts the first one to Unicode. This is the usual source of problems.
You need to either convert both variables to bytes with Encode::encode() or to perl's internal format with Encode::decode() before concatenation.
See perldoc Encode.
Expanding on the previous answer, here's a little more information that I found useful when I started messing with character encodings in Perl.
This is an excellent introduction to Unicode in perl: http://perldoc.perl.org/perluniintro.html. The section "Perl's Unicode Model" is particularly relevant to the issue you're seeing.
A good rule to use in Perl is to decode data to Perl characters on it's way in and encode it into bytes on it's way out. You can do this explicitly using Encode::encode and Encode::decode. If you're reading from/writing to a file handle you can specify an encoding on the filehandle by using binmode and setting layer: perldoc -f binmode
You can tell which of the strings in your example has been decoded into Perl characters using Encode::is_utf8:
use Encode qw( is_utf8 );
print is_utf8($stuff) ? 'characters' : 'bytes';
A colleague of mine found the answer to this problem. It really had something to do with the fact that AJAX started the call.
The file structure is as follows:
1 Handler, accessed by Apache
1 Handler, accessed by Apache but who only contains AJAX responders. We call it the AJAX-Handler
1 package, which contains functions relevant for the entire software, who access yet other packages from our own Framework
Inside of the AJAX-Handler, we print the result as such
sub handler {
my $r = shift;
# processing output
$r->print($output);
return Apache2::Const::OK;
}
Now, when I replace $r->print($output); by print($output);, the problem disappears! I know that this is not the recommended way to print stuff in mod_perl, but this seems to work.
Still, any ideas how to do this the proper way are welcome.
I'm attempting to run a CGI script in the current environment from another Perl module. All works well using standard systems calls for GET requests. POST is fine too, until the parameter list gets too long, then they get cut off.
Has anyone ran in to this problem, or have any suggestions for other ways to attempt this?
The following are somewhat simplified for clarity. There is more error checking, etc.
For GET requests and POST requests w/o parameters, I do the following:
# $query is a CGI object.
my $perl = $^X;
my $cgi = $cgi_script_location; # /path/file.cgi
system {$perl} $cgi;
Parameters are passed through the
QUERY_STRING environment variable.
STDOUT is captured by the calling
script so whatever the CGI script
prints behaves as normal.
This part works.
For POST requests with parameters the following works, but seemingly limits my available query length:
# $query is a CGI object.
my $perl = $^X;
my $cgi = $cgi_script_location; # /path/file.cgi
# Gather parameters into a URL-escaped string suitable
# to pass to a CGI script ran from the command line.
# Null characters are handled properly.
# e.g., param1=This%20is%20a%20string¶m2=42&... etc.
# This works.
my $param_string = $self->get_current_param_string();
# Various ways to do this, but system() doesn't pass any
# parameters (different question).
# Using qx// and printing the return value works as well.
open(my $cgi_pipe, "|$perl $cgi");
print {$cgi_pipe} $param_string;
close($cgi_pipe);
This method works for short parameter lists, but if the entire command gets to be close to 1000 characters, the parameter list is cut short. This is why I attempted to save the parameters to a file; to avoid shell limitations.
If I dump the parameter list from the executed CGI script I get something like the following:
param1=blah
... a bunch of other parameters ...
paramN=whatever
p <-- cut off after 'p'. There are more parameters.
Other things I've done that didn't help or work
Followed the CGI troubleshooting guide
Saved the parameters to a file using CGI->save(), passing that file to the CGI script. Only the first parameter is read using this method.
$> perl index.cgi < temp-param-file
Saved $param_string to a file, passing that file to the CGI script just like above. Same limitations as passing the commands through the command line; still gets cut off.
Made sure $CGI::POST_MAX is acceptably high (it's -1).
Made sure the CGI's command-line processing was working. (:no_debug is not set)
Ran the CGI from the command line with the same parameters. This works.
Leads
Obviously, this seems like a character limit of the shell Perl is using to execute the command, but it wasn't resolved by passing the parameters through a file.
Passign parameters to system as a single string, from HTTP input, is extremely dangerous.
From perldoc -f system,
If there is only one scalar argument, the argument is checked for shell metacharacters, and if there are any, the entire argument is passed to the system's command shell for parsing (this is /bin/sh -c on Unix platforms, but varies on other platforms). If there are no shell metacharacters in the argument,..
In other words, if I pass in arguments -e printf("working..."); rm -rf /; I can delete information from your disk (everything if your web server is running as root). If you choose to do this, make sure you call system("perl", #cgi) instead.
The argument length issue you're running into may be an OS limitation (described at http://www.in-ulm.de/~mascheck/various/argmax/):
There are different ways to learn the upper limit:
command: getconf ARG_MAX
system header: ARG_MAX in e.g. <[sys/]limits.h>
Saving to a temp file is risky: multiple calls to the CGI might save to the same file, creating a race condition where one user's parameters might be used by another user's process.
You might try opening a file handle to the process and passing arguments as standard input, instead. open my $perl, '|', 'perl' or die; fprintf(PERL, #cgi);
I didn't want to do this, but I've gone with the most direct approach and it works. I'm tricking the environment to think the request method is GET so that the called CGI script will read its input from the QUERY_STRING environment variable it expects. Like so:
$ENV{'QUERY_STRING'} = $long_parameter_string . '&' . $ENV{'QUERY_STRING'};
$ENV{'REQUEST_METHOD'} = 'GET';
system {$perl_exec} $cgi_script;
I'm worried about potential problems this may cause, but I can't think of what this would harm, and it works well so far. But, because I'm worried, I thought I'd ask the horde if they saw any potential problems:
Are there any problems handling a POST request as a GET request on the server
I'll save marking this as the official answer until people have confirmed or at least debated it on the above post.
Turns out that the problem is actually related to the difference in Content-Length between the original parameters and the parameter string I cobbled together. I didn't realize that the CGI module was using this value from the original headers as the limit to how much input to read (makes sense!). Apparently the extra escaping I was doing was adding some characters.
My solution's trick is simply to piece together the parameter string I'll be passing and modify the environment variable the CGI module will check to determine the content length to be equal to the .
Here's the final working code:
use CGI::Util qw(escape);
my $params;
foreach my $param (sort $query->param) {
my $escaped_param = escape($param);
foreach my $value ($query->param($param)) {
$params .= "$escaped_param=" . escape("$value") . "&";
}
}
foreach (keys %{$query->{'.fieldnames'}}) {
$params .= ".cgifields=" . escape("$_") . "&";
}
# This is the trick.
$ENV{'CONTENT_LENGTH'} = length($params);
open(my $cgi_pipe, "| $perl $cgi_script") || die("Cannot fork CGI: $!");
local $SIG{PIPE} = sub { warn "spooler pipe broke" };
print {$cgi_pipe} $params;
warn("param chars: " . length($params));
close($cgi_pipe) || warn "Error: CGI exited with value $?";
Thanks for all the help!