I have a Perl script I wrote, saved it as index.cgi, and it reads in a table of shortlinks and their expansion from a TSV:
redirects.tsv:
$ head redirects.tsv
about http://tobilehman.com/about
gapminder http://www.gapminder.org/
speed http://speedof.me/
delete http://justdelete.me/
terms http://tosdr.org/
re https://www.debuggex.com/
1 http://www.diffen.com/difference/Diffen_vs_Wikipedia
2 http://arstechnica.com/information-technology/2013/10/google-fiber-now-explicitly-permits-home-servers/
3 https://www.senate.gov/legislative/LIS/roll_call_lists/roll_call_vote_cfm.cfm?congress=113
ifs http://tobilehman.com/blog/2013/10/19/revisiting-spaces-in-file-names/
index.cgi:
#!/usr/bin/perl
print "Content-type: text/html\n\n";
my $current_time = time();
my $file = "redirects.tsv";
open my $lines, $file or die "Could not open redirects.tsv";
my $redirect_html = "";
while ( my $line = <$lines> ) {
$line =~ /^([0-9a-z_-]+)\t(.*)/;
#$redirect_html = "$redirect_html\n<li><a href='$1'>tblh.mn/$1</a> ($2)</li>";
my $link = "http://tblh.mn/$1";
$redirect_html
= "$redirect_html<tr><td><a href='$link'>$link</td><td>→</td><td style='padding-left:15px'>$2</td></tr>\n";
}
print <<HTML;
<html>
<head>
<link href="/favicon.png" rel="icon">
<title>tblh.mn → tobilehman.com</title>
</head>
<body>
<h1>Current Time: $current_time</h1>
<h1>Short Links</h1>
<table>
$redirect_html
</table>
</body>
</html>
HTML
exit;
Right now, the links are being printed as in Last-In-Last-Out, but I'd like them to be printed as Last-In-First-Out.
I tried solving this by using reverse(<$lines>) and <reverse($lines)>, neither of which worked. I am not very familiar with Perl, so forgive me if this is a noob problem.
As has already been pointed out, you can use reverse to reverse an array, or File::ReadBackwards to do what its name implies.
Additionally, I would like to encourage you to do some basic error checking in your script:
Always include use strict; and use warnings; in EVERY script.
Include use autodie any time you're doing file processing.
Make sure that your regular expression matched before using the capture variables.
And stylistically:
Use the concatenation operator to build a string $string .= "more string";
Use alternative delimiters like qq{ } whenever you want to include double quotes in a string.
Including these changes with a couple other small fixes:
#!/usr/bin/perl
use strict;
use warnings;
use autodie;
print "Content-type: text/html\n\n";
my $current_time = time();
my $file = "redirects.tsv";
open my $fh, '<', $file;
my $redirect_html = "";
for ( reverse <$fh> ) {
chomp;
if ( my ( $shortlink, $full ) = /^([0-9a-z_-]+)\t(.*)/ ) {
my $link = "http://tblh.mn/$shortlink";
$redirect_html
.= qq{<tr><td><a href="$link">$link</td><td>→</td><td style="padding-left:15px">$full</td></tr>\n};
#$redirect_html .= "\n<li><a href='$1'>tblh.mn/$1</a> ($2)</li>";
}
}
print <<"HTML";
<html>
<head>
<link href="/favicon.png" rel="icon">
<title>tblh.mn → tobilehman.com</title>
</head>
<body>
<h1>Current Time: $current_time</h1>
<h1>Short Links</h1>
<table>
$redirect_html
</table>
</body>
</html>
HTML
exit;
for my $line (reverse(<$lines>)) {
...
}
Alternatively, File::ReadBackwards.
Related
I have a simple web form in which I want the user to select a folder to upload (I only care about the contents of the folder). So my attempt to make this work is as followed:
#!/usr/bin/perl
use strict;
use warnings;
use CGI;
use Data::Dumper;
my $q = CGI->new();
print $q->header;
my $upload_folder = '/var/www/html/uploads';
my $name = $q->param('name');
my $email = $q->param('email');
my $comments = $q->param('comments');
my #files = $q->param('multi_files');
foreach my $upload(#files){
print "Upload this please -- $upload<br>";
my $upload_file = $q->upload($upload);
if ($upload_file){
open (OUTFILE,">$upload_folder/$upload") or die $!;;
binmode OUTFILE;
while (<$upload_file>) {
print OUTFILE;
}
}
else {
print "<b>Guess it's broken</b><br/>";
}
}
print "<p><b>Name</b> -- $name<br><b>Email</b> -- $email<br><b>Comments</b> -- $comments<br>";
print $q->end_html;
When I run the script all the parameters are correct, the files print out as expected but when I execute the upload query it returns blank. This is my first attempt at using CGI as I tend to use other languages to process forms.
Here is the code from the form just in case :
<html>
<head>
<title>Stupid Test Site</title>
<script src="//ajax.googleapis.com/ajax/libs/jquery/1.11.3/jquery.min.js" ></script>
</head>
<body>
<h1>This is a test</h1>
<form action="/cgi-enabled/test.pl" method="POST">
Your Name: <input type="text" name="name"><br>
Email Address: <input type="text" name="email"><br>
Select PO folder: <input name="multi_files" type="file" webkitdirectory multiple /><br/>
Comments:<br>
<textarea name="comments" rows="5" cols="60"></textarea><br>
<input type="submit" value="Send">
</form>
</body>
</html>
I've also posted this question on perlmonks.com as well.
The main issue with my code was it was missing the
enctype="multipart/form-data"
from the form. I made some changes as well. Here is my working code.
#!/usr/bin/perl
use strict;
use warnings;
use CGI;
use Data::Dumper;
use CGI::Carp qw( fatalsToBrowser );
use HTML::Entities qw/encode_entities/;
use File::Copy qw' copy ';
use File::Basename;
my $q = new CGI;
print $q->header;
my $upload_folder = '/var/www/html/uploads';
my $name = $q->param('name');
my $email = $q->param('email');
my $comments = $q->param('comments');
my #files = $q->param('multi_files');
my #io_handles=$q->upload('multi_files');
my %file_hash;
foreach my $item(#files){
next unless $item =~ /.+\/(M.+\.pdf)/;
foreach my $sub_item(#io_handles){
if($item eq $sub_item){
$file_hash{$item} = $sub_item;
}
}
}
chdir $upload_folder or die "Cannot chdir to upload destination directory: $!\n";
print '<ul>';
foreach my $key(keys %file_hash){
my $base = basename($file_hash{$key});#
my $tmpfilename = $q->tmpFileName($file_hash{$key});
my $destFile = File::Spec->catfile($upload_folder,$base);
copy( $tmpfilename, $destFile ) or die "Copy to ($destFile) failed: $!\n";
print '<li>Sucessfully uploaded --- <b>', CGI->escapeHTML(basename($key)), '</b></li>';
}
print '</ul>';
print "<p><b>Name</b> -- $name<br><b>Email</b> -- $email<br><b>Comments</b> -- $comments<br>";
print $q->end_html;
Thanks for the help. (I posted this answer on perlmonks.com as well)
I have a script which fetches the summary file from the NCBI website using command line argument (accession number).
Example:
./efetch.pl NM_000040
Now I am trying to fetch the same file using a HTML webpage which takes the form request via a CGI script.
My question: Is it possible to combine the CGI and my Perl script in one file and pass the HTML form argument from the CGI portion of the code to the perl script in single run?
I have tried to do some scripting but it seems that the argument from the CGI is not getting passed to the Perl script.
Any help will be greatly appreciated.
CGI and Perl Script in one single file:
#!/usr/bin/perl -wT
use strict;
use warnings;
use LWP::Simple;
use CGI::Carp qw(warningsToBrowser fatalsToBrowser);
################### Environmental Variables ###################
my ($buffer, #pairs, $pair, $name, $value, %FORM);
# Read in text
$ENV{'REQUEST_METHOD'} =~ tr/a-z/A-Z/;
if ($ENV{'REQUEST_METHOD'} eq "POST")
{
read(STDIN, $buffer, $ENV{'CONTENT_LENGTH'});
} else {
$buffer = $ENV{'QUERY_STRING'};
}
#print "$buffer\n";
# Split information into name/value pairs
#pairs = split(/&/, $buffer);
foreach $pair (#pairs) {
($name, $value) = split(/=/, $pair);
$value =~ tr/+/ /;
#$value =~ s/%(..)/pack("C", hex($1))/eg;
$FORM{$name} = $value;
}
my $access = $FORM{accession};
if ($access =~ m{\A(\w+\d+)\z}) {
$access = $1;
}
print "Content-type:text/html\r\n\r\n";
print "<html>";
print "<head>";
print "<title> CGI Program</title>";
print "</head>";
print "<body>";
if ($access eq "") {
print "<h2> Please check the accession number</h2>";
exit;
}
print "<h2>$access</h2>";
print "</body>";
print "</html>";
print <HEADING
<html>
<head>
<title> Output result of the program </title>
</head>
<body>
<h1> Summary result </h1>
<table border=1>
<tr>
<th>S.No.</th>
<th>Fragment</th>
<th>Position</th>
<th>Region</th>
<th>GC%</th>
</tr>
HEADING
;
######################## INPUT PARAMETERS #####################
my $utils = "http://www.ncbi.nlm.nih.gov/entrez/eutils";
my $db = "nuccore";
my $query = $access; #"$ARGV[0]" or die "Please provide input for the accession number. $!";
############### END OF INPUT PARAMETERS ######################
############### FILE DOWNLOAD FROM NCBI ######################
my $report = "gb"; # downloads the summary text file
open (IN,">", $query.".summary");
my $esearch = "$utils/esearch.fcgi?" . "db=$db&retmax=1&usehistory=y&term=";
my $esearch_result = get($esearch . $query);
$esearch_result =~ m|<Count>(\d+)</Count>.*<QueryKey>(\d+)</QueryKey>.*<WebEnv>(\S+)</WebEnv>|s;
my $Count = $1; my $QueryKey = $2; my $WebEnv = $3;
my $retstart; my $retmax=3;
for($retstart = 0; $retstart < $Count; $retstart += $retmax) {
my $efetch = "$utils/efetch.fcgi?" .
"rettype=$report&retmode=text&retstart=$retstart&retmax=$retmax&" .
"db=$db&query_key=$QueryKey&WebEnv=$WebEnv";
my $efetch_result = get($efetch);
print IN $efetch_result, "\n";
}
close (IN);
Print command in the perl script prints the $access but it fails to pass the value of $access to $query.
HTML form:
<form action="/cgi-bin/efetch.cgi" method="post" id="myform">
<div>
NCBI accession number:<label for="accession"> <input type="text" name="accession"> </label><br>
<input type="submit" value="Submit" form="myform">
</div>
</form>
Your script is much more complicated than it needs to be. Specifically - you're using the CGI module (which is deprecated, so you might want to consider something else*) but then you're trying to roll your own input handling in your script.
You can write a single script that sends 'POST' or 'GET' data to itself for processing. That's not too difficult at all.
A simple example might be
#!/usr/bin/perl
use strict;
use warnings;
use Data::Dumper;
print "Content-Type: text/html\n\n";
my %param;
while ( <STDIN> ) {
my ( $key, $value ) = split ( "=" );
$param{$key} = $value;
}
print Dumper \%param;
print "<FORM METHOD=\"POST\">\n";
print " <INPUT TYPE=\"text\" NAME=\"access\">\n";
print " <INPUT TYPE=\"submit\">\n";
print "</FORM>\n";
This isn't a good example, but it'll work, and hopefully it'll give you an idea of what's going on - POSTed stuff comes on STDIN. GET stuff comes in the URL.
You can test for the existence of such input, and either render your basic form or process the input you got.
if ( $param{'access'} ) {
#process it;
else {
#print form;
}
There are many modules that make this easier (you're even using one already, in the form of CGI), so I wouldn't EVER suggest doing it this way 'for real' - this is purely an illustration of the basics.
With the CGI module, which is perhaps the thing that'd require least code alteration, you could use the 'CGI::param()' method to retrieve parameters:
use CGI;
print CGI::header;
print CGI::param('access');
#form stuff.
But a more complete one would be to consider a bit more of an in-depth rewrite, and consider using one of the more up to date 'web handling' frameworks. There really are a lot of potential gotchas. (Although it does depend rather, on how much control over your environment you have - internal/limited user scripts I'm a lot more relaxed about than internet facing).
* See: CGI::Alternatives
Writing a CGI program in 2014 is a lot like using a typewriter. Sure, it'll work, but people will look at you very strangely.
But given that you already have a CGI program, let's look at what it might look like if you used techniques that weren't out of date in the last millennium.
There are basically two underlying problems with your code.
You open a file using a name that comes from user input. And that violates the taint mode rules, so your program dies. You would have seen this in your web server error log, had you looked there.
You don't actually need to write the data to a file, because you want to send the data to the user's web browser.
So here's an improved version of your code. It fixes the two problems I mentioned above but it also uses modern tools.
The CGI module has a param() method which makes it far easier for us to get the parameters passed to our program. We also use its header() method to output the CGI header (basically just the Content-type header).
We use the Template module to move all of the HTML out of out code and put it in a separate area. Here I've cheated slightly and have just put it in the DATA section of the CGI program. Usually we'd put it in a completely separate file. Notice how separating the Perl and the HTML makes the program look cleaner and easier to maintain.
It wasn't clear to me exactly how you wanted to format the data you're getting back from the other web site. So I've just stuck it in a "pre" tag. You'll need to work that out for yourself.
Here's the code:
#!/usr/bin/perl -T
use strict;
use warnings;
use LWP::Simple;
use Template;
use CGI ':cgi';
use CGI::Carp qw(warningsToBrowser fatalsToBrowser);
my $access = param('accession');
my $utils = "http://www.ncbi.nlm.nih.gov/entrez/eutils";
my $db = "nuccore";
my $query = $access;
my $report = "gb"; # downloads the summary text file
my $esearch = "$utils/esearch.fcgi?" . "db=$db&retmax=1&usehistory=y&term=";
my $esearch_result = get($esearch . $query);
my $data = '';
if (my ($Count, $QueryKey, $WebEnv) = $esearch_result =~ m|<Count>(\d+)</Count>.*<QueryKey>(\d+)</QueryKey>.*<WebEnv>(\S+)</WebEnv>|s) {
my $retstart;
my $retmax=3;
for ($retstart = 0; $retstart < $Count; $retstart += $retmax) {
my $efetch = "$utils/efetch.fcgi?" .
"rettype=$report&retmode=text&retstart=$retstart&retmax=$retmax&" .
"db=$db&query_key=$QueryKey&WebEnv=$WebEnv";
my $efetch_result = get($efetch);
$data .= $efetch_result;
}
}
my $tt = Template->new;
print header;
$tt->process(\*DATA, { data => $data })
or die $tt->error;
__END__
<html>
<head>
<title> CGI Program</title>
</head>
<body>
<h1>Input</h1>
<form action="/cgi-bin/efetch.cgi" method="post" id="myform">
<div>NCBI accession number:<label for="accession"> <input type="text" name="accession"></label><br>
<input type="submit" value="Submit" form="myform"></div>
</form>
[% IF data -%]
<h1>Summary Result</h1>
<pre>
[% data %]
</pre>
[% END -%]
</body>
</html>
reference an earlier problem I have since changed my code to this(I am getting closer) however it now reads out the whole file and not the line I am trying to ask for. (I want to read out a line that contains a value that the user enters on the form.
form code:
#!\xampp\perl\bin\perl.exe
use CGI qw/:standard/; # load standard CGI routines
use CGI::Carp('fatalsToBrowser');
print header(); # create the HTTP header
print <<HTML
<head>
<title>Shop Here</title>
</head>
<body>
<h1>list</h1>
<br />
<form action="doSearch.pl">
animalname: <input type="text", name="search" size=5><br><br>
<input type="submit" value="select">
</form>
</body>
</html>
HTML
# <>;
response form////////////////////////
use CGI qw(:standard);
use CGI::Carp('fatalsToBrowser');
$search = new CGI;
#animallist = param;
print header, start_html("animal list"); #prints title on tab
$inFile = "animal.txt";
open (IN, $inFile) or
die "Can't find file: $inFile";
#animallist = (<IN>);
# print #animallist, "\n" ;
foreach $line (#animallist)
{
if ($line =~ $value)
{
print $line;
}
}
print end_html;
You really should put a question in your question.
I assume your code is not working.
I know that the line $line =~ $value may very well not do what you want if $value contains special characters.
have a look here ( \Q \E may be what you want ) IF that is the problem and you need to solve it.
I have a html as the following:
<!DOCTYPE html
PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en-US" xml:lang="en-US">
<head>
<title></title>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
</head>
<body bgcolor="white">
<h1>foo.c</h1>
<form method="post" action=""
enctype="application/x-www-form-urlencoded">
Compare this file to the similar file:
<select name="file2">
<option value="...">...</option>
</select>
<input type="hidden" name="file1" value="foo.c" /><br>
Show the results in this format:
</form>
<hr>
<p>
<pre>
some code
</pre>
I need to get value of input name = 'file' and the contents of HTML pre tag. I don't know on perl language, by googling I wrote this small program(that I believe isn't "elegant"):
#!/usr/bin/perl
package MyParser;
use base qw(HTML::Parser);
#Store the file name and contents obtaind from HTML Tags
my($filename, $file_contents);
#This value is set at start() calls
#and use in text() routine..
my($g_tagname, $g_attr);
#Process tag itself and its attributes
sub start {
my ($self, $tagname, $attr, $attrseq, $origtext) = #_;
$g_tagname = $tagname;
$g_attr = $attr;
}
#Process HTML tag body
sub text {
my ($self, $text) = #_;
#Gets the filename
if($g_tagname eq "input" and $g_attr->{'name'} eq "file1") {
$filename = $attr->{'value'};
}
#Gets the filecontents
if($g_tagname eq "pre") {
$file_contents = $text;
}
}
package main;
#read $filename file contents and returns
#note: it works only for text/plain files.
sub read_file {
my($filename) = #_;
open FILE, $filename or die $!;
my ($buf, $data, $n);
while((read FILE, $data, 256) != 0) {
$buf .= $data;
}
return ($buf);
}
my $curr_filename = $ARGV[0];
my $curr_file_contents = read_file($curr_filename);
my $parser = MyParser->new;
$parser->parse($curr_file_contents);
print "filename: ",$filename,"file contents: ",$file_contents;
Then I call ./foo.pl html.html But I'm getting empty values from $filename and $file_contents variables.
How to fix this?
Like always, there's more than one way to do it. Here's how to use the DOM Parser of Mojolicious for this task:
#!/usr/bin/env perl
use strict;
use warnings;
use Mojo::DOM;
# slurp all lines at once into the DOM parser
my $dom = Mojo::DOM->new(do { local $/; <> });
print $dom->at('input[name=file1]')->attr('value');
print $dom->at('pre')->text;
Output:
foo.c
some code
Using xpath and HTML::TreeBuilder::XPath Perl module ( very few lines ):
#!/usr/bin/env perl
use strict; use warnings;
use HTML::TreeBuilder::XPath;
my $tree = HTML::TreeBuilder::XPath->new_from_content( <> );
print $tree->findvalue( '//input[#name="file1"]/#value' );
print $tree->findvalue( '//pre/text()' );
USAGE
./script.pl file.html
OUTPUT
foo.c
some code
NOTES
in the past, I was using HTML::TreeBuilder module to do some web-scraping. Now, I can't go back to complexity. HTML::TreeBuilder::XPath do all the magic with the useful Xpath expressions.
you can use new_from_file method to open a file or a filehandle instead of new_from_content, see perldoc HTML::TreeBuilder ( HTML::TreeBuilder::XPath inherit methods from HTML::TreeBuilder)
using <> in this way is allowed here because HTML::TreeBuilder::new_from_content() specifically allows reading multiple lines in that way. Most constructors will not allow this usage. You should provide a scalar instead or use another method.
You don't generally want to use plain HTML::Parser unless you're writing your own parsing module or doing something generally tricky. In this case, HTML::TreeBuilder, which is a subclass of HTML::Parser, is the easiest to use.
Also, note that HTML::Parser has a parse_file method (and HTML::TreeBuilder makes it even easier with a new_from_file method, so you don't have to do all of this read_file business (and besides, there are better ways to do it than the one you picked, including File::Slurp and the old do { local $/; <$handle> } trick.
use HTML::TreeBuilder;
my $filename = $ARGV[0];
my $tree = HTML::TreeBuilder->new_from_file($filename);
my $filename = $tree->look_down(
_tag => 'input',
type => 'hidden',
name => 'file1'
)->attr('value');
my $file_contents = $tree->look_down(_tag => 'pre')->as_trimmed_text;
print "filename: ",$filename,"file contents: ",$file_contents;
For information on look_down, attr, and as_trimmed_text, see the HTML::Element docs; HTML::TreeBuilder both is a, and works with, elements.
How would I write a Perl CGI script that receives a file via a HTTP post and saves that to the file system?
Use the CGI module.
my $fh = $query->upload('upload_field');
while(<$fh>) {
print SAVE_FILE $_;
}
Just a note: however you will write it, don't save it in a place accessible from your web-server.
And now to the point: below is a script which I was using for some time for photo-uploading. It might need some tweaking, but should show you the way.
As the image isnt uploaded to web-accesible directory, we then have separate process checking it, resizing, putting a watermark and placing it where it can be accessed.
#!/usr/bin/perl -wT
use strict;
use CGI;
use CGI::Carp qw ( fatalsToBrowser );
use File::Basename;
$CGI::POST_MAX = 1024 * 5000;
my $safe_filename_characters = "a-zA-Z0-9_.-";
my $upload_dir = "/home/www/upload";
my $query = new CGI;
my $filename = $query->param("photo");
my $email_address = $query->param("email_address");
if ( !$filename )
{
print $query->header ( );
print "There was a problem uploading your photo (try a smaller file).";
exit;
}
my ( $name, $path, $extension ) = fileparse ( $filename, '\..*' );
$filename = $name . $extension;
$filename =~ tr/ /_/;
$filename =~ s/[^$safe_filename_characters]//g;
if ( $filename =~ /^([$safe_filename_characters]+)$/ )
{
$filename = $1;
}
else
{
die "Filename contains invalid characters";
}
my $upload_filehandle = $query->upload("photo");
open ( UPLOADFILE, ">$upload_dir/$filename" ) or die "$!";
binmode UPLOADFILE;
while ( <$upload_filehandle> )
{
print UPLOADFILE;
}
close UPLOADFILE;
print $query->header ( );
print <<END_HTML;
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Thanks!</title>
</head>
<body>
<p>Thanks for uploading your photo!</p>
</body>
</html>
END_HTML
See the CGI.pm documentation for file uploads.
I'd start by using CGI and reading CREATING A FILE UPLOAD FIELD, and using open to create a file and print to write to it. (and then close to close it).