I am having an issue where I am using cURL inside a perl script to execute a http request. I believe my issue is related to special characters in the URL string but I cannot figure out how to make it work.
I can confirm that the URL is correct as I can run it from my browser.
My perl script is
#!/usr/bin/perl
use strict;
use warnings;
$url = "http://machine/callResync?start=2017-02-01 00.00.00.000&end=2017-02-01 23.23.999";
system "curl $url
It fails when it reaches the first whitespace. I tired to escape that using %20.
After that I put in %26 to escape the & but then I get another issue. I have tired a number of different combinations but it keeps failing.
Any idea's.
Use the URI module to correctly build a URL, and rather than shelling out to cURL you should use a Perl library like LWP::Simple to access the page
The disadvantage of LWP::Simple is that it may be too simple in that it provides no diagnostics if the transaction fails. If you find you need something more elaborate then you should look at
HTTP::Tiny,
LWP::UserAgent, or
Mojo::UserAgent.
If you need help with these then please ask
use strict;
use warnings 'all';
use URI;
use LWP::Simple 'get';
my $url = URI->new('http://machine/callResync');
$url->query_form(
start => '2017-02-01 00.00.00.000',
end => '2017-02-01 23.23.999',
);
my $content = get($url) or die "Failed to access URL";
Problem number 1: You used an invalid URL. Spaces can't appear in URLs.
my $url = "http://machine/callResync?start=2017-02-01%2000.00.00.000&end=2017-02-01%2023.23.999";
Problem number 2: Shell injection error. You didn't correctly form your shell command.
system('curl', $url);
or
use String::ShellQuote qw( shell_quote );
my $cmd = shell_quote('curl', $url);
system($cmd);
Related
I have a perl cgi script that is exactly the following:
#!/usr/bin/perl -w
use CGI;
$query = new CGI;
print $query->redirect("http://www.yahoo.com");
At the command line things look OK:
$perl test.pl
Status: 302 Moved
Location: http://www.yahoo.com
When I load it in the browser, http://localhost/cgi-bin/test.pl, the request gets aborted, and depending on the browser I get various messages:
Connection reset by server.
No data received.
The only research I could find on this issue, stated that a common problem is printing some data or header before the redirect call, but I am clearly not doing that here.
I'm hosting it from a QNX box with the default slinger server.
The code works fine on my machine, check the following
Check the error logs, eg: tail /var/log/http/error_log
Do the chmod/chown permissions match other working CGi scripts, compare using ls -l
Does printing the standard hello world work? Change your print statement to
print $query->header(), 'Hello World';
Add the following for better errors
use warnings;
use diagnostics;
use CGI::Carp 'fatalsToBrowser';
at the command line use slinger will return some basic use options. For logging you need both syslogd and -d enabled in slinger. Ie
slinger -d &
Then look to /var/log/syslog for errors
i'm trying to implement some REST API functionality in PERL using WWW::Curl::Easy.
I suffer on curl error Code 3: "URL using bad/illegal format or missing URL No URL set!
While debugging this issue it seems that there is some issue with setting the URL CURLOPT.
For Example: IMHO this testcode should return the URL http://www.example.com
#!/usr/bin/perl
use strict;
use WWW::Curl::Easy;
my $curl = WWW::Curl::Easy->new;
$curl->setopt("WWW::Curl::Easy::CURLOPT_URL", 'http://www.example.com');
print $curl->getinfo("WWW::Curl::Easy::CURLINFO_EFFECTIVE_URL");
but i'm getting "43" as output.
Is there a bug in my module?
CURL WWW::Curl::Easy::Version 4.17
perl Version v5.16.2
System OSX 10.9
What are you using for documentation?
The following is how the WWW::Curl docs for WWW::Curl::Easy demonstrate getting started:
use strict;
use warnings;
use WWW::Curl::Easy;
my $curl = WWW::Curl::Easy->new;
$curl->setopt(CURLOPT_HEADER,1);
$curl->setopt(CURLOPT_URL, 'http://example.com');
When I try to run script on my second computer I get this message:
malformed JSON string, neither array, object, number, string or atom, at character offset 0
(before "LWP will support htt...") at iptest.pl line 21, line 2.
On my first computer, the script works fine.
Line 21:
my $data = decode_json($resp->content);
Does anyone know what the problem can be?
Thanks in advance
I'm a bit surprised that the JSON error is the only error you get. But it does contain a tiny little hint: "LWP will support htt...". I bet that LWP is missing a module it needs to be able to make https connections. You now have two options:
print $response->content to see the full error message.
On the command line, do something like lwp-request https://google.com/. You should see the full error message.
Then install the missing module.
And of course: please, please, please:
use strict and use warnings
Clean that script up and throw away every use-line you don't need: IO::Socket, LWP::Simple, YAML::Tiny.
Read the documentation of the modules that you actually are using. What are you trying to achieve with LWP::UserAgent->new(keep_alive)? Hint: It won't help to quote keep_alive.
Some issues:
Always use use strict; use warnings;.
Never use $response->content. What it returns is useless. Instead, use $response->decoded_content( charset => 'none').
You need to chomp the values you get from STDIN.
You should never use our unless forced to (e.g. our #ISA = ok). my should be used instead.
my $format = '$format'; "$format" is a silly way of writing "\$format".
I applied most of the changes ikegami suggested. Then perl gave me good error messages to fix the remaining issues. It looks like it works now. Don't ask why it didn't work before. Your code is weird that it's hard to say what exactly went wrong. With strict and warnings you're forced to write better code. Maybe add some nicely named subroutines to add more clarity.
#!/usr/bin/perl
use strict;
use warnings;
use IO::Socket;
use LWP::UserAgent;
use open qw(:std :utf8);
use LWP::Simple;
use YAML::Tiny;
use JSON;
use URI;
use List::MoreUtils qw(uniq);
print "Enter Qve:";
my ( $qve, $loc, $key, $href );
chomp( $qve = <STDIN> );
print "Enter Location:";
chomp( $loc = <STDIN> );
$key = '';
my $format = '$format';
$href =
"https://api.datamarket.azure.com/Bing/Search/v1/Web?Query='$qve [loc:$loc]'&Latitude=43&Longitude=19&$format=JSON";
my $ua = LWP::UserAgent->new('keep_alive');
$ua->credentials( "api.datamarket.azure.com" . ':443', '', '', $key );
my $resp = $ua->get($href);
my $data = decode_json( $resp->decoded_content( charset => 'none' ) );
my #urls = map { $_->{'Url'} } #{ $data->{d}->{results} };
my #za;
for my $i ( 0 .. $#urls ) {
my $trz = "www.";
my $host = URI->new( $urls[$i] )->host;
$host =~ s/$trz//g;
push( #za, $host );
}
For the record, this fixed the issue for me (CentOS):
# yum install perl-Crypt-SSLeay
I got in the same situation. After I used 'yum' to install perl, I still got the error when run the perl script.
$ sudo yum install perl
Updating Subscription Management repositories.
Unable to read consumer identity
This system is not registered to Red Hat Subscription Management
...
Eventually I did these, and it works.
$ lwp-request https://google.com/
It returned an error message .. (LWP::Protocol::https not installed)
$ sudo rpm -ivh ~/perl-LWP-Protocol-https-6.07-4.el8.noarch.rpm
$ lwp-request https://google.com/
it returned a HTML page.
and my perl script can run without error.
(my system is:
$ cat /etc/os-release
PRETTY_NAME=Red Hat Enterprise Linux 8.0 )
I'm unable to execute the HTML::Template function in the CGI.
I'm following a simple tutorial that I found here: http://metacpan.org/pod/HTML::Template
I created a new file on my server in the home path as test.tmpl.
I created a new file named frt.cgi ... (is that the issue here? should it be a different file extention??)
#!/usr/local/bin/perl -w
use HTML::Template;
# open the html template
my $template = HTML::Template->new(filename => '/test.html');
# fill in some parameters
$template->param(HOME => $ENV{HOME});
$template->param(PATH => $ENV{PATH});
# send the obligatory Content-Type and print the template output
print "Content-Type: text/html\n\n", $template->output;
I've modified the 1st line to reflect my host provided program path for perl. I don't know what the -w does I just know I've tried this with and without it. Also I've tried changing the code a bit like this:
use warnings;
use strict;
use CGI qw(:standard);
use HTML::Template;
I've searched...
https://stackoverflow.com/search?q=HTML%3A%3ATEMPLATE+&submit=search
https://stackoverflow.com/search?q=HTML%3A%3ATEMPLATE
https://stackoverflow.com/search?q=HTML%3A%3ATEMPLATE+PERL&submit=search
Yet I still do not see the answer.
I even searched google for .TMPL Encoding because I thought there may be some special type needed. Please help.
If you look in your server logs, you'll probably see an error message along the lines of:
HTML::Template->new() : Cannot open included file /test.html : file not found.
You need to provide the path on the file system, not a URI relative to the generated document.
First, you likely specified the wrong path - change /test.html to test.html.
Also, it is possible that there is no $ENV{HOME} variable in your system so set up flag die_on_bad_params to 0:
my $template = HTML::Template->new(
filename => 'test.html',
die_on_bad_params => 0,
);
Also, don't forget to mark your Perl file as executable by chmod 755.
Option -w makes Perl to enable warnings, so there is no point to write use warnings; afterwards.
You can check what Perl command line options do by using module B::Deparse, like this ($^W variable disables/enables warnings):
perl -w -MO=Deparse -e "print;"
This would print:
BEGIN { $^W = 1; }
print $_;
Perl Question. I'm trying to get this script running in a debugger.
I've got Aptana + Epic + ActivePerl 5.12.4 working on Windows 7x64. The script is starting fine but I'm getting an error:
curl -sS http://intranet.mycompany.org/directory/directory.xml
The above command works fine... but if I start the debugger I get this error:
curl: (1) Protocol 'http not supported or disabled in libcurl
First part of the script below:
#!/usr/bin/perl
use strict;
use XML::Parser;
use Data::Dumper;
my $url = 'http://intranet.atlanticgeneral.org/directory/directory.xml';
my $output = 'C:\global.gabook';
my $file = "curl -sS '$url' |";
my $parser = new XML::Parser(Style => 'Tree');
my $tree = $parser->parsefile($file)->[1];
Windows doesn't like single quotes in commands. Try using double quotes in the command, using qq{} escaping. Just change one line:
my $file = qq{curl -sS "$url" |};
Wooble~
"I'm guessing it's the extra single quotes around $url that's
causing it"
When I removed the quotes around the '$url' it worked. Quotes worked in redhat perl, but didn't work in my windows perl debugger:
#!/usr/bin/perl
use strict;
use XML::Parser;
use Data::Dumper;
my $url = 'http://intranet.atlanticgeneral.org/directory/directory.xml';
my $output = 'C:\global.gabook';
my $file = "curl -sS $url |";
my $parser = new XML::Parser(Style => 'Tree');
my $tree = $parser->parsefile($file)->[1];
Posting as answer since Wooble didn't.
I was getting the same error when I was using the curl command in my java program as follows
String command = "curl 'http://google.com'";
try
{
Process proc = Runtime.getRuntime().exec(command);
.......
}catch(Exception e){}
Changing command to the following fixed this error
String command = "curl http://google.com";
Actually, It may be an issue because of shell interpretor.
I used curl command like below example
String command = "curl -duser.name=hdfs -dexecute=select+*+from+logdata.test; -dstatusdir=test.output http://hostname:50111/templeton/v1/hive";
As an alternative (and not needing an external program), you could use LWP::UserAgent to fetch the document.
You pass a wrongly spelled protocol part as in "htpt://example.com" or as in the less evident case if you prefix the protocol part with a space as in " http://example.com/".
Source : https://curl.haxx.se/docs/faq.html