Okay I'm working on finding the latitude and longitude of cities using Perl. I found the Geo::Coder::Google module and have it installed properly, but I'm getting an error when attempting to use it. The first time I got the error I went out and got an API key from Google thinking that was what I was missing, but that didn't solve the error either. Can someone help me figure out what I'm missing?
Here is the error I'm receiving:
Google Maps API returned error: 500 Can't verify SSL peers without knowing which Certificate Authorities to trust at test.pl line 7.
Here is the code I'm using right now:
1:#!/usr/bin/perl
2:use strict;
3:use warnings;
4:
5:use Geo::Coder::Google;
6:my $geocoder = Geo::Coder::Google->new( apiver => 3, gl => 'us', apikey => 'My API Key Here' );
7:my $location = $geocoder->geocode( location => 'Mount Vernon, IN' );
8:
9:print "$location->{'geometry'}->{'location'}->{'lat'}";
10:print "$location->{'geometry'}->{'location'}->{'lng'}";
The code you have written works fine as it stands—you don't need an API key. The problem is that your Secure Socket Layer support for the HTTPS is incomplete
You should start by reinstalling the LWP library to make sure that it is up to date. You should also install Mozilla::CA to make sure that the Certificates of Authority are current. (This is the most likely cause of the problem given the error message that you're getting.)
If it is still not working after that, then the only other culprits I can think of are IO::Socket::SSL and Crypt::SSLeay, but I would be surprised if they are out of date as they should be updated as dependencies of LWP
#!/usr/bin/perl
use strict;
use warnings 'all';
use Geo::Coder::Google;
my $geocoder = Geo::Coder::Google->new( apiver => 3 );
my $info = $geocoder->geocode( location => 'Mount Vernon, IN' );
my $location = $info->{geometry}{location};
printf "%s %s\n", $location->{lat}, $location->{lng};
output
37.9322662 -87.8950267
Try:
sudo cpan
install LMP::UserAgen Mozilla::CA
Related
I have migrated my scrilpts from CentOS 7 to 8 and there's a new Perl version. I have the folowing snippet that uses head to check if a URL exists:
#!/bin/perl
use strict;
use warnings;
use LWP::Simple;
my $sitemapurl = "https://www.prosinger.net";
if (head($sitemapurl)) {
...
}
else {
print "The $sitemapurl doesn't exist\n";
exit(1);
}
It now always returns that the URL doesn't exist. I'm quite sure that this has to do something with https (I have perl-LWP-Protocol-https installed), but I'm not sure how to get any feedback information from head method to check what the error code is.
Any ideas?
You can use LWP::UserAgent instead of LWP::Simple, which allows you to get an error message:
my $ua = LWP::UserAgent->new;
my $sitemapurl = "https://www.prosinger.net";
my $req = $ua->head($sitemapurl);
if ($req->is_success) {
...
} else {
die "Could not head($sitemapurl): " . $req->status_line;
}
Running this code prints:
Could not head(https://www.prosinger.net): 403 Forbidden at head.pl line 15.
You can fix this (for this specific website; this will not work for all website) by setting a User-Agent in your LWP::UserAgent object:
my $ua = LWP::UserAgent->new( agent => 'anything seems to work' );
Of interest is the decoded_content method of HTTP::Response that allows you to get the content of the request (you don't need it in that case, but you might later):
my $req = $ua->get(...);
if ($req->is_success) {
my $content = $req->decoded_content;
...
}
Your code that uses LWP::Simple and Dada's version that switches to LWP::UserAgent are basically doing the same thing, except that you can get details of the error when using LWP::UserAgent.
Running the LWP::UserAgent version gives this error:
Could not head(https://www.prosinger.net): 500 Can't connect to www.prosinger.net:443 (SSL connect attempt failed error:2707307E:OCSP routines:OCSP_check_validity:status not yet valid)
And Googling that error message gives this SO answer as the first result. Is it possible that the clocks on your your client machine and the server are out of sync?
I have a very simple script (on a SLES11 system) to send a http1.1 request to a server. This worked fine for a long time. Since a couple of days it stopped working. I have no idea why. After some investigation I found out forcing the script to use http1.0 it is working again. I have no idea why. And I want to know why it is not working in its default http1.1 mode.
I already enabled debugging as much as I know it (see code below). But I do not see anything - my script just hangs. There is no network activity. I neiter see that a socket will be openend (netstat -a), nor I see any incoming traffic form my client on the server. (A "telnet myserver myport" works fine.)
Can someone please help me how to track down this problem? How can I enable more debug to see where the real problem is?
#!/usr/bin/perl -w
use strict;
use warnings;
use Data::Dumper;
use HTTP::Request::Common;
use LWP::Debug qw(+);
use LWP::UserAgent;
# Workaround: forcing http1.0 instead of using http1.1, it works again!
use LWP::Protocol::https10 ();
LWP::Protocol::implementor('https', 'LWP::Protocol::https10');
# EO workaround
my $ua = LWP::UserAgent->new;
$ua->ssl_opts( verify_hostname => 0 );
my $response = $ua->request(
POST 'https://myuser:mypassword#myserver:8888/service/myservice',
Connection => 'close', # Edit: added, see comments below
Content_Type => 'text/xml',
Content => '... my content ...'
);
$ua->request() does not return! I need to kill/^C the script!
Edit: ok, no one seems to have an idea how to continue. So I startet debugging it using perl debugger.
LWP::UserAgent::post(/usr/lib/perl5/site_perl/5.10.0/LWP/UserAgent.pm:418):
418: return $self->request( HTTP::Request::Common::POST( #parameters ), #suff );
So I can see it is not returning from request().
Anyway, due to the fact that LWP::UserAgent calls HTTP::Request::Common I changed my example code above back to use HTTP::Request::Common to skip that step while debugging.
Ok... New result:
LWP::Protocol::implementor(/usr/lib/perl5/site_perl/5.10.0/Net/HTTPS.pm:26):
26: eval { require IO::Socket::SSL; };
Inside request() it hangs at IO::Socket::SSL. Means, this script it enough for further debug:
#!/usr/bin/perl
require IO::Socket::SSL;
This statement does not come back.
Further down, inside IO::Socket::SSL it hangs at:
IO::Socket::SSL::CODE(0x1274370)(/usr/lib/perl5/site_perl/5.10.0/IO/Socket/SSL.pm:92):
92: Net::SSLeay::SSLeay_add_ssl_algorithms();
Ahhhh! There is already a bugreport concerning this issue: Net-SSLeay hangs on Suse 11 P2 pointing to Bug #81575 that says:
I enountered this same issue on SLES 11 SP2, which has openssl-0.9.8j installed.
An upgrade to openssl-0.9.8r did resolve the problem. ...
The packages for 0.9.8r can be found in this repository:
http://download.opensuse.org/repositories/security:/fips/
Guess that's it!
There is already a bugreport concerning this issue: Net-SSLeay hangs on Suse 11 P2 pointing to Bug #81575 that says:
I enountered this same issue on SLES 11 SP2, which has openssl-0.9.8j installed. An upgrade to openssl-0.9.8r did resolve the problem. ... The packages for 0.9.8r can be found in this repository: http://download.opensuse.org/repositories/security:/fips/
Guess that's it!
I am having difficulty using perl to visit a website via TOR if it is an https site but not if it is an http site.
#!/usr/bin/perl
use strict;
use WWW::Mechanize;
use LWP::Protocol::socks;
use LWP::Protocol::https;
use utf8;
my $mech = WWW::Mechanize->new(timeout => 60*5);
$mech->proxy(['http', 'https'], 'socks://localhost:9150');
$mech->get("https://www.google.com");
I am receiving the error message: Error GETing https://www.google.com: Status read failed: Bad file descriptor at line 10," where line i10 is the last line of the program.
In the TOR browser, I can successfully view: "https://www.google.com" with a port of 9150.
I am using ActivePerl 5.16.2; Vadalia 0.2.21 and Tor 0.2.3.25.
I have a Windows machine and my primary internet browser is Mozilla.
I have tried installing packages with the commands:
cpan LWP::UserAgent
ppm install LWP::Protocol::https
cpan LWP::Protocol::https
ppm install LWP::Protocol::socks
cpan LWP::Protocol::socks
ppm install Mozilla::CA
ppm install IO::Socket::SSL
ppm install Crypt::SSLeay
cpan Crypt::SSLeay
Thank you for any help! Please let me know whether there is any further information that I can provide.
Time ago, i'd found the way to go throught https sites with Tor using WWW::Curl::Easy to fetch those kind of sites, because using LWP i found the same problems.
After that i save all html in files and parsing them using WWW::Mechanzie or HTML::TreeBuilder.
If you want more interactivity with site like post forms , etc. This solutions may be more tedious because you'll need to interact with curl.
package Curl;
use warnings;
use WWW::Curl::Easy;
use WWW::UserAgent::Random;
my $curl = WWW::Curl::Easy->new;
my $useragent = rand_ua("browsers");
my $host = 'localhost';
my $port = '9070';
my $timeout = '20';
my $connectTimeOut= '20';
&init;
sub get
{
my $url = shift;
$curl->setopt(CURLOPT_URL, $url);
my $response_body;
$curl->setopt(CURLOPT_WRITEDATA,\$response_body);
my $retcode = $curl->perform;
if ($retcode == 0) {
print("Transfer went ok Http::Code = ".$curl->strerror($retcode)."\n");
my $response_code = $curl->getinfo(CURLINFO_HTTP_CODE);
# judge result and next action based on $response_code
return \$response_body;
} else {
# Error code, type of error, error message
print("An error happened: $retcode ".$curl->strerror($retcode)." ".$curl->errbuf."\n");
return 0;
}
}
sub init
{
#setejem el proxy
$curl->setopt(CURLOPT_PROXY,"$host:".$port);
$curl->setopt(CURLOPT_PROXYTYPE,CURLPROXY_SOCKS4);
#posem les altres dades
$curl->setopt(CURLOPT_USERAGENT, $useragent);
$curl->setopt(CURLOPT_CONNECTTIMEOUT, $connectTimeOut);
$curl->setopt(CURLOPT_TIMEOUT, $timeout);
$curl->setopt(CURLOPT_SSL_VERIFYPEER,0);
$curl->setopt(CURLOPT_HEADER,0);
}
Hope this will help you!
Maybe the proxy that you are using is already an HTTPS proxy (ie. CONNECT proxy). In that case this should work (untested):
#!/usr/bin/perl
use strict;
use WWW::Mechanize;
use LWP::Protocol::socks;
use LWP::Protocol::https;
use utf8;
my $mech = WWW::Mechanize->new(timeout => 60*5);
$mech->proxy(['http'], 'socks://localhost:9150');
$mech->proxy(['https'], 'https://localhost:9150'); ### <-- make https go over https-connect proxy
$mech->get("https://www.google.com");
I cannot find the origin but I fought with that a long time ago. Basically the problem I had was with the imlpementation that LWP::UserAgent used for the https requests.
Possibly this question can help you: How do I force LWP to use Crypt::SSLeay for HTTPS requests?
I'm trying to script some interaction between a Linux box and an ESXi host, using the VMware Perl library (which appears to call a SOAP service on the ESXi host).
I am getting an error for which I can't find a solution: Undefined subroutine &LWP::Protocol::https::Socket::can_read called at /usr/local/share/perl5/LWP/Protocol/http.pm line 22
I am currently running Net-HTTP-6.03. I did try with Net-HTTP-6.05 but it makes the web-based calls very slow (and subsequent ones also eventually fail, instead with about 1.2MB of XML as the error message) -- which various forums suggest is only resolvable by reverting to 6.03 (via cpan install GAAS/Net-HTTP-6.03.tar.gz).
The crux of my code is as follows:
use VMware::VIRuntime;
my $context = {
options => ParseOptions(), # result of using GetOpts::Long to parse command-line
session => undef
};
$ENV{PERL_NET_HTTPS_SSL_SOCKET_CLASS} = "Net::SSL"; # Suppresses SSL_VERIFY_NONE
if ($context->{options}->{'ignore-ssl-errors'}) {
$ENV{PERL_LWP_SSL_VERIFY_HOSTNAME} = 0; # Ignores certificate errors
}
## next line fails
$context->{session} = Vim::login(
service_url => $context->{options}->{url},
username => $context->{options}->{username},
password => $context->{options}->{password}
);
Is there something else I (can|need to) do to get this working?
I've got it. I needed to reinstall Bundle::LWP from an older version, not just Net-HTTP. Without it, I had LWP v6.05 trying to talk to Net-HTTP v6.03.
Once I realised that LWP == libwww-perl (yes, I'm that new), I found the overall package I needed:
cpan install GAAS/libwww-perl-6.03.tar.gz
Reproduced with LWP 6.05 and Net::HTTP 6.01, which is the cocktail that MacPorts installs at the time of this writing. Upgrading Net::HTTP to 6.06 (current) solved it:
sudo cpan Net::HTTP
I'm attempting to use an existing CAS server to authenticate login for a Perl CGI web script and am using the AuthCAS Perl module (v 1.3.1). I can connect to the CAS server to get the service ticket but when I try to connect to validate the ticket my script returns with the following error from the IO::Socket::SSL module:
500 Can't connect to [CAS Server]:443 (Bad hostname '[CAS Server]')
([CAS Server] substituted for real server name)
Symptoms/Tests:
If I type the generated URL for the authentication into the web browser's location bar it returns just fine with the expected XML snippet. So it is not a bad host name.
If I generate a script without using the AuthCAS module but using the IO::Socket::SSL module directly to query the CAS server for validation on the generated service ticket the Perl script will run fine from the command line but not in the browser.
If I add the AuthCAS module into the script in item 2, the script no longer works on the command line and still doesn't work in the browser.
Here is the bare-bones script that produces the error:
#!/usr/bin/perl
use strict;
use warnings;
use CGI;
use AuthCAS;
use CGI::Carp qw( fatalsToBrowser );
my $id = $ENV{QUERY_STRING};
my $q = new CGI;
my $target = "http://localhost/cgi-bin/testCAS.cgi";
my $cas = new AuthCAS(casUrl => 'https://cas_server/cas');
if ($id eq ""){
my $login_url = $cas->getServerLoginURL($target);
printf "Location: $login_url\n\n";
exit 0;
} else {
print $q->header();
print "CAS TEST<br>\n";
## When coming back from the CAS server a ticket is provided in the QUERY_STRING
print "QUERY_STRING = " . $id . "</br>\n";
## $ST should contain the received Service Ticket
my $ST = $q->param('ticket');
my $user = $cas->validateST($target, $ST); #### This is what fails
printf "Error: %s\n", &AuthCAS::get_errors() unless (defined $user);
}
Any ideas on where the conflict might be?
The error is coming from the line directly above the snippet Cebjyre quoted namely
$ssl_socket = new IO::Socket::SSL(%ssl_options);
namely the socket creation. All of the input parameters are correct. I had edited the module to put in debug statements and print out all the parameters just before that call and they are all fine. Looks like I'm going to have to dive deeper into the IO::Socket::SSL module.
As usually happens when I post questions like this, I found the problem. It turns out the Crypt::SSLeay module was not installed or at least not up to date. Of course the error messages didn't give me any clues. Updating it and all the problems go away and things are working fine now.
Well, from the module source it looks like that IO::Socket error is coming from get_https2
[...]
unless ($ssl_socket) {
$errors = sprintf "error %s unable to connect https://%s:%s/\n",&IO::Socket::SSL::errstr,$host,$port;
return undef;
}
[...]
which is called by callCAS, which is called by validateST.
One option is to temporarily edit the module file to put some debug statements in if you can, but if I had to guess, I'd say the casUrl you are supplying isn't matching up to the _parse_url regex properly - maybe you have three slashes after the https?