I'm trying to do a download of files from a remote host using LWP. Here is a simplified version of my code that reproduces the error:
#!/usr/local/bin/perl5.8 -w
use strict;
use LWP::UserAgent;
my $userAgent = LWP::UserAgent->new;
$userAgent->agent("p_o_c");
my $request = HTTP::Request->new(GET=>"https://ajax.googleapis.com/ajax/libs/prototype/1.7.2.0/prototype.js");
my $response = $userAgent->request($request);
When I run this code, I get the following:
[dev] /home/me > ./tmp/poc.pl
/usr/local/bin/perl5.8: symbol lookup error: /usr/local/ext/perl/5.8.0/lib/site_perl/i686-linux-thread-multi/auto/Crypt/SSLeay/SSLeay.so: undefined symbol: OpenSSL_add_all_algorithms
Also, as seen above, I'm running perl 5.8. I'm running RHEL 6.4 (Santiago).
I've tried downloading a file over http (not https) and that works.
Does anyone have any idea on how to get this https download to work? I'm open to other methods too besides LWP, but this seemed like the easiest way.
As Oesor mentioned, I am using (not by choice) a very old version of Perl. Using a newer version fixed the issue.
Related
My Perl script sends some information to a remote server.
Below is a portion of the code
#!/var/hvmail/libexec/perl
use strict;
use HTTP::Request::Common qw(POST);
use LWP::UserAgent;
use constant HANDLER_URL => "https://www.website.com/handler.php";
$ENV{HTTPS_DEBUG} = 1;
my $ua = LWP::UserAgent->new;
# Some DB stuff, not applicable
my $row; # This is a DB row ($sth->fetchrow_hashref())
my $req = POST ''.HANDLER_URL, [ %$row ];
my $res = $ua->request($req);
$res->is_success is false with $res->status_line being
500 SSL negotiation failed
We are running CentOS 6.4, Perl 5.10.1, OpenSSL 1.0.1e-fips.
Update
Here's the full output:
SSL_connect:before/connect initialization
SSL_connect:SSLv2/v3 write client hello A
SSL3 alert read:fatal:handshake failure
SSL_connect:error in SSLv2/v3 read server hello A
SSL_connect:before/connect initialization
SSL_connect:SSLv3 write client hello A
SSL3 alert read:fatal:handshake failure
SSL_connect:failed in SSLv3 read server hello A
SSL_connect:before/connect initialization
SSL_connect:SSLv2 write client hello A
SSL_connect:failed in SSLv2 read server hello A
Error: [ 500 SSL negotiation failed: ]
Requested Command Output
Can't locate Net/SSLeay.pm
Can't locate LWP/Protocol/https.pm
You seem to be relying on Crypt::SSLeay. You shouldn't. It's outdated and incomplete.
Install the latest LWP::Protocol::https which will upgrade your LWP and install the preferred SSL/TLS stack consisting of the IO::Socket::SSL and Net::SSLeay.
A web search shows there are CentOS6 repositories with RPM packages for LWP::Protocol::https.
The server has disabled SSLv3 support which means the negotiation fails.
Once you install the package, if you are still seeing the same error, make sure your script is not forcing the use of Crypt::SSLeay. That is, make sure none of the following appears anywhere in your script:
use Net::HTTPS;
$Net::HTTPS::SSL_SOCKET_CLASS = 'Net::SSL';
or
local $ENV{PERL_NET_HTTPS_SSL_SOCKET_CLASS} = 'Net::SSL';
or
use Net::SSL;
If you are still running into problems, make sure there is no PERL_NET_HTTPS_SSL_SOCKET_CLASS environment variable in the script's run time environment.
Also, try
$ /var/hvmail/libexec/perl -MNet::SSLeay -le 'print $Net::SSLeay::VERSION'
and
$ /var/hvmail/libexec/perl -MLWP::Protocol::https -le 'print $LWP::Protocol::https::VERSION`'
and report the output.
I suspect the issue is that the new packages were installed for the system's perl whereas it seems you may have a separate perl.
If that is the case, you should install each package individually using /var/hvmail/libexec/perl. For example:
$ curl -O https://cpan.metacpan.org/authors/id/M/MI/MIYAGAWA/App-cpanminus-1.7043.tar.gz
$ tar xvf App-cpanminus-1.7043.tar.gz
$ cd App-cpanminus-1.7043
$ /var/hvmail/libexec/perl Makefile.PL
$ make install
Figure out where cpanm was installed. I am hoping /var/hvmail/libexec.
$ /var/hvmail/libexec/cpanm LWP::Protocol::https
See also Updating all outdated Perl modules, but that may be risky on a production. Still, installing App::cpanoutdated, and seeing how outdated your Perl modules are might be useful
Now, keep in mind that tinkering with a production install like this is risky. Make sure you have a way to undo changes in case something goes wrong.
Finally, note that OpenSSL 1.0.1 versions are no longer supported:
With regards to current and future releases the OpenSSL project has adopted the following policy:
Version 1.1.0 will be supported until 2018-08-31.
Version 1.0.2 will be supported until 2019-12-31 (LTS).
Version 1.0.1 is no longer supported.
Version 1.0.0 is no longer supported.
Version 0.9.8 is no longer supported.
There is no need for ''.HANDLER_URL. It looks ugly, and HANDLER_URL is fine
You don't explain what is in $row or what the POST call requires, but it looks like this
my $req = POST ''.HANDLER_URL, [ %$row ];
my $res = $ua->request($req);
should be
my $res = $ua->post(HANDLER_URL, $row);
I try to convert odt-Files to doc-Files using OpenOffice. Installed Version is 3.1.1 and can't be changed at the moment. Perl Version is 5.18.
The Perl-module OpenOffice::UNO is used for this conversion. Unfortunately in newer Versions of OpenOffice/LibreOffice do not support Perl anymore.
The Script calls OpenOffice headless using xvfb.
Here is the code used:
`# Launch OpenOffice.org as a server
$ ooffice \
"-accept=socket,host=localhost,port=8100;urp;StarOffice.ServiceManager"
use OpenOffice::UNO;
# connect to the OpenOffice.org server
$uno = OpenOffice::UNO->new;
$cxt = $uno->createInitialComponentContext('file:///.../path/perluno');
$sm = $cxt->getServiceManager;
$resolver = $sm->createInstanceWithContext
("com.sun.star.bridge.UnoUrlResolver", $cxt);
$rsm = $resolver->resolve
("uno:socket,host=localhost,port=8100;urp;StarOffice.ServiceManager");
# get an instance of the Desktop service
$rc = $rsm->getPropertyValue("DefaultContext");
$desktop = $rsm->createInstanceWithContext("com.sun.star.frame.Desktop", $rc);
.....`
On the last included line to create $desktop i get following Error message:
terminate called after throwing an instance of 'com::sun::star::container::NoSuchElementException'
Is there any way to fix this problem? Tried to understand the Code of the UNO-interface, especially UNO.xs but there has not been any information about the call "createInstanceWithContext".
Looking through the OpenOffice-documentation does not provide any information about this either.
It would also help just to get the complete java error message, to make sure what element is missing.
The file "perluno" has the content:
[Bootstrap]
UNO_TYPES=/usr/lib64/openoffice.org//program/types.rdb
UNO_SERVICES=/usr/lib64/openoffice.org//program/services.rdb
I have a very simple script (on a SLES11 system) to send a http1.1 request to a server. This worked fine for a long time. Since a couple of days it stopped working. I have no idea why. After some investigation I found out forcing the script to use http1.0 it is working again. I have no idea why. And I want to know why it is not working in its default http1.1 mode.
I already enabled debugging as much as I know it (see code below). But I do not see anything - my script just hangs. There is no network activity. I neiter see that a socket will be openend (netstat -a), nor I see any incoming traffic form my client on the server. (A "telnet myserver myport" works fine.)
Can someone please help me how to track down this problem? How can I enable more debug to see where the real problem is?
#!/usr/bin/perl -w
use strict;
use warnings;
use Data::Dumper;
use HTTP::Request::Common;
use LWP::Debug qw(+);
use LWP::UserAgent;
# Workaround: forcing http1.0 instead of using http1.1, it works again!
use LWP::Protocol::https10 ();
LWP::Protocol::implementor('https', 'LWP::Protocol::https10');
# EO workaround
my $ua = LWP::UserAgent->new;
$ua->ssl_opts( verify_hostname => 0 );
my $response = $ua->request(
POST 'https://myuser:mypassword#myserver:8888/service/myservice',
Connection => 'close', # Edit: added, see comments below
Content_Type => 'text/xml',
Content => '... my content ...'
);
$ua->request() does not return! I need to kill/^C the script!
Edit: ok, no one seems to have an idea how to continue. So I startet debugging it using perl debugger.
LWP::UserAgent::post(/usr/lib/perl5/site_perl/5.10.0/LWP/UserAgent.pm:418):
418: return $self->request( HTTP::Request::Common::POST( #parameters ), #suff );
So I can see it is not returning from request().
Anyway, due to the fact that LWP::UserAgent calls HTTP::Request::Common I changed my example code above back to use HTTP::Request::Common to skip that step while debugging.
Ok... New result:
LWP::Protocol::implementor(/usr/lib/perl5/site_perl/5.10.0/Net/HTTPS.pm:26):
26: eval { require IO::Socket::SSL; };
Inside request() it hangs at IO::Socket::SSL. Means, this script it enough for further debug:
#!/usr/bin/perl
require IO::Socket::SSL;
This statement does not come back.
Further down, inside IO::Socket::SSL it hangs at:
IO::Socket::SSL::CODE(0x1274370)(/usr/lib/perl5/site_perl/5.10.0/IO/Socket/SSL.pm:92):
92: Net::SSLeay::SSLeay_add_ssl_algorithms();
Ahhhh! There is already a bugreport concerning this issue: Net-SSLeay hangs on Suse 11 P2 pointing to Bug #81575 that says:
I enountered this same issue on SLES 11 SP2, which has openssl-0.9.8j installed.
An upgrade to openssl-0.9.8r did resolve the problem. ...
The packages for 0.9.8r can be found in this repository:
http://download.opensuse.org/repositories/security:/fips/
Guess that's it!
There is already a bugreport concerning this issue: Net-SSLeay hangs on Suse 11 P2 pointing to Bug #81575 that says:
I enountered this same issue on SLES 11 SP2, which has openssl-0.9.8j installed. An upgrade to openssl-0.9.8r did resolve the problem. ... The packages for 0.9.8r can be found in this repository: http://download.opensuse.org/repositories/security:/fips/
Guess that's it!
I am having difficulty using perl to visit a website via TOR if it is an https site but not if it is an http site.
#!/usr/bin/perl
use strict;
use WWW::Mechanize;
use LWP::Protocol::socks;
use LWP::Protocol::https;
use utf8;
my $mech = WWW::Mechanize->new(timeout => 60*5);
$mech->proxy(['http', 'https'], 'socks://localhost:9150');
$mech->get("https://www.google.com");
I am receiving the error message: Error GETing https://www.google.com: Status read failed: Bad file descriptor at line 10," where line i10 is the last line of the program.
In the TOR browser, I can successfully view: "https://www.google.com" with a port of 9150.
I am using ActivePerl 5.16.2; Vadalia 0.2.21 and Tor 0.2.3.25.
I have a Windows machine and my primary internet browser is Mozilla.
I have tried installing packages with the commands:
cpan LWP::UserAgent
ppm install LWP::Protocol::https
cpan LWP::Protocol::https
ppm install LWP::Protocol::socks
cpan LWP::Protocol::socks
ppm install Mozilla::CA
ppm install IO::Socket::SSL
ppm install Crypt::SSLeay
cpan Crypt::SSLeay
Thank you for any help! Please let me know whether there is any further information that I can provide.
Time ago, i'd found the way to go throught https sites with Tor using WWW::Curl::Easy to fetch those kind of sites, because using LWP i found the same problems.
After that i save all html in files and parsing them using WWW::Mechanzie or HTML::TreeBuilder.
If you want more interactivity with site like post forms , etc. This solutions may be more tedious because you'll need to interact with curl.
package Curl;
use warnings;
use WWW::Curl::Easy;
use WWW::UserAgent::Random;
my $curl = WWW::Curl::Easy->new;
my $useragent = rand_ua("browsers");
my $host = 'localhost';
my $port = '9070';
my $timeout = '20';
my $connectTimeOut= '20';
&init;
sub get
{
my $url = shift;
$curl->setopt(CURLOPT_URL, $url);
my $response_body;
$curl->setopt(CURLOPT_WRITEDATA,\$response_body);
my $retcode = $curl->perform;
if ($retcode == 0) {
print("Transfer went ok Http::Code = ".$curl->strerror($retcode)."\n");
my $response_code = $curl->getinfo(CURLINFO_HTTP_CODE);
# judge result and next action based on $response_code
return \$response_body;
} else {
# Error code, type of error, error message
print("An error happened: $retcode ".$curl->strerror($retcode)." ".$curl->errbuf."\n");
return 0;
}
}
sub init
{
#setejem el proxy
$curl->setopt(CURLOPT_PROXY,"$host:".$port);
$curl->setopt(CURLOPT_PROXYTYPE,CURLPROXY_SOCKS4);
#posem les altres dades
$curl->setopt(CURLOPT_USERAGENT, $useragent);
$curl->setopt(CURLOPT_CONNECTTIMEOUT, $connectTimeOut);
$curl->setopt(CURLOPT_TIMEOUT, $timeout);
$curl->setopt(CURLOPT_SSL_VERIFYPEER,0);
$curl->setopt(CURLOPT_HEADER,0);
}
Hope this will help you!
Maybe the proxy that you are using is already an HTTPS proxy (ie. CONNECT proxy). In that case this should work (untested):
#!/usr/bin/perl
use strict;
use WWW::Mechanize;
use LWP::Protocol::socks;
use LWP::Protocol::https;
use utf8;
my $mech = WWW::Mechanize->new(timeout => 60*5);
$mech->proxy(['http'], 'socks://localhost:9150');
$mech->proxy(['https'], 'https://localhost:9150'); ### <-- make https go over https-connect proxy
$mech->get("https://www.google.com");
I cannot find the origin but I fought with that a long time ago. Basically the problem I had was with the imlpementation that LWP::UserAgent used for the https requests.
Possibly this question can help you: How do I force LWP to use Crypt::SSLeay for HTTPS requests?
I'm trying to script some interaction between a Linux box and an ESXi host, using the VMware Perl library (which appears to call a SOAP service on the ESXi host).
I am getting an error for which I can't find a solution: Undefined subroutine &LWP::Protocol::https::Socket::can_read called at /usr/local/share/perl5/LWP/Protocol/http.pm line 22
I am currently running Net-HTTP-6.03. I did try with Net-HTTP-6.05 but it makes the web-based calls very slow (and subsequent ones also eventually fail, instead with about 1.2MB of XML as the error message) -- which various forums suggest is only resolvable by reverting to 6.03 (via cpan install GAAS/Net-HTTP-6.03.tar.gz).
The crux of my code is as follows:
use VMware::VIRuntime;
my $context = {
options => ParseOptions(), # result of using GetOpts::Long to parse command-line
session => undef
};
$ENV{PERL_NET_HTTPS_SSL_SOCKET_CLASS} = "Net::SSL"; # Suppresses SSL_VERIFY_NONE
if ($context->{options}->{'ignore-ssl-errors'}) {
$ENV{PERL_LWP_SSL_VERIFY_HOSTNAME} = 0; # Ignores certificate errors
}
## next line fails
$context->{session} = Vim::login(
service_url => $context->{options}->{url},
username => $context->{options}->{username},
password => $context->{options}->{password}
);
Is there something else I (can|need to) do to get this working?
I've got it. I needed to reinstall Bundle::LWP from an older version, not just Net-HTTP. Without it, I had LWP v6.05 trying to talk to Net-HTTP v6.03.
Once I realised that LWP == libwww-perl (yes, I'm that new), I found the overall package I needed:
cpan install GAAS/libwww-perl-6.03.tar.gz
Reproduced with LWP 6.05 and Net::HTTP 6.01, which is the cocktail that MacPorts installs at the time of this writing. Upgrading Net::HTTP to 6.06 (current) solved it:
sudo cpan Net::HTTP