Here is my code:
my $lwpcurl = LWP::Curl->new(CURLOPT_SSL_VERIFYHOST => 0,CURLOPT_SSL_VERIFYPEER=>0);
my $content;
$content = $lwpcurl->get($url);
I am getting this error:
`SSL peer certificate was not ok`
LWP::Curl doesn't accept CURLOPT_SSL_VERIFYHOST/CURLOPT_SSL_VERIFYPEER parameters for it's constructor!
Use LWP::Protocol::Net::Curl instead:
use LWP::Protocol::Net::Curl ssl_verifyhost => 0, ssl_verifypeer => 0;
use LWP::UserAgent;
my $ua = LWP::UserAgent->new;
my $content = $ua->get($url);
Note that LWP::Protocol::Net::Curl alters the default LWP::UserAgent behavior, so you still use $ua = LWP::UserAgent->new, while it uses libcurl internally.
Related
I can connect to a remote url using the proxy from tsocks in the following way:
tsocks telnet host port
How can I do the same using the Perl's LWP::UserAgent module?
I've been so far trying it like this, but it doesn't work:
use strict;
use warnings;
use v5.16;
use LWP::UserAgent;
use HTTP::Request::Common;
use Data::Dumper;
#my $ua = LWP::UserAgent->new();
my $ua = LWP::UserAgent->new(timeout => 10,
ssl_opts => {
#verify_hostname => 0,
verify_hostname => 0,
SSL_verify_mode => '0x01',
SSL_version => 'SSLv23:!SSLv3:!SSLv2',
}
);
$ua->proxy(['http', 'https' ], 'https://proxy_host:proxy_port' );
my $request = GET ( 'https://remote_url', Accept => 'application/json' );
$request->authorization_basic( 'username', 'password' );
say $request->as_string();
my $response = $ua->request( $request );
say $response->as_string();
By the way, I don't have socks installed on this server. So I'll need to do it without them.
Thanks!
LWP::UserAgent and SOCKS
You may use LWP::Protocol::socks perl package make LWP::UserAgent SOCKS capable.
IMHO it is much better way for your own perl scripts.
tsocks may be better for legacy perl scripts you do not want to modify.
I found a solution which might be very specific for my case:
Before running the script, I just call tsocks like this:
tsocks script_name.pl
Getting 500 handshaker error:443 over https. The host service I am sending XML to does not support TLS 1.2, they do support 1.0 and 1.1. Currently using LWP 6.03 on CentOS 6. Using the code below they claim I am still sending using TLS1.2
use LWP::UserAgent;
$ua = LWP::UserAgent->new(ssl_opts => { verify_hostname => 0,SSL_version => 'SSLv23:!TLSv12' });
$req = HTTP::Request->new(GET => 'https://secure-host-server');
$res = $ua->request($req);
if ($res->is_success) {
print $res->content;
} else {
print "Error: " . $res->status_line . "\n";
}
Is it possible to print the TLS version as it is sent to the host? Anything I can do to verify I am using TLS1.1?
Setting SSL_version via LWP::UserAgent would not work. I tried countless methods to try to get my code to send XML via TLSv1 without luck, The following code did the trick.
use Net::SSLGlue::LWP;
use IO::Socket::SSL;
my $context = new IO::Socket::SSL::SSL_Context(
SSL_version => 'tlsv1',
SSL_verify_mode => Net::SSLeay::VERIFY_NONE(),
);
IO::Socket::SSL::set_default_context($context);
use LWP::UserAgent;
use HTTP::Request::Common;
I'm porting an application from PHP/cURL to Perl and LWP::UserAgent. I need to do a POST request to a web server and provide a client certificate and key file. The PHP code I'm trying to replicate is this:
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_SSLCERT, "/path/to/certificate.pem");
curl_setopt($ch, CURLOPT_SSLKEY, "/path/to/private.key");
curl_setopt($ch, CURLOPT_SSLKEYPASSWD, "secretpassword");
And here's my Perl code:
my $ua = LWP::UserAgent->new();
$ua->ssl_opts(
SSL_verify_mode => 0,
SSL_cert_file => '/path/to/certificate.pem',
SSL_key_file => "/path/to/private.key",
SSL_passwd_cb => sub { return "secretpassword"; }
);
The PHP code successfully connects to the server but the Perl code fails with:
SSL read error error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure
I can't figure out what I'm missing.
sub send_command(){
my $command = shift;
my $parser = XML::LibXML->new('1.0','utf-8');
print color ("on_yellow"), "SEND: ", $command, color ("reset"), "\n";
# Create a request
my $req = HTTP::Request->new( GET => $Gateway.$command );
# Pass request to the user agent and get a response back
my $res;
eval {
my $ua;
local $SIG{'__DIE__'};
$ua = LWP::UserAgent->new(); # или
$ua->ssl_opts( #$key => $value
SSL_version => 'SSLv3',
SSL_ca_file => '/ca.pem',
#SSL_passwd_cb => sub { return "xxxxx\n"; },
SSL_cert_file => '/test_test_cert.pem',
SSL_key_file => '/test_privkey_nopassword.pem',
); # ssl_opts => { verify_hostname => 0 }
$ua->agent("xxxxxx xxxx_tester.pl/0.1 ");
$res = $ua->request($req);
};
warn $# if $#;
# Check the outcome of the response
if ( $res->is_success ) {
open xxxLOG, ">> $dir/XXXX_tester.log";
my $without_lf = $res->content;
$without_lf =~ s/(\r|\n)//gm;
print PAYLOG $without_lf,"\n";
close PAYLOG;
}
else {
return $res->status_line;
}
print color ("on_blue"), "RESPONSE: ", color ("reset"), respcode_color($res->content), color ("reset"),"\n\n";
return $res->content;
}
The answer from emazep above solved my problem. I'm using the sample Perl code from UPS to connect to their Rate service via XML. From my tests, this will work any time LWP::UserAgent is being called without arguments that you can control directly, which makes it handy if you're using some other module which makes calls to LWP for you. Just use Net::SSL (in addition to whatever packages have already used LWP) and set a few environment variables:
...
use Net::SSL;
$ENV{HTTPS_VERSION} = 3;
$ENV{PERL_LWP_SSL_VERIFY_HOSTNAME} = 0;
my $browser = LWP::UserAgent->new();
...
That's it! You shouldn't even need to specify the path to your server's root certificate with $ENV{PERL_LWP_SSL_CA_FILE}.
Indeed this is a messy bit. Depending on your setup LWP::UserAgent may use one of (at least) two SSL modules to handle the SSL connection.
IO::Socket::SSL
Net::SSL
The first one should be the default for newer versions of LWP::UserAgent. You can test which of these are installed by running the standard command in a terminal for each module:
perl -e 'use <module>;'
IO::socket::SSL requires the SSL configuration with the ssl_opts as in your example.
Net::SSL requires the SSL configuration in environment variables as in goddogsrunnings answer.
Personally I fall in the second category and had good inspiration from the Crypt::SSLeay page. Particularly the section named "CLIENT CERTIFICATE SUPPORT ".
My symptom is that I cannot use a proxy with HTTPS requests with LWP. This seems to be a common problem, and the hints on Google and even here all suggest a work-around for setting the HTTPS_PROXY environment variable for use by Crypt::SSLeay.
My specific problem appears to be that LWP::Protocol::https is loading IO::Socket::SSL rather than Crypt::SSLeay. How can I force Crypt::SSLeay's use instead?
My code:
#!/usr/bin/perl
use strict;
use warnings;
$ENV{HTTPS_PROXY} = 'http://10.0.3.1:3128';
use LWP::UserAgent;
my $ua = LWP::UserAgent->new();
my $req = HTTP::Request->new('GET','https://www.meritrustcu.org/');
my $res = $ua->request($req);
print "$_\n" for grep { $_ =~ /SSL/ } keys %INC;
And it's output, showing that Crypt::SSLeay is not being used:
Net/SSLeay.pm
IO/Socket/SSL.pm
/usr/lib/perl5/auto/Net/SSLeay/autosplit.ix
/usr/lib/perl5/auto/Net/SSLeay/set_proxy.al
/usr/lib/perl5/auto/Net/SSLeay/randomize.al
Simply adding an explicit use Crypt::SSLeay to my script has proven ineffective. It loads the module, but it continues to load IO::Socket::SSL, and use it for the HTTPS requests.
Try this:
use strict;
use warnings;
use Net::SSL (); # From Crypt-SSLeay
BEGIN {
$Net::HTTPS::SSL_SOCKET_CLASS = "Net::SSL"; # Force use of Net::SSL
$ENV{HTTPS_PROXY} = 'http://10.0.3.1:3128';
}
use LWP::UserAgent;
my $ua = LWP::UserAgent->new();
my $req = HTTP::Request->new('GET','https://www.meritrustcu.org/');
my $res = $ua->request($req);
print "$_\n" for grep { $_ =~ /SSL/ } keys %INC;
I don't have a suitable proxy, so I haven't tried it myself.
This is what I did to get LWP and SOAP::Lite to work with our proxy at GE. This was after much digging on CPAN, google etc. I finally figured it out after running the test script in the Crypt::SSLeay package called net_ssl_test and it was able to connect through the proxy. The key is forcing Crypt::SSLeay to use Net::SSL as was mentioned above. This is not documented very well on CPAN however.
use LWP::UserAgent;
# override HTTPS setting in LWP to work with a proxy
$ENV{PERL_NET_HTTPS_SSL_SOCKET_CLASS} = "Net::SSL";
$ENV{PERL_LWP_SSL_VERIFY_HOSTNAME} = 0;
$ENV{HTTPS_PROXY} = 'http-proxy.ae.ge.com:80';
$ENV{HTTPS_PROXY_USERNAME} = 'username';
$ENV{HTTPS_PROXY_PASSWORD} = 'password';
$ua = new LWP::UserAgent;
# make a https request
my $req = HTTP::Request->new(GET => 'https://mail.google.com/');
my $res = $ua->request($req);
print $res->as_string;
The following code ...
my $user_agent = LWP::UserAgent->new;
my $request = HTTP::Request->new(GET => $url);
my $response = $user_agent->request($request);
if ($response->is_success) {
print "OK\n";
} else {
die($response->status_line);
}
.. will fail with ..
500 Can't connect to <hostname> (Bad hostname '<hostname>')
.. if the hostname in $url is an IPv6 only address (that is: presence of an AAAA record, but no A record).
My questions are:
How do I enable IPv6 support in LWP?
How do I configure LWP's settings for "prefer-IPv4-over-IPv6" (A vs. AAAA) / "prefer-IPv6-over-IPv4" (AAAA vs. A)?
It looks like you just need to use Net::INET6Glue::INET_is_INET6. To quote its example:
use Net::INET6Glue::INET_is_INET6;
use LWP::Simple;
print get( 'http://[::1]:80' );
print get( 'http://ipv6.google.com' );
I believe you'll have to change the module to use the IPV6 net module. By default it does not have this enabled: http://eintr.blogspot.com/2009/03/bad-state-of-ipv6-in-perl.html. I don't believe there is something as simple as "prefer-ipv6"
Debian Wheezy (perl 5.14)
Work nice:
use LWP::Simple;
print get( 'http://ip6-localhost:80' );
Not working (1)
use LWP::Simple;
print get( 'http://[::1]:80' );
Not working (2) [Return: Bad hostname]
use LWP::Simple;
$ua = new LWP::UserAgent();
my $req = new HTTP::Request("GET", "http://[::1]/");
my $res = $ua->request($req);
Not working (3) [Return: Connection refused]
use Net::INET6Glue::INET_is_INET6;
use LWP::Simple;
$ua = new LWP::UserAgent();
my $req = new HTTP::Request("GET", "http://[::1]/");
my $res = $ua->request($req);
Soo, if you don't need IPv6 address in http request, it's fine. :(