Using LWP with SSL and client certificates - perl

I'm porting an application from PHP/cURL to Perl and LWP::UserAgent. I need to do a POST request to a web server and provide a client certificate and key file. The PHP code I'm trying to replicate is this:
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_SSLCERT, "/path/to/certificate.pem");
curl_setopt($ch, CURLOPT_SSLKEY, "/path/to/private.key");
curl_setopt($ch, CURLOPT_SSLKEYPASSWD, "secretpassword");
And here's my Perl code:
my $ua = LWP::UserAgent->new();
$ua->ssl_opts(
SSL_verify_mode => 0,
SSL_cert_file => '/path/to/certificate.pem',
SSL_key_file => "/path/to/private.key",
SSL_passwd_cb => sub { return "secretpassword"; }
);
The PHP code successfully connects to the server but the Perl code fails with:
SSL read error error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure
I can't figure out what I'm missing.

sub send_command(){
my $command = shift;
my $parser = XML::LibXML->new('1.0','utf-8');
print color ("on_yellow"), "SEND: ", $command, color ("reset"), "\n";
# Create a request
my $req = HTTP::Request->new( GET => $Gateway.$command );
# Pass request to the user agent and get a response back
my $res;
eval {
my $ua;
local $SIG{'__DIE__'};
$ua = LWP::UserAgent->new(); # или
$ua->ssl_opts( #$key => $value
SSL_version => 'SSLv3',
SSL_ca_file => '/ca.pem',
#SSL_passwd_cb => sub { return "xxxxx\n"; },
SSL_cert_file => '/test_test_cert.pem',
SSL_key_file => '/test_privkey_nopassword.pem',
); # ssl_opts => { verify_hostname => 0 }
$ua->agent("xxxxxx xxxx_tester.pl/0.1 ");
$res = $ua->request($req);
};
warn $# if $#;
# Check the outcome of the response
if ( $res->is_success ) {
open xxxLOG, ">> $dir/XXXX_tester.log";
my $without_lf = $res->content;
$without_lf =~ s/(\r|\n)//gm;
print PAYLOG $without_lf,"\n";
close PAYLOG;
}
else {
return $res->status_line;
}
print color ("on_blue"), "RESPONSE: ", color ("reset"), respcode_color($res->content), color ("reset"),"\n\n";
return $res->content;
}

The answer from emazep above solved my problem. I'm using the sample Perl code from UPS to connect to their Rate service via XML. From my tests, this will work any time LWP::UserAgent is being called without arguments that you can control directly, which makes it handy if you're using some other module which makes calls to LWP for you. Just use Net::SSL (in addition to whatever packages have already used LWP) and set a few environment variables:
...
use Net::SSL;
$ENV{HTTPS_VERSION} = 3;
$ENV{PERL_LWP_SSL_VERIFY_HOSTNAME} = 0;
my $browser = LWP::UserAgent->new();
...
That's it! You shouldn't even need to specify the path to your server's root certificate with $ENV{PERL_LWP_SSL_CA_FILE}.

Indeed this is a messy bit. Depending on your setup LWP::UserAgent may use one of (at least) two SSL modules to handle the SSL connection.
IO::Socket::SSL
Net::SSL
The first one should be the default for newer versions of LWP::UserAgent. You can test which of these are installed by running the standard command in a terminal for each module:
perl -e 'use <module>;'
IO::socket::SSL requires the SSL configuration with the ssl_opts as in your example.
Net::SSL requires the SSL configuration in environment variables as in goddogsrunnings answer.
Personally I fall in the second category and had good inspiration from the Crypt::SSLeay page. Particularly the section named "CLIENT CERTIFICATE SUPPORT ".

Related

Perl LWP - need get timings: DNS resolving time, ssl connect, etc

I am using LWP to check website accessibility for HTTPS.
But, sometimes there are delays of 3000 ms
sub get_url{
my $url = shift;
my $browser = LWP::UserAgent->new;
$browser->timeout($alltimeout);
eval {
local $SIG{ALRM} = sub { die "alarm\n" }; # NB: \n required
alarm $alltimeout;
$response = $browser->get($url,
'User-Agent' => 'CHECKER (Win98; U)',
'Accept-Charset' => 'utf-8',
);
alarm 0;
};
if ($#) {print "$#"; return "Timeout $alltimeout - error!"}
if ($response->is_success){$resp[0]=200; return $response->content;}
else {return "ERROR".$response->status_line}
}
i want to check separately: DNS resolving time, ssl connect time, etc for HTTPS.
LWP, and many other modules you'll use, aren't instrumented to produce this sort of tracing, and certainly not in any coherent or consistent fashion. You'd have to delve into the individual modules to provide your own hooks for this.
Do you see the same delays with other browsers, such as curl or wget?

LWP::UserAgent Can't Post with TLS1.1

Getting 500 handshaker error:443 over https. The host service I am sending XML to does not support TLS 1.2, they do support 1.0 and 1.1. Currently using LWP 6.03 on CentOS 6. Using the code below they claim I am still sending using TLS1.2
use LWP::UserAgent;
$ua = LWP::UserAgent->new(ssl_opts => { verify_hostname => 0,SSL_version => 'SSLv23:!TLSv12' });
$req = HTTP::Request->new(GET => 'https://secure-host-server');
$res = $ua->request($req);
if ($res->is_success) {
print $res->content;
} else {
print "Error: " . $res->status_line . "\n";
}
Is it possible to print the TLS version as it is sent to the host? Anything I can do to verify I am using TLS1.1?
Setting SSL_version via LWP::UserAgent would not work. I tried countless methods to try to get my code to send XML via TLSv1 without luck, The following code did the trick.
use Net::SSLGlue::LWP;
use IO::Socket::SSL;
my $context = new IO::Socket::SSL::SSL_Context(
SSL_version => 'tlsv1',
SSL_verify_mode => Net::SSLeay::VERIFY_NONE(),
);
IO::Socket::SSL::set_default_context($context);
use LWP::UserAgent;
use HTTP::Request::Common;

Using Perl LWP::UserAgent I get a response on port 443 but not from port 8443

I am using LWP::UserAgent to check the response from a server. I get a response from port 443 but I am not able to get any response from port 8443.
When I use cURL for Windows I get a response code from both ports.
Please help me.
This example program (adapted from perldoc lwpcook) shows how to connect with a different port
It also allows turning off of the SSL verify, in case you have a home brew cert that is causing a problem
#!/usr/bin/perl
$port = $ARGV[1] || 443;
$host = $ARGV[0] || 'pause.perl.org';
$verify =$ARGV[2] || 0;
use LWP::UserAgent;
$ua = LWP::UserAgent->new(ssl_opts => { verify_hostname => $verify});;
#$ua->agent("$0/0.1 " . $ua->agent);
$ua->agent("Mozilla/8.0"); # pretend we are very capable browser
$req = HTTP::Request->new( GET => "https://$host:$port" );
$req->header( 'Accept' => 'text/html' );
# send request
$res = $ua->request($req);
# check the outcome
if ( $res->is_success ) {
print $res->decoded_content;
}
else {
print "Error: " . $res->status_line . "\n";
}

Perl JSON::RPC::Client Not a HASH reference Error

When i call remote echo procedure my call reachs to rpc server which is written in python with txjsonrpc module. but this script says:
malformed JSON string, neither array, object, number, string or atom, at character offset 0 (before "read failed: at /us...") at /usr/local/lib/perl/5.14.2/Moose/Meta/Method/Delegation.pm line 110
Any idea? Code is here:
use LWP::UserAgent;
use JSON::RPC::LWP;
use Net::SSL ();
use JSON::RPC::Client;
$ENV{HTTPS_DEBUG} = 1;
# CA cert peer verification
$ENV{HTTPS_CA_FILE} = 'ca_file';
$ENV{HTTPS_CA_DIR} = 'ca_dir';
# Client PKCS12 cert support
$ENV{HTTPS_PKCS12_FILE} = 'pkcs12 cert';
$ENV{HTTPS_PKCS12_PASSWORD} = 'password';
my $ua = eval { LWP::UserAgent->new() }
or die "Could not make user-agent! $#";
$ua->ssl_opts( verify_hostname => 0);
print "JSON RPC CONNECTION\n";
my $rpc = JSON::RPC::Client->new(
ua => $ua,
version => '2.0'
);
my $result = $rpc->call( 'https://...', {method=>'echo', params=>['param']});
print $result->result, "\n";
What version of JSON::RPC::Client are you using? If it is the most current one, that error is happening because the server is returning something other than a JSON object.

Extracting cookies from a Mojolicious user agent response

I started using the Mojolicious library for testing and everything was working fine in until I tried to extract cookies from a response.
I've tried several variants of:
$ua = Mojo::UserAgent->new();
$ua->on( error => sub { my ($ua, $error) = #_; say "This looks bad: $error"; } );
$ua->max_redirects(1)->connect_timeout(10)->request_timeout(20);
$ua->cookie_jar(Mojo::CookieJar->new);
# ... later ...
my $tx = $ua->get($url);
my $jar = $ua->cookie_jar->extract($tx); # This is undef
I can however extract the cookies via LWP::UserAgent. However, LWP has several different issues that make that option unworkable for now. Just for a comparison here is the LWP code that does extract the cookies.
my $lwp = LWP::UserAgent->new(cookie_jar => {}, timeout => 20, max_redirect => 1);
push #{ $lwp->requests_redirectable }, 'POST';
my $response = $lwp->get($url);
die $response->status_line unless $response->is_success;
$lwp->cookie_jar->scan(\&ScanCookies);
sub ScanCookies {
my ($version, $key, $value) = #_;
say "$key = $value";
}
So I know that I have the $url etc. correct.
Edit: I should mention that i'm using strawberry 5.14
Edit2: I should also mention that the cookies are getting into the user agent for sure, because the session ID is getting handled properly. Unfortunately, I have a need to access another cookie (for testing the site) and I just don't seem to be able the get the right incantation to access them... saying that I believe this to be a programmer problem and nothing more.
Use this:
$tx->res->cookies