I am trying to call a web service using ssl. It gives following error:
500 SSL negotiation failed:
I searched forums and applied offered methods but none of them worked.
2 methods I applied are listed below:
1-) setting enviroment before call:
$ENV{PERL_LWP_SSL_VERIFY_HOSTNAME} = 0;
2-) passing parameter ssl_opts => [ SSL_verify_mode => 0 ] to proxy:
my $soap = SOAP::Lite
-> on_action( .... )
-> uri($uri)
-> proxy($proxy, ssl_opts => [ SSL_verify_mode => 0 ])
-> ns("http://schemas.xmlsoap.org/soap/envelope/","soapenv")
-> ns("http://tempuri.org/","tem");
$soap->serializer()->encodingStyle(undef);
Is there any solution for this?
... Connection reset by peer at /usr/lib/perl5/vendor_perl/5.8.5/i386-linux-thread-multi/Net/SSL.pm line 145
You are running a very old version of Perl (from 2004) together with an old version of the SSL libraries (i.e. Crypt::SSLeay instead of IO::Socket::SSL) and my guess is that this goes together with using a very old version of the OpenSSL libraries for TLS support. This combination means that there is no support for SNI, no support for TLS 1.2 and no support for ECDHE ciphers. Many modern servers need at least one of these things supported. But connection reset by peer could also mean that some firewall is blocking connections or that there is no server listening on the endpoint you've specified. Or it could mean that the server is expecting you to authorize with a client certificate. Hard to tell but with a packet capture of the connection one might provide more information. And, if the URL is publicly accessible publishing it would help too in debugging the problem.
Related
I am getting below error while running my script, I am on AIX
Reading configuration data /ecmsq1vg1/home1/aradmin/.ssh/config
Reading configuration data /etc/ssh_config
Connecting to www.mftcatapp.firstdataclients.com, port 22.
Remote version string: SSH-2.0-Sun_SSH_1.1.8
Remote protocol version 2.0, remote software version Sun_SSH_1.1.8
Net::SSH::Perl Version 2.14, protocol version 2.0.
No compat match: Sun_SSH_1.1.8.
Connection established.
Sent key-exchange init (KEXINIT), waiting for response.
No matching mac found: client hmac-sha2-512-etm#openssh.com,hmac-sha2-256-etm#openssh.com,hmac-sha2-512,hmac-sha2-256 server hmac-sha1 at /apps/perl/lib/site_perl/5.18.1/aix-thread-multi/Net/SSH/Perl/SSH2.pm line 273
Can anyone help me out why i am facing the above error?
perldoc Net::SSH::Perl:
Integrity checking is performed by the hmac-sha2-256, hmac-sha2-512, hmac-sha2-256-etm#openssh.com, or hmac-sha2-512-etm#openssh.com algorithms. The deprecated hmac-sha1 or hmac-md5 algorithms are available but not enabled by default. Many older SSH server installations still use hmac-sha1 as the main accepted MAC algorithm. To enable this, use the following options parameter:
options => [ "MACs +hmac-sha1" ]
So either configure your server not to use hmac-sha1 for integrity checking or tell your script to accept hmac-sha1.
I need to issue a series of parallelized web requests from the same server to a specific domain, but control what IP address these requests actually go to. Originally, I came up with a scheme where I would request the IP I wanted specifically, and then manually set the Host: www.example.com header on the request, and use a series of handlers to make sure that redirects issued followed the same pattern.
This seemed to work for some time, but lately I've been having trouble with redirects to HTTPS. The handshake will fail, and the request in turn. I have tried disabling SSL verification in a variety of ways, including:
local $ENV{ PERL_LWP_SSL_VERIFY_HOSTNAME } = 0;
local $ENV{ HTTPS_DEBUG } = 1;
$ua->ssl_opts(
SSL_ca_file => Mozilla::CA::SSL_ca_file(),
verify_hostname => 0,
SSL_verify_mode => 0x00,
);
IO::Socket::SSL::set_ctx_defaults(
SSL_verifycn_scheme => 'www',
SSL_verify_mode => 0,
);
I have also tried using LWP::UserAgent::DNS::Hosts to solve the problem, but it persists.
<edit>I should note that the reason why turning off peer validation for SSL is not solving the problem is likely because for some reason requesting this way is actually causing the handshake to fail, not failing on a validation point.</edit>
One thing that works is making an entry in /etc/hosts to point the domain at the appropriate IP, however this is not practical, because I may need to run tens, or hundreds, of tests, in parallel, on the same domain.
Is there a way to emulate the functionality of adding an entry to /etc/hosts that does not involve requesting the IP specifically and overriding the Host: ... HTTP header?
EDIT: SSL Debug Info
DEBUG: .../IO/Socket/SSL.pm:1914: new ctx 140288835318480
DEBUG: .../IO/Socket/SSL.pm:402: socket not yet connected
DEBUG: .../IO/Socket/SSL.pm:404: socket connected
DEBUG: .../IO/Socket/SSL.pm:422: ssl handshake not started
DEBUG: .../IO/Socket/SSL.pm:455: not using SNI because hostname is unknown
DEBUG: .../IO/Socket/SSL.pm:478: set socket to non-blocking to enforce timeout=180
DEBUG: .../IO/Socket/SSL.pm:491: Net::SSLeay::connect -> -1
DEBUG: .../IO/Socket/SSL.pm:501: ssl handshake in progress
DEBUG: .../IO/Socket/SSL.pm:511: waiting for fd to become ready: SSL wants a read first
DEBUG: .../IO/Socket/SSL.pm:531: socket ready, retrying connect
DEBUG: .../IO/Socket/SSL.pm:491: Net::SSLeay::connect -> -1
DEBUG: .../IO/Socket/SSL.pm:1388: SSL connect attempt failed with unknown error
DEBUG: .../IO/Socket/SSL.pm:497: fatal SSL error: SSL connect attempt failed with unknown error error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure
DEBUG: .../IO/Socket/SSL.pm:1948: free ctx 140288835318480 open=140288835318480
DEBUG: .../IO/Socket/SSL.pm:1953: free ctx 140288835318480 callback
DEBUG: .../IO/Socket/SSL.pm:1956: OK free ctx 140288835318480
And in the response I get:
Can't connect to redacted.org:443
SSL connect attempt failed with unknown error error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure at /System/Library/Perl/Extras/5.18/LWP/Protocol/http.pm line 51.
It fails equally well on our server (using an older legacy version of Perl, which I will not disclose here as it seems irrelevant).
The server initially responds to a non-HTTPS request with a 301 redirect to the HTTPS site. Then the failure occurs. I will post reproducing code with the specific details of my request removed, but any site which redirects non-HTTPS traffic to HTTPS should suffice.
use IO::Socket::SSL qw/ debug4 /;
use LWP::UserAgent;
use LWP::UserAgent::DNS::Hosts;
use HTTP::Request;
use Mozilla::CA;
use Data::Dumper;
LWP::UserAgent::DNS::Hosts->register_hosts(
'recacted.org' => '127.0.0.1', # no I am not redirecting to loopback in reality, this is anonymized
'www.redacted.org' => '127.0.0.1',
);
LWP::UserAgent::DNS::Hosts->enable_override;
my $ua = LWP::UserAgent->new;
$ua->ssl_opts( SSL_ca_file => Mozilla::CA::SSL_ca_file() );
my $request = HTTP::Request->new(GET => 'http://redacted.org/');
my $response = $ua->request($request);
print $response->content; #Dumper ( $response->is_success ? $response->headers : $response );
Again, that is not the production code, just enough code to reproduce the issue. It doesn't seem to have anything to do with SSL verification, but moreover an inability to negotiate the request, presumably because LWP::UserAgent::DNS::Hosts is doing exactly what I was doing: changing the request target to the desired IP, and then writing the Host: ... header manually. Why this causes the SSL handshake to fail, I do not know.
On my local machine debugging
openssl version -a: 1.0.2j 26 Sep 2016
IO::Socket::SSL->VERSION == 1.966
Net::SSLeay->VERSION == 1.72
On a server of ours
openssl version -a: 1.0.1t 3 May 2016
IO::Socket::SSL->VERSION == 1.76
Net::SSLeay->VERSION == 1.48
Given that it works with an explicit /etc/hosts file but not with just replacing PeerAddr or using LWP::UserAgent::DNS::Hosts this looks like a problem with the SNI extension. This TLS extension is used to provide the TLS server with the requested hostname (similar to the HTTP Host header) so that it can choose the appropriate certificate. If this SNI extension is missing some servers return a default certificate while others throw an error, like in this case.
The fix is to provide the hostname using SSL_hostname in ssl_opts. Such fix could probably also help with LWP::UserAgent::DNS::Hosts, i.e in LWP/Protocol/https/hosts.pm:
12 if (my $peer_addr = LWP::UserAgent::DNS::Hosts->_registered_peer_addr($host)) {
13 push #opts, (
14 PeerAddr => $peer_addr,
15 Host => $host,
16 SSL_verifycn_name => $host,
NEW SSL_hostname => $host, # for SNI
17 );
18 }
I can see here how to set the socket adapter for Zend_Http_Client
http://framework.zend.com/manual/1.12/en/zend.http.client.adapters.html
The examples they give are tls or sslv2.
Does anyone know what the setting is for tls1.2?
I've tried a few but I'm just guessing. I get errors along the lines of:
Unable to find the socket transport "tls1.2" - did you forget to enable it when you configured PHP?'
If I try tls on it's own I get:
Unable to Connect to tls://www.sandbox.paypal.com:443
(For others Googling this is to fix our IPN verification with PayPal which gives the following error on our SSL connection:
Error in cURL request: error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure
)
You were close! Set ssltransport to tlsv1.2.
$config = array(
'adapter' => 'Zend_Http_Client_Adapter_Socket',
'ssltransport' => 'tlsv1.2'
);
$client = new Zend_Http_Client('https://www.sandbox.paypal.com', $config);
$response = $client->request();
echo $response->getStatus();
Figured it out by first checking what Zend_Http_Client_Adapter_Socket uses to send HTTP requests, which turned out to be stream_socket_client(). You can run the stream_get_transports() on your system to view the list of available socket transports.
See SSL/TLS version selection in the OpenSSL changes in PHP 5.6.x migration guide for more examples of how to select specific SSL/TLS versions.
Tested with PHP 5.6 on Ubuntu 14.04 Trusty, which supports TLSv1.2 out of the box.
Fiddler Error Connecting to HTTPS Applications
Fiddler Log:
!SecureClientPipeDirect failed: Authentication failed because the remote party has closed the transport stream. on pipe to (CN=services.bigpond.com, O=DO_NOT_TRUST_BC, OU=Created by http://www.fiddler2.com)
I have followed other posts but no answers
The typical explanation for this message, as documented in many places, is that the client application has not been configured to trust Fiddler's root certificate. As such, the client closes the connection to Fiddler when it sees the untrusted certificate.
http://fiddler2.com/documentation/Configure-Fiddler/Tasks/TrustFiddlerRootCert
In Kestrel I'm using an SSL cert.
I 'downgraded' the TLS protocol in order to get this to work.
This is not something you'd do in production - but in production you shouldn't be using kestrel. I'm not saying this is the best overall config, but this is mainly to show the SslProtocols option.
WebHost.CreateDefaultBuilder(args)
.UseKestrel(options =>
{
options.Listen(IPAddress.Any, 5000); // http:localhost:5000
options.Listen(IPAddress.Any, 44300, listenOptions =>
{
// https://dotnetthoughts.net/enable-http2-on-kestrel/
//listenOptions.Protocols = Microsoft.AspNetCore.Server.Kestrel.Core.HttpProtocols.Http2;
listenOptions.UseHttps(#"S:\WORK\SSL\example.com.pfx", "cert-password", httpsOptions =>
{
httpsOptions.SslProtocols = System.Security.Authentication.SslProtocols.Tls;
});
});
})
.UseStartup<Startup>();
How can I get LWP to verify that the certificate of the server I'm connecting to is signed by a trusted authority and issued to the correct host? As far as I can tell, it doesn't even check that the certificate claims to be for the hostname I'm connecting to. That seems like a major security hole (especially with the recent DNS vulnerabilities).
Update: It turns out what I really wanted was HTTPS_CA_DIR, because I don't have a ca-bundle.crt. But HTTPS_CA_DIR=/usr/share/ca-certificates/ did the trick. I'm marking the answer as accepted anyway, because it was close enough.
Update 2: It turns out that HTTPS_CA_DIR and HTTPS_CA_FILE only apply if you're using Net::SSL as the underlying SSL library. But LWP also works with IO::Socket::SSL, which will ignore those environment variables and happily talk to any server, no matter what certificate it presents. Is there a more general solution?
Update 3: Unfortunately, the solution still isn't complete. Neither Net::SSL nor IO::Socket::SSL is checking the host name against the certificate. This means that someone can get a legitimate certificate for some domain, and then impersonate any other domain without LWP complaining.
Update 4: LWP 6.00 finally solves the problem. See my answer for details.
This long-standing security hole has finally been fixed in version 6.00 of libwww-perl. Starting with that version, by default LWP::UserAgent verifies that HTTPS servers present a valid certificate matching the expected hostname (unless $ENV{PERL_LWP_SSL_VERIFY_HOSTNAME} is set to a false value or, for backwards compatibility if that variable is not set at all, either $ENV{HTTPS_CA_FILE} or $ENV{HTTPS_CA_DIR} is set).
This can be controlled by the new ssl_opts option of LWP::UserAgent. See that link for details on how the Certificate Authority certificates are located. But be careful, the way LWP::UserAgent used to work, if you provide a ssl_opts hash to the constructor, then verify_hostname defaulted to 0 instead of 1. (This bug was fixed in LWP 6.03.) To be safe, always specify verify_hostname => 1 in your ssl_opts.
So use LWP::UserAgent 6; should be sufficient to have server certificates validated.
There are two means of doing this depending on which SSL module you have installed. The LWP docs recommend installing Crypt::SSLeay. If that's what you've done, setting the HTTPS_CA_FILE environment variable to point to your ca-bundle.crt should do the trick. (the Crypt::SSLeay docs mentions this but is a bit light on details). Also, depending on your setup, you may need to set the HTTPS_CA_DIR environment variable instead.
Example for Crypt::SSLeay:
use LWP::Simple qw(get);
$ENV{HTTPS_CA_FILE} = "/path/to/your/ca/file/ca-bundle";
$ENV{HTTPS_DEBUG} = 1;
print get("https://some-server-with-bad-certificate.com");
__END__
SSL_connect:before/connect initialization
SSL_connect:SSLv2/v3 write client hello A
SSL_connect:SSLv3 read server hello A
SSL3 alert write:fatal:unknown CA
SSL_connect:error in SSLv3 read server certificate B
SSL_connect:error in SSLv3 read server certificate B
SSL_connect:before/connect initialization
SSL_connect:SSLv3 write client hello A
SSL_connect:SSLv3 read server hello A
SSL3 alert write:fatal:bad certificate
SSL_connect:error in SSLv3 read server certificate B
SSL_connect:before/connect initialization
SSL_connect:SSLv2 write client hello A
SSL_connect:error in SSLv2 read server hello B
Note that get doesn't die, but it does return an undef.
Alternatively, you can use the IO::Socket::SSL module (also available from the CPAN). To make this verify the server certificate you need to modify the SSL context defaults:
use IO::Socket::SSL qw(debug3);
use Net::SSLeay;
BEGIN {
IO::Socket::SSL::set_ctx_defaults(
verify_mode => Net::SSLeay->VERIFY_PEER(),
ca_file => "/path/to/ca-bundle.crt",
# ca_path => "/alternate/path/to/cert/authority/directory"
);
}
use LWP::Simple qw(get);
warn get("https:://some-server-with-bad-certificate.com");
This version also causes get() to return undef but prints a warning to STDERR when you execute it (as well as a bunch of debugging if you import the debug* symbols from IO::Socket::SSL):
% perl ssl_test.pl
DEBUG: .../IO/Socket/SSL.pm:1387: new ctx 139403496
DEBUG: .../IO/Socket/SSL.pm:269: socket not yet connected
DEBUG: .../IO/Socket/SSL.pm:271: socket connected
DEBUG: .../IO/Socket/SSL.pm:284: ssl handshake not started
DEBUG: .../IO/Socket/SSL.pm:327: Net::SSLeay::connect -> -1
DEBUG: .../IO/Socket/SSL.pm:1135: SSL connect attempt failed with unknown errorerror:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
DEBUG: .../IO/Socket/SSL.pm:333: fatal SSL error: SSL connect attempt failed with unknown errorerror:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
DEBUG: .../IO/Socket/SSL.pm:1422: free ctx 139403496 open=139403496
DEBUG: .../IO/Socket/SSL.pm:1425: OK free ctx 139403496
DEBUG: .../IO/Socket/SSL.pm:1135: IO::Socket::INET configuration failederror:00000000:lib(0):func(0):reason(0)
500 Can't connect to some-server-with-bad-certificate.com:443 (SSL connect attempt failed with unknown errorerror:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed)
I landed on this page looking for a way to bypass SSL validation but all answers were still very helpful. Here are my findings. For those looking to bypass SSL validation (not recommended but there may be cases where you will absolutely have to), I'm on lwp 6.05 and this worked for me:
use strict;
use warnings;
use LWP::UserAgent;
use HTTP::Request::Common qw(GET);
use Net::SSL;
my $ua = LWP::UserAgent->new( ssl_opts => { verify_hostname => 0 }, );
my $req = GET 'https://github.com';
my $res = $ua->request($req);
if ($res->is_success) {
print $res->content;
} else {
print $res->status_line . "\n";
}
I also tested on a page with POST and it also worked. The key is to use Net::SSL along with verify_hostname = 0.
If you use LWP::UserAgent directly (not via LWP::Simple) you can validate the hostname in the certificate by adding the "If-SSL-Cert-Subject" header to your HTTP::Request object. The value of the header is treated as a regular expression to be applied on the certificate subject, and if it does not match, the request fails. For example:
#!/usr/bin/perl
use LWP::UserAgent;
my $ua = LWP::UserAgent->new();
my $req = HTTP::Request->new(GET => 'https://yourdomain.tld/whatever');
$req->header('If-SSL-Cert-Subject' => '/CN=make-it-fail.tld');
my $res = $ua->request( $req );
print "Status: " . $res->status_line . "\n"
will print
Status: 500 Bad SSL certificate subject: '/C=CA/ST=Ontario/L=Ottawa/O=Your Org/CN=yourdomain.tld' !~ //CN=make-it-fail.tld/
All the solutions presented here contain a major security flaw in that they only verify the validity of the certificate's trust chain, but don't compare the certificate's Common Name to the hostname you're connecting to. Thus, a man in the middle may present an arbitrary certificate to you and LWP will happily accept it as long as it's signed by a CA you trust. The bogus certificate's Common Name is irrelevant because it's never checked by LWP.
If you're using IO::Socket::SSL as LWP's backend, you can enable verification of the Common Name by setting the verifycn_scheme parameter like this:
use IO::Socket::SSL;
use Net::SSLeay;
BEGIN {
IO::Socket::SSL::set_ctx_defaults(
verify_mode => Net::SSLeay->VERIFY_PEER(),
verifycn_scheme => 'http',
ca_path => "/etc/ssl/certs"
);
}
You may also consider Net::SSLGlue ( http://search.cpan.org/dist/Net-SSLGlue/lib/Net/SSLGlue.pm ) But, take care, it depends on recent IO::Socket::SSL and Net::SSLeay versions.
You are right to be concerned about this. Unfortunately, I don't think it's possible to do it 100% securely under any of the low-level SSL/TLS bindings I looked at for Perl.
Essentially you need to pass in the hostname of the server you want to connect to the SSL library before the handshaking gets underway. Alternatively, you could arrange for a callback to occur at the right moment and abort the handshake from inside the callback if it doesn't check out. People writing Perl bindings to OpenSSL seemed to have troubles making the callback interface consistently.
The method to check the hostname against the server's cert is dependent on the protocol, too. So that would have to be a parameter to any perfect function.
You might want to see if there are any bindings to the Netscape/Mozilla NSS library. It seemed pretty good at doing this when I looked at it.
Just perform execute the following command in Terminal:
sudo cpan install Mozilla::CA
It should solve it.