SSL error calling Perl web service via https using LWP - perl

I have a Perl script that acts as a web service to other Perl scripts running under the same domain. I recently created a new SSL certificate (AlphaSSL) for this domain and installed it on the server and it shows up fine when accessing the site using https.
However, now when a client script calls the web service using htttps via LWP, an error is issued where previously it was not. The code to call the web service looks like this:
$useragent = LWP::UserAgent->new();
$useragent->agent("someagentid");
$postdata = {
'action' => $apiaction,
'xauth' => $REQUEST_AUTH_KEY,
};
$response = $useragent->post($BILLING_INFO_GATEWAY, $postdata);
The gateway in the post is an https: URL to the web service script running under the same domain.
The error it produces is:
500 Can't connect to xxxxx.com:443 (certificate verify failed)
Content-Type: text/plain
Client-Date: Tue, 25 Nov 2014 01:52:20 GMT Client-Warning: Internal response
Can't connect to xxxxx.com:443 (certificate verify failed)
LWP::Protocol::https::Socket: SSL connect attempt failed with unknown error
error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify
failed at /usr/perl/lib/site_perl/5.10.1/LWP/Protocol/http.pm line 49.
If I add:
$ENV{PERL_LWP_SSL_VERIFY_HOSTNAME} = 0;
then the error goes away, but then the secure connection seems less secure.
I have located the certificate directory on my server which is /ssl/certs/ which has one .crt file in it, so I tried adding:
$ENV{HTTPS_CA_PATH} = '/ssl/certs';
but I still receive the error. I also tried:
$ENV{HTTPS_CA_FILE} = '/ssl/certs/xxxxxxxxxx.cabundle';
after copying a CA bundle file to the same folder (there was no bundle file there previously).
This folder contained a .crt file which already existed there after installing the certificate via WHM/cPanel.)
I also tried specifying just the HTTPS_CA_FILE environment variable without the HTTPS_CA_PATH variable but no luck. Still the error persists.
Any ideas about what I might be missing either in the code or on the server to allow the script to verify the hostname successfully? This is a dedicated web server running Linux and cPanel.
Thanks!

If it works in the browser but not in LWP script it might be because intermediate certificates are missing. Browser often receive these intermediates certificates from earlier SSL connections and cache them, but scripts usually don't do that. You can check this if you check the host with SSLLabs and watch out for "Chain Issues".
If this is not the problem please provide the versions of the modules you are using and preferable also the URL of the host you are trying to reach. To get the versions:
#!/usr/bin/perl
for (qw(LWP::UserAgent LWP::Protocol::https IO::Socket::SSL)) {
eval "require $_; warn '$_: '.$_->VERSION";
}

Related

Not able to access AEM author using JcrUtils.getRepository with https

I am trying to access the repository from a standalone java application using JcrUtils.
**repository = JcrUtils.getRepository("http://localhost:4502/crx/server"); this works**
**repository = JcrUtils.getRepository("https://localhost:4502/crx/server"); this doesn't work**
Exception in thread "main" javax.jcr.RepositoryException: Unable to
access a repository with the following settings:
org.apache.jackrabbit.repository.uri: https://localhost:4502/crx/server The following
RepositoryFactory classes were consulted:
org.apache.jackrabbit.commons.JndiRepositoryFactory: declined
org.apache.jackrabbit.core.RepositoryFactoryImpl: declined
org.apache.jackrabbit.jcr2dav.Jcr2davRepositoryFactory: declined
org.apache.jackrabbit.jcr2spi.Jcr2spiRepositoryFactory: declined Perhaps the repository you are trying to access is not available at
the moment. at
org.apache.jackrabbit.commons.JcrUtils.getRepository(JcrUtils.java:223)
at
org.apache.jackrabbit.commons.JcrUtils.getRepository(JcrUtils.java:263)
at com.hero.jcr.util.CqHelper.getSession(CqHelper.java:20) at
com.hero.jcr.commandline.CheckConnection.getDamAssets(CheckConnection.java:36)
at
com.hero.jcr.commandline.CheckConnection.main(CheckConnection.java:29)
Thanks in advance
The problem can have different causes.
HTTPS configured and on which port
It looks like you're trying to access http and https in the same situation on the same port. Http and Https don't run on the same port in parallel.
Please use the SSL Wizard as hinted by ronnyfm to check if SSL is activated and on which port it is used: https://docs.adobe.com/content/help/en/experience-manager-65/administering/security/ssl-by-default.html
For instance the default port for https in AEM is 8443, so try https://localhost:8443 to access it, if you have configured it there.
Untrusted Certificate
Also connecting to https might not work if you use an untrusted certificate, which you need to add to the Java security keystore first. In my case I got the same "Unable to access" message, because the original exception regarding the certificate got silently swallowed in the JCR library I was using.
Wrong certificate
When you added the certificate to the keystore and it still doesn't work, check if the certificate was created for them same host you request. If you created it for bla.host, but try to access it via localhost, it also gives you the same exception, while the original message is silently swallowed.

SSL for mail server

I don't know if I am asking this in the right place.
I have an SSL cert for my website, and I am trying to setup a mail server (same domain) using the same cert.
I am using Postfix and Dovecot. When I try logging in from Evolution mail client, I get an error "Peer failed to perform TLS handshake". When I try an online service to verify I get "Recipient address rejected: User unknown in local recipient table."
I guess my actual question is, can I actually use the same SSL cert for my website and my email server?
What do I do to debug next?
You can use a certificate you have for the web server also for your mail server as long as it matches the hostname you use to access your mail server. Of course the certificate need to properly setup at the mail server, i.e. include the necessary intermediate certificates similar to how it is (hopefully) setup on the web server.
I am using Postfix and Dovecot.
This means you need to take care of multiple configurations, both for SMTP in Postfix and IMAP/POP3 (whatever you use) in Dovecot. And in all cases the certificates subject/SAN must match the hostname you use to connect to the server.
When I try logging in from Evolution mail client, I get an error "Peer failed to perform TLS handshake".
There are not enough information about this setup to find out what exactly is causing the TLS error. It is not even clear if the error is caused when retrieving mail (IMAP/POP3, i.e. Dovecot) or while sending (Postfix).
When I try an online service to verify I get "Recipient address rejected: User unknown in local recipient table."
This has nothing to do with TLS at all. The test server simply tried to use a recipient which your mail server (Postfix) will not accept.
What do I do to debug next?
The next steps would probably be to check if the certificate matches the names you use in the first place and to look into log files for error messages or warnings. Following steps depend on what the result of these steps is.

Facebook - Curl Error SSL_CACERT SSL certificate

I am getting "Curl Error : SSL_CACERT SSL certificate problem: unable to get local issuer certificate" when asking Facebook to scrape my page over https. How can I fix this so that Facebook can scrape my page without errors?
The page is hosted via Apache 2.4 proxying to IIS 10. Apache handles all certificates and IIS is on the local network. My page is running asp code (so no php) and solutions similar to these: edit the php.ini file or adding curl.pem to php folder will not work fix my problem ... or so I think?!?
IIS has no certificate installed.
I do have extension=php_curl.dll enabled -- and extension_dir = 'C:\64bit\php-7.0.6-Win32-VC14-x64\ext' defined in my php.ini file. I followed these steps to install Curl on Windows. And phpinfo.php confirms that cURL is enabled (cURL Information 7.47.1).
My proxy setup in my Apache config file is:
<IfModule mod_proxy.c>
ProxyRequests Off
ProxyPass / http://192.168.1.101:88/com_ssl/
ProxyPassReverse / http://192.168.1.101:88/com_ssl/
RewriteRule ^(.+)$ https://www.domainname.com/$1 [P,L]
</IfModule>
I have no RequestHeader defined in my Apache proxy config file, such as suggested here in Step 10:
RequestHeader set "X-RP-UNIQUE-ID" "%{UNIQUE_ID}e"
RequestHeader set "X-RP-REMOTE-USER" "%{REMOTE_USER}e"
RequestHeader set "X-RP-SSL-PROTOCOL" "%{SSL_PROTOCOL}s"
RequestHeader set "X-RP-SSL-CIPHER" "%{SSL_CIPHER}s"
Is this what is missing to fix the error?
"unable to get local issuer certificate" is almost always the error message you get when the server doesn't provide an intermediate certificate as it should in the TLS handshake, and as WizKid suggests, running the ssllabs test against the server will indeed tell you if that is the case.
If you are using nodejs server and getting this error 'Curl Error SSL_CACERT SSL certificate' then you need to add your CA along with your SSL CRT.
var fs = require('fs');
var https = require('https');
var options = {
key: fs.readFileSync('server-key.pem'),
cert: fs.readFileSync('server-crt.pem'),
ca: fs.readFileSync('ca-crt.pem'), // <= Add This
};
https.createServer(options, function (req, res) {
console.log(new Date()+' '+
req.connection.remoteAddress+' '+
req.method+' '+req.url);
res.writeHead(200);
res.end("hello world\n");
}).listen(4433);
This may not have been the case at the time but I will add this info in case others encounter the same issue.
If you are using a CDN, like cloudflare, it is important to set up your SSL before adding to cloudflare as it can generate issues.
It is also important to ensure that all domains are correctly annotated in the DNS control of cloudflare, otherwise you may end up serving your main domain via cloudflare and your subdomain(s) directly from your server. Whilst this wont matter much to the user (still shows secure, still have access, still passes SSL tests) it may flag issues with sharing apps onto social media. Basically, I replicated the error by splitting the DNS setup as above and achieved the flagged error as highlighted by the op. Then I added the DNS for the subdomain into cloudflare, tested a few hours later (after resetting the page in debudder: https://developers.facebook.com/tools/debug/sharing/?q=https%3A%2F%2Fus.icalculator.info%2Fterminology%2Fus-tax-tables%2F2019%2Fvirginia.html). and, hey presto, the error goes. So, if you encounter that issue and you use cloudflare, that is something to check you have set up correctly.

Test Dropbox webhooks implementation with self-signed certificate?

I'm setting up use of Dropbox webhooks in a web application. I'd like to do the development on my local machine. I can access my local machine via https from the Internet, but when I try to add the URL as a webhook URI in Dropbox App Console I found that it seems to require the SSL certificate to not be self-signed. Is there a way to do testing on a local machine with a self-signed certificate? The error message is below. After complaining about the cert it also says there is no response body, but I'm guessing this is because it didn't even get as far as checking the body (I've tested my URL by loading it directly in a browser and via wget from a remote server and the response contains the desired content).
Error: SSL certificate problem: self signed certificate
Request: GET
https://my.domain.com/dropbox-webhook?challenge=urFYi8g-jCkVM7aP676BAyYlH7Z3u04RAJH5Lu0AYLg
Response:(No headers)
Response Body (First 256 bytes):(No response body)
For local development, I leaned on https://github.com/dropbox/dropbox_hook pretty hard to test everything. It's what I would recommend for local development of dropbox webhooks.

Perl LWP SSL connection: certificate verify fails

My application requires LWP under SSL, but I can't seem to get it to connect properly without receiving an error.
certificate verify failed)LWP::Protocol::https::Socket: SSL connect attempt failed with unknown error error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed at C:/Perl/site/lib/LWP/Protocol/http.pm line 51.
Whenever I load up the website in FireFox on my server, I receive
This connection is untrusted (Technical error: sec_error_unknown_issuer)
Whenever I load it up in Chrome, it verifies absolutely fine.
I can load the page up in FireFox and Chrome on my computer and receive no SSL verification errors at all.
I have no clue what is different between my PC and my server; they're both using the exact same FireFox versions so I'm not sure why it wouldn't verify properly on there.
I've tried workarounds by putting in $ENV{PERL_LWP_SSL_VERIFY_HOSTNAME} = 0 in the Perl script but it seems to throw the same exact error still.
This is the script I'm using to test HTTPS
Does anyone know any solution to this or any workaround?
I've tried workarounds by putting in $ENV{PERL_LWP_SSL_VERIFY_HOSTNAME} = 0
As the name of the option implies it cares only about disabling the check of the hostname against the certificate. It does not disable the validation of the certificate chain.
Whenever I load up the website in FireFox on my server, I receive ...
Since you get validation errors with Firefox and LWP but not with Chrome, Chrome has probably an additional CA as trusted which they others don't have.
From your code it looks like that you are using Windows and from my understanding Firefox comes with its own CA store, LWP uses Mozilla::CA (which contains the CAs usually shipped with Firefox) but Chrome uses the system CA store. Thus there is probably a CA in the system store which Firefox and LWP don't know about.
While it might be that there is some special esoteric CA which is known to the default Windows CA store but not to Firefox, it is more likely that you are behind some SSL inspecting firewall and that the administrators added the necessary proxy certificate to Windows CA store but not to Firefox or LWP's store. I would suggest you check the certificate chain inside the Chrome browser and look at the top (builtin) certificate.
If you consider the CA trusted you might export it and import it into Firefox. You might also make LWP use this CA by saving it in PEM format and then set the PERL_LWP_SSL_CA_PATH environment variable to point to the saved file or use the SSL_ca_path option in the ssl_opts setting to make LWP use this CA certificate for validation.
If this explanation does not lead to a solution please provide more details about the target URL you are trying to access and the network setup you have, especially if there is a SSL intercepting proxy or firewall. Please provide also information about the version of LWP you are using, because there changed a lot with version 6.