I've had IPN working for years: I'm using a classic Paypal Button to initiate a purchase, and then Paypal sends me an IPN to my web server so that I can record the payment and send the customer an email.
But since Oct 4, 2021, Paypal has not been able to send me IPNs any more.
The "Instant Payment Notification (IPN) history" shows all IPNs since then as either Failed or Pending.
My server, however, responds just fine to the requests if I invoke the IPN URL from a browser. My server (Debian 10 with Apache 2) has not had any configuration changes or package updates in the past week, especially not since the time when it still received IPNs (Oct 3).
The only change I could find around that date (on Oct 3) is an automatic renewal of the Let's Encrypt SSL cert, which happened many times before without causing any such trouble with IPN. I have since recreated the Let's Encrypt cert, even as a dedicated one just for that domain given to Paypal (before the cert was shared between multiple domains on that server), with no improvement.
What works
Invoke the URL (https://paypal-api.tempel.org/) from curl or a web browser. My CGI script is invoked, and the Apache access.log shows the GET request.
What does not work
Use the IPN Simulator, specifying the same URL - then the simulator page shows
IPN was not sent, and the handshake was not verified. Review your information
at the top and I see no request in the Apache access.log
What has changed?
I have checked my firewall (which I hadn't changed in a while), even disabled it, to no avail.
Why would Paypal suddenly fail to even contact my server?
I suspect it has to do with the SSL cert, as its renewal date is the only date relation I could find and since Paypal not accepting the SSL cert would probably also lead to the effect that it won't even issue the GET or POST request to my server, hence no log entry about it.
Am I doing the testing with the Simulator right? How else could I find out why Paypal doesn't perform the "handshake"?
SSL cert oddities
Even though certbot says that the SSL cert is up-to-date, I get this output from openssl for the domain:
% openssl s_client -connect paypal-api.tempel.org:443 -showcerts -CApath /etc/ssl/certs/
CONNECTED(00000005)
depth=1 O = Digital Signature Trust Co., CN = DST Root CA X3
verify error:num=10:certificate has expired
notAfter=Sep 30 14:01:15 2021 GMT
verify return:0
depth=1 O = Digital Signature Trust Co., CN = DST Root CA X3
verify error:num=10:certificate has expired
notAfter=Sep 30 14:01:15 2021 GMT
verify return:0
depth=3 O = Digital Signature Trust Co., CN = DST Root CA X3
verify error:num=10:certificate has expired
notAfter=Sep 30 14:01:15 2021 GMT
verify return:0
---
Certificate chain
0 s:/CN=git.tempel.org
i:/C=US/O=Let's Encrypt/CN=R3
-----BEGIN CERTIFICATE-----
…
-----END CERTIFICATE-----
---
Server certificate
subject=/CN=git.tempel.org
issuer=/C=US/O=Let's Encrypt/CN=R3
---
No client certificate CA names sent
Server Temp Key: ECDH, X25519, 253 bits
---
SSL handshake has read 4574 bytes and written 281 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-CHACHA20-POLY1305
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol : TLSv1.2
Cipher : ECDHE-RSA-CHACHA20-POLY1305
Session-ID: 523AEDD29014010191C7B3DD16F4DC7179B8D91372C3063CEF8D6AD2ED7B800E
Session-ID-ctx:
Master-Key: 36D105B8EAD319DA6B8D2EA30484FC6676F2A2D64CA0EE8889B16396ECA71178FE5154638DCE1E9AF051D8D2FA44472E
Start Time: 1633776598
Timeout : 7200 (sec)
Verify return code: 0 (ok)
---
closed
Why does it say "certificate has expired" at the top, even though it's ok at the end? Could this be the issue?
More strangely, when I give the IPN Simulator another domain of mine, which is also using Let's Encrypt, e.g. www.tempel.org, the Simulator responded yesterday with a positive result, yet today, it gives the same "handshake" error. Also, openssl already gave the same "certificate has expired" error yesterday when the IPN Simulator still accepted it.
So, this all seems to have to do with Let's Encrypt's certificates, somehow. But how do I fix this, apart from buying a different SSL cert?
Or is this just some odd caching issue? When I open https://paypal-api.tempel.org/ in Safari, and check its cert info, it does not indicate any expired certs. What's going on here?
Related to recent expiry of X3 root cert?
I have a hunch that this is related to the expiry of one of Let's Encrypt's root certificate on Sept 30, 2021. The openssl command I've shown above mentions that X3 certificate, indeed.
I am still not clear on how to fix this, however. I understand the instructions in the SO article about disabling the outdated X3 cert, but don't understand what to do then, yet. Probably force-renew the certs with certbot. But even that doesn't make the Simulator happy, yet, and the openssl cmd keeps mentioning the X3 root regardless.
Updated the certs
After reading here about fixing the X3 expiry issues, I renewed my cert by forcing the X1 root (h1.tempel.org is another domain on the same server):
certbot renew --cert-name h1.tempel.org --preferred-chain "ISRG Root X1" --force-renewal
and then restarted Apache (apachectl graceful).
Oddly, since then, when I give https://h1.tempel.org/ to the IPN Simulator, I get the desired "IPN was sent and the handshake was verified." message. But using https://paypal-api.tempel.org/, which is the same certificate, but just a different Apache site on the same server, I still get the "the handshake was not verified" error msg. I am out of ideas.
Can anyone see a difference in the certs between https://h1.tempel.org/ and https://paypal-api.tempel.org/?
I've finally managed to get it to work again - in parts.
After I had found out that the simulator accepts https://h1.tempel.org/ but not https://paypal-api.tempel.org/, even though both are valid pages that return status 200, and both are on the same server and use the same SSL certificate, I reconfigured the web server so that the h1 subdomain would also serve the cgi handler. Then I updated the handler URL at https://www.paypal.com/cgi-bin/webscr?cmd=_profile-ipn-notify to use the h1 subdomain (instead of paypal-api, and that fixed the issue, at least for NEW Paypal transactions.
However, trying to resend the previously failed notifications still fails, because they keep using the old URL.
Paypal Support responds
After finally having contacted Paypal business support, I learned:
They're using Cisco / Talos Intelligence to rate ANY outside page they contact, apparently.
And for some inexplicable reason, Talos currently rates my IPN URL's domain as "malware"! No other blacklisting site rates my site, because it's virtually invisible, and there's no content delivered by that site, apart from the IPN cgi handler, which delivers nothing but an empty reply with either status 200 or 500, depending on the content.
Which means I now have to get Talos to remove that malware rating (which is still pending 24 hours later!). Interestingly, Talos only rates the domain, not all sites served by that server's IP address. Therefore, even payments.tempel.org, which is literally nothing but a ServerAlias to paypal-api.tempel.org, is rated fine.
Paypal support also explained that the notification URL is stored in the database, and gets re-used when trying to re-send a failed notification, hence it would keep using the old (currently malware-rated) URL despite me having told Paypal to use a different URL now.
But now, a day later, trying resend again on the failed message, suddenly does use the newly set URL, regardless of Paypal support's explanation that this would not happen with re-sends.
So, the moral of the story: Sometimes it's not your fault but it's Paypal who gets this wrong. Or Cisco.
Also worth noting: Paypal won't alert you if they suddenly decide to consider your IPN URL part of a malware site. Nothing in the help or their testing tools will tell you why they're not contacting you any more, leaving you guessing until you contact them (In that regard, the initial answer (now deleted) was right to suggest that I need to contact Paypal directly because Paypal had blacklisted me for some reason). And they can't even whitelist you, but instead leave you fighting with Cisco over getting your rating back to "harmless".
I am currently trying to integrate paypal checkout with our online store. We are testing against Sandbox. Everything but the IPN (Instant payment notification) works.
We read a lot about paypal changing their security model so we tried to follow their guide but we are still getting an error:
The SSL certificate for the host could not be verified
the error we get on IPN is:
The SSL certificate for the host could not be verified
Now, we are using a G2 cert from GoDaddy (supports sha256). Not sure if this has anything to do with it or not.
Any help would be appreciated.
I am running out of ideas. We already installed the G5 root cert from Verisign, the site runs in SSL via G2 GoDaddy cert.
Thanks,
Moz
Are you using cURL to get the VERIFIED response?
If yes then contact GoDaddy and get the version number of SSL you have.
cURL by default uses SSLv3.
I had a similar issue, solved it by adding this
curl_setopt($ch, CURLOPT_SSLVERSION, 2);
to my cURL headers.
I have integrated paypal express checkout with my app. Few days back I got following email from Paypal.
Dear Customer,
In keeping with industry standards set by the Certification Authority/Browser (CA/B) Forum, PayPal will discontinue supporting 1024-bit key length certificates and will migrate to 2048-bit certificates before the end of 2013.
We have completed the installation of 2048-bit certificates for all API endpoints in our PayPal Sandbox and Payflow Pilot environments, and we will be doing the same for our production environments starting on August 6, 2013.
We strongly encourage merchants to thoroughly test any existing integration(s) in the PayPal Sandbox and/or Payflow Pilot environments to ensure this migration will not cause any unforeseen issues.
Please have the team or person responsible for your integration refer to the following:
If you need to import the new PayPal Sandbox and/or Payflow Pilot server certificates to your application or system truststore, you can download production and Sandbox certificates from https://ppmts.custhelp.com/app/answers/detail/a_id/952.
If you don’t typically import the server certificates to your truststore, you can proceed with testing with no other action required.
If you have any questions, please contact PayPal Merchant Technical Services by filing a ticket; refer to PP-LIVE-3503. You may also visit our Live Site Status blog.
Sincerely,
PayPal
I am using API signatures instead of certificates. So, I really dont need to do anything here, right?
It's not the API certificate that is changing, it's the endpoint certificate that's changing to 2048. So whether your API credentials consist of either a API Signature or an API Certificate shouldn't matter.
You will only need to change anything if you're somehow storing and validating PayPal's API endpoint SSL certificate against a locally stored copy of the (same) SSL certificate. Often this is done in a so called 'truststore'.
Since PayPal's API endpoint certificate will change, you would need to update the certificate in the truststore accordingly.
So yes, you won't need to change anything if you're merely using an API signature or API certificate for API authentication.
I am not sure it is neccessary if I am using PayPal express checkout for recurring payment, do I need https/ssl for my website to connect to PayPal? I am using Ruby on Rails and there is a gem called "paypal-express". It is working well without ssl in sandbox environment. Any suggestion? Thank you.
If by 'https/ssl' you mean whether your site itself needs to support SSL traffic over HTTP (and thus have a valid SSL certificate): no, it doesn't.
You do however need to be able to establish an SSL connection. Specifically, to PayPal's API endpoint.
This means you need to allow outbound SSL traffic via your firewall (if you have any) and your environment needs to support this.
Ensure you have a valid copy of the root certs (I usually suggest http://curl.haxx.se/ca/cacert.pem) to valid the SSL certificates against.
Callbacks required SSL as stated here: https://developer.paypal.com/docs/classic/express-checkout/integration-guide/ECInstantUpdateAPI/
Working with the Paypal API, yes i've checked my config files, yes i've checked username and password.... i'm outa ideas. I'm using the ExpressCheckout API downloaded from and everytime i try call the setExpressCheckout method i get the following error:
NSS: client certificate not found (nickname not specified)
I know it's something to do with an SSL error, how do i go about solving the problem?
On August 3 and August 5 PayPal renewed the SSL certificates for the following API endpoints:
api.paypal.com
api-3t.paypal.com
api-aa.paypal.com
api-aa-3t.paypal.com
If you need to import the new PayPal SSL certificates into your application or system keystore/truststore you can download them from https://ppmts.custhelp.com/app/answers/detail/a_id/920/. If you don't typically import the PayPal SSL certificates into your keystore/truststore, no action is required on your part.
For Sandbox please use this link: https://ppmts.custhelp.com/app/answers/detail/a_id/924 to Download the new SSL Certificate.
Check the paypal's server domain you send request to.
It should be api.paypal.com for certificate based authorization and api-3t.paypal.com for signature based authorization.
https://cms.paypal.com/us/cgi-bin/?cmd=_render-content&content_ID=developer/e_howto_api_nvp_NVPAPIOverview#id084E30V0RE9