I've had IPN working for years: I'm using a classic Paypal Button to initiate a purchase, and then Paypal sends me an IPN to my web server so that I can record the payment and send the customer an email.
But since Oct 4, 2021, Paypal has not been able to send me IPNs any more.
The "Instant Payment Notification (IPN) history" shows all IPNs since then as either Failed or Pending.
My server, however, responds just fine to the requests if I invoke the IPN URL from a browser. My server (Debian 10 with Apache 2) has not had any configuration changes or package updates in the past week, especially not since the time when it still received IPNs (Oct 3).
The only change I could find around that date (on Oct 3) is an automatic renewal of the Let's Encrypt SSL cert, which happened many times before without causing any such trouble with IPN. I have since recreated the Let's Encrypt cert, even as a dedicated one just for that domain given to Paypal (before the cert was shared between multiple domains on that server), with no improvement.
What works
Invoke the URL (https://paypal-api.tempel.org/) from curl or a web browser. My CGI script is invoked, and the Apache access.log shows the GET request.
What does not work
Use the IPN Simulator, specifying the same URL - then the simulator page shows
IPN was not sent, and the handshake was not verified. Review your information
at the top and I see no request in the Apache access.log
What has changed?
I have checked my firewall (which I hadn't changed in a while), even disabled it, to no avail.
Why would Paypal suddenly fail to even contact my server?
I suspect it has to do with the SSL cert, as its renewal date is the only date relation I could find and since Paypal not accepting the SSL cert would probably also lead to the effect that it won't even issue the GET or POST request to my server, hence no log entry about it.
Am I doing the testing with the Simulator right? How else could I find out why Paypal doesn't perform the "handshake"?
SSL cert oddities
Even though certbot says that the SSL cert is up-to-date, I get this output from openssl for the domain:
% openssl s_client -connect paypal-api.tempel.org:443 -showcerts -CApath /etc/ssl/certs/
CONNECTED(00000005)
depth=1 O = Digital Signature Trust Co., CN = DST Root CA X3
verify error:num=10:certificate has expired
notAfter=Sep 30 14:01:15 2021 GMT
verify return:0
depth=1 O = Digital Signature Trust Co., CN = DST Root CA X3
verify error:num=10:certificate has expired
notAfter=Sep 30 14:01:15 2021 GMT
verify return:0
depth=3 O = Digital Signature Trust Co., CN = DST Root CA X3
verify error:num=10:certificate has expired
notAfter=Sep 30 14:01:15 2021 GMT
verify return:0
---
Certificate chain
0 s:/CN=git.tempel.org
i:/C=US/O=Let's Encrypt/CN=R3
-----BEGIN CERTIFICATE-----
…
-----END CERTIFICATE-----
---
Server certificate
subject=/CN=git.tempel.org
issuer=/C=US/O=Let's Encrypt/CN=R3
---
No client certificate CA names sent
Server Temp Key: ECDH, X25519, 253 bits
---
SSL handshake has read 4574 bytes and written 281 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-CHACHA20-POLY1305
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol : TLSv1.2
Cipher : ECDHE-RSA-CHACHA20-POLY1305
Session-ID: 523AEDD29014010191C7B3DD16F4DC7179B8D91372C3063CEF8D6AD2ED7B800E
Session-ID-ctx:
Master-Key: 36D105B8EAD319DA6B8D2EA30484FC6676F2A2D64CA0EE8889B16396ECA71178FE5154638DCE1E9AF051D8D2FA44472E
Start Time: 1633776598
Timeout : 7200 (sec)
Verify return code: 0 (ok)
---
closed
Why does it say "certificate has expired" at the top, even though it's ok at the end? Could this be the issue?
More strangely, when I give the IPN Simulator another domain of mine, which is also using Let's Encrypt, e.g. www.tempel.org, the Simulator responded yesterday with a positive result, yet today, it gives the same "handshake" error. Also, openssl already gave the same "certificate has expired" error yesterday when the IPN Simulator still accepted it.
So, this all seems to have to do with Let's Encrypt's certificates, somehow. But how do I fix this, apart from buying a different SSL cert?
Or is this just some odd caching issue? When I open https://paypal-api.tempel.org/ in Safari, and check its cert info, it does not indicate any expired certs. What's going on here?
Related to recent expiry of X3 root cert?
I have a hunch that this is related to the expiry of one of Let's Encrypt's root certificate on Sept 30, 2021. The openssl command I've shown above mentions that X3 certificate, indeed.
I am still not clear on how to fix this, however. I understand the instructions in the SO article about disabling the outdated X3 cert, but don't understand what to do then, yet. Probably force-renew the certs with certbot. But even that doesn't make the Simulator happy, yet, and the openssl cmd keeps mentioning the X3 root regardless.
Updated the certs
After reading here about fixing the X3 expiry issues, I renewed my cert by forcing the X1 root (h1.tempel.org is another domain on the same server):
certbot renew --cert-name h1.tempel.org --preferred-chain "ISRG Root X1" --force-renewal
and then restarted Apache (apachectl graceful).
Oddly, since then, when I give https://h1.tempel.org/ to the IPN Simulator, I get the desired "IPN was sent and the handshake was verified." message. But using https://paypal-api.tempel.org/, which is the same certificate, but just a different Apache site on the same server, I still get the "the handshake was not verified" error msg. I am out of ideas.
Can anyone see a difference in the certs between https://h1.tempel.org/ and https://paypal-api.tempel.org/?
I've finally managed to get it to work again - in parts.
After I had found out that the simulator accepts https://h1.tempel.org/ but not https://paypal-api.tempel.org/, even though both are valid pages that return status 200, and both are on the same server and use the same SSL certificate, I reconfigured the web server so that the h1 subdomain would also serve the cgi handler. Then I updated the handler URL at https://www.paypal.com/cgi-bin/webscr?cmd=_profile-ipn-notify to use the h1 subdomain (instead of paypal-api, and that fixed the issue, at least for NEW Paypal transactions.
However, trying to resend the previously failed notifications still fails, because they keep using the old URL.
Paypal Support responds
After finally having contacted Paypal business support, I learned:
They're using Cisco / Talos Intelligence to rate ANY outside page they contact, apparently.
And for some inexplicable reason, Talos currently rates my IPN URL's domain as "malware"! No other blacklisting site rates my site, because it's virtually invisible, and there's no content delivered by that site, apart from the IPN cgi handler, which delivers nothing but an empty reply with either status 200 or 500, depending on the content.
Which means I now have to get Talos to remove that malware rating (which is still pending 24 hours later!). Interestingly, Talos only rates the domain, not all sites served by that server's IP address. Therefore, even payments.tempel.org, which is literally nothing but a ServerAlias to paypal-api.tempel.org, is rated fine.
Paypal support also explained that the notification URL is stored in the database, and gets re-used when trying to re-send a failed notification, hence it would keep using the old (currently malware-rated) URL despite me having told Paypal to use a different URL now.
But now, a day later, trying resend again on the failed message, suddenly does use the newly set URL, regardless of Paypal support's explanation that this would not happen with re-sends.
So, the moral of the story: Sometimes it's not your fault but it's Paypal who gets this wrong. Or Cisco.
Also worth noting: Paypal won't alert you if they suddenly decide to consider your IPN URL part of a malware site. Nothing in the help or their testing tools will tell you why they're not contacting you any more, leaving you guessing until you contact them (In that regard, the initial answer (now deleted) was right to suggest that I need to contact Paypal directly because Paypal had blacklisted me for some reason). And they can't even whitelist you, but instead leave you fighting with Cisco over getting your rating back to "harmless".
I am ensuring the logic that if I have upgraded my SSl and have successfully completed a transaction in Paypal's sandbox, that this means that I'm good to go as far as their coming upgrades?
It is my understanding that their test sandbox like https://api-3t.sandbox.paypal.com/nvp are setup with the new requirements an will fail if the system does not have the new requirements.So a successful transaction on this test sandbox means we are good to go, Paypal upgrade wise right?
Am I correct?
Exactly, an end-to-end payment test against the sandbox API endpoint will ensure your server compatibility.
Or you may simply put a connection test from your host to verify:
openssl s_client -connect api-3t.sandbox.paypal.com:443 -showcerts
I am trying to get information from PayPal by using IPN (sandbox).
When I enter address:
https://my.site.com:4576/script.py
I get an error:
paypal URL with port number is not allowed for IPN.
According to this PayPal IPN on port other than 80 PayPal does not allow ports other than 80 and 443.
However when I change the address to:
https://my.site.com/script.py
I get an error:
IPN was not sent, and the handshake was not verified. Please review
your information.
According to this paypal IPN simulator getting error? this means that SSL is outdated however when I run this command openssl s_client -connect api-3t.sandbox.paypal.com:443 -showcerts -CApath /etc/ssl/certs/ in my server I get Verify return code: 0 (ok) which means that SSL is up to date.
How can I make this IPN work?
As per the paypal security upgrade on Jan 17th 2016 they are saying that the server needs to be installed with a ssl server with ssl of algorithm SHA-256 and the certificate needs to verify with a G5 ROOT certificate. But my doubt here previusly i can test the paypal sandbox payment in my localhost(a server without https) and it was worked perfectly. But as per the new upgrade from the paypal team is there any option to check the api service in sandbox in a http server(may be on localhost). When i try this i got a handshake_failure exception.
This was my mistake and finally i understood that ssl not needed. When i have update the paypal sdk jar to 1.4 the issue cleared.
I'm getting the following error when using the IPN Simulator -
IPN was not sent, and the handshake was not verified. Please review your information.
The URL is non-SSL HTTP (dev environment).
When I hit it in a browser I get the expected output (and a matching entry in my servers access.log), however when I hit it via the IPN Simulator, I get the above error, and nothing in the servers access log.
On advice from these questions:
IPN was not sent, and the handshake was not verified. Please review your information.
IPN Simulator: "IPN was not sent, and the handshake was not verified. Please review your information"
I tried the command openssl s_client -connect api-3t.sandbox.paypal.com:443 -showcerts -CApath /etc/ssl/certs
and got the output they suggested:
Verify return code: 0 (ok)
I don't think this is particularly important as I'm not attempting to make an SSL request.
Any other ideas?
I'm attempting this as part of a bug fix, I developed the IPN stuff approximately 3 weeks ago and the IPN Simulator worked just dandily, now not so much.
Thanks in advance.
I found out that IP-based http such as http://182.100.25.4/foo/bar/baz do not work. I used dyndns to create a DNS record to point to 182.100.25.4 and suddenly voila - http://somedns.ddns.net/foo/bar/baz started working properly in the IPN simulator.
http-based URLs do work, you do not need to use https-based URLs.
Check web access logs on your server.
In my case there were 500 Internal Server Error on my side and I got same message in IPN Simulator.
But IPN Simulator were trying to send request any way...
This is incredible but in my case I needed to add a trailing slash to the end of the URL. Truly incredible.