I've spent quite a few hours fighting with these issues so I though a quick recap might be helpful for somebody else too.
First, when trying to import an RP from a metadata URL:
I was getting this error:
An error occured during an attempt to read the federation metadata. Verify that the specified URL or hostname is a valid federation metadata endpoint.
...
Error message: The underlying connection was closed: An unexpected error occured on a send.
The problem turned out to be caused by the fact that Windows Server at least up to 2016 is using TLS 1.0 for .NET framework (in which the ADFS configuration wizard is implemented) while my service hosting the metadata document only allowed TLS 1.2 as the minimum version:
Dropping the minimum version to TLS 1.0 is a no-go from security point of view, so the proper fix would be to enable TLS 1.2 as the default version on the ADFS server.
That would solve the issue (which I confirmed with a test) but then some of the other RPs only supporting TLS 1.0 would stop working, so I had to give up on importing metadata directly from a URL and use the file import option:
In this case another error popped up, which happened to be:
An error occured during an attempt to read the federation metadata. Verify that the specified URL or hostname is a valid federation metadata endpoint.
...
Error message: Entity descriptor '...'. ID6018: Digest verification failed for reference '...'.
This one turned out to be caused by me when I formatted the XML in the metadata file with line breaks and tabs to improve readability - it's all on a single line originally. ADFS won't allow that so the document must be exactly the same it came out of the metadata endpoint.
The same issue might result in different error messages and codes, depending on Windows and ADFS versions. For example this one is possible caused by a failed metadata integrity check as well:
An error occured during an attempt to read the federation metadata. Verify that the specified URL or hostname is a valid federation metadata endpoint.
...
Error message: Entity descriptor '...'. ID6013: The signature verification failed.
After having successfully imported a raw metadata file and having added a suitable Claim Issuance Policy I've got it finally working:
Related
We have set our Kafka Connect to be able to read credentials from a file, instead of giving them directly in connector config. This is how a login part of connector config looks like:
"connection.user": "${file:/kafka/pass.properties:username}",
"connection.password":
"${file:/kafka/pass.properties:password}",
We also added these 2 lines to "connect-distributed.properties" file:
config.providers=file
config.providers.file.class=org.apache.kafka.common.config.provider.FileConfigProvider
Mind that it works perfectly for JDBC connectors, so there is no problem with the pass.properties file. But for other connectors, such as couchbase, rabbitmq, s3 etc. it causes problems. All these connectors work fine when we give credentials directly but when we try to make Connect to read them from a file it gives some errors. What could be the reason? I don't see any JDBC specific configuration here.
EDIT:
An error about couchbase in connect.log:
[2021-12-02 11:50:19,580] ERROR [com.couchbase.io][SaslAuthenticationFailedEvent][20ms] Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"circuitBreaker":"DISABLED","coreId":"0xbf785c7500000001","remote":"10.30.142.109:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"ae3ce600-7097-4077-9231-8ced290cd399"}} (com.couchbase.io:533)
[2021-12-02 11:50:19,580] WARN [com.couchbase.endpoint][EndpointConnectionFailedEvent][23ms] Connect attempt 9 failed because of AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"circuitBreaker":"DISABLED","coreId":"0xbf785c7500000001","remote":"10.30.142.109:11210","type":"KV"} (com.couchbase.endpoint:523)
com.couchbase.client.core.error.AuthenticationFailureException: Authentication Failure - Potential causes: invalid credentials or if LDAP is enabled ensure PLAIN SASL mechanism is exclusively used on the PasswordAuthenticator (insecure) or TLS is used (recommended) {"circuitBreaker":"DISABLED","coreId":"0xbf785c7500000001","remote":"10.30.142.109:11210","status":"UNKNOWN","type":"KV","xerror":{"ref":"ae3ce600-7097-4077-9231-8ced290cd399"}}
at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.failConnect(SaslAuthenticationHandler.java:488)
at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.maybeFailConnect(SaslAuthenticationHandler.java:293)
at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.channelRead(SaslAuthenticationHandler.java:250)
at com.couchbase.client.core.io.netty.kv.MemcacheProtocolVerificationHandler.channelRead(MemcacheProtocolVerificationHandler.java:84)
at java.lang.Thread.run(Thread.java:748)
It says something about authentication but works fine when credentials are given directly. If the masking is not working correctly, how does it work for JDBC connectors?
Looks like the problem was quote marks in pass.properties file. The interesting thing is, even if credentials are typed with or without quote marks, JDBC connectors work well. Maybe the reason is it is the first line in the file but just a small possibility.
So, do NOT use quote marks in your password files, even if some of the connectors work this way.
I am attempting to connect from Quarkus to a MongoDB instance in the cloud which requires TLS. I have the certificate file for the server but cannot see how to use it with Quarkus.
I currently have the following properties set
quarkus.mongodb.connection-string = mongodb://blah:blah#mydomain.com:27017
quarkus.mongodb.database=school
quarkus.mongodb.tls=true
There does not appear to be anywhere to set the certificate file.
I cannot get past this error
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
Warren
There is no specific TLS settings for MongoDB with Quarkus.
If your certificate has a known root (which didn't seem to be the case), there is nothing more to do.
If your certificate is not knownw by your JVM, you need to use the keetool of your JVM to import it.
Be careful that if you deploy your application as a native executable, there is some steps to follow: https://quarkus.io/guides/native-and-ssl
We have an api deployed on Azure that uses Google authentication. Over the weekend, the API started to throw 500 errors that were resolved after restarting the API. Is there a way to identify what the underlying cause for these errors might be?
Check if you have custom error mode in web.config file to “on” or “Remoteonly”. If
yes then turn it off. Add the following line to System.web element in web.config
Enable custom logging/instrumentation in the code which can help you in more
information.
ASP.NET applications can use the System.Diagnostics.Trace class to log information to
the application diagnostics log. For example
System.Diagnostics.Trace.TraceError("If you're seeing this, something bad happened");
Enable Detailed Error Messages - Detailed version of the html files produced when
your website responds with an error message. This is good to enable for debugging
some error responses in your website. It is stored in the website's file system.
Web Server Logging - Also known as HTTP logs or IIS logs, this will log all requests
to your website in W3C Extended Log File Format.
Failed Request Tracing - Also known as FREB, here you can get lots of information
from IIS through its different stacks for each failing request.
Server: Windows Server 2012
Cold Fusion: 9,0,1,274733
Update-Level: hf901-00011.jar
Running on java version: 1.7.0_79
Java home points to the right path.
All certificates (for live and sandbox) are imported into the keystore of this JDK 1.7.0_79. I've tested it and renaming the cacerts file results in an error when connecting to the live API.
Testing the cacerts file using the keytool shows that the apropriate certificates are properly imported.
A little order app provides payment using PayPal.
First step is a connection to PayPal using the method "setExpressCheckout".
The connection to the live API using NVP at URL https://api-3t.paypal.com/nvp works and delivers the token URL-string.
The connection to the sandbox API using NVP at URL https://api-3t.sandbox.paypal.com/nvp fails with the error response:
I/O Exception: peer not authenticated
Connection Failure
Unable to determine MIME type of file.
Connection Failure. Status code unavailable.
Calling the URL https://api-3t.sandbox.paypal.com/nvp from the server works.
Test 1
imported the sandbox certificate for api-3t.sandbox.paypal.com
restart CF service
connection test failed with the same error
Test 2
renamed the cacerts file to cacerts.bak
copied the cacerts file from jre1.8.0_31\lib\security to the 1.7.0_79\lib\security
there is no specific PayPal cert in the cacerts file
restart the CF service
connection to live API works
connection to sandbox api fails with the same error
The weird thing is that the connection to the live api works without importing the specific certificate into the keystore when using the 1.8xx cacerts file.
I can't figure out why the connection to the sandbox fails. Maybe I can get new hints here?
If there are more informations needed please let me know. Thank you in advance.
Problem solved!
Scott Jibben (see his comment above) mentioned that the PayPal sandbox may already decline TLS1 connections in order to the upcoming change to do so in the PayPal live API.
This is absolutely right. But while in CF11 this isn't a problem because CF11 and its JRE are already using TLSv1.2, CF8-10 are using the default protocol of their JRE.
If not, one can force CF to use TLSv1.2 with the JVM argument
-Dhttps.protocols=TLSv1.2
Short:
CF8-10 are using TLSv1 while running with JDK1.70_79 and below no matter what the JVM startup argument -Dhttps.protocols was set to.
This is because the default protocol of these Java versions is TLSv1 and CF8-10 are simply ignoring the JVM startup argument -Dhttps.protocols and always use the JDK's default protocol.
This has changed with CF11 therefore it works fine with it.
Read detailed informations in a post from Wil Genovese at:
https://www.trunkful.com/index.cfm/2014/12/8/Preventing-SSLv3-Fallback-in-ColdFusion
What I did
I've installed the JDK1.8.0_144 and set up Cold Fusion 9 to use this one. Since then the connection to the PayPal sandbox API at api-3t.sandbox.paypal.com works pretty well.
Hope this may help others with this old and by now a little odd CF9.
When trying to hit an environment with improperly configured SSL certificates, I get the following error:
javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated
at com.sun.net.ssl.internal.ssl.SSLSessionImpl.getPeerCertificates(SSLSessionImpl.java:352)
at org.apache.http.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:128)
at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:390)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:148)
at org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:149)
at org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:121)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:562)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:415)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:776)
at dispatch.BlockingHttp$class.dispatch$BlockingHttp$$execute(Http.scala:45)
at dispatch.BlockingHttp$$anonfun$execute$1$$anonfun$apply$3.apply(Http.scala:58)
at dispatch.BlockingHttp$$anonfun$execute$1$$anonfun$apply$3.apply(Http.scala:58)
at scala.Option.getOrElse(Option.scala:108)
at dispatch.BlockingHttp$$anonfun$execute$1.apply(Http.scala:58)
at dispatch.Http.pack(Http.scala:25)
at dispatch.BlockingHttp$class.execute(Http.scala:53)
at dispatch.Http.execute(Http.scala:21)
at dispatch.HttpExecutor$class.x(executor.scala:36)
at dispatch.Http.x(Http.scala:21)
at dispatch.HttpExecutor$class.when(executor.scala:50)
at dispatch.Http.when(Http.scala:21)
at dispatch.HttpExecutor$class.apply(executor.scala:60)
at dispatch.Http.apply(Http.scala:21)
at com.secondmarket.cobra.lib.delegate.UsersBDTest.tdsGet(UsersBDTest.scala:130)
at com.secondmarket.cobra.lib.delegate.UsersBDTest.setup(UsersBDTest.scala:40)
I would like to ignore the certificates entirely.
Update: I understand the technical concerns regarding improperly configured SSL certs and the issue isn't with our boxes but a service we're using. It happens mostly on test boxes rather than prod/stg so we're investigating but needed something to test the APIs.
You can't 'ignore the certificates entirely' for the following reasons:
The problem in this case is that the client didn't even provide one.
If you don't want security why use SSL at all?
I have no doubt whatsoever that many, perhaps most, of these alleged workarounds 'for development' have 'leaked' into production. There is a significant risk of deploying an insecure system if you build an insecure system. If you don't build the insecurity in, you can't deploy it, so the risk vanishes.
The following was able to allow unsafe SSL certs.
Http.postData(url, payload).options(HttpOptions.allowUnsafeSSL,
HttpOptions.readTimeout(5000))
For the newest version of Dispatch (0.13.2), you can use the following to create an http client that accepts any certificate:
val myHttp = Http.withConfiguration(config => config.setAcceptAnyCertificate(true))
Then you can use it for GET requests like this:
myHttp(url("https://www.host.com/path").GET OK as.String)
(Modify accordingly for POST requests...)
I found this out here: Why does dispatch throw "java.net.ConnectException: General SSLEngine ..." and "unexpected status" exceptions for a particular URL?
And to create an Http client that does verify the certificates, I found some sample code here: https://kevinlocke.name/bits/2012/10/03/ssl-certificate-verification-in-dispatch-and-asynchttpclient/.