When trying to hit an environment with improperly configured SSL certificates, I get the following error:
javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated
at com.sun.net.ssl.internal.ssl.SSLSessionImpl.getPeerCertificates(SSLSessionImpl.java:352)
at org.apache.http.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:128)
at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:390)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:148)
at org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:149)
at org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:121)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:562)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:415)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:776)
at dispatch.BlockingHttp$class.dispatch$BlockingHttp$$execute(Http.scala:45)
at dispatch.BlockingHttp$$anonfun$execute$1$$anonfun$apply$3.apply(Http.scala:58)
at dispatch.BlockingHttp$$anonfun$execute$1$$anonfun$apply$3.apply(Http.scala:58)
at scala.Option.getOrElse(Option.scala:108)
at dispatch.BlockingHttp$$anonfun$execute$1.apply(Http.scala:58)
at dispatch.Http.pack(Http.scala:25)
at dispatch.BlockingHttp$class.execute(Http.scala:53)
at dispatch.Http.execute(Http.scala:21)
at dispatch.HttpExecutor$class.x(executor.scala:36)
at dispatch.Http.x(Http.scala:21)
at dispatch.HttpExecutor$class.when(executor.scala:50)
at dispatch.Http.when(Http.scala:21)
at dispatch.HttpExecutor$class.apply(executor.scala:60)
at dispatch.Http.apply(Http.scala:21)
at com.secondmarket.cobra.lib.delegate.UsersBDTest.tdsGet(UsersBDTest.scala:130)
at com.secondmarket.cobra.lib.delegate.UsersBDTest.setup(UsersBDTest.scala:40)
I would like to ignore the certificates entirely.
Update: I understand the technical concerns regarding improperly configured SSL certs and the issue isn't with our boxes but a service we're using. It happens mostly on test boxes rather than prod/stg so we're investigating but needed something to test the APIs.
You can't 'ignore the certificates entirely' for the following reasons:
The problem in this case is that the client didn't even provide one.
If you don't want security why use SSL at all?
I have no doubt whatsoever that many, perhaps most, of these alleged workarounds 'for development' have 'leaked' into production. There is a significant risk of deploying an insecure system if you build an insecure system. If you don't build the insecurity in, you can't deploy it, so the risk vanishes.
The following was able to allow unsafe SSL certs.
Http.postData(url, payload).options(HttpOptions.allowUnsafeSSL,
HttpOptions.readTimeout(5000))
For the newest version of Dispatch (0.13.2), you can use the following to create an http client that accepts any certificate:
val myHttp = Http.withConfiguration(config => config.setAcceptAnyCertificate(true))
Then you can use it for GET requests like this:
myHttp(url("https://www.host.com/path").GET OK as.String)
(Modify accordingly for POST requests...)
I found this out here: Why does dispatch throw "java.net.ConnectException: General SSLEngine ..." and "unexpected status" exceptions for a particular URL?
And to create an Http client that does verify the certificates, I found some sample code here: https://kevinlocke.name/bits/2012/10/03/ssl-certificate-verification-in-dispatch-and-asynchttpclient/.
Related
I am trying to test an API on my site. The tests work just fine from one machine, but running the code from a different machine results in the SSLCertVerificationError - which is odd because the site has an SSL cert and is NOT self signed.
Here is the core of my code:
async def device_connect(basename, start, end):
url = SERVER_URL
async with aiohttp.ClientSession() as session:
post_tasks = []
# prepare the coroutines that post
for x in range(start, end):
myDevice={'test':'this'}
post_tasks.append(do_post(session, url, myDevice))
# now execute them all at once
await asyncio.gather(*post_tasks)
async def do_post(session, url, data):
async with session.post(url, data =data) as response:
x = await response.text()
I tried (just for testing) to set 'verify=False' or trust_env=True, but I continue to get the same error. On the other computer, this code runs fine and no trust issue results.
That error text is somewhat misleading. OpenSSL, which python uses, has dozens of error codes that indicate different ways certificate validation can fail, including
X509_V_ERR_SELF_SIGNED_CERT_IN_CHAIN -- the peer's cert can't be chained to a root cert in the local truststore; the chain received from the peer includes a root cert, which is self-signed (because root certs must be self-signed), but that root is not locally trusted
Note this is not talking about the peer/leaf cert; if that is self signed and not trusted, there is a different error X509_V_ERR_DEPTH_ZERO_SELF_SIGNED_CERT which displays as just 'self signed certificate' without the part about 'in certificate chain'.
X509_V_ERR_UNABLE_TO_GET_ISSUER_CERT_LOCALLY (displays in text as 'unable to get local issuer certificate') -- the received chain does not contain a self-signed root and the peer's cert can't be chained to a locally trusted root
In both these cases the important info is the peer's cert doesn't chain to a trusted root; whether the received chain includes a self-signed root is less important. It's kind of like if you go to your doctor and after examination in one case s/he tells you "you have cancer, and the weather forecast for tomorrow is a bad storm" or in another case "you have cancer, but the weather forecast for tomorrow is sunny and pleasant". While these are in fact slightly different situations, and you might conceivably want to distinguish them, you need to focus on the part about "you have cancer", not tomorrow's weather.
So, why doesn't it chain to a trusted root? There are several possibilities:
the server is sending a cert chain with a root that SHOULD be trusted, but machine F is using a truststore that does not contain it. Depending on the situation, it might be appropriate to add that root cert to the default truststore (affecting at least all python apps unless specifically coded otherwise, and often other types of programs like C/C++ and Java also) or it might be better to customize the truststore for your appplication(s) only; or it might be that F is already customized wrongly and just needs to be fixed.
the server is sending a cert chain that actually uses a bad CA, but machine W's truststore has been wrongly configured (again either as a default or customized) to trust it.
machine F is not actually getting the real server's cert chain, because its connection is 'transparently' intercepted by something. This might be something authorized by an admin of the network (like an IDS/IPS/DLP or captive portal) or machine F (like antivirus or other 'endpoint security'), or it might be something very bad like malware or a thief or spy; or it might be in a gray area like some ISPs (try to) intercept connections and insert advertisements (at least in data likely to be displayed to a person like web pages and emails, but these can't always be distinguished).
the (legit) server is sending different cert chains to F (bad) and W (good). This could be intentional, e.g. because W is on a business' internal network while F is coming in from the public net; however you describe this as 'my site' and I assume you would know if it intended to make distinctions like this. OTOH it could be accidental; one fairly common cause is that many servers today use SNI (Server Name Indication) to select among several 'certs' (really cert chains and associated keys); if F is too old it might not be sending SNI, causing the server to send a bad cert chain. Or, some servers use different configurations for IPv4 vs IPv6; F could be connecting over one of these and W the other.
To distinguish these, and determine what (if anything) to fix, you need to look at what certs are actually being received by both machines.
If you have (or can get) OpenSSL on both, do openssl s_client -connect host:port -showcerts. For OpenSSL 1.1.1 up (now common) to omit SNI add -noservername; for older versions to include SNI add -servername host. Add -4 or -6 to control the IP version, if needed. This will show subject and issuer names (s: and i:) for each received cert; if any are different, and especially the last, look at #3 or #4. If the names are the same compare the whole base64 blobs to make sure they are entirely the same (it could be a well-camoflauged attacker). If they are the same, look at #1 or #2.
Alternatively, if policy and permissions allow, get network-level traces with Wireshark or a more basic tool like tcpdump or snoop. In a development environment this is usually easy; if either or both machine(s) is production, or in a supplier, customer/client, or partner environment, maybe not. Check SNI in ClientHello, and in TLS1.2 (or lower, but nowadays lower is usually discouraged or prohibited) look at the Certificate message received; in wireshark you can drill down to any desired level of detail. If both your client(s) and server are new enough to support TLS1.3 (and you can't configure it/them to downgrade) the Certificate message is encrypted and wireshark won't be able to show you the contents unless you can get at least one of your endpoints to export the session secrets in SSLKEYLOGFILE format.
I am trying the simplest example I can, pulled directly from their website. Here is my entire html file, with code taken exactly from https://peerjs.com/index.html:
<script src="https://unpkg.com/peerjs#1.3.1/dist/peerjs.min.js"></script>
<script>
var peer = new Peer();
var conn = peer.connect('another-peers-id');
// on open will be launch when you successfully connect to PeerServer
conn.on('open', function(){
// here you have conn.id
conn.send('hi!');
});
</script>
In Chrome and Edge I get this in the console:
peerjs.min.js:64 GET https://0.peerjs.com/peerjs/id?ts=15956160926060.016464029424720694 net::ERR_CONNECTION_REFUSED
In Firefox I get this:
Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://0.peerjs.com/peerjs/id?ts=15956162489620.8436734374800061. (Reason: CORS request did not succeed).
What am I doing wrong?
#reyad has requested "a full trace of requests and responses". Here's what I see in my network tab in Firefox:
And here's Chrome:
And a tiny bit more Chrome:
[Note: It would have been better if you could provide a full trace of requests and responses. This problem may occur for several reasons. I'll state two solutions. So, try those. If those doesn't work, provide full trace of requests and responses.]
1. First Solution:
Sometimes, this type of error occurs because of self-signed certificate. To solve this problem, open developer tools/options, then go to network tab. You'll see a list of requests. Select the request which was failed because of CORS(i.e. which gave you this Reason: CORS request did not succeed). Open it(i.e. click it). If your problem is related to cert you'll see the following error message:
AN ERROR OCCURED: SEC_ERROR_INADEQUATE_KEY_USAGE
To solve this problem, go to url that is the reason of this problem and accept the certificate manually.
2. Second solution:
Check the request(which is the reason of CORS) in the network tab of developers tools/options(same as described in 1. First Solution). You'll find a Transferred column. See, what's written in the Transferred column of the failed request. If it is written Blocked By Some Ad-Blocker, then disable the Ad-Blocker. Your request will work fine.
[P.S.]: These solutions are proposed on assumptions. Hope these works. If these two do not work, then please provide more info about requests and responses. And also check this.
3. Third and final solution:
[Note: This solution may not solve your problem directly, but it'll give you alternative solution and also insight about what your problem is and how to work around it]
Before reading the solution below, read this to understand how Access-Control-Allow-Origin works(it is the reason for CORS error).
Let me first explain how peerjs works:
PEERJS works based on PEER ID. So, you've to get some PEER ID either from the PEERJS CLOUD SERVER or you've to provide yourself one in the PEER CONSTRUCTOR i.e. new Peer("some-peer-id"). Peer id has to be unique, cause its necessary to detect all the users uniquely. And, peerjs uses this PEER ID to send and receive data from user to user.
Now, you should know that, you're using PEERJS CLOUD SERVER to get/generate unique peer id which is the default server PEERJS uses unless you specified some other server to use.
Now let me explain why you're facing this problem:
As you already know how CORS works, you may have already guessed, that https://unpkg.com/peerjs#1.3.1/dist/peerjs.min.js(the downloaded js file) is calling https://0.peerjs.com to retrieve/generate new unique PEER ID. But, this request by https://your.website.com does not have Access-Control-Allow-Origin access for some reason, it may also be a middleware problem. So, its difficult to tell where the problem is actually occuring. But one thing for sure, it's not your fault of writing code :D.
I hope all the concepts is clear to you I've stated above.
Now, to solutions:
Alternative-appraoch-1 (Using PEERJS CLOUD SERVER AND Your own provided id):
In this approach you've to generate your own unique PEER ID. So, "https://your.website.com" does not have to call "https://0.peerjs.com" for unique peer id. [Note: make your peer id large enough so that its always unique, at least 64 chars long]
In this way, you can avoid the CORS problem.
Update:
I just saw an new issue in github, which says the public peerjs cloud server is now unstable or does not work properly. It just gives error like: Firefox cannot establish a connection with the server at the address wss://0.peerjs.com/peerjs?key=peerjs&id=123222589562487856955685485555&token=ocyxworx62i and in Chrome: Error in connection establishment: net::ERR_CONNECTION_REFUSED. For details check here. So, its better, you use your own server(see the next approach).
Alternative-appraoch-2 (Using your own peerjs server):
You can host your own peerjs server instead of PEERJS CLOUD SERVER. In this way, you can allow access to anyone/any website you want. If you want know how to host a peerjs server, you may visit here.
[P.S.]: I have studied pearjs issues in github. After reading all those issues, it seems, it is better to use your own server rather than using pearjs cloud. There are a lot of various problems with each new release of peerjs. And mostly related with connection with peerjs cloud and also peerjs cloud is not stable I guess. They were hosting it in 0.peerjs.com:9000 before(not secure). But now in 0.peerjs.com:443.
I haven't use peerjs before nor set up peerjs server. If you want to set up one, I hope the community would be able help you on how to do that properly.
What I understand from your question is that there is an issue of (CORS => Cross-origin resource sharing ), Maybe what I am suggesting is not very intuitive.
First : download the "https://unpkg.com/peerjs#1.3.1/dist/peerjs.min.js" in your local directory . and then incklude the local javascript code to the html.
like: <script src="./peerjs.min.js"></script>
Second :
you are using var peer = new Peer();
but please provide an extra unique id from your side. for example, I just created a random id and provided it.
StackOverflow link: https://stackoverflow.com/questions/21216758/peerjs-set-your-own-peerid#:~:text=1%20Answer&text=Provide%20a%20peer%20id%20when,to%20under%20Create%20a%20peer.
var a_random_id = Math.random().toString(36).replace(/[^a-z]+/g, '').substr(2, 10);
var peer = new Peer(a_random_id, {key: 'myapikey'});
Third : the best option is to run PeerServer: A server for PeerJS of your own.
If you don't want to develop anything, just enter a few commands below.
Install the package globally:
$ npm install peer -g
Run the server:
$ peerjs --port 9000 --key peerjs --path /myapp
Started PeerServer on ::, port: 9000, path: /myapp (v. 0.3.2)
Check it: http://127.0.0.1:9000/myapp It should return JSON with name, description, and website fields.
details:https://github.com/peers/peerjs-server
In PostgreSQL, whenever I execute an API URL with secure connection with query
like below
select *
from http_get('https://url......');
I get an error
SSL certificate problem: unable to get local issuer certificate
For this I have already placed a SSL folder in my azure database installation file at following path
C:\Program Files\PostgreSQL\9.6\ssl\certs
What should I do to get rid of this? Is there any SSL extension available, or do I require configuration changes or any other effort?
Please let me know the possible solutions for it.
A few questions...
First, are you using this contrib module: https://github.com/pramsey/pgsql-http ?
Is the server that serves https://url....... using a self-signed (or invalid) certificate?
If the answer to those two questions is "yes" then you may not be able to use that contrib module without some modification. I'm not sure how limited your access is to PostgreSQL in Azure, but if you can install your own C-based contrib modules there is some hope...
pgsql-http only exposes certain CURLOPTs (see: https://github.com/pramsey/pgsql-http#curl-options) values which are settable with http_set_curlopt()
For endpoints using self-signed certificates, I expect the CURLOPT you'll want to include support for to ignore SSL errors is CURLOPT_SSL_VERIFYPEER
If there are other issues like SSL/TLS protocol or cipher mismatches, there are other CURLOPTs that can be patched-in, but those also are not available without customization of the contrib module.
I don't think anything in your
C:\Program Files\PostgreSQL\9.6\ssl\certs
folder has any effect on the http_get() functionality.
If you don't want to get your hands dirty compiling and installing custom contrib modules, you can create an issue on the github page of the maintainer and see if it gets picked up.
You might also take a peek at https://github.com/pramsey/pgsql-http#why-this-is-a-bad-idea because the author of the module makes several very good points to consider.
I am new to this OPC-UA world and Eclipse Milo.
I do not understand how the security works here,
Discussing about client-example provided by eclipse-milo
I see few properties of security being used to connect to the OPCUA Server:
SecurityPolicy,
MessageSecurityMode,
clientCertificate,
clientKeyPair,
setIdentityProvider,
How the above configurations are linked with each other?
I was trying to run client-examples -> BrowseNodeExample.
This example internally runs the ExampleServer.
ExampleServer is configured to run with Anonymous and UsernamePassword Provider. It is also bound to accept SecurityPolicy.None, Basic128Rsa15, Basic256, Basic256Sha256 with MessageSecurityMode as SignandEncrypt except for SecurityPolicy.None where MessageSecurityMode is None too.
The problem is with AnonymousProvider I could connect to the server with all SecurtiyPolicy and MessageSecurityMode pair mentioned above (without client certificates provided).
But I could not do the same for UsernameProvider, For UsernameProvider only SecurityPolicy MessageSecurityMode pair with None runs successfully.
All others pairs throw security checks failed exception (when certificate provided) else user access denied (when client certificate not provided). How to make this work?
Lastly, It would be really nice if someone could point me to proper User documentation for Eclipse Milo. Since I could not see any documentation except examples codes, and they are not documented.
SecurityPolicy and MessageSecurityMode go hand-in-hand. The security policy dictates the set of algorithms that will be used for signatures and encryption, if any. The message security mode determines whether the messages will be signed, signed and encrypted, or neither in the case where no security is used.
clientCertificate and clientKeyPair must be configured if you plan to use security. You can't use encryption or signatures if you don't have a certificate and private key, after all.
IdentityProvider used to provide the credentials that identify the user of the session, if any.
When the ExampleServer starts up it logs that its using a temporary security directory, something like this: security temp dir: /var/folders/z5/n2r_tpbn5wd_2kf6jh5kn9_40000gn/T/security. When a client connects using any kind of security its certificate is not initially trusted by the server, resulting in the Bad_SecurityChecksFailed errors you're seeing. Inside this directory you'll find a folder rejected where rejected client certificates are stored. If you move the certificate(s) to the trusted folder the client should then be able to connect using security.
I am running into the following problem with an iOS app we are developing. On my server I am running a WCF-service. This service is running in HTTPS. I wrote a small test program to see if I can connect to it from another PC:
ServicePointManager.ServerCertificateValidationCallback = (sender, certificate, chain, sslPolicyErrors) => true;
Console.WriteLine("Press key to start webrequest");
Console.ReadKey();
WebRequest wr = WebRequest.Create("https://<serveraddress.here.com>");
Stream stream = wr.GetResponse().GetResponseStream();
Console.WriteLine(new StreamReader(stream).ReadToEnd());
Console.ReadKey();
This works as expected and returns html-code. Notice we're using ServicePointManager.ServerCertificateValidationCallback to ensure that the certificate is being trusted in any case.
The same code in MonoTouch on the iPhone however returns the following error:
System.Net.WebException has been thrown
Error getting response stream (Write: BeginWrite failure): SendFailure
at System.Net.HttpWebRequest.EndGetResponse (IAsyncResult asyncResult)
Connecting to another a signed https site (paypal for example) works without problems. Also connecting to the server without https (regular http) works fine.
It seems like it's ignoring the ServerCertificateValidationCallback and failing somewhere. Is there any way to fix this?
SSL connection with unsigned certificate
I suspect you're using a self-signed (not an unsigned) certificate that you made yourself (e.g. makecert) for temporary use.
Like others said in comments, this should work as many people are doing similar things when developing their applications.
It seems like he's ignoring the ServerCertificateValidationCallback and failing somwhere. Is there any way to fix this?
Mono (and MonoTouch) supports both the ServerCertificateValidationCallback and the older ICertificatePolicy methods to allow application to have the last word on accepting (or refusing) an X.509 certificate. You can try the other method - but I'm not sure that's the issue.
You can also compare your (full, not partial) stack trace with the same code where ServerCertificateValidationCallback is commented. If the error is identical then it's likely certificate related, otherwise it's not (there's a lot more that can go wrong with SSL/TLS ;-)
Sadly I can't give you more specific help because your question lacks important details.
What version of MonoTouch are you using ?
The full stack trace of the exception MonoTouch gives you ? from the partial one you gave it's not 100% clear that the issue happens when certificates are validated.
Where/how did you execute your sample code ? e.g. was it on Mono or Microsoft .NET ? which version ?
I suggest you to open a bug report on http://bugzilla.xamarin.com (and include the missing details) since it's likely that we'll require even more data (e.g. a wireshark log of the SSL communication).