SSL TLS Version issue in BigQuery Linked Service in ADF - azure-data-factory

I am creating a Linked Service to connect to BigQuery with Service Account authentication and p12 file has already been placed in IR VM directory.
Receiving following error:
ERROR [HY000] [Microsoft][DriverSupport] (1160) Cannot enable SSL for the connection when connecting to a server that has not enabled SSL. If the server has SSL enabled, please check if it has been configured to use a SSL protocol version that is lower than what is allowed for the connection. The minimum SSL protocol version allowed for the connection is: TLS 1.2.
But when i place p12 file in another IR VM, linked service is working correctly.
Tried to check TLS version in both VM's and everything looks same.
Please suggest ways to check SSL and TLS version in VM's and how to resolve this issue.

Based on the error message it seems the issue might be w.r.t SSL certificate so 1st step would be to compare the SSL/TLS across both the servers
In case if they are same, would suggest to monitor the traffic across both the server when trying to access GCP Big Query.
You might get the exact cause of failure which can be like :
Firewall blockage
some missing set up like Big query need Google_base set up as a signature
etc.

You could create a PowerShell script that checks the TLS & SSL registry entries mentioned in the following documentation:
https://learn.microsoft.com/en-us/windows-server/security/tls/tls-registry-settings
Even if after enabling TLS/SSL settings the same issue arises then please try re-installing IR in the VM

Related

Why is ADF failing the SSL handshake with mongodb.net?

For the past few days, we've been getting errors during a daily pipeline run. The errors are similar to this:
Message:Failure happened on 'Source' side.
ErrorCode=UserErrorFailedToConnectOdbcSource,'
Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,
Message= Error from MongoDB Client: No suitable servers found
(serverSelectionTryOnce set): TLS handshake failed:
error:1416F086:SSL routines:tls_process_server_certificate:
certificate verify failed calling ismaster on
'.mongodb.net:27017'
I am able to connect to the Mongo database using Mongo CLI. I used Wireshark to confirm the SSL handshake is working fine. It shows that the Mongo cert is issued by ‘Let’s Encrypt' (https://letsencrypt.org/). The ‘Let’s Encrypt’ cert is included in the list of certs and it issued by ‘DST Root CA X3’, which is in the trusted root certs on my computer. Interestingly enough, the Mongo cert was just renewed a few days ago.
Just for fun (see '...verify calling ismaster...' in the message above), in the pipeline configuration, I updated the datasource dataset to ‘Allow Self-Signed Server Cert’ and the connection started working.
I’ll set that flag and allow the pipeline to run a few days to see if it fixes it. But I have no explanation for why this would work other than Azure has a weird way of implementing SSL.
Is there a way to see the list of trusted certs on the Azure server? Is there a way to capture the packets of the SSL handshake on the Azure side?
Microsoft suggested I upgrade the MongoDB Collection dataset type to MongoDB Collection V2. The pipeline I was working with had been developed a while back. The V2 version does not have 'Allow Self-Signed Server Cert'. This turned out to work for me though.

change security policy in milo opc example server

How to change security policy in milo example server? I've tried to connect to milo's opc server with UaExpert, and set the security policy. No matter what security policy I choose, the result is the same error:
ERROR o.e.m.o.s.s.t.u.UascServerAsymmetricHandler - [remote=/127.0.0.1:33762] Exception caught; sent ErrorMessage{error=StatusCode{name=Bad_SecurityChecksFailed, value=0x80130000, quality=bad}, reason=certificate path validation failed}
io.netty.handler.codec.DecoderException: UaException: status=Bad_SecurityChecksFailed, message=certificate path validation failed
How can I configure it correctly?
Second question: The example server seems quite complicated. What is the simplest way to create a server and connect to it? Is there a minimal example for that?
Welcome to stackoverflow
I do not know the Milo server and I have not almost used UAExpert but:
The client must trust the server certificate and the server must do the same with the client's certificate and of course both sides must have their certificate, sometimes the client and the server automatically create one if it doesn't exist but sometimes it should be created previously by external ways.
Help about UAExpert http://documentation.unified-automation.com/uaexpert/1.4.3/html/first_steps.html
If you don't want to deal with certificates then choose then endpoint with SecurityPolicy None when connecting:
Otherwise, note the security directory the example server logs on startup. Something like:
INFO o.e.m.examples.server.ExampleServer - security temp dir: /var/folders/1v/2pxlxd_x4bsdxz25_fv7r0940000gn/T/security
Navigate to the security directory and you'll find the UaExpert client certificate in the pki/rejected folder. You can move it to pki/trusted/certs and then connect with security from UaExpert.

SOAP Error "iaik.security.ssl.SSLException: Peer sent alert: Alert Fatal: handshake failure" in webMethods 6.5

I have encountered the below error while executing the pub.client:soapHTTP service that is available in WmPublic.
java.io.IOException: said.security.ssl.SSLException: Peer sent alert: Alert Fatal: handshake failure
On further investigation, I found that TLSv1.2 is being used by the partner server and I believe wM 6.5 only supports TLS v1.0. This might be a TLS version issue.
I am using webMethods 6.5.
Trusted Certificates > CA Certificate Directory is unspecified and watt.security.cert.wmChainVerifier.trustByDefault is set as TRUE for my server. Therefore, all CAs should be trusted.
Is there any way to make this work as upgrading wM is not an option, unfortunately?
I found this page (link PFB) where there is a section called TLS/SSL Handshake Issues & Debugging. There my handshake issue is mentioned and as a solution, it said:
handshake failed as Integration Server using Entrust IAIK connects to a TLS 1.2 enabled server. In this case, change from IAIK to JSSE option and set "watt.net.jsse.client.enabledProtocols=TLSv1.2"
I tried the above way but it did not work and yes, I restarted the server after changing the configuration.
Moreover, it also stated that
For outbound SSL connection (i.e. Integration Server is acting as SSL client), the following watt properties control the protocol version and ciphersuites:
IAIK:watt.net.ssl.client.handshake.minVersion,watt.net.ssl.client.handshake.maxVersion, watt.net.ssl.client.strongcipheronly,watt.net.ssl.client.cipherSuiteList
JSSE:watt.net.jsse.client.enabledProtocols,watt.net.jsse.client.enabledCipherSuiteList.
But I don't have any of those parameters configured in server.cnf and even if I had to I don't know what values I should configure them with. I'd like some help with those if it'll help with the situation.
https://techcommunity.softwareag.com/pwiki/-/wiki/Main/Debugging%20TLS%20SSL%20connections%20in%20Integration%20Server
try to set in extended settings
watt.net.ssl.client.useJSSE=true
i dont know the old version but it could be that the useJsse parameter that normally is set on the .http and .soapClient service is just taken from the default properties

Npgsql 3.0.3 error with Power BI Desktop

I'm receiving the following error when connecting to an AWS Postgres database that requires SSL. I recently upgraded from npgsql 2.3.2 (which was buggy) to 3.0.3 which won't connect. Any suggestions would be appreciated.
DataSource.Error: TlsClientStream.ClientAlertException:
CertificateUnknown: Server certificate was not accepted. Chain status:
A certificate chain could not be built to a trusted root authority. .
at TlsClientStream.TlsClientStream.ParseCertificateMessage(Byte[] buf,
Int32& pos) at
TlsClientStream.TlsClientStream.TraverseHandshakeMessages() at
TlsClientStream.TlsClientStream.GetInitialHandshakeMessages(Boolean
allowApplicationData) at
TlsClientStream.TlsClientStream.PerformInitialHandshake(String
hostName, X509CertificateCollection clientCertificates,
RemoteCertificateValidationCallback
remoteCertificateValidationCallback, Boolean
checkCertificateRevocation) Details:
DataSourceKind=PostgreSQL
I was able to fix the issue by installing the Amazon RDS public certificate on my machine. Once I did this, I was able to connect.
Steps I followed:
Download the AWS RDS public certificate 1
Create a .crt file from the .pem file downloaded. Sample instructions
here 2
Install the certificate (.crt file) on the machine. 3
Connect!
The docs from npgsql give the solution as changing the default trust server certificate of 'false' to 'true' in the connection string.
Unfortunately, neither Excel (AFAIK) nor Power BI will allow you to edit the connection string. So if you are unable to get the SSL certificate from the DB admin (as suggested in another answer), or the SSL cert has a different server name to the name you connect to (in my case an IP address), there is not much that can be done.
I can see two ways of fixing this. Either Shay & co from npgsql (who are doing an excellent job btw) provide some way for users to change the default settings for the connection string parameters. Or Microsoft allows users to send keywords in the connection dialog of Power BI (and Excel).
Npgsql 2.x didn’t perform validation on the server’s certificate by default, so self-signed certificate were accepted. The new default is to perform validation, which is probably why your connection is failing. Specify the Trust Server Certificate connection string parameter to get back previous behavior.
You can read more on the Npgsql security doc page, note also that this change is mentioned in our migration notes.
I had the same issue connecting PowerBI to a locally hosted PostgreSQL server and it turned out to be easy to solve if you can get the right information. Recent Npgsql versions will only connect over SSL if it trusts the certificate of the server. As a Windows application PowerBI uses the windows certificate store to decide what to trust. If you can get the SSL cert for the PostgreSQL server (or the CA cert used to sign that one) then tell Windows to trust that certificate, PowerBI will trust it too.
In the configuration folder for the PostgreSQL server there is a postgresql.conf file, search it for ssl settings, there is one with the location of the ssl cert. Note NOT the key file which contains the private key, only the cert file which contains the public key. copy it or its content to the machine running PowerBI and import using Run | mmc | Add Plugin... Certificates (Google it)
Look at the server name once you imported the cert and connect from PowerBI using the same server name (so the cert matches the connection). That solved the problem for me. If PostgreSQL is configured to insist on a SSL connection you might have to do the same for a ODBC connection too.
Its not best way but worked for me since if u dont need encryption for security reason.
Go to Postgres config file on your DB server and go from
ssl = true
to
ssl = false
Then open your power bi desktop File-> Options and settings -> Data source settings -> then in global you will have saved your connection press Edit Permissions and uncheck "ENCRYPT CONNECTIONS"
Then it will work
WARNING: THIS IS NOT RECOMMENDED IF YOUR DB IS OPEN TO PUBLIC.
Regards,
Davlik

Installing Wildcard SSL Certificate on Azure VM

I'm developing an application on Azure VM and would like to secure it by using the wildcard SSL certificate that I'm already using with my main domain. The SSL cert works with any *.mydomain.com and the application on Azure VM is accessible through myapplication.cloudapp.net
Based on the research that I've done, CNAME should be the best option to do that (I can't use A record since we need to shutdown the VMs every week and turn them back on the next week and will lose the ip addresses).
My two questions are:
How can I have myapplication.cloudapp.net be shown as subdomain.mydomain.com?
Will doing that make it possible for wildcard SSL certificate to be used for Azure application too?
How can I have myapplication.cloudapp.net be shown as
subdomain.mydomain.com?
Yes - this is just the CNAME forwarding and ensuring that the appropriate SSL certificate is installed on the server.
Will doing that make it possible for wildcard SSL certificate to be used for Azure application too?
Well as you're already exposing the Application through the VM - this should happen seemlessly.
Just a word of caution, you mention that you're using the certificate on the main domain, but haven't mentioned where you're using this. Be aware that, out-of-the-box, you can only assign one SSL per HTTPS endpoint. You can enable multiple SSL certificates on an Endpoint for Azure / IIS using Server Name Identification and can be enabled directly or automatically. If you do take this route, remember to configure your SNI bindings first, then apply the default binding - it kinda screws up otherwise.