I am using a 3rd party tool that can read data stored in parquet formaton hdfs and execute different functionalities on it. But this tool runs fine in environment which have Kerberos authentication disabled. I have Kerberos authentication enabled in my CDH-5.15 environment which cannot be disabled due to security reasons.
Is there a way to disable Kerberos authentication only for a particular user in CDH-5.15 ? Note that Parquet version that i have is 2.1.0
thanks
Related
We are currently planning to use Databricks as compute platform and Snowflake as our DWH system. We have planned to use SSO-based login for both, with our corporate ADFS as the IdP and we are still in the planning phase.
Wanted to check if having SSO enabled at Snowflake will restrict our ability to run jobs on Databricks that interacts with Snowflake for reading/writing data. If yes, what are our alternatives for better login security?
If this set-up is actually possible, can someone please point to any documentation talking about connecting to Snowflake from Databricks through SSO. Didn't really find anything on the topic. The document below mentions that MFA, SSO or any browser based login won't work with Snowflake's Spark connector, not sure if that's relevant to this use case.
https://docs.snowflake.com/en/user-guide/spark-connector-use.html#authenticating-through-a-browser-is-not-supported
For Spark connector use OAuth for authentication.
It can be configured with Microsoft Azure AD, see here
I have an application which uses LDAP authentication against AD, which works fine. However the UserDN and password for accessing LDAP are in clear text in the config files, and I would like to avoid clear text passwords.
In Softerra LDAP browser you have the option to authenticate with "Currently logged in user", does anyone know if this type of Windows Authentication is possible with Spring LDAP? Or is there perhaps another way to avoid credentials in config files (like using the credentials of the user trying to authenticate)?
With Microsoft SQL server, it is possible to do Windows Authentication via JDBC, since Microsoft provides some native code, has anyone heard about a similar functionality when using LDAP against AD?
"Currently logged in user"
this may mean that Kerberos based SASL authentication is used
You may check
https://docs.spring.io/spring-security-kerberos/docs/current/reference/htmlsingle/#ssk-kerberosldap
and
https://docs.spring.io/autorepo/docs/spring-security-kerberos/1.0.1.RELEASE/api/org/springframework/security/kerberos/client/ldap/KerberosLdapContextSource.html
The latter shows how the Kerberos ticket cache can be used.
We are using Java Kerberos authentication to connect to our SQL Server DB from Linux. Here we had used the prinicipal name and the password to generate a keytab file on the Linux system. Currently the connectivity works fine.
But there has been an additional requirement to use expiring passwords, which expire every 3 months. In our other applications we use an API called CyberArk which retrieves the password from a vault and Ops team need not bother about changing the password on the application server located on the Linux system.
Does anyone have any experience on using Kerberos in such an enironment? We are basically looking at avoiding to regenerate the keytab file every time the password expires.
I don't think you can avoid to regenerate the keytab file in the event of password change or expiring. What you can do, however, is to make it painless to generate the keytab file on the Linux server. this require the Linux server joining the Active Directory, using RHEL native tool realm or Centrify software.
RHEL tool document is here https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/windows_integration_guide/realmd-domain
For Centrify user, https://community.centrify.com/t5/Centrify-Express/Replace-SSH-Keys-with-Kerberos-Keytabs/td-p/10112
According to documentataion for WebHDFS REST API
https://hadoop.apache.org/docs/r2.7.3/hadoop-project-dist/hadoop-hdfs/WebHDFS.html#Delegation_Token_Operations
It is mentioned when security is on there is 2 mechanism
Authentication using Kerberos SPNEGO when security is on
Authentication using Hadoop delegation token when security is on
If i choose to use second option i.e Authentication using Hadoop delegation token when security is on
Does it mean it can run without Kerberos configuration in hadoop setup?
Do i have to setup Kerberos in my hadoop configuration in this case ?
To put things in context: typically, you use SPNEGO when you start your HTTP session, then cache your credentials somehow to avoid the complex rounds of 3-way communication between client, server, and Kerberos KDC.
AFAIK, all the Hadoop UIs and REST APIs use a signed cookie after the initial SPNEGO, and it's completely transparent for you -- with the exception of WebHDFS.
Now, with WebHDFS, you have to manage your "credentials cache" explicitly:
start your session with a GET ?op=GETDELEGATIONTOKEN -- you don't present any credentials, therefore it will trigger a SPNEGO authentication, then generate a Hadoop delegation token server-side
retrieve that delegation token from the JSON result
use that token to present your session credentials explicitly in the following GET / POST / PUT, by appending &delegation=XXXXXX to all URLs
Bottom line: yes, you have to set up your Kerberos configuration on client side. The delegation token only allows you to minimize the authentication overhead.
Everything I've seen and experienced so far suggests that you must use Windows Authentication if you want to access a file via Win32 from within the a databases filestream store but is it possible to use jsut SQL authentication somehow?
No, you have to use Windows Authentication.
See the official team Web Log for Microsoft Customer Service and Support (CSS) SQL Support:
How It Works: File Streams Requires Integrated Security (Windows Authentication)